But what interests me here, is that in my experience the difficulty of redesigning a product – both the subjective experience of effort and the objective expenditure of time and attention in completing a redesign, start to finish – seems to vary with the product's success, but not in a particularly straightforward manner. That is, how successful the product to be redesigned currently is affects the difficulty of the process in a complex, non-intuitive way. Why?
Well, first of all there are at least three big classes of factors impacting the difficulty of redesigning a site. Some of these are important in designing the site from scratch, too, but the way they interact with the current success of the product when embarking on a redesign is what interests me. The three factors are Market Information, User Entrenchment, and The Organization.
Market Information
When redesigning, we look to the market for clues, approach our users for tests, and effectively try to tease out guidance from the field. Ultimately, we're looking for the formula for design success. But this stuff is incredibly complicated, including such things as the design conventions determining current user expectations; the competitive landscape identifying areas of relevant differentiation; trend dynamics suggesting potential "innovations"; etc. But the most subtly difficult factors are the complex social dynamics of product acceptance. I've written about these before: the hard to discern workings of network effects, informational cascades, and the resultant path dependencies cause enormous confusion among stakeholders, greatly increasing the difficulty of redesigning social functionality. In our misguided search for a simple formula for redesign success, these factors, which decrease the role of product qualities (the stuff of design) in predicting product success, can really lead you astray.
In a nutshell, the externalities inherent in the social web (i.e. we initially evaluate a product based on others' actions and then get the most value from the products that everyone else uses) make predicting success from product qualities almost impossible. Success is usually a matter of context, timing and accidents of the product's acceptance history rather than objective design quality. So, beyond a set of user expectation-defining standards – the competitive baseline, which you should have already identified in the initial design round – market analysis and user research won't help you identify a redesign guaranteed to succeed.
Even worse, up to a certain point, the more relative success you have, the murkier the information derivable from testing users and combing the market becomes. Your variance from the competitive baseline tends to be smaller the more successful you are, and the true drivers of success are rarely clear cut product design issues beyond this competitive baseline. Very few stakeholders understand these subtle dynamics and insist that more market research, testing and "innovation" will crack the "guaranteed success" design code. As a result, redesigns of moderately successful products tend to bog down in obsessive, inconclusive research, the results of which are infinitely and inconsistently interpretable (especially if testing is public). Research is necessary, but will never provide a guarantee.
User Entrenchment
Perceived usability – as opposed to laboratory usability – is more about familiarity than objective human factors metrics. Users invest their time and attention in your product, amassing a sort of practical capital of product-specific know-how. When you start changing things, they view this as effectively theft of a precious resource: their investment in practical capital has been thrown out the window without their explicit blessing.
As anyone who has ever redesigned anything knows, users get angry when they feel they've wasted time and attention; they must reinvest to get back to the same level of practical capital. So, up to the point where nearly everybody's using your product and there's no real alternative (i.e. where the network effect is so strong and the average user's "sunk cost" so great they overcome practically all anger over perceived losses associated with change), redesigns get dicier the greater the success of the product. One consequence of this is that testing redesigns with current users will significantly skew your results toward the negative, regardless of "actual" quality. If these tests are public, you've just created a significant political battle and, thus, headaches.
The Organization
There are two related but distinct sources of organizational inertia. First, people within the organization simply get used to doing things one way and – being human – are reluctant to change. If the product's relatively successful, that felt reluctance can be rationalized by arguing that any redesign is a dangerous and unneeded rocking of the boat that endangers continuing success. Slowdowns, endless discussion, and frequent miscommunications are usually the result.
Next, as much as I hate calling simple regularities or covariances "laws" for comic effect (Moore's Law, Metcalfe's Law, etc.), Conway's Law, or something like it, seems to be pretty prevalent. In 1968, Melvin Conway observed that organizations produce products that mirror their structure or at least their internal communication patterns. He was referring specifically to the development of intercommunicating software systems produced by different design teams existing in some sort of institutional, social or communication structure. His claim is that this structure will manifest itself in the design of the interfaces between the systems.
Conway thought this was inevitable, which is way too strong by my lights. But stretching the idea a bit, some products, particularly websites, often mirror the structures of the organizations that created them. Furthermore, it seems to go the other way as well. Sometimes the organization and product have co-evolved in such a way that they're truly intermingled or even identified in many stakeholder's minds. Stakeholders view the product as a direct manifestation of the organization, and any change to the former necessarily impacts the latter.
Whatever the underlying mechanisms that generate this feeling, the upshot is that redesigning a product often has organizational implications, at least in terms of many stakeholders' perceptions of the project. Tampering with the product is often perceived as tampering with roles, responsibilities, and the delicately negotiated distribution of power within the organization. Understandably, this can lead to a significant amount of internal resistance, particularly when the product has been relatively successful and actual power and prestige have accumulated. The resulting inertia and sometimes downright sabotage can lead to massive struggles and delays.
These factors interact with the current success of the product (and with each other) to impact the difficulty of redesigning the product. Drawing from my highly subjective experience designing, redesigning, and "researching" social functionality, the situation seems to look something like this:
On the x axis is increasing success and on the y axis, increasing difficulty of redesigning the product. Some of the more interesting areas have been called out with letters. I'll discuss them briefly.
Why is a more difficult than b? After all, nobody's using the product, so redesign should be really easy. At this level of market failure, if you're asked to do a redesign, you should consider it carefully: this product should be completely abandoned and a different product built. But if you have to do a redesign, the whole organization probably understands and embraces the need to change and analyzing the market may help you identify appropriate conventions, standards and design directions. However, you are re-starting from a hole with little prospect of success (remember the importance of network externalities). Your potential users have already invested somewhere else (this a redesign… you had an initial chance and blew it. At this point your users have invested elsewhere and people burned earlier won't come back.). Finally, you have absolutely no user data on which to base your redesign recommendations and no prospect of launching then optimizing, bootstrapping your way to a successful design. After all, there's not enough activity to get a picture of where it's failing or succeeding; analyzing the market will suggest directions, but you can't really put it out there and optimize on the fly as no one's using your product.
As success modestly improves to b, the ease of redesign increases dramatically. At this point, your stakeholders are still ready for change. Analyzing the market could probably still help with design standards, conventions and trends. But now you have some data on what might work and what definitely won't. You have real users that you can gently test against, but not so many that changes will be met with a loud protest. You also have at least a foothold, which, with luck could be turned into something more by doing everything you can to generate an informational cascade through targeted design differentiation and smart positioning/marketing.
But things get progressively tougher from b to c. You're probably nearing the conventional designs as you're at least competitive at this level of success, so analyzing the market too much could lead to frustration and confusion among your stakeholders. Selling designs internally becomes more difficult as the product's modest success has been enough to spread a bit of influence and power throughout the organization. But, there are enough users to make switching costs to another product at least non-negligible to many users. However, the User Entrenchment process begins to become an issue at this point, as you've enough success for your users to have begun internalizing the system.
From c the difficulty quickly rises with success, slowing to a peak at d. This is the zone of incredible difficulty for a redesign; throughout this region all of the factors are potentially against you. Unless you're just adding hot new functionality pioneered by a competitor, culling the market for tips will be largely fruitless. You're most likely conventional – if not the leader – at this level of success, so most of the factors determining further success are those frustrating, inconsistently interpretable, and highly contingent social dynamics. Thus prolonged research and analysis in this situation actually pays little – and often costs much in terms of focus, morale and transactions. But most organizations in this range still get hung up searching the field and bugging their users in an obsessive search for a magic design formula when they should just be looking for suggestive trends to fuel experimentation. But you have to remember, users in this range become ever more entrenched, so the "right" changes are both difficult to identify and nearly impossible to justify through testing or interviews. Users want what they know now, not what they will want later, so at high levels of entrenchment asking them to evaluate a change (as opposed to just testing lab usability) often confuses the issue or leads to the agonizing death of bold new designs. Finally, organizationally, great success can make people scared to change for fear of losing it (people are, after all, far more risk averse than gain hungry). The organization has "rigidified." Success has pumped prestige and power into it's structure, creating significant vested interests in maintaining status quo. Projects in this range can quickly become nightmares.
At d, however, things turn around a little. Though it's never as easy as it is when you've poor to moderate success, the network effect is so strong, that the specter of losing position is largely mooted. That effectively removes the rationalization for status quo from within the organization and greatly mitigates the sting of negative test results from entrenched users. Also, at this point, you lead the market and most of the tips you're looking for are in terms of competitive and trend analysis, not the elusive success guaranteeing design formula. You largely co-determine the conventions defining the completive baseline along with other highly successful players. Thus, redesign at this point becomes slightly easier, but there's still there's a lot of reluctance from those who might fear loss of position and entrenched users will definitely make a lot of noise, which is never pleasant.
Redesigns are necessary to stay competitive. But along with the excitement and product / organizational rejuvenation they generate, they can also be headaches for the designers involved. Hopefully I've managed to shed some light on why and how. As I said, all of this stems form my subjective observation that redesign difficulty seems somehow oddly related to product success. Whether or not your experience of the relation between the two mirrors mine, I hope you'll at least agree that there's some relationship and that it's impacted by the factors I've suggested.
