Builders and Shoppers

In the IT world, I think there are two types of decision-makers: Builders and Shoppers. A Builder is a person who investigates how new technologies might enable new approaches to current needs. A Builder might study a new technology and ask - how would we work differently if we used this? Would things be better somehow? In contrast, a Shopper usually learns about a new technology in the context of a product evaluation.

Builders are not simply do-it-yourself types. Builders try new products all the time. Builders are extremely interested in knowing what leading technology providers are coming up with. So it's wrong to say that the difference between Builders and Shoppers is that Builders will only build solutions and Shoppers can only buy solutions. But I think it's fair to say that Builders set the direction and definition for their objectives independently of what's commercially available, while Shoppers tend to rely more on product claims to set the direction of their project and define requirements.

There's also a difference between Builders and Shoppers after a technology has been implemented. Builders accept the fact that new requirements will emerge, and that even after project is "done" there will inevitably be a second project based on changes in the business's operations or emerging requirements. Shoppers tend to see a project as "done" until the vendor releases an upgrade or patch.

When I began working with Web services some ten years ago, I found that one of the greatest problems we had to overcome was helping line-of-business managers understand why they would want to use Web services to do anything. Web services were a technology solution looking for a business problem, so within different organizations they were advocated in a variety of ways:

- As a better, faster cheaper kind of EAI.
- As the back-end connectivity for B2B integration projects.
- As the secret sauce in next-generation portals.
- As the secret sauce in next-generation business intelligence
- As a short-cut to customer data integration.
- As the foundation for improved BPM.
- As a mechanism for consolidation of IT operations.

Looking back on these messages, I can see that some of them were designed for Shoppers - Web services can displace an expensive EAI application, portal solution or some other product. Other messages were designed for Builders - consolidate IT operations, or improve BPM.

But the challenge we faced was that while most enterprises were thinking like Shoppers, the greatest benefits of Web services only resonated with people who thought like Builders. Web services were a new way to provide interfaces for applications. The question was - what capabilities could we (or should we) build on the organization's existing applications (and new ones as well) with this new type of interface? To really get excited about Web services required the imagination of a Builder.

Then, as always happens with new technology, all of a sudden every enterprise technology product or service was labeled a "Web Services" offering as well. In a desperate attempt to convince the Shoppers that they weren't missing out on an essential technology, a "Web Services" sticker was put on every box -- the enterprise IT equivalent of "Zero Trans-Fats" or "Heart Healthy".... And so confusion reigned.

Where were the experts in all of this? Many of them were right there in the trenches, working with the Builders to get things right, sharing knowledge and designing architectures that would have previously been very difficult, if not impossible, to manage and maintain. Many experts weighed in with thoughtful, detailed analyses of the major vendors' efforts and cautioned against the perils of thinking like a Shopper when making Builder decisions.

But too many experts became a kind of "super-Builder". They veered off into the weeds, piling complexity on top of confusion. Dogmatic debates over architectural principles led to a tangle of standards that almost nobody even tried to use. Different vendors supported entirely incompatible families of Web services standards, which had the ironic result of turning a technology that was supposed to be both vendor and technology neutral into yet another means to lock a customer's architecture into a single vendor's product line.

Meanwhile, many in the open source and hacker community - rather than taking sides in a fight they had no stake in - took a different course. Many began stripping out the bits that they didn't need. Such hackery wasn't sanctioned by standards committees, but it was the slow pace of those bodies and (for some) their closed-doors discussions that had frustrated the technologists in the first place. Besides, given the choice between standards compliance and just getting the job done, the hackers chose the latter.

Across technology domains, there has been a steady divergence between the simplicity of lightweight technologies and the complexity of "enterprise" technologies. It would be wrong to say that one trend has been driven by Builders and the other by Shoppers. Builders are involved in everything, but while the deep pockets of the Shoppers kept the enterprise technologies moving forward, it was a groundswell of Builder communities that enabled lightweight technologies to flourish as they have.

As the Java Enterprise Edition specifications grew more sophisticated, developers flocked to Tomcat, POJOs and the Spring Framework. For every student in an expensive RDBMS class, a hundred developers downloaded MySQL and began to tinker. As Web services specifications became impossibly convoluted, the practice of caching a bit of XML on the browser (which had been going on for years without any official industry-wide sanction) was suddenly blessed with the wonderfully marketable acronym AJAX (Asynchronous Javascript and XML). Lightweight technologies didn't eliminate the need for more robust ones. But they did empower Builders to choose the right tool for each job, depending on the criticality of its function and operational requirements.

And as they did so, an interesting thing happened. Many of the "lightweight" technologies overtook their "enterprise" counterparts. Giant server farms running MySQL on Linux, open source Java technologies, and a host of other proof points showed that the price-performance trade-off had become a scatterplot - "most expensive" no longer meant "most reliable" or "best performing", and you needed Builders to figure out the real story.

Now we are living in a world of Web 2.0, cloud computing, open source, and phones that behave like computers. Our systems are mashed-up, distributed and accessible from our iPhones. We still live in a world of Builders and Shoppers, but the number of Builders continues to expand. Implementations of the Apache web server, GNU-Linux operating system and MySQL database number in the millions. Within the first two years after opening the Google Maps API, more than 50,000 mashups were built. In the six months following the launch of Apple's iTunes App Store, some 15,000 applications have been made available for the iPhone. The Builders are taking over.

So where are the Shoppers? They're still alive and well, mostly in corporate IT departments. Organizational structures and policies evolve slowly, so corporate IT Shoppers are relatively safe, with RFP processes and lists of Approved Vendors. But there's a problem - while the options available to Builders have exploded, the Shoppers are looking at almost the opposite scenario. Driven by Builders' curiosity and creativity, options for programming languages, platforms and frameworks have proliferated. Driven by Shoppers' financial constraints, the software industry is consolidating. It's true that the Builders' world may seem chaotic and fraught with risk. But the Shoppers' world is suffering from a diminishing range of choices.

Of course, no organization is purely Shoppers or purely Builders. But I'm sticking with my distinction, because I think that an IT organization tends to have one kind of culture or the other. Also, it gives some perspective on Nick Carr's famous question "Does IT Matter?" If you've worked in an organization that has an IT culture of Builders, you know that it matters. If you depend on superior knowledge work or operational performance for competitive advantage, I hope your IT organization has a Builder culture. It does matter. A lot.