Software Architecture: 7 Keys for Success


As a technology executive, I'm often appalled by the lousy track record of software development organizations. The Standish Group reported a few years ago that only 30% of all IT projects are considered to be an "unqualified success" - satisfying business requirements and completed on time and on budget. 53% of all in-flight projects were said to be challenged or seriously compromised. In 2004, it was reported that the average schedule overrun was 84% with cost overruns averaging 56%. Computerworld Today also reported in 2004 that fully 1/3 of all IT spending went to repair botched projects. Many IT projects fail completely, with the plug being pulled after enormous financial cost - sometimes as high as $500MM.

There are exceptions, of course - technology teams who consistently deliver quality systems on time and on budget. I like to think that the technology organizations that I've led at American Express and MapQuest are among these exceptions. Nobody's perfect, and we've certainly had our failures, but I believe our track record is generally characterized by successful projects.

There are a number of reasons for this, including great engineers and testers, careful attention to business requirements, agile development methodologies, automated testing, and solid engineering practices, and I intend to write about these things in the future. But perhaps the single most important reason for our success has been the great software architects that I've been privileged to lead. Great architects can see an entire blueprint in their heads, identify risky areas, and avoid the traps of poor design and incomplete requirements. Great software architects are the glue that ties the IT organization together, and I'm not exaggerating when I say that the architects on your technology development team can make the difference between business success and failure.

So, what are the marks of a great software architect? The following thoughts provide an answer to this question and are drawn from my experience as an architect and as a technology leader.

Don't be a Perfectionist.

Good software architects are first engineers and are, by definition, extremely analytical. They not only understand things like Fibonacci series and Functional Closure, but they like them. They tend to enjoy mathematics and like things that are perfect, complete, pure, and unambiguous. Good architects are able to easily assess the strengths and weaknesses of systems. They can can spot brittleness and unnecessary complexity. But it's all too easy to fixate on weaknesses of system and spend much time and energy trying to eliminate its weaknesses.

CTO's can also get caught up in quests for engineering perfection, but the good one's will resist such urges. It's interested to note that in my career I've NEVER been able to win the support of the business for a significant project that ONLY fixed a system imperfection.

Great architects focus on what drives business success vs. engineering perfection. Only business results matter. As CTO, one of my most important jobs is to clearly understand the business strategy, then develop and implement technology strategy that helps drives the business strategy forward. I count on my architects to help me maintain tight a tight connection between business goals and the work actually being done.

Be Like a Spider

I believe that good architects must have good people skills. We will sometimes forgive talented software engineers who are hermits living in their darkened cubicles, cut off from the outside world by noise-canceling headphones and pacified with an occasional offering of raw meat. But architects cannot be such Neanderthals. This is because architects must interact with all types of stakeholders and get buy-in for expensive ideas from a lot of people. This requires careful attention to relationships. Like a spider, architects sit in the middle of a web of important relationships. If too many strands break, then the entire web will collapse.

When I read grim statistics about failed IT projects I often wonder how many of them failed - not because of bad architecture - not because of incompetent engineers - but because of failed relationships. Great architects recognize the importance of this web of relationships and carefully invest time in keeping their web intact.

Balance is Everything

Technology architects are constantly caught in tension. On one hand, there is constant tactical pressure to rapidly deliver business results. The modern business environment is fast-paced and competitive. Business owners are under tremendous pressure to deliver results fast. As a result, they sometimes seem like my four year-old son, who in the space of 20 minutes rides his bike, rides his scooter, has a mock sword fight with his brother, and eats a grape popsicle.

This is especially true in public companies who are focused on posturing for Wall Street and often struggle to see beyond the next quarter's earnings report. In my experience there is no tolerance in a public company for investing in projects that don't produce a positive ROI within 18 months of inception. In an Internet company, the collective attention span of the business is even shorter. Most technology teams suffer in such environments if they cannot deliver most projects in is 60-90 days.

On the other hand, the CTO/CIO is accountable to deliver and maintain sustainable systems for long-term success; systems that are flexible, robust, extensible. Strategic investments must be made in people and technology in order to ensure that the company's technology remains viable over the long haul. In The 7 Habits of Effective People, Stephen Covey talks about the difference between Production (P) and Production Capacity (PC). This is illustrated by the fable of the Goose that Laid the Golden Eggs. Production is the golden egg. Production Capacity is the goose. Fail to care for the goose and you'll soon have no more golden eggs. The CTO's challenge is to meet Production goals while continuously investing in improving Production Capacity - constantly evolving systems, improving testing methodologies, refining the SDLC, refactoring code, re-platforming systems, upgrading networks. If he doesn't do this, he places the company at risk. Yet there is constant pressure to throw all of the company's technology resources at the current hot project.

A good architect understands this tension and works to maintain a healthy balance between tactical and strategic concerns. A great architect leverages energy from tactical business projects to achieve strategic technology goals.

Be a Weaver

At some point you are going to need to ask the business to invest in strategic technology project - re-platforming, refactoring of components, swapping out your database, etc. This will test your credibility with the business because it will require you to divert resources that could be used for short-term tactical projects. It's natural for business partners to become tense and upset when such work gets in the way of tactical projects. So don't be surprised when you hear them in the hallway muttering about how the Lunatics are Running the Asylum.

Agile Development can be your friend here. By running in short iterations you have the opportunity to knock out strategic work without asking your business partners to wait too long for valuable development resources. Most business partners will wait - but not too long. So good architects help the business break big projects into sub-projects that yield early ROI and make scheduling easier. This can be easier than it sounds, since product managers are fantastic at pork barreling. (I've seen single projects that easily contained five standalone projects and included requirements to refinish the product manager's hardwood floors.) My solution for this is to have the development teams work in two-week development cycles (sprints) - producing "happy", releasable code at the end of each sprint. I depend on my architects to work with business partners to decompose projects into sub-projects that rarely exceed 30 days. This minimizes the time that we lock up development resources and gives the business a lot of flexibility.

A good architect is adept all of this - working with business partners to decompose large projects and at working with the development teams to schedule project work in manageable two-week iterations. This makes it easier to weave strategic work in with tactical business projects.

Don't be a Bottleneck.

As valuable as architects are in my organization, I personally know of IT shops where the architects are actually an obstacle to progress. This generally happens when architects are too far removed from the actual development work or do not have the right level of technical expertise. It is a sad thing when an architect's engagement in a project becomes overhead in the form of never-ending "architectural reviews", "JAD" sessions, etc.. Such reviews are sometimes necessary, but can easily become a costly burden to the business. In my opinion the key to avoiding this overhead is putting the architects as close to the work as possible. CTO's and CIO's can help this by thinking carefully about the location of architects in their organization. If you are an architect, you probably have little control over the design of the organization, so the best you can do is to try to stay as close to the work as possible in your day-to-day activities.

Architects also form bottlenecks when they don't maintain a working knowledge of the systems and technology being used by the development teams. The most severe examples of this in my experience have been cases where architects were expert in mainframe computing, but didn't have sufficient grasp of modern languages and distributed computing to make intelligent design decisions. Yet, because of their tenure, they were in senior positions of great architectural influence. Architects need to be seen as experts, but no single person can know everything about technology. There's no crime in this, but it's important for architects to constantly work to update their knowledge, and then to recognize their limitations and manage around them by building a network of experts who can be called upon for information and advice.

Don't Become a Dinosaur

In IT, software engineers sometimes become software architects and architects sometimes become CTO's. Indeed, I think that the role of CTO makes an excellent goal for any successful architect. But ambitious architects should think carefully about this. Not everyone wants to put up with the hassles of being a CTO, since the role of CTO often involves managing people and dealing with business more than the technology itself. The role of the CIO also involves managing people and almost always involves dealing more with finances and corporate policies than technology. For many, the role of software architect is a natural and more satisfying place to live out the balance of one's technology career.

They key is to keep your technical knowledge up-to-date so that you can provide valuable leadership to your company over a long period of time. This, again, involves proactively networking with other technologists and engaging in regular study to stay abreast of technology trends. On a side note, vendors can provide valuable intelligence on technology trends but architects must not rely on vendors for their ongoing technical education.

Build on Your Strengths

I love reading, and read something from at least one book nearly every day of my life. I grew up in a home with no TV, but my mom would take us to the local library every couple of weeks and fill up a cardboard box with 20-30 books. Today, my kids have TV, but my wife - who is a talented homeschoolinging mom - checks out 30-40 books a week for our four kids. Reading is big around our house.

I've found that reading good books gives me access to the thinking of some of the smartest people who have lived on Earth and provides wisdom for working through tough business and management problems. When you read enough books on the same subject you begin to detect recurring, timeless themes. Peter Drucker wrote about one of these themes 40 years ago in a great book called "The Effective Manager". Drucker said that effective managers Build to Strengths and Manage Around Weaknesses.

We hate weakness of any kind. As Americans this is probably related to our culture of independence and autonomy. As managers, we try to hire people with no weaknesses and subsequently end up with a bunch of mediocre people. In school, we insist that our kids be great in everything rather than helping them discover and build on their individual strengths. I like what Marcus Buckingham says about this in his book, "Now, Discover Your Strengths". He says that in our attempt to avoid weakness, we tend to treat people like checker pieces where everyone has precisely the same abilities. But people are more like chess pieces than checker pieces. Every person has unique strengths just as the knights in chess have unique capabilities that are different from the bishops and rooks. Great managers know this, and hire people for the particular strengths that can be powerfully brought to bear on the needs of the business

Building to Strengths is vital in management, but understanding and building on strengths is perhaps even more important on a personal level. When the Gallup Organization polled employees a few years ago, they found that only 20% of those surveyed responded "yes" when asked if they had the opportunity to do what they're best at every single day. This is tragic. If you do work that takes advantages of your strengths, you are sure to have a life that is more successful than if you you're in a job merely because it provides a higher salary.

Architects need to ensure that they truly have the technical and analytical abilities that make them a great fit for their roles, then focus on building on their strengths rather than trying to overcome weaknesses.

Manage around your weaknesses and invest your time and energy in the things that you do best every single day.

This entry is a summary of a speech that I recently presented to the International Association of Software Architects - Denver.

About this Entry

This page contains a single entry by Kevin Survance published on August 11, 2008 12:37 PM.

In Vain You Rise Up Early was the previous entry in this blog.

Death to Boring Staff Meetings! is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.