How to Fix The Productivity Problem That’s Killing the Software Industry

Back in October, 2008 I wrote an article for CIO.com entitled, “How Better Software Can Save the World.” In the article I expressed my amazement and frustration over the tragically flawed methods used to produce software.

So much has happened since October that it feels as though a century has already passed. My amazement and frustration over the state of software development has been replaced by sadness and anger.

So, where do we go from here? Where in the wreckage do we find the resources we need to engineer a genuine business recovery? Most companies have already slashed their human capital investments to minimum levels.

Many of the managers and executives I’ve spoken with in recent weeks now use staff cuts as a catch-all excuse for inaction. A feeling of paralysis has set in. My great fear is that if this sense of helplessness becomes the prevailing mindset, we will become incapable of generating the emotional energy required to achieve a lasting recovery.

IT is uniquely positioned to act against this danger. In Automated Defect Prevention (Wiley, 2007), Dorota Huizinga and I described a software development infrastructure that automates repetitive tasks, organizes project activities, tracks project status, and seamlessly collects project data to facilitate decision making. As we wrote in our book, this infrastructure forms a software production line. It also represents a paradigm shift in the software industry, and by extension, a paradigm shift in the way IT looks at productivity.

My new book, The Next Leap in Productivity, builds on this concept to make a strong connection between IT productivity and enterprise productivity. Boost the productivity of IT and you will boost the productivity of the business.

The best and most logical place to start doing this is at the software development level, since every service delivered by IT relies on software.

When the business needs to change direction quickly, new software is required to enable the change. That’s the fundamental reality of modern business. You really can’t do much of anything until IT has the right software in place to make it happen.

But as we all know, it can take forever to develop a new piece of software. When the worldwide economy was booming, the glacially slow pace of software development was taken for granted. People would say, “It is what it is, like bad weather or visiting the in-laws.”

Their fatalism was perhaps understandable. Until recently, the only way to hasten the pace of software development was by throwing more bodies onto the production line. Occasionally this strategy worked, but most often it just resulted in even higher levels of inefficiency, more bugs and no appreciable gains in speed.

With the economy getting worse by the minute, we can no longer afford the fatalistic attitudes of the recent past. Today, 30 or 40 properly trained and well-managed people can do the work of 300 or 400 developers—in a fraction of the time.

This is not science fiction. This is fact. And this fact has consequences. Let me give you an example. I recently spoke with an executive at a technology company and we discussed the current economic crisis. The executive told me that the company had laid off many of its developers and was scaling down the size of its software projects as a result.

I told the executive that my firm, Parasoft, had recently finished a Java- based project to create version 6.0 of our SOAtest product. The project required us to write about 5.5 million lines of code. We completed the project in 18 months with only 34 developers.

Based on what I knew about its approach to productivity, I estimated that the executive’s company would have needed 500 developers to complete a similar project in the same timeframe.

We were able to accomplish this “miracle” by following the methodology described in Automated Defect Prevention. There was no simple silver bullet or secret sauce—just a clear methodology that enabled continuous process improvement within an automated infrastructure.

I honestly don’t believe that it takes a genius to see how this kind of productivity can be turned into a huge leap in enterprise productivity.

For example, a business that gets new software up and running faster than a competitor will get its innovations to market long before competitors who stick with traditional development methodologies.

Maybe I’m just shouting into the wind, but it seems to me as though we no longer have a choice. If companies intend to remain competitive, they will need new software. But they don’t have enough people to develop new software using old-fashioned ways.

The basic technologies for improving the pace and quality of software development, such as work distribution and monitoring, static analyses, unit testing, code review, protocol testing and so on, already exist.

In the past, fear of change prevented management from giving these technologies a serious examination. Now is the moment for us to get over our fear and make the leap into a new world of much greater productivity. The key is for management to re-envision the software development process as a modern production line in which the quality of work is validated at every stage and defects are caught during the process, instead of at the end of it.

Adam Kolawa holds a Ph.D. in theoretical physics from the California Institute of Technology, and has been granted 16 patents for his recent inventions. He and Dorota Huizinga are coauthors of Automated Defect Prevention: Best Practices in Software Management (Wiley, 2007). Kolawa’s new book is The Next Leap in Productivity: What Top Managers Really Need to Know About Information Technology (Wiley, 2009).