Creating Synergy and Pride in Software Testing Departments

Excerpted from Managing the Test People: A Guide to Practical Technical Management, by Judy McKay. (Rocky Nook, $39.95 US, $51.95 CA)

We Love Us, Why Doesn’t Everyone Else?

It’s time to be honest here. QA is often considered a necessary evil. This isn’t exactly the description we were hoping for when we took the job. I bet you didn’t go to college and look at the career center postings thinking, “Ah, Necessary Evil, that’s what I want to be.” On the other hand, the “necessary” part is a good thing. QA has to be recognized as necessary. Once you’ve hurdled that obstacle, you can fix the evil (unless of course you like being referred to as evil, in which case you might want to consider a different career—we don’t need you cluttering up the world of QA).

Emphasize the Necessary

How do you make sure your department is considered to be necessary? By providing a tangible product. Remember, you may be competing for funding and headcount with development. If the VP of Engineering has money to spend, how will he decide? If he spends it on development, he’ll get this cool new program that does stuff. If he spends it on QA he’ll get…what exactly? More bugs identified? Well, what good is that? You need to be sure your department is producing a tangible product that can be understood and described to your management. Did you already do the test plans?

Great. You probably also keep track of the test cases, and the bugs you find. But what does that mean to your management when they are thinking about spending money on you? Can you guarantee to your boss that if he gives you two more people, you will find 500 more bugs and make the product safe for humanity? No. But the development manager can promise that two more people will result in two more cool programs that can be seen, demonstrated, and sold for revenue.

Determining ROI (Return on Investment) for QA

This has always been a quandary for us in QA, and particularly for QA management. For all the years I’ve done QA management work, this has been one of the most difficult aspects of the job. Fortunately there is a good way to track and project what the QA group will accomplish, and how additional resources will affect the outcome of the goals. I’m a strong advocate of approaching quality assurance as a risk management function. Given any project (of any size), you first define the risks that you are trying to mitigate. What are all the bad things that could happen if this software doesn’t work? It is a scary thought, isn’t it? But that’s where you start.

Clarify your goals and contributions.

For example, what happens if the software will not install? That is bad: a high-priority risk that must be mitigated. Is it something you can catch in testing? Yes. Can you build a test case or a suite of test cases to check for this problem? Yes. By creating a risk analysis and a mitigation strategy (testing and quality assurance), you are able to present a picture that management will understand. You can offer a giant list of all the risks inherent in this software. You can offer a list of all the things we can do to mitigate the risks. And you can explain how long it will take to carry them out. This can mean executing test cases, participating in design reviews, or even verifying the documentation. Each action requires time and resources. If you have a limited amount of time and resources, you can draw an accurate picture of how much risk mitigation you can offer.

It is important to emphasize that your department’s goal is not to find and report bugs, but rather to provide a risk management system that will help deliver a higher quality product to your customers. For more information regarding how to build and maintain this approach, see Rex Black’s book, Managing the Testing Process.

But They Still Think We’re Evil!

We will assume you’ve established that your department is a necessary cog in the corporate machine. Now we have to dispel the evil aura that sometimes surrounds the QA group. Our industry is not immune to the occasional bad apple. I personally have met some really obnoxious QA people with whom I had absolutely no desire to work. I pity their developers. In the QA world (and more often than in other technical fields), diplomacy is a requirement. As was discussed in the section on interviewing and hiring, you have to start with people who can do their job and who can work effectively with others. Still, even with a group of technically sound and diplomatic folks, there is sometimes the predisposition among developers to judge QA as evil.

Problem Solver or Problem Causer?

The good news is that there is one thing you personally can do to help improve your department’s reputation. You must always be perceived as part of the team that will solve the project problems. This isn’t an easy role, because you are often in the position of pointing the project problems out in the first place. Everyone in management knows it is better to be presented with a problem and a solution, rather than just a problem and shrugged shoulders. So, think about it. Let’s say the schedule is in the tank. What can you do?

Don’t take ownership of a problem that isn’t yours.

First of all, meet with the development manager and see if he has any ideas. He may be the reason your project is in trouble, so perhaps he has thought of ways to get it back out of trouble. If that doesn’t work, get out your carefully prepared risk analysis. You were planning on testing 75 percent of the high-risk items. Now you may only have time for 50 percent. Do you simply move the cutoff line up the chart and get back to work? Probably not. One of the common mistakes we make in QA management is taking ownership of a problem that isn’t ours. So, now our schedule is in trouble, but not as a result of testing issues. What do we do next?

This isn’t a decision you can make. It is time to reconvene the project group (including project management and development), and get an opinion. This is objective data, so you can treat it as such. “Here’s the plan, here’s where we are. We could test just to this point, but I’m worried that we’ll leave these important areas untouched.” It may be that you can reorganize the priorities, and get a wider range of testing done at a higher level. This might be safer in the long run, depending upon the product. The actual approach you settle on will vary by project. The important thing is that you’re working integrally as a part of the project team to find a solution to the problem. It’s not just your problem; it is the team’s problem. By getting the entire team involved, ownership becomes shared (and, negatively speaking, the blame is also shared), proper exposure is given, and a more creative solution may be possible.

Unrealistic Expectations

If expectations are unrealistic, who set them?

As a manager or lead, you may find yourself defending your group against someone’s unrealistic expectations. That’s never a comfortable position, but if you find yourself in it, you need to figure out how those unreasonable expectations got set in the first place. Did you fail to do adequate project preparation, making clear how the testing would be executed? Did they expect the performance problems to be found at the beginning of testing, before you had adequate functionality to run the load tests? Anytime you find yourself on the defensive, you should be able to point to documentation that was previously presented that explains your position. If you don’t have that documentation, maybe you didn’t do your job in the project planning stages. Now you know what to improve upon next time.

The same applies if you find yourself giving anything but an objective response with supporting data. The only reason to be giving an emotional or off-the-cuff response to project issues is that you didn’t do the planning and gather the documentation you needed to support yourself. Have you ever been asked how you know a product is ready to ship? The correct answer is not, “Because it feels right.” The correct answer is demonstrated in charts and graphs showing the risks that were mitigated by the successful execution of the specified test cases. Since (after reading this) you will be doing excellent planning and documenting, remember to regularly publish your progress documents so issues can be raised early.

Presenting Your Information

I worked with a QA manager who had been backed into a corner by a project that was on fire. The development manager had been telling the executive staff that the project was on schedule. The QA manager had not wanted to contradict the development manager in a meeting, but clearly there were still glitches in the software. After much discussion with him, it became apparent that the plan still called for the software to be delivered on schedule, but not in such an order that allowed any functional testing to take place. This was a huge problem. The development manager could honestly say that he had delivered 90 percent of the software to QA, but the vast majority of the software was not testable because the missing 10 percent provided the communication between the UI (user interface) and the database backend processes.

We worked out a way to present the data clearly and accurately to our management, and got the development manager to agree that it was an accurate representation. We took the development manager’s original slide that showed the percent of software delivered by subsystem, and added a column indicating how much of each subsystem was testable, and another column indicating how much had actually been tested. The chart looked like the following:

This gave quite a different picture to management—not a happy picture, but one they could clearly understand, which was our goal. We had the data and were able to present it objectively. Now everyone understood that we had a problem project, and we could work together on a solution. It was also vitally important that the development manager agreed with our data. When we presented this information, he was the first one to be questioned about its veracity. Fortunately, he was honest. He was also an excellent developer and manager. We were lucky.

What if he hadn’t been willing to share the blame and acknowledge the problem? We had documentation for that too. We were prepared to show the detailed risk analysis, which showed the prioritization of our test cases. In our risk analysis, we had highlighted which risks we had been able to mitigate—a pitifully small number. We also had the list of all the necessary test cases and which ones we could actually execute. Knowing that this would bore our non-technical executive team, we picked out a few selected cases that showed the glaring lack of functionality. All of this documentation supported our information (which, by the way, was correct and unembellished).

Present a united front with development.

These kinds of situations highlight the importance of working with the development team and the development manager, right from the start of the project. If you don’t have a solid relationship built on honesty, this kind of crisis will escalate. If you present your data and the development team debates it, you can haul out all your backup information (as we were prepared to do), although management is still left with the uncomfortable impression of internal quarrels and unnecessary impediments to getting the work done. If development is favored in your work environment, you are likely to be the recipient of the blame for the problems of not being able to “work together.”

This is an unfortunate reality in many organizations. The only way to fix it is to build a positive relationship from the beginning, so you are always working together on the project with the developers. This is easy to say, but hard to do. Unfortunately, you can be technically sound and absolutely correct with all your information, yet still find yourself in one of these situations where the development group has managed to cloud the water with enough squid ink that no one is sure who is at fault. Uh-oh. It is best not to infuriate the squid. Work with him, and make him work with you throughout the project.

Job Satisfaction

That’s right, we’re cool. But sometimes, even with all the correct planning and documentation, a product ships before it should. I worked in one company that was completely driven by hardware schedules. When the hardware was ready, the accompanying software shipped, regardless of its quality. And, to make matters worse, my division only did the initial software release. Another company did all subsequent maintenance releases, so we never got to see any product improvements over time.

Pride and Ownership

Good QA people want to deliver good products. The more ownership of a product or subsystem of a product they feel, the more they care about its quality. Part of their job satisfaction comes from knowing they did a good job managing the risks of the product and saving the customer from the horrors that the previously buggy software would have caused. Unfortunately, that’s not really what the job is. As a manager, you need to make clear to your people what the job is not. It is not to ensure that the software has no bugs when it gets to the customer. It is not to ensure that the software is the coolest stuff on the market. It is not to ensure that every release will be shipped on time, and will be a stunning success in the field.

Our job is to provide factual information to the decision makers.

So what is the job? The job is to execute all the scheduled test cases to mitigate the priority risks. And, yes: You also want to find and document bugs, and accurately identify and classify those bugs. The job is to make a summary appraisal on the quality of the software by evaluating the risks. And, probably most importantly, the job is to provide factual information to the decision makers. This can be a very different picture from the expectations your people may have for their jobs, and the areas from which they expect to gain job satisfaction.

Risk Mitigation

Personally, I love to find bugs—the more complicated and elusive the better. That is what satisfies me. I am downright unhappy if I run a bunch of test cases and don’t find any defects. If my test cases don’t find bugs, does that mean I’m not doing a good job? Not at all. My job is to mitigate risks, not find bugs. The fact that I ran the test cases means I mitigated the risks those cases cover. You want to orient your group so that they are driven to mitigate risks, not find bugs. If the goal is to find bugs (or worse), and if evaluation is based on the number of bugs found, what happens if you get stuck testing software that’s very solid and works well? We spend a lot of time testing things that work (we hope!).

You want to be sure you get the credit for all the time spent, not just the time that resulted in bug detection. Be sure to publish the test case execution results, and the risk mitigation results along with your bug reports. Upper management is often fixated on bugs found rather than risks mitigated, and these are very different results from the testing effort. Spend the time to train your management so they understand the goals. In this way you can reinforce those goals with your people, and increase their job satisfaction when they have completed test cases (whether or not bugs were found).

Ship It!

It’s your job to constantly remind people that they don’t have to agree with the decision to ship, but they are expected to provide accurate information. They don’t have to fix the bugs, they just identify and classify them correctly. They don’t need to take on a personal crusade for the quality of the software. If you can get these messages across to your group, they will be happier in their jobs, and will enjoy their contribution to their projects.

Those who like what they do will do a better job, work harder at it, learn what they need to know on their own time, and make their own rewards. You can’t control someone else’s job satisfaction, but you can provide an environment which will allow them to glean what they need.

Synergy, Pride and Celebration

Celebrate the little victories.

As you rise higher in management, it can be easy to forget to celebrate the little victories. You move from one project to the next; as soon as one project is past the crisis stage, you’re on to planning the next one (the next project, not the next crisis!). It is all too common to forget to celebrate the completion of the previous project before turning to the next one. Don’t forget. Just because your own reward system may have changed, you still have people who were fixated on a project for some amount of time. They need to celebrate the successful completion—have a party!

Creating synergy and pride within your department is your job. Be a leader! How much does it mean to you to get a pat on the back? Remember, as you go higher, there are fewer tangible rewards. (On the day I received a promotion, a friend rudely pointed out to me the higher up the tree the monkey goes, the more of what is seen is the monkey’s behind.) Just because you don’t get them doesn’t mean you shouldn’t give them. Be sure you recognize successes, and the effects of those successes. Your group may have just facilitated an early launch of a highly competitive product. They may not know that they were part of a project that just snuffed out the competition. Tell them! Celebrate the victories. Acknowledge and console when there are defeats as well. My department was once involved with a software release that never should have happened. It shipped (to our horror) on the scheduled date, despite the overwhelming evidence we presented. We knew the product would fail in the field, but we had done our job and supplied accurate information to the decision makers. Even though we felt rotten about the product, we had a project-completion luncheon anyway. We needed to spend a little time collectively as a group to discuss the business reasons why the decision to ship the product had been made, and I needed to remind everyone that we had done the job we were supposed to do. Everyone left the lunch feeling better about the work they’d done, and were energized to tackle the next project.

Never miss a chance to celebrate together as a group. Bring in doughnuts for hard days and nights. Give time off for extra hours spent. Treat people as if you are grateful for the work they’ve done, and the commitment they’ve shown. Remember, you are only as successful as your team makes you, so create genuine rewards for genuine effort. The little things do matter, so remember to say thank you, and send a nice e-mail now and then. Food is always appreciated, so take people to lunch or buy them bagels for breakfast. Make people glad they work for you.

Deflecting Criticism and Building Respect

You’re part of the team that will provide the solution to the project’s problems

You have to defend your department from internal and external forces. You may find yourself in the role of deflecting criticism. QA generally receives a lot of criticism, and that is partly because we generate a lot of criticism. Isn’t that what identifying a bug is? We can say that it’s documentation ofaberrant behavior, but to the developer, it is a criticism of their work. Frustration can run very high with developers, particularly when schedules are tight and bad bugs are found late in the process. Sometimes this will cause them to lash out, to you or to their management. So, what can be done toprevent hard feelings? As we’ve discussed in previous sections, a well-planned and well-documented test process will help relieve the frustration and finger pointing when the schedule blows up. Everyone will have already seen and agreed to the plan. You may not be able to follow that plan, but now you are completely within your rights to discuss any deviations with the stakeholders in the project. You’re part of the team that will provide the solution to the problem; you’re not the problem.

But What About Internal Problems?

In addition to deflecting criticism from outside your group, be sure your team does not explode from internal combustion. If you have problems brewing within (and you usually will), be sure you’re constantly working to alleviate them. It is always best to learn about an internal problem from internal sources, but sometimes things boil over when you’re not around. Be sure to listen! People need to feel free to talk to you or they won’t bring you problems. If you’re hearing about internal problems from external sources, you have communication issues in your department that need to be solved.

Be sure your group knows they can count on you. You need to be ready and able to act appropriately when faced with a personnel issue. If you criticize your group members to other people, they will find out and their trust in you will be destroyed. If you criticize group members to other group members, you should be shot. Well, maybe that’s a little strong, but you get the point. There is no excuse for a manager to criticize one’s peers. It makes everyone uncomfortable and destroys trust. The worst manager I’ve ever worked with criticized his people to non-management and management people, within and outside his department. Usually, a group with a bad manager will tend to band together against him. This is one of the least effective team-building strategies! In the case of this manager of mine, he created so much distrust between the team members that they hated him and they distrusted each other. It became the most dysfunctional group I’ve ever had the misfortune with which to work.

Don’t confide in others inappropriately. If you need to talk about an employee, find another manager at your level. Or get a dog—they can always be trusted to keep your confidences. If you do decide to confide in another manager regarding your personnel problems, be sure you have an action plan. Don’t just whine. If you’re asking for advice, do so. But be prepared to act on that advice or explain why you won’t. You don’t want to appear weak and unable to deal with an employee problem. Be professional and think about what you want to gain from the discussion before you have it.

Remember to Remain Objective and Honest

When someone brings you an internal issue, what should you do? You want them to confide in you, but you don’t want to give the appearance that you are agreeing with them. At this point, you need to be objective. You’ll have to check the data to see if what you were told is valid, and determine if action is needed. Be very careful what you say, as it is likely to be repeated. If someone comes in your office and says, “I hate working with Joe; he’s always such a grump in the morning,” don’t nod your head and say, “Yeah, I noticed.” You can bet as soon as your complainer leaves the office, the whole group will know you said Joe’s a grump in the morning, and you hate working with him. Depending on the speed of your grapevine, you can probably count on a visit from Joe before lunch, and he definitely will be grumpy!

When someone does bring a problem to you, be sure you are honest with them. Be sure they leave your office confident that they have given you information and you will act appropriately. This doesn’t mean to encourage tattling. The last thing you want is to be resolving every little personnel issue that occurs. You are paying people adult wages and they should act like adults. At least that is what I keep telling myself. There is nothing wrong with sending someone from your office with the advice that maybe they need to work out the problem on their own, and they should come back if there are further issues.

Encourage and reward honesty.

You need to be sure that you have people in your department who will be honest with you, even with something you don’t want to hear. I once made a presentation to my department in which I discussed a job classification change that affected the Senior Engineer requirements. Unfortunately, during the presentation, I misspoke and said Engineer instead of Senior Engineer. That would have made almost all the Engineers unqualified for their current jobs. I was very grateful for the person who raised their hand and asked for clarification. That would have been a terrible message to leave with the group. Encourage and reward honesty, and be sure you are honest in return. It generates and reinforces trust.

Include Your Staff Members in the Solution Process

Sometimes your department may be under fire, and you need to make some changes. What do you do? I once had a situation in which I had been told our customer felt the QA department was lazy. After significant investigation, it turned out they thought we were lazy because we allowed food in the lab, and they felt that people who were eating weren’t working. On the contrary, by allowing food in the lab, people worked through breaks and lunchtime, and we were able to get maximum utilization of the equipment. The customer would not be convinced that this was an efficient way to work. They felt that chip bags and Coke cans in the lab indicated slovenly test habits. Rather than make a no-food ruling that would be seen as a significant negative to the work environment, I called a meeting with the entire department. We discussed the problem and reached the agreement that all food and drink would vanish from the lab when the customer was on the premises. At all other times, food was acceptable. Since they had been included in the problem resolution and had agreed to the solution, the group complied with the new rules and even enjoyed being part of the conspiracy.

If your problem is such that you can include your team into the solution process, do so. It allows them to see some of the issues you are dealing with, and makes them a part of the overall organization. In addition to getting your group comfortable with you, it’s a good idea to make friends elsewhere in the organization. Create a solid, open working relationship with your peers, particularly the development peers. They can be your strongest allies or your worst enemies. You choose. You will almost always win your battle for more resources if you can present your case with backup from the development manager. You will almost always lose the battle if you don’t have that backing. It is up to you to cultivate these valuable friendships and alliances. It may cost you a few lunches, but hey. You were going to eat anyway.

Excerpted from Managing the Test People: A Guide to Practical Technical Management, by Judy McKay. (Rocky Nook, $39.95 US, $51.95 CA. You can order it here.

Judy McKay has spent the past 20 years working in the high-tech industry with particular focus on software quality assurance. She has managed departments encompassing all aspects of the software lifecycle including requirements design and analysis, software development, database design, software quality assurance, software testing, technical support, professional services, configuration management, technical publications and software licensing. Her career has spanned across commercial software companies, aerospace, foreign-owned R&D, networking and various Internet companies.