The secret to successful projects: Peer reviews

A project has a high probability of success when the project manager understands the key factors that determine success or failure and has the authority to make adjustments when needed. Unfortunately, this is difficult, because even the best project managers encounter situations that are foreign to them, and even when they do identify a situation that needs attention, there can be political or other reasons that prevent them from making adjustments.

In addition, project managers are often too close to the details, and when things go poorly they tend to adopt overly optimistic views of future progress. The secret to making sure projects have the best chance of success is to conduct peer reviews to help the project teams understand their points of vulnerability and to make sure the obstacles to success are addressed.

Key success factors

Before discussing the mechanics of a peer review, there are a few key success factors that must be noted upfront as being critical to the final outcome.

The selection of the peer reviewer is critical

The person conducting the peer review must be someone the project team respects, so their conclusions are taken seriously. He or she must also be someone senior management respects, so they can influence the senior team to make changes if they are necessary.

The purpose of the peer review should be to improve the probability of project success

It should be a collaborative process between the reviewer, the project team and senior management. If the project team believes the review will help them understand where they really are with the project and overcome obstacles, they will be open and the review should be successful. If, instead, the project team believes the purpose of the review is to identify their shortcomings and report these findings to senior management, they will feel threatened and will attempt to conceal issues.

With senior management, I have seen many situations where peer reviews are either a “check the box” exercise or an inquisition to identify failures. In these cases, the project manager will react accordingly and do whatever they can to simply survive the review. Successful reviews happen when the senior management team understands that all projects have risks, and the purpose of the peer review is to allow them to understand whether a specific project requires their assistance to mitigate a particular risk.

Peer reviews should be done multiple times during a project

The first review should be done very early in the project and subsequent reviews should be performed soon after the start of a new project phase. The key is to conduct reviews when adjustments can best be made to improve the probability of success. Early reviews allow the project team to take preventative steps to improve the probability of success, while later reviews can be reactive if they are done after issues have already begun to surface.

The point about conducting the first review very early on is important, because many large projects struggle significantly to get started. I have seen several projects that don’t start moving forward until several months have passed, due to management focusing on staffing the project before they determine how to best move forward. In these situations, the project team consumes much of the project budget without making much progress. Peer reviews near the start of a project can help the project team focus on the right items.

Mechanics of a peer review

The following sections provide an overview of how to do a peer review. At first glance, this may seem like quite a bit of work, but you can scale up or down depending on the importance of the project.

Mission critical projects will want to adhere closely to the process as the cost of failure easily justifies the expenditure of time. While it is easy to use a dollar figure to determine whether a project should do a full peer review (all projects great than $1 million in cost, for example), the best approach is to have the senior management team review the project portfolio and identify the projects that will go through this rigor.

Less critical projects might use some peer review techniques but stop short of involving senior management, unless there is a gap that needs to be addressed. A technique I have seen used for less critical projects is to simply convene the project team to discuss how the project is doing in the areas that often derail IT projects. The questions listed in the following section provide an excellent framework for this discussion.

Conduct a project team questionnaire

The first step in a peer review is to obtain the perspective of the project team, and the easiest way to do this is to have every team member fill out a questionnaire.  The questionnaire should ask questions that are intended to surface issues that often lead to project failures. Over time, I developed a questionnaire that I think does this job well. It contains 11 questions for team members to evaluate on a 1 to 5 scale and concludes with an open-ended question. It can be filled out and analyzed quickly so this step in the peer process does not consume a great deal of time. Here are the questions I use:

1. Are the requirements complete and understood?
2. Is the scope of the project stable?
3. Is there executive support for the project, and is the executive sponsor engaged?
4. Are the right people (business, technology) and enough resources assigned to the project?
5. Does the project team have the technology skills to do the work?
6. If the project is using new technology, is the new technology well understood and working well?
7. Are clear metrics in place to track project status?
8. Is the delivery timeframe realistic?
9. Is ownership and accountability for deliverables and milestones clear?
10. Does the team have high morale and a collaborative team dynamic?
11. Does a formal quantitative business case justify the project?
12. What is the one thing you would change on the project and why?

Analyze the questionnaire results

The second step in the peer review process is to analyze the results of the questionnaire. While much of this work is simply calculating the scores and looking for outliers, the reviewer also will want to be able to look at the results from different perspectives. For example, it is important to understand if the business participants on the project score questions much differently than the IT participants; if junior participants view things differently than more senior participants; and if certain groups have a different perspective — if, for example, the programmers have a different view than the analysts.

While the first 11 questions will provide a very good understanding of how the project team views the project, it is the last question that usually provides the most benefit. In my career, I have found the answers to this question almost always become key components of the final report.

Conduct project team interviews

Once the view of the project team is understood, the next step is to conduct interviews to obtain more detailed information. Interviews should be done at all levels, and the focus should be for project team members to communicate their views of the project health. 

The reviewer should specifically ask the interviewee if they answered the open-ended question and spend some time probing any subjects raised. In addition, the reviewer should ask the interviewee about any low scores or inconsistencies that were seen from the results of the questionnaire. For example, if the programmers feel the requirements are not as complete as the analysts think they are, the reviewer should ask for further information in the interviews with programmers and analysts.

Document the findings and develop conclusions

Following the interviews, the reviewer should document the results and develop their preliminary conclusions. Then, they should review their findings with the project manager. If the reviewer and the project manager are not in sync, they should determine how to best close the gap. For example, the project manager and the reviewer may want to invite project members into their meeting to discuss certain findings.

It is in the best interest of both the project manager and the reviewer to come to an agreement, because their chances of obtaining senior management help to close gaps will be much better if they are united. Finally, they should outline the key points to be made in the final report.

Write the final report and present the findings to senior management

The final step in the peer review process is to document conclusions in a final report and present these findings to senior management. The purpose of the presentation is to share with senior management where there are potential vulnerabilities on the project and to obtain their assistance in closing these gaps. It should be a joint presentation with both the project manager and the reviewer offering their thoughts, and they should clearly define what it is they need from senior management to help the project.

To emphasize a point made previously, the focus of senior management must be to help the project team going forward rather than to look for ways to punish suboptimal results. It is very important to give the project team a chance to “right the ship” or the information shared in peer reviews will deteriorate significantly. The time for senior management to take a tougher approach is only after a review has surfaced an issue, and they have taken steps to help the team address the issue but the issue remains a problem.

Summary

I remember the first time I was on the receiving end of a peer review. At that time in my career, I was working in the consulting business for a company with partners at the top of the pyramid. The peer reviewer told me he knew it was my first review, and he wanted to share a perspective with me.

He said when a partner in the company has a problem on a job, they don’t hide it. In fact, they do exactly the opposite by telling every other partner who will listen about their issue. In this way, their problem becomes everyone’s problem. He told me his job was to help me assess the project status and then, if needed, to help me obtain any help I needed to be successful. When peer reviews are conducted in this manner, they can be a powerful tool in helping projects succeed.