The Final Findings Report is what company officials present internally to the big brass and to customers knowledgeable enough to ask for it. But there’s no obligation to do it. They can declare their CMM level without producing any evidence. They can even hire their own lead appraisers inside the company and assess their CMM capabilities themselves. They don’t have to hire a lead appraiser from the outside who might be under less pressure to give a good assessment. And they can characterize their CMM level any way they want in their marketing materials and press releases.

SEI officials say they are not in the business of controlling what companies say about their assessments. Nor will they reveal to the public which companies have been assessed or what the assessments consisted of. "We weren’t chartered to be policemen?we’re a research and development group," Hayes says.

Instead, the SEI exerts control through the relatively small lead appraiser community (approximately 220 are authorized to do CMM assessments). From the beginning, the SEI has reserved the right to discipline or even remove appraisers who cheat or do their jobs badly. But in the early days, the SEI rarely followed through on those threats, say longtime appraisers.

More recently, the SEI toughened up the CMM itself and plans to completely replace it (as of December 2005) with a broader, more in-depth model called CMMI. In the process, it has increased the training requirements and controls on appraisers. According to Hayes, under CMMI, the SEI reviews each appraisal that comes in for irregularities. And under CMMI, appraisers have to file a report called an Appraisal Disclosure Statement that clearly states which parts of the organization and projects were assessed, as well as all the people who took part in the assessment (though assessed companies are not required to reveal that report publicly, either). The SEI, along with the lead appraiser community, is also developing a "code of ethics" for appraisers.

Yet if CIOs want to get the true picture about appraisers, to check if they’ve ever been reprimanded for performing faulty assessments or thrown out altogether for cheating, they are out of luck. The SEI will not reveal any information about errant appraisers.

And the SEI has no intention of becoming a governing body like the American National Standards Institute (ANSI), which controls ISO 9000 certification in the United States. ANSI requires companies to be reassessed every six months if they want to maintain their ISO 9000 certification and reassesses all its appraisers each year. "No one has asked us to become a governing body, and that’s not our mandate. And if we did, what would that solve?" the SEI’s Humphrey asks. "It wouldn’t excuse anyone from doing their homework."