At lunchtime one day last January, Jill Hein, an Iowa mother of\n eight, took a jar of Peter Pan peanut butter\u2014the kind\n with Peter, in his feathered cap, on the label\u2014out of her\n pantry. She opened the lid. Everything seemed fine. No funny\n odor. No odd color.\n\nRELATED LINKS\nBeyond Peter Pan: Lessons from ConAgra's Recalls\n\nFive Best Practices for Recalls\n\nSoftware to Smooth the Recall Road\n\nUse Disasters to Fine-Tune Your Supply Chain\n\nTime Line: 15 Years of Recalls\n\nTainted Goods from China and Other Supply Chain Risks\nHein fed the Peter Pan to one son and one daughter. Within\n hours, they were cramping and vomiting. Hein\u2019s\n 3-year-old boy, Bowen, had to be taken to the emergency\n room the next day. Hein ate some herself two weeks later and\n was hospitalized for dehydration. And renal failure.Alone or with jelly, peanut butter is as classic as Elvis,\n who preferred his on white bread, mashed with bananas and\n fried. Americans eat 700 million pounds of crunchy and creamy\n each year\u2014enough, the Peanut Advisory Board says, to coat\n the floor of the Grand Canyon.Hein never expected a simple peanut butter sandwich to go so\n wrong.Neither did ConAgra Foods, the $12 billion conglomerate that\n makes Peter Pan.One of ConAgra\u2019s oldest and best-known brands, Peter\n Pan brought in $109 million in sales last year, says\n Information Resources, which tracks retail spending. ConAgra\n also supplies some of Wal-Mart\u2019s Great Value house brand\n and sells peanut butter toppings to companies like Carvel and\n Sonic, bringing total peanut butter sales to $147 million last\n year. But when an outbreak of a rare salmonella strain was\n traced to ConAgra peanut butter, the company would have to try\n to get it all back.The Peter Pan recall eventually involved 326 million pounds\n of its own and Wal-Mart\u2019s peanut butter, plus 99,953\n cases of toppings. So far, ConAgra has spent more than $78\n million dealing with an estimated $1 billion worth of\n potentially infected product. Its peanut butter sales were down\n 63 percent in fiscal 2007, the company says.No one knows how much ConAgra will need to spend to\n re-establish trust in its product. Hiring Tinker Bell to ask\n people to clap if they believe in Peter Pan won\u2019t fix\n this.Why Recalls Depend on the Supply ChainPeanut butter isn\u2019t ConAgra\u2019s only recall\n trouble, either. The company has had to call back hundreds of\n pounds of ground beef in the past few years, and this month\n ConAgra\u2019s Banquet pot pies were recalled when at least\n 211 people in the U.S. got salmonella poisoning, which the\n Centers for Disease Control and Prevention links to the pot\n pies. That recall is ongoing.But it\u2019s not just ConAgra. Recalls are blooming like\n flowers in spring: Dole\u2019s e.coli bagged salads; Metz\n Fresh\u2019s salmonella spinach; REI\u2019s faulty\n children\u2019s bikes; Mattel\u2019s lead-painted and\n choking-hazard toys, just to name a few. Federal records show\n at least 628 recalls so far this year, and another\n 941 in 2006. (For more, check our History of Famous Recalls.)\n Globalization accounts for some of this surge. Many U.S.\n companies depend on overseas production, where quality controls\n are difficult to monitor. And it\u2019s not just hard goods\n like toys from China. Food, too, arrives by container ship from\n other countries, and sometimes it\u2019s contaminated. So far\n this year, for example, more than 8,660 cartons of cantaloupe\n from Costa Rica have been recalled for salmonella risks,\n according to U.S. Food and Drug Administration (FDA)\n records.But mainly, things go wrong. That\u2019s business.\n That\u2019s life.\u201cOne risk every company faces is a recall,\u201d says\n Jane Barrett, an analyst at AMR Research in Boston. So if\n recalls are inevitable, a CIO must help create a supply chain\n ready to cope with them, she says, by quickly providing the\n relevant data to facilitate the process. And a recall conducted\n under pressure from federal regulators, an angry public and\n plaintiff\u2019s lawyers tests every supply chain management\n decision a CIO makes.Best practice calls for companies to be able to track and\n trace the pedigree and whereabouts of the raw materials used to\n make their products all the way through the manufacturing\n process and out to end-point customers, says Steve David,\n former CIO of Procter & Gamble. But many companies, for\n many reasons, don\u2019t have supply chains that can do that,\n and that becomes evident in clumsy recalls that go on too long,\n cost too much, and have the potential to damage corporate and\n product reputations.\u201cThe ugliness of bad data management really hits you\n when you have a product recall,\u201d David says.As soon as the decision is made to recall a product,\n companies should release consistent, correct information to\n minimize brand damage, says Joe Barkai, a practice director at\n Manufacturing Insights, a consulting firm that is a sister\n company to CIO\u2019s publisher. \u201cBut,\u201d he says,\n \u201cnow [traceability] is mainly a manual procedure.\n Companies don\u2019t have it automated.\u201dAnd that\u2019s a problem. ConAgra, for example, had to\n revise its recall twice as it learned more about how much\n infected product it could have manufactured and where it might\n have gone, according to FDA records. The original\n Valentine\u2019s Day 2007 announcement recalled peanut butter\n made after May 2006. In early March, ConAgra expanded the scope\n to December 2005 and added toppings made in its Humboldt,\n Tenn., plant using peanut butter from its Sylvester, Ga.,\n plant, where the original contamination had occurred. A week\n later, ConAgra pushed the date back to October 2004\u201422\n months before the first reported illness.ConAgra declined to comment on the recall. Talking about how\n its IT and supply chain managers perform recalls, a spokeswoman\n says, \u201cdoesn\u2019t align with our priorities.\u201d\n But it\u2019s clear the experience revealed problems at\n ConAgra. Paul Hall, vice president of global food safety, gave\n a talk at the Food Marketing Institute in April, after the\n recall was under way. The peanut butter recall taught ConAgra lessons other food manufacturers can use, he noted, including knowing where all of your product is going, such as toppings, and assessing ahead of time overall recall and traceability processes across your supply chain.Hall himself is at ConAgra as a result of the recall. The\n company created a global food safety group after the recall and hired him\n to lead it.Lead Paint and Other Horror StoriesWhen your boss sweats through hostile questions from Wall\n Street or is sworn in to testify before Congress, as were both\n ConAgra Senior Vice President of Operations David Colo and\n Mattel CEO Bob Eckert, he doesn\u2019t tap on a keyboard\n looking for answers from the company\u2019s factory floor\n software. He\u2019s got a piece of paper in his hands and a\n trusted adviser whispering in his ear. He\u2019s not working\n with data; he needs information. IT leaders have to provide it,\n turning numbers into ideas.For example, in his testimony, Eckert outlined the history\n of Mattel\u2019s early August recall of those Chinese-made\n toys containing lead paint.When product samples failed lead tests performed for a\n Mattel trading partner in France, Eckert said, Mattel product\n integrity employees in China then inspected the manufacturing\n records the company requires its Chinese contractors to keep,\n showing data such as pigment tests that can be matched to\n specific containers of paint in use on the factory floor. The\n containers get stickers with batch number, test number and\n other information. That data must be kept available for\n periodic audits.By assessing data the factory handed over, as well as data\n it couldn\u2019t\u2014such as authorization to use paint from\n a source not approved by Mattel\u2014Mattel tracked the lead\n to yellow pigment in paint used on some parts of certain Dora\n the Explorer and Sesame Street toddler toys.\u201cBased on what I saw within the first week of this\n hitting them, they had a pretty solid contingency plan in\n place,\u201d says John Quelch, a business administration\n professor at Harvard Business School. (Mattel declined a\n request for an interview.)CIOs who want to mitigate risk during a recall must move\n information to where it needs to be: to the FDA or U.S.\n Department of Agriculture or Consumer Product Safety\n Commission, for example. Or to factory employees and to the\n public, says Rick Blasgen, former senior vice president of integrated logistics at ConAgra.Blasgen, who left ConAgra in 2005 to become president and\n CEO of the Council of Supply Chain Management Professionals,\n declined to talk specifically about ConAgra. But during his\n three years there and his five years leading Nabisco\u2019s\n supply chain operations before that, he was part of several\n recalls. \u201cParticularly if there\u2019s product that can\n hurt someone, you stop what you\u2019re doing and focus on\n that,\u201d he says. \u201cYou want to be as accurate as\n possible.\u201dWho Wants Yesterday\u2019s Data? (You Do)Accuracy, unfortunately, is not an absolute. Manufacturing\n systems commonly fail to feed production-line information into\n corporate data warehouses, where it could be kept to trace bad\n products or bad ingredients down the road, says Fernando\n Gonzalez, CIO of Byer California, a $300 million clothing\n company. Gonzalez has managed IT at aerospace, medical device\n and chemical companies before coming to Byer, and has been\n involved in several recalls, including one of railroad car room\n deodorizers contaminated by bacteria.\u201cIf you can\u2019t tell it was this production line\n on this shift on these days,\u201d Gonzalez says, \u201cthen\n you\u2019re stuck with an estimate and doing an expansive\n recall just to be safe.\u201dManufacturing execution systems, such as packages from Atos\n Origin or Manugistics, keep that kind of data. SmartOps offers\n a supply chain dashboard that managers inside and outside the\n factory can use to watch manufacturing activity and be alerted\n to problems, such as failures in quality control. (For more on manufacturing execution systems, see "Software for Hard Times.")On the deodorizer recall, Gonzalez\u2019s company could\n trace the offending ingredient back to a supplier in the U.K.\n But that supplier didn\u2019t keep records to show whether the\n raw materials had come in contact with other ingredients.\n \u201cAll we could do was assume that everything from the U.K.\n supplier was contaminated,\u201d he says, adding that the\n recall cost \u201cseveral million dollars.\u201dSometimes data isn\u2019t stored long enough or outside\n companies hired to manage your warehouses or ship your products\n don\u2019t retain their data as long as you do. Barkai,\n at Manufacturing Insights, remembers wanting to study one year\n of supply chain activities at a U.S. auto maker. \u201cI was\n told I would not be able to receive [the data] because they had\n limited server space, so they purged it,\u201d he says.\n \u201cSomeone decided six months was good enough.\u201dWhen deciding how long to hold data, factor in the\n product\u2019s life span, as well as any local health\n department requirements, federal antiterrorism legislation and\n the potential for litigation if something goes wrong,\n recommends a data security manager at ConAgra, who asked not to\n be named. He said data collection during a recall is \u201can\n arduous task\u201d at the company, but \u201cnot a\n scramble.\u201d (For more tips, check out "5 Recall Best Practices.")How to Manage a RecallAs Blasgen explains, the central goal in a recall is to\n figure out how much of the bad product was manufactured and\n where it went.To do that, he recommends starting by querying your\n manufacturing execution system, which keeps data on production\n lines, product batches, ingredients and conditions inside the\n factory when products were made. ConAgra, according to its\n website and other corporate communications, uses a mix of\n Manugistics and SAP software in its factories to schedule and\n monitor manufacturing runs.Next, Blasgen says, IT managers should pull shipping records\n from their transportation management systems. (Kewill Ship and\n HighJump Software are typical systems.) Search for date, lot or\n batch codes to figure out which distribution centers or\n customers were sent the suspected bad product, Blasgen says.\n Then match that information against data in the distribution\n centers\u2019 and customers\u2019 receiving systems.Those are the steps Blasgen recommends. But most supply\n chains are not configured to allow a CIO to follow\n them. \u201cDo you see what\u2019s happening?\u201d Blasgen\n asks. \u201cMultiple systems with their own ways of querying a\n nd no magic \u2018recall button\u2019 to press for any of\n this.\u201dFixing a Hole Where the Rain Leaked InConAgra had streamlined its supply chain with its $300\n million \u201cProject Nucleus\u201d started in 2003 and\n mostly led by Blasgen. The company closed factories,\n consolidated distribution centers and standardized on SAP\n enterprise resource planning software. Blasgen says that any\n company that reworks its supply chain to be more efficient will\n make and move products faster. That\u2019s good when the\n products are good and bad when they\u2019re not.Steve David, the former Procter & Gamble CIO, notes the\n irony. \u201cYou take the bad with the good,\u201d he\n says.In recent financial statements filed with the Securities and\n Exchange Commission, ConAgra claims savings of $275 million so\n far from its supply chain revamp. That pot has helped offset\n the $78 million ConAgra reportedly has spent to date on the\n recall\u2014costs that include customer notifications,\n installing and staffing a toll-free hotline, consumer refunds,\n and collecting and disposing of bad peanut butter.Another $15 million to $20 million went to overhauling the\n Sylvester factory. ConAgra bought a new peanut roaster,\n upgraded finished-product testing procedures, built new factory\n walls and designed other ways to keep peanut dust and raw\n peanuts away from already roasted peanuts (heating to 165\n degrees kills salmonella). It also put up a new roof and\n installed a new sprinkler system. Why?Because the company has a theory as to how salmonella got\n into Peter Pan in the first place. For an undetermined period\n of time, broken sprinkler heads dripped inside the Sylvester\n plant while rainwater seeped through the factory\u2019s leaky\n roof.The moisture from those sources may have awakened dormant\n salmonella bacteria in peanut dust or in stocks of raw peanuts.\n That what ConAgra\u2019s Colo told Congress in April.\n ConAgra\u2019s finished-product tests missed the\n contamination, Colo testified. The company then trucked jars to\n distribution centers and from there to Wal-Mart, Dollar\n General, Kroger and other supermarkets, warehouse clubs, food\n distributors and restaurants across the U.S.At last count, the Centers for Disease Control and\n Prevention reports that 628 people in 47 states got sick.\u201cWhen we asked people who were sick, \u2018Can you\n take out your jar and tell us the brand and lot code?\u2019\n says Anandi Sheth, a CDC doctor who investigated the outbreak,\n \u201cwe repeatedly saw Peter Pan and Great Value and lot code\n 2111.\u201dThat\u2019s the data string ConAgra\u2019s uses to\n identify products made in Sylvester. For example, one bad batch\n was coded \u201c21115055 00 1037A.\u201d After the 2111, the\n 5 indicated 2005, the year made; 055 was Julian date, for the\n 55th day of the year; 1037 was military time; and A showed\n which production line within the factory.How Recalls (and Peanut Butter) StickWhile Peter Pan returned to stores in time for\n back-to-school shopping in late August, and a new \u201cMiss\n Georgia Peanut Festival\u201d was crowned on schedule last\n month at the annual event ConAgra sponsors, the company\n predicts sales won\u2019t return to pre-salmonella heights\n anytime soon.ConAgra CFO Andr\u00e9 Hawaux told financial analysts in\n June that peanut butter \u201cwill be a lower profit\n contributor\u201d next year, even compared with this troubled\n year\u2019s figure. That\u2019s \u201cdue to relaunch\n investments and lower [sales] volume planned,\u201d he\n said.Lawsuits abound. Parents are suing on behalf of their\n children; a prisoner in upstate New York serving 12 years for\n manslaughter got sick and he\u2019s suing, too. In another\n suit, a man says his wife, after eating Peter Pan, had to have\n her gallbladder removed, which meant he subsequently\n \u201csuffered the loss of spousal and other services commonly\n provided by his wife.\u201dSixty-seven cases, including that of Hein, the Iowa mom,\n were consolidated in Julyin U.S. District Court in Atlanta,\n accusing the company of, among other charges, negligence and\n liability for product defects. ConAgra has denied all\n charges.Hein and her children have recovered, but they no longer eat\n Peter Pan.