After geologists assess the information, it's sliced and diced against financial realities. "The amount of money we spend is very high—$100 million for a well alone," de Souza says. "We want to get it right."

To reach its goal of becoming one of the five biggest oil companies in the world by 2020, Petrobras has to take some calculated risks. Recovering oil from this find will be expensive partly because it's so far down in the earth. "No company has tried to explore under it," he says. But promising data has triggered major staffing decisions: Petrobras has created a new group of senior managers to oversee exploration of this area and plans to hire 14,000 drillers, geologists and engineers. It takes years to go from initial exploration to crude oil production and sales of finished gasoline, so companies have to model markets five, 10, 15 years out. They use a mix of their own intelligence and public data, such as from the Energy Information Administration (EIA), says researcher Knapp.

For example, automakers continue to improve the fuel efficiency of their cars and light trucks, as well as to build electric-gas hybrids. By 2030, the average light-duty vehicle will get 27.9 miles per gallon, 40 percent more than in 2006, according to the EIA. A highly simplified analysis suggests that if people use less gasoline, gas prices should drop, which makes expensive drilling less profitable, Knapp explains.

Although demand for gas is growing in China and India, so far it's not enough to offset the expected drop in U.S. demand. New well and rig technologies could take some of the cost out of drilling, but no one knows exactly when or by how much. There is no shortage of data points; the value is in interpretation. "It's about filtering rather than finding a piece of information," he says. "Understanding what this whole pile of stuff can do for you is the key."

Adjusting to Change in Real Time

Every Wednesday morning, the shouts and hand gestures that make the Nymex trading floor in New York frantic begin to calm. Petroleum traders are waiting for the release of data from the U.S. Energy Information Administration (EIA) on countries' inventories of crude oil and gasoline, as well as world crude prices.

At 10:30 a.m., the EIA's website sees a storm of activity: 1,000 page views per second for 15 seconds, says Charlie Riner, a lead analyst for the site. Oil companies, commodities traders, analyst firms, and government agencies in the United States and other countries have written bots to collect the data. Then traffic ebbs.

Inventories are the most closely watched data in the industry, says Joanne Shore, a senior petroleum analyst at the EIA, the statistics keeper for the U.S. Department of Energy. "This is what moves markets when it comes out," Shore says. If, for example, U.S. supplies fall sharply from the week before, that can mean demand is rising and prices likely will, too. It's not only traders who want this data. Corporations such as Valero fold it into its analysis of inventories and market activity so that they always know their standing compared to rivals.

In the oil and gas business, you are what you own. The amount of crude waiting to be refined, or the already-processed liquid in storage tanks ready to be sold and delivered, represents much of a company's value at a given moment. As a refiner, Valero buys barrels of oil to heat and pressurize into other products, such as diesel fuel, asphalt and lubricants. The $95 billion downstream company owns 17 refineries that together can produce 3.1 million barrels of product per day.

But Valero doesn't sell that much in a given day so it must store finished goods until they're ready to be shipped to customers. The company tracks its own inventory movements the way a first-time mother studies her infant. How much of which products did we sell this morning? How about now? And now?

Market analysts run inventory reports "a few hundred times a day," says Kirk Hewitt, vice president of accounting processing optimization . As the cost of crude fluctuates during trading hours, Valero sales and marketing staff want frequent updates so they can sell products at the most profitable price and buy crude to feed their refineries at the best price.

"We're dealing with a commodity whose price changes every second," Hewitt explains. "So our margins change every minute. Our costs change every minute."

Companywide, Valero employees generate more than 20,000 reports per month. These range from gas station profit-and-loss statements to the status of payable invoices to telecommunications charges.

Valero uses WebFocus tools from Information Builders for nearly all its BI reporting, but not for inventory reporting, which has to be quick and dirty. For that, users query Valero's SAP Business Warehouse system, which collects operations data from the SAP R/3 system at Valero's refineries. Data includes items such as the volume of crude processed and the amount of products made from it. The information is presented in an Excel spreadsheet, he says, because "it's a fast way to get a snapshot."