That’s a lot of disk space?and possibly millions of dollars in hardware costs.

Stamford, Conn.-based Corworks aims to help its customers deal with the exponential growth of data by, quite simply, making it smaller through compression. As an added benefit, users can also retrieve the shrunken bits more quickly at lower cost. With a foothold in data-intensive companies such as MasterCard, credit giant Experian Information Solutions and financing company CIT Group, Corworks has a solid chance of sticking around as the next chapters in the book of big data are written.

Digging for Data

Corworks was founded in 1996 by Chairman and CTO Andrew Dysart, with help from Principal Software Engineer Norbert Hom. After employing that age-old startup strategy of cold-calling potential customers ("That didn’t work," says Farrell), the company decided to focus its efforts on those companies that were in the business of managing massive amounts of information. Sears, Roebuck and Co. was an early customer, as was credit bureau Equifax.

Farrell and his cohorts use the term tera-inflation to describe the overhead that relational database technology requires to support data. For example, 1 terabyte of data may actually consume as much as 5 terabytes of hardware processing space. "Relational technology alone just can’t do it, especially in large installations," says Farrell. Andrew Clyne, vice president of information delivery product for global technology and operations at MasterCard’s offices in Purchase, N.Y., would agree. On a peak day, MasterCard handles as many as 40 million transactions. Using Corworks technology, MasterCard compressed 42 terabytes of credit card data down to just 9 terabytes.

In the past, says Clyne, transactions were spun to tape. "It was cost prohibitive to buy that much disk and manage it over time," he says. If his staff needed to do research beyond 13 months, it went to the mainframe, spun tapes for the transactions, and processed them on the mainframe, a process that could take upward of eight weeks. "Now I can do that same effort in just a few days," Clyne says.

Farrell likens the technology to an Iomega Zip disk, which also compresses technology, but that’s where the analogy ends. Unlike a Zip disk, Corworks can capture intelligence and store it in the data object so that companies can operate on their data?read, query, update, delete?while its compressed. "We see this as a competitive advantage?our ability to leverage information and turn it into business intelligence for our member banks," says Clyne. The technology reduces the time and complexity involved when exchanging data with its members and gives MasterCard the capability to design new product offerings. Customers can also slice and dice the data more easily.