Corworks Aids Customers in Data Compression

According to Brian Farrell, CEO of Corworks, data volume doubles roughly every eight months: If you have 20 terabytes of data today, you’ll be reaching the petabyte plateau sometime in 2005.

That’s a lot of disk space?and possibly millions of dollars in hardware costs.

Stamford, Conn.-based Corworks aims to help its customers deal with the exponential growth of data by, quite simply, making it smaller through compression. As an added benefit, users can also retrieve the shrunken bits more quickly at lower cost. With a foothold in data-intensive companies such as MasterCard, credit giant Experian Information Solutions and financing company CIT Group, Corworks has a solid chance of sticking around as the next chapters in the book of big data are written.

Digging for Data

Corworks was founded in 1996 by Chairman and CTO Andrew Dysart, with help from Principal Software Engineer Norbert Hom. After employing that age-old startup strategy of cold-calling potential customers (“That didn’t work,” says Farrell), the company decided to focus its efforts on those companies that were in the business of managing massive amounts of information. Sears, Roebuck and Co. was an early customer, as was credit bureau Equifax.

Farrell and his cohorts use the term tera-inflation to describe the overhead that relational database technology requires to support data. For example, 1 terabyte of data may actually consume as much as 5 terabytes of hardware processing space. “Relational technology alone just can’t do it, especially in large installations,” says Farrell. Andrew Clyne, vice president of information delivery product for global technology and operations at MasterCard’s offices in Purchase, N.Y., would agree. On a peak day, MasterCard handles as many as 40 million transactions. Using Corworks technology, MasterCard compressed 42 terabytes of credit card data down to just 9 terabytes.

In the past, says Clyne, transactions were spun to tape. “It was cost prohibitive to buy that much disk and manage it over time,” he says. If his staff needed to do research beyond 13 months, it went to the mainframe, spun tapes for the transactions, and processed them on the mainframe, a process that could take upward of eight weeks. “Now I can do that same effort in just a few days,” Clyne says.

Farrell likens the technology to an Iomega Zip disk, which also compresses technology, but that’s where the analogy ends. Unlike a Zip disk, Corworks can capture intelligence and store it in the data object so that companies can operate on their data?read, query, update, delete?while its compressed. “We see this as a competitive advantage?our ability to leverage information and turn it into business intelligence for our member banks,” says Clyne. The technology reduces the time and complexity involved when exchanging data with its members and gives MasterCard the capability to design new product offerings. Customers can also slice and dice the data more easily.

There are some hurdles to Corworks’ growth in the marketplace, however. Richard Winter, president of database consultancy Winter Corp. in Waltham, Mass., says the Corworks product doesn’t currently accept SQL, the standard language for data access. “That means they aren’t a candidate for the mainstream data warehousing market,” he says, though he adds that the company can still be successful as a specialty tool. He also notes that the big players in high-end data warehousing, such as IBM, NCR and Oracle, are also investing in data compression technology and won’t make it easy for startups like Corworks to hone in on their territory.

Corworks flagship product is Knowledge Server, an open-platform software product. It also offers Relationship Builder, which allows users to create customer definitions any way they choose using existing data, and Data Exchange, which allows users to move data across the Internet or an intranet at faster speeds (the data is compressed to 20 percent of its original size). As far as Farrell’s strategy for growth, he’s focusing on a new marketing campaign to spread the word to other large data hubs such as large credit card issuers, government agencies and telecom companies. He’s confident that Corworks has no place to go but up: “We have what people need, and the need is only getting bigger.”