Machine intelligence: Build your own vs. as-a-service

Fans of HBO’s “Silicon Valley” may recall the plotline earlier this season in which Erlich Bachman secures $200,000 in VC funding for See Food, a camera app that recognizes various kinds of food and instantly surfaces useful information, such as nutritional data.

Bachman is 5% technologist and 95% charlatan, give or take, so naturally there’s a hitch: See Food doesn’t exist. The funding is the result of a misunderstanding that Bachman quickly compounded into a lie. Antics ensue as Bachman, determined to keep the money, attempts to transmute his vaporware into a working prototype.

Here’s what struck me: Many of these antics, such as Bachman’s attempt to con a class of Stanford undergrads into training a machine learning model, are predictably hilarious—but from a technical standpoint, virtually none of them is implausible.  

Indeed, whereas just a few years ago, an app like See Food would have been literally impossible to build, today it’s not a stretch that a few Bachman-like misfits could actually cobble together the pieces. “Silicon Valley” is TV fantasy but Bachman and company rely on the same resources developers would use in the real world: cloud infrastructure, neural nets, etc. That these resources have become so accessible—accessible enough to be casually mentioned in a mainstream TV show—is a testament to how much and how quickly things have changed.

Crucially, this shift is about more than camera apps; it’s about machine intelligence (MI) moving from niche applications to ubiquity. As we’ve written previously, thanks to APIs and cloud services, we’re entering the first years in which virtually any enterprise that wants to harness machine intelligence will be able to—which means that enterprises that don’t harness intelligence will risk being left behind.

The prospect of digital ecosystems that widely incorporate MI raises a critical question for CIOs: When it comes to machine intelligence, how does one assess when to build a system from the ground up versus when to invest in third-party solutions?

Rent vs. build: Machine intelligence’s core ingredients

Machine intelligence requires three ingredients: computing muscle, algorithms, and data. The degree to which a company is strong in any one area informs when that company should go proprietary and when it should fill gaps with third-party as-a-service offerings. Strength should be assessed not only in terms of technologies and budgets but also human talent and ability to execute. 

Compute

Prior to the cloud, harnessing the computing power necessary for machine learning (ML) typically required building one’s own supercomputer—a spectacularly forbidding prospect in terms of cost, time, and requisite expertise. Cloud infrastructure has changed that, rapidly diminishing the marginal cost for additional computing power, and enabling companies to rent when they previously would have been forced to build or to enter spectacularly expensive partnerships.

Consequently, fewer scenarios exist in which building one’s own ML compute infrastructure confers a competitive advantage. The expense and effort might be justified if you’re building a unique service or require specialized hardware—but even then, the benefit is debatable. If a custom system performs 5% better than as-a-service offerings but involves 10 times the cost and takes 10 times longer to develop and deploy, the system can still easily lose money in the end, despite its superior performance.

That’s not to say all cloud infrastructure is equal, of course. Top providers such as Google (our employer) and Microsoft don’t just scale up spare computing cores, for example; they build custom chips specifically for MI. This sort of specialty hardware reinforces the challenges companies face in building proprietary systems that can compete with top cloud services—but it also reinforces that companies must carefully vet cloud infrastructure providers.  

Algorithms

Proprietary algorithms can be incredibly lucrative. If your company’s code can do something better than anyone else’s—such as extracting profit signals from noisy datasets—the IP may be valuable enough to justify the cost of development.

But that cost can be prohibitively high, especially given that many, if not most, companies don’t already possess the requisite in-house expertise. Moreover, in many cases, what’s important isn’t owning algorithms—it’s quickly delivering improved experiences to the market and staying ahead of the competition. Many apps include a search function, for example, but most app makers don’t build search engines themselves—they use Google or Bing. Whatever advantage a company might gain with a proprietary search solution usually ends up negated by the lost opportunity cost the company sacrifices during development. Machine intelligence is likely to function similarly.

It’s likely that legions of future apps will include machine intelligence technologies being pioneered today, such as natural language processing, speech-to-text features, and image recognition. These capabilities are increasingly packaged as pretrained models exposed via APIs—meaning that for many companies, it may be more important to develop API management skills than to invest heavily in in-house ML algorithms.

Data

Whereas relatively few businesses organically possess the compute and algorithm ingredients, data strength is somewhat more common. There may eventually be a degree of commodification in the infrastructure and algorithms many services use, but the data those services analyze will often vary from company to company. Take the same neural net technology and feed it drastically different training sets, and its conclusions over time may well be quite different.

Indeed, companies have been pining for years for ways to activate their data—that’s why “big data” became a mainstream phrase and “data scientist” remains one of today’s hottest job titles. We’re starting to see more companies find creative ways to turn their data into revenue opportunities. Vivanda, a company that matches flavor preferences to food, for example, was spun out of spice company McCormick’s FlavorPrint technology, which leveraged the company’s expertise and data to create a “digital fingerprint” for any food or recipe. Imagine how many other data-to-product transitions are possible now that ML algorithms and compute infrastructure have become so accessible.

Machine learning will be defined by ecosystems: You don’t have to do it all yourself

In the last decade, we’ve seen innumerable examples of platforms and ecosystems that redefine the way products and services are built. Google Maps, for example, is both an aggregator of demand because it can be infinitely replicated for consumers and developers, and also a fulfiller of demand because it can be integrated into services developers build, such as ride-hailing services. The takeaway is that opportunities continue to explode for companies to leverage one another’s technology, creating end-user experiences defined by multiple parties and distributing the work of demand generation and value creation across ecosystems of participants.

This phenomenon is extending to machine learning and artificial intelligence. When you have strengths in core ML ingredients, leverage them. When you have blank spots, look for partners with adjacent needs and skills, and look to the growing as-a-service market to move quickly without spending years and millions of dollars in development. Our intelligent future is waiting to be assembled—but CIOs don’t need to do it alone.