Tesla, the data company

Mention Tesla to the average investor and the reaction will predictably elicit one of two responses: praise from the bulls or criticism from the bears.

In other words, it’s either changing the auto industry forever, or going nowhere fast.

But let’s set aside the financials, the competition, the shenanigans perpetrated by its ever-entertaining-polymath-celebrity CEO Elon Musk to take a look at the company from a different perspective.

There is one aspect of Tesla where it is miles ahead of the competition. And that is in its use of data to build what might just be the world’s most sophisticated, cutting-edge neural network anywhere.

When big data was the next big thing

Silicon Valley loves buzz words that metaphorically underscore the “next big thing.”

Data, for instance, was “the new oil.” It was just waiting there, an unrefined asset, ready to be tapped, refined and harnessed to drive competitive advantage.

But the hype and hoopla over big data has been eclipsed by the nitty gritty reality of the technical challenges in actually converting that structured, unstructured and semi-structured stuff into something truly valuable.

In fact, we have already sunk into Gartner’s proverbial “trough of disillusionment” around big data. Hadoop never became the big unified data platform in the sky that seemed so promising 10 years ago.

Meanwhile, artificial intelligence and machine learning are gaining more prominence. But other than the big social media platforms (who are eager to optimize their algorithms to sell you more stuff you didn’t know you needed), who is doing anything of significance here?

Tesla’s use of data, AI and ML to build a neural network — a system of sensors, data, communications, CPUs, peripheral hardware, and software that collectively processes information and adapts and learns like a human — is where the company really shines.

The race is on

The total available market for autonomous transportation is in the trillions, according to analysts.

That’s why Tesla, Google’s Waymo, Uber and all the big traditional auto manufacturers are scrambling to figure this new model out.

Autonomous driving has, in fact, been infiltrating our human-driving habits for years. Cruise control, ABS braking, lane change guidance, even air bags, can all be considered steps in the path to achieve the Holy Grail, in which humans aren’t involved at all. (Tesla’s Autopilot mode is arguably the most sophisticated driver-assistance system to date.)

Getting to fully autonomous operation, though, is hard work. How can you ensure that a computer on wheels can think, react and make smart decisions when faced with the unpredictable world of insanity that we call driving?

It will take millions of hours of coding, defining and refining algorithms, sophisticated 3D modeling and simulations, test tracks, even beta testing in real life situations.

At least, that would be the case if you used the traditional approach. But that is not what Tesla is doing. It is doing something quite novel.

With 600,000 cars on the road, Tesla treats each vehicle, each sensor, each “event” (i.e. human interaction with the steering wheel, brake pedals, etc.) as data points.

It is then taking that data, analyzing it and utilizing it to improve its algorithms, create new algorithms and send those improvements over the air to the vehicles.

As of November 2018, Tesla has amassed 1 billion miles of Autopilot data. For comparison, Waymo has collected about 15 million miles.

When you look at all miles driven on Tesla vehicles (whether Autopilot is engaged or not), the total number of miles logged is around 10 billion, according to Tasha Keeny, an analyst with ARK, an investment firm that specializes in disruptive technologies and markets.

That is a massive library that Tesla is able to tap into to teach its neural network new things, to adapt and improve.

But the truly important distinction between Tesla and the other guys is this:

• Tesla’s data is gleaned from real world miles in real world
• Tesla owners are not just driving a car to commute to work or run errands. They are simultaneously training the Tesla AI/ML engines as they go.

“When one vehicle learns something,” said Musk, “they all learn it.” (Fortune, “How Tesla Autopilot Learns.”)

This has got to be one of the most effective crowd-sourced AI/ML training initiatives around today.

Autonomy day

How does Tesla achieve those numbers and drive those continuous improvements into the system?

In a recent event for investors that Tesla billed as “Autonomy Day,” engineering VP Stuart Bowers* laid out the approach (which can be seen on YouTube; Bowers’ remarks come at 2:50–3:00 in the presentation).

“To start all of this we begin by trying to understand the world around us,” said Bowers. “We have eight cameras, but then we have 12 ultrasonic sensors (radar), an inertial measurement unit, GPS, and the one thing we usually forget about: the pedal and steering actions.”

Each of those sensors, noted Bowers, has “overlapping fields” that audit each other. With this approach, Tesla is able to get “an extremely precise understanding of what’s happening.”

Each new event, each new interaction between driver and machine, is logged and uploaded to its database. It’s then used to create 3D simulations, which Tesla software engineers can study to improve and refine the algorithms.

Updates, changes or modifications to the overall system can then be transmitted over the air to Tesla vehicles.

Shadow mode

And here’s the brilliant part: Before going “live” with those changes, Tesla operates the modifications in “shadow mode.” This is, without a doubt, a major improvement over simulation, or standard beta testing, since shadow mode is operating in real time, in the real world, but “thinking” in the background, to create a continuous feedback loop.

In simple terms, think of it as your teenage self, eager to get your license, sitting in the passenger seat as Dad drives and explains what he’s doing. You pay very close attention, because you’re going to be in that driver seat soon.

“When we have new algorithms we want to try out, we can put them on the fleet and see what they would have done in the real-world scenarios,” commented Bowers.

“Ultimately, we can do more and more through machine learning and then go into a controlled deployment, which for us is the early access program.”

One feature Tesla is testing right now is a predictive behavior as to what a pedestrian or cyclist in front of a vehicle might do.

“We have the ability to detect an obstacle in the roadway and a pedestrian is an obstacle,” said Bowers. “We can actually look at bicycles and people and our next-generation automatic emergency breaking system will not just stop for people in your path, it will also stop for people that are going to be in your path.”

This new feature is operating in shadow mode right now, noted Bowers. It will then go out to the entire fleet of Tesla vehicles but will be further beta-tested by a hard-core group of drivers who have signed up to participate.

Another example is lane changing on a freeway. Tesla now has amassed 9 million successful completions using Autopilot without one accident. “We are doing about 100,000 of these a day,” said Bowers.

And as of April 2019, it has logged 70 million miles in which Autopilot combined with its GPS navigation have taken drivers and passengers door to door.

The ultimate endgame, according to Bowers, is “to look across all the neural networks and all the cars and bring all that information together and ultimately output one source of truth for the world around us.”

Mobility as a Service

The rush to autonomous driving is not to sell you a robot car. It’s to rent you a robot taxi service.

This is the emerging field of Mobility as a Service. In a recent interview with me, ARK’s Tasha Keeny argued that MaaS is already under way in a sense, if you consider our increased use of Uber and Lyft, along with traditional taxi services.

But it is still more expensive to “rent” than buy, according to her numbers. Today, car ownership runs about 70 cents a mile, cheaper than using a service. But once human labor is out of the equation, the price drops precipitously to 22 cents a mile for MaaS, far cheaper than owning.

And rest assured that Milennials — the sharing generation — will be first in line for this service.

This tipping point — when we no longer own cars but tap an app on the phone to summon a driverless vehicle — is going to create a market worth $5 trillion (that’s trillion with a T), says Keeny.

This is the moon shot for all the companies involved in autonomous driving. This is why Uber’s investors are willing to accept $6 billion in losses this year alone. This is why Waymo has partnered with Avis and Autonation.

Tesla plans to get into the MaaS game as well. New Tesla Model 3s, for example can be leased, but you can’t buy the car at the end of the lease. Tesla is preparing to use them for a semiautonomous type of MaaS as it transitions to fully autonomous MaaS in the near future.

The stakes are high. The trophy is big. And a world of mobility as a service is coming at us fast. The end result will be game changing.

At least from a technology perspective, Tesla is a full lap ahead of the competition.

The big auto manufacturers are doing what they do best: designing and building vehicles at scale to drive down cost and remain competitive in selling vehicles.

But as is usually the case with disruptive companies, Tesla doesn’t abide by those rules. Tesla is building its own chips, its own hardware, software (from the kernel up), and most importantly, perhaps, the neural network, all in an effort to offer its flavor of MaaS. The production of physical vehicles is just one piece of the equation.

Tesla’s game changer in this holistic systems approach is, without a doubt, data.

*Bowers has since left Tesla.