How Facebook uses IT automation to manage 16,000 MacBooks

(Editor’s note: This story has been modified to correct an error related to Mike Dodge’s title at Facebook.)

When Mike Dodge joined Facebook about five years ago he was an IT team of one, managing 1,800 MacBooks. The company’s employee base has grown rapidly ever since, and the tools Dodge, now Facebook’s client platform engineer, uses to manage its massive fleet of MacBooks have also changed considerably.

Facebook now has more than 16,000 MacBooks deployed across its workforce, and Dodge says key changes he made during the past years help him deploy an ever-increasing number of Apple devices, and he no longer has to deal with the headaches he faced just a few years ago.

[Related News: How and why Facebook excels at data center efficiency]

“Scale can be scary, especially if you don’t have all the right tools and processes in place,” Dodge said on Tuesday, at the MacIT Conference in Santa Clara, Calif. “When you take the time and bandwidth to get it right early, your fleet can double overnight and you won’t even feel it.” 

Dodge encouraged his IT counterparts to automate certain Mac managements tasks now so they don’t weigh the tech team down later, and he also suggested that they start writing their own associated code and stop relying heavily on out-of-the-box scripts. Then he walked through a brief history of Facebook’s IT automation goals and challenges to illustrate his points and explain what he and his (now much larger) team have done to overcome past problems.

Facebook’s constant growth keeps IT on its toes

When Facebook originally transitioned to a more automated approach to manage its fleetof Apple MacBooks, Dodge was using Bash, a UNIX command shell and language included in Apple’s OS X, and JAMF‘s Casper Suite, which lets IT pros manage, deploy and secure any Apple product. Facebook quickly became limited by the Bash code, and Dodge started searching for alternatives. “I was making Bash do things it was probably never intended to do,” he says. 

[Related News Analysis: Why IT should be skeptical of Facebook at Work]

Soon thereafter, Dodge and his team ditched Bash for a combination of Python and Casper, and quickly narrowed down hundreds of lines of code to just a few lines in Python. That system worked relatively well for Facebook for the next couple of years, especially after it added custom code to supplement inadequacies, according to Dodge. However, when his team grew to five full-time administrators and three interns, things started to get “really messy” again due to a lack of source control.

“Sometimes it was more risk than we were willing to take on,” he says. So he went back to the drawing board, searching for open source tools that could provide Facebook with a level of flexibility it hadn’t enjoyed previously.

Facebook embraces open-source for IT automation 

In early 2014, Facebook decided to make the switch to Chef, a set of IT automation tools that turn infrastructure into code, and Munki, another set of utilities that OS X administrators can use to manage software installs on client machines. Before that could happen, though, Dodge and his team had to completely rewrite all of their code during a two-month sprint.

[Related Feature: IT execs on Facebook at Work strengths and weaknesses]

Munki and Chef now power all of Facebook’s IT automation, and Dodge says he’s confident the pair of open-source tools will meet the company’s needs for years to come. There’s a learning curve involved, especially with Chef, but in his standing-room-only session at Tuesday’s conference he insisted that attendees’ efforts to learn more about the tools would pay off in spades. They did for Facebook, at least.