Sweetwater Sound retunes in-house tech for expansion

Business expansion plans often pose IT leaders a thorny question: Can the software at the heart of the enterprise support our business ambitions?

For Sweetwater Sound, an online retailer of musical instruments and audio equipment, this crossroads came in the wake of an October 2018 decision to build a new distribution center — nearly four times the size of its existing facility — next to its Fort Wayne, Indiana, headquarters.

In-house software development had been a way of life at Sweetwater since the beginning: Founder and CEO Chuck Surack built the company’s CRM, ordering and inventory management system himself.

“Until the late nineties our CRM server used to sit underneath his desk,” says Jason Johnson, senior vice president of IT.

One consequence of that do-it-yourself approach, Johnson says, was that Sweetwater’s distribution logic was tightly coupled to its CRM platform — logic that, until 2006, was essentially a print job: Staff would grab the printout and run down to the warehouse to fulfill the order, checking it off on the computer once it shipped.

“That works really well when your distribution center is connected to your building, when you have a low latency,” Johnson says.

Over the years, the system gained in sophistication — but sales and shipping stayed physically close, and the software remained a monolith. And while its planned new distribution center would be adjacent to the headquarters, the next one might be much farther afield, perhaps even on the West Coast, meaning that the one CRM system would have to communicate with many distribution centers.

“We really wanted a system that would stand on its own, that would decouple the execution of the warehouse operations from our CRM platform and let both of those platforms really grow independently,” says Johnson.

That challenged the foundation of how the existing software was built, prompting Johnson to say, “Let’s scrap everything we have and let’s start something new.”

Johnson initially considered adopting one of the many commercial software packages out there, but he learned from peers that tweaking out-of-the-box software will never get it just right. Moreover, with the construction project scheduled to take around 18 months, there was another problem with implementing commercial software: time.

“You sit down with this off-the-shelf company and you think, ‘Oh, we should be able to implement quickly,’ and they’re like, ‘We don’t have any customers that implement that at faster than two and a half years,’” he says.

And so, with the original programmer’s blessing, Johnson and his team set out to re-create the company’s CRM and warehouse management system in-house.

“We turned a big physical project into a giant IT project where, in many ways, we became the focal point,” says Johnson, acknowledging that many of his peers would see this is a huge mistake.

Keeping it in the family

Sweetwater’s IT team spent around six months on planning and defining the processes, and six months on core development — a tight schedule they were able to keep in part because of the things they could leave out. “I don’t have to support every language under the sun,” he says, as an example: Most of Sweetwater’s warehouse staff speak English, and most of the others speak Burmese.

There was also no need to build a massive settings interface, as the software would already be tailored to Sweetwater’s needs. “I can roll out with just a database where we keep the settings, because I know I’ll have people around to change them if they need to,” he says.

The corporate culture also helped. In any company, IT workers benefit by understanding the jobs of those they’re supporting. At Sweetwater, says Johnson, newly hired IT staff begin by doing those jobs.

“I have people in IT that are some of the best packers, some of the best pickers, that know how to operate a forklift,” he says.

In addition, Johnson’s fifty-person IT group is split into teams of four to six — “We keep them incredibly lean to reduce red tape,” he says — and one of those had spent several years working with the distribution center led by one of Sweetwater’s “delivery managers.”

“Other companies might call them a business analyst. We call them a delivery manager because we want to make sure that they’re focused on delivering things and not just analyzing things,” Johnson says.

That intimate knowledge of the warehouse floor helped the team identify how things should work and where the hiccups were, to come up with a set of functional specifications for the new system.

Then came time to develop the software. In the fall of 2019, Sweetwater hosted a talk by Robert “Uncle Bob” Martin, one of the co-authors of the “Manifesto for Agile Software Development” — but Johnson and his team aren’t agile purists in their methodology.

“We call it swagile, Sweetwater Agile,” he says, meaning that they will focus on being nimble, on short iteration cycles and prioritization, without worrying about whether they have a Scrum master. “We try to chunk things down into two-week periods we call build cycles, and focus on what’s going to be the highest impact.”

At the peak of the project developers were deploying to production six or seven times a day. “We would have a developer sitting out next to somebody packing boxes, writing code as they were packing boxes and then going back and testing it.”

Microservices management

That rhythm was possible because of the microservices architecture the team adopted to decouple the various pieces, making for tightly defined areas of responsibility around each system. “That makes it easy to make a change: All you have to worry about is your external interfaces and whether they changed and whether other systems will see that change,” says Johnson. “That was definitely a big takeaway for us, that that works super well.”

It was not without its difficulties, though: This being the first time Sweetwater had tried such an architecture, there would sometimes be confusion about whose responsibility it was to fix a problem — or on which side of an interface it should be fixed. “Getting all those teams onto the same page and making that a positive, healthy conversation I think was by far the most limiting factor we had in the first couple of weeks in getting changes out,” he says.

While it proved possible for the team to learn to build microservices, “One of the things that is really hard to change about people is their ability to communicate,” says Johnson. That’s why it’s one of the key skills he looks for in new hires — alongside the humility to admit when they don’t understand something or need help. “If we have those two things then we’ll teach you whatever you need to know.”

The distribution center went live in February, a few weeks before the COVID-19 lockdown caused a spike in demand for instruments from would-be musicians seeking a creative outlet during their enforced time at home.

The software was ready too — but not finished.

“We’re nowhere near maintenance mode on the project,” says Johnson. “I don’t know that that project will ever really hit maturity: I have a team of three or four people out there long-term, constantly iterating on the software, trying to make it better and better and better.”