Blue Cross NC brings IT back in house to improve agility

When Petar Bojovic joined Blue Cross and Blue Shield of North Carolina three years ago, his task was to build an engineering department — and the IT infrastructure it would manage — from the ground up.

The organization had no engineering team because it had outsourced most of its IT operations: Pretty much anything that had an IP address was run by a managed service provider (MSP), he says.

“I had a unique opportunity,” says Bojovic, now director of systems engineering.

Years earlier, Blue Cross NC handed over its two data centers and many of the staff that ran them to Fujitsu, following a fashion at the time for businesses to focus on core competences and to cut costs by outsourcing what many saw as mundane tasks.

As a result, Blue Cross NC IT became little more than a vendor manager tracking KPIs: “If there was an application outage, if there were application issues, performance issues, it was not handled by Blue Cross internally. It would have been handled by the MSP,” he says.

The world has changed, though, with IT now seen as an engine of growth capable of responding to challenges with flexible, innovative solutions. So, for Bojovic, bringing IT back in house was not something he had to sell to upper management. “It was already sold before I got there,” he says. His goals in facilitating the datacenter migration were to cut costs and to regain control.

Black box blues

One of the challenges Bojovic faced in preparing for the migration was knowing what needed migrating.

“When I joined, I would ask questions like, ‘Hey, what does our environment look like? What version of VMware are we running?’ Well, nobody knew that answer. So it was essentially a black box,” he says.

Bojovic decided to build a new infrastructure with enough headroom to run old workloads alongside new ones for years to come. He set about recruiting a team with the expertise and fresh ideas to help him. “Some had some healthcare and insurance backgrounds, which is great, but at the end of the day, it was all technology,” he says.

The new hybrid infrastructure combines cloud capabilities from Amazon Web Services with on-premises servers from Dell and Lenovo running VMware, and uses NetApp to create a single data fabric with flash storage.

The team completed the data center migration in February 2019, moving workloads more or less as it found them to expedite the process. The team, which is still growing, is now in the process of optimizing Blue Cross NC’s applications and systems.

“I call it a fun phase, where we’re doing continual cleanup,” says Bojovic, who seeks to make Blue Cross NC’s infrastructure less of a concern for its developers, “so nobody cares about where that infrastructure’s running, what services on the backend are called.”

To do that, the team is building a container-based environment on OpenShift, a Kubernetes-based platform that runs on premises and in the cloud. “Now, if you’re an internal developer or refactoring an application, it no longer matters, ‘Am I in this data center? Am I in that data center? Am I in this region, this zone?’”

He’s taking a multicloud approach to governance, allowing for access to Google Cloud Platform and Microsoft Azure in the future. “That doesn’t mean we open up any and all services that they provide. That, I think, would be foolish,” he says. The governance tools will be there “to make sure that we can consume it properly.”

Providing value-based care

With its new infrastructure, Blue Cross NC can now deliver projects at speed. Under its former MSP, it could take 90 days after submitting a request to get a new service or server approved and delivered, says Bojovic. “With us, you submit a request, you get the approvals within minutes.”

One application that has already been upgraded is Blue Connect. The application, through which Blue Cross members can check their claims, bills and services received, started its life on premises, moved to AWS, and is now being refactored into containers. This, Bojovic says, means it runs quicker and will be quicker to repair in the event of any issues.

The new infrastructure is also making it easier for Blue Cross NC to access and analyze its data — and as a health insurer, it has plenty of that.

Behind all this is a wider goal at Blue Cross NC: to deliver value-based care rather than just processing claims. “When you see your primary carer, they care more about your overall outcome versus submitting ICD 10 codes,” says Bojovic.

The new infrastructure has also proved beneficial in navigating the COVID-19 pandemic. Shelter-in-place orders quickly sent 98 percent of the organization’s workforce to work from home. The core infrastructure suddenly had to support 7,000 simultaneous VPN connections, up from between 1,000 to 1,500 before.

“With the former MSP, it would have just crippled our business. Now we have so much throughput, so much capability that we can easily spin up virtual, for lack of a better term, end points for the VPN,” he says.

Blue Cross NC members, Bojovic says, will barely have noticed the difference as call-center workers switched to working from home.

The migration to WFH has been a revelation for Bojovic, who wasn’t a fan before: “I loved being on site and I loved having my team on site.” While he wouldn’t want to work from home indefinitely, he says, “I could see myself working from home more days out of the week than being onsite.”