By repurposing the Amazon IoT Button, Flight Centre Travel Group A/NZ CIO Chris Locke was able to fix long running performance issues affecting stores.
The leisure travel sales businesses unit counts on more than 1000 stores across Australia and employs more than 5000 people. With a complex network and application landscape, performance problems are no strangers to Locke and his team.
Locke says the help desk team handles regular reports of performance related issues, but when an intermittent issue occurs, it’s difficult to associate a root cause to it.
“These types of issues can end up taking long periods of time to resolve and cause significant frustration to both the end user and our technology support team,” Locke explains. “The issues never resurface when the support team are shadowing the user and the user can never accurately tell you when the performance was degraded.”
Locke had two challenges to handle: a way to track precisely when the performance impacts were being felt and to find a solution that would result in zero friction to travel consultants or “they would not adopt it”.
He says that he’d seen some interesting opportunities for the business after participating in some Amazon Web Services (AWS) executive briefing sessions. And the Amazon IoT Button was one of those solutions and was now available for enterprise users.
“Imagine if we could give the user an easy way to trigger a message that they were having performance issues right now with the click of a button,” Locke says.
“The team got to work learning more about the opportunity the IoT Button gave us. We managed to get it working fairly quickly and then the clever stuff actually started to happen.
“The team integrated the button first with Splunk to pick up real-time data analysis and insights and then into ServiceNow so we could automatically raise a service ticket, attaching the captured telemetry, and assign it to the appropriate team to investigate.”
Locke says that what followed was that the team was able to know exactly, down to the second, when the issues occurred. It was also provided with data to start diagnosing it.
In a matter of weeks the solution was ready to be trialledand named the button, Flight Recorder.
A store in Queensland that was experiencing long running performance issues and had not been able to effectively diagnose its origin was where Flight Recorder was first tested.
“The next day, during a period of degraded performance, one of the consultants in store triggered the button and our team picked up the information and started to diagnose the problem with more detail than they had ever had in the past,” Locke says. “Within 24 hours the problem had been properly diagnosed and resolved.
“The problem had to do with tiny memory spikes that had been occurring on the team’s server. All of our previous attempts to diagnose had been done against data sets that were too coarse grained. Knowing the exact time the issue occurred allowed the team to drill right in on the data and discover the true root cause of the problem.”
The solution is being deployed across other stores.
Locke says one thing that has helped him in the past few years was borrowing facets from both lean and agile methods – all of his communications with senior business leaders are short and concise addressing specific topics.
He works on addressing leadership concerns fast following an “iterative method”.
“That way I can ensure tight business alignment with IT delivery,” he says.
Leading a gender and cultural diverse team
Locke started with Flight Centre 20 years ago as a developer. He has grown through a number of roles, also having worked for other organisations during the years, and he became the CIO in 2016.
He says he uses his own personal journey both inside and outside of the organisation to help inspire his team, including successes and failures.
Within the IT team there’s a lot of flexibility for returning parents after maternity or paternity leave.
Locke also says an internal survey revealed that 80 per cent of staff had at least one parent born overseas.
“Our staff represent every corner of the globe and we support internal transfers, allowing people to work in Europe, North America or Asia,” he says.
Approximately 75 per cent of all Flight Centre staff are women, according to Locke, who says the organisationhas a program called Womenwise that supports women in the workplace and promotes growth into senior leadership positions
“One of our team [members], recently promoted to group CISO, is a committee member for the group,” he says.
“We organise regular leadership days within the IT group where we engage in topical discussion, but also deliver training designed to make us better leaders (e.g. conflict resolution or the power of Influence). Over the last 12 months, we’ve also started working on driving business planning and budget management down to each of our teams, further growing our leader’s skills.”
There is also a seminar series where members of the team present some of the “cool stuff” they’re working on as well as sharing content from conferences attended or to talk about newly introduced technologies. This allows them to practice presenting to a group.
“It’s also very important that we evangelise our successes outside of our organisation and promote and showcase our talent. It’s the only way we will continue to be able to engage new talent. We present our story as often as possible and participate in industry events wherever we can to promote our organisation,” Locke says.