At SecureWorks, a Dell Technologies company, we built a lab over a 15-year period that contained more than 500 physical and virtual servers. Our software development teams use the lab daily to design, develop, test and deploy critical applications to protect our customers.
Due to company growth and expansion and with minimal interruption to the teams that use the lab daily, we needed to relocate the lab across the country to our new Software Defined Data Center (SDDC). We were given 90 days to complete the move. Here we share what we did to meet the deadline, to work with other teams and to garner support from management.
Executive sponsors: Before beginning the project, we outlined our goals and the steps needed to meet the deadline. We created realistic and achievable SMART (Specific, Measurable, Achievable, Results-focused, and Time-bound) goals to gauge the success of the migration. As we worked, we documented the steps we took and any difficulties we encountered. We communicated weekly with sponsors, which garnered their confidence in the teams and helped us gain their support in supplying us with whatever we needed to resolve issues.
Expectation management: To manage expectations, weekly we reported on our progress, successes and challenges. We made it clear to executives and to IT and engineering teams, directly or indirectly involved in the migration, what we needed to do to meet our goals. We informed the teams and stakeholders why the move was necessary, keeping everyone invested in completing the project on time.
We set expectations by informing all parties involved in the migration that some of the technologies being used before the move might not work with newer technologies in the new lab. We explained that the SDDC environment would change over time and that some features and capabilities would need to be enhanced based on feedback from the teams.
Some older technologies would not be supported in the new lab, and others would need significant changes before they would work in the SDDC. We kept detailed notes of the successes and failures of the migration to share with other team members who might have suggestions on how to accomplish our goals.
Each team had different technologies it was responsible for migrating. One of the important lessons we learned during the migration was that the teams needed clearly documented instructions on how best to migrate their software and servers. Once we created and communicated those written instructions, the migration went much quicker.
We also created a document that listed specific terminology to be used during the migration since different teams (especially IT and engineering teams) sometimes use different terms when referring to specific technical capabilities. To keep everyone on the same page, we created a lexicon that listed terminology we would all use to refer to common technologies and processes. This created one culture for all the teams and helped process changes run smoothly.
We created an “agile scrum” team, consisting of a representative from each stakeholder group, which met daily. Those representatives met with their teams multiple times daily, informing them of the current status, open risks and next steps, which included the name of the person who owned them and the deadline for completion. We also created an online forum where migration team members could chat in real time, collaborate, get updates, and ask and answer questions. Because teams work around the clock in multiple time zones around the world, any time of day a team member could go onto the forum, ask a question and would often get a quick response, limiting any delays. The combination of the scrum meetings and real-time chat helped speed the project and united IT and engineering teams, creating solid relationships in a true DevOps spirit.
Advice for Moving an SDDC
Create a plan that documents the actions needed to meet your SMART goals. Include those steps that may not directly relate to SDDC migration but may affect other teams or technologies that are dependent upon other components running in the SDDC. Prioritize tasks for the migration project so the responsible teams will know which tasks they should focus on and how to define their top goals.
Below are some of the unexpected issues that could prolong the project:
- Older operating systems and hardware may not be supported in the new environment.
- Network architecture changes may require changes to firewalls and routers.
- New machine naming conventions may be required, creating the need to update configurations to reflect the current names.
- The new data center may have constraints, such as power, cooling, rack-space, storage and network bandwidth, which may affect the migration.
Remind the migration teams that migrating systems takes a coordination effort similar to that of an original installation. So you can prepare accordingly, discuss current challenges with existing technologies that might be exacerbated by the move. Communicate with the teams running and depending on software in the labs, document the responsibilities and deadlines for each of the teams, and make sure they validate their collaboration tools.
Procure pre-assigned resources from all teams, including those indirectly involved in the migration, such as networking teams, database administrators and security architects. Sometimes migration tasks will be on hold until you can obtain assistance from teams not directly working with the project but indirectly affecting it. Let those teams know in advance that you may be giving them short notice that you need their help.
Analyze servers and applications in advance of moving them to see which ones are not being used and which ones are being used so rarely they could be powered down. Although some people on the team feared those older servers and applications might be needed in the future, we were able to come to agreement and eliminated approximately 10 percent of the systems, reducing the scope of work and costs.
Preparing a Server for Migration
Below are some items that may be time consuming and may make server migration to SDDC more challenging:
- Systems that do not have automated and thorough availability monitoring will be harder to migrate because you may not know if they are working after they have been moved to the new lab. The users of the lab will have to let you know whether the systems are working.
- Systems that do not have automated and validated install procedures will be the most problematic to move. Older systems may cause difficulty, as the original team members who installed them either may no longer remember how to install them or may no longer work at the company.
- Servers/services that other servers/systems depend on will be among the most challenging. If they aren’t accessible after migration or if they don’t work, neither will those systems that depend on them to operate. Map out the dependencies and plan accordingly.
Celebrate successes with the entire team along the way, and celebrate again after the project has been completed. Encourage executive sponsors to reach out to those who went beyond the call of duty in making the project a success.
We migrated a lab of more than 500 servers and completed the project one day ahead of schedule. We closed our old office and moved our technologies across the country with minor impacts to the teams that depended on the lab to deliver features to our customers. Our software development teams experienced minimal downtime.