Conductor is an amazing piece of software. It’s used by the world’s largest companies to orchestrate their initiatives, providing increased productivity and real-time visibility.
It all started at Klick Health, a healthcare-focused agency which started developing their internal task-tracking system over 15 years ago. Since then, it’s expanded from a simple internal task management tool to a fundamental building block of the business to now being used outside the organization – by some of the largest companies around the world, who use it to orchestrate their most business-critical programs!
Like any piece of software which has grown organically over time, it has legacy issues that eventually need to be addressed. While the team is continuously adding new features and improving the architecture over time, the architectural foundation is built on older principles and the many developers that have touched the software have all left their mark.
As Sensei Labs moves into its next phase of growth, we’ve decided to dedicate a team whose sole focus is to address the technical debt and modernize the architecture of the platform. Over the course of the next two years, Conductor will be transformed from a legacy monolith software application, to a modern service-oriented architecture which can scale with the growth of the company.
When I joined the team, a lot of progress had already been made on this front. The application was already partitioned into front-end and back-end applications, communicating via a well-defined web service layer. The front-end code has been refactored significantly and the CI-CD pipeline and development process is mature.
The last big remaining hurdle with the application is the back-end monolith. It’s 375k lines of legacy code with a high degree of interdependencies. Our first task is to break apart the monolith into more manageable chunks, then we can start modernizing the architecture piece by piece.
I like to compare this task to changing a tire on a moving car. Just because we’ve decided to re-architect the application, this doesn’t mean that the Product team will slow down feature requests or the Dev team will stop building them. We need to deal with legacy issues while having a minimum impact on the evolving product roadmap.
After a lot of upfront planning, here’s how we decided to approach it! As we work our way through this plan, we’re bound to discover things and evolve the plan, but this is where we’re currently at. In future blog posts we’ll expand on portions of this and share our learnings and progress.
Establish a baseline
Since Sensei Labs is a data-driven company, of course we want to start with data. In order to determine how much progress we’ve made, we need to first determine where we’re at.
We’ve made extensive use of a tool called NDepend – which analyzes your application and provides a report of issues within the application, as well as a high-level score measuring technical debt using the industry accepted SQALE rating.
Achieve base test coverage
As anyone who’s dealt with legacy software before can tell you, your best tool for mitigating risk of large-scale changes is to have robust unit test coverage. This is your only safety net to ensure that you aren’t changing the behavior of the application.
Two approaches we’re taking here – one was to work with the dev teams to set go-forward standards for unit tests and ensure that all new code is tested. Secondly, in order to backfill large portions of the application, we looked at methods to auto-generate unit tests and create a regressions suite of tests which capture the current behavior of the application.
Delete unused code
The next step is to reduce the amount of work we need to do later. Given the age of the application, there are many retired features which are still hanging around. The more code we delete now, the easier our reorganization steps will be in the future.
Once we’ve gotten rid of as much unused code as we can, we start the step of organizing our code into conceptual domains.
We start this by first cleaning up and re-organizing our namespaces to align with our conceptual domains. This allows us to see the interdependencies between domains more clearly, and to start breaking these dependencies using interfaces etc. The next step will be to break out these namespaces into different .NET projects.
As part of this re-organization, we’ll also introduce service layers. This will allow us to more easily break the dependencies between the different namespaces/ conceptual domains, as well as make our code easier to test by enabling mocking.
The final step in breaking up the monolith will be to break up the back-end database to correspond with the conceptual domains identified in the previous step.
We plan on taking a similar approach as we do with the code reorganization, where we’ll first move tables and views into different schemas, then once we have landed on a structure, we can start moving schemas into their own databases. At this juncture we can start making decisions about whether to migrate any new databases into a different datastore.
So that’s our current plan as it sits. As we start working our way through the code deletion, we’ve identified the need to create some custom tooling to identify the dependencies within our application.
But that’s a topic for another article…