Legacy system modernisation: rewrite, refactor or gradually replace the system nobody dares to touch?

Many companies that have been around for a while have a system that people talk about cautiously in meetings. It runs invoicing, inventory or production, a lot of people have modified it, few understand it in detail and every change is slow and expensive.

When the time finally comes to decide what to do with it, the proposal on the table is often: let’s rewrite the whole thing from scratch. It sounds tempting, but it is risky: the old code is an archive of business rules, exceptions and fixes accumulated over the years, which are often not written down anywhere, and anyone who throws the code away in one go throws that knowledge away with it. Yet legacy system modernisation comes in more forms than “leave it as it is” or “rewrite it”, and with a large, business-critical system it is usually safer to replace it piece by piece while it remains fully operational.

Symptoms: a system the company is afraid of

Michael Feathers, author of the book Working Effectively with Legacy Code, defines legacy code simply as code without tests. So you identify a legacy system not by its age but by whether it can be changed safely. Typical symptoms include:

• The one person who understands the system. When that person leaves, the company loses the ability to change a key process.
• Fear of deployments. Changes pile up into large batches, so when something goes wrong, the cause is harder to track down.
• Outdated technology that is hard to recruit for and that the vendor has stopped supporting.
• Security risks that cannot be eliminated without touching the core of the system.
• No way to connect new tools. The system has no API layer, in other words a defined interface for exchanging data, so every integration with an online shop or an AI tool is a project in its own right.

All of these are forms of technical debt, which the article A company is one product examines as a principle that applies to the whole company.

The public sector is no exception. In a July 2025 report, the US Government Accountability Office (GAO) identified the 11 most critical federal systems in need of modernisation. Eight of them use outdated languages such as COBOL and assembly language, and according to the agencies, seven are running with known vulnerabilities that cannot be fixed without modernisation.

Why “let’s rewrite the whole thing from scratch” is so tempting

A big bang rewrite means building a new system from scratch and switching the entire company over to it in one go. It promises a clean start and modern technology. In an essay from 2000, Joel Spolsky offers a less flattering explanation: programmers regard old code as a mess mainly because it is harder to read code than to write it. He called rewriting code from scratch the worst strategic mistake a software company can make. He cited Netscape as an example: version 6.0 went into its first public beta almost three years after version 4.0, and in the meantime the company’s market share was falling sharply.

It is an uncompromising view, but the specific mechanisms of failure can be identified:

• Hidden rules. According to Spolsky, the messy parts of old code are often fixes for bugs that only surfaced after weeks of real-world use. A rewrite throws them away, and the same bugs can reappear, once again only in production.
• Parallel development. According to authors from Thoughtworks, routine business requests continue to be handled in the old system in the meantime, so the company pays for two development streams at once.
• A long period without value. Martin Fowler points out that replacing a serious IT system takes a long time, and users cannot wait that long for new features.
• A moving target. The Thoughtworks authors describe an organisation that went through a series of modernisation programmes lasting three to five years each. In every one of them, changing business needs overtook the technology strategy and the organisation had to start again. Where a programme was run as a big bang, that meant throwing away most of the work.

Six options and when each one fits

1. Retain and stabilise. Put error and outage monitoring, backups and documentation in place, and train a second person to understand the system. This fits when the system does its job and rarely changes. It is usually the cheapest way to reduce risk.
2. Refactor. According to Martin Fowler, refactoring is a change to the internal structure of software that makes it easier to understand and cheaper to modify without changing its observable behaviour. This fits when the technology is sound but the code is convoluted.
3. Move to new infrastructure, for example from an on-premises server to the cloud. This fits when the pain lies in operations or hardware. A move will not cure flawed application logic.
4. Replace gradually. This fits large, critical systems that cannot afford downtime, whose behaviour nobody knows in full and which can keep running alongside the new ones for a long time.
5. Buy an off-the-shelf solution. For a standard process such as accounting or time and attendance, custom development is usually unnecessary, although the process has to adapt to the product. This choice is examined in the article Seven SaaS tools or one system?
6. Retire. If the system supports a process that no longer exists, you still need to check whether other systems or reports draw data from it.

Strangler fig: the new system grows alongside the old one

The name strangler fig pattern comes from Martin Fowler’s metaphor: in 2001, in the rainforests of Queensland, he saw strangler figs, which gradually grow around a host tree until they replace it. In the same way, the new system is not built in place of the old one but next to it. It takes over one function after another until the old system can be switched off.

Microsoft’s documentation describes the mechanism: a facade, which acts as an intermediary, is placed between the users and both systems and gradually shifts requests from the old system to the new one, starting, for example, with just the calculation of price quotes. Users are not aware that a migration is taking place. According to Feathers, the old system remains in place as a fallback in case anything goes wrong.

Fowler acknowledges the cost: a transitional architecture that allows both systems to run side by side and disappears once the work is complete. In his view, this is outweighed by the lower risk and earlier value, because the parts being replaced are small. He warns, however, that without a change in culture and leadership, the new system will also end up in a similar mess.

Data is the hardest part

Code can be replaced piece by piece, but during the transition both systems need the data at the same time. Data migration, meaning the transfer of data from the old structure to the new one, is not simply a matter of copying. Researchers at Trinity College Dublin point out that data from legacy systems tends to be of poor quality, has to be mapped onto the new structure and often needs cleansing as well. In practice, this means things like duplicates, fields used for a different purpose and values that only the old code understands.

Microsoft recommends separating the data gradually, one area at a time: populate the new database with an initial load; carry over subsequent changes using change data capture, that is, by capturing changes in the old database; verify that the data matches before switching over; and remove the old objects only once the new system has been validated. Until then, rolling back is possible. After that, it is considerably more difficult and risky.

For each type of data, it must be clear throughout the transition which system is the single source of truth. If a customer’s address can be changed in both systems, there is a risk that each will end up with a different one. Our practical conclusion: people should always change a given piece of data in one system only, and the other systems should simply take it from there. The broader role of data is covered in the article Your data is the advantage.

How to find out what the system actually does

Feathers points out that people carry around a mental picture of what the system does and forget to look at what it does in reality. Three sources can help:

• Observing users. Watch what people do with the system, not what they say about it. This is where workarounds, spreadsheets kept on the side and manual steps come to light.
• Logs, the records of the system’s activity. They show which features are used, how often and by whom, and help uncover unexpected dependencies.
• Tests of current behaviour. Feathers describes characterisation tests: you ask the code a question and record the answer you get as the expected result. This gives you regression tests: automated checks that verify after every change that what used to work still works.

The Thoughtworks authors describe a team that was testing, in production, a replacement for its integration middleware, the software that connects the other systems. Shortly afterwards, critical management reports stopped adding up. After a long investigation, it turned out that the middleware database was being replicated into a data warehouse that the reports drew on. The problem was only resolved by a temporary mechanism that backfilled the missing data into the warehouse.

Sequencing: greatest pain, lowest risk

Migration should not be the first step. Here too, we apply the sequence Remove, Simplify, Automate, Augment: remove first, then simplify, and only then automate and augment. The Thoughtworks authors point out that most legacy systems become bloated over time with features nobody uses, citing a 2014 Standish Group report that puts the figure at 50% of features. For any particular system, this is only a rough indication. The logs will show the real picture. A feature that has been removed does not need to be analysed, migrated or tested, and workarounds for the old system’s limitations are worth simplifying during the transfer rather than copying.

The first part to be replaced should cause noticeable pain, for example by holding back sales, while also being relatively decoupled from the core. Feathers advises focusing on the parts that are going to grow and are also causing difficulties, and, to redirect requests, finding what he calls seams: places where one behaviour can be replaced with another.

Risks and how to measure them during the transition

Gradual replacement does not eliminate risk. It breaks it down into smaller, measurable pieces. The Final Notice issued by the UK’s Financial Conduct Authority (FCA) on 20 December 2022 illustrates the opposite. Over the weekend of 20 to 22 April 2018, TSB moved most of its operations and customer data to a new, as yet unproven platform. Serious problems with internet and mobile banking and with card payments followed, and by 7 April 2019 the bank had received 225,492 complaints. The FCA identified failings in planning, testing and risk management, as well as in the oversight of a third-party supplier. It pointed out that after go-live the bank was in practice unable to return to its original platform, and it imposed a fine of £29.75 million.

Vicent Martí of GitHub described a safer route in 2015. The team was replacing one of the most critical parts of the codebase directly in production: the old and new code ran in parallel, the results were compared and users always received the result from the old code. The experiment started on 1% of requests and was gradually expanded until it had run at 100% for 24 hours without a single mismatch. The comparison also uncovered 2 serious bugs in the original implementation that nobody had noticed for years.

What to track, then:

• Result parity between the old and new solutions on real data before the switchover.
• Error rates and performance after every redirection of requests.
• The way back: how quickly the company can return to its original state.
• Data consistency and the number of manual corrections.
• The share of the workload taken over. If it does not grow over a long period, the project turns into dual running with no end in sight.

When a complete rewrite is justified

A complete rewrite is not always a mistake. Feathers describes it as a business decision about return on investment, although he himself tries refactoring first and points out that the whole system usually does not need to be rewritten and that targeted rewrites of individual parts tend to be very effective. Microsoft lists the situations in which gradual replacement is not appropriate: when the system is small and can easily be replaced as a whole, when there is no access to the source code, or when the original system needs to be decommissioned quickly. The Thoughtworks authors accept like-for-like replication of the old functionality for systems with a well-understood specification, where a given input must produce a given output, but they regard such cases as exceptional.

Our view: a rewrite is also defensible when the business has changed so much that the old system supports a way of working the company is moving away from. In that case, however, it is a new product with a new brief, not a copy of the old features.

Legacy system modernisation therefore does not start with choosing a technology, but with understanding what work the system does for the company, which of its parts are really used and where exactly the pain arises. Anyone who skips this diagnosis is choosing between a rewrite and refactoring in the dark.

Key takeaways

• You are not just choosing between “leave it” and “rewrite it”, but among six options. Stabilisation, an off-the-shelf solution or retirement tends to be cheaper than a rewrite.
• Before migrating, find out what the system really does: by observing people, from the logs and with tests of its current behaviour.
• Remove unused features first, otherwise you will end up migrating and testing them needlessly.
• Replace a large, critical system piece by piece: start where the pain is greatest and the risk lowest, and make sure you have a way back.
• Define a single source of truth for each type of data, and measure result parity, error rates and the share of the workload taken over.