Integrating two systems: six questions to answer before writing code

Connecting two systems looks like a small job. A CRM should send new customers to the invoicing tool, or an online shop should update stock in the warehouse system. A developer can often get a first version running in a day. The trouble starts a month later, when a customer exists twice, an invoice is missing, and nobody knows which system is right.

Almost all of these problems come from questions that were not asked at the start. This article lists six of them. Answer them on one page, together with the people who use both systems, and the code that follows will be smaller and easier to trust. For the wider picture of how connections grow over time, see API integration patterns for growing companies.

1. Who owns the data?

For every piece of data that moves, name one system as the owner. The owner is the place where people are allowed to change it. The other system only receives a copy.

Take a customer address. If staff can edit it in both the CRM and the invoicing tool, the next sync will overwrite one edit with the other, and someone will lose work without noticing. Go through the fields one by one: name, address, tax number, price, status. Write the owner next to each. Where the business truly needs edits in both places, say so openly and agree on a rule, such as "the most recent change wins", knowing that it has limits.

2. Which direction does it flow?

One-way is far simpler than two-way. A one-way flow from the owner to the copy has one possible source of conflict: none. Two-way flows need conflict rules, loop protection (system A updates B, which updates A again) and more testing.

Ask for each field whether the other system really needs to send it back. Often the answer is only a status, such as "invoice paid", and not the whole record. Moving that one field back is a small, separate flow, much easier than a full two-way sync.

3. What happens when it fails?

It will fail. The other system will be down for maintenance, a required field will be empty, a token will expire. Decide beforehand:

• Who is told? A named person or a shared mailbox, not "the logs".
• What happens to the failed item? It should wait in a visible list and be retried, not disappear.
• How many retries, and how often? Three tries over an hour is reasonable for a short outage.
• What is the manual fallback? If the integration is down for a day, how does the work continue?

A useful test: imagine the connection breaks on Friday evening. On Monday morning, can a colleague who is not a developer see what is missing and fix it? If not, add that to the plan.

4. How often does it need to run?

"Real-time" sounds like the safe answer, but it costs more to build and operate than a regular batch, and it makes errors appear instantly and in volume. Start from the decision the data supports. Stock levels for a shop may need minutes. A monthly management report does not need anything faster than daily.

Write one sentence: "This data must be no older than X, because Y." If you cannot finish the sentence with a real reason, choose the slower option. You can always speed it up later, which is easier than slowing down a flow people have begun to depend on.

5. How will we check that it works?

Before launch, agree on a few checks that anyone can repeat:

• A count. Number of records in the source for a given day against the number in the target.
• A sample. Ten records compared field by field, including some with unusual values such as long names, empty fields or special characters.
• A total. For money, the sum of invoices on both sides for the same period.

Use these both for the first test and after launch. A short weekly check catches slow drift that no error message will report. If the integration touches many systems, integrating CRM, email, documents and chat without breaking work has more on keeping the picture clear for staff.

6. How do we go back?

Before the first live run, know how to undo it. Can you stop the flow with one switch? Is there a copy of the target data from before the start? If a thousand records were changed wrongly, can you identify exactly which ones?

This does not need to be elaborate. A backup, a "dry run" mode that shows what would change without changing it, and a log of which records were touched are often enough. A plan for the way back makes the way forward less frightening, and it lets you start with a small group of records instead of everything at once.

The one-page brief

Put the answers in a short document. The following example is invented, only to show the level of detail.

One page like this takes an hour or two to write and several people to agree on. It usually reveals a disagreement or a missing owner, which is exactly the point: it is far cheaper to find it now than after launch.

When the answers do not fit

Sometimes the honest answers show the project is bigger than expected, for example because there is no clear owner of customer data at all. That is a useful result. Fix the ownership first, or reduce the scope to one flow that is clear, and leave the rest for later. For larger landscapes with many systems, an enterprise integration strategy looks at how to order the work.

What to do this week

Pick the integration you are planning, or the one that gives you the most trouble now. Write the six questions on a page and fill in the answers with one person from each team that uses the data. Mark every answer you are unsure about. Those marks are your list of things to settle before any developer starts.

Sources

• Our related articles linked above.
• The example table is invented for illustration and does not describe a real client or system.