Bad data rarely starts in the database. It starts in the form: a field left empty because nobody knew what to write, a phone number typed three different ways, a "Notes" box used for everything. Cleaning this up later is slow and expensive. Preventing a part of it takes a few decisions made once, when the form is designed.
The decisions pull in opposite directions. Too few rules and the data is unusable. Too many and staff type "x" or "n/a" just to get past the screen, which is worse than an empty field because it looks like real data. This article gives a short rule for finding the balance.
Rule one: required means someone uses it
Before marking a field as required, ask one question: who will use this value, and for what? If you cannot name a person, a report or a step in the process, the field should not be required, and often should not exist.
A practical test is to look at what happens when the field is empty:
- The next step cannot happen. An order without a customer cannot be delivered. Required.
- The next step is harder, but possible. A missing phone number means sending an email instead. Optional, perhaps with a hint.
- Nothing happens. Nobody reads it. Remove it or leave it optional.
A related point is timing. A field may be needed later but not now. A quote does not need a delivery address; the order does. Require it at the step where it matters, not on the first screen. If your process hands work between teams, the list of fields that must be present at the handover is a good starting point; see the quote-to-order handoff for an example.
Rule two: let the machine check what it can check
Some things can be checked reliably, and checking them saves a lot of work later.
- Format. An email address contains an @ and a domain. A postal code has the expected length. A date is a real date.
- Range. A quantity is not negative. A discount is not more than 100 percent. A delivery date is not in the past.
- Choices from a list. Where only a few answers are valid, such as status, country or product group, offer a drop-down instead of a text box. Spelling variants disappear.
- Duplicates. If a customer with the same tax number already exists, say so before saving a second one. This is far cheaper than the cleanup of duplicate customer and supplier records later.
Keep these checks simple. A check that is too clever will sooner or later reject a valid entry, and the person at the screen will stop trusting the tool.
Rule three: do not validate what only a human can judge
Some fields look like they could be checked but cannot be, at least not without annoying people.
- Names. People have short names, long names, one name, apostrophes and hyphens. Do not set a minimum length or a list of allowed characters beyond what the database needs.
- Phone numbers. Accept spaces, plus signs and brackets, and store them in a consistent way behind the scenes instead of rejecting how someone typed them.
- Addresses. Offer enough room and do not force a format that works in one country only.
- Free text. A notes field is for things you did not predict. Do not require it and do not limit it more than you must.
The general principle from the GOV.UK Design System validation pattern is that validation should help people complete a task, not test them. Use that as a guide, not a rule.
Messages that help
A rule is only as good as its message. "Invalid input" tells the person nothing. A good message says which field, what is wrong and how to fix it: "Quantity must be 1 or more" or "Enter the date as day, month and year, for example 14 11 2026".
Show the message next to the field and keep what the person has already typed. Nothing is more frustrating than a form that clears itself after an error. The same thinking applies to the wider question of what staff see when something goes wrong; the article on error messages in an internal tool covers it.
Go through the form field by field
Take your current form, or the draft of a new one, and fill in a small table. It takes about an hour with the person who uses the form daily.
| Field | Who uses it | Required? | What the machine checks | Message if wrong |
|---|---|---|---|---|
| Customer | Dispatch | Yes | Must exist in the customer list | "Choose a customer from the list" |
| Quantity | Warehouse | Yes | Whole number, 1 or more | "Quantity must be 1 or more" |
| Delivery date | Dispatch | Yes, at order step | Real date, not in the past | "Choose a date from today onwards" |
| Contact phone | Driver | No | Digits, spaces, plus sign | "Use digits, spaces or a plus sign" |
| Notes | Anyone | No | Nothing | None |
The example is illustrative. The point is that every row has a named user, and every "Yes" has a reason.
Check again after a month
Rules made at the desk meet reality when staff start to use the form. After a month, look at the saved data for three signs:
- A required field full of dummy values such as "x", "-" or "test". Either the field should not be required or people do not know what to enter. Ask them.
- An optional field that is almost always empty. Probably nobody needs it. Consider removing it.
- A rule that people keep hitting. If many entries are rejected for the same reason, the rule or the message needs a change. Collect these cases in your usual place for change requests after launch.
Common mistakes
- Making everything required "to be safe". It produces dummy data and slows people down.
- Checking only on the screen. Rules should also apply when data arrives from an import or another system.
- Rejecting unusual but valid entries. Prefer a warning that can be confirmed over a block.
- Vague messages. "Error" helps nobody.
- Never reviewing the rules. The process changes and the form should follow.
A first step
Pick the form your staff use most. Write down its fields, name the person who uses each one, and mark as required only those where the next step cannot happen without them. Then add two or three machine checks where bad entries have hurt you before. That is usually enough to see a difference within a few weeks.
Sources
- GOV.UK Design System, Validation pattern
- MDN Web Docs, Client-side form validation
- Nielsen Norman Group, Required Fields in Web Forms
- This article is a planning aid. The example table is illustrative. No client results are claimed.