"Our data is pretty good" is the sentence we hear most often at the start of an AI project, and it is almost always sincere. It is also almost always untested against the specific use the team has in mind. Data that is good enough for a monthly finance report can be useless for a model that needs to know what happened yesterday, and data that is perfect for one team's dashboard can be inaccessible to the system that needs it.

We break readiness into four separate tests, because they fail independently and they are fixed by different people.

Four tests, not one score

Clean asks whether the values mean what they say. Duplicates, inconsistent categories, free-text fields where a code should be, dates in three formats. Connected asks whether the data can be joined and reached: shared keys across systems, an API or export that a pipeline can use, refresh timing that matches the need. Secure asks whether we know who can see what, and whether that policy survives the data moving into a new system. Useful asks the question people skip: does this data actually contain the signal the use case needs?

The four readiness tests, who typically owns each, and what fixing it looks like.
The four readiness tests, who typically owns each, and what fixing it looks like.

The project where "clean" was not the problem

An insurance client asked for a model to flag claims likely to need extra review. Their claims data was genuinely clean: well-typed, consistent, deduplicated. We built a first model and it learned almost nothing. The problem was the fourth test. The signal that made a claim complex lived in the adjuster's notes and the attached documents, and neither was in the dataset. The structured fields recorded the outcome of the review, not the reasons for it.

We spent the next phase not on modelling but on getting the notes and documents into a form the system could read, with the access rules that came with them. The model that followed was useful. The lesson was that a clean dataset can still be the wrong dataset.

Ask what evidence a human uses to make the decision. If that evidence is not in the data, the model cannot learn the decision.

How we run the assessment

We do not run a generic data-quality tool across everything. We start from the use case and work backwards to the handful of tables and documents it depends on. For each source we record five things: where it lives, who owns it, how it is accessed, how fresh it is, and a sample of what is actually in it. The sample is the important part. Reading two hundred real rows tells you more than any profile report.

Then we run the specific checks the use case implies:

  • For analytics: are the business entities consistent across systems, and is there one agreed definition of the metrics that matter?
  • For retrieval and assistants: are documents current, is there a way to know when one is superseded, and do permissions on the documents map to people who will ask?
  • For prediction: is the outcome we want to predict actually recorded, recorded consistently, and recorded before the point at which the prediction would be used?

Connected is usually the slowest fix

Cleaning can be scripted. Connecting depends on other people. The most common blocker is a system with no API and an owner who exports a spreadsheet on Fridays. We plan for it: the first pipeline reads the spreadsheet from a shared location on a schedule, validates it against a contract, and raises a flag when the format changes. It is not elegant. It is reliable, and it stops the project waiting on an integration that may never be prioritised.

A minimal pipeline for a source that only exists as a periodic export: read on schedule, validate against a contract, alert on drift.
A minimal pipeline for a source that only exists as a periodic export: read on schedule, validate against a contract, alert on drift.

Security: the test that changes the architecture

Data that is well-governed in its source system loses its governance the moment it is copied into a new store, unless the copy carries the rules with it. For AI systems this matters twice: once for the pipeline that moves the data, and again for the model or assistant that answers questions over it. An assistant that can read every document in the index will answer questions about documents the asker was never allowed to open.

So we treat access as part of the data model. Every record and document in the system carries who may see it, and every query is filtered by the asker's identity before the model sees anything. We also decide explicitly what leaves the client's environment: which fields go to an external model provider, whether they are anonymised first, and what the provider's retention terms are. These decisions are documented alongside the data map, because they are the answers the security team will ask for.

Using the assessment purposefully

The output is not a score. It is a short plan: which sources are ready, which need a specific fix and who owns it, and which use cases are blocked until a fix lands. Often the honest result is that the first project should be a data project, with the AI feature following once there is something reliable to build on. Teams sometimes hear that as bad news. It is the opposite. It is the difference between a model that ships and one that stays a demo.

  1. Start from the use case and list only the sources it needs.
  2. Read real rows from each source before trusting any profile.
  3. Test clean, connected, secure and useful separately.
  4. Fix connection problems with a contract and a schedule, not by waiting for an integration.
  5. Make access rules part of the data, so they travel with it.