Why Data Readiness Decides the Project Before the Model Does
Most AI projects that stall do not stall on the model. They stall because the data the plan assumed existed turned out to sit in another system, in a worse format, under an owner who never agreed to release it. An NVIDIA 2026 survey reported 48 percent of respondents citing data-related issues as a blocker to AI adoption, ahead of a shortage of AI expertise and a lack of clarity on return. That matches what we see in delivery: the technical risk in a first document or forecasting project is rarely the model choice and almost always the real state of the corpus.
This article is the assessment itself, written out so you can run it without us. Nine dimensions, each scored zero to three against a rubric that describes observable conditions rather than opinions, for a total of 27 points. Each dimension carries a remediation band in engineering effort, so a low score converts into weeks of work rather than into a vague concern. The output is a number, a slice definition and a remediation plan with named owners. Run it before you write the scope document, because a scope written against imagined data fails in week six rather than week one, as we argue in how to define an AI project scope.
Two warnings before the rubric. A readiness score is not a judgement of your organisation: nearly every enterprise we assess scores badly on at least two dimensions, and the median first assessment lands in the middle band. The score exists to route the project, not to grade the client. And do not let anyone answer from memory. Every score must be backed by evidence somebody actually pulled: a row count, a sample of files, a screenshot of a permissions dialogue, a written answer from legal. The gap between what a data owner believes about their store and what a random sample shows is the most expensive surprise in this work.
The Scorecard: Nine Dimensions, Zero to Three, 27 Points
The nine dimensions are findability and inventory, access and permissions, format and machine-readability, volume and long-tail coverage, labelling status and label agreement, freshness and update cadence, lineage and ownership, quality defects, and legal and consent posture. Each scores zero to three, so the range is zero to 27. Zero means the condition is absent and nobody can describe it. One means it exists informally and has never been measured. Two means it is documented, measured and workable for a first production workflow. Three means it is engineered: automated, monitored and reproducible by someone other than whoever built it.
Read the total in four bands. Twenty-two to 27 is green: build now, and expect data work to stay under roughly fifteen percent of total project effort. Sixteen to 21 is amber: proceed, but insert a remediation sprint of typically three to six weeks before the model work, written into the plan as a phase with its own acceptance criteria. Ten to 15 is red-amber: this is a data engineering engagement with an AI outcome attached, usually two to four months before a pilot means anything, and pretending otherwise moves the same work later at a higher cost. Below ten, do not start.
Three dimensions can veto a good total on their own. A zero on legal and consent posture stops the project regardless, since no amount of clean text makes unlawful processing lawful. A zero on access means the team cannot obtain a copy, which turns every other score into speculation. A zero on machine-readability at the scale you need means the first deliverable is an OCR programme, not an AI system. Score in one two-hour session with the evidence already collected, resolve disagreement downwards, and record a line of evidence per score. For document-heavy work we double the weight on access, machine-readability and legal posture, giving a 36-point scale with proportionally scaled bands.
Dimensions One to Three: Can You Find It, Get It, and Read It
Findability and inventory asks whether you can name every store the workflow will touch, with a count and a named human owner. Zero: nobody can list the systems, and the answer to where the contracts live is somewhere on the shared drive. One: two or three systems are named, with no counts and no owners, and discovery happens by asking colleagues. Two: each system is named with an order-of-magnitude count and a named owner. Three: a written inventory with exact counts, storage location, owner, retention rule and refresh cadence. Building that inventory from scratch typically takes three to eight engineer-days for five to ten stores, plus one to two days per additional system with its own authentication.
Access and permissions asks whether the project team can hold a copy, not whether access is theoretically possible. Zero: no path, no precedent for bulk export, and every request routes to a committee that meets monthly. One: possible in principle, file by file, with six to ten weeks of approvals. Two: bulk export of a defined slice is approved within two to three weeks and a read-only service account can be issued. Three: a scoped service account exists or can be issued in under five working days, with a documented API or dump procedure. Integration after approval is small, typically two to five engineer-days; the approvals are calendar time, and we budget three to eight weeks for them.
Format and machine-readability most often produces an unpleasant zero. Zero: scanned images only, mixed resolution with a meaningful share below 200 dpi, no text layer, handwriting on the fields that matter. One: a mix, perhaps 40 to 60 percent native text, with an OCR layer from an old engine that visibly mangles Turkish diacritics. Two: over 80 percent native text with recoverable tables and consistent encoding. Three: near-total native text or structured records with a documented schema. An OCR pipeline over 100,000 to 500,000 pages typically runs three to six weeks including quality sampling; re-scanning paper is a physical operation measured in months. Measure this rather than debating it: pull a random 200 pages, run your intended OCR, and report character error rate overall and on Turkish specific characters separately, because a system reading Latin text at 98 percent still fails on the letters that carry meaning in a name.
Dimensions Four to Six: Coverage, Labels and Freshness
Volume and long-tail coverage asks whether the rare cases that carry the risk are represented at all. Zero: only recent, common examples, and the rare classes have between zero and five instances. One: the head is well covered but the classes that matter for exceptions have under 30 examples each. Two: every class the first workflow must handle has at least 50 examples and the rarest has 20 or more. Three: 200 or more per class, sampled deliberately across time, region and business unit rather than taken as the most recent slice. Collecting long-tail examples is mostly business effort, typically two to six weeks of a domain expert pulling and confirming cases from the archive.
Labelling status and label agreement is where most assessments discover that nobody wrote down what correct means. Zero: no ground truth anywhere. One: labels exist as a by-product of a business process, definitions undocumented, agreement never measured. Two: a written labelling guide, 200 to 500 labelled items, and a double-labelled sample where two annotators choose the same answer on roughly four items in five. Three: guide, adjudication process, and a frozen held-out evaluation set of 300 to 1,000 items nobody trains or tunes against. A 300 to 500 item gold set with double labelling typically costs eight to fifteen person-days of domain-expert time plus two to four engineer-days of tooling. Treat disagreement as information about the task: if two senior experts disagree on 30 percent of items, no model will measure above that ceiling, and the honest fix is to rewrite the definition until disagreement drops.
Freshness and update cadence asks how stale the snapshot is and how the next one arrives. Zero: a snapshot of unknown age with no route to a new one. One: an annual or ad-hoc dump, the last one older than six months, refreshed by raising a manual ticket. Two: a documented monthly refresh in which changed records can be identified. Three: an incremental feed with change data capture, reliable modification timestamps or an event log, with lag under 24 hours. Turning a manual dump into a scheduled incremental pipeline for a single source typically takes two to four weeks, and considerably longer when the source has no trustworthy modified timestamp and you must diff whole snapshots to find changes.
Dimensions Seven to Nine: Lineage, Defects and Legal Posture
Lineage and ownership asks whether any answer the system produces can be traced back to a source record. Zero: no route from a row or a document back to a system of record, and no named owner. One: informally known, with the owner being a team rather than a person. Two: each dataset has a named owner, a documented source and usable join keys. Three: stable identifiers end to end, so a user can be shown the exact record behind an answer. Retro-fitting stable identifiers and a provenance field typically takes one to three weeks per source. Say it plainly to the sponsor: without lineage you cannot produce citations, and without citations you will not get sign-off in a regulated workflow.
Quality defects covers duplicates, contradictions, superseded documents still in circulation and encoding damage. Zero: none of it is quantified, obsolete versions circulate freely, and mangled characters are visible in Turkish text. One: problems known but unmeasured, with 10 to 30 percent near-duplicates suspected. Two: measured, duplicate rate under 10 percent, a written rule for which version wins, encoding normalised to UTF-8. Three: measured and monitored, supersession modelled so each document knows its successor, plus contradiction checks on a sample. Deduplication across 100,000 to 500,000 documents typically takes two to four weeks, mostly business validation rather than code. Work the arithmetic: 200,000 documents at a 22 percent near-duplicate rate means 44,000 redundant items clustered around the documents people care about most, so a top-ten result carries four copies of the same circular and genuine coverage collapses from six positions to three.
Legal and consent posture is the veto dimension. Zero: lawful basis unknown, special-category data present and unflagged, or a supplier contract prohibiting third-party processing of the content. One: lawful basis debatable, no documented assessment, no clear answer on whether processing may run outside your own infrastructure. Two: lawful basis documented under Law No. 6698 articles 5 and 6, a recorded decision on where processing may run, contracts reviewed. Three: all of that plus a designed minimisation or redaction step, an approved processing location, and written answers for the article 10 duty to inform and the article 12 security obligation. KVKK's generative AI guide is the natural reference, and we work through it in our read of the KVKK generative AI guidance.
Turning the Total Into a Decision
Take a worked example of the shape we commonly see. An organisation scores findability 2, access 1, machine-readability 1, coverage 2, labelling 0, freshness 2, lineage 1, defects 1 and legal posture 2. The total is 12 out of 27, in the red-amber band. The zero on labelling is not a formal veto, but it is decisive, because there is no way to measure whether the system works. The recommendation is not no. It is a ten to fourteen week first phase delivering an access path, a 400-item gold set, an OCR baseline and a deduplicated slice, with model work starting only after that phase passes its own acceptance test.
That reframing is what makes the scorecard worth running: it moves the data cost from a hidden overrun into a line item. In the projects we run, data preparation sits somewhere between fifteen and forty percent of first-phase effort, and the assessment tells you which end of that range you are at before you sign anything. A proposal showing 12 out of 27 and a costed remediation phase is far easier to approve than one showing an optimistic schedule that asks for more money in month three. We break the cost structure down further in what an AI project actually costs.
Re-score after remediation rather than declaring victory. The point of a rubric with observable levels is that the second run is cheap: evidence collection is already scripted and the session still takes two hours. We typically see a well-executed remediation sprint move three dimensions by one level each, enough to cross from red-amber into amber but rarely enough to reach green in one pass. Re-score again before the production decision, because access and freshness routinely regress when a pilot moves off an engineer laptop onto infrastructure somebody else must provision, a transition we cover in moving a pilot into production.
Good Enough Is a Slice, Not a Corpus
You almost never need clean data. You need a defined slice that is clean enough for the first workflow, and the discipline to write down what the slice is. Cut it on four axes at once: one document type, one department or business unit, one date range, one language. A slice defined this way is testable, because a single domain expert can look at every edge case inside it, and it is negotiable, because you can point at exactly what is out of scope. A corpus defined as everything in the archive is neither. The most common cause of an eighteen-month pilot that never ships is a scope that was never cut.
Compare two corpora. Five thousand documents that are eighty percent clean give you four thousand usable items, and crucially you can identify which thousand are not. Five hundred thousand documents that are forty percent clean give two hundred thousand usable items in principle, but you cannot say which ones, so every retrieval result is a coin flip and every evaluation number is an average over noise you cannot localise. Cleanliness you cannot localise is not cleanliness. The smaller corpus also makes evaluation affordable: a 300-item sample from 5,000 documents of one type is representative, while a 300-item sample across 500,000 documents of forty types is representative of nothing.
Cut the slice narrow, but do not cut the difficulty out of it. The temptation is to select the clean, recent, well-formed documents, which produces a pilot that demonstrates nothing and collapses on contact with the real distribution. Our rule is that the slice must contain the long tail of its own scope: if the department handles nine document subtypes, all nine belong even though one appears eleven times. Drop the adjacent department instead. Write the expansion rule at the same time: expand only when the slice holds its evaluation numbers over two consecutive refresh cycles, and expand one axis at a time. Adding a second document type to a working RAG system is typically one to three weeks including a new evaluation set, while a second language costs more because retrieval quality and chunking behaviour both change, as we discuss in building RAG on Turkish legal data.
The Failure Modes This Catches in Week Zero Instead of Week Six
The digital archive that is 300,000 scanned images. Somebody says the archive was digitised in 2015, which is true, and the assessment discovers that digitised meant bitonal scans at 200 dpi with no text layer, sorted into folders named after the scanning batch. Found at week zero, this is an OCR programme of three to six weeks with a known accuracy ceiling and an honest conversation about which fields are recoverable. Found at week six, it is a schedule that has already promised a demo, a team that built a retrieval stack against a clean sample of 500 files somebody emailed, and a sponsor hearing the word delay for the first time.
Legal will not approve the export. This one is fatal when it arrives late, because it invalidates the architecture rather than the timeline. A team that spent six weeks building against an external processing assumption discovers that the supplier contract prohibits sending content to a third party, or that the data set carries special-category records nobody flagged. Asked in week zero, the same question produces a design constraint: processing stays on infrastructure you control, model choice narrows accordingly, and the cost estimate reflects it from the start. Ask a specific question. Not is this allowed, but who signs the approval, what do they need to see, and how long have comparable approvals taken here.
Then the quiet ones. The label that is not a label: a category field populated by whoever filed the document, so the model learns filing habits and the pilot reports high accuracy against labels wrong roughly a third of the time. The single source that turns out to be three: two departments migrated at different times, records overlap, no shared key, so 180,000 documents is really 140,000 plus 40,000 partial duplicates with divergent metadata. The freshness trap: the pilot runs on a week-one snapshot and nobody owns the refresh. And encoding damage, where a legacy export mangles Turkish characters and exact-match filters silently miss those records because the demo queries were typed in English. All four are invisible to a schema review and obvious in a random sample of 300 records.
Running the Assessment: The Room, the Clock, the Artefacts
Six roles need to be in the room, and five of them are not engineers. A data owner for each store, who can say what is actually in it. One person from infrastructure who can issue credentials, not someone who can request them. A domain expert with the authority to define what correct means. Someone from legal or data protection who can answer lawful basis questions rather than escalate them. One engineer who will do the work and therefore has to believe the answers. And the sponsor, for thirty minutes at the end, to hear the score and the recommendation directly rather than through a summary slide two weeks later.
Typical elapsed time is two to three weeks for a single-workflow scope, against five to ten engineer-days of actual effort. The gap is waiting: for a credential, a legal answer, an export job. With more than six source systems, or legal review starting from nothing, four to six weeks is the honest range. Our sequence is a half-day kickoff to build the inventory, one week of sampling and measurement, a two-hour scoring session, a readout. Never accept a curated sample: ask for a random draw defined mechanically, by identifier modulus or date interval, and pull it yourself where you can. A sample chosen by a helpful colleague is systematically cleaner than the population, because helpful colleagues pick documents that are easy to explain.
Three hundred records is enough to estimate a duplicate rate or an OCR error rate to a precision that supports a go decision; a thousand is worth it when the population is heterogeneous and you need per-class estimates. Five artefacts come out, all short. An inventory sheet, one row per store, with counts, owner, access route and refresh cadence. A scorecard with one line of evidence per dimension. A slice definition of one paragraph. A remediation plan with effort bands and a named owner per item. And a measured baseline: OCR character error rate, duplicate rate, label agreement. That baseline becomes the reference point behind measuring return on an AI investment.
How We Run This at HatsonTech
We run this assessment before quoting on any document-heavy engagement, and we run it as paid work rather than pre-sales, because it consumes real engineering days and produces a document the client owns whether or not they continue with us. The deliverable is the five artefacts above. The first thing we ask for is a random draw of 300 to 1,000 records defined by a rule we can check; the second is a named person on the infrastructure side. If neither is available in the first week, that is itself a finding, and we say so in the readout rather than working around it.
Two measurements we always take. First, OCR character error rate on a 200-page random sample, reported separately for Turkish specific characters, because that number decides whether exact-match filtering and hybrid search are viable or whether everything has to lean on embeddings. Second, near-duplicate rate with a written rule for which version supersedes which, validated by the business rather than assumed by us. Both habits come from our own products: caseon.ai and DiligenceAI operate on document sets where a superseded version returned confidently is worse than no answer, and that constraint shaped how we score defects and lineage.
We say no more often than clients expect. Below roughly twelve out of 27 we will not start a model engagement, and we propose a data engineering phase instead, with its own scope and acceptance criteria. That is not caution for its own sake: a model built on a corpus nobody can refresh or trace becomes unsupportable within a quarter, and the cost of unwinding it exceeds the cost of doing the pipeline work first. If your score lands in that band, the useful next step is our data engineering work, which is where the inventory, the access path, the OCR baseline and the refresh schedule actually get built.