Why doesn't plain text extraction work for PDF tables?
Because a PDF does not store a table as a grid of rows and columns; it stores characters placed at coordinates on a page. A text extractor that tries to put those characters into reading order can mix up columns, split one cell across two lines or separate a value from the heading that gives it meaning. "12,400" on its own says nothing. Which product, which year and which currency it belongs to, and whether a footnote attaches a condition to it, can only be known if the table structure survives.
The problem grows at the chunking stage. A chunker designed for paragraphs cuts the table at a fixed length, leaving the header row in one chunk and the data rows in another. Even if retrieval finds the right row, the model receives a run of numbers without column headings. The hard cases come up again and again: multi-level headers ("2026" above "EUR" and "USD"), merged cells, units that sit in the header ("thousand TRY"), footnotes marked with an asterisk, subtotal rows, tables that run across several pages with the header repeated on each, rotated pages and scanned documents.
Some PDFs already carry the structure. In PDFs tagged for accessibility, a table is marked up with the Table, TR, TH and TD structure elements described in W3C's PDF6 technique, which is how screen readers understand the relationship between rows and headers. A large share of documents in company archives are untagged or wrongly tagged, though, so check the tags on a sample before relying on them.
Which parsing method fits which kind of PDF?
No single parser does well on every kind of PDF; the right approach is to sort documents by type and route each type to the method that suits it. The table below is a starting point for that routing:
| PDF type | Suitable route | Watch out for |
|---|---|---|
| Tagged digital PDF | Read the structure tags directly | Tags may be missing or wrong; verify on a sample |
| Digital table with ruling lines | Line-based parsing | Does not work on tables without lines |
| Borderless table aligned with whitespace | Parsing based on text alignment | Column boundaries are inferred; narrow columns can merge |
| Scanned or image-only PDF | OCR plus table structure recognition | OCR errors corrupt numbers; track per-cell confidence |
| Multi-level headers and merged cells | A layout model plus human review | You still have to rebuild the header hierarchy from the output |
The "How it works" page of the open-source Camelot library shows the distinction well: its Lattice method resolves tables that have lines between cells from where those lines intersect, while its Stream method infers the table from the whitespace between cells. Both are meant for PDFs with a text layer; a scanned page needs OCR first.
Cloud services return richer structure. Amazon Textract's table output returns each cell as its own block with row and column indexes, links merged cells through a separate relationship, can mark cells as column headers, table titles, footers, section titles and summary cells, and gives every block a confidence score. Microsoft's Document Intelligence layout model also returns row and column counts, row and column spans and header cells; in its v4.0 Markdown output, tables are written as HTML tables so that merged cells and multi-row headers survive. The same page contains an important detail: in this version a table's bounding region does not include its caption or footnotes. If you don't link the footnote back to the table yourself, a condition such as "prices exclude VAT" disappears.
How should table data be written to the index?
Write each row as a self-contained record that carries its own headers and context. A user's question usually lands on one row or one cell, so the unit of retrieval should be the row as well. Each row record needs at least these fields:
- Source: Document ID, version, page number, table ID and table caption.
- Row label: The row's name and its parent group, if any ("Support plans > Premium").
- Column header path: The full path for multi-level headers ("2026 > EUR").
- Value: Raw text, normalised number and unit. "12.400,50" and "12,400.50" are the same number in Turkish and English formatting; parse according to the document's language, and carry units such as "thousand TRY" from the header into every value.
- Footnote: The text the marker refers to, not just the marker.
- Quality data: Parser name and version, cell confidence and the cell's coordinates on the page.
Build the searchable text from these fields: "2026 price list, Premium support plan, annual price, EUR, excluding VAT: 10,300". Keep the structured cells separately; you will use them to verify answers and to calculate. In long tables, repeat the header row in every chunk. For table-level questions such as "Which table lists the discount rates?", a separate summary record with the table caption and column names also helps. Do not treat an empty cell as zero; unless the source defines a rule, store it as "no value". Mark subtotal and total rows too, so they are not mixed up with ordinary data rows.
Hypothetical example: retrieving the right cell from a price table
Imagine a procurement team's supplier quotes include a table like the one below, with the footnote "* Prices exclude VAT. 2026 prices are provisional." underneath. The figures are illustrative.
| Support plan | 2025 EUR | 2025 USD | 2026 EUR* | 2026 USD* |
|---|---|---|---|---|
| Basic | 4,200 | 4,550 | 4,400 | 4,750 |
| Premium | 9,800 | 10,600 | 10,300 | 11,100 |
A user asks, "What is the 2026 EUR price of the Premium plan?" The correct answer is 10,300 EUR, and it comes with two conditions: it excludes VAT and it is provisional. With plain text extraction the row becomes "Premium 9,800 10,600 10,300 11,100"; the model has to pick one of four numbers, and if the column order shifted during extraction it will state the wrong one with confidence. If the chunker dropped the header row into a separate chunk, nothing tells the model which column is 2026. If the footnote was kept apart from the table, the answer leaves out the "provisional" warning. In all three failures the answer can still cite the correct page, so a page-level citation check will not catch them. With a proper row record, retrieval brings back the "Premium" row, the "2026 > EUR" header path and the footnote text together, and the answer can be verified down to the cell.
How should you compare parsers?
Choose a parser by measuring it on tables from your own documents, not by its product page. A workable test design:
- Pick tables of every type from the archive: tagged and untagged digital PDFs, scanned pages, tables with and without ruling lines, tables spanning several pages and tables with merged headers. A few of each, 20–40 tables in total, is enough to start.
- Write down each table's expected grid by hand: cell values, row labels, column header paths, units and footnote links.
- Run every candidate method (plain text, a line- or alignment-based parser, a cloud layout service) on the same tables.
- Measure cell content accuracy, the share of values linked to the correct row and column header, whether units and footnotes survive, and whether the normalised number matches exactly. The GriTS metric from table structure recognition research treats a table as a matrix and evaluates cell topology, location and content in one framework; it is a good reference when you build your own scoring.
- Record processing time and cost per page.
- Then test retrieval: write 30–50 questions taken from real user questions, mark the expected row ID for each, and measure whether the right row appears among the first few results and whether the number in the answer matches the number in the cell.
The decision rule can stay simple: for each document type, pick the cheapest method that clears your threshold and route documents by type. For types where no method clears it, open a separate route with human review.
How do you answer comparison and aggregate questions?
Row-level retrieval works well for questions about a single value, but not for questions that require scanning the whole table, such as "Which plan is cheapest?" or "Which item rose most from 2025 to 2026?", because retrieval only returns the few most similar rows. If such questions are common, also write the tables to a structured store, such as a relational database table, and add a query step that routes these questions there. Show the query result with its source table and page as well. Decide which questions take this route from your test questions; sending every question to a structured query adds an unnecessary source of errors to simple value lookups.
How do you prevent table errors at answer time?
Give the model the relevant row with its headers, unit, footnote and page, not a loose list of values, and require citations down to page, table and row. Do not leave sums, differences or ratios to the model: calculate them in code from the structured cells and show the operands with the answer so a reviewer can reproduce the result. A simple automatic check goes a long way: every number in the answer must appear exactly in the cited cell, and if it does not, the answer goes to review. If cell confidence is low, the header path is ambiguous or two versions of the same table disagree, show an image of the source page and ask the user to confirm. Keep the original PDF available at all times; the parsed structure is a derived view of the evidence, not the evidence itself.
Common mistakes
- Leaving tables to the paragraph chunker and separating headers from data rows.
- Not carrying the header's unit and the footnote marker into the values.
- Treating empty cells as zero and subtotal rows as ordinary rows.
- Indexing the header row repeated on each page as a data row.
- Checking only the page citation and never comparing the number in the answer with the cell.
Where should you start?
Start with an inventory of the table types in your archive: how many documents are scanned, how many have multi-level headers, which questions land on tables? Then run the small test above and choose the method by measurement. If scanned tables dominate, add the OCR quality gates from our guide to building RAG from scanned PDFs to the same test. To see how ingestion that extracts text, tables and metadata from PDF and Word documents fits together with search, permissions and citations, see our enterprise RAG and semantic search service page.