Why Visual Context Fixes Indian Document Classification.

Document classification is the process of identifying and categorising documents based on their content, structure and context. For Indian enterprises, the challenge goes further: organisations also need to identify the personal and sensitive data contained within scanned KYC forms, loan applications, identity documents and other unstructured files.

In an earlier piece, we looked at why Indian PII is so difficult to classify. The formats that vary by state, the identifiers that change structure across decades, the documents that look different depending on which bank, which year, or which language they came from. That blog established the problem. This one answers the technical question it raised: if Indian documents are this complex, what approach actually solves them?

The answer starts with reframing the problem itself. Most document classification tools treat a document as a container of text. Open it, pull out the words, run the words through a set of rules, and flag whatever matches. This works well enough on a clean spreadsheet or a database export. It falls apart on the documents that carry the bulk of Indian personal data.

Consider a scanned loan application from a regional lender. The applicant's Aadhaar sits in a handwritten field. The PAN is stamped in one corner. The address runs across two languages. The form itself has no consistent template because the lender has used three different versions over the years. A driving licence is attached on the last page. A human looking at this document understands it instantly; they see where the Aadhaar is because of where it sits on the page, what surrounds it, and what the form is asking for. A text-based classifier sees none of that. It sees a jumble of extracted characters stripped of everything that gave them meaning.

Document classification for Indian data cannot be treated as a text-only problem. Visual and structural context, including layout, field position, script, and surrounding elements, can be critical to understanding what the data in a document actually represents. To see why, it helps to walk through the three approaches most document classification tools rely on, in order of increasing sophistication, and understand exactly where each one breaks.

Approach 1: Pattern Matching Shape Without Context

The simplest approach to document classification is pattern matching. You define what an identifier looks like: Aadhaar is twelve digits, a PAN is five letters followed by four digits followed by a letter, and you scan text for anything that fits the shape. Pattern matching has one virtue: it is fast and cheap. It also has one fatal limitation. It understands shape, and nothing else.

What a regex cannot do:

Determine whether a twelve-digit number is an Aadhaar, a transaction reference, or a device serial number
Handle an Aadhaar entered without spaces, with a typo, or sitting in a field the rule never anticipated
Distinguish a PAN from a product SKU, an employee code, or a batch number with a similar alphanumeric structure
Adapt when the same identifier appears in dozens of different contextual formats across document types

So it does one of two things, both bad. Either it flags every twelve-digit number as a potential Aadhaar, burying real PII detection findings under thousands of false positives, or it tightens the rule to reduce noise and starts missing every Aadhaar that does not fit the exact expected format. For Indian documents, where the same identifier appears in dozens of formats and dozens of contexts, pattern matching is not a foundation. It is a guess dressed up as a rule.

Approach 2: OCR Plus Regex Reading Without Seeing

The next step up is to add OCR. If pattern matching fails on scanned documents because there is no text to match against, the obvious fix is to extract the text first. OCR reads the characters off the image, converts them into a string, and hands that string to the same pattern matching engine. This feels like progress. It is not, or at least not nearly enough.

What can be lost when an OCR pipeline reduces a document to flattened text:

The spatial position of every field (whether something appears in a header, a form field, or a footer)
The visual relationship between adjacent elements (what sits next to what, what sits above what)
The document's structural identity (whether this is a KYC form, an invoice, or a salary slip)
Script context (whether a value is written in Devanagari or English, and what that implies about how to classify it)

Take the same loan application. OCR might successfully read the twelve-digit number. But it has no idea that the number sits in a field labelled Aadhaar, next to a photograph, above a signature block. It just has the number, floating in a stream of other extracted text, handed back to a regex that is right back to guessing. Extraction is not understanding. Reading the characters is not the same as knowing what they mean.

The most sophisticated text-based approach brings machine learning into the picture. Instead of rigid regex rules, you use named entity recognition and language models trained to identify personal data in text. These models understand that a name is a name even in an unusual format. They pick up on surrounding words as clues. They reason about language in ways a regex never could.

This is where many modern data classification tools and intelligent document processing platforms sit, and it is a genuine improvement. But it shares the same fundamental flaw as the approach before it, just hidden under a layer of sophistication.

Why language models over OCR still fail on Indian documents:

A language model reasoning over OCR output is still reasoning over a document it cannot see
It receives the same flattened text stream, with the same structure already stripped away
When OCR misreads a character in a Devanagari field, the model receives corrupted input it cannot recover from
When the visual cue that distinguished a KYC form from an invoice never made it into the text stream, the model has no way to reconstruct it
When an unlabelled field says only "Government ID," the model has neither the label nor the visual context to determine what value was entered

The sophistication masks the problem. You have a better reader, but it is still reading a shadow of the original document.

The Common Thread: All Three Approaches Discard the Visual Layer

Step back and look at these three approaches together, and the pattern becomes clear. They are not three different solutions. They are three points on a single line, each a more sophisticated version of the same underlying mistake. That mistake: they all convert a visual object into text before trying to understand it. They treat the visual nature of the document as an obstacle to strip away so the real work of text analysis can begin. For most Western documents, you can sometimes get away with this; those documents are often born digital, cleanly formatted, and text native. For Indian documents, you cannot.

Why the visual layer is not noise in Indian documents: it is the signal:

The position of a field tells you what kind of identifier it contains
The structure of a form tells you what the document is asking for
The script a value is written in affects how it should be read and classified
The document version (which generation of Aadhaar card, which era of PAN card) determines what layout to expect and how to interpret each element

Discard all of that, and you have discarded the answer before the sophisticated analysis even begins. This is why every text-based approach to document classification hits the same ceiling on Indian documents. You cannot analyse your way to the right answer using a method that deletes the answer in its first step.

What Visual Context Actually Means

Visual context is a different starting point entirely. Instead of converting a document into text and then analysing that text, a visual context approach reads the document the way a human analyst does.

What visual context sees that text-based approaches cannot:

The whole page at once: layout, structure, field positions, spatial relationships between elements
What sits next to what and above what, and what that positioning implies about the value in question
Script and text in place, without ripping them out of their surrounding structure
Document identity: whether this is a form, a header, a signature block, and how that structure interprets every element on the page

The difference is not incremental. It is categorical.

A text-based data classification tool asks: what words are on this page, and do any of them match my rules? A visual context approach asks: what is this document, what is it asking for, and given all of that, what does this value actually represent? The first question can only ever produce a guess. The second produces understanding. The question then becomes technical: what kind of system can deliver visual context at scale, across millions of documents, reliably? This is where the document visual language model comes in.

Why a Document Visual Language Model Is the Right Technical Answer

A document visual language model (DocVLM) is a class of AI built specifically to understand documents as visual objects rather than as extracted text. It is the technical realisation of everything the visual context argument demands and the only architecture that solves the document classification problem at the root rather than adding sophistication on top of a flawed foundation.

Where a traditional pipeline runs OCR to extract text and then hands that text to a separate analysis step, a DocVLM does not separate seeing from understanding. It processes the image of the document and its content together, in a single unified step.

What a DocVLM processes simultaneously, in one pass:

The layout of the page and the position of every field
The text content of each field, read in place
The visual relationships between elements
The document type and structural identity
Script variation across languages on the same page

Nothing is flattened. Nothing is discarded. The structure that every text-based approach throws away is exactly the information a DocVLM is built to use. For a document classification problem where the answer lives in layout, position, script, and structure, this is not one option among several. It is the only class of technology that actually operates on the information that matters.

How Privy is Built for This

Data discovery and classification are built on a document visual language model, purpose-built for the reality of Indian data. Because it reads documents the way a human analyst would visually, contextually, and as a whole, it handles precisely the failure cases that defeat every text-based approach.

Walk through the Indian document challenges from Part 1, and the difference is clear in each case:

Format variety: When an Aadhaar appears with spaces, without spaces, with a typo, or embedded in a sentence, a pattern matcher needs a new rule for each variation and misses the ones it did not anticipate. Data Compass recognises the Aadhaar from its role in the document, not from a rigid string match, so format variation stops being a failure point.

Version changes: The Aadhaar card has been redesigned repeatedly. The PAN card has changed across two decades. Bank statements differ across every institution. A data classification tool trained on one layout misses the next. Data Compass understands documents structurally, so a redesign is a variation it can interpret. The 2010 Aadhaar card and the 2024 Aadhaar card are not the same visual object, and Data Compass knows the difference.

Mixed scripts: When a document carries Devanagari, Tamil, and English on the same page, OCR fragments and the downstream text becomes unreliable. Data Compass reads script in place, as part of the document, rather than depending on a clean extraction that Indian documents rarely allow.

Unlabelled and ambiguous fields: When a form field says only "Government ID" and the value could be a Voter ID, a Passport number, or a Driving Licence, a text-based tool has no label to reason from. Data Compass uses the surrounding document, the layout, the position, and the neighbouring fields to determine what the value actually is.

In every case, the pattern is the same. The text-based approach fails because it discarded the information it needed. Data Compass succeeds because it kept that information and used it. To understand where document classification fits in your DPDP compliance programme, consider it the foundation layer the step that makes every obligation that follows (consent, rights management, breach scoping, data deletion) possible to execute correctly. And if you are assessing what a complete solution requires, the seven capabilities an ideal DPDP platform should cover show how document classification connects to the full stack.

The Right Questions for Evaluators

If you are evaluating a document classification tool for Indian data, the instinct is to ask how large its pattern library is, how many identifiers it covers, and how accurate its rules are. These are questions about the wrong layer. The question that actually determines whether a tool will work on your data is simpler.

Does it see the document, or does it only read the text extracted from it?

Every tool built on regex, on OCR plus regex, or on language models reasoning over OCR output answers that question the same way: it reads the text and does not see the document. For clean, text-native, predictable data, that can be enough. For the scanned, multi-format, multi-script, endlessly varied reality of Indian documents, it is not, and no amount of sophistication layered on top of text extraction changes the answer, because the failure happens in the first step, before the sophistication ever gets a chance. Data Compass answers the question differently because it is built differently. It sees the document. For Indian data, that is not a feature. It is the whole game.

Conclusion

Document classification for Indian data is an architectural question before it is a capability question. The three dominant approaches pattern matching, OCR plus regex, and OCR plus language models all share the same structural failure: they discard the visual layer that Indian documents carry most of their meaning in.

A document visual language model does not discard that layer. It builds on it. That is the difference between a document classification approach that produces a report and one that produces an accurate inventory. Under the DPDP Act's security safeguard obligations, the distinction between those two things is not academic. To see how Privy by IDfy handles the Indian documents in your environment, write to shivani@idfy.com to book a walkthrough.

FAQ's

What is document classification and why does it matter for DPDP compliance?

Document classification is the process of identifying what type of personal data a document contains and labelling it accordingly. Under the DPDP Act, data fiduciaries must implement security safeguards across all personal data they hold. That obligation applies regardless of whether the data lives in a structured database or an unstructured document like a scanned KYC form or a loan application. Classification is the step that makes the obligation enforceable; you cannot protect data you have not identified.

Why do pattern matching and OCR-based tools fail on Indian documents specifically?

Both approaches convert a document into text before analysing it. In doing so, they discard the visual structure, field positions, spatial relationships, document type, and script context that Indian documents carry their meaning in. Indian PII identifiers appear in dozens of formats across different documents, versions, and entry conditions. Without visual context, a data classification tool has no way to distinguish an Aadhaar from a transaction reference, a PAN from a product code, or a Voter ID from any other alphanumeric string. The failure is architectural, not a matter of how many patterns the library covers.

What is a document visual language model (DocVLM)?

A DocVLM is a class of AI that processes documents as visual objects rather than as extracted text. Instead of running OCR to produce a text string and then analysing that string separately, a DocVLM reads the document image and its content together in a single unified step. It understands layout, field positions, spatial relationships, script, and document structure simultaneously, the way a trained human analyst reads a document.

How does Data Compass handle documents that have changed design over the years?

Data Compass understands documents structurally rather than by matching a fixed template. Because it processes a document as a visual object and understands the relationships between elements, a redesigned document is a structural variation it can interpret rather than an unknown format it fails on. The 2010 Aadhaar card and the 2024 Aadhaar card are not the same visual object. Data Compass recognises both because it understands how Aadhaar cards are structured across generations.

What should enterprises ask when evaluating a document classification tool for Indian data?

The most important question is whether the tool sees the document or only reads the text extracted from it. Any tool built on regex, OCR plus regex, or language models reasoning over OCR output discards the visual layer before analysis begins. For Indian documents, that visual layer is where the meaning lives. A tool that cannot see it cannot classify reliably, regardless of how large its pattern library is or how sophisticated its language model is.

Why Visual Context Matters for Accurate Indian Document Classification