DPDP and AI Governance: The Inference Gap

The old data governance question was simple: what personal data do you hold, and is it protected? A recent IDfy webinar on AI Governance and DPDP compliance, hosted by Tridib Mukherjee with Pramod Pal, CISO at TataCliQ, and Animesh Mishra, who leads engineering, compliance, and governance at Jupiter Money, landed on a harder one. What does your AI now know about your customers that they never told you, and did they ever agree to it knowing that?

That question is the gap this piece is about. India's enterprises are moving fast from AI assistants to autonomous AI agents, while the DPDP Rules 2025 push in the opposite direction toward accountability and consent. Sitting between those two forces, most AI governance frameworks watch data going into a model and ignore the data coming out. The output is where the DPDP exposure now lives.

To watch the webinar, click here

The Concept DPDP Drafters Did Not Plan For

Personal data, as most privacy programmes treat it, is what a customer hands over directly: name, phone number, email, address, and order history. Pramod Pal's argument in the webinar was that AI has quietly created a second category that no privacy notice covers.

Take a customer who uses a food delivery app, a UPI app, and a social platform. Each holds data the person consented to share. Combine those signals through a model, and it starts inferring things nobody typed into a form: late-night ordering habits, heavy spending on medicines and hospitals, frequent searches for mental-health content, and a spending pattern that reads as financial stress. From there, the model concludes the person may have anxiety and unhealthy eating habits and flags them as a target for an insurance or loan offer priced accordingly. The customer never disclosed any of it. The model produced it.

Pal called this inferred data and made the case that it deserves protection as a sensitive asset even though no one consented to its creation. His retail example was sharper still. A fashion recommendation engine trained on three signals – what a customer browsed, what they returned, and how long they lingered on a page – starts inferring body-image preferences, size insecurity, and price sensitivity. As he put it, nobody consented to that; nobody designed it; the model just learned it.

Why Consent Alone Does Not Cover Inference

DPDP is built on a consent model: a data principal consents to a purpose, the data is processed for that purpose, and accountability follows the purpose. Inference breaks the chain because AI does not merely process the data; it transforms it into something qualitatively different from what was agreed.

A customer consents to share a delivery address for logistics, purchase history for recommendations, and support-chat logs for issue resolution. Each is a clean, bounded permission. Then a model combines all three and outputs a household income bracket, a life stage, whether the person is a new parent or recently married, and what Pal described as brand-loyalty elasticity, exactly how much discount it takes to stop them from churning. None of that appears in any consent notice or privacy policy. It sits in a feature store, driving decisions about real people.

This is why the governance-first view that consent alone is not DPDP compliance matters more in the AI era than before. Consent governs the inputs. Inference happens after consent, on the output side, where most frameworks are not looking.

The Erasure Gap: Deleting the Input Does Not Delete the Inference

The webinar surfaced a problem that sits directly against the DPDP right to erasure. When a data principal asks a company to delete their record, the company deletes the row from its customer systems. But the inference the model already drew from that record persists in the vector databases, embedding stores, and fine-tuning datasets that rarely appear in a data inventory or a backup policy, and rarely in the erasure workflow.

Pal put the gap plainly: deleting the input is not deleting the inference. The customer exercised their right, the record is gone from the database, and a shadow profile of them still lives inside the model. Closing that gap starts with knowing those stores exist and are classified as personal data in the first place, which is a data discovery and classification problem before it is a deletion problem. You cannot erase what your inventory never recorded.

His practical prescription was twofold. First, classify AI-generated data as sensitive personal data: if a model predicts a customer sits in a high-income group, protect that prediction like the personal data it effectively is. Second, build purpose and control into the AI system itself, so data collected to improve recommendations cannot later train a credit-scoring model without fresh consent. Governance built into the technology, he argued, not written only into a policy document.

The Agentic Problem: Can You Explain Why the AI Decided That?

The second half of the webinar moved from what AI knows to what AI does. Enterprises are shifting from assistants that answer questions to agents that take actions: an agent that checks an order, reads the fraud score, applies the return policy, and issues a refund with no human approval. Pal's scenario for the Data Protection Board was pointed out: a customer speaks to your AI, and afterwards their loyalty tier is downgraded. The Board asks you to explain why. Can you?

Traditional audit assumes a clean chain: a user clicks, a system logs an API call, and you read the log. Agentic AI breaks that in three ways he named. Decisions become multi-step chains across several agents, sometimes spanning vendors. Agents act rather than just read, occasionally with more permission than the human role they stand in for. And the reasoning behind a decision, the inference, stays inside the model where a log cannot reach it. A log shows the agent accessed the data and made a call. It cannot show what the model weighed to reach the decision.

Animesh, engineering director at Jupiter, answer was to treat privacy as a horizontal layer, the way security teams treat authentication and authorization. At Jupiter Money, he described building an in-house consent service from day one, a data protection officer appointed two years ago, automated deletion scripts, and role-based access that extends to AI agents themselves: every agent gets a token-based role, so it can only reach data for the purpose it is entitled to, exactly as a human employee would. His view on inferred data was that it can be governed the same way as raw data, through tagging and annotation, so the privacy-engineering policy fires whether the data is raw or derived. Building this into an AI governance framework from the start is what keeps an agent auditable, whereas bolting it on afterward does not.

The Third-Party You Cannot See Inside

The third theme was the vendor chain, and here the webinar was blunt about a limit of current practice. An Indian e-commerce company rarely builds its own models. It plugs in a chatbot provider, a cloud host, an analytics engine, and a support-automation vendor, and customer data flows out to all of them. Any one of them might store the conversation, train a model on it, route it to servers outside India, or pass it to a subcontractor the company never sees.

Under DPDP, the company that collected the data stays accountable no matter how far down that chain a breach happens. Pal's reframing of the standard vendor question was the takeaway: stop asking, “Is my vendor secure?” and start asking, “Is my vendor's AI accountable to me and through me to my customer?” That means AI-specific contract clauses (does this vendor plug an AI model into our data, and will they notify us if that model is breached?) and assessing risk by data exposure rather than vendor brand size, because a small partner with weak controls can carry more risk than a large one. This is the point where classic third-party risk management has to extend from software vendors to the models those vendors quietly run, and where vendor contracts become the biggest blind spot in an AI supply chain.

Mishra was honest about the ceiling here. Inside your own boundary, observability, audit logs, and policy enforcement are an engineering problem you can solve. Once data leaves for a partner, contracts and audit rights give some assurance, but you can never be fully certain what a vendor does with it. A vendor can show data deleted from its primary store while copies persist in a cache or a vector database you will never see. Both speakers named the same blind spot when asked to pick one: the loss of visibility once data enters the AI ecosystem.

What an AI Governance Framework Has to Cover Now

Pulling the webinar together, the host summarised four things that have to be engineered in from the start rather than patched on later: protect the data, protect the inference, limit the purpose, and instrument all of it for visibility. Translated into a framework, that means a few concrete commitments.

Classify inferred and derived data as personal data, with the same tagging, protection, and erasure obligations as data a customer handed over directly. Extend purpose limitation to model outputs, so data gathered for one purpose cannot silently train a model for another. Give AI agents scoped, role-based identities so an agent can only touch data for its entitled purpose. Build an audit trail that records what an agent did and, as far as the technology allows, the data and reasoning behind the decision. And extend vendor due diligence to the models inside your suppliers, with AI-specific contract terms and continuous monitoring. A structured 90-day path to operationalising DPDP is a useful way to sequence this without trying to do everything at once.

The through-line is continuous visibility. As both speakers stressed, AI cannot be audited once a year, because models drift, user behaviour changes, and prompts evolve. Governance is a process, with a human in the loop, not a launch-day checkbox.

Conclusion

AI governance under DPDP is no longer just about protecting the data customers provide. It is about governing what AI infers, how autonomous systems make decisions, and where customer data travels once it leaves your organisation. As AI becomes embedded across enterprise operations, inferred data, agentic workflows, and third-party models create new accountability obligations that traditional privacy programmes were never designed to manage. The organisations that will be ready for May 2027 are the ones building continuous visibility into their AI ecosystem not only over the data that goes in but also the decisions, outputs, and downstream processing that follow.

Privy by IDfy helps enterprises operationalise that governance. Data Compass discovers and classifies personal data across structured, unstructured, vector, and embedding stores where inferred data often resides, while AI Compliance Co-pilot continuously monitors AI-driven data flows, purpose drift, and vendor risk, connecting them to consent, rights management, and a unified audit trail. Together, they enable organisations to demonstrate what their AI knows, why an automated decision was made, and how accountability is maintained across every system, agent, and third party.

To see how Privy by IDfy helps build an AI governance framework that is ready for DPDP, write to shivani@idfy.com to schedule a demo.

FAQ’s

What is inferred data under the DPDP Act? 

Inferred data is information an AI creates about a person by combining signals they did share, such as predicting income bracket, life stage, health risk, or financial stress from browsing, purchase, and payment history. The person never disclosed it directly, and it typically is not covered by any consent notice, which is why it sits in a governance gap.

Does deleting a customer's record delete what the AI inferred about them? 

Usually not. Deleting the input record removes it from customer systems, but inferences already drawn often persist in feature stores, vector databases, and fine-tuning datasets that are not in the standard erasure workflow. Closing that gap requires classifying inferred data as personal data and including those stores in the inventory.

Why is agentic AI a DPDP audit problem? 

Autonomous agents make multi-step decisions, act rather than just read, and hold their reasoning inside the model. Logs can show what an agent did but not the inference behind it, so a company may struggle to explain to the Data Protection Board why an AI made a specific decision about a customer.

How should DPDP vendor risk management change for AI? 

Move from asking whether a vendor is secure to whether a vendor's AI is accountable to you. Add AI-specific contract clauses, require notification if a vendor plugs a model into your data or that model is breached, and assess risk by data exposure rather than vendor size.

Can an AI governance framework be added after a system is built? 

The webinar's consistent answer was no. Purpose limitation, scoped agent identities, inference classification, and audit trails have to be engineered in from the start, because patching governance onto a live system leaves exactly the gaps a regulator looks for.

Conclusion

The uncomfortable centre of the webinar was that most Indian enterprises are not yet ready to govern inferred data, not because they are careless but because the tools, frameworks, and regulatory guidance have not caught up with what AI can now do. The old governance question, what data do we hold and is it protected, no longer covers the risk. The new question is what the AI knows, why an autonomous system decided to act, and whether that reasoning survives regulatory scrutiny. An AI governance framework built only around inputs will keep missing the outputs, and the outputs are where DPDP now bites.

To see how Privy by IDfy classifies inferred data, connects it to consent and erasure, and keeps an audit trail an agent's decisions be traced through, write to shivani@idfy.com.