India's First DPDP Penalty Won't Come From a Hack. It Will Come From an AI Workflow.

The Real Shape of a DPDP Violation

The first major DPDP penalty in India will probably not come from a hacker. It will not be a ransomware attack or a database leak. Those things will happen, of course. But on looking closely at how organisations are actually using AI today, the interesting risks are much less dramatic

Somewhere inside an enterprise, an employee is going to paste customer information into an AI tool because they are trying to do their job faster. That is it. No criminal intent. No sophisticated attack. No shadowy actor sitting behind monitors. Just an employee on a crunch trying hard to summarise customer feedback, draft email responses, analyse a spreadsheet, or generate a report before the end of the day.

The stakes are real. Penalties under the DPDP Act can reach ₹250 crore per violation, and they stack. A single incident involving a security failure and a missed breach notification could expose an organisation to ₹450 crore in combined penalties. Full enforcement under Phase 3 begins in May 2027. Yet according to an EY India survey, 83% of Indian organisations have not yet begun comprehensive DPDP implementation. That gap is where the risk lives.

The thing about privacy incidents is that we still imagine them as moments when somebody breaks into a system. Increasingly, that is not what they look like. The data is leaving through the door, often with good intentions. An employee uploads a spreadsheet containing customer names and phone numbers to generate a cohort analysis. A support executive asks ChatGPT to summarise a complaint. A recruiter uses an AI screening tool to rank candidates.

Nobody thinks of these actions as privacy events. Most people would describe them as productive. The problem is that privacy laws do not care whether a processing activity feels productive. They care whether it was lawful. That distinction sits at the heart of the challenge India now faces as AI and data privacy in India collide with the Digital Personal Data Protection Act.

How AI Breaks the Old Consent Model

For years, organisations treated consent as something to be collected at the beginning of a customer relationship and largely forgotten afterwards. A checkbox. A policy link. A paragraph in a terms-of-service document. Once obtained, consent became a record sitting somewhere in the background while the business moved on. AI has exposed how fragile that approach really is. The value of AI comes from finding uses for existing data. That is what makes it powerful. A dataset collected for one purpose suddenly becomes useful for another. Customer interactions become training material. Transaction histories become prediction engines. Support tickets become inputs into automated decision-making systems.

Here, the conversation moves beyond technology and into governance. Most enterprises have spent a lot of years investing into systems that help them collect, store, analyse and monetise data effectively. Almost overnight, AI increases the value of those datasets because once the information has served the organization, it can serve dozens of other functions and use cases. The temptation is obvious. If historical customer interactions can improve service quality, automate operations, reduce costs, or generate new insights, organisations naturally want to use them.

The challenge is that DPDP compliance in India does not disappear simply because a new technology creates a compelling business case. The fact that data can be used does not automatically mean it should be used. Effective consent management under DPDP requires organisations to ask a harder question: does the consent already collected actually cover what the AI system is now doing with that data?

Many organisations are discovering that their greatest AI challenge is not model selection or technical capability. It is understanding whether the data they already possess can legally and ethically support the new purposes they have in mind.

Purpose Limitation: The Question AI Forces You to Answer

The data remains the same. The purpose changes. That is where privacy teams start getting uncomfortable.

Imagine a bank that collected customer transaction data three years ago for account management. Today, that same data is being used to improve an AI-powered risk model. The question is not whether the model is effective. The question is whether the customer ever agreed to that use in the first place.

Purpose limitation under DPDP is one of the Act's most operationally demanding requirements. Data Fiduciary obligations under AI-driven systems require that personal data be used only for the specific, clearly stated purpose for which consent was obtained. When AI repurposes existing datasets, it almost always creates a problem, whether or not anyone in the organisation recognises it as one.

Privacy laws have a habit of reducing technological debates to surprisingly simple questions. Did the individual know? Were they informed? Did they agree? Can you prove it? Those questions become much harder to answer when AI enters the picture. DPDP Rules 2025 make clear that the lawfulness of processing is tied to the original consent record. If that record does not cover a new AI use case, the processing is not lawful, regardless of how valuable the output is.

The Deletion Problem: What "Machine Unlearning" Actually Means

The challenge does not end with consent. It gets messier when you start talking about deletion. Most people understand data deletion intuitively. If a company stores information about you and you ask them to remove it, they should remove it. That is how filing cabinets work. That is how databases are supposed to work. AI models do not behave like filing cabinets. Once personal information contributes to a model's training, its influence becomes distributed throughout the system. It is no longer sitting neatly inside a record waiting to be deleted. Researchers call the proposed solution "machine unlearning." Despite years of work, it remains difficult, expensive, and largely unproven at enterprise scale.

Machine unlearning under DPDP is not a theoretical problem. The Act gives Data Principals the right to erasure. The technology responds by asking what erasure actually means when the data has already shaped a model's weights and outputs. That conversation is still being resolved. Meanwhile, Indian enterprises are adopting AI faster than governance programs can keep up with, and most have no documented answer to the question of what happens when a customer exercises their right to deletion and their data has already been used in an AI workflow.

Why Most Enterprises Can't Answer a Regulator's Questions Today

That is what makes this moment unusual. Most regulatory transitions happen before adoption. Rules are introduced, organisations prepare, and then technology scales. In India's case, AI has already arrived. Generative AI usage in India is among the highest globally. Employees are experimenting with tools every day. Vendors are embedding AI into products that companies already use. Workflows are changing faster than enterprise AI governance in India can keep up.

The result is that many organisations do not actually know where AI is touching data. That is a visibility problem before it is a DPDP penalty problem. Ask an enterprise which AI systems currently process customer information, and you will often get a surprisingly incomplete answer. They will tell you about the approved projects. The strategic vendor relationships. The initiatives that went through procurement. They will struggle with the browser tabs. The free tools. The pilot programs. The features that appeared during a software update and were switched on by default.

Those are often the places where privacy incidents begin. Not with an actor, but with an undocumented workflow. History offers a lesson. Some of the significant GDPR penalties in Europe were not driven by cyberattacks. They were driven by failures of consent and transparency. The issue was not that data had been stolen. The issue was that data had been processed without clarity and accountability. India may follow a similar path. The organisations facing the steepest DPDP compliance risk may not be the ones that suffer the biggest breaches. They may be the ones that cannot explain how personal data moves through their AI ecosystem when a regulator, customer, or Data Principal asks the question.

Because eventually somebody will ask. What data do you hold about me? Where did it come from? Who has access to it? Which AI systems use it? What did I consent to? For most enterprises, those questions still do not have complete answers.

What Compliant AI Governance Actually Looks Like

The organisations approaching this well are not necessarily the ones slowing down AI adoption. They are the ones creating visibility around it. They know which systems are processing personal data. They understand how information moves between applications, vendors, and teams. They can connect a consent record to a processing activity and explain why a particular dataset is being used in the first place. That sounds like a compliance exercise. It is not. It is slowly becoming an essential component of the business due to the DPDP Rules 2025.

When customers ask how their information was used, when regulators demand evidence, or when boards require confidence that AI does not create unmanaged risk, organizations must respond to these challenges with data, not assumptions. However, the problem is that many organizations are still trying to manage AI using mechanisms created for a completely different technology epoch. Now, privacy, technology, legal, and business teams are facing the same issue.

Conclusion

AI is not purely a technology issue. It is a governance issue, a trust issue, and increasingly, a DPDP compliance issue. The organisations that recognise that connection early will be far better prepared than those that continue treating these disciplines as separate conversations.

The challenge is that most enterprises are still trying to govern AI with processes designed for a very different era of technology. Visibility into where personal data moves, which AI systems touch it, and whether the right consent exists for each use case, these are not questions most organisations can answer today. That is exactly the problem Privy by IDfy is built to solve, giving enterprises the data discovery, consent governance, and compliance infrastructure they need to navigate DPDP with confidence.

When the first DPDP penalty finally arrives, there is a good chance it will not start with a breach notification. It will start with an ordinary workflow that was not thought of as a privacy issue at all. If your organisation is navigating DPDP readiness, AI governance, or consent management, we'd be happy to exchange perspectives. Reach out to us at shivani@idfy.com.

FAQ's

What is the DPDP Act, and how does it apply to AI?
The Digital Personal Data Protection Act is India's data protection law, enacted in August 2023 with Rules notified in November 2025. It applies to any organisation that processes the digital personal data of Indian residents. AI systems that use, analyse, or process personal data fall squarely within its scope, regardless of whether the AI use case was the original purpose for which data was collected.

Can employees using ChatGPT create DPDP compliance risk?
Yes. When an employee inputs customer or employee personal data into an external AI tool, that constitutes processing under the DPDP Act. If the original consent collected from the Data Principal does not cover that use, the processing is unlawful. This is one of the most common and least-tracked compliance risks enterprises currently face.

What is the purpose limitation under the DPDP Rules 2025?
Purpose limitation requires that personal data be used only for the specific purpose for which consent was obtained. Under DPDP Rules 2025, organisations cannot repurpose data for new AI use cases simply because the data is already in their possession. A new purpose requires fresh consent or a valid legal basis under the Act.

What is machine unlearning and why does it matter for DPDP?
Machine unlearning refers to the technical process of removing the influence of a specific individual's data from an already-trained AI model. It matters for DPDP because the Act grants Data Principals the right to erasure. When an AI model has been trained on personal data, fulfilling that right becomes very technically difficult that, in many cases, is currently infeasible at an enterprise scale.

What obligations do Data Fiduciaries have for AI-processed data?
Data Fiduciaries must make sure that personal data that has been processed by AI systems is only collected from data subjects through valid consent, not used for any other purpose, protected by proportional security measures, and is erased if so requested. Moreover, they must inform the Data Protection Board of the incident without unnecessary delay.