How to stop company data from leaking into ChatGPT

The Dutch Alert Online 2025 study, run by the Ministry of Economic Affairs, found that 49% of Dutch employees say their organization uses generative AI tools. Only 26% say their organization has clear guidelines. So almost three-quarters use AI without any real framework for safe use.

You see it on the work floor. A colleague asks ChatGPT to draft an email, has Copilot review a piece of code, or pastes a customer list into Gemini for a quick summary. Useful, and almost never ill-intentioned. Yet a single paste is enough to push personal data, credentials, or confidential business information outside your organization.

This article lines up the numbers and shows what goes wrong when sensitive data leaks into an AI chatbot. We look at what the GDPR, NIS2, and the EU AI Act say about this use. After that, you’ll read how to prevent it in practice, without banning AI altogether.

What goes wrong when data leaks into an AI chatbot

What you type into a chatbot doesn’t stay in your browser. The text goes to the provider’s servers, often outside the European Economic Area. There it can be stored and, depending on the plan and settings, used to train the model further. With the free consumer versions, that last part is the rule rather than the exception. That’s why the Dutch data protection authority (AP) warns against entering trade secrets and personal data into chatbots. With non-western chatbots like DeepSeek, data can even be processed outside the EU, including in China.

This isn’t a theoretical risk. The Dutch supervisory authority received dozens of AI-related data breach reports in 2024 and 2025, with more reports in 2025 than in 2024. A concrete example comes from the municipality of Eindhoven, which announced in December 2025 that employees had uploaded CVs, youth-care files, and internal reports to public AI chatbots.

Once it’s sent, you don’t get that data back. You can’t undo a prompt. And because nobody signed a data processing agreement with that provider in advance, your organization formally doesn’t even know where the data ends up or how long it’s kept. In the worst case, that’s a data breach you have to report.

Which data you should never drop into a prompt

Not everything is equally sensitive, but a few categories simply don’t belong in an AI chatbot:

• Personal data, such as a name combined with a national ID, address, date of birth, or phone number.
• Customer and patient files, quotes, and other documents with identifiable data about other people.
• Credentials and API keys from providers like OpenAI, AWS, GitHub, or Stripe.
• Source code and trade secrets that aren’t public.
• Financial data, such as IBANs, credit card numbers, and internal revenue figures.
• Special category data about health, religion, or criminal history.

An example makes it concrete. This innocent-looking prompt already contains five pieces of data you’d rather keep in-house:

Write a polite email to Jan de Vries (j.devries@acme.nl) about his
quote OFF-2026-0914. His national ID is 123456782 and IBAN NL91ABNA0417164300.

Name, email address, reference, national ID, and IBAN all travel to the chatbot’s server in one go. The employee only wanted a quick email.

How big is the problem, really?

Usage is high, and policy hasn’t caught up with it. Research by KPMG (2025) found that nearly seven in ten Dutch people who use AI do so at work, while many organizations have no clear AI policy. International research by Software AG (2024) among 6,000 knowledge workers shows that 75% use AI tools, and that 46% keep doing so even when their employer explicitly forbids it. A ban alone doesn’t stop people.

We ran our own market research among thirty IT managers, security officers, and compliance staff, across both the private and public sector and from small business to large enterprise. The results show that organizations lack control:

Only 10% have a technical control such as a DLP tool. And where one exists, it often works poorly. Just 10% of respondents say their DLP solution functions properly. The international picture is no better. According to Proofpoint’s Data Loss Landscape Report 2024, only 38% of organizations consider their own DLP program mature. The best-known option, Microsoft Purview, costs at least €12 per user per month and requires a Microsoft 365 license, which puts it out of reach for many smaller organizations.

What the GDPR, NIS2, and EU AI Act say about it

Sharing personal data with an AI service is processing under the GDPR. That requires a legal basis, you have to respect data minimization, and a data processing agreement belongs with it. A free ChatGPT account has no such agreement, and the data often goes outside the EEA.

Since February 2025, the EU AI Act imposes a duty of AI literacy. Organizations have to make sure their people know enough to use AI responsibly. Knowing which data doesn’t belong in a prompt falls squarely under that.

For companies covered by NIS2, risk management is added on top. Uncontrolled use of AI tools, also known as shadow AI, is a risk you have to map and address. You can’t manage something you have no visibility into, and the numbers above show that visibility is usually missing.

How to prevent it in practice

Writing a policy is a good start, but a document in a folder won’t stop anyone at the moment things go wrong. Awareness training helps, yet the temptation to quickly paste something stays strong. The fact that 46% carry on despite a ban shows you need something that steps in at the moment itself.

You can do that with detection that runs locally in the browser. The prompt is scanned before it’s sent. If it contains sensitive information, the employee gets a warning and the choice to redact or replace the data. With pseudonymization, fictional data takes its place, so the chatbot still gives a usable answer without any real data being sent.

Redactprompt is built for exactly this. Detection and redaction happen entirely locally, so no data goes to external servers. You can set what’s allowed per chatbot, and see in a dashboard where most detections come from, without exposing the content of the prompts themselves.

No measure catches everything. But by checking at the moment of pasting instead of finding out afterwards that something went wrong, you reduce the risk considerably. That saves a lot of hassle, and in the best case a reportable data breach.

Want to see how this works for your organization? You can try Redactprompt for free with no strings attached, or get in touch to discuss the options for your team.