Shadow AI
When staff paste company data into an AI tool
Entering data into an AI tool you have not approved can be a breach in itself. Find out what went in, where it went and whether it concerns people.
Last reviewed against legislation.gov.uk, National Cyber Security Centre and OWASP GenAI Security Project. Sources
The case
Samsung in 2023
CSO Online reported on a Samsung memo saying that in April 2023 engineers had put internal source code into ChatGPT by mistake [10]. According to the report, the memo gave no detail of the code itself, nor of whether outsiders inspected it [10]. Samsung called its restriction on generative AI temporary, until security measures were ready [10].
Response
Steps after a paste
-
1
Ask the person what was entered, into which tool, on which account and when. Do this without blame, or the next case will be hidden. -
2
Establish whether it included personal data, credentials or confidential material. -
3
Check the tool's settings, retention and training terms for the account used, and ask the supplier what it can delete. -
4
Rotate any credentials that were pasted. -
5
Decide whether this is a reportable personal data breach and record the outcome [1].
Fix
What to change afterwards
Say what may be entered
OWASP's advice is to educate users against entering anything sensitive, and to publish plain rules on how long data is kept and when it is deleted [5]. A written rule beats a verbal one.
Offer an approved route
A ban alone left Samsung describing its own restriction as temporary [10]. Staff will turn to the tools anyway unless a sanctioned alternative exists.
Know what is in use
The NCSC wants leaders to understand how their data and software reach them through suppliers [3]. Start with a list of AI tools in use, including the ones nobody approved.
Keep the first report easy
If owning up to a mistake is awkward, you will hear about it late. Provide a short route to the incident lead and mention it in training.
See the response plan for where this fits.
Keep reading
Read next
Output leaks
What to do when an AI tool reveals one person's data to another, or discloses confidential content in its answers: scoping, containment and fixes.
Read the guide
Examples
Four documented AI-related data exposures, ChatGPT, Samsung, Microsoft and DeepSeek, with the cause of each and the lesson for UK organisations.
Read the guide
What to do
A practical order of work for UK organisations after data leaks through an AI tool: awareness, containment, scoping, reporting, supplier and plan.
Read the guideHow CyPro can help
Need a hand working through this?
Finding out whether an incident reaches you is one job. Checking your own systems, tightening access and briefing the board are several more. CyPro's responders and consultants can take any of them on.
Incident response
If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.
More on cypro.co.uk
24/7 security monitoring
Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.
More on cypro.co.uk
Fractional CISO for the board
A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.
More on cypro.co.uk
Penetration testing
A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.
More on cypro.co.uk
Hardening and segmentation
Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.
More on cypro.co.uk
Vulnerability scanning
Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.
See the scanning service
Talk to a specialist
Worried this affects you?
When data has leaked through an AI tool, its logs or its storage, the early decisions shape what you must report and to whom. An incident response team helps you contain the exposure, establish what was involved and prepare your notifications.