AI Chatbot Privacy — What AI Chatbots Remember About You
What you type into an AI chatbot does not necessarily disappear when the conversation ends. Two real 2023 incidents show what that can mean in practice.
In the spring of 2023, engineers at Samsung's semiconductor division pasted confidential source code into ChatGPT on at least three separate occasions in under three weeks, hoping it would help them debug faster or summarize notes. Once submitted, that code had been sent to a third-party service outside Samsung's internal environment — no longer something the company's own systems controlled, regardless of how useful the tool's response turned out to be.
That is the quiet risk underneath a genuinely useful tool: a conversation with a cloud-hosted AI chatbot should not automatically be treated like a private scratch pad. It is a request sent to a company's servers, and what happens to it afterward depends on that company's systems and policies, not yours.
The two incidents happened for entirely different reasons, which is part of what makes them useful to look at together. The OpenAI incident was a technical bug, not an attack — a caching error in a shared library briefly mixed up data between simultaneous user sessions, which is a category of software fault that can affect any service handling large amounts of concurrent traffic. OpenAI published a detailed explanation of the bug and the fix soon after, which is part of why the incident is well documented.
The Samsung case was different in nature: no bug was involved. Employees, apparently trying to use ChatGPT to help debug code or summarize notes, pasted genuinely confidential company material directly into the conversation. Once submitted, that information had left the company's own systems and become part of a conversation with a third-party service, with retention and usage governed by that service's own policies rather than Samsung's internal ones.
These two incidents point to the same underlying fact from different directions: a conversation with a cloud-hosted AI chatbot should not automatically be treated like a private scratch pad. It is a request sent to a company's servers, processed and often stored there, and — depending on the service and its settings — potentially used to improve future versions of the underlying model. Even without any bug or mistake at all, that alone means anything typed into a chatbot should be treated the way you would treat information sent to any external company, not the way you would treat a private note to yourself.
The Samsung case in particular is a reminder that the risk is not only technical. It is also about habit: AI chatbots are convenient enough that it is easy to paste in something sensitive without pausing to consider where it is actually going, especially under the same kind of everyday time pressure that makes other social engineering attacks effective.
In the time since these incidents, it has become increasingly common for organizations to publish explicit policies on which AI tools employees may use for work, and what categories of information may never be entered into them. That shift mirrors an older one: the same way organizations eventually wrote clear policies about email attachments and USB drives, AI chatbots are now going through the same process of being treated as a genuine data-handling channel, not just a productivity convenience.
None of this means AI chatbots should be avoided entirely — it means treating them the same way you would treat any other service that processes what you send it.
- Never paste confidential information — source code, internal documents, personal data, passwords — into a public AI chatbot.
- Review the current data controls, retention policy, and model-training settings of the specific AI tool you're using — these vary by service and change over time, and turning off training use does not necessarily mean your data is deleted immediately or not retained at all.
- Follow your employer's specific policy on AI tool use for work-related tasks, since many organizations now set clear rules about what can and cannot be shared this way.
None of this means treating every AI chatbot as unsafe to use. It means treating it the way you would treat any other outside company you're sending information to — fine for most things, not the place for anything you would not want to see outside your own control.
A useful reminder
A chatbot conversation is a request to a company's servers, not a private note. Treat what you type accordingly, the same way you would think twice before emailing sensitive information to an unfamiliar address.
The Takeaways
Before pasting anything work-related into a chatbot, ask whether you'd be comfortable emailing the same thing to an outside company — because that's effectively what you're doing.
Look up the current data controls, retention policy, and model-training settings for the specific AI service you use — don't assume; these details vary by provider and change over time.
Check your employer's AI-use policy before using any chatbot for work tasks — many organizations now have one specifically because of cases like these.
If you would like a visual explanation, continue with the accompanying CyberBlink video.
AI Chatbot Privacy — What AI Chatbots Remember About You
What actually happens to a chatbot conversation after you send it, based on two real 2023 incidents — one a technical bug, one a habit worth changing.