Private AI: how to give your team the power of ChatGPT without sending your data out

    Employees paste company data into ChatGPT every day. What leaks, what the account tier changes, and how to run private AI on your own servers.

    Clara Lehnert
    Published October 1, 2026
    7 min read

    Your team probably uses ChatGPT already. Someone pastes in a client email to get a quick reply, someone else drops in a contract they don't have time to read before a meeting. Every one of those prompts lands on an outside provider's servers. Private AI gives people the same kind of help, except the whole system runs on infrastructure you control, from the models to the company documents they draw on. Nothing they type leaves the company.

    In its 2025 report, the data security company Cyberhaven found that 34.8% of the corporate data employees put into AI tools is sensitive. Two years earlier, it was 10.7%. A global study led by the University of Melbourne with KPMG, covering more than 48,000 people in 47 countries, points the same way: almost half of employees use AI in ways that break company policy, including uploading sensitive company information into free public tools like ChatGPT.

    Share of sensitive data in AI tools rising from 10.7% in 2023 to 34.8% in 2025

    Is ChatGPT safe for business data?

    It depends on the account. On ChatGPT Business and ChatGPT Enterprise, OpenAI does not use your data to train its models by default. On the personal Free, Plus and Pro plans many employees use, sharing for training is switched on until each person turns it off.

    That second group is the risky one. An employee who pastes a contract into a personal account has made a choice nobody in IT can see. Switching the setting off later only covers new conversations, not the ones before.

    Even on business plans, the data still leaves your infrastructure. It is processed and stored on the provider's servers, under the provider's retention rules and the laws of the country where those servers sit. For plenty of companies, that's an acceptable trade. It isn't for an accounting or law firm bound by professional secrecy, or for a hospital with patient records. Banks, insurers and defense suppliers usually say no too. So does a chip maker sitting on designs nobody outside should see yet.

    What actually leaks, and how?

    Mostly ordinary work documents, pasted in to save time. A contract someone wants summarized before a call. A bug a developer still can't find at 6 pm.

    Even security professionals do it. Cisco's 2025 Data Privacy Benchmark Study surveyed more than 2,600 privacy and security professionals in 12 countries. 64% of them worry about sharing sensitive information by accident. Close to half admitted putting personal or non-public data into AI tools anyway.

    The best-known case dates from 2023. Engineers at Samsung Electronics pasted internal source code into ChatGPT to get help with their work. A few weeks later, Samsung banned generative AI tools on company computers, phones and internal networks. Nobody meant any harm. They were trying to get their work done faster.

    What are your options?

    Most companies end up choosing between three. You can ban AI tools, buy an enterprise subscription, or run private AI on your own hardware. What separates them is where the data goes and how you pay for it.

    Ban it. Many companies tried this first. People kept using AI on their phones and personal laptops, and the company lost the little visibility it had. We wrote about what those workarounds tell you in our article on shadow AI.

    Buy an enterprise subscription. ChatGPT Enterprise, Microsoft Copilot and similar plans settle the training question and give IT an admin console. The data still goes to the provider, and the bill grows with every seat you add.

    Run private AI. This is the main alternative to ChatGPT Enterprise for companies that can't send data out. The models, the connection to your documents and the access controls all run on servers you own, inside your network. No prompt ever leaves the building. And since nobody bills you per seat or per question, a busy month costs the same as a quiet one. A technical team can build it with open-source tools, or you can use a turnkey private AI platform that arrives with the hardware, the models and the interface already set up.

    Comparison of banning AI tools, an enterprise subscription and private AI on your own hardware

    What does private AI look like in practice?

    A private AI system has three parts: the models, the connection to your own documents, and the controls around them, such as who can use which assistant and a record of every question asked.

    Private AI system with models, documents and controls inside your infrastructure

    From the user's side, a private AI chatbot looks like ChatGPT. People log in with their company account, ask questions, upload documents and get answers, while the model runs on a server in your building and each answer can point back to the internal document it came from.

    At Understand Tech, this is what our secure enterprise AI assistants do on AI-in-a-Box. Each assistant gets its own isolated portal, and people sign in with the company account they already have. Every question ends up in an audit log. If you need it, the whole system runs air-gapped, with no internet connection at all. The details are on our security page.

    Cerfrance Maine-et-Loire, an accounting firm in western France, runs it on a Dell Pro Max GB300 in its own offices. Its teams use it every day, mostly to pull figures out of accounting documents and to check internal procedures and professional doctrine. They also analyze CSV exports with it and draft replies to clients. None of that data leaves the firm, and they have since built a business application of their own on the same system.

    Private AI also goes further than a chatbot. The same platform runs apps built for one specific job. A standards body can use one to turn a long technical specification into test cases. A chip company can let its distributors ask questions about datasheets without opening up its internal documentation. You can see them on our page of AI apps built for your industry.

    Is private AI as good as ChatGPT?

    For most daily work, yes. The top closed models still win on the hardest reasoning problems. But summarizing a report or answering a question about an internal procedure is well within reach of open-weight models such as gpt-oss, Qwen or DeepSeek.

    gpt-oss actually comes from OpenAI. It was released with open weights so companies can run it on their own hardware. Private AI can run an OpenAI model, just not on OpenAI's servers.

    The model also matters less than people expect. What makes answers useful at work is access to the right documents. A private AI assistant that has read your own procedures often answers better than a stronger public model that has never seen them.

    How much does private AI cost?

    Less than most people assume, once more than a small team uses it. Subscriptions charge per seat, every month, so hiring ten people means paying for ten more seats. Private AI runs on fixed capacity. You pay for the hardware and the software, and nobody meters how much your team uses it.

    We measured the difference on our own workloads. When we moved our AI from cloud GPUs to two NVIDIA DGX Spark units, our first-year cost fell by 88%. The full numbers are in our comparison of on-prem AI vs cloud cost.

    How do you roll it out without creating new shadow AI?

    Start with one team. Pick a single use case and the documents it needs, then watch how often people come back to it over the first few weeks.

    The best first use case is one people already handle in ChatGPT today, like summarizing long reports. If the private version answers just as fast and also knows the company's own documents, nobody needs to be told to switch. From there, add teams one at a time, and tell employees plainly which tool is approved for which data.

    Frequently asked questions

    Does ChatGPT Enterprise train on my data?

    No, not by default. OpenAI states that it does not use data from ChatGPT Business, ChatGPT Enterprise or its API to train its models. The data is still processed and stored on OpenAI's servers.

    Can I run ChatGPT itself on-premise?

    No. ChatGPT only runs on OpenAI's infrastructure. You can run gpt-oss, OpenAI's open-weight model, on your own hardware, along with other open-weight models such as Qwen or DeepSeek.

    What is private AI?

    A complete AI system that runs on infrastructure you control: the models, the connection to your own documents, the access controls and the apps people use. In practice it often starts as a ChatGPT-style assistant, sometimes called a private ChatGPT. Nothing you type is sent to an outside AI provider.

    Is private AI GDPR compliant?

    Keeping data on your own servers removes the transfer to a third-party AI provider, which is one of the main GDPR questions. You still need the usual basics: a legal basis for processing, access controls and a retention policy.

    Do we need an AI team to run it?

    Not with a turnkey system. AI-in-a-Box arrives with the models and the interface installed, and goes into production in about 30 minutes.

    Curious about the hardware behind private AI? Have a look at our on-prem LLM server.

    Want to see private AI working on your own documents? Book a demo.

    Sources

    All articles
    Chat with AI Assistant