What does a private ChatGPT on company data actually mean?
A private ChatGPT on company data is an assistant that answers questions from your own documents and systems under rules your organisation controls: who may see which source, where prompts are processed, what is logged and for how long. "ChatGPT" is used here as shorthand. The same questions apply to any general-purpose assistant or language model you might build on.
Whatever the vendor calls them, these systems have three parts. The model writes the answer. The retrieval layer finds the relevant passages in your documents at the moment a question is asked, an approach known as retrieval-augmented generation (RAG). The control layer handles sign-in, permissions, logging, retention and the source shown to the user. Most buying decisions are really decisions about the second and third parts. Models can be swapped to a degree; the way your data is connected, filtered and governed cannot.
It also helps to separate two needs that usually arrive bundled together. The first is a general writing and analysis assistant for employees: drafting an email, summarising a report someone pastes in, translating a paragraph. The second is an assistant that knows your company. It answers "What does our travel policy say about weekend flights?" or "Which customer contracts have a change-of-control clause?" from your own sources and shows where each answer came from. The first is mostly a licensing and usage-policy question. The second is a data project, and it is the subject of most of this guide.
What are the options for an enterprise ChatGPT?
There are three main routes, and many organisations end up combining two of them.
- The enterprise plan of a SaaS assistant. ChatGPT's business plans, Microsoft Copilot, Gemini in Google Workspace and similar products. You buy licences, connect the vendor's connectors to your mail, files and chat, and manage users from an admin console.
- Your own application on a cloud model API. You, or a supplier, build the retrieval and control layers, and the model runs as a managed service in your own cloud subscription, for example as a model deployment in Microsoft Foundry.
- Self-hosted. An open-weight model runs on servers you control, on premises or in a private data centre, together with your own retrieval layer. Nothing leaves your network unless you decide it should.
| Question | SaaS enterprise plan | Own app on a cloud API | Self-hosted |
|---|---|---|---|
| Where are prompts processed? | In the vendor's regions, under its terms | In the region and deployment type you choose | On your own servers |
| How are permissions applied? | Through the vendor's connectors and your existing file permissions | You design the permission filter | You design the permission filter |
| Which sources can it use? | Those the vendor offers connectors for | Any source you can build a connector to | Any source you can build a connector to |
| Control over answers and citations | Limited to product settings | Full | Full |
| Operational burden | Low | Medium | High: hardware, capacity, updates |
| Shape of the cost | Licence per user | Build cost plus usage | Build cost plus hardware and operations |
| Fits best when | Data lives in one office suite and permissions are tidy | Answers must come from business systems or follow your own quality rules | Data may not leave the organisation, by law, contract or policy |
For the details that matter, read the vendor's documentation rather than its marketing page. Microsoft's data, privacy and security page for Copilot states that prompts, responses and data accessed through Microsoft Graph are not used to train foundation models, and that Copilot only surfaces organisational data to which the individual user has at least view permission. The same page states that customers outside the EU may have their queries processed in the US, the EU or other regions, which is a data transfer question for any company in Türkiye. Google's privacy hub for generative AI in Workspace makes comparable commitments about model training and existing access controls. For the cloud API route, Microsoft's data privacy page for models sold by Azure explains that prompts are processed within the geography you specify unless you choose a Global or Data Zone deployment type, and that content flagged by abuse monitoring may be stored for human review unless you have been approved for modified abuse monitoring. These are exactly the points your data protection review will ask about.
Is RAG over company data different from uploading files to ChatGPT?
Yes. Uploading a file to a chat gives the model one document for one conversation; RAG connects the assistant to your sources permanently, keeps them current and decides, for every question, which passages this particular user may see.
A RAG pipeline has a handful of stages, and each is a place where quality is won or lost:
- Connectors read from file shares, SharePoint, the document management system, the wiki, the ticketing system or the ERP, together with each document's permissions.
- Parsing turns PDFs, scans, spreadsheets and slides into clean text while keeping headings and tables intact. Scanned documents need OCR first.
- Indexing splits documents into passages and stores them for both keyword and semantic search, so that exact terms such as product codes and clause numbers are not lost.
- Permission filtering removes, inside the search itself, every passage the user cannot open in the source system.
- Generation gives the model only the selected passages and instructs it to answer from them.
- Citation links each statement to the passage it came from. When there is no supporting passage, the assistant says it could not find the answer instead of guessing.
The citation step is what makes an assistant usable for real work, because people can check an answer in seconds instead of having to trust it. Caseon, the LegalTech product Hatson Tech is building for law firms' due diligence and contract review, is one example of this principle: every answer is shown with the document excerpt it is based on, and the lawyer reviews the sources and approves the work. The Caseon case study describes the approach. The same design choice applies to an HR policy assistant or a technical support assistant. If users cannot see the source, they either stop using the tool or, worse, stop checking it.
Which KVKK questions must be answered before rollout?
At minimum: who is the data controller and who is the processor, where personal data is processed and on what basis it is transferred, what employees and customers are told, what is stored and for how long, and whether the vendor uses your data for anything beyond providing the service.
The Personal Data Protection Authority's guide on generative AI and the protection of personal data, published in November 2025, applies Law No. 6698 to these systems. Two of its points bear directly on a company assistant. First, when an organisation in Türkiye uses a generative AI system through a service provider established abroad and personal data is transferred abroad as a result, the transfer must comply with Article 9 of the Law and the 2024 regulation on transfers abroad. Second, the guide recommends separate, explicit information notices for processing to use the system and for processing to develop or improve models. Article 9 of Law No. 6698 permits transfers on the basis of an adequacy decision, or one of the appropriate safeguards listed in the article, such as a standard contract announced by the Board, which must be notified to the Authority within five business days of signing. Only where neither is available may data be transferred under the incidental exceptions in paragraph 6.
Work through the checklist below with your legal and information security teams. It is not legal advice; it is the list of questions that should have written answers before the first department goes live.
| Question | Why it matters | What to ask for |
|---|---|---|
| Which personal data will the assistant see? | Contracts, HR files and support tickets often contain names, ID numbers and health information | A source inventory with a personal data column |
| Who is the controller and who is the processor? | It decides who answers to the Authority and to data subjects | The data processing agreement and its annexes |
| Where are prompts, indexes and logs processed and stored? | Processing abroad brings Article 9 into play | Regions, deployment type and the list of subprocessors |
| What is the transfer mechanism? | Adequacy decision, appropriate safeguard or incidental exception, each with its own conditions | The signed standard contract and proof of notification, or the documented alternative |
| Does the vendor train on or review your data? | Model training and human review are separate processing purposes | The contract clause rather than the marketing page; abuse monitoring settings |
| How long are prompts, answers and logs kept? | Retention has to follow the purpose and your retention and destruction policy | Configurable retention, a deletion procedure, the contents of the logs |
| Have employees and customers been informed? | The information obligation in Article 10 | Updated notices, and a separate notice if data is used to improve models |
| Are special categories of data excluded or protected? | Health, biometric and similar data carry stricter conditions | Excluded folders, masking rules, additional measures |
| How are data subject requests handled? | A person may ask what is held about them, or ask for deletion | Search and deletion across the index and the logs |
If the answer to "where is it processed?" is "abroad" and the transfer basis is unclear, that is not a reason to drop the project. It is a reason to choose the architecture deliberately: some sources can go to a cloud service, others can stay with a model on your own servers. Our guide to in-house and sovereign AI covers the deployment side in more depth.
How do permissions work, and why do they decide the project?
A company assistant may only use passages that the person asking is allowed to open in the source system. Everything else follows from this rule, and in a real project much of the effort goes into making it true.
With a SaaS assistant, permissions come from your existing file shares. That is convenient, but it has a consequence: every folder that was shared with "everyone in the company" years ago becomes searchable in plain language. Before you switch a SaaS assistant on, review broad sharing links and sensitive folders such as payroll and legal, and check whether sensitivity labels are in use. With your own application, the permission filter is something you build: each passage carries the identifiers of the users and groups allowed to read it, and the search applies the user's current groups before ranking. We describe the details, along with a leakage test plan, in permission-aware retrieval for enterprise RAG.
OWASP's 2025 risk list for LLM applications names both failure modes. Its entry on vector and embedding weaknesses covers context leaking between users who share a vector database and recommends permission-aware vector stores with strict logical partitioning of datasets. Its entry on sensitive information disclosure recommends least-privilege access to data sources and clear policies on retention and deletion. A practical test before go-live: ask the same question as an ordinary employee, a manager and a guest account, and compare the raw search results, not only the final answers.
Which option fits our situation?
Answer the questions below honestly. The pattern of answers usually points to one route, or to a deliberate combination.
| Question | If yes, lean towards |
|---|---|
| Are most of the relevant documents in one office suite, with permissions you trust? | A SaaS enterprise plan |
| Is the main need general writing help rather than company knowledge? | A SaaS enterprise plan plus a usage policy |
| Must answers come from the ERP, CRM, document management system or a custom database? | Your own application on a cloud API |
| Do answers need a specific citation format, quality checks in Turkish or an approval step? | Your own application |
| Do law, a customer contract or internal policy forbid data leaving the organisation? | Self-hosted, at least for those sources |
| Is the transfer basis for a foreign cloud service still unresolved? | Self-hosted, or a hybrid while it is resolved |
| Should the assistant also act, for example create records or open requests? | Your own application with an agent layer and human approval |
| Is there nobody to run servers and update models? | Avoid self-hosting unless a supplier operates it for you |
If the table leaves you between two routes, ask who will notice when an answer is wrong. If nobody checks, citations and a regular sample review matter more than the choice of model.
Hypothetical example: one company, two assistants
This is a hypothetical example, not a client story. A manufacturer with around 400 employees asks for "a ChatGPT that knows our company". The first conversation separates three needs:
- Office staff want help drafting emails and summarising reports. The material is mostly not confidential.
- Production engineers want answers from quality procedures and work instructions, which sit in SharePoint with tidy departmental permissions.
- The sales team wants to ask questions across customer contracts in the document management system. Several contracts contain confidentiality clauses that restrict processing outside the company, and the signature pages contain personal data.
Running each need through the table gives a clear split. Office drafting and the procedures assistant go to a SaaS enterprise plan, after a permission clean-up of the procedure library and a short usage policy that tells staff what not to paste in. The contracts assistant becomes a separate application on an open-weight model hosted on the company's own servers, with a permission filter mapped from the document management system and a citation shown for every answer. The legal team signs off the KVKK checklist for the SaaS plan, including the processing regions and the transfer mechanism; for contracts, the self-hosted assistant removes the transfer question altogether. The pilot starts with one sales team and 50 real questions collected from its members, each written down with the expected answer and the clause it should cite.
What decided the architecture was not any model's quality, but where the data sat and what the contracts allowed.
What does a sensible rollout plan look like?
Roll out in stages, each with a clear exit criterion, rather than switching the assistant on for everyone at once.
- Inventory and policy. List the sources, their owners, their permissions and the personal data they hold. Find out which AI tools employees already use, and publish a short acceptable-use policy.
- Golden question set. Collect real questions from future users, each with the expected answer and its source. This becomes the acceptance test.
- Pilot. One department, one or two sources. Measure whether the right passage is retrieved, whether citations are correct, how often the assistant correctly says it does not know, and whether the permission tests pass.
- Expansion. Add sources and departments one at a time, and re-run the question set after each change.
- Training. Show users how to read citations and how to report a wrong answer. Adoption depends on this more than on the model.
- Operations. Monitor index freshness, permission sync, cost per answered question and user feedback. Re-run the question set whenever the model version changes.
Common mistakes
- Buying licences before cleaning up who can see which folder.
- Judging the system on a vendor demo instead of on your own question set.
- Accepting answers without citations for work that has to be checked.
- Treating the vendor's marketing page as if it were the data processing agreement.
- Indexing everything "just in case", including archives full of personal data with no business purpose.
- Forgetting the logs: prompts and answers are personal data too, with their own retention period.
How Hatson Tech approaches this
We build assistants on your own documents and systems, in the cloud or on your own servers, as part of our enterprise RAG and semantic search solutions. Every answer shows its source, permissions are enforced in the search, and a golden question set is part of the acceptance criteria. For the regulatory background, see our KVKK generative AI guide. The first meeting and the discovery work for a software project are free. A useful way to prepare is to bring a list of the sources you have in mind and a few questions your teams ask every week.