RAG & Enterprise Semantic Search

What is RAG?

RAG (retrieval-augmented generation) is a method in which a language model finds and reads the relevant documents before answering. The model answers from the company documents retrieved at that moment rather than from general knowledge it memorised, and it ties each claim to a source. The approach was defined in the 2020 paper by Lewis et al., and it underpins most enterprise AI assistants today.

Three advantages matter most to companies. When a document changes, the answer changes too, without retraining the model. Because answers arrive with their sources, users can check them, which noticeably reduces the risk of hallucination. And because access rules can be applied during retrieval, each person only gets answers from documents they are allowed to read. From Ankara, Hatson Tech builds document-based assistants and enterprise semantic search systems for organisations across Türkiye.

How does RAG architecture work?

A good RAG system is not a single model but a connected pipeline. Most of the answer quality is decided before the model is involved:

1. Ingestion: Text, tables and metadata are extracted from PDFs, Word files, wikis, email and business systems; scanned documents go through OCR.
2. Chunking: Documents are split into pieces that keep their meaning. Not losing headings and tables is critical; see our article on chunking at scale.
3. Embeddings and a vector index: Each chunk becomes a vector that represents its meaning and is indexed. See our articles on tokens, embeddings and vectors and on vector databases.
4. Hybrid search and reranking: Semantic similarity and keyword matching work together, so exact matches such as product codes and clause numbers are not lost; the most relevant passages are then reranked (hybrid search and reranking).
5. Permission filter: A passage the user cannot read in the source system never reaches the model (permission-aware retrieval).
6. Context and generation: The selected passages are placed into a limited context budget and the model answers only from that evidence (context engineering).
7. Citation and verification: Every claim links to the passage that supports it; without evidence, the system says it could not find an answer instead of inventing one (verifying citations in RAG answers).

Semantic search, RAG or fine-tuning?

Not every need calls for RAG. "Semantic search vs RAG" is a common question: semantic search finds the right document, while RAG writes a sourced answer from the documents it finds. The table summarises the criteria we use:

Need	Better fit
Employees should find the right document quickly	Semantic search
Questions should be answered with sources	RAG
The model's tone, format or domain language must change	Fine-tuning
Work happens on a single, very long document	A long context window
Questions are about relationships between people, organisations and events	GraphRAG (knowledge graph + RAG)
An answer needs several tools and steps	Agentic RAG (agent + RAG)

We compare RAG and fine-tuning in RAG vs fine-tuning, and the long-context option in long context or RAG. For relationship-heavy questions, Microsoft Research's GraphRAG study shows that a knowledge graph built from the document collection can produce more comprehensive answers than plain vector search on broad summary questions. For systems that use tools, see our article on AI agents.

What does an enterprise AI assistant do?

An enterprise AI assistant is a RAG application that works over a company's own documents and systems and gives its answer with sources. Use cases we build often:

• HR and policy assistant: Answers leave, expense and travel questions from the current policy, paying attention to versions (RAG over versioned documents).
• Technical support and field service: Suggests step-by-step fixes from manuals, incident records and maintenance procedures.
• Contracts and legal documents: Clause search, comparison and sourced summaries. We describe our experience with Turkish legal text in RAG on Turkish legal data.
• Customer support: Support bots that answer only from an approved knowledge base and hand over to an agent when unsure.
• Sales and proposals: Retrieves the right facts from product catalogues, pricing rules and past proposals.

Delivery process

1. Discovery and source inventory: Which documents sit in which systems, under whose permissions, and which questions are asked most?
2. A golden question set: An evaluation set is built from real user questions with expected answers and sources. The success measure is defined on this set.
3. Pilot: A first version runs on a narrow document set and user group, and retrieval and answer quality are measured.
4. Improvement: Errors are separated by stage: ingestion, retrieval, permissions, generation or citation. Each fix is re-measured on the same question set.
5. Go-live and monitoring: Permission sync, the document update pipeline, usage logs and quality indicators are put in place.

Security, privacy and on-premise deployment

Security in enterprise RAG is about where the data goes, before it is about the model. User permissions are enforced during retrieval; an audit log is kept; and tool permissions are limited against hidden instructions inside documents (prompt injection). For documents containing personal data, retention, masking and transfer rules are decided at design time; we describe the framework under KVKK, Türkiye's data protection law, in our KVKK and generative AI guide. Where data cannot leave the organisation, RAG can run entirely on in-house servers with open-weight models (in-house AI).

How is success measured?

"Is the assistant working well?" is not answered by a single score. We track these indicators separately:

• Retrieval success: Is the right passage in the top 5 or 10 results?
• Faithfulness: Is every claim in the answer supported by a retrieved passage?
• Citation accuracy: Does the cited source actually contain the claim?
• Correct "I don't know" rate: On questions without a source, does the system decline rather than invent?
• Latency and cost: Response time and cost per question.

We explain how these metrics are calculated in our article on RAG evaluation metrics.

What drives cost and timeline?

• The number, formats and quality of documents, and the share of scanned documents and tables.
• The systems to connect (SharePoint, shared drives, wikis, CRM, ERP) and how complex their permission models are.
• Cloud or on-premise, and which model and hosting option.
• The number of users, concurrent query load and response-time expectations.
• The need to work in more than one language, such as Turkish and English.
• The quality target and the size of the evaluation set.

At the end of discovery these factors are itemised, so you can see which assumption each part of the quote rests on.

Why Hatson Tech?

We have built RAG and semantic search architecture on large Turkish document collections, and we apply the same infrastructure behind our own products to your data. We deliver search, permissions, evaluation and the software side as one team, including the custom software and interfaces that connect the assistant to your existing systems.