RAG for Customer Support: Cutting Ticket Volume With a Document-Grounded Answer Engine
Every B2B SaaS team we talk to about AI features starts with the same request: an assistant that answers customer questions from the product documentation before anyone opens a ticket. The instinct is sound. RAG for customer support is the most forgiving first retrieval project you can pick. The source material already exists, wrong answers surface quickly, and the business case has a number attached to it: tickets that never got filed. This post describes the architecture we build for clients, from the ingestion pipeline through pgvector storage and hybrid retrieval to the confidence threshold that decides when the engine should stay quiet and hand over to a human.
What RAG for customer support has to get right
A support answer engine has one job. Given a customer question, it finds the passages in your documentation and your resolved tickets that answer it, then generates a reply grounded in those passages, with links back to the sources. The grounding is the point. A bare LLM will happily answer questions about features you do not have, in a confident tone your customers will believe. A grounded engine can only say what your documents say, and when the documents say nothing useful, it should admit that and create a ticket instead.
That second behavior separates useful deployments from embarrassing ones. A large share of real support questions are account specific. "Why was I charged twice" has no answer in your docs, and no retrieval trick will change that. Design the handover to a human agent on day one rather than bolting it on later.
The ingestion pipeline
Everything downstream depends on what you feed in, and support content is messier than any demo suggests. In practice you ingest from three places: Markdown documentation living in a repo, wiki exports from a tool like Notion, and resolved conversations from a helpdesk like Intercom.
Documentation is the easy part. Split it along heading boundaries, keep the heading path (Billing, then Invoices, then Credit notes) as metadata on every chunk, and store the source URL so answers can link back. We went deep on sizing in our post on chunking strategies; the short version is that structure-aware chunks of roughly 200 to 500 tokens, with the heading path prepended to the text before embedding, beat fixed-size windows for support content.
Helpdesk exports need real cleaning. A resolved Intercom conversation contains greetings, small talk, screenshots, three rounds of "did that work for you", and somewhere in the middle, the actual resolution. Index the raw transcript and you index noise. What works instead: run each resolved conversation through an LLM once at ingestion time, extract a clean question and answer pair, and index that pair. The cost is a fraction of a cent per ticket and the payoff is large, because past tickets capture the phrasing customers actually use. Documentation rarely does.
Attach metadata to every chunk: source type, product version, language, and a last updated timestamp. You will filter on all four. A customer on version 2 of your product should never receive an answer that only applies to version 3, and a German speaking customer should not get English passages mixed into a German answer.
Storage and retrieval: pgvector plus BM25
You do not need a dedicated vector database for this. A support corpus is small by vector search standards, tens of thousands of chunks for most products. Postgres with pgvector handles that comfortably and keeps the vectors in the same database as the metadata, so a version filter is a WHERE clause rather than a second system to operate. The full argument is in our production RAG blueprint, and if a tooling decision like this is exactly where your team is stuck, that is what our tech stack strategy engagements are for.
For embeddings, text-embedding-3-small from OpenAI is the default choice: cheap and good enough for support content. If your compliance requirements keep customer text inside your own infrastructure, a self-hosted multilingual embedding model works too. The architecture does not change.
Vector similarity alone will disappoint you, though. Support queries are full of exact identifiers such as error codes and API endpoint names, and embeddings are weak at exact string matching while Postgres full text search is excellent at it. Run both and merge the result lists with reciprocal rank fusion. We described the SQL for this hybrid setup in a separate post.
The last retrieval stage is reranking. Pull 30 to 50 candidates with hybrid search, which is tuned for recall, then let a cross-encoder rerank them and keep the top five for the prompt. A cross-encoder reads the question and the passage together, so it is far better at precision than either retrieval method on its own. The added latency is well under a second, which a support widget can afford.
The confidence threshold
This is the most consequential design decision in the whole system, and most teams skip it. Every retrieval returns similarity and reranker scores. Define a threshold below which the engine does not answer at all. Instead it says it could not find anything reliable, opens a ticket, and attaches the question plus the best retrieved passages so the agent starts with context instead of a blank screen.
Calibrate the threshold with data you already have. Take a few hundred past customer questions, run them through retrieval, and have someone judge whether the top passages actually answer them. Plot the scores for both groups and pick the cutoff that keeps precision high, accepting that some answerable questions will fall through to humans. That trade is correct. A confident wrong answer costs customer trust and usually produces a second, angrier ticket. A handover costs you nothing, because that ticket was coming anyway.
The Symfony pipeline and the Next.js widget
On the backend we build the ingestion side as a Symfony Messenger pipeline. A console command or a webhook enqueues a document reference, and workers fetch, clean, chunk, embed, and upsert it into pgvector. Make every step idempotent by keying on a content hash, so re-running ingestion after a failure or a docs update is safe. Embedding API calls belong in the workers, behind a rate limiter, never in a web request.
The query path is deliberately thin: a controller that normalizes the question, embeds it, runs the hybrid query and the reranker, applies the threshold, and streams the generated answer over server sent events. The frontend is a small Next.js widget you embed in the support portal. It streams the answer as it generates, renders source links underneath, and always shows an escape hatch that files a ticket with the question prefilled. Never trap a customer in a bot loop.
If you are adding this to an existing product rather than building fresh, the integration concerns are covered in our retrofit playbook.
Measuring deflection honestly
"Questions answered" is a vanity metric. The number that matters is the deflection rate: the share of answer sessions where the customer saw a grounded answer and did not file a ticket on the same topic within a day. Measuring it takes discipline, because you need to join widget sessions against helpdesk data, but without it you cannot tell whether the engine saves agent time or merely delays tickets.
Log every session with the question, the retrieval scores, whether an answer was shown or the query fell below threshold, the customer's thumbs up or down, and any ticket filed afterwards. Two byproducts of that logging earn their keep. The below-threshold questions, clustered weekly, are a ranked backlog of documentation gaps. And the logged retrieval results become the dataset for a proper evaluation harness, which we covered in our post on RAG evaluation metrics, so when answer quality drops you can tell whether retrieval or generation is at fault.
Ticket volume is a cost line, and this is one of the few AI features where the savings are directly measurable. If you want a second pair of eyes on your architecture before you commit, or a team to build the engine end to end alongside yours, this sits squarely in our custom software development work. Write to hello@wolf-tech.io or have a look around wolf-tech.io.