A Beginner's Guide to Evaluating RAG Systems with LangSmith

Introduction to RAG

Retrieval-Augmented Generation (RAG) combines retrieval and generation to improve the quality of responses in natural language tasks. A RAG system has two main parts: a retriever that fetches the information needed to answer a query, and a generator that produces the answer from the retrieved information. RAG lets LLMs reach external knowledge, so they give more accurate, contextual answers and hallucinate less.

Why use LangSmith evaluations?

LangSmith gives you a platform to evaluate, monitor, and improve RAG systems. With it you can gain insight into how your RAG components perform, spot areas to improve, monitor the system in production, and apply various metrics to assess different parts of the pipeline.

Setting up your environment

You need a Python environment with the required libraries and access to LangSmith and OpenAI API keys. Install the packages with: pip install -qU langsmith langchain-community langchain langchain_openai chromadb langchain-chroma.

Then set your environment variables. A small helper prompts for any missing key with getpass, sets OPENAI_API_KEY and LANGCHAIN_API_KEY, turns on tracing with LANGCHAIN_TRACING_V2="true", and points LANGCHAIN_ENDPOINT at https://api.smith.langchain.com.

Building the RAG pipeline

We use LangChain to build a retriever and a generator. A RecursiveUrlLoader fetches the LangChain LCEL documentation pages, a RecursiveCharacterTextSplitter cuts them into chunks of about 4500 characters with 200 characters of overlap, and the chunks are embedded with OpenAIEmbeddings and stored in a Chroma vector store, which becomes the retriever.

The core logic lives in a RagBot class. Its retrieve_docs method uses the retriever to fetch relevant documents for a question. Its invoke_llm method calls OpenAI's chat completions with the retrieved docs as context, using a system prompt that casts the model as a code assistant with expertise in LCEL. Its get_answer method chains retrieval and generation together to return the final answer.

Every method carries the @traceable() decorator, which enables LangSmith tracing so you can monitor and evaluate performance in detail. The result is a RAG system that answers questions about the loaded documentation, such as "What is LCEL?".

Creating the dataset for evaluation

To evaluate a RAG system you need a high-quality dataset of question-answer pairs. Using the LangSmith Client, you define lists of questions and expert answers about LCEL, zip them into QA pairs, create a dataset (here named RAG_test_LCEL with a short description), and populate it with create_examples, passing the questions as inputs and the answers as outputs.

The process, step by step: define the QA pairs, combine them into dictionaries, initialize the LangSmith Client, set the dataset name and description, create the dataset, and add the examples. Once created, you can view and manage it through the LangSmith interface. A well-structured dataset is the foundation for a thorough evaluation.

Evaluating the RAG system

LangSmith provides a suite of metrics to assess the pipeline. This guide focuses on four types of evaluation.

• Response vs reference answer: how correct the generated answer is compared to a ground-truth label.
• Response vs retrieved documents: how faithful the response is to the retrieved context, which helps detect hallucinations.
• Retrieved documents vs input: how relevant the retrieved documents are to the query.
• Hallucination detection: whether the response contains information not supported by the retrieved documents.

Two small wrapper functions feed the evaluators: predict_rag_answer returns just the answer for answer evaluation, and predict_rag_answer_with_context returns the answer plus its contexts for evaluating retrieval and hallucinations.

1. Response vs reference answer

This method compares the generated answer to a pre-defined, expert-crafted reference answer. Use it when you have questions with known high-quality answers — for benchmarking against human-level performance, spotting systematic errors or bias, and tracking answer quality over time. The key metric is a correctness score, usually 0 to 1, where 1 is a perfect match. In code, a LangChainStringEvaluator using "cot_qa" (chain-of-thought QA) runs against the dataset via evaluate().

2. Response vs retrieved docs (hallucination detection)

This method checks whether the response is faithful to the retrieved documents, catching cases where the model invents information. Use it when accuracy is paramount — medical, legal, or financial applications — and when you need traceability between responses and sources. The key metric is a faithfulness score from 0 to 1, where 1 means perfect alignment. It uses a labeled_score_string evaluator with a rubric scoring how grounded the answer is, normalized by 10.

3. Retrieved docs vs input (document relevance)

This method judges the quality and relevance of the retrieved documents for a query, which helps optimize the retrieval component. Use it with a large corpus where efficient retrieval matters, when improving precision and recall, or when tuning retrieval parameters. Metrics include a relevance score, and others such as NDCG, Mean Reciprocal Rank, or Precision@k. A score_string evaluator scores document relevance on a 1-to-10 rubric, normalized by 10.

4. Hallucination detection

This method specifically flags information in the response that the retrieved documents do not support. Use it when factual accuracy is critical and you want to minimize false or unsupported claims. The key metric is a binary score: 1 means the answer is grounded in the retrieved facts, 0 means possible hallucination. A custom grader pulls the retrieved documents and the generation from the run tree, then asks an LLM with structured output (a GradeHallucinations model holding a binary_score) whether the generation is grounded in the facts.

Together these four methods form a robust framework: response accuracy against references, response faithfulness to retrieved data, retrieval quality against the query, and explicit hallucination detection. Review the results in the LangSmith interface, where you can visualize trends and compare versions of your system.

Conclusion

Adding LangSmith evaluations to your RAG pipeline is a key step toward a reliable, high-performing system. This approach lets you assess answer accuracy against references, ensure faithfulness to retrieved documents, and evaluate the relevance of retrieval. Evaluation is not a one-time task but an ongoing process — as you refine the system, add data, or adapt to new needs, regular evaluation keeps performance improving over time.

Originally published on the FutureSmart AI blog.