Skip to content

RAGBOT CLI

By University of Cantabria.

@author Rubén Martínez Amodia
@email  ruben.martinezam@alumnos.unican.es
@supervisor Cristina Tirnauca
@supervisor Marta Zorrilla

RAGBOT is a command-line tool designed for running and evaluating Retrieval-Augmented Generation (RAG) chatbots. Developed as part of a bachelor thesis project, it supports the creation of intelligent virtual assistants through flexible experimentation and principled evaluation.

Built atop the powerful LangChain framework and integrated with LangSmith for seamless experiment tracking and logging, RAGBOT offers a complete workflow for iterating on RAG pipelines.

The tool provides a structured and extensible framework that enables:

  • Interactive development and testing of RAG chatbot configurations.
  • Customizable control over fine-tuning parameters, LLM selection, embedding strategies, and retriever behavior.
  • Lightweight, modular evaluation inspired by the RAGAS framework—yet more affordable and adaptable to diverse experimentation needs.

Evaluation in RAGBOT combines multiple methodologies, including:

  • LLM-as-a-judge techniques for subjective assessments,
  • Embedding-based similarity scoring,
  • And additional custom metrics for deeper analysis.

Whether you're optimizing a chatbot's responses or conducting rigorous academic evaluations, RAGBOT offers the tools to explore, refine, and understand the behavior of modern RAG systems from the command line.

🧪 See the CLI Overview to get started.
📚 Dive into the API Reference for detailed documentation.
📂 To explore the source code, use the GitHub link in the upper-right corner of this site.