Skip to content

Dataset Schema

Schema definitions for evaluation samples used in RAG evaluation.

BaseSample

Bases: BaseModel

Base class for evaluation samples providing feature extraction.

Source code in ragbot\evaluation\dataset_schema.py 
 8
 9
10
11
12
class BaseSample(BaseModel):
    """Base class for evaluation samples providing feature extraction."""

    def get_features(self):
        return list(self.to_dict().keys())

Sample

Bases: BaseSample

Structured sample used for evaluating RAG responses.

Attributes:

Name Type Description
question Optional[str]

The user's question or input.
answer Optional[str]

The model-generated answer.
retrieved_context Optional[List[str]]

The context retrieved by the RAG pipeline.
reference_context Optional[List[str]]

The ground truth context.
reference_answer Optional[str]

The ground truth answer.
Source code in ragbot\evaluation\dataset_schema.py 
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
class Sample(BaseSample):
    """Structured sample used for evaluating RAG responses.

    Attributes:
        question (Optional[str]): The user's question or input.
        answer (Optional[str]): The model-generated answer.
        retrieved_context (Optional[List[str]]): The context retrieved by the RAG pipeline.
        reference_context (Optional[List[str]]): The ground truth context.
        reference_answer (Optional[str]): The ground truth answer.
    """

    question: Optional[str] = None
    answer: Optional[str] = None
    retrieved_context: Optional[List[str]] = None
    reference_context: Optional[List[str]] = None
    reference_answer: Optional[str] = None