StringEvaluator#
- class langchain.evaluation.schema.StringEvaluator[source]#
Grade, tag, or otherwise evaluate predictions relative to their inputs and/or reference labels.
Attributes
evaluation_name
The name of the evaluation.
requires_input
Whether this evaluator requires an input string.
requires_reference
Whether this evaluator requires a reference label.
Methods
aevaluate_strings
(*, prediction[, ...])Asynchronously evaluate Chain or LLM output, based on optional input and label.
evaluate_strings
(*, prediction[, reference, ...])Evaluate Chain or LLM output, based on optional input and label.
- async aevaluate_strings(
- *,
- prediction: str,
- reference: str | None = None,
- input: str | None = None,
- **kwargs: Any,
Asynchronously evaluate Chain or LLM output, based on optional input and label.
- Parameters:
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
kwargs (Any) – Additional keyword arguments, including callbacks, tags, etc.
- Returns:
The evaluation results containing the score or value.
- Return type:
dict
- evaluate_strings(
- *,
- prediction: str,
- reference: str | None = None,
- input: str | None = None,
- **kwargs: Any,
Evaluate Chain or LLM output, based on optional input and label.
- Parameters:
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
kwargs (Any) – Additional keyword arguments, including callbacks, tags, etc.
- Returns:
The evaluation results containing the score or value.
- Return type:
dict