ExactMatchStringEvaluator#
- class langchain.evaluation.exact_match.base.ExactMatchStringEvaluator(
- *,
- ignore_case: bool = False,
- ignore_punctuation: bool = False,
- ignore_numbers: bool = False,
- **kwargs: Any,
Compute an exact match between the prediction and the reference.
Examples
>>> evaluator = ExactMatchChain() >>> evaluator.evaluate_strings( prediction="Mindy is the CTO", reference="Mindy is the CTO", ) # This will return {'score': 1.0}
>>> evaluator.evaluate_strings( prediction="Mindy is the CTO", reference="Mindy is the CEO", ) # This will return {'score': 0.0}
Attributes
evaluation_name
Get the evaluation name.
input_keys
Get the input keys.
requires_input
This evaluator does not require input.
requires_reference
This evaluator requires a reference.
Methods
__init__
(*[, ignore_case, ...])aevaluate_strings
(*, prediction[, ...])Asynchronously evaluate Chain or LLM output, based on optional input and label.
evaluate_strings
(*, prediction[, reference, ...])Evaluate Chain or LLM output, based on optional input and label.
- Parameters:
ignore_case (bool)
ignore_punctuation (bool)
ignore_numbers (bool)
kwargs (Any)
- __init__(
- *,
- ignore_case: bool = False,
- ignore_punctuation: bool = False,
- ignore_numbers: bool = False,
- **kwargs: Any,
- Parameters:
ignore_case (bool)
ignore_punctuation (bool)
ignore_numbers (bool)
kwargs (Any)
- async aevaluate_strings(
- *,
- prediction: str,
- reference: str | None = None,
- input: str | None = None,
- **kwargs: Any,
Asynchronously evaluate Chain or LLM output, based on optional input and label.
- Parameters:
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
kwargs (Any) – Additional keyword arguments, including callbacks, tags, etc.
- Returns:
The evaluation results containing the score or value.
- Return type:
dict
- evaluate_strings(
- *,
- prediction: str,
- reference: str | None = None,
- input: str | None = None,
- **kwargs: Any,
Evaluate Chain or LLM output, based on optional input and label.
- Parameters:
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
kwargs (Any) – Additional keyword arguments, including callbacks, tags, etc.
- Returns:
The evaluation results containing the score or value.
- Return type:
dict