Type alias LabeledCriteria

Configuration to load a "LabeledCriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria and also provides a "ground truth" label for the evaluator to incorporate in its evaluation.

Param

The criteria to use for the evaluator.

Param

The language model to use for the evaluator.

Returns

The configuration for the evaluator.

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "labeled_criteria",
    criteria: "correctness"
  }],
};
Copy

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "labeled_criteria",
    criteria: { "mentionsAllFacts": "Does the include all facts provided in the reference?" }
  }],
};
Copy