Type alias Criteria

Configuration to load a "CriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria.

Param

The criteria to use for the evaluator.

Param

The language model to use for the evaluator.

Returns

The configuration for the evaluator.

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "criteria",
    criteria: "helpfulness"
  }]
};
Copy

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "criteria",
    criteria: { "isCompliant": "Does the submission comply with the requirements of XYZ"
  }]
};
Copy