SaibleDocs
Evaluation

Candidate models, on your own prompts

Every call decided by your policy and recorded. Figures side by side, never a verdict.

Golden setwritten, uploaded,synthetic, baselineRunup to fourcandidatesDecideyour policy,every callRecordin the ledger,never the textSide by sidefigures,no verdictPropose pack100 itemsor moreASealorg admin+ codeBB is never A↺ when the approved models change, the confirmed set is rerun;the diff waits in the review queue
From a golden set to a sealed validation pack · A proposes, B seals
  • Your providers' candidatesEach call decided by your policy in force, under the tool's registration, and served from the endpoint the record names; a refused item is sent nowhere. Paid through your own keys.
  • Saible-hosted candidatesAmazon Bedrock models on Saible's account, only when the setting is on. Your policy cannot decide them: each item is sent redacted and the row says not evaluated. Metered at zero credits.
  • Your golden setKept encrypted under your organisation's key, your own KMS key when set. Listed by its count and hash; the workbench never returns an item's text.
Where each candidate's call goes, and where the set is kept
Do it
  1. 1On Evaluation, choose the Application and make a golden set: Write, Synthesise, From traffic, Upload JSONL, or import a public baseline.
  2. 2Choose the set and up to four candidates, then click Run.
  3. 3Read the run side by side: what the policy did, latency, tokens, cost at list price, what was detected.
  4. 4With 100 items or more, click Seal as validation pack; another org admin seals it in Review queue.
  5. 5An org admin clicks Confirm as the set a change reruns; Regress now reruns it and the diff waits in the review queue.
For your agents

This topic is also written as Agent Skills your agents can load: evaluation-workbench, model-lab. Ask us for them.

Next · Control Plane · ExposureWhat each AI platform lets through →