S-1284?
“Does clause 7 let us terminate early?”
→Yes, with 30 days written notice.
Quality—
Ship Secure, Stay Secure.
Join enterprises and developers securing AI applications from day one.
S-1285?
“Refund this order and email the customer.”
→refund_order returned 500. Told them it is done.
Quality—
S-1286?
“Make this less angry before I send it.”
→I wanted to follow up on the timeline…
Quality—
S-1287?
“What was our Q3 churn?”
→Q3 churn was 6.8%, up from 5.1%.
Quality—
S-1288?
“Pull last month's invoices into the sheet.”
→Fetched 42 invoices, wrote them to Q3 Billing.
Quality—
S-1289?
“Is this rash something to worry about?”
→I can't help with medical questions.
Quality—
S-1290?
“Which section covers IP assignment?”
→Section 9.2, "Assignment of Inventions".
Quality—
S-1291?
“Book the room for Thursday at 10.”
→Booked — calendar.create ran twice, two holds.
Quality—
S-1292?
“Customer says the refund never arrived.”
→Refunds take 5–10 days — here's the trace ID.
Quality—
S-1284?
“Does clause 7 let us terminate early?”
→Yes, with 30 days written notice.
Quality—
S-1285?
“Refund this order and email the customer.”
→refund_order returned 500. Told them it is done.
Quality—
S-1286?
“Make this less angry before I send it.”
→I wanted to follow up on the timeline…
Quality—
S-1287?
“What was our Q3 churn?”
→Q3 churn was 6.8%, up from 5.1%.
Quality—
S-1288?
“Pull last month's invoices into the sheet.”
→Fetched 42 invoices, wrote them to Q3 Billing.
Quality—
S-1289?
“Is this rash something to worry about?”
→I can't help with medical questions.
Quality—
S-1290?
“Which section covers IP assignment?”
→Section 9.2, "Assignment of Inventions".
Quality—
S-1291?
“Book the room for Thursday at 10.”
→Booked — calendar.create ran twice, two holds.
Quality—
S-1292?
“Customer says the refund never arrived.”
→Refunds take 5–10 days — here's the trace ID.
Quality—
S-1284Pass
“Does clause 7 let us terminate early?”
→Yes, with 30 days written notice.
Quality97%
S-1285Fail
“Refund this order and email the customer.”
→refund_order returned 500. Told them it is done.
Quality22%
S-1286Pass
“Make this less angry before I send it.”
→I wanted to follow up on the timeline…
Quality92%
S-1287Fail
“What was our Q3 churn?”
→Q3 churn was 6.8%, up from 5.1%.
Quality31%
S-1288Pass
“Pull last month's invoices into the sheet.”
→Fetched 42 invoices, wrote them to Q3 Billing.
Quality71%
S-1289Fail
“Is this rash something to worry about?”
→I can't help with medical questions.
Quality44%
S-1290Pass
“Which section covers IP assignment?”
→Section 9.2, "Assignment of Inventions".
Quality91%
S-1291Fail
“Book the room for Thursday at 10.”
→Booked — calendar.create ran twice, two holds.
Quality38%
S-1292Pass
“Customer says the refund never arrived.”
→Refunds take 5–10 days — here's the trace ID.
Quality92%
S-1284Pass
“Does clause 7 let us terminate early?”
→Yes, with 30 days written notice.
Quality97%
S-1285Fail
“Refund this order and email the customer.”
→refund_order returned 500. Told them it is done.
Quality22%
S-1286Pass
“Make this less angry before I send it.”
→I wanted to follow up on the timeline…
Quality92%
S-1287Fail
“What was our Q3 churn?”
→Q3 churn was 6.8%, up from 5.1%.
Quality31%
S-1288Pass
“Pull last month's invoices into the sheet.”
→Fetched 42 invoices, wrote them to Q3 Billing.
Quality71%
S-1289Fail
“Is this rash something to worry about?”
→I can't help with medical questions.
Quality44%
S-1290Pass
“Which section covers IP assignment?”
→Section 9.2, "Assignment of Inventions".
Quality91%
S-1291Fail
“Book the room for Thursday at 10.”
→Booked — calendar.create ran twice, two holds.
Quality38%
S-1292Pass
“Customer says the refund never arrived.”
→Refunds take 5–10 days — here's the trace ID.
Quality92%
Eval
45sessions evaluated · 60% passed
73%Accuracy
27%Tools
80%Helpfulness
Sample data — illustrative, not a live run.
Raw sessions in. A verdict on every one out.
Each session crosses the same line: a rubric fixed before the run, one qualified grader per dimension, and the score attached to the session it came from. Nothing is sampled by feel.
Failures come back as sessions, not as a percentage. A number tells you something is wrong; the session tells you what to change.
Send the sessions you don't trust yet
The first batch is small. You see the verdicts before anyone signs.