Concept & Theory

A new model dropping at a lower price can look like free money. In practice, it is a production change with real blast radius. The right question is not whether the model is cheaper. The right question is whether it passes a small, repeatable gate on the workloads that matter to you.

A useful 30-minute gate is narrow by design. You are not trying to prove the model is "best" in the abstract. You are checking whether it survives the parts of your stack that usually fail first: strict JSON output, tool-call discipline, short context retention, refusal handling, and consistency under messy inputs. That is the difference between a benchmark and a migration plan.

The point of this prompt collection is to make that gate fast enough that teams will actually run it before they switch. It is built for developers evaluating a cheap model drop like DeepSeek-Coder-V2 against an incumbent such as Grok-2, but the method applies to any current model API that accepts chat-style messages and returns text or structured output.

A strong gate has four properties:

It uses the same prompts your app already depends on.
It includes a small adversarial set, not just happy-path cases.
It asks for outputs you can score mechanically or inspect quickly.
It ends with a clear decision rule: ship, hold, or retest.

The practical win here is speed. In 30 minutes, you can often learn enough to reject a bad move, or at least avoid an impulsive migration based on price alone.

The Prompts

Use these prompts in order. They are written to be copied into a chat model, a test harness, or a small evaluation script. Replace the variables with your own schemas, examples, and policies.

1) Production Fit Triage Prompt

This prompt is for the first pass. It asks the model to summarize how it would handle your real workload and to expose any hidden assumptions early.

You are evaluating whether a new model can replace an existing production model.

Task:
Review the workload below and assess whether this model is a credible drop-in candidate.

Workload:
- Product domain: {{DOMAIN}}
- Primary use case: {{PRIMARY_USE_CASE}}
- Output format: {{OUTPUT_FORMAT}}
- Critical constraints: {{CRITICAL_CONSTRAINTS}}
- Failure modes that matter most: {{TOP_FAILURE_MODES}}
- Current incumbent model: {{CURRENT_MODEL_NAME}}
- Candidate model: {{CANDIDATE_MODEL_NAME}}

Instructions:
1. Identify the top 5 risks of switching to the candidate model.
2. Call out any places where the workload depends on deterministic formatting, exact wording, or stable ordering.
3. State whether this model should be considered for a 30-minute gate, a longer pilot, or blocked immediately.
4. Keep the answer concrete. No generic model hype.

Return the result in this exact structure:
- Verdict: one of [30-minute gate, longer pilot, blocked]
- Risks: numbered list
- Unknowns: numbered list
- Recommendation: 2-4 sentences

2) Schema Fidelity Prompt

This is the most important test for many production systems. Cheap models often look fine until you ask them to obey a schema under pressure.

You are a strict formatter for production JSON.

Given the user request and schema below, produce only valid JSON that conforms exactly to the schema. Do not add commentary, markdown, code fences, or extra keys.

User request:
{{USER_REQUEST}}

Schema:
{{JSON_SCHEMA}}

Rules:
- Use only fields defined in the schema.
- Preserve required field names exactly.
- If a field cannot be determined, use null only when the schema allows it.
- Respect enum values exactly as written.
- Preserve numeric types and date formats exactly.
- Do not explain your reasoning.

Edge condition to test:
{{EDGE_CONDITION}}

Return only the final JSON object.

Run this prompt against at least three cases: a normal request, a partial request, and a malformed request. Score whether the model produces valid JSON every time, not just plausible JSON.

3) Failure-Mode and Refusal Prompt

A model that is cheap but unreliable can create silent operational debt. This prompt checks how it behaves when the request is ambiguous, conflicting, or outside policy.

You are assisting with a production workflow.

The request below may contain ambiguity, conflicting requirements, or invalid assumptions.

Request:
{{TRICKY_REQUEST}}

Context:
- The system depends on accurate extraction and concise output.
- It is better to ask for clarification than to invent missing facts.
- If the request cannot be completed safely, say so directly.

Instructions:
1. Identify the exact parts of the request that are ambiguous or unsafe.
2. Say whether you can proceed, need clarification, or must refuse.
3. If you can proceed, give the safest minimal answer.
4. Do not over-explain.

Return format:
- Status: [proceed | clarify | refuse]
- Issues: numbered list
- Response: short answer or clarification question

Use this to catch models that over-answer. A good production model knows when to stop and ask a question.

4) Side-by-Side Scoring Prompt

This prompt is useful when you want a clean comparison between the incumbent and the candidate. Run the same test case twice, then compare the answers manually or with a simple rubric.

You are scoring two model outputs for the same task.

Task:
{{TASK_DESCRIPTION}}

Reference constraints:
- Must satisfy: {{MUST_SATISFY}}
- Must avoid: {{MUST_AVOID}}
- Preferred style: {{PREFERRED_STYLE}}

Model A output:
{{MODEL_A_OUTPUT}}

Model B output:
{{MODEL_B_OUTPUT}}

Score each output from 1 to 5 on:
1. Instruction adherence
2. Format correctness
3. Completeness
4. Factual restraint
5. Practical usefulness

Then answer:
- Winner: A, B, or tie
- Why: 3-5 sentences
- Blocker issues: list only issues that would stop production use
- Next test: one targeted follow-up prompt

This is the fastest way to turn an opinionated debate into a shared decision. It also makes the tradeoff explicit: cheaper is only useful if the score stays high where it counts.

5) Migration Decision Prompt

Use this last. It helps the team turn test results into an actual recommendation instead of another round of discussion.

You are advising a developer team on whether to migrate from one model to another.

Inputs:
- Incumbent model: {{CURRENT_MODEL_NAME}}
- Candidate model: {{CANDIDATE_MODEL_NAME}}
- Monthly request volume: {{MONTHLY_VOLUME}}
- Cost difference: {{COST_DIFFERENCE}}
- Gate results: {{GATE_RESULTS}}
- Known regressions: {{KNOWN_REGRESSIONS}}
- Business tolerance for failures: {{FAILURE_TOLERANCE}}

Decision criteria:
- Reject any candidate that fails schema fidelity.
- Reject any candidate that increases manual correction time.
- Prefer a slower, safer migration over an immediate cost win.

Instructions:
1. Summarize the evidence.
2. State whether the candidate is ready for limited rollout, broader rollout, or rejection.
3. Name the exact blocker if you reject it.
4. Keep the answer grounded in operational risk, not model fandom.

Return:
- Decision: [limited rollout | broader rollout | rejection]
- Evidence: numbered list
- Blocker: one sentence
- Rollout plan: 3 concrete steps or null

Variables

Keep the variable set small. The gate should be fast to run, not a project of its own.

{{DOMAIN}}: your product area, such as fintech ops, developer tools, support automation, or internal admin workflows.
{{PRIMARY_USE_CASE}}: the exact task the model performs.
{{OUTPUT_FORMAT}}: JSON, Markdown, plain text, or tool-call style.
{{CRITICAL_CONSTRAINTS}}: ordering, tone, length, schema, citations, or latency requirements.
{{TOP_FAILURE_MODES}}: the failures that hurt you most, such as hallucinated fields, broken JSON, or overconfident answers.
{{USER_REQUEST}}: a realistic prompt from your application.
{{JSON_SCHEMA}}: the exact schema your app expects.
{{EDGE_CONDITION}}: one nasty but realistic edge case.
{{TRICKY_REQUEST}}: a request with ambiguity, missing data, or conflicting instructions.
{{TASK_DESCRIPTION}}: the shared task used for comparison.
{{MUST_SATISFY}} and {{MUST_AVOID}}: hard rules for evaluation.
{{GATE_RESULTS}}: a short summary of what happened in the tests.

A practical tip: keep the same 8 to 12 test cases every time you evaluate a candidate. If you keep changing the set, you are comparing vibes, not models.

Expected Output

A good 30-minute gate should end with one of three conclusions:

Proceed - the candidate matches the incumbent on the behaviors that matter, with no serious regression in schema fidelity, refusal behavior, or formatting.
Pilot - the candidate is promising, but you need a limited rollout or more targeted testing before a full migration.
Reject - the candidate introduces brittle behavior, higher correction cost, or unacceptable inconsistency.

The output of the gate should be short and operational:

A clear verdict.
A list of failed test cases.
A note on whether any failures are cosmetic or production-blocking.
A recommendation on next action.

A cheap model drop can still be a good move. But only after it survives the gate. That is the difference between making a cost-aware decision and making an expensive mistake in slow motion.