Testing and Quality

Building a great AI Flow is an iterative process. Unlike traditional software, the same input can produce different output every time the model runs - so the only way to know if your Flow is actually working is to test it. FormWise gives you a full set of testing and evaluation features in the Quality tab of your Flow editor.

Why testing matters: small wording changes in your prompt, a new model version, or a tweak to your Notebooks can all affect output quality in ways you would not catch with a single preview run. Tests catch regressions before your users do.

[Screenshot: The Quality tab inside a Flow, showing rubric rules on the left and test cases on the right]

---

Preview testing

The quickest way to sanity-check your Flow is with the built-in preview:

1. Open your Flow and go to the Design tab.
2. Click the Preview button to open the preview panel.
3. Fill in sample inputs (or type a message for chatbot Flows).
4. Click Run to execute the workflow.
5. Review the output to see if it matches your expectations.

This is great for quick, hands-on checks while you are building. You can run the preview as many times as you want and adjust your workflow between runs. Preview runs also appear in Execution History so you can dig into the per-node trace afterward.

---

Test cases

Test cases are the inputs you want to check against. Each test case is paired with the Flow's current workflow and rubric - when you run the evaluator, FormWise executes the workflow for every test case and grades each output.

What a test case looks like

A test case has:

• A name or description - so you can recognize it in the list ("Empty URL," "Long product description," "User asks for a refund").
• Inputs - for form Flows, a value for every form question. For chatbot Flows, a single message string.
• Optional expected behavior - a short note describing what a good output should look like. This is fed to the grader so it knows what to check for.

You do not need a hard-coded "expected output" string. FormWise uses rubric-based grading (see below) - you describe the qualities a good answer should have, and the grader judges every output against those qualities.

Creating test cases manually

1. Go to the Quality tab.
2. Click Add Test Case.
3. Fill in the input fields with sample values.
4. Add a short description so you can find it later.
5. Optionally, add expected behavior - a sentence or two describing the ideal response.
6. Save the test case.

You can create as many test cases as you need. Good test coverage means checking a variety of inputs - not just the happy path. Include edge cases: empty fields, weirdly long input, off-topic requests, things you would not expect a real user to send.

Auto-generating test cases

Do not want to write every test case by hand? FormWise can generate them for you:

1. In the Quality tab, click Generate Tests.
2. Choose how many test cases you want.
3. The AI reads your Flow's configuration - input fields, system instructions, and workflow - and produces a set of realistic test cases.
4. Review the generated tests and edit or remove any that do not fit your use case.

Auto-generated tests are a great starting point, especially when you are setting up the Quality tab for the first time. Always review them and add edge cases the AI might have missed.

---

Rubric rules

A rubric rule is a criterion the grader uses to score each output. Each rule has:

• A description - the rule itself, written in plain language. For example: "The response always includes a citation," or "The tone is friendly and never apologetic."
• An assertion - the specific thing the grader checks for.
• A severity - Critical, Major, or Minor. A Critical failure on one rule fails the whole test case; Major and Minor failures are surfaced but weighed less heavily.

You can write rubric rules by hand, or click Generate Rules and describe what you want to evaluate in plain language ("never use the word 'delve', always include citations, maintain a professional tone"). FormWise turns that into a structured rule set you can edit.

Rubric grading is faster and more flexible than exact-match testing - the grader can recognize that two differently worded responses both meet the same criterion.

[Screenshot: The rubric rules section with three example rules and their severity badges]

---

Running the evaluator

Once you have at least one test case and one rubric rule:

1. Click Run Evaluator at the top of the Quality tab.
2. FormWise runs your current workflow against every test case.
3. Each output is graded against every rubric rule.
4. The result is recorded as a new Execution - a numbered run you can come back to later.

While the evaluator is running, you will see a live progress indicator. A typical suite of 5--10 test cases takes anywhere from 30 seconds to a few minutes, depending on the models and tools your workflow uses.

Each evaluator run charges credits - one for every test case execution plus the grader passes. Keep your test suite focused so runs stay cheap.

---

Reading the results

Every evaluator run produces an Execution entry in the history list. Click into one to see:

• Pass / fail summary - how many test cases passed all rules, and how many had at least one failure.
• Per-rule breakdown - which rules failed and on which test cases.
• Per-test-case detail - the input, the actual output, and the grader's reasoning for any failures.
• Latency - how long each test case took to run end-to-end.
• Full execution trace - click View run on any test case to open the same per-node waterfall view used in Execution History. This shows exactly what each node received, what it produced, and where time was spent.

Comparing one execution to the next is how you measure progress. If Execution #4 fails three rules and Execution #5 only fails one, your most recent change moved things in the right direction.

[Screenshot: An execution detail panel showing the pass/fail breakdown by rubric rule]

---

Turning failures into fixes

A failed test case is information, not a verdict. FormWise can use the failures to suggest concrete improvements:

1. Open a failed Execution from the history list.
2. Scroll down to review the AI's suggested fixes for each failing rule.
3. Each suggestion targets a specific rule and explains what to change in the prompt or workflow configuration.
4. Click Apply Fix to Model to rewrite the target node's prompt. A diff preview shows you exactly what is changing before you confirm.

You can also pull insights from failures manually:

• If a rule fails on most test cases, the rule might be too strict, or your system instruction is missing something important. Try rewording the instruction or adding a few-shot example.
• If a rule fails on one specific test case, that input might be an edge case worth handling explicitly.
• If failures cluster around a specific node, that node's prompt is a good place to start refining.

If your Flow uses Notebooks, check that the relevant knowledge is actually in your Notebooks and is being retrieved - the per-node trace will show you what context the model received.

---

When to run tests

You do not need to run the evaluator on every save. A few good moments:

• Before publishing a version. Always run the evaluator before promoting a draft to production. See Versioning.
• After big changes. Switched models? Rewrote a system instruction? Added a node? Run the suite to make sure nothing broke.
• After updating your Notebooks. New documents in your Notebooks can change retrieved context and shift outputs.
• Periodically on a schedule. Model providers update their models occasionally. Re-running your suite catches drift before users notice.

For one-off bug reports from real users, jump straight to Execution History - that is where you can replay actual production runs and inspect what happened.

---

Best practices

• Test with a variety of inputs. Short, long, edge cases, unusual requests. Your users will surprise you.
• Keep rules independent. Each rubric rule should test one thing. Bundling multiple checks into a single rule makes failures harder to act on.
• Use Critical sparingly. Reserve Critical severity for behaviors that would genuinely make the Flow unusable - safety, factual accuracy, brand-breaking tone.
• Re-run after changes. Every time you update your system instruction, switch models, or adjust your workflow, run the suite again.
• Start testing early. You do not need a perfect Flow before you start. Early tests help you spot problems before they compound.

---

Next steps

Happy with your test results? Learn how to save your progress with Versioning so you can track changes and roll back if needed. To debug real user runs after you publish, see Execution History.