How to use the Product Experiment Design Tool.
Define the hypothesis, measurement and interpretation before running an experiment. Consider what result would change the decision, not only what would confirm the idea.
Make the workflow fit your task.
State the proposed mechanism, comparison and outcome measure before launching the test. Define what would change the product decision and record practical limits on the sample, observation window and interpretation.
- What you provide
- Hypothesis, metrics and constraints.
- What you get
- Experiment design with success and stopping criteria.
See the input and the result.
Illustrative input and output · a teaching example, not a live WebAct run
Example input
Design a formative experiment for the hypothesis that previewing changed rows reduces incorrect approvals. Use comparable invoice-review tasks, current and proposed views, and record correct completion plus incorrect approvals.
Completed example
Hypothesis: the preview makes changed rows easier to identify before approval. Comparison: current view versus proposed preview on comparable tasks. Primary observation: incorrect approvals per review attempt. Additional observations: correct completion and time needed to finish. Controls: keep task difficulty and instructions comparable; vary order to reduce a simple practice effect. Decision: investigate whether observed differences support further testing; sample size and evaluation window must be agreed before running the study.
Load this input into the prompt, then copy it to WebAct to try the task. Your result may differ from the illustration.
Decisions and troubleshooting.
Should an experiment measure only the desired positive outcome?
Include relevant failure or guardrail measures. An improvement in one metric may come with a cost elsewhere in the user experience.
Why does a positive result fail to answer the original question?
Check whether the test changed several things or measured a proxy unrelated to the decision. Narrow the claim to what was actually observed.
Try it with your own source.
Replace the example with your material in the task prompt. Keep the requirements you need, then copy the task into WebAct.
Customize and copy the task ↑