SKILL FILE

Experiment Designer with AI

Design product experiments end-to-end — hypothesis to decision rule. Covers A/B tests, feature betas, pilots, fake-door tests, and pricing tests.

Download Skill File ↓

What this skill file teaches Claude

Drop one markdown file into your repo. Claude Code learns how to run this entire workflow.

1

Hypothesis writing

Forces the hypothesis into a falsifiable shape: "If X, then Y, measured by Z."

2

Experiment-type selection

Helps you pick A/B test, fake-door, pilot, beta, painted door, or qualitative — based on the question you're asking.

3

Sample size guidance

Estimates how many users you need to detect a meaningful effect. Saves you from running underpowered tests.

4

Decision rule (pre-committed)

You agree in advance what each possible result means — ship, kill, iterate, dig deeper.

5

Risk + ethics flagging

Surfaces ethical concerns (pricing tests with vulnerable users, dark patterns) and rollout risks before you commit.

What you can build with this

A/B test a feature variant

Design the test with proper sample size, decision rule, and guardrails before turning the flag on.

Pricing test

Test a new pricing model with the right segment, control group, and exit criteria.

Fake-door MVP

Validate demand for a feature before building it — with a clear "yes/no" threshold for "build the real thing."

Pilot or beta program

Design the success criteria and timeline up front so the pilot can actually end.

Get the full skill file

Everything above is 80% of the skill file. Download the complete version with full implementation details, agent prompts, and ready-to-run scripts.

Common questions

No — the skill handles the standard sample-size math for typical product experiments. For high-stakes or complex experiments (medical, financial), still loop in a stats expert.
Show users a button or link for a feature that doesn't exist yet. Measure how many click. If demand is high, build it; if not, kill the idea cheaply. Useful for validating before investing in build.
Pre-commit your decision rule and sample size BEFORE looking at the data. The skill outputs the experiment plan in a format you can share with the team so the commitment is shared.
Yes — sequential testing (run price A for a period, then price B), geographic tests, or new-user-only tests all avoid the awkwardness of differential pricing for existing customers.
When the cost of the experiment exceeds the value of the answer, when ethics make it impossible to do cleanly, when the result wouldn't change your behaviour either way, or when qualitative research would be faster.

Ready to automate?

SoloStack gives you every skill pre-installed — scraping, marketing, sales, CRM, and more. One repo. Every department.

Book a Call →