Autopilot and AI review
What the AI writes, how its variants are reviewed, and exactly what autopilot may do on its own.
Three levels of automation#
Avenue Lift can do as much or as little of the optimization loop as you want. Each level builds on the one before, and each is a deliberate switch — nothing becomes automatic by default.
- AI proposes, you decide
- Ask for test ideas for a page or for challenger variants for an experiment. Everything the AI writes waits for a person. This is how every site starts.
- Autopilot recommends
- The experiment page shows what autopilot concludes from the current results and what it would do — promote, stop a loser, add a challenger — for you to carry out. This works for every experiment, with autopilot on or off.
- Autopilot acts
- With autopilot enabled for the site and for the experiment, the same conclusions are carried out on a schedule, within the limits you set. Included in the Growth and Agency plans.
What the AI writes#
- Ideas
- Avenue Lift reads a page of your site and writes test ideas for it: a title, a hypothesis, the reasoning, the concrete changes, and rough estimates of impact, confidence and effort. Ideas land in the site's backlog, where you shortlist, edit or dismiss them. An idea becomes a draft experiment in one step.
- Challenger variants
- For an existing experiment, the AI sees the page, the hypothesis, the variants that already exist and how they are performing — so it writes something different from what is losing.
- Rewrites
- In the visual editor, alternative copy for one element, or a revision of a whole variant from an instruction such as “keep the button, shorten the subhead”.
- Explanations
- A plain-language reading of an experiment's results. It is written from the computed statistics and may only quote numbers that are in them.
Each of these counts as one AI generation against the plan's monthly allowance. When the allowance is used up, or AI is not configured for your installation, the buttons say so and everything else keeps working.
Brand context and guardrails#
The AI writes for your brand from what you tell it. Each site has a brand context, which you can fill in by hand or have drafted from your home page and then correct:
| Field | What it is for |
|---|---|
| Voice | How the brand sounds: “plain, warm, no jargon”. |
| Audience | Who the site sells to. |
| Value propositions | Claims the brand can truthfully make. The AI is told not to invent others. |
| Banned phrases | Words and phrases that must never appear in generated copy. |
| Guardrails | Hard rules: legal and compliance constraints, “never change pricing”, “never mention competitors”. |
| Industry and notes | Anything else worth knowing. |
Independently of what you write there, some limits are built in:
- The AI can only produce the same mutations a person can make in the editor. It cannot add scripts, and its HTML goes through the same sanitizer.
- A proposed change that fails validation is discarded, not repaired. You are never shown a variant that was silently altered to make it pass.
- The content of your page is treated as material to work on, not as instructions. Text on a page cannot tell the AI what to do.
- When you reject a variant, your reason is kept and given to later generations for the site, along with what earlier experiments taught.
Reviewing AI variants#
A variant written by the AI for an experiment that is already running does not go live. It is saved as pending review and served to nobody. People with the member role or higher are notified by email, and the variant.pending_review webhook fires.
For each pending variant you see the changes, the AI's rationale, and a preview link. Then:
- Approve
- The variant becomes active and starts receiving new visitors. You can edit it in the same step. In a bandit experiment it enters at the average weight of the live variants and earns its share from there.
- Reject
- The variant is never served. Give a reason — “we do not offer free returns” — and later generations will take it into account.
Two cases skip the queue. Variants generated for a draft experiment are active immediately, because a draft serves nobody and you review it as a whole before starting. And a site with auto-approve AI variants turned on puts them live without review — see below before enabling that.
What autopilot does#
Autopilot looks at every running experiment that is flagged for it, on sites where it is enabled, and applies the decision rules to the primary goal. Depending on what it finds:
| It finds | It does |
|---|---|
| A sample-ratio mismatch | Flags the experiment and decides nothing until assignment and tracking have been checked. |
| A winner | Promotes it — only when “promote winners automatically” is on. Otherwise it recommends the promotion and waits for you. |
| The original wins, or no variant can plausibly win | Ends the experiment, keeping the original page. |
| A clear loser among several variants | Stops that variant so its traffic goes to the others. It never stops the last remaining challenger this way. |
| Fewer than 2 challengers in play | Asks the AI for new ones, up to 10 variants per experiment. They wait for review unless auto-approve is on. |
| Room for another experiment | Starts one from the idea backlog, and asks for new ideas when the backlog is empty. Without auto-approve it only starts ideas a person wrote or accepted. |
Everything autopilot does goes through the same operations a person would use. Each action is in the audit log under the actor “autopilot”, counts against the same plan limits, and sends the same webhooks and notifications.
Autopilot settings#
Autopilot is configured per site, by an admin. An experiment additionally has its own autopilot flag, so you can leave autopilot on for the site and still run an experiment entirely by hand.
| Setting | Default | Effect |
|---|---|---|
| Autopilot enabled | Off | The master switch. Off means recommendations only. |
| Auto-approve AI variants | Off | AI-written variants go live without review, and autopilot may start experiments from AI ideas nobody has looked at. |
| Promote winners automatically | Off | A winning variant is promoted and rolled out to all targeted visitors without waiting for you. |
| Minimum visitors per variant | 500 | No promote or stop decision before every variant has this many visitors. |
| Win probability | 95% | The P(best) a variant needs to be called the winner. |
| Maximum concurrent experiments | 2 | Autopilot does not start a new experiment while this many are running on the site. |
Before turning on auto-approve
A sensible progression: start with review on and promotion off; turn on automatic promotion once you trust the decision thresholds; turn on auto-approve last, if at all.