4. Create an optimization

Define design factors, choose a model type, and link a reward metric to create a live optimization.

An optimization is the core unit in Levered. It defines what you are testing (design factors and their levels), how you measure success (a reward metric), and what model allocates traffic (MAB or CMAB).

Key concepts

Design factor. A variable you want to optimize. Examples: headline, cta_text, hero_image.
Level. A possible value for a design factor. Example: the headline factor might have levels "Ship faster", "Save time", and "Build better".
Variant. A specific combination of levels across all design factors. With 3 headlines and 2 CTA texts, you have 6 variants.

Levered explores these variants automatically and shifts traffic toward the ones that produce the most reward.

Model types

Model	Best for	How it works
MAB (Multi-Armed Bandit)	Simple optimizations, small variant spaces	Treats each variant independently. Learns which variant wins overall.
CMAB (Contextual Multi-Armed Bandit)	Personalization, larger variant spaces	Uses context factors (device, location, user segment) to learn which variant works best for each type of user.

Start with MAB if you are new to Levered. Switch to CMAB when you want to personalize variant selection based on user attributes.

What users outside the optimization see

Decide this before you create it. Part of your traffic, the holdout, stays outside the optimization so Levered can measure the gain (10% by default, set with --holdout). What the holdout sees is the control type:

Control type	When	Flag
`variant` (default)	One of the variants is what users see today. It's the baseline, and the holdout is served it. By default the first level of every factor; name another combination with `--baseline`.	`--control variant`
`external`	The holdout keeps today's version, but that is none of the variants (for example a legacy component). Holdout users never train the model. Needs a holdout.	`--control external`
`none`	Every variant is new. No holdout, no guardrails; lift is measured against an even split of traffic.	`--control none`

Create an optimization

Optimizations are created with the Levered CLI or the Levered plugin for Claude Code, not from the dashboard. (The dashboard's create form exists only in non-production for Levered staff; customers create via the CLI or a coding agent.)

With the Levered plugin installed, describe what you want to test, and Claude builds the factor JSON, picks the model, and runs the CLI for you:

Prompt for Claude Code

Using Levered, create an optimization for my landing page hero: test the headline (Ship faster, Save time, Build better) and the CTA text (Start free, Try it now), optimizing for my "Signup Conversion" metric.

levered optimizations create \
  --name "Landing Page Hero" \
  --design-factors '[
    {"name": "headline", "levels": ["Ship faster", "Save time", "Build better"]},
    {"name": "cta_text", "levels": ["Start free", "Try it now"]}
  ]' \
  --reward-metric-id <metric-uuid> \
  --model-type mab

--reward-metric-id is the metric you created in the previous step (run levered metrics list for the id).
--model-type is mab to start, or cmab for personalization (add --context-factors).
--control and --holdout set what users outside the optimization see (see above). Without them, the control type is variant with a 10% holdout.

The optimization is created as a draft: nothing is served to real users yet. You integrate the SDK, test the setup, and then set it live with levered optimizations update <id> --status live.

What happens after creation

Once you've integrated the SDK and set the optimization live:

Users see variants based on the bandit's current allocation. A share of them, the holdout, keeps today's experience so Levered can measure the gain.
Exposure and reward events land in your warehouse.
Levered trains on a recurring schedule, reads new data from your warehouse, and updates the model.
Traffic shifts toward variants with higher reward rates.

After going live there's nothing to run by hand: the optimization learns as data flows.

Next step

Your optimization is created. Now integrate the SDK to start serving variants and logging exposures.