Optimizations
The top-level entity in Levered -- a specific product decision you want to optimize using multi-armed bandits.
An optimization is the core unit of work in Levered. It represents a specific product decision you want to improve: which headline converts best, which checkout flow drives the most revenue, which pricing display maximizes sign-ups.
What an optimization contains
Each optimization bundles everything Levered needs to run and improve a decision:
- Design factors -- the variables you control (headline text, button color, layout) and their possible values. See Design Factors.
- A model -- the bandit that learns which variants perform best. See Models.
- Linked metrics -- the reward signals that define success, written as SQL queries against your warehouse.
- Training data config -- how Levered connects exposures to rewards, including the conversion window and warehouse tables.
Example
Suppose you want to optimize a landing page hero section. You might create an optimization with:
- Design factors:
headline(5 options) andcta_text(2 options), producing 10 variants. - Reward metric: a SQL query that counts sign-ups within 24 hours of seeing the page.
- Model: a CMAB that personalizes by country and device type.
Once live, the SDK serves the best variant for each visitor, and the model retrains as new data flows into your warehouse.
Status lifecycle
Every optimization moves through a simple lifecycle:
| Status | Meaning |
|---|---|
live | Actively serving variants and collecting data. The model retrains periodically as new observations arrive. |
completed | The optimization has concluded. Levered locks in the winning variant(s) and stops exploring. You can still query results and serve the winner. |
archived | No longer active or visible in the default dashboard view. Data is retained for reference. |
How this differs from A/B testing
Traditional A/B tests have a fixed design: you choose a sample size, split traffic evenly, wait for statistical significance, then manually pick a winner. This means a lot of traffic goes to underperforming variants while you wait.
Levered optimizations work differently:
- Continuous optimization. The model starts shifting traffic toward better variants as soon as it has enough data. There is no waiting period.
- No fixed sample size. You do not need to calculate a required sample size upfront. The bandit adapts as data accumulates.
- Automatic traffic allocation. Variants that perform well get more traffic. Variants that underperform get less. This happens automatically via Thompson Sampling.
- Exploration never fully stops. Even while exploiting the best variant, the bandit occasionally explores alternatives. If user behavior shifts, the model adapts.
The result: faster decisions, less wasted traffic, and the ability to keep optimizing as conditions change.