Commands

All commands support --json for structured output unless noted otherwise. Use levered <command> --help for inline documentation on any command.

Authentication

Manage your session and credentials.

Configuration

Read and write CLI settings stored in ~/.levered/config.json.

Optimizations

Create, manage, and inspect optimizations.

List optimizations

levered optimizations list [--limit N] [--include-archived] [--json]

List optimizations in the current organization. Returns the most recent optimizations, up to --limit (default 10). Archived optimizations are hidden unless --include-archived is set.

Show optimization details

levered optimizations show <id> [--json]

Print full details for a single optimization, including design factors, model configuration, linked metric, and current status.

Create an optimization

levered optimizations create \
  --name NAME \
  --design-factors JSON \
  --reward-metric-id UUID \
  [--model-type cmab] \
  [--context-factors JSON] \
  [--control variant|external|none] \
  [--holdout PERCENT] \
  [--baseline JSON] \
  [--json]

Create a new optimization as a draft.

Example. Create a hero section optimization with two design factors:

levered optimizations create \
  --name "Landing Page Hero" \
  --design-factors '[
    {"name": "headline", "levels": ["Ship faster", "Build better", "Scale up", "Go live today", "Start free"]},
    {"name": "cta_text", "levels": ["Get started", "Try it free"]}
  ]' \
  --reward-metric-id 8a3b12c4-5678-4def-abcd-1234567890ab \
  --context-factors '[{"name": "country"}, {"name": "device_type"}]' \
  --json

Update optimization status

levered optimizations update <id> --status {live|paused|shipped|completed|archived}

Transition an optimization to a new status. See Optimizations for the status lifecycle.

Turn the baseline on or off

levered optimizations update <id> --no-baseline
levered optimizations update <id> --baseline

--no-baseline says every variant is new: lift is reported against an even split of traffic instead of the baseline variant (every factor at its first level). The holdout must be 0 and all guardrails removed first; otherwise the API refuses the change and the CLI prints why. --baseline makes the all-first-levels variant the baseline again.

Ship the winner

levered optimizations ship <id>

Serves the leading variant for each context to 100% of traffic. Learning stops, the holdout is removed, and results are frozen. Prints the shipped policy: the variant each context segment is served. Pass -y to skip the confirmation and --json to get the policy as JSON.

To go back to learning, run levered optimizations update <id> --status live. This restores the holdout.

Stop an optimization

levered optimizations stop <id>

Stops the optimization for good (status completed). Levered stops serving variants, so users get the fallback defined in your app. This cannot be undone. Pass -y to skip the confirmation.

Archive an optimization

levered optimizations archive <id> [-y]

Archive an optimization. Prompts for confirmation unless -y is passed.

View results

levered optimizations results <id> [--json]

Print performance results for an optimization: exposures and rates per variant, measured lift against the holdout, and the model's figures: policy and best-variant lift with their intervals, each variant's probability of being best and expected reward, and importance. --json returns the API's results with the model figures under model, in one shape for every optimization; model.source says where they came from.

The figures come from the posterior, where importance is a lift for every design factor, context factor and their interaction, on one scale. An optimization created before the posterior store (model_source: "bandit") shows the model's own figures as before, with design-factor importance as a share of the reward's variance.

View lift history

levered optimizations lift-history <id> [--days N] [--json]

Print the model's policy and best-variant lift, one point per day: the last posterior of each day, or the daily snapshots of an optimization created before the posterior store. --days limits it to a trailing window.

View progress

levered optimizations progress <id> [--matured-only] [--json]

Show how far along an experiment is: days running (from first measured traffic), enrollment per arm as unique users and converters with the holdout share, and convergence, covering progress to best, P(beats baseline), and the variant the model currently ranks highest with its probability of being best (its share of optimized traffic).

Enrollment counts are participant-level (a user exposed or converting multiple times counts once) and come from the measured holdout series; they are not comparable to the bandit's per-event exposure counts shown in results.

View guardrails

levered optimizations guardrails <id> [--matured-only] [--json]

Show each guardrail metric's bandit and holdout rates, the measured lift with its 95% confidence interval and p-value, and whether the guardrail is ok, violated, or still pending data, plus the overall rollup status.

Capture variant screenshots

levered optimizations screenshot <id> --url <page-url> \
  [--selector <css>] [--viewport 1280x800] [--wait-ms 800] \
  [--full-page] [--only 1,4] [--out-dir <dir>] [--no-upload]

Screenshot every servable variant of an optimization against a running app and upload the images to Levered, so each variant shows a visual preview in the dashboard's variants table.

For each variant, the command opens the page in a headless browser, intercepts the SDK's serve call, and forces that variant's factor values, so the page renders exactly what users in that arm see. Point --url at your local dev server (or a staging URL); the page must already be wired up with useVariant for this optimization.

Requires Node.js on PATH. The first run installs a pinned Playwright and downloads the Chromium browser (~130 MB, one-time) into ~/.levered/screenshot-runner/. Disabled and excluded variants are skipped, and re-running overwrites the previous screenshots.

List observations

levered optimizations observations <id> [--page N] [--size N] [--json]

Paginate through the raw observation data (exposures matched to rewards) for an optimization.

Models

Inspect and manage the bandit models behind optimizations.

Posterior

levered posterior query <optimization-id> --sql '<sql>' [--param name=value] [--version <v>] [--json]

Run read-only SQL against an optimization's posterior: the source of every model figure the dashboard and CLI show. See the posterior API for the tables, views and standard queries.

Warehouse

Inspect and query your data warehouse. Warehouse connections are managed through the dashboard or the API.

These commands are especially useful for AI agents that need to discover table schemas before writing metric SQL.

Metrics

Define reward metrics that tell Levered what counts as a successful outcome.

List metrics

levered metrics list [--json]

Create a metric

levered metrics create \
  --name NAME \
  --sql SQL \
  --anonymous-id-col COL \
  --timestamp-col COL \
  [--value-col COL] \
  [--reward-type TYPE] \
  [--json]

Example. Create a sign-up conversion metric:

levered metrics create \
  --name "Sign-ups (7d)" \
  --sql "SELECT anonymous_id, signed_up_at FROM analytics.sign_ups WHERE signed_up_at >= CURRENT_DATE - 7" \
  --anonymous-id-col anonymous_id \
  --timestamp-col signed_up_at \
  --json

Preview metric data

levered metrics preview <id> [--limit N] [--json]

Run the metric's SQL query and display a sample of matching rows. Useful for verifying the query returns the expected data before linking the metric to an optimization.

Delete a metric

levered metrics delete <id> [-y]

Delete a metric. Prompts for confirmation unless -y is passed. Metrics linked to active optimizations cannot be deleted.

Variant Serving

Request a variant assignment for a user outside of the SDK.

levered serve <optimization-id> [--anonymous-id ID] [--context JSON] [--dry-run] [--json]

Returns the selected variant, meaning the set of factor-level assignments, for the given user and context, and is_holdout.

Verify

Check an optimization's setup before launch.

levered verify <optimization-id> [--anonymous-id ID] [--timeout SECONDS] [--days N] [--json]

Always runs the identity check on the last --days (default 14) days of your exposures table: the share of exposure ids that also appear in the reward metric, and, if the table has an is_holdout column, the share of is_holdout rows that Levered's holdout hash also puts in the holdout. A low share warns of a possible id mismatch; interpret it using representative non-test data, as described in Test before launch.

With --anonymous-id on the Managed Warehouse, it first waits up to --timeout seconds (default 120) for a test exposure and a test reward from that user. JSON output includes exposure and reward for arrival, plus the aggregate report under identity. Top-level ok is true only when arrival and identity.ok are both true.

Exits with status 1 when either check fails or an API error occurs. Inspect the JSON fields and errors rather than treating every exit 1 as a missing event. If arrival succeeds but aggregate reward overlap cannot be established solely because representative non-test data is absent or QA rewards are intentionally excluded, record identity as pending under the test-before-launch rules. Query/authentication errors, failed holdout checks, and failures with representative data still block. An authenticated managed event-arrival read returns only the event booleans; it does not mark aggregate identity as passed.

Interactive Dashboard

levered dashboard [--refresh SECONDS]

Launch a terminal UI (TUI) that displays live optimization data, including variant distributions, reward rates, and model state. The display refreshes on the interval specified by --refresh (default: 10 seconds).

This is a read-only view. Use it to monitor running optimizations without leaving the terminal.