Skip to main content
designADR-0040..0044, 0050runnable examples

AI research agents, as a typed operator kind.

The platform already enforces every primitive an AI research agent's outputs would have to honour — bitemporal discipline, citation refusal, content-addressed feature hashes, a named-human promotion gate. The agent work is therefore not "bolt an LLM on": it's name agents as a second operator kindalongside the human operator, give them a typed tool surface, and let the existing gates do their job.

The six ADRs below cover the substrate. Three runnable examples exercise it end-to-end — thepropose example(happy-path proposal pipeline), therefusal-recovery example(what happens when a tool's backend is structurally absent), and thefeature-search example(bounded discovery through the agent surface). All three are contract-tested in CI.

ADR-0040

Agent as first-class operator

Operator(kind='human'|'agent') with a typed identifier. Agents must reference a registered model_artifact; the validator refuses construction otherwise.

Operator type + cited_artifacts: agent:<model>/<run_id> typed prefix

ADR-0041

Agent tool surface

JSON-RPC 2.0 / MCP server over stdio. Read + bounded-discovery tools only. Write tools (register, promote, record_order) return 403 by construction.

McpServer + ToolRefusedError typed refusal contract

ADR-0042

Agent-run eval harness

Three-axis score per agent run: citation quality (must be 1.0), reproducibility (Jaccard floor), backtest IC/Sharpe. Lands on BOOK_PROVENANCE_SCHEMA.verification_score.

AgentRunScore + classify_run_outcome → eligible/flagged/not_promotable/refused

ADR-0043

Cost ceilings + model routing

Per-run token / wall / tool-call ceilings. Per-phase model routing (planning → Opus, tool_calls → Haiku, synthesis → Sonnet). Daily operator-level spend cap.

AgentRunBudget + project_run_cost_usd worst-case projector

ADR-0044

Self-consistency + hallucination refusal

Jaccard similarity between an agent's original and re-run drafts on three axes (features, citations, config). Below the floor → flagged for operator review.

compute_self_consistency + classify_prompt_injection

ADR-0050

Typed refusal vocabulary

Backend-state absence is a typed refusal (no_state_root), never an empty result. Every tool publishes its refusal_reasons slugs in the catalogue — drift-tested against the handler source.

_open_book_state() + ToolDefinition.refusal_reasons

Citation refusal at propose-time

A proposal citing a slug the writing manifest doesn't resolve is rejected before any row lands. The most common hallucination shape — fabricated source attribution — is refused by construction.

Bitemporal floor at the tool surface

features.materialize(as_of=...) refuses any timestamp greater than utc_now(). Agents cannot read future data; the data layer refuses, not the prompt.

Self-consistency check before promotion

A proposal's feature set / cited notes / inputs hash are re-generated at a higher temperature. Drift past the Jaccard threshold flips the outcome from eligible to flagged for operator review.

Human approver between propose and live

Agents write Provenance rows. They never call record_order. The ADR-0019 promotion contract still requires a named human approver to flip the live switch.

ADR-0041's bundled tool catalogue, snapshotted at build time from alphakernel.server.mcp.builtin_tools(). A Python drift test (test_catalogue_matches_live) keeps this list in sync with the live JSON-RPC server. Write tools (register,promote,record_order) are not on this surface — the agent reads broadly but writes only to its own leaderboard entry.

  • backtest.run

    2 required · 2 optional

    Execute a walk-forward backtest against a registered model + feature set (ADR-0005). Returns the EvalReport shape every discovery method emits (ADR-0023). Body lands in #IMPL-0041 follow-up; argument validation is live.

    strategy_slug · required
    string — Registered strategy slug.
    features · required
    array — Feature names from the alphakernel DAG.
    horizon · optional
    integer — Forward-return horizon (bars).
    max_bars · optional
    integer — Max bars in the panel ([1, 100000]). Higher counts force chunked invocation.
    refuses with
    • bad_strategy_slug
    • bad_features
    • bad_horizon
    • bad_max_bars
    • not_implemented
  • book.attribution_by_thesis

    0 required · 2 optional

    Roll up realised PnL bucketed by cited research note (ADR-0013/ADR-0014). Reads the BookState rooted at `$ALPHAKERNEL_BOOK_STATE_ROOT`; refuses with `no_state_root` when unset. Optional ISO-8601 since/until window bounded by the ADR-0016 bitemporal floor — since maps to the window start, until to as_of. Read-only. Returns rows sorted by absolute realised PnL descending.

    since · optional
    string · date-time — ISO-8601 lower bound on the attribution window (UTC).
    until · optional
    string · date-time — ISO-8601 upper bound (bitemporal as_of, UTC).
    refuses with
    • bad_since
    • bad_until
    • since_in_future
    • until_in_future
    • bad_window
    • no_state_root
  • book.recent_provenance

    0 required · 2 optional

    List the most recent decision-provenance rows from the book (ADR-0011/ADR-0040). Returns each row's event_id, exchange_ts, strategy_slug, operator_kind, cited_artifacts. Reads the BookState rooted at `$ALPHAKERNEL_BOOK_STATE_ROOT`; refuses with `no_state_root` when the env var is unset. Read-only — agents cannot write provenance through this surface.

    limit · optional
    integer — Max rows to return ([1, 100]). Defaults to the runner's chosen page size.
    strategy · optional
    string — Optional strategy_slug filter — return only rows produced by this strategy.
    refuses with
    • bad_limit
    • bad_strategy
    • no_state_root
  • discovery.event_window_search

    3 required · 3 optional

    Search pre × post × reduction event-window configurations against a synth target panel + seeded synthetic event stream (ADR-0032 / ADR-0023). Reductions: indicator, count, mean_score, time_since_last. Ranks by Spearman rank-IC; candidates with fewer than min_events matches are excluded.

    pre_windows · required
    array — Polars duration strings — '0m', '5m', '1d'.
    post_windows · required
    array — Same shape as pre_windows.
    reductions · required
    array — Window reductions to try (ADR-0032 closed set).
    target · optional
    string — Target column — a panel base column or registered feature (default fwd_ret_1).
    max_candidates · optional
    integer — Cap on pre × post × reductions product.
    min_events · optional
    integer — Small-sample floor — candidates below it are excluded.
    refuses with
    • bad_pre_windows
    • bad_post_windows
    • bad_reductions
    • bad_target
    • bad_max_candidates
    • bad_min_events
    • cross_product_too_large
    • bad_spec
  • discovery.factor_mining

    2 required · 3 optional

    Randomised symbolic-expression factor mining over a synthetic L1 panel (ADR-0023). Builds short alpha expressions from `inputs` and ranks by |rank_ic| or signed rank_ic against `target`. Returns the top-20 leaderboard. Bounded by spec — over-wide requests refused at construction. MCP shares the HTTP service's threat model; CLI callers can use wider bounds via `FactorMiningSpec` directly.

    inputs · required
    array — Input column names from the panel.
    target · required
    string — Target column on the panel (e.g. fwd_ret_5).
    max_depth · optional
    integer — Max expression-tree depth.
    n_candidates · optional
    integer — Random candidate count.
    objective · optional
    string — Score the candidate by IC magnitude or signed IC.
    refuses with
    • bad_inputs
    • bad_target
    • bad_max_depth
    • bad_n_candidates
    • bad_objective
    • bad_spec
    • unknown_feature
  • discovery.feature_search

    1 required · 4 optional

    Greedy forward-selection or exhaustive subset search over a pool of features against a default ridge/walk-forward Model (ADR-0023). Returns a leaderboard ranked by `objective`. The estimator + CV windows are operator-set (not agent-chosen); agents drop to the CLI to vary them. Bounded by spec — over-wide requests refused at construction.

    pool · required
    array — Candidate feature names (panel base columns or registered features).
    method · optional
    string — Greedy forward selection or full subset walk.
    min_features · optional
    integer — Minimum subset size.
    max_features · optional
    integer — Max subset size — bounds combinatorial growth.
    objective · optional
    string — Score per evaluated subset.
    refuses with
    • bad_pool
    • bad_method
    • bad_max_features
    • bad_min_features
    • bad_objective
    • bad_spec
    • unknown_feature
  • discovery.sweep

    1 required · 4 optional

    Grid or seeded-random sweep over a default Model's hyperparameters (ADR-0023). Returns a leaderboard ranked by `objective`. Estimator + base CV are operator-set; agents drop to the CLI to vary the baseline. Seed + panel dims are fixed — agents only vary the listed dims.

    dims · required
    array — Search axes; each {name, choices}.
    method · optional
    string — Cartesian product or seeded random draw.
    n_random · optional
    integer — Draw count when method=random.
    max_configs · optional
    integer — Hard ceiling on the grid product.
    objective · optional
    string — Score per evaluated config.
    refuses with
    • bad_dims
    • bad_dim_name
    • bad_method
    • bad_max_configs
    • bad_n_random
    • bad_objective
    • bad_spec
    • unknown_feature
  • features.list

    no params

    Enumerate every registered feature in the alphakernel DAG (ADR-0006). Returns name, doc, lag, is_label, deps, content-addressed hash, license_tier, and namespace per feature. A non-empty namespace (e.g. 'kr') marks a feature outside the generic universe — generic callers should filter to namespace == '' unless they specifically want it.

  • features.materialize

    2 required · 0 optional

    Materialise a feature panel as-of a given timestamp (ADR-0016 PIT). Refuses as_of in the future. Body lands in #IMPL-0041 follow-up PR; currently returns a typed not_implemented refusal.

    slug · required
    string — Feature name.
    as_of · required
    string · date-time — ISO-8601 timestamp (UTC). Must be <= now.
    refuses with
    • missing_as_of
    • bad_as_of
    • as_of_in_future
    • not_implemented
  • thermometer.latest

    no params

    Return the latest immutable AI supply-demand thermometer print: two axes, categorical verdict, 0-100 diffusion heat index, agreement/dispersion, freshness, component provenance, and methodology hash (ADR-0100).

    refuses with
    • no_thermometer_print
  • writing.get_note

    1 required · 0 optional

    Fetch one writing entry by slug. Refuses unknown slugs with a typed error so agents cannot propose citations against missing or hallucinated notes (ADR-0015).

    slug · required
    string — Writing note slug (kebab-case identifier).
    refuses with
    • missing_slug
    • unknown_slug
  • writing.list_notes

    no params

    Enumerate every published research note (ADR-0012). Returns slug, content hash, status, title, published date, visibility, and superseded_by per entry — the same shape the ADR-0015 citation validator resolves against.

Refusal vocabulary followsADR-0050: every tool publishes the typedreason slugs its handler may raise, so a planner can branch on a known vocabulary instead of parsing prose. Absent backend = typed refusal (no_state_root), never an empty result.

decision
ADR-0050
Backend absence is a typed refusal, not an empty result.
design note
refusal as planning hint
Why typed slugs are the part of a tool surface that decides whether an agent can plan.
thesis
audit isn't a feature
Why the same substrate that answers the auditor is what lets agents compose with humans.
runnable
examples/agent_handles_no_state_root
A no-op planner that hits no_state_root and routes around it. Contract-tested in CI.

Same tool, same arguments — the only difference between the two payloads is whether ${ALPHAKERNEL_BOOK_STATE_ROOT} resolved. The planner branches on _meta.reason, never on the message text or the absence of rows.

happy path

isError: false
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "isError": false,
    "structuredContent": [
      {
        "event_id": "0195a1c2-...",
        "exchange_ts": "2026-05-21T09:14:30Z",
        "strategy_slug": "xs_momentum_live",
        "operator_kind": "agent",
        "cited_artifacts": [
          "note:walk-forward-without-leakage",
          "agent:claude-opus-4-7/0195a1c2-..."
        ]
      }
    ]
  }
}

refused

_meta.reason: no_state_root
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "isError": true,
    "_meta": { "reason": "no_state_root" },
    "content": [
      {
        "type": "text",
        "text": "ALPHAKERNEL_BOOK_STATE_ROOT is unset — the agent has no book to read."
      }
    ]
  }
}
1 of 5
all runs →

Rendered from theexamples/agent_propose_xs_momentum/sample_run*.json fixtures — actual output ofpython agent.py with pinned UUIDs and clock. A drift test (test_sample_run_matches_live) re-runs the agent against every pinned config and asserts bytewise equality, so this tile cannot fall behind the live code.

n_buckets
5
features = ["momentum_126", "momentum_to_vol_ratio_20"]
horizon = 5, rebalance = "daily"
config_id = 8a4c09dc05fb

A no-op agent that walks the full pipeline. Run it from a clone:

$ python examples/agent_propose_xs_momentum/agent.py --out-dir agent_out

agent operator       = claude-opus-4-7/01951b40-...
provenance event_id  = 01951b40-...
cited artifacts      = ('note:walk-forward-without-leakage',
                       'note:the-promotion-gate',
                       'agent:claude-opus-4-7/01951b40-...')
leaderboard entries  = 1
book directory       = agent_out
leaderboard json     = agent_out/leaderboard.json
  1. Operator binds

    Operator(kind='agent', model_artifact='claude-opus-4-7', identifier='<model>/<uuid7>') — pydantic validator refuses construction without a model artifact.

  2. Enumerate features via MCP

    builtin_tools()['features.list'].handler({}) — same callable JSON-RPC tools/call dispatches to. No subprocess, no network.

  3. Enumerate notes via MCP, refuse hallucinated citations

    builtin_tools()['writing.list_notes'].handler({}) returns every published slug. Cited notes outside that set raise ValueError before any write — ADR-0015 enforced at the tool boundary.

  4. Land one provenance row

    cited_artifacts = (note:walk-forward-without-leakage, note:the-promotion-gate, agent:claude-opus-4-7/<run_id>). One write, no orders. Promotion stays a human-only gate (ADR-0019).

  • 2026 frontier models still hallucinate in domain tasks(Stanford AI Index 2026 reports 22–94% rates depending on task). The platform's defence isn't that the agent won't hallucinate — it's that hallucinations get refused before they reach any artefact.
  • The promotion gate is the load-bearing piece.Without a named human approver between the agent's leaderboard entry and the live runner (ADR-0019), the entire stack would be a different kind of thing. The platform commits to keeping the human in the loop.
  • The differentiator is the substrate, not the agent."AI doing research" has been done many times (Quantopian had bots, Numerai is essentially crowdsourced ML). What's different here is that the agent operates on a citable, PIT-honest, schema-strict substrate. Its runs are reproducible; its outputs are auditable; its cost is bounded.
deskgdfindgfstrategiesgsfeaturesgepromotionsgpapigabotgbwritinggw

Shortcuts are no-ops while typing in an input or textarea.