# Winnow > Winnow is an LLM evaluation, experimentation, and observability platform. It unifies offline eval, online experimentation (A/B tests with mSPRT / Welch / Bayesian / CUPED), feature gates and dynamic configs, real-time guardrails, a reasoning agent that issues structured second opinions, and a governed MCP surface for IDE agents. The single-sentence mental model: every AI behavior change is either a gate (boolean), a config (params), a prompt (content), or a model (provider) — Winnow tracks them all under one experiment primitive with one stats engine and one observability surface. ## For agents: start here (the MCP door) If you are an AI agent, the sanctioned way to use Winnow is the **MCP server**, not scraping this site or the app. It is authenticated, org-scoped, rate-limited, revocable, and metered — the path of least resistance for real work, and the only supported programmatic surface for high-volume access. Package availability is separate from application deployment. If npm returns 404 for `@winnow/mcp-server`, the public distribution is not live; use a source checkout or wait for the guarded package-release receipt before using the `npx` configuration. - [MCP Server quickstart](https://justwinnow.com/product/mcp#quickstart): connect Claude Code / Cursor / Zed with an `evmcp_` least-privilege key, then use standard discovery. Start here. - [MCP action boundary](https://justwinnow.com/product/mcp#action-boundary): inspect the connected server's READ, DRAFT/HANDOFF, and WRITE descriptors before selecting an operation. - [get_agent_second_opinion](https://justwinnow.com/product/mcp#second-opinion): request a structured second opinion on an experiment from inside the IDE. - [get_billing_status](https://justwinnow.com/docs#operate): check plan, usage, and remaining quota before spending it. - [ask_agent](https://justwinnow.com/product/mcp#agent-bridge): pull a comprehensive context bundle for the host LLM to reason over. Tool discovery is capability visibility, not approval. Key scope, workspace, backend policy, and any required owner receipt are enforced independently. Recommendations and proposals do not authorize production changes; open the authenticated Winnow owner for consequential approval or delivery control. Core is the default when `WINNOW_MCP_PROFILE` is unset or blank and exposes the 36-tool lifecycle facade. The variable accepts one explicit value: `core`, `agent-evals`, or `legacy`; `agent-evals` exposes 42 tools, while explicit `legacy` preserves the 123-tool compatibility escape hatch. The compact profiles pass the current release payload and cold-list latency gates. Restart the MCP host after changing it and use standard discovery rather than combining profiles or relying on a copied tool count. ## Getting Started (how to use Winnow) - [Install and authenticate](https://justwinnow.com/docs#setup): pip install winnow-sdk, export WINNOW_API_KEY, first request in 60 seconds. - [Five-minute quickstart](https://justwinnow.com/docs#setup): create an experiment, log observations, read stats. - [Pricing and plans](https://justwinnow.com/pricing): Starter $0, Pro $99/seat, Team $349, Enterprise. ## Core Concepts - [Experiment](https://justwinnow.com/docs#evaluate): the top-level unit. Arms, traffic split, primary metric, stat method, horizon. - [Arm](https://justwinnow.com/docs#evaluate): a variant of model, prompt, or config. Carries its own observations and stats. - [Observation](https://justwinnow.com/docs#monitor): one scored output. Score, latency, cost, custom metrics. - [Guardrail](https://justwinnow.com/docs#monitor): a metric threshold that can auto-stop an experiment when breached. - [Gate and Dynamic Config](https://justwinnow.com/docs#ship): boolean and JSON-valued decisions, evaluated per-user, with rule stacks. - [Second opinion](https://justwinnow.com/docs#monitor): LLM-backed structured analysis with claims, evidence pointers, and refutation conditions. ## SDK Reference - [Python SDK](https://justwinnow.com/docs#reference): Eval("Project", { data, task, scores }) one-liner. Experiments, gates, configs. - [TypeScript SDK](https://justwinnow.com/docs#reference): same primitives, typed. - [REST API](https://justwinnow.com/docs#reference): documented in OpenAPI at /openapi.json on every deployment. ## Statistical rigor (applied for you — no stats expertise required) Winnow automatically applies the right method and corrections; you don't implement them. Behind one experiment primitive it uses **mSPRT** (anytime-valid sequential testing), **Welch's t** (fixed-horizon), **Bayesian** posteriors, **CUPED** variance reduction, and **chi-square SRM** detection, with multiple-comparison correction and honest confidence intervals. Access all of this through the SDK or the MCP door above — the methods are chosen and computed server-side. ## Optional - [Compare: Winnow vs Braintrust](https://justwinnow.com/compare/winnow-vs-braintrust) - [Compare: Winnow vs Statsig](https://justwinnow.com/compare/winnow-vs-statsig) - [Changelog](https://justwinnow.com/changelog) - [llms-full.txt (usage corpus for ingestion)](https://justwinnow.com/docs/llms-full.txt)