> For the complete documentation index, see [llms.txt](https://seekspeed.gitbook.io/seekspeed-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://seekspeed.gitbook.io/seekspeed-docs/getting-started/core-concepts.md).

# Core Concepts

| Concept            | What it is                                                                                                                  |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------- |
| **Workspace**      | Top-level tenant. Owns connectors, agents, benchmarks, reports. RLS-scoped to the auth user.                                |
| **Connector**      | An OpenAI-compatible endpoint + credentials. Server-side only.                                                              |
| **Agent Profile**  | Prompt + connector + tools + memory mode + token budget. The unit under test.                                               |
| **Benchmark Set**  | A named list of prompts + an agent profile.                                                                                 |
| **Benchmark Run**  | One execution of a set. Captures TTFT, total latency, tokens, cost, errors, per-iteration values.                           |
| **Recommendation** | A rule-generated proposal with expected impact, difficulty, confidence, and an `apply` function that synthesises a variant. |
| **Spec Lab Job**   | A speculative-decoding run with a draft model, target model, and an adapter (vLLM, TGI, llama.cpp, DeepSpec stub).          |
| **Report**         | A frozen snapshot of one or more runs with notes, exportable to JSON/CSV/HTML.                                              |

## TTFT vs throughput

The single most important conceptual point in the entire product:

> **Tokens/second is throughput. TTFT + inter-token latency is responsiveness. They are almost independent.**

A model that pumps 500 tok/s with 400 ms TTFT feels slower in a voice agent than a 150 tok/s model with 60 ms TTFT and smooth pacing. SeekSpeed measures both and forces you to look at both before you ship.

## Statistical significance

Every A/B comparison in SeekSpeed uses Welch's *t*-test (unequal variance). A green "significant" badge means `p < 0.05` across the per-iteration latency distributions of baseline and variant. Without that, an "improvement" might just be noise from a single fast retry.

```ts
// src/lib/stats.ts
export function welchTTest(a: number[], b: number[]) {
  const ma = mean(a), mb = mean(b);
  const va = variance(a), vb = variance(b);
  const se = Math.sqrt(va / a.length + vb / b.length);
  const t  = (ma - mb) / se;
  const df = Math.pow(va/a.length + vb/b.length, 2) /
             (Math.pow(va/a.length, 2)/(a.length-1) + Math.pow(vb/b.length, 2)/(b.length-1));
  return { t, df, pValue: studentT2Tailed(t, df) };
}
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://seekspeed.gitbook.io/seekspeed-docs/getting-started/core-concepts.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.