> For the complete documentation index, see [llms.txt](https://seekspeed.gitbook.io/seekspeed-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://seekspeed.gitbook.io/seekspeed-docs/regression-guard/thresholds.md).

# Thresholds

A **threshold** is what turns a scheduled run into an alert. It answers "how much regression is a regression?"

## Shape

```ts
type Threshold = {
  metric: 'p50_ms' | 'p95_ms' | 'mean_ms' | 'tok_s' | 'cost_usd' | 'error_rate';
  op:     'pct_increase' | 'pct_decrease' | 'abs_gt' | 'pp_increase';
  value:  number;      // interpretation depends on `op`
  severity: 'warn' | 'critical';
};
```

## Operator reference

| Op             | Fires when                                             | Typical use                    |
| -------------- | ------------------------------------------------------ | ------------------------------ |
| `pct_increase` | `observed` is ≥ `value`% above baseline                | `p95_ms +15%` — latency creep  |
| `pct_decrease` | `observed` is ≥ `value`% below baseline                | `tok_s -20%` — throughput drop |
| `abs_gt`       | `observed > value` (raw units)                         | `error_rate > 0.01` — hard SLA |
| `pp_increase`  | `observed − baseline` (in percentage points) ≥ `value` | `error_rate +2pp`              |

## The significance gate

Latency and throughput are noisy. A single tick with a 20% p95 spike could be network jitter. Guard applies **Welch's&#x20;*****t*****-test** on the per-iteration total-latency arrays and refuses to fire a latency/cost/throughput alert unless `p < 0.05` (and both sides have `n ≥ 3`).

Error-rate style metrics don't get this gate — a 5% error rate is a real problem even from a single tick.

```ts
// src/lib/guard-rules.ts (excerpt)
if (needsSignificance(threshold.metric) && pValue != null && pValue >= 0.05) continue;
breached.push({ ... });
```

The alert row records both the raw delta and the *p*-value, so you can audit whether the gate is helping or hurting for your workload.

## Recommended defaults

Applied when you create a schedule without customizing thresholds:

| Metric       | Op             | Value | Severity |
| ------------ | -------------- | ----- | -------- |
| `p95_ms`     | `pct_increase` | 15    | warn     |
| `p95_ms`     | `pct_increase` | 35    | critical |
| `cost_usd`   | `pct_increase` | 20    | warn     |
| `error_rate` | `abs_gt`       | 0.01  | critical |

## Recipes

**Voice / real-time agents** — tail latency dominates:

```
p95_ms   pct_increase  10   critical
mean_ms  pct_increase  20   warn
```

**Batch summarizer** — nobody cares about tail, cost is everything:

```
cost_usd  pct_increase  10   warn
cost_usd  pct_increase  30   critical
tok_s     pct_decrease  15   warn
```

**Prod SLA gate** — coarse but catches meltdowns:

```
error_rate  abs_gt        0.02  critical
p95_ms      abs_gt        2000  critical
```

## Interpreting the alert

Each `guard_alerts` row includes the observed metrics, the baseline metrics, the threshold that fired, the raw delta, `pctChange`, and (for gated metrics) the Welch *p*-value. In the UI, unacknowledged alerts sort first; ack them once you've triaged or fixed the underlying issue.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://seekspeed.gitbook.io/seekspeed-docs/regression-guard/thresholds.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.