benchmark_baseline.ts

Benchmark baseline storage and comparison utilities. Save benchmark results to disk and compare against baselines for regression detection.

Declarations
#

12 declarations

view source

benchmark_baseline_compare
#

benchmark_baseline.ts view source

(results: BenchmarkResult[], options?: BenchmarkBaselineCompareOptions): Promise<BenchmarkBaselineComparisonResult>

Compare benchmark results against the stored baseline.

results

current benchmark results

type BenchmarkResult[]

options

comparison options including regression threshold and staleness warning

default {}

returns

Promise<BenchmarkBaselineComparisonResult>

comparison result with regressions, improvements, and unchanged tasks

examples

const bench = new Benchmark(); bench.add('test', () => fn()); await bench.run(); const comparison = await benchmark_baseline_compare(bench.results(), { regression_threshold: 1.05, // Only flag regressions 5% or more slower staleness_warning_days: 7, // Warn if baseline is older than 7 days }); if (comparison.regressions.length > 0) { console.log('Performance regressions detected!'); for (const r of comparison.regressions) { console.log(` ${r.name}: ${r.comparison.speedup_ratio.toFixed(2)}x slower`); } process.exit(1); }

benchmark_baseline_format
#

benchmark_baseline_format_json
#

benchmark_baseline_load
#

benchmark_baseline.ts view source

(options?: BenchmarkBaselineLoadOptions): Promise<{ version: number; timestamp: string; git_commit: string | null; git_branch: string | null; node_version: string; entries: { ...; }[]; } | null>

Load the current baseline from disk.

options

load options

default {}

returns

Promise<{ version: number; timestamp: string; git_commit: string | null; git_branch: string | null; node_version: string; entries: { name: string; mean_ns: number; p50_ns: number; std_dev_ns: number; ... 7 more ...; sample_size: number; }[]; } | null>

the baseline, or null if not found or invalid

examples

const baseline = await benchmark_baseline_load(); if (baseline) { console.log(`Baseline from ${baseline.timestamp}`); }

benchmark_baseline_save
#

benchmark_baseline.ts view source

(results: BenchmarkResult[], options?: BenchmarkBaselineSaveOptions): Promise<void>

Save benchmark results as the current baseline.

results

benchmark results to save

type BenchmarkResult[]

options

save options

default {}

returns

Promise<void>

examples

const bench = new Benchmark(); bench.add('test', () => fn()); await bench.run(); await benchmark_baseline_save(bench.results());

BenchmarkBaseline
#

benchmark_baseline.ts view source

ZodObject<{ version: ZodNumber; timestamp: ZodString; git_commit: ZodNullable<ZodString>; git_branch: ZodNullable<ZodString>; node_version: ZodString; entries: ZodArray<...>; }, $strip>

Schema for the complete baseline file.

BenchmarkBaselineCompareOptions
#

benchmark_baseline.ts view source

BenchmarkBaselineCompareOptions

Options for comparing against a baseline.

inheritance

regression_threshold

Minimum speedup ratio to consider a regression. For example, 1.05 means only flag regressions that are 5% or more slower. Default: 1.0 (any statistically significant slowdown is a regression)

type number

staleness_warning_days

Number of days after which to warn about stale baseline. Default: undefined (no staleness warning)

type number

min_percent_difference

Minimum percentage difference to consider meaningful, as a ratio. Passed through to benchmark_stats_compare. See BenchmarkCompareOptions. Default: 0.10 (10%)

type number

BenchmarkBaselineComparisonResult
#

benchmark_baseline.ts view source

BenchmarkBaselineComparisonResult

Result of comparing current results against a baseline.

baseline_found

Whether a baseline was found

type boolean

baseline_timestamp

Timestamp of the baseline

type string | null

baseline_commit

Git commit of the baseline

type string | null

baseline_age_days

Age of the baseline in days

type number | null

baseline_stale

Whether the baseline is considered stale based on staleness_warning_days option

type boolean

comparisons

Individual task comparisons

type Array<BenchmarkBaselineTaskComparison>

regressions

Tasks that regressed (slower with statistical significance), sorted by effect size (largest first)

type Array<BenchmarkBaselineTaskComparison>

improvements

Tasks that improved (faster with statistical significance), sorted by effect size (largest first)

type Array<BenchmarkBaselineTaskComparison>

unchanged

Tasks with no significant change

type Array<BenchmarkBaselineTaskComparison>

new_tasks

Tasks in current run but not in baseline

type Array<string>

removed_tasks

Tasks in baseline but not in current run

type Array<string>

BenchmarkBaselineEntry
#

benchmark_baseline.ts view source

ZodObject<{ name: ZodString; mean_ns: ZodNumber; p50_ns: ZodNumber; std_dev_ns: ZodNumber; min_ns: ZodNumber; max_ns: ZodNumber; ... 5 more ...; sample_size: ZodNumber; }, $strip>

Schema for a single benchmark entry in the baseline.

BenchmarkBaselineLoadOptions
#

benchmark_baseline.ts view source

BenchmarkBaselineLoadOptions

Options for loading a baseline.

path

Directory to load baseline from (default: '.gro/benchmarks')

type string

BenchmarkBaselineSaveOptions
#

benchmark_baseline.ts view source

BenchmarkBaselineSaveOptions

Options for saving a baseline.

path

Directory to store baselines (default: '.gro/benchmarks')

type string

git_commit

Git commit hash (auto-detected if not provided)

type string | null

git_branch

Git branch name (auto-detected if not provided)

type string | null

BenchmarkBaselineTaskComparison
#

Depends on
#