Google Online Boutique Benchmark
A reproducible benchmark for adaptive test selection using canonical Quantik Mind™ results.
Benchmark architecture
The Google Online Boutique Benchmark uses the Google Online Boutique services and a Playwright E2E test suite to evaluate adaptive test selection under reproducible conditions.
Quantik Mind™ does not execute the tests. Playwright executes the tests. Quantik Mind™ selects which Playwright tests should run.
The reproduction path is intentionally simple: clone the repository, follow the README, run the documented CLI workflow, and inspect the resulting selection decision in the dashboard.
Methodology and selection process
The benchmark compares selection approaches by measuring test savings and defect recall. Quantik Mind™ also reports runtime-critical risk coverage.
Collect signals
Load test metadata, runtime observability, change context, and service dependency information.
Select tests
Run Quantik Mind™ and baseline approaches against the same benchmark conditions.
Compare outcomes
Measure savings, defect recall, and runtime-critical risk coverage using canonical results.
Reproducible benchmark results
These values are canonical for the Quantik Mind™ Google Online Boutique Benchmark.
Test Savings
The share of the suite not executed because the selected scope carried the useful signal.
Defect Recall
The percentage of known defects still detected by the selected test scope.
Runtime-Critical Risk Coverage
Coverage of tests connected to current runtime risk, service behavior, and dependency impact.
Reproduction guide
Reproduce the benchmark from the Quantik Mind™ benchmark repository. The repository README contains the current CLI workflow and any environment-specific setup.
Clone the benchmark repository.
Follow the repository README.
Run the documented CLI workflow.
Inspect the results in Quantik Mind™.
Run the exact benchmark used to validate Quantik Mind™.
Inspect selected Playwright tests, skipped tests, decision rationale, runtime-critical risk coverage, test savings, and defect recall for the Google Online Boutique Benchmark run.
Ready to run selection on your own codebase?
Install Quantik Mind™ in minutes and start validating the tests that matter most.