ACCURACY BENCHMARK

Validation.

Every statistical method in Analysez AI is checked against independent references: NIST certified values and the same analysis run in R. This page shows the latest results.

Last run: 22 October 2026 · engine 31250c69 · R 4.5.2

80/80

checks matched the reference

methods and safety checks verified

1,643

automated engine tests passing

Results by method.

Each method was run on public reference datasets and compared, number by number, with the certified or R answer. Every method matched on every check.

Method	Reference	Match
Descriptive statistics (mean, SD)	NIST certified values	100% · 9/9
ANOVA F statistic	NIST certified values	100% · 11/11
Linear regression coefficients	NIST certified values	100% · 9/9
t-test (Welch)	R t.test	100% · 2/2
Mann-Whitney U	R wilcox.test	100% · 2/2
One-way ANOVA	R oneway.test / aov	100% · 2/2
Kruskal-Wallis	R kruskal.test	100% · 1/1
Chi-square test	R chisq.test	100% · 4/4
Fisher's exact test	R fisher.test	100% · 1/1
Pearson and Spearman correlation	R cor.test	100% · 3/3
Logistic regression	R glm (binomial)	100% · 1/1
Poisson and negative binomial	R glm / MASS::glm.nb	100% · 2/2
Ordinal and multinomial regression	R MASS::polr / nnet::multinom	100% · 2/2
ROC curve / AUC	R rank AUC	100% · 1/1
Kaplan-Meier and log-rank	R survival::survdiff	100% · 3/3
Cox proportional hazards	R survival::coxph	100% · 3/3
Choosing the right test	Expected method per design	100% · 5/5
Refusing unsafe analyses	Must refuse with a reason	100% · 12/12
Detecting corrupted data	Known injected problems	100% · 6/6

Side by side: reference vs Analysez AI.

Engine values are unrounded; differences appear only after about the 10th significant digit, normal floating-point precision. The last three rows come from a real 128-row survey run through the full workspace and checked again in R; the screen shows them rounded.

Test	Dataset	R / NIST	Analysez AI
Welch's t-test (p)	MASS::birthwt	0.00700254817307696	0.007002548173076962
Chi-square (p)	MASS::birthwt	0.0395769693252326	0.039576969325232546
Kruskal-Wallis (p)	survival::pbc	4.14976262609015e-10	4.1497626260901583e-10
Log-rank (p)	survival::lung	0.00131116452035551	0.0013111645203554897
Cox regression (HR (bilirubin))	survival::pbc	1.14	1.14
Linear regression (B1)	NIST Longley	15.0618722713733	15.061872271541649
Chi-square (Yates) (χ²)	ADR survey	4.987374	4.9874
Chi-square (χ² (df 4))	ADR survey	21.19881	21.1988
Chi-square, Monte Carlo (p)	ADR survey	0.0207	0.020

How the benchmark works

The reference answers are computed once, in R or taken from NIST, and frozen. They are never edited to match the engine.
A case passes only if the engine picks the expected method and every number is within the stated tolerance.
Unsafe analyses (for example, a column with only one value) must be refused with a clear reason. Silent blanks count as failures.
The datasets are public: NIST StRD, and the R packages survival, MASS and medicaldata.

Who sets the reference answers

NIST ↗

The US National Institute of Standards and Technology publishes Statistical Reference Datasets with answers certified to 15 digits. Statistical software is tested against them worldwide.

R ↗

R is the open-source statistics language used by universities, journals and regulators. We run the same analysis in R and compare every number.

← Back to home