These metrics help you judge whether your findings are statistically meaningful and practically relevant — especially useful when analysing raw data exports.
p-Value (Statistical Significance)
The probability that the observed difference occurred by chance. A p < 0.05 is the conventional threshold — less than 5% chance the result is random.
Cohen's d (Effect Size)
Measures how large the difference between two groups is, relative to their variability. Unlike p-values, Cohen’s d tells you how meaningful the difference is.
Confidence Interval (CI)
A range within which the true value likely falls. A 95% CI of [3.2, 4.1] means 95% confidence the real average is between 3.2 and 4.1.
Two fundamental ways a statistical test can go wrong:
Type I Error (False Positive)
You conclude there’s a difference when there isn’t one. Controlled by your significance level (alpha, typically 0.05). Running many tests without correction inflates this risk.
Type II Error (False Negative)
You miss a real difference. Controlled by statistical power, which depends on sample size, effect size, and variance. Larger samples reduce this risk.
