01
Hypothesis Testing
Decide if observed data supports a claim (H₁) or can be explained by chance (H₀) using p-values and significance thresholds.
H₀ = null (no effect)
H₁ = alternative (your claim)
p < 0.05 → reject H₀
Type I error = α · Type II = β
A/B Test Simulator — CTR Comparison
Control CTR 12%
Treatment CTR 15%
Sample size (n per group) 1,000
Control CTR
—
Treatment CTR
—
p-value
—
Lift
—
from scipy import stats import numpy as np # A/B test: CTR comparison control = np.random.binomial(1, 0.12, 1000) # 12% CTR treatment = np.random.binomial(1, 0.15, 1000) # 15% CTR t_stat, p_val = stats.ttest_ind(control, treatment) print(f"p-value: {p_val:.4f}") # p < 0.05 -> reject H0, treatment wins print("Significant!" if p_val < 0.05 else "Not significant")
02
Confidence Intervals
A range that, over repeated experiments, would contain the true population parameter ~95% of the time. It describes the estimation procedure, not one interval.
CI = x̄ ± z* × (σ/√n)
95% CI → z*=1.96
Wider CI = less data or more variance
More uncertainty → collect more data
CI = x̄ ± t* × (s / √n)
Repeated Sampling Simulator
Sample size (n) 10
Confidence level 95%
Contains true mean
Misses true mean
Click "Run 30 samples" to begin
from scipy import stats import numpy as np data = np.array([23,25,28,22,27,30,24,26,21,29]) mean = data.mean() se = stats.sem(data) ci = stats.t.interval(0.95, df=len(data)-1, loc=mean, scale=se) print(f"Mean: {mean:.2f}") print(f"95% CI: ({ci[0]:.2f}, {ci[1]:.2f})")
03
Bayes' Theorem
Updates prior beliefs with new evidence — the foundation of probabilistic reasoning and the base rate fallacy that trips up interviewers.
P(A|B) = P(B|A)·P(A) / P(B)
Prior → Likelihood → Posterior
Base rate fallacy: rare disease trap
99% accurate ≠ 99% PPV
P(disease | positive) = P(pos | disease) × P(disease) / P(positive)
Medical Test — Base Rate Fallacy
Prevalence 1.0%
Sensitivity 99%
Specificity 95%
True Positives
—
False Positives
—
All Positives
—
FP Rate
—
P(DISEASE | POSITIVE TEST)
—
—
# Medical test — base rate fallacy p_disease = 0.01 # 1% prevalence p_pos_given_disease = 0.99 # sensitivity p_pos_given_healthy = 0.05 # 1 - specificity p_pos = (p_pos_given_disease * p_disease + p_pos_given_healthy * (1 - p_disease)) ppv = (p_pos_given_disease * p_disease) / p_pos print(f"P(disease | positive test): {ppv:.1%}") # ~16.5% despite 99% accurate test!
04
Distributions & CLT
Different distributions model different real-world phenomena. The Central Limit Theorem guarantees sample means converge to Normal regardless of population shape.
Normal: heights, errors
Poisson: event counts
Binomial: binary outcomes
CLT: n≥30 → Normal means
Distribution Explorer
—
from scipy.stats import poisson, binom # Poisson: P(exactly k tickets in a day) lam = 10 # avg 10 tickets/day print(f"P(8 tickets): {poisson.pmf(8, lam):.3f}") print(f"P(<=12): {poisson.cdf(12, lam):.3f}") # Binomial: 100 coin flips n, p = 100, 0.5 print(f"P(>60 heads): {1 - binom.cdf(60, n, p):.4f}") # CLT: means from ANY distribution → Normal import numpy as np samples = [np.random.exponential(2, 30).mean() for _ in range(2000)] print(f"Means approx Normal: μ={np.mean(samples):.2f}, σ={np.std(samples):.3f}")
