Statistics & Probability

Distributions, estimators, expected values and hypothesis tests with clear derivations.

• How to Calculate Probabilities in a Markov Chain
  Calculate Markov-chain probabilities using transition matrices, initial distributions, matrix powers, path products, and hitting-probability equations.
• How to Find a Sinusoidal Regression Equation
  Learn how to fit a sinusoidal regression model, estimate its frequency by least squares, and convert the result to amplitude-phase form.
• How to Find a PMF and CDF
  Learn how to find a probability mass function and cumulative distribution function from a discrete distribution, with a worked example.
• How to Test the Mean of an Exponential Distribution
  Test an exponential population mean with an exact chi-square statistic, and use the same pivot to construct a confidence interval for the mean.
• How to Use the Normal Distribution to Find Probabilities
  Learn to use the normal distribution by standardizing values to z-scores, finding cumulative probabilities, and checking whether the model fits.
• How Many Times Should an Experiment Be Repeated?
  There is no universal number of experimental repetitions. Plan the sample size from your desired precision or statistical power and expected variability.
• How to Find Probabilities for a Uniform Distribution
  For a continuous uniform distribution on [a, b], find an event’s probability by dividing its overlap length with [a, b] by b − a.
• What Is the Expectation of a Function Symmetric About Zero?
  A function symmetric about zero is even, so its expectation is not generally zero; symmetry makes expectations of integrable odd functions vanish.
• What Is the Expected Maximum of Two Exponential Variables?
  For independent exponential variables with rates λ₁ and λ₂, the expected maximum is 1/λ₁ + 1/λ₂ − 1/(λ₁ + λ₂).
• How to Find the Expected Value of an Exponential Function
  Find E[e^X] by integrating against X's distribution. For beta and gamma random variables, standard formulas give the result and its conditions.
• How Does Expected Value Change with the Probability Measure?
  Expected value depends on the probability measure: integrate the same random variable under each measure, or use a density to convert between them.
• How to Find the Expected Squared Distance in a Random Walk
  For a random walk on a graph, calculate expected squared distance by weighting each squared graph distance by the probability of reaching that vertex.
• How to Determine Sample Size for a Normal Mean
  Calculate sample size for estimating a normal population mean from the confidence level, desired margin of error, and standard deviation.
• What Is the Difference Between the Rician and Gaussian Distributions?
  A Rician variable is the magnitude of a Gaussian vector with a possible nonzero mean; compare its support, density, limiting cases, and uses.
• How to Calculate Probabilities in Minesweeper
  Calculate a Minesweeper square’s mine probability by counting all mine layouts consistent with clues and the total mine count.
• Closed Form for a 2D Random Walk Probability Distribution
  For a square-lattice walk with four equally likely directions, count paths to (x, y) and divide by 4^n; the formula also identifies unreachable points.
• Conditional Mean and Variance of an Exponential Random Variable
  For an exponential random variable conditioned to exceed a threshold, memorylessness gives a shifted exponential: mean threshold plus 1/rate and variance 1/rate squared.
• How to Calculate the Probability of a Shared Birthday
  Calculate the chance that at least two people share a birthday by finding the probability that all birthdays differ, then subtracting it from 1.
• How to Compute the MGF of a Compound Poisson Distribution
  Compute a compound Poisson MGF by conditioning on the Poisson count: the result is exp(lambda times (the jump MGF minus one)).
• What Is a False Negative in a Confusion Matrix?
  A false negative is a positive case incorrectly predicted as negative. Learn where it appears in a confusion matrix and how to calculate its rate.
• How to Find a Gaussian Classifier’s Decision Boundary
  For Gaussian class-conditional data, derive the Bayes decision boundary from class means, covariances, and priors, and see when it is linear or quadratic.
• Continuity Correction vs. Confidence Interval: What’s the Difference?
  A continuity correction improves normal approximations to discrete probabilities; a confidence interval estimates an unknown parameter. Here is when each applies.
• Are Credit Card PINs Qualitative Ordinal Data?
  A credit card PIN is nominal categorical data, not ordinal: its digits form an identifying code with no meaningful ranking or arithmetic.
• How to Find Probabilities in a Box of Red and Black Balls
  For a draw without replacement, use combinations to find the probability of a given number of red and black balls, or adapt the method for replacement.
• How to Add a Line to a Bar Chart
  Overlay a line on a bar chart by aligning values to category centers; use a secondary axis only when units differ and label it clearly.
• How to Apply Bayes’ Theorem to Multiple Random Variables
  Use the joint form of Bayes’ theorem to update several unknown variables, compute the evidence by summing or integrating, and marginalize variables you do not need.
• What Is a Sufficient Statistic for the Beta Distribution?
  For an iid Beta(alpha, beta) sample with both shape parameters unknown, the minimal sufficient statistic is the pair of sums of log observations and log complements.
• How to Compute SSreg and SSres in Regression
  Compute regression and residual sums of squares from observed and fitted values, check the total-sum-of-squares identity, and interpret R-squared.
• How to Prove the Main Properties of Conditional Expectation
  Prove linearity, positivity, the tower property, and other conditional expectation identities from its defining integral property.
• How to Find a Confidence Interval for a Uniform Distribution
  For independent Uniform(0, θ) data, use the sample maximum to derive an exact confidence interval, with formulas and a worked 95% example.
• How Is the Multivariate Gaussian Density Derived?
  Derive the multivariate Gaussian density by transforming a standard normal vector, and see how the covariance determinant and inverse enter the formula.
• What Is the Entropy of the Sum of Two Uniform Random Variables?
  For independent X and Y uniform on [0,1], their sum has a triangular density and differential entropy 1/2 nat (about 0.721 bits).
• How to Calculate E[XY] for Dependent Random Variables
  For dependent random variables, compute E[XY] from their joint distribution or use covariance; the marginal means alone are not enough.
• What Are the Expected Value and Standard Deviation of Dice Rolls?
  For a fair six-sided die, the expected roll is 3.5 and the standard deviation is about 1.708; for n independent rolls, use the scaling rules.
• Expected Value of 1/(X + 1) for a Poisson Distribution
  For X ~ Poisson(λ), E[1/(X+1)] = (1−e^−λ)/λ. See the series derivation, a worked example, and the special case λ = 0.
• Expected Value of a Geometric Distribution by First-Step Analysis
  Use first-step analysis to derive the mean of a geometric waiting time, then see how the answer changes when you count failures instead of trials.
• What Is the Expected Value of a Poisson Distribution?
  For a Poisson random variable with parameter λ, the expected value is λ. See the derivation, a worked rate example, and common mistakes.
• How to Prove the Law of Total Expectation
  The law of total expectation says E[Y] = E[E[Y|X]]. Learn its proof from conditional expectation's definition, with discrete, continuous, and worked numerical examples.
• How to Find an Unbiased Estimator
  An estimator is unbiased when its expected value equals the target for every parameter value; identify the model, compute its expectation, and verify it.
• How to Find the Mean and Variance of an Exponential Distribution
  For an exponential distribution with rate λ, the mean is 1/λ and the variance is 1/λ²; see the derivation and scale-parameter version.
• How to Find the Median and Modes of a Discrete Uniform Distribution
  For a discrete uniform distribution, every support value is a mode. The median is the middle value for odd support size and spans two middle values for even size.
• How to Find the UMVUE of an Exponential Parameter
  For an i.i.d. exponential sample, the UMVUE depends on whether theta is the scale or rate: use the sample mean for scale and (n−1)/sum Xi for rate.
• What Is the Fisher Information for an Exponential Distribution?
  For an exponential distribution, Fisher information is 1 over the squared rate per observation, or n over the squared rate for n iid observations.
• What Is the Fisher Information of a Geometric Distribution?
  For a geometric random variable with success probability p, the Fisher information is 1/[p²(1−p)] per observation and n times that for n observations.
• What Is the Fisher Information of a Binomial Distribution?
  For X ~ Binomial(n,p), Fisher information about p is n/[p(1-p)]. Derive it, compare parameterizations, and see a worked example.
• What Is the Fisher Information of a Normal Distribution?
  For n independent normal observations, the Fisher information is diagonal: n/σ² for the mean and n/(2σ⁴) for the variance.
• What Is the Fisher Information of the Rayleigh Distribution?
  For the standard Rayleigh scale parameter, one observation has Fisher information 4 divided by sigma squared; n independent observations have 4n divided by sigma squared.
• How to Generate a Poisson Random Variable from a Uniform One
  Use inverse-transform sampling: compare a standard uniform draw with cumulative Poisson probabilities, updating each probability recursively until you find the count.
• How to Generate Random Numbers From a Probability Equation
  To sample from a probability equation, define a valid distribution, then use inverse-CDF sampling or rejection sampling when inversion is difficult.
• How to Find a Geometric Mean When Values Are Negative
  The standard geometric mean requires positive values. Learn how to handle signed data, when shifting is misleading, and how to average negative returns.
• Why Can’t I Compute the Expected Value from a Uniform MGF?
  For X uniform on [a,b], the MGF has an apparent 0/0 at t=0; extend it by continuity, then differentiate to get E[X]=(a+b)/2.
• How to Calculate a Z-Score Without a Standard Deviation
  A z-score needs a standard deviation or equivalent information. Learn when it cannot be calculated and how to find or estimate the missing spread.
• How to Find an Exponential Percentile on a TI-84
  Find an exponential-distribution percentile on a TI-84 using the inverse CDF, with rate-versus-mean formulas and a worked keystroke example.
• How to Find a Probability Density Function (PDF)
  Learn how to derive a probability density function from a CDF, identify a known distribution, or transform a random variable, then check its support and integral.
• How to Find the Inverse of a Poisson Distribution Function
  The inverse Poisson CDF is a discrete quantile: find the smallest integer k whose cumulative probability is at least p; it is computed numerically.
• Law of Iterated Expectations: A Worked Example
  The law of iterated expectations says to average a conditional mean over the variable you conditioned on; a weighted two-bag example shows each step.
• What Is the MLE for a Uniform(θ, 0) Distribution?
  For iid data from Uniform(θ, 0) with θ < 0, the likelihood is maximized at the sample minimum; if endpoints are reversed, it is the maximum.
• Mean and Variance of the Weibull Distribution
  For a Weibull distribution with shape k and scale λ, the mean is λΓ(1+1/k) and the variance is λ²[Γ(1+2/k)−Γ(1+1/k)²].
• How to Find the Median of Discrete and Continuous Variables
  Find a median from the cumulative distribution function: use the 0.5 crossing for continuous laws and account for jumps in discrete or mixed laws.
• How to Estimate Uniform Distribution Parameters by Moments
  For a uniform model, match sample moments to population moments; see formulas for U(0, θ) and U(a, b), with a worked example.
• How to Use the Method of Moments for a Gamma Distribution
  Estimate a gamma distribution’s shape and scale by matching sample moments, with formulas, a worked example, and the rate-parameter alternative.
• What Is the MGF of the Negative Binomial Distribution?
  For the failures-before-rth-success convention, the negative binomial MGF is [p/(1−(1−p)e^t)]^r; its domain and other conventions are explained.
• What Is the Minimal Sufficient Statistic for Uniform(θ, θ)?
  For Uniform(θ, θ), the endpoints coincide, so the model is degenerate; for a nonzero interval, the minimal statistic depends on its endpoints.
• How to Find the MLEs of a Negative Binomial Distribution
  For negative-binomial data, the MLE of p has a closed form when the size is fixed; estimate unknown size by solving a digamma score equation numerically.
• How to Find the MLE of a Uniform Distribution
  For an i.i.d. sample from Uniform(0, θ), the maximum-likelihood estimate is the sample maximum; endpoint estimates differ when both bounds are unknown.
• What Is the MGF of a Uniform Distribution?
  For a continuous uniform random variable on [a,b], the moment-generating function is (e^{bt}-e^{at})/[t(b-a)], with value 1 at t=0.
• How Does a Poisson Distribution Converge to Normal?
  As a Poisson mean λ grows, centering by λ and scaling by √λ gives a standard normal limit; use a continuity correction for count probabilities.
• How to Derive the Power of the Wald Test
  The Wald test has asymptotic power given by a noncentral chi-square distribution under local alternatives.
• What Are the Properties of the Maximum and Minimum of Exponential Random Variables?
  For independent exponential variables, the minimum is exponential with summed rate; derive the maximum's CDF, density, mean, and variance.
• How to Prove the Variance of a Geometric Distribution
  Derive Var(X) = (1-p)/p² for a geometric random variable using its probability-generating function, and see how the 0-based convention differs.
• Real Examples of Misleading Statistics and How to Spot Them
  Learn how misleading statistics arise from averages, risk comparisons, selective samples, and graphs, with worked examples and checks for interpreting claims.
• What Is the Standard Deviation of an Exponential Distribution?
  For an exponential distribution with rate λ, the standard deviation is 1/λ; with scale θ or mean μ, it equals θ or μ. See the derivation and example.
• How Do You Sum a Random Number of Independent Variables?
  For an independent random count and iid summands, derive the random sum’s mean and variance, see a worked example, and learn when the formulas fail.
• Expected Value of 1/(1 + X) for a Geometric Random Variable
  For a geometric random variable X, E[1/(1+X)] is -p ln(p)/(1-p) on support starting at zero; starting at one gives a different formula.
• What Is the Tower Property of Conditional Expectation?
  The tower property says that conditioning first on finer information and then on coarser information is the same as conditioning directly on the coarser information.
• What Is an Unbiased Estimator of a Bernoulli Distribution?
  For independent Bernoulli observations, the sample proportion of successes is an unbiased estimator of the unknown success probability.
• How Do Confidence Bounds Relate to a One-Tailed Test?
  For a one-tailed test at significance level alpha, use a 1−alpha one-sided confidence bound: lower for a right-tail test and upper for a left-tail test.
• What Is Wald’s Second Equation?
  Wald’s second equation gives the mean squared centered sum at a stopping time: the increment variance times the expected stopping time.
• What Does π Mean in Statistics?
  In statistics, π often denotes a population proportion, event probability, prior distribution, or mixture weight; its exact meaning comes from context.
• Why Does Sampling Density Scale as n^(1/p)?
  In a fixed p-dimensional region, n samples give about n^(1/p) points per coordinate and spacing n^(-1/p); distinguish this from probability density.
• What Is the Intuitive Relationship Between SVD and PCA?
  For a centered data matrix, PCA directions are the right singular vectors of its SVD; squared singular values give component variances.
• Does Standard Deviation Change When You Add or Multiply by a Constant?
  Adding a constant leaves the standard deviation unchanged; multiplying by a scales it by |a| and the variance by a². Proof, examples and the general rule for aX + b.
• Given the Mean and Standard Deviation, How Do You Find a Probability?
  Convert values to z-scores and use the standard normal table to find P(X < b), P(X > a) and P(a < X < b) from a mean and standard deviation, with examples.
• How to Find Quartiles from a Histogram (Grouped Data)
  Find Q1, the median and Q3 from a histogram using cumulative frequencies and the grouped-data interpolation formula, with a fully worked example.
• Inverting a Gaussian (RBF) Kernel Matrix Without Numerical Trouble
  A Gaussian kernel matrix is positive definite for distinct points, so it is invertible, but it is often badly conditioned. Use a ridge term and a Cholesky solve instead.
• What Is the Standard Deviation of a Dice Roll?
  The standard deviation of one fair six-sided die is √(35/12) ≈ 1.708. See the derivation, the formula for any die, and how it scales for sums and averages.