What Does Probability Really Mean?
At a glance
- 1. Define the event
State exactly what outcome or proposition you are assigning probability to.
- 2. Specify the information
A probability is conditional on assumptions, data, and the model used.
- 3. Choose an interpretation
Use long-run frequency, degree of belief, or a physical chance according to the context.
- 4. Apply the axioms
Probabilities must lie between zero and one and obey complement and addition rules.
- 5. Update or test it
Compare predictions with data, or revise beliefs when new evidence arrives.
Probability does not simply mean “how often something happens” in every situation. It is a way to quantify uncertainty about a precisely defined event, given a model and some information.
For example, “the probability of rain tomorrow” refers to a future event under current meteorological information. “The probability that a die shows 6” refers to a repeated experiment under assumptions about the die. “The probability that a defendant is guilty” concerns uncertainty about a proposition, not a repeatable physical experiment in the same way.
The numerical rules are shared across these examples, but what the number represents can differ.
The mathematical meaning
A probability model consists of a sample space , a sigma-algebra of events, and a probability measure defined on . For an event , is its probability.
The basic axioms (for and pairwise disjoint ) are:
for pairwise disjoint events . The last rule says that probabilities add when events cannot occur together.
Useful consequences include
and, for any two events,
These axioms tell you how probabilities must behave. They do not, by themselves, tell you what a particular probability means philosophically. That is the role of interpretation.
Probability as long-run frequency
The frequentist interpretation connects probability with repeated trials. If a fair coin is tossed many times, the proportion of heads is expected to approach as the number of tosses grows.
If is the number of heads in tosses, then the observed relative frequency is
Under the usual independent fair-coin model, this relative frequency converges to in the long run, with probability 1. This does not mean that the proportion is exactly after every finite number of tosses. After 10 tosses, 7 heads is entirely possible.
This interpretation works naturally for repeatable processes: manufacturing defects, survey samples, dice rolls, or the proportion of patients responding to a treatment under defined conditions.
It becomes less straightforward for one-off events. There is no literal infinite sequence of identical “tomorrows” from which to calculate the frequency of rain tomorrow. Statistical methods can still assign a probability using a model fitted to past weather data, but the interpretation is then model-based rather than a directly observed frequency.
Probability as degree of belief
The Bayesian interpretation treats probability as a coherent degree of belief in a proposition, conditional on information. A probability of means that, given the stated information and assumptions, the proposition is believed more strongly than one assigned probability .
This does not mean that every personal guess is automatically a useful probability. Beliefs should obey the probability rules. If someone assigns probabilities that violate those rules, their assessments can be made internally inconsistent.
For example, suppose you believe there is a probability that a package arrives today. After receiving a tracking update, you may revise that number to . The event has not occurred more often in a repeated experiment; your information has changed.
Bayesian probability is especially useful for uncertain parameters, prediction, medical diagnosis, and decision-making. A statement such as “there is a 95% probability that this parameter lies between these values” has a Bayesian interpretation if the probability is assigned to the parameter itself under a specified posterior model.
Physical chance and model assumptions
Sometimes probability is understood as an objective tendency or physical chance. A well-designed random number generator may be constructed so that each output has a physical chance close to . Quantum mechanics also uses probabilities to describe outcomes of measurements.
Even then, the numerical probability depends on how the experiment is defined and what assumptions are made. A die may have probability for each face in an ideal model, but a real die can be biased. The probability is therefore not a property of the symbol “die” alone; it is a property of the die, the procedure, and the model.
The same point applies to statistical data. Saying “the probability of disease is 0.02” is incomplete unless you specify the population, test result, time period, and information being conditioned on.
A worked example: a medical test
Suppose a disease affects of a population. A test has sensitivity , meaning
and a false-positive rate of , meaning
What is the probability that a randomly selected person has the disease after testing positive?
Use Bayes’ rule:
The total probability of a positive result is
Therefore,
So the probability is about , not . The figure is the probability of a positive result given disease; it is not the probability of disease given a positive result. This example shows why a probability must always be attached to a clearly stated event and conditioning information.
What probability does not mean
A probability of does not guarantee that an event will happen. It expresses uncertainty before the outcome is known. A chance can fail to occur, just as a chance can occur.
Nor does probability necessarily mean that an event is “partly happening.” The event either occurs or does not occur in a particular trial. The probability describes uncertainty about which outcome will occur, or the model’s long-run behaviour across comparable trials.
Probability also does not remove the need for a model. Data, assumptions, and the reference population determine the number. Changing those inputs can change the probability without any arithmetic error.
Common mistakes
- Confusing with . Conditional probabilities generally differ; the medical-test example demonstrates this.
- Treating independence as automatic. Two events are independent only when under the specified model.
- Assuming every probability is a frequency. Probabilities can represent uncertainty about a one-time event or an unknown parameter.
- Ignoring the reference population. “A 5% risk” is incomplete without knowing who is at risk and over what period.
- Using past outcomes as if they force the next one. In independent coin tosses, a run of heads does not make tails more likely on the next toss.
A sound probability statement therefore has three parts: the event being considered, the information or conditions, and the model or interpretation used to assign the number.
References
- Foundations of the Theory of Probability — A. N. Kolmogorov
- Probability Theory: The Logic of Science — E. T. Jaynes
- Introduction to Probability — Joseph K. Blitzstein and Jessica Hwang
Spotted an error? Send a correction. Read how we write and check articles in our editorial policy.