Conditional Probability and the Multiplication Rule

Conditional probability — the probability of $E$ given that $F$ has occurred: $$P(E \mid F) = \frac{P(E \cap F)}{P(F)}, \qquad P(F) \neq 0$$ Conditioning shrinks the sample space to $F$: count favourable outcomes inside $F$ only. Useful facts: $P(F \mid F) = 1$, and $P(E' \mid F) = 1 - P(E \mid F)$.

Multiplication rule (the same identity, rearranged): $$P(E \cap F) = P(F)\,P(E \mid F) = P(E)\,P(F \mid E)$$ This is the engine for without-replacement problems — multiply along the branches of the tree, updating the counts at each draw.

Independent Events

$E$ and $F$ are independent when knowing one changes nothing about the other: $$P(E \cap F) = P(E)\,P(F)$$ equivalently $P(E \mid F) = P(E)$. Test with numbers, never with intuition — compute both sides and compare. If $A$ and $B$ are independent, so are $A$ and $B'$, $A'$ and $B$, and $A'$ and $B'$.

Do not confuse independent with mutually exclusive: mutually exclusive events ($E \cap F = \varnothing$) with nonzero probabilities are never independent — one occurring forbids the other.

Standard consequences for independent $A, B$: $P(A \cup B) = P(A) + P(B) - P(A)P(B)$, and $P(\text{at least one}) = 1 - P(A')P(B')$ — the complement route is nearly always shorter.

Total Probability and Bayes' Theorem

Let $E_1, E_2, \ldots, E_n$ be a partition of the sample space (mutually exclusive, exhaustive, nonzero probabilities) and $A$ any event.

Theorem of total probability — average the conditional probabilities, weighted by the partition: $$P(A) = \sum_{i=1}^{n} P(E_i)\,P(A \mid E_i)$$

Bayes' theorem — reverse the conditioning: $$P(E_k \mid A) = \frac{P(E_k)\,P(A \mid E_k)}{\sum_{i=1}^{n} P(E_i)\,P(A \mid E_i)}$$ The recipe: name the hypotheses $E_i$ (which bag, which machine, diseased or not), write their prior probabilities, write each $P(A \mid E_i)$, and assemble. The denominator is always the total-probability sum — compute it once and reuse it.

Random Variables and Their Distributions

A random variable $X$ assigns a number to each outcome. Its probability distribution is the table of values $x_i$ with probabilities $p_i$, where $$p_i \geq 0, \qquad \sum p_i = 1$$ The condition $\sum p_i = 1$ is how unknown constants in a distribution table are found — a guaranteed board question.

Mean (expectation): $$E(X) = \mu = \sum x_i p_i$$ Build the distribution table first, then add an $x_i p_i$ row and sum it — tabulate, never improvise.

Worked Examples — Conditional Probability and Independence

Example 1 — The definition at work

Given $P(E) = 0.6$, $P(F) = 0.3$ and $P(E \cap F) = 0.2$, find $P(E \mid F)$ and $P(F \mid E)$.

Step 1 — apply the formula both ways: $P(E \mid F) = \dfrac{0.2}{0.3} = \dfrac{2}{3}$ and $P(F \mid E) = \dfrac{0.2}{0.6} = \dfrac{1}{3}$.

Answer: $\dfrac{2}{3}$ and $\dfrac{1}{3}$ — the two conditionals differ; the denominator is always the given event.

Example 2 — The two-children classic

A family has two children. Find the probability that both are boys, given that (i) at least one is a boy, (ii) the elder is a boy.

Step 1 — sample space: $\{BB, BG, GB, GG\}$, equally likely.

Step 2 — case (i): given set $\{BB, BG, GB\}$, favourable $\{BB\}$: probability $\dfrac{1}{3}$.

Step 3 — case (ii): given set $\{BB, BG\}$, favourable $\{BB\}$: probability $\dfrac{1}{2}$.

Answer: $\dfrac{1}{3}$ and $\dfrac{1}{2}$ — different information shrinks the sample space differently; this pair is the cleanest illustration of conditioning on the paper.

Example 3 — Conditioning on a dice sum

A die is thrown twice. Given that the sum is $6$, find the probability that $4$ appeared at least once.

Step 1 — the given event: sum $= 6$ means $\{(1,5), (2,4), (3,3), (4,2), (5,1)\}$ — five outcomes.

Step 2 — favourable inside it: $(2,4)$ and $(4,2)$.

Answer: $\dfrac{2}{5}$ — count within the reduced sample space; the original $36$ never appears in the final ratio.

Example 4 — Without replacement

Two cards are drawn from a well-shuffled pack without replacement. Find the probability that both are kings.

Step 1 — multiply along the tree: $P(\text{first king}) = \dfrac{4}{52}$, then $P(\text{second king} \mid \text{first king}) = \dfrac{3}{51}$.

Answer: $\dfrac{4}{52}\cdot\dfrac{3}{51} = \dfrac{1}{221}$ — the multiplication rule is the tree diagram in symbols.

Example 5 — Testing independence with numbers

A die is thrown. $E$ = "the number is even", $F$ = "the number is at most $4$". Are $E$ and $F$ independent?

Step 1 — the three probabilities: $P(E) = \dfrac{1}{2}$, $P(F) = \dfrac{4}{6} = \dfrac{2}{3}$, and $E \cap F = \{2, 4\}$ so $P(E \cap F) = \dfrac{1}{3}$.

Step 2 — compare: $P(E)P(F) = \dfrac{1}{2}\cdot\dfrac{2}{3} = \dfrac{1}{3} = P(E \cap F)$.

Answer: independent — the product test decides, and it can surprise; always compute.

Example 6 — Union of independent events

$A$ and $B$ are independent with $P(A) = 0.3$ and $P(B) = 0.6$. Find $P(A \cup B)$ and $P(A \cap B')$.

Step 1 — union: $P(A \cup B) = 0.3 + 0.6 - (0.3)(0.6) = 0.72$.

Step 2 — with the complement (independence passes to complements): $P(A \cap B') = P(A)P(B') = (0.3)(0.4) = 0.12$.

Answer: $0.72$ and $0.12$.

Example 7 — At least one, via complements

A problem is given to two students who work independently. Their chances of solving it are $\dfrac{1}{2}$ and $\dfrac{1}{3}$. Find the probability that (i) the problem is solved, (ii) exactly one of them solves it.

Step 1 — problem solved = at least one solves: $1 - P(\text{both fail}) = 1 - \dfrac{1}{2}\cdot\dfrac{2}{3} = \dfrac{2}{3}$.

Step 2 — exactly one: $\dfrac{1}{2}\cdot\dfrac{2}{3} + \dfrac{1}{2}\cdot\dfrac{1}{3} = \dfrac{1}{3} + \dfrac{1}{6} = \dfrac{1}{2}$.

Answer: $\dfrac{2}{3}$ and $\dfrac{1}{2}$ — "at least one" should trigger the complement route automatically.

Worked Examples — Total Probability, Bayes and Random Variables

Example 8 — Total probability

Bag I contains $3$ red and $4$ black balls; Bag II contains $5$ red and $6$ black. A bag is chosen at random and a ball drawn. Find the probability that it is red.

Step 1 — partition by bag: $P(\text{I}) = P(\text{II}) = \dfrac{1}{2}$; $P(R \mid \text{I}) = \dfrac{3}{7}$, $P(R \mid \text{II}) = \dfrac{5}{11}$.

Step 2 — weighted average: $$P(R) = \frac{1}{2}\cdot\frac{3}{7} + \frac{1}{2}\cdot\frac{5}{11} = \frac{3}{14} + \frac{5}{22} = \frac{33 + 35}{154}$$

Answer: $\dfrac{68}{154} = \dfrac{34}{77}$.

Example 9 — Bayes reverses Example 8

In the setup of Example 8, the drawn ball is red. Find the probability that it came from Bag I.

Step 1 — Bayes: $P(\text{I} \mid R) = \dfrac{P(\text{I})\,P(R \mid \text{I})}{P(R)} = \dfrac{3/14}{34/77}$.

Step 2 — simplify: $\dfrac{3}{14}\cdot\dfrac{77}{34} = \dfrac{33}{68}$.

Answer: $\dfrac{33}{68}$ — the denominator is exactly the total probability already computed; never recompute it from scratch.

Example 10 — Bayes with machines

Machines $A$ and $B$ produce $60\%$ and $40\%$ of a factory's items; $2\%$ of $A$'s output and $1\%$ of $B$'s output is defective. An item drawn at random is defective. Find the probability it came from machine $A$.

Step 1 — the two branches: $P(A)P(D \mid A) = 0.6 \times 0.02 = 0.012$ and $P(B)P(D \mid B) = 0.4 \times 0.01 = 0.004$.

Step 2 — Bayes: $P(A \mid D) = \dfrac{0.012}{0.012 + 0.004} = \dfrac{12}{16}$.

Answer: $\dfrac{3}{4}$ — lay out the prior-times-likelihood products first and the answer is one division.

Example 11 — Bayes against intuition

A disease affects $0.1\%$ of a population. A test detects it with probability $0.99$ if present, but also gives a false positive with probability $0.005$ for a healthy person. Given a positive test, find the probability the person actually has the disease.

Step 1 — the two branches: diseased and positive: $0.001 \times 0.99 = 0.00099$; healthy and positive: $0.999 \times 0.005 = 0.004995$.

Step 2 — Bayes: $$P(\text{disease} \mid +) = \frac{0.00099}{0.00099 + 0.004995} = \frac{990}{5985} = \frac{22}{133}$$

Answer: $\dfrac{22}{133} \approx 0.17$ — a $99\%$-accurate test, yet a positive result means the disease is still unlikely, because healthy people vastly outnumber diseased ones. Boards love asking for the interpretation sentence.

Example 12 — A probability distribution and its mean

Two fair coins are tossed. Let $X$ be the number of heads. Write the distribution of $X$ and find its mean.

Step 1 — the table: $P(X = 0) = \dfrac{1}{4}$, $P(X = 1) = \dfrac{1}{2}$, $P(X = 2) = \dfrac{1}{4}$ (sum $= 1$).

Step 2 — mean: $E(X) = 0\cdot\dfrac{1}{4} + 1\cdot\dfrac{1}{2} + 2\cdot\dfrac{1}{4}$.

Answer: $E(X) = 1$ — exactly one head expected on average, as symmetry suggests.

Example 13 — Finding an unknown in a distribution

A random variable $X$ takes values $0, 1, 2, 3$ with probabilities $k, 2k, 3k, 4k$. Find $k$, $P(X \geq 2)$ and $E(X)$.

Step 1 — probabilities sum to $1$: $k + 2k + 3k + 4k = 10k = 1$, so $k = \dfrac{1}{10}$.

Step 2 — read off: $P(X \geq 2) = 3k + 4k = \dfrac{7}{10}$.

Step 3 — mean: $E(X) = 0(k) + 1(2k) + 2(3k) + 3(4k) = 20k = 2$.

Answer: $k = \dfrac{1}{10}$, $P(X \geq 2) = \dfrac{7}{10}$, $E(X) = 2$ — the $\sum p_i = 1$ condition unlocks the whole question.