Total probability and Bayes theorem

Objectives and prerequisites

Build a disjoint, exhaustive partition; calculate a marginal probability from branch probabilities; reverse a conditional using Bayes theorem; and account for base rates when evidence is common among a large background group. Prerequisites: events, conditional probability and the multiplication rule. Bayes theorem is explicitly in the GATE 2026 CS syllabus. The examples here are original mathematical models, not empirical claims about real filters or factories.

1. Partitions prevent overlap and missing cases

Events H₁,…,Hₖ form a partition if they are mutually disjoint and their union is Ω. Exactly one of them occurs on every experiment. If every used branch has P(Hᵢ)>0, the law of total probability is

P(E)=Σᵢ P(Hᵢ)P(E|Hᵢ).

Proof: E is the disjoint union of E∩Hᵢ. Add their probabilities and apply the multiplication rule. A zero-probability branch contributes zero and can be omitted without defining a conditional on it. The Hᵢ need not be equally likely, and the conditional probabilities need not be equal. Independence is not assumed.

Example: a uniformly selected item comes from machine M₁ with probability .6 and M₂ with probability .4. Their defect probabilities are .02 and .05. The defect probability is .6(.02)+.4(.05)=.012+.020=.032. This is a weighted average. The unweighted average (.02+.05)/2=.035 would assume equally likely machines, contrary to the item-level proportions.

The sample procedure matters: choosing a machine uniformly and then choosing one of its items is a different experiment from selecting an item uniformly from combined output. Always attach the branch weights to the specified procedure.

2. Deriving Bayes theorem

Suppose P(E)>0. For a partition branch Hⱼ,

P(Hⱼ|E)=P(Hⱼ∩E)/P(E) =P(Hⱼ)P(E|Hⱼ)/[Σᵢ P(Hᵢ)P(E|Hᵢ)].

P(Hⱼ) is the prior probability; P(E|Hⱼ) is the likelihood of the evidence under that branch; P(Hⱼ|E) is the posterior probability after observing evidence. The denominator includes every route by which E could occur. A likelihood is not a posterior, and the numerator alone is a joint probability, not a conditional probability.

For the machines, P(M₂|defect)=.020/.032=5/8=.625. Although M₂ supplies only 40% of items, its higher defect rate means it supplies 62.5% of defective items in this model. P(defect|M₂)=.05 answers a different question. Check the other branch: P(M₁|defect)=.012/.032=3/8; the two posteriors add to one.

A count model makes the same calculation visible. Among 10,000 model items, 6,000 are from M₁ and 4,000 from M₂. Expected defect counts are 120 and 200. Of 320 defects, 200/320 come from M₂. These counts illustrate expected proportions; they do not promise exact counts in a random real sample.

3. Base rates and false alarms

Suppose 1% of messages are spam. A filter flags 90% of spam and 5% of nonspam. Let S=spam and F=flagged. Then P(S)=.01, P(F|S)=.90, P(F|Sᶜ)=.05. Total flagged probability is .01(.90)+.99(.05)=.0585. Therefore

P(S|F)=.009/.0585=2/13≈.153846.

The posterior is about 15.4%, not 90%. In 10,000 model messages, 100 are spam and 9,900 are not. Expected flagged counts are 90 spam and 495 nonspam, totaling 585. The large background group creates more false positives despite its smaller individual flag probability. A high detection rate alone does not determine the posterior.

Now use unflagged evidence Fᶜ. P(Fᶜ|S)=.10 and P(Fᶜ|Sᶜ)=.95. Thus P(S|Fᶜ)=.001/[.001+.9405]=2/1883≈.001062. Complement the likelihoods within each branch; do not simply take 1−P(S|F). That latter expression equals P(Sᶜ|F), with the original evidence still fixed.

4. Three or more hypotheses

Suppose a source is H₁,H₂,H₃ with probabilities 1/2,1/3,1/6. An observed event E has conditional probabilities 1/5,1/2,3/5. The joint weights are 1/10,1/6,1/10. Their sum is 11/30. Dividing each by 11/30 gives posterior weights 3/11,5/11,3/11. Check: they are nonnegative and sum to one.

A convenient workflow is “prior × likelihood, then normalize.” Keep exact fractions or several significant digits until the final division. If you round small joint weights too early, posteriors may be visibly distorted. You can compare which hypothesis is most probable after evidence by comparing joint weights, because all share the same positive denominator.

5. Evidence from repeated observations

Do not multiply likelihoods unless the model provides the necessary conditional independence. Suppose an urn U is selected once: U₁ with probability 1/2 has red probability 3/4, and U₂ with probability 1/2 has red probability 1/4. Two draws with replacement from that selected urn are conditionally independent given U. For two reds R₁∩R₂, the likelihoods are (3/4)²=9/16 and (1/4)²=1/16. Bayes gives P(U₁|R₁∩R₂)=9/(9+1)=9/10.

Before observing the urn, the red draws are not independent: P(R₁)=P(R₂)=1/2, but P(R₁∩R₂)=(1/2)(9/16)+(1/2)(1/16)=5/16, not 1/4. The shared unknown urn links them. Conditional independence and unconditional independence are distinct. If draws were without replacement, even the within-urn likelihood would need updating rather than squaring a fixed probability.

Pitfalls and solving checklist

Name the hypotheses and check that they are disjoint and exhaustive. Mark which number is a prior and which is conditional. Draw a probability tree or write the joint branch weights. Add all branches producing the evidence to obtain its total probability. Verify that denominator is positive, then divide the relevant joint weight. Check posteriors sum to one. Never reverse P(E|H) by intuition, omit the background branches, average unequally weighted rates, or assume repeated observations are independent without the appropriate conditioning.

Original practice: six single-correct MCQs

These are original questions, not previous-year questions. Choose exactly one option.

BT01. Machines M₁,M₂ supply 60%,40% of items and have defect rates 2%,5%. Find the defect probability of a uniformly selected item from combined output.

A. 0.035 B. 0.07 C. 0.032 D. 0.012

Answer: C

A: This unweighted mean ignores the output proportions.

B: Adding conditional rates ignores branch probabilities.

C: Use .6×.02+.4×.05=.032.

D: This includes defects from M₁ only.

BT02. With the machine proportions and defect rates 60%,40% and 2%,5%, what is P(M₂|defect)?

A. 5/8 B. 2/5 C. 1/20 D. 1/50

Answer: A

A: The M₂ defect joint weight .02 divided by total .032 is 5/8.

B: This is the prior probability of M₂.

C: This is P(defect|M₂), the reversed condition.

D: This is the joint probability of M₂ and defect before normalization.

BT03. Spam prevalence is 1%; a filter flags 90% of spam and 5% of nonspam. Find P(spam|flagged).

A. 0.90 B. 0.01 C. 0.95 D. 2/13

Answer: D

A: This is the likelihood of flagging given spam.

B: This is the prior before observing a flag.

C: This is the nonspam unflagged rate, not the requested posterior.

D: Use .009/(.009+.0495)=2/13.

BT04. H₁,H₂,H₃ form a mutually exclusive exhaustive partition, with priors 1/2,1/3,1/6 and evidence likelihoods 1/5,1/2,3/5. What is P(H₂|E)?

A. 1/2 B. 5/11 C. 1/6 D. 3/11

Answer: B

A: This is the likelihood P(E|H₂).

B: The joint weight 1/6 divided by total 11/30 is 5/11.

C: This is the unnormalized H₂ joint weight.

D: This is the posterior for H₁ or H₃.

BT05. Which conditions are sufficient for using P(E)=ΣᵢP(Hᵢ)P(E|Hᵢ), with all displayed conditionals defined?

A. The Hᵢ only need to overlap B. The Hᵢ must be equally likely C. The Hᵢ form a disjoint exhaustive partition and each has positive probability D. E must be independent of every Hᵢ

Answer: C

A: Overlapping branches can double-count evidence.

B: Unequal branch probabilities are allowed.

C: A positive-probability partition makes the decomposition valid and all ratios defined.

D: Independence is unnecessary; conditional rates may differ.

BT06. Choose one of two urns with equal probability, with red fractions 3/4 and 1/4. Make two uniform random draws with replacement, independent given the chosen urn. Given two reds, what is the probability the first urn was chosen?

A. 3/4 B. 1/2 C. 5/16 D. 9/10

Answer: D

A: This is the posterior after one red, not two.

B: This is the prior before the evidence.

C: This is the total probability of two reds, not the posterior.

D: Conditional likelihoods are 9/16 and 1/16; equal priors give 9/(9+1).

Source and scope

Bayes theorem is explicitly listed under Section 1 Probability and Statistics of the official GATE 2026 CS syllabus. Total probability supplies its denominator. These original lessons do not claim complete coverage of Probability and Statistics or Engineering Mathematics.

Official GATE 2026 CS syllabus (verified 2026-10-14).