List the outcomes, condition with the intersection in the numerator, and standardise every Gaussian. That is Exercise 1.
Notation follows the 2023–24 reader. Event probability is \(\Pr[A]\). Discrete PMF and continuous PDF are both \(p_X\); the CDF is \(P_X(x)=\Pr[X\le x]\). SLT sheets sometimes write \(P[\cdot]\) for \(\Pr\), \(f_{X,Y}\) for a joint PDF, and \(\Phi\) for the standard-normal CDF. Those are aliases; this page stays with \(\Pr\), \(p\), \(P_X\) and \(Q(x)=1-\Phi(x)\).
An experiment is a procedure you can repeat. One run is a trial. The sample space \(\mathcal{S}\) is the list of everything that can happen. An event is a subset of that list — one outcome, many, or all of \(\mathcal{S}\). Two events are disjoint (mutually exclusive) if they share no outcomes. The empty set \(\emptyset\) is the null event.
Two fair coins:
$$ \mathcal{S}=\{hh,ht,th,tt\}. $$\(ht\) and \(th\) are different outcomes. \(A=\{\text{first coin heads}\}=\{hh,ht\}\) and \(B=\{\text{second coin tails}\}=\{ht,tt\}\) are not disjoint: \(ht\) sits in both. A partition (event space) of \(\mathcal{S}\) is a collection of disjoint events whose union is \(\mathcal{S}\); here \(\{hh,ht\}\) and \(\{th,tt\}\) (first coin H vs T).
Venn picture: the rectangle is \(\mathcal{S}\). Complement \(A^C\): everything in \(\mathcal{S}\) not in \(A\). Intersection \(A\cap B\): both. Union \(A\cup B\): in \(A\), in \(B\), or in both. The reader also writes \(\Pr[AB]=\Pr[A,B]=\Pr[A\cap B]\).
If every atom of \(\mathcal{S}\) is equally likely and \(A\) contains \(N_A\) of the \(N\) atoms, the classic (Laplace) definition is \(\Pr[A]=N_A/N\). Frequentist story: repeat \(N\) times, \(f_A=N(A)/N\), and \(\Pr[A]=\lim_{N\to\infty}f_A\). Two fair coins, \(A=\{hh\}\): \(N=4\), so \(\Pr[A]=1/4\). Do not use \(N_A/N\) when the atoms are not equally likely.
Every probability on this course obeys three rules (reader §2.4.1). Everything else is a consequence.
Consequences you must be able to write.
Empty set. \(\mathcal{S}=\mathcal{S}\cup\emptyset\) and the two pieces are disjoint, so \(1=\Pr[\mathcal{S}]+\Pr[\emptyset]\) and \(\Pr[\emptyset]=0\).
Complement. \(A\) and \(A^C\) partition \(\mathcal{S}\), so \(\Pr[A^C]=1-\Pr[A]\).
Union, any two events (not necessarily disjoint):
$$ \Pr[A\cup B]=\Pr[A]+\Pr[B]-\Pr[A\cap B]. $$Derivation. Split the union into three disjoint pieces:
$$ A\cup B=(A\cap B^C)\cup(B\cap A^C)\cup(A\cap B). $$\(A\) itself splits as \((A\cap B^C)\cup(A\cap B)\), so \(\Pr[A\cap B^C]=\Pr[A]-\Pr[A\cap B]\). Same for \(B\). Add the three pieces:
$$ \begin{aligned} \Pr[A\cup B] &=(\Pr[A]-\Pr[A\cap B])+(\Pr[B]-\Pr[A\cap B])+\Pr[A\cap B]\\ &=\Pr[A]+\Pr[B]-\Pr[A\cap B]. \end{aligned} $$If \(A\) and \(B\) are disjoint then \(\Pr[A\cap B]=0\) and you recover axiom 3. If \(A\subset B\) then \(\Pr[A]\le\Pr[B]\).
For a partition \(\{B_1,\dots,B_m\}\) of \(\mathcal{S}\),
$$ \Pr[A]=\sum_{i=1}^m\Pr[A\cap B_i]=\sum_{i=1}^m\Pr[AB_i]. $$Picture: oval \(A\) cut by the partition strips. Fair coins, four equally likely atoms: axiom 2 plus axiom 3 give \(1=4\Pr[\{hh\}]\), so each atom is \(1/4\).
After you learn that \(B\) happened, the world shrinks to \(B\). The probability of \(A\) inside that smaller world is the overlap over the size of \(B\):
$$ \Pr[A\mid B]=\frac{\Pr[AB]}{\Pr[B]}=\frac{\Pr[A\cap B]}{\Pr[B]},\qquad \Pr[B]\gt 0. $$A priori \(\Pr[A]\); a posteriori \(\Pr[A\mid B]\). Counting: \(\Pr[AB]=N_{AB}/N\), \(\Pr[B]=N_B/N\), so \(N_{AB}/N_B=\Pr[AB]/\Pr[B]\). Properties: \(\Pr[A\mid B]\ge 0\), \(\Pr[B\mid B]=1\), and if the \(A_i\) are disjoint with union \(A\) then \(\Pr[A\mid B]=\sum_i\Pr[A_i\mid B]\).
Two coins. \(A_2=\{hh\}\), \(A_1=\{\text{first toss H}\}=\{hh,ht\}\):
$$ \Pr[A_1 A_2]=\Pr[\{hh\}]=\tfrac14,\quad \Pr[A_1]=\tfrac12,\quad \Pr[A_2\mid A_1]=\frac{1/4}{1/2}=\tfrac12. $$If the first toss is T, \(\Pr[\{hh\}\mid\{th,tt\}]=0\).
Disjoint: the circles do not overlap, \(\Pr[AB]=0\). Independent: learning one does not change the other,
$$ \Pr[AB]=\Pr[A]\Pr[B], $$which is equivalent to \(\Pr[A\mid B]=\Pr[A]\) when \(\Pr[B]\gt 0\). These are not synonyms.
If \(A\) and \(B\) are disjoint and both have positive probability, they cannot be independent: independence would need \(\Pr[AB]=\Pr[A]\Pr[B]\gt 0\), but disjointness gives \(\Pr[AB]=0\). Two coin tosses are independent: \(\Pr[\{hh\}]=(1/2)(1/2)=1/4\). “Male” and “aged 22” in a uniform-age group can happen together (not disjoint) and can still be independent.
For \(M\) events, mutual independence needs the product formula for every subcollection, not just every pair. Pairwise independence does not imply mutual independence of the whole collection.
Split the world into a partition \(\{B_i\}\) with \(\Pr[B_i]\gt 0\). Then \(A\) happens by happening with one of the \(B_i\):
$$ \Pr[A]=\sum_{i=1}^M \Pr[A\mid B_i]\Pr[B_i]. $$Every step: from the partition, \(\Pr[A]=\sum_i\Pr[A\cap B_i]\). By definition of conditional, \(\Pr[A\cap B_i]=\Pr[A\mid B_i]\Pr[B_i]\). Substitute. Two-set form, used constantly with a complement:
$$ \Pr[A]=\Pr[A\mid B]\Pr[B]+\Pr[A\mid B^C]\Pr[B^C]. $$Do not sum only the conditionals, and do not use overlapping \(B_i\).
You care about a hidden cause \(B\) (disease, card type). You observe data \(A\) (test, colour). Bayes flips \(\Pr[\text{data}\mid\text{cause}]\) into \(\Pr[\text{cause}\mid\text{data}]\). From \(\Pr[A\mid B]\Pr[B]=\Pr[AB]=\Pr[B\mid A]\Pr[A]\),
$$ \Pr[B\mid A]=\frac{\Pr[A\mid B]\,\Pr[B]}{\Pr[A]}. $$Posterior = likelihood \(\times\) prior / evidence. Expand the evidence with total probability. The exam workhorse is the two-hypothesis (complement) form:
$$ \Pr[B\mid A]=\frac{\Pr[A\mid B]\Pr[B]}{\Pr[A\mid B]\Pr[B]+\Pr[A\mid B^C]\Pr[B^C]}. $$Lung disease, order trap. \(A=\) lung disease, \(B=\) smoker. Given \(\Pr[A]=0.1\), \(\Pr[B]=0.2\), \(\Pr[B\mid A]=0.3\). Want \(\Pr[A\mid B]\):
$$ \Pr[A\mid B]=\frac{\Pr[B\mid A]\Pr[A]}{\Pr[B]}=\frac{0.3\cdot 0.1}{0.2}=0.15. $$The given 30% is “among patients with lung disease, 30% smoke” = \(\Pr[B\mid A]\), not \(\Pr[A\mid B]\).
HIV, complement in the denominator (reader Ex. 2.6). Prior \(\Pr[H]=1/5000=0.0002\), so \(\Pr[H^C]=0.9998\). Test correct 99% of the time: \(\Pr[+\mid H]=0.99\) and \(\Pr[+\mid H^C]=0.01\).
$$ \begin{aligned} \Pr[H\mid +] &=\frac{\Pr[+\mid H]\Pr[H]}{\Pr[+\mid H]\Pr[H]+\Pr[+\mid H^C]\Pr[H^C]}\\ &=\frac{(0.99)(0.0002)}{(0.99)(0.0002)+(0.01)(0.9998)}=0.0194. \end{aligned} $$Rare disease: most positives are false. A second independent test: yesterday’s posterior is today’s prior. \(\Pr[H\mid +]=0.0194\), \(\Pr[H^C\mid +]=0.9806\), and each test still has \(\Pr[++\mid H]=0.99\), \(\Pr[++\mid H^C]=0.01\):
$$ \Pr[H\mid ++]=\frac{(0.99)(0.0194)}{(0.99)(0.0194)+(0.01)(0.9806)}=0.662. $$Three-card colour (SLT1 2024–25 1.2). Cards RR, BB, RB, drawn uniformly. You see a red face. Probability the other side is black = \(\Pr[RB\mid R]\).
$$ \begin{aligned} \Pr[RB\mid R] &=\frac{\Pr[R\mid RB]\Pr[RB]}{\Pr[R\mid RR]\Pr[RR]+\Pr[R\mid RB]\Pr[RB]+\Pr[R\mid BB]\Pr[BB]}\\ &=\frac{(1/2)(1/3)}{(1)(1/3)+(1/2)(1/3)+(0)(1/3)}=\frac{1/6}{1/2}=\frac13. \end{aligned} $$Naive “two remaining cards, RR vs RB” gives \(1/2\) and is wrong: RR has two red faces, so it is twice as likely to produce the observed red. Count faces, not cards.
Lyme ticks (SLT1 2024–25 1.1). \(\Pr[L]=0.16\), \(\Pr[H]=0.10\), and “10% of the ticks that had either Lyme or HGE carried both”:
$$ \Pr[LH\mid L\cup H]=0.10. $$Because \(LH\subset L\cup H\), that is \(\Pr[LH]/\Pr[L\cup H]=0.10\). Union: \(\Pr[L\cup H]=\Pr[L]+\Pr[H]-\Pr[LH]\). So
$$ \Pr[LH]=0.10\bigl(\Pr[L]+\Pr[H]-\Pr[LH]\bigr). $$Collect \(\Pr[LH]\):
$$ \Pr[LH]+0.10\Pr[LH]=0.10(0.16+0.10)\implies 1.1\,\Pr[LH]=0.026\implies \Pr[LH]=\frac{0.026}{1.1}=0.0236. $$Then \(\Pr[H\mid L]=\Pr[LH]/\Pr[L]=0.0236/0.16=0.1475\).
Intuition picture (“Where is J from?”): prior Dutch \(17/445\approx 4\%\). Description fits 60% of Dutch and 20% of non-Dutch. In a sample of 250: 10 Dutch, 240 not. Fits: \(0.6\times 10=6\) Dutch, \(0.2\times 240=48\) non-Dutch. Posterior \(6/(6+48)\approx 11\%\), not 60% and not 90%. Lots of non-Dutch \(\times\) a weaker match outnumbers few Dutch \(\times\) a stronger match.
SLT1 2025–26 1.1. Deck \(\{2,3,4\}\), dealt in random order, six equally likely deals. Even cards: 2 and 4. Odd: 3. \(E_i=\{\text{\(i\)-th card even}\}\), \(O_i=\{\text{\(i\)-th card odd}\}\). List \(\mathcal{S}\), then compute
\(\mathcal{S}=\{234,243,324,342,423,432\}\), each \(1/6\).
(a) \(E_1=\{234,243,423,432\}\) (four deals: first is 2 or 4). \(E_1\cap E_2=\{243,423\}\) (the two evens in the first two seats, 3 last). \(\Pr[E_2\mid E_1]=(2/6)/(4/6)=1/2\).
(b) First two even given third even: then 2 and 4 occupy the first two seats, so the third card is 3 (odd). Impossible: \(\Pr[E_1 E_2\mid E_3]=0\).
(c) First odd \(\Rightarrow\) first is 3 \(\Rightarrow\) remaining 2, 4 are both even \(\Rightarrow\) second is always even: \(\Pr[E_2\mid O_1]=1\).
(d) Two odds in a three-card deck with only one odd card: \(\Pr[O_2\mid O_1]=0\).
Outcomes may be labels. An RV \(X\) is a rule \(\mathcal{S}\to\mathbb{R}\). Capital \(X\) is the RV; a realization is lowercase \(x\); the range is \(S_X\).
A deterministic signal is a formula that gives \(x[n]\) exactly (a square wave). A stochastic signal cannot be predicted sample by sample; at best you know statistics. Real measurements are useful part plus noise, so treat them as random. Additive model: \(x(t)=s(t)+n(t)\) with \(s\) deterministic, \(n\) random. A random process is a waveform per outcome; that is the next chapter. Here one experiment still produces one number.
Dice sum: discrete. Voltage in \([-5,5]\) mV: continuous. Phone-call duration rounded to 1 min if it lasted \(\lt 60\) s: mixed.
PMF, discrete: probability of each point, bars that sum to 1,
$$ p_X(x)=\Pr[X=x],\qquad P_X(x)=\Pr[X\le x]=\sum_{x_i\le x}p_X(x_i). $$Continuous CDF and PDF:
$$ P_X(x)=\Pr[X\le x]=\int_{-\infty}^x p_X(u)\,du,\qquad p_X(x)=\frac{dP_X(x)}{dx}. $$CDF properties: \(0\le P_X(x)\le 1\); \(P_X(-\infty)=0\), \(P_X(+\infty)=1\); nondecreasing; \(P_X(x_2)-P_X(x_1)=\Pr[x_1\lt X\le x_2]\) for \(x_2\gt x_1\).
PMF: \(0\le p_X(x)\le 1\), zero off the range, \(\sum_x p_X(x)=1\). PDF: \(p_X(x)\ge 0\), no upper bound (a thin spike can be taller than 1), \(\int_{-\infty}^{\infty}p_X(x)\,dx=1\). For a continuous RV, \(\Pr[X=x_0]=0\).
Why \(\Pr[X=0.5]=0\) for Uniform\((0,1)\). Infinitely many points. If each had probability \(\varepsilon\gt 0\), the total would be infinite. Axiom 2 forces point probabilities to 0; only intervals have positive probability.
Mixed (reader Ex. 2.8). \(X\sim\mathrm{Exp}(\lambda=1)\), \(p_X(x)=e^{-x}\) for \(x\ge 0\). Administrative \(Y=1\) if \(X\lt 1\), else \(Y=X\). Atom at 1:
$$ \Pr[Y=1]=\Pr[X\lt 1]=\int_0^1 e^{-x}\,dx=1-e^{-1}. $$ $$ p_Y(y)=(1-e^{-1})\,\delta(y-1)+e^{-y}\,u(y-1). $$The Dirac \(\delta\) is the only way a density can carry a finite probability at a point.
A family is a formula plus a few knobs. You need the experiment, the PMF, mean and variance. You do not need every appendix derivation.
| Family | Experiment | Mean | Variance |
|---|---|---|---|
| Bernoulli\((p)\) | one yes/no trial | \(p\) | \(p(1-p)\) |
| binomial\((n,p)\) | \(n\) independent Bernoulli | \(np\) | \(np(1-p)\) |
| geometric\((p)\) | trial number of first success, support \(1,2,\dots\) | \(1/p\) | \((1-p)/p^2\) |
| Poisson\((\alpha)\) | counts in an interval, mean \(\alpha\) | \(\alpha\) | \(\alpha\) |
Success \(X=1\) with probability \(p\), failure \(X=0\) with \(1-p\).
$$ p_X(x)=\begin{cases}1-p,& x=0\\ p,& x=1\\ 0,& \text{otherwise.}\end{cases} $$\(E[X]=0\cdot(1-p)+1\cdot p=p\). Variance, every step:
$$ \begin{aligned} E[X^2]&=0^2(1-p)+1^2 p=p,\\ \mathrm{Var}[X]&=E[X^2]-(E[X])^2=p-p^2=p(1-p). \end{aligned} $$\(n\) independent Bernoulli\((p)\) trials, \(X=\) number of successes. Conditions: \(n\) fixed in advance; two outcomes; independent trials; constant \(p\).
$$ p_X(x)=\binom{n}{x}p^x(1-p)^{n-x},\quad x=0,1,\dots,n,\qquad \binom{n}{x}=\frac{n!}{x!(n-x)!}. $$Write \(X=X_1+\cdots+X_n\) with each \(X_i\) Bernoulli. Linearity does not need independence: \(E[X]=\sum E[X_i]=np\). Independence makes variances add: \(\mathrm{Var}[X]=\sum\mathrm{Var}[X_i]=np(1-p)\). One Bernoulli is binomial\((1,p)\).
\(X=\) trial number of the first success, support \(x=1,2,\dots\). (The other convention, number of failures before the first success, starts at 0. This course starts at 1.)
$$ p_X(x)=p(1-p)^{x-1},\quad x=1,2,\dots $$CDF for \(x\ge 1\): geometric series
$$ P_X(x)=\sum_{n=1}^x p(1-p)^{n-1}=p\frac{1-(1-p)^x}{p}=1-(1-p)^x. $$Mean \(1/p\). Each day you buy a ticket with probability \(1/2\); \(K=\) day of first ticket (SLT1 2024–25 1.4(b)):
$$ p_K(k)=(1/2)(1/2)^{k-1}=(1/2)^k,\quad k=1,2,\dots $$\(E[X]=\mathrm{Var}[X]=\alpha\). Recognition test: mean equals variance. \(\alpha\) is a mean count, not a probability.
SLT1 2024–25 1.3. Five children, \(\Pr(\text{girl})=1/2\), independent, \(X=\) number of girls. Name the family, write \(p_X(x)\), and compute \(\Pr[X\ge 1]\).
\(X\sim\mathrm{binomial}(n=5,p=0.5)\), \(p_X(x)=\binom{5}{x}(1/2)^5\). Complement of “at least one” is “none”:
$$ \Pr[X\ge 1]=1-\Pr[X=0]=1-\binom{5}{0}(1/2)^5=1-\frac{1}{32}=0.96875\approx 0.97. $$Bell curve, location \(\mu\), width \(\sigma\). After you add many small independent noises, you get (approximately) this shape — that is the CLT, last section. Thermal noise is modelled Gaussian for that reason.
$$ p_X(x)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right),\quad\sigma\gt 0. $$\(E[X]=\mu\), \(\mathrm{Var}[X]=\sigma^2\). Symmetric about \(\mu\). Standard normal: \(\mu=0\), \(\sigma^2=1\). VL picture: professor arrival \(T\sim\mathcal{N}(0,5^2)\), about 68% within \(\pm 5\) min of 9:00.
Standardisation must be automatic. If \(Y\sim\mathcal{N}(\mu_Y,\sigma_Y^2)\),
$$ Z=\frac{Y-\mu_Y}{\sigma_Y}\sim\mathcal{N}(0,1). $$Q-function: right tail of the standard normal,
$$ Q(x)=\Pr[Z\gt x]=\frac{1}{\sqrt{2\pi}}\int_x^{\infty}e^{-u^2/2}\,du. $$Relations: \(Q(x)=1-Q(-x)=1-\Phi(x)\), and \(\Phi(x)=P_Z(x)=\Pr[Z\le x]\). Reader also writes \(Q(-x)=\Phi(x)\). For a general Gaussian \(Y\),
$$ \Pr[Y\gt y]=Q\left(\frac{y-\mu}{\sigma}\right),\qquad \Pr[Y\le y]=Q\left(-\frac{y-\mu}{\sigma}\right)=\Phi\left(\frac{y-\mu}{\sigma}\right). $$If the A4 or the calculator has error functions:
$$ \Phi(x)=\frac12\left(1+\mathrm{erf}\frac{x}{\sqrt{2}}\right),\qquad Q(x)=\frac12\mathrm{erfc}\left(\frac{x}{\sqrt{2}}\right). $$Empirical rules: \(\Pr[|X-\mu|\lt\sigma]\approx 68.2\%\), \(\lt 2\sigma\approx 95.4\%\), \(\lt 3\sigma\approx 99.7\%\). Put a short \(\Phi/Q\) table on the A4. Values used on SLT1 2025–26: \(\Phi(0)=0.5\), \(\Phi(0.5)=0.692\), \(\Phi(1.5)=0.933\), \(\Phi(2)=0.977\).
Template, \(V\sim\mathcal{N}(0,2^2)\), \(\sigma=2\):
$$ \Pr[V\gt 4]=1-\Pr[V\le 4]=1-\Phi\!\left(\frac{4-0}{2}\right)=1-\Phi(2)=1-0.977=0.023. $$Same statement: \(Q(2)=0.023\). Recipe for every Gaussian tail: write the event, subtract \(\mu\), divide by \(\sigma\) (not \(\sigma^2\)), then \(\Phi\) or \(Q\), and use \(Q(-x)=1-Q(x)\).
SLT1 2025–26 1.3. Use \(\Phi(0)=0.5\), \(\Phi(0.5)=0.692\), \(\Phi(1.5)=0.933\), \(\Phi(2)=0.977\).
(a) \(\Pr[V\gt 4]=1-\Phi(4/2)=1-\Phi(2)=1-0.977=0.023\). Same: \(Q(2)=0.023\).
(b) \(\Pr[W\le 2]=\Phi((2-2)/5)=\Phi(0)=1/2\).
(c) \(\Pr[X\le\mu+1]=\Phi(1/2)=\Phi(0.5)=0.692\).
(d) \(\Pr[Y\gt 65]=1-\Phi((65-50)/10)=1-\Phi(1.5)=1-0.933=0.067\).
Mean = centre of gravity of the distribution, or the long-run average of infinitely many independent draws. Variance = expected squared distance from the mean. LOTUS: to get \(E[g(X)]\) you do not need the law of \(g(X)\); average \(g(x)\) with the law of \(X\).
$$ E[X^m]=\begin{cases}\sum_{x\in S_X} x^m p_X(x) & \text{discrete}\\[4pt] \int_{-\infty}^{\infty} x^m p_X(x)\,dx & \text{continuous.}\end{cases} $$Mean: \(m=1\), \(\mu_X=E[X]\). LOTUS:
$$ E[g(X)]=\sum g(x)p_X(x)\quad\text{or}\quad \int g(x)p_X(x)\,dx. $$Pairs: \(E[g(X,Y)]=\sum\sum g(x,y)p_{X,Y}(x,y)\) or \(\iint g\,p_{X,Y}\,dx\,dy\). Linearity: \(E[aX+bY]=aE[X]+bE[Y]\), no independence required. Constants: \(E[z]=z\).
Variance \(\mathrm{Var}[X]=\sigma_X^2=E[(X-\mu_X)^2]\). The identity \(\mathrm{Var}[X]=E[X^2]-\mu_X^2\), every step:
$$ \begin{aligned} \mathrm{Var}[X] &=E[(X-\mu_X)^2]\\ &=E[X^2-2\mu_X X+\mu_X^2]\\ &=E[X^2]-2\mu_X E[X]+E[\mu_X^2]\\ &=E[X^2]-2\mu_X\cdot\mu_X+\mu_X^2\\ &=E[X^2]-\mu_X^2. \end{aligned} $$Std: \(\sigma_X=\sqrt{\mathrm{Var}[X]}\).
\(Y=aX+b\). Mean: \(E[Y]=aE[X]+b=a\mu_X+b\). Variance, every step:
$$ \begin{aligned} \mathrm{Var}[Y] &=E[(Y-E[Y])^2]\\ &=E[(aX+b-(a\mu_X+b))^2]\\ &=E[a^2(X-\mu_X)^2]\\ &=a^2 E[(X-\mu_X)^2]\\ &=a^2\mathrm{Var}[X]. \end{aligned} $$The shift \(b\) dies; the scale \(a\) squares. Sign of \(a\) does not matter. This identity is a free-standing SLT (2025–26 1.4).
Sample mean \(\bar x=(1/n)\sum x_i\). Sample variance: biased uses \(1/n\), unbiased uses \(1/(n-1)\) because the mean was already estimated from the same data. Sample moments become true moments only as \(n\to\infty\).
Two numbers from one experiment. The joint law lives on the plane. A marginal pretends the other variable does not exist (integrate or sum it out). A conditional is the joint restricted to a slice, renormalised.
Joint CDF: \(P_{X,Y}(x,y)=\Pr[X\le x,\,Y\le y]\). Joint PDF: \(p_{X,Y}=\partial^2 P_{X,Y}/(\partial x\partial y)\). Joint PMF: \(p_{X,Y}(x,y)=\Pr[X=x,Y=y]\). SLT alias for the joint PDF: \(f_{X,Y}\).
Marginalisation:
$$ p_X(x)=\sum_y p_{X,Y}(x,y)\qquad\text{or}\qquad p_X(x)=\int_{-\infty}^{\infty}p_{X,Y}(x,y)\,dy. $$Proof, continuous: \(P_X(x)=\Pr[X\le x]=\Pr[X\le x,Y\le\infty]=\int_{-\infty}^x\int_{-\infty}^{\infty}p_{X,Y}(v,u)\,du\,dv\); differentiate in \(x\). Conditional given the other RV:
$$ p_{X\mid Y}(x\mid y)=\frac{p_{X,Y}(x,y)}{p_Y(y)}. $$Independence of RVs: \(p_{X,Y}(x,y)=p_X(x)p_Y(y)\). Then \(p_{X\mid Y}=p_X\). Given an event \(B\), the conditional joint is \(p_{X,Y}(x,y)/\Pr[B]\) on \(B\) and 0 else — restrict the region and divide by \(\Pr[B]\).
Wedge (reader Ex. 2.11). Joint \(p_{X,Y}(x,y)=C e^{-x}\) on \(x\ge 0\) and \(-x\le y\le x\). Normalise:
$$ \int_0^\infty\int_{-x}^x C e^{-x}\,dy\,dx=C\int_0^\infty e^{-x}\cdot 2x\,dx=2C=1\implies C=\tfrac12. $$Marginal \(X\): \(\int_{-x}^x (1/2)e^{-x}\,dy=x e^{-x}\) for \(x\ge 0\). Marginal \(Y\): integrate \(x\) from \(|y|\) to \(\infty\), get \(p_Y(y)=\frac12 e^{-|y|}\). Conditional: \(p_{Y\mid X}(y\mid x)=1/(2x)\) on \(-x\le y\le x\) (uniform on that interval).
Always sketch the support. Limits are the whole problem. Integrating over a rectangle when the support is a triangle is the standard way to lose the marks.
SLT1 2025–26 1.2. Joint density \(f_{X,Y}(x,y)=1/2\) on \(-1\le x\le y\le 1\) (the triangle above the line \(y=x\), between \(x=-1\) and \(y=1\)).
(a) Region \(X\gt 0\): \(x\) from 0 to 1, for each \(x\) the coordinate \(y\) from \(x\) to 1.
$$ \Pr[X\gt 0]=\int_0^1\int_x^1 \tfrac12\,dy\,dx=\int_0^1\frac{1-x}{2}\,dx=\tfrac14. $$(b) For \(x\in[-1,1]\), \(y\) runs from \(x\) to 1:
$$ p_X(x)=\int_x^1\tfrac12\,dy=\frac{1-x}{2}. $$(c) LOTUS on the marginal:
$$ \begin{aligned} E[X] &=\int_{-1}^1 x\cdot\frac{1-x}{2}\,dx=\frac12\int_{-1}^1(x-x^2)\,dx =\frac12\left[\frac{x^2}{2}-\frac{x^3}{3}\right]_{-1}^1\\ &=\frac12\left(\Big(\tfrac12-\tfrac13\Big)-\Big(\tfrac12+\tfrac13\Big)\right) =\frac12\Big(\tfrac16-\tfrac56\Big)=-\frac13. \end{aligned} $$Independence: joint = product of marginals (no information either way). Uncorrelated: covariance 0 — the linear part of one does not help linearly predict the other. Orthogonal: \(E[XY]=0\) (raw product, not centred).
Independence \(\Rightarrow\) uncorrelated. Converse false in general. Converse true for jointly Gaussian.
$$ \mathrm{Cov}[X_1,X_2]=E[(X_1-\mu_1)(X_2-\mu_2)]=E[X_1 X_2]-\mu_1\mu_2. $$This course’s correlation of two RVs is \(r_{X_1,X_2}=E[X_1 X_2]\). Uncorrelated: \(\mathrm{Cov}=0\), i.e. \(E[X_1 X_2]=\mu_1\mu_2\). Orthogonal: \(r_{X_1,X_2}=0\). If either mean is 0, uncorrelated \(\Leftrightarrow\) orthogonal. Correlation coefficient:
$$ \rho=\frac{\mathrm{Cov}[X_1,X_2]}{\sigma_1\sigma_2},\qquad -1\le\rho\le 1. $$\(\rho\) normalises covariance, not \(E[XY]\). Do not write \(\rho=E[XY]/(\sigma_1\sigma_2)\) without centering.
Jointly Gaussian vector \(\mathbf{X}\sim\mathcal{N}_k(\mu,\Sigma)\):
$$ p_{\mathbf{X}}(\mathbf{x})=\frac{1}{\sqrt{(2\pi)^k|\Sigma|}}\exp\Big(-\frac12(\mathbf{x}-\mu)^T\Sigma^{-1}(\mathbf{x}-\mu)\Big). $$Zero off-diagonal in \(\Sigma\) \(\Leftrightarrow\) the quadratic form separates \(\Leftrightarrow\) the joint factors \(\Leftrightarrow\) independence. A linear image of a Gaussian vector is Gaussian, so independence of coordinates of \(\mathbf{Y}=Q\mathbf{X}\) is equivalent to \(\mathrm{Cov}[Y_1,Y_2]=0\).
Rotation (SLT2 2025–26 2.1): \(Y=QX\) with
$$ Q=\begin{bmatrix}\cos\theta & -\sin\theta\\ \sin\theta & \cos\theta\end{bmatrix},\quad C_X=\mathrm{diag}(\sigma_1^2,\sigma_2^2),\quad C_Y=Q C_X Q^T. $$Off-diagonal: \(\mathrm{Cov}[Y_1,Y_2]=(\sigma_1^2-\sigma_2^2)\sin\theta\cos\theta\). If \(\sigma_1^2=\sigma_2^2\), covariance is 0 for all \(\theta\) (isotropic Gaussian: independent for all rotations). If \(\sigma_2^2\neq\sigma_1^2\), you need \(\sin\theta\cos\theta=0\): axis-aligned rotations only.
Average (or sum) many independent similar random pieces. The histogram turns into a Gaussian, even if each piece is uniform or exponential. That is why measurement noise is modelled \(\mathcal{N}\).
i.i.d. case. \(X_1,\dots,X_N\) i.i.d., finite mean \(\mu\), finite \(\sigma\). For \(N\) large enough, the CDF of the sum (or of the average) converges to a Gaussian CDF. Average \(Y=\frac1N\sum_i X_i\):
$$ E[Y]=\mu,\qquad \mathrm{Var}[Y]=\frac{\sigma^2}{N}. $$Independent but not i.i.d.: still Gaussian in the limit if none of the terms dominates (Lyapunov). Skewed pieces need larger \(N\). Symmetric i.i.d. converge faster.
CD player (VL2-A). Each of 8 independent measurements \(X_i\sim\mathrm{Uniform}(v-0.5,v+0.5)\) mV, so \(E[X_i]=v\) and \(\mathrm{Var}[X_i]=1^2/12=1/12\). \(U=\frac18\sum_{i=1}^8 X_i\). CLT: \(U\approx\mathcal{N}(v,\mathrm{Var}[U])\) with
$$ \mathrm{Var}[U]=\frac{8\cdot(1/12)}{64}=\frac{1}{96}. $$Error \(U-v\approx\mathcal{N}(0,1/96)\). Two-sided tail:
$$ \Pr[|U-v|\gt 0.1] =2Q\left(\frac{0.1}{\sqrt{1/96}}\right) =2\bigl(1-\Phi(0.1\sqrt{96})\bigr)\approx 0.327. $$Course licence: sum of many independent noise sources \(\Rightarrow\) Gaussian (AWGN). Computational licence: replace an ugly 8-fold convolution by one \(Q\). Do not treat \(N=2\) of a very skewed law as exact, and do not mix the variance of a sum with the variance of an average.
5CTA0 Q1 2026–27 · from the course reader, lectures and official keys.