Estimation outputs a number. Detection outputs a label: \(\mathcal H_0\) or \(\mathcal H_1\). The two densities overlap, so every rule makes mistakes. The optimal rule is always a likelihood ratio test. Neyman–Pearson picks the threshold from \(P_{\mathrm{FA}}=\alpha\). Bayes picks it from costs and priors. After algebra you test a scalar. Which side of that scalar is \(\mathcal H_1\) is not always the right tail.
Exam block, every year: write \(L=p(\mathbf x\mid\mathcal H_1)/p(\mathbf x\mid\mathcal H_0)\), simplify to \(T\gtrless\lambda'\), re-derive the inequality, shade, then \(Q\) / erf / exponential CDF.
Model: \(\mathcal H_0:\ \mathbf x\sim p(\mathbf x\mid\mathcal H_0)\), \(\mathcal H_1:\ \mathbf x\sim p(\mathbf x\mid\mathcal H_1)\). A detector partitions observation space into \(\mathcal X_0\cup\mathcal X_1=\mathbb R^N\). If \(\mathbf x\in\mathcal X_i\), decide \(\mathcal H_i\).
| true \(\mathcal H_0\) | true \(\mathcal H_1\) | |
|---|---|---|
| decide \(\mathcal H_0\) | true negative (TN) | miss / false negative / type II |
| decide \(\mathcal H_1\) | false alarm / type I | detection / true positive |
Row = decision, column = truth. Memorise by axes, not by the words “positive / negative”.
$$P_D=\int_{\mathcal X_1}p(\mathbf x\mid\mathcal H_1)\,d\mathbf x,\qquad P_{\mathrm{FA}}=\int_{\mathcal X_1}p(\mathbf x\mid\mathcal H_0)\,d\mathbf x,\qquad P_M=1-P_D.$$If the test on a 1-D statistic is “decide \(\mathcal H_1\) when \(T>\lambda'\)” (usual right-tail mean-shift):
$$P_{\mathrm{FA}}=1-F_{T\mid\mathcal H_0}(\lambda'),\qquad P_D=1-F_{T\mid\mathcal H_1}(\lambda').$$The inequality can flip. If \(\mathcal H_1\) is the narrower density (small variance, large exponential rate), large \(|x|\) favours \(\mathcal H_0\), and you decide \(\mathcal H_1\) when the statistic is small. Then \(P_{\mathrm{FA}}\) is a left tail of \(p(\,\cdot\mid\mathcal H_0)\). Re-derive \(\gtrless\); do not memorise “right tail”.
Shading, always:
Reader Fig. 14.4 (mean-shift, decide \(\mathcal H_1\) if \(\bar x>\lambda'\)): left panel \(P_{\mathrm{FA}}\) shades the right tail of \(p(\bar x\mid\mathcal H_0)\) (centred at 0). Right panel \(P_D\) shades the right tail of \(p(\bar x\mid\mathcal H_1)\) (centred at \(A\)). The reader’s OCR caption on the left panel wrongly says “under \(\mathcal H_1\)”. Trust the figure, not the OCR.
You cannot move the threshold to improve both error types. Liberal (threshold toward the noise mean): more hits and more FAs. Conservative (threshold toward the signal mean): fewer FAs and more misses.
Regardless of NP or Bayes, the optimal test compares the likelihood ratio to a threshold:
$$\Lambda(\mathbf x)=L(\mathbf x)=\frac{p(\mathbf x\mid\mathcal H_1)}{p(\mathbf x\mid\mathcal H_0)}\;\underset{\mathcal H_0}{\overset{\mathcal H_1}{\gtrless}}\;\gamma.$$Decide \(\mathcal H_1\) if \(L\ge\gamma\), else \(\mathcal H_0\). \(L\) is a scalar even if \(\mathbf x\in\mathbb R^N\). \(\ln L\) is monotone, so the log-LRT is equivalent for \(\gamma>0\). After simplification one compares a sufficient statistic \(T(\mathbf x)\) to a new threshold \(\lambda'\). That \(\lambda'\) is not \(\gamma\); it absorbs constants, \(\sigma^2\), \(N\), \(\ln\gamma\).
Some old keys write \(L=p(\mathcal H_0)/p(\mathcal H_1)\) (Jan 2019 Answers_oldexams). Use \(L=p_1/p_0\). The probability table in that key is the standard right-tail one; the inverted ratio is a typesetting bug, not a different test.
Maximise \(P_D\) subject to \(P_{\mathrm{FA}}\le\alpha\). The optimum is an LRT. Any other region with the same \(P_{\mathrm{FA}}\) has strictly smaller \(P_D\).
Lowering \(\gamma\) enlarges \(\mathcal X_1\): both \(P_D\) and \(P_{\mathrm{FA}}\) increase.
Priors \(P_0=P(\mathcal H_0)\), \(P_1=1-P_0\). Cost \(C_{i,j}=\) cost of deciding \(\mathcal H_i\) when \(\mathcal H_j\) is true. Minimising Bayes risk again yields an LRT, now with a known threshold
$$\gamma=\frac{(C_{1,0}-C_{0,0})P_0}{(C_{0,1}-C_{1,1})P_1}.$$Usually \(C_{1,1}=C_{0,0}=0\). Then \(\gamma=C_{1,0}P_0/(C_{0,1}P_1)\). Minimum probability of error: \(C_{0,1}=C_{1,0}=1\), so \(\gamma=P_0/P_1\) and \(\mathcal R=P_0 P_{\mathrm{FA}}+P_1 P_M\). Equivalent MAP: pick the hypothesis with larger posterior.
NP vs Bayes \(\gamma\). NP: \(\lambda'\) from \(P_{\mathrm{FA}}=\alpha\) using only \(p(\,\cdot\mid\mathcal H_0)\). Bayes: \(\gamma\) from costs and priors, no \(\alpha\). “Bayesian detection maximises Bayes risk” is false (it minimises). Do not set \(\gamma=1\) unless the costs and priors actually give that.
Plot \(P_D\) vs \(P_{\mathrm{FA}}\) as \(\gamma\) runs from \(0\) (always say \(\mathcal H_1\): \(P_D=P_{\mathrm{FA}}=1\)) to \(\infty\) (always say \(\mathcal H_0\): both 0).
IID samples. Always start from \(L=\prod_n p(x_n\mid\mathcal H_1)/p(x_n\mid\mathcal H_0)\), take \(\ln\), dump constants into \(\gamma\).
Right-tail test on the sample mean. Under \(\mathcal H_0\), \(\bar x\sim\mathcal N(0,\sigma^2/N)\); under \(\mathcal H_1\), \(\bar x\sim\mathcal N(A,\sigma^2/N)\).
$$P_{\mathrm{FA}}=Q\Bigl(\frac{\lambda'}{\sigma/\sqrt N}\Bigr),\qquad P_D=Q\Bigl(\frac{\lambda'-A}{\sigma/\sqrt N}\Bigr).$$NP: \(\lambda'=(\sigma/\sqrt N)\,Q^{-1}(\alpha)\). Same algebra as the matched filter when the “mean” is a known waveform \(s_n\) instead of a DC (see below).
Nov 2019 Q4, official key.
$$\mathcal H_0:\ x[n]\sim\mathcal N(0,\sigma_0^2),\qquad\mathcal H_1:\ x[n]\sim\mathcal N(0,\sigma_1^2),\quad\sigma_1>\sigma_0.$$Both centred at 0; \(\mathcal H_1\) is wider. Large \(|x|\) favours \(\mathcal H_1\).
$$p(\mathbf x\mid\mathcal H_i)=(2\pi\sigma_i^2)^{-N/2}\exp\Bigl(-\frac1{2\sigma_i^2}\sum_{n=0}^{N-1}x[n]^2\Bigr).$$ $$L=\Bigl(\frac{\sigma_0}{\sigma_1}\Bigr)^N\exp\Bigl[\Bigl(\frac1{2\sigma_0^2}-\frac1{2\sigma_1^2}\Bigr)\sum_n x[n]^2\Bigr].$$Log-LRT (official with-answers key; the Answers_oldexams file drops the squares — typesetting bug):
$$\ln L=N\ln\frac{\sigma_0}{\sigma_1}-\frac{\sum x[n]^2}{2\sigma_1^2}+\frac{\sum x[n]^2}{2\sigma_0^2}\;\gtrless\;\ln\gamma.$$Because \(1/\sigma_0^2>1/\sigma_1^2\), the coefficient of \(\sum x^2\) is positive \(\Rightarrow\) decide \(\mathcal H_1\) when \(\sum x[n]^2\) is large (two-sided tails in 1-D). Energy detector, not a mean detector.
Single sample, \(L=1\): the PDFs cross at \(\pm x_{\mathrm{th}}\). Narrow tall peak \(=\mathcal H_0\), wide short peak \(=\mathcal H_1\). \(\mathcal X_1\) is outside the lines.
$$P_D=2\int_{x_{\mathrm{th}}}^\infty p(x\mid\mathcal H_1)\,dx,\qquad P_{\mathrm{FA}}=2\int_{x_{\mathrm{th}}}^\infty p(x\mid\mathcal H_0)\,dx.$$Official numbers. \(\sigma_1=3\sigma_0\), \(x_{\mathrm{th}}=\sigma_1=3\sigma_0\). Given for \(x\sim\mathcal N(0,\sigma^2)\):
$$\Pr(0\le x\le\sigma)=0.341,\quad\Pr(\sigma\le x\le 2\sigma)=0.136,\quad\Pr(2\sigma\le x\le 3\sigma)=0.021.$$Under \(\mathcal H_1\), \(x_{\mathrm{th}}=\sigma_1\) is one own-standard-deviation out:
$$P_D=2\bigl(0.5-0.341\bigr)=2\cdot 0.159=0.318.$$Under \(\mathcal H_0\), \(x_{\mathrm{th}}=3\sigma_0\) is three own-sd out:
$$\Pr(0\le x\le 3\sigma_0)=0.341+0.136+0.021=0.498,\qquad P_{\mathrm{FA}}=2(0.5-0.498)=0.004.$$Official: \(P_D=0.318\), \(P_{\mathrm{FA}}=0.004\).
2022 4c, no official key. Algebra follows the printed exam (including the flipped \(\gtrless\)).
$$p(x\mid\mathcal H_0)=\tfrac12 e^{-|x|},\qquad p(x\mid\mathcal H_1)=e^{-2|x|}.$$\(\mathcal H_1\) has larger rate \(\Rightarrow\) narrower Laplace. Large \(|x|\) favours \(\mathcal H_0\). One sample: \(L(x)=2e^{-|x|}\). IID:
$$L(\mathbf x)=2^N\exp\Bigl(-\sum_{n=0}^{N-1}|x_n|\Bigr).$$ $$2^N e^{-\sum|x_n|}\;\underset{\mathcal H_0}{\overset{\mathcal H_1}{\gtrless}}\;\gamma\qquad\Rightarrow\qquad -\sum|x_n|\;\underset{\mathcal H_0}{\overset{\mathcal H_1}{\gtrless}}\;\ln\gamma-N\ln 2.$$Multiply by \(-1\) (flip the inequality):
$$\sum_{n=0}^{N-1}|x_n|\;\underset{\mathcal H_1}{\overset{\mathcal H_0}{\gtrless}}\;N\ln 2-\ln\gamma=:\gamma'.$$Decide \(\mathcal H_0\) when the sum of absolute values is large. That is the printed 2022 4c inequality. Threshold \(\gamma'=\ln(2^N/\gamma)\).
For \(N=1\), \(Y=|X|\) is exponential: \(p(y\mid\mathcal H_0)=e^{-y}\), \(p(y\mid\mathcal H_1)=2e^{-2y}\), \(y>0\), and \(y\;\underset{\mathcal H_1}{\overset{\mathcal H_0}{\gtrless}}\;\gamma'\). At \(\gamma'=1\), decide \(\mathcal H_1\) for small \(y\) (left of 1).
Do not shade the usual right tail. Numerical 2022 4e–g values (\(P_{\mathrm{FA}}=0.1\Rightarrow P_D=0.19\), Bayes \(y=\ln 3\)) are derived from the printed exam; there is no official key for those numbers. Do not treat them as published letters.
Jan 2020 Q5, official key.
$$\mathcal H_0:\ p(x)=\lambda_0 e^{-\lambda_0 x},\qquad\mathcal H_1:\ p(x)=\lambda_1 e^{-\lambda_1 x},\qquad x\ge 0,\ \lambda_1>\lambda_0>0.$$\(\mathcal H_1\) decays faster \(\Rightarrow\) more mass near 0. Large \(x\) favours \(\mathcal H_0\).
$$L(x)=\frac{\lambda_1}{\lambda_0}\exp\bigl[-(\lambda_1-\lambda_0)x\bigr].$$\(L>\gamma\) iff \(x\) is smaller than a threshold (inequality flips). For \(\gamma=1\):
$$x_{\mathrm{th}}=\frac{\ln(\lambda_1/\lambda_0)}{\lambda_1-\lambda_0}.$$Official numbers \(\lambda_1=2\), \(\lambda_0=0.1\): \(x_{\mathrm{th}}=\ln 20/1.9\approx 1.58\).
CDF of \(\mathrm{Exp}(\lambda)\): \(P(X\le x_{\mathrm{th}})=1-e^{-\lambda x_{\mathrm{th}}}\). Because we decide \(\mathcal H_1\) when \(x<x_{\mathrm{th}}\):
$$P_D=1-e^{-\lambda_1 x_{\mathrm{th}}},\qquad P_{\mathrm{FA}}=1-e^{-\lambda_0 x_{\mathrm{th}}}.$$(Official writes \(\int_{-\infty}^{x_{\mathrm{th}}}\) but the exponential lives on \(x\ge 0\); the closed form is right.) Hint on the paper: design so \(P_D>P_{\mathrm{FA}}\) — that is the left-tail choice.
Gaussian, right-tail test \(x>\lambda'\), \(x\sim\mathcal N(\mu,\sigma^2)\):
$$P(x>\lambda')=Q\Bigl(\frac{\lambda'-\mu}{\sigma}\Bigr)=\frac12\Bigl[1-\operatorname{erf}\Bigl(\frac{\lambda'-\mu}{\sigma\sqrt 2}\Bigr)\Bigr].$$Two-sided energy test \(|x|>x_{\mathrm{th}}\), zero-mean: double the one-sided tail.
Exponential / Laplace folded to \(y>0\), rate \(\lambda\), CDF \(1-e^{-\lambda y}\):
\(\mathcal H_0=\mathcal N(0,1)\), \(\mathcal H_1=\mathcal N(\mu_1,\sigma_1)\), threshold \(\lambda'\), right-tail.
$$P_{\mathrm{FA}}=\tfrac12\bigl[1-\operatorname{erf}(\lambda'/\sqrt 2)\bigr],\qquad P_D=\tfrac12\bigl[1-\operatorname{erf}((\lambda'-\mu_1)/(\sigma_1\sqrt 2))\bigr].$$The table they give is \(\operatorname{erf}(Z/\sqrt 2)\) vs \(Z\), so you look up \(Z=(\lambda'-\mu)/\sigma\) directly. Values used in the key: \(Z=1\Rightarrow\operatorname{erf}=0.683\), \(Z=2\Rightarrow\operatorname{erf}=0.954\), and \(\operatorname{erf}\) is odd.
| \(\lambda'\) | \(\mu_1\) | \(\sigma_1\) | \(P_D\) | \(P_{\mathrm{FA}}\) |
|---|---|---|---|---|
| 1 | 1 | 1 | 0.5 | 0.159 |
| 1 | 3 | 1 | 0.977 | 0.159 |
| 1 | 3 | 2 | 0.842 | 0.159 |
| 2 | 3 | 1 | 0.842 | 0.023 |
How, first row: \(\lambda'=1\), \(\mathcal H_0\): \(Z=1\), \(\operatorname{erf}=0.683\), \(P_{\mathrm{FA}}=\frac12(1-0.683)=0.1585\approx 0.159\). Row \(\lambda'=1,\mu_1=3,\sigma_1=1\): \(Z=(1-3)/1=-2\), \(\operatorname{erf}(-2)=-0.954\), \(P_D=\frac12(1-(-0.954))=0.977\). Last row \(P_{\mathrm{FA}}\): \(Z=2\), \(\frac12(1-0.954)=0.023\).
\(P_{\mathrm{FA}}\) does not depend on \(\mu_1,\sigma_1\). Raising \(\mu_1\) at fixed threshold raises \(P_D\) only. Raising \(\sigma_1\) at fixed threshold drops \(P_D\) here (more \(\mathcal H_1\) mass leaks left of \(\lambda'\)). Raising \(\lambda'\) drops both.
Official key. \(p(y\mid\mathcal H_1)=\frac14 e^{-|y|/2}\) (Laplace), \(p(y\mid\mathcal H_0)=(2\pi)^{-1/2}e^{-y^2/2}\). Set \(L=1\):
$$\frac{\tfrac14 e^{-|y|/2}}{(2\pi)^{-1/2}e^{-y^2/2}}=1\Rightarrow -|y|+y^2=2\ln 4-\ln(2\pi)\Rightarrow y_{\mathrm{th}}=\pm 1.5884.$$Decide \(\mathcal H_1\) (Laplace, heavier tails) for large \(|y|\). Shade: \(P_M\) is the centre of the Laplace between \(\pm y_{\mathrm{th}}\); \(P_{\mathrm{FA}}\) is the Gaussian tails outside \(\pm y_{\mathrm{th}}\).
$$P_D=\int_{1.5884}^\infty\tfrac12 e^{-y/2}\,dy=e^{-1.5884/2}=0.4519,$$ $$P_{\mathrm{FA}}=2Q(1.5884)=2\times 0.0559=0.1118.$$(The integral uses evenness / the one-sided Laplace piece; equivalent to \(2\int_{y_{\mathrm{th}}}^\infty\frac14 e^{-y/2}\,dy\).) Official: \(y_{\mathrm{th}}=\pm 1.5884\), \(P_D=0.4519\), \(P_{\mathrm{FA}}=0.1118\).
Detect a known length-\(N\) waveform \(s_n\) in WGN:
$$\mathcal H_0:\ x_n=w_n,\qquad\mathcal H_1:\ x_n=s_n+w_n,\qquad w_n\sim\mathrm{IID}\ \mathcal N(0,\sigma^2).$$Mean-shift with a vector mean \(s\), not a DC. LRT:
$$\ln L=\frac1{\sigma^2}\sum_{n=0}^{N-1}x_n s_n-\frac1{2\sigma^2}\sum_n s_n^2\;\gtrless\;\ln\gamma\qquad\Rightarrow\qquad\sum_{n=0}^{N-1}x_n s_n\;\underset{\mathcal H_0}{\overset{\mathcal H_1}{\gtrless}}\;\sigma^2\ln\gamma+\tfrac12\sum_n s_n^2=\lambda'.$$Left side = correlator (inner product \(\mathbf x^\top\mathbf s\)). Implement as an FIR filter, sample at the end:
$$y_n=\sum_{l=0}^{N-1}x_l h_{n-l},\qquad h_n=s_{N-1-n}\quad\text{(time-reversed }s\text{)}.$$Then \(y_{N-1}=\sum_l x_l s_l\). Compare \(y_{N-1}\) to \(\lambda'\).
Why this maximises SNR. \(\mathrm{SNR}=\mathbb E^2[y_{N-1}\mid\mathcal H_1]/\mathrm{Var}[y_{N-1}\mid\mathcal H_0]=(\mathbf h^\top\mathbf s)^2/(\sigma^2\,\mathbf h^\top\mathbf h)\). Cauchy–Schwarz: \((\mathbf h^\top\mathbf s)^2\le(\mathbf h^\top\mathbf h)(\mathbf s^\top\mathbf s)\), equality iff \(\mathbf h=c\mathbf s\). Among all linear filters, the matched filter is SNR-optimal.
Resit 2020 / 2022 4a, official: “SNR at the output is maximised” is the true statement. \(h[n]=s[n]\) without time-reversal is false unless you correlate instead of convolve. The filter also exists for coloured noise after whitening; “only for uncorrelated noise” is not the definition.
Jan 2020 Q5b, official. Same exponentials as above, \(\lambda_0=0.1\). Find \(x_{\mathrm{th}}\) so that \(P_{\mathrm{FA}}=0.1\). Then find \(\lambda_1\) so that \(P_D=0.9\) at that threshold. Decide \(\mathcal H_1\) for small \(x\).
\(P_{\mathrm{FA}}=1-e^{-0.1\,x_{\mathrm{th}}}=0.1\Rightarrow x_{\mathrm{th}}=-10\ln 0.9\approx 1.053\approx 1.05\).
\(P_D=1-e^{-\lambda_1\cdot 1.05}=0.9\Rightarrow\lambda_1=\ln 10/1.05\approx 2.19\).
Official: \(x_{\mathrm{th}}=1.05\), \(\lambda_1=2.19\). Same paper, ROC points: \(P_{\mathrm{FA}}=0.5\Rightarrow x_{\mathrm{th}}\approx 6.93\), \(P_D\approx 1\); \(P_{\mathrm{FA}}=0.9\Rightarrow x_{\mathrm{th}}\approx 23\), \(P_D\approx 1\). Larger \(\lambda_1\) lifts the ROC toward the top-left.
Resit 24 Jan 2020 4a, official. Detect a known pulse in WGN. Which statement is true? (a) SNR at the output is maximised. (b) \(h[n]=s[n]\). (c) The matched filter exists only for uncorrelated noise.
Official: (a). Impulse response is \(s[-n]\) or \(s[N-1-n]\), not \(s[n]\), unless you correlate instead of convolve. A matched filter exists for coloured noise too, via whitening.
One sample, \(\mathcal H_0:\mathcal N(0,\sigma_0^2)\), \(\mathcal H_1:\mathcal N(0,\sigma_1^2)\) with \(\sigma_1>\sigma_0\). Thresholds at \(\pm x_{\mathrm{th}}\) where \(L=1\). Shade \(P_{\mathrm{FA}}\) and \(P_D\). What is the sufficient statistic for \(N\) samples?
Decide \(\mathcal H_1\) in the outer tails (wider density). \(P_{\mathrm{FA}}\): two tails of the narrow \(\mathcal H_0\) Gaussian outside \(\pm x_{\mathrm{th}}\). \(P_D\): two tails of the wide \(\mathcal H_1\) Gaussian outside \(\pm x_{\mathrm{th}}\). Statistic: \(\sum x[n]^2\). Official Nov 2019 numbers at \(x_{\mathrm{th}}=\sigma_1=3\sigma_0\): \(P_D=0.318\), \(P_{\mathrm{FA}}=0.004\).
5CTA0 Q1 2026–27 · from the course reader, lectures and official keys.