A process is a waveform per outcome. WSS is two checks; the spectrum is the DTFT of \(r[l]\); the innovation filter keeps every root inside the unit circle.
Same symbols as the previous chapter. Discrete-time lag \(l=n_1-n_2\); continuous lag \(\tau\). Autocorrelation \(r_X[n_1,n_2]\), or \(r_X[l]\) if WSS. Autocovariance \(C_X\) (SLT sometimes writes \(\gamma\)). PSD \(P_X(e^{j\theta})\) (VL also \(P(e^{j\omega})\)). Innovation filter \(L(z)\), \(\sigma_i^2\), \(l[0]=1\). Discrete time \(x[n]\); continuous \(x(t)\). Capital \(X\) is the process; lowercase is a realization. The reader writes Wiener–Kintchine; this page writes Wiener–Khinchin for the same theorem.
An RV is one random number. A random (stochastic) process is a random waveform: each outcome \(\zeta\) of the experiment is mapped to a whole time function \(x[n,\zeta]\) (or \(x(t,\zeta)\)).
The ensemble is all realizations. The process is the mapping from \(\mathcal{S}\) to time functions. A length-\(N\) snapshot is the random vector \(\mathbf{X}[n]=\bigl[X[n],X[n-1],\dots,X[n-N+1]\bigr]^T\). One recorded sequence is one realization, not the process. Two recordings are two realizations, not two processes.
Deterministic vs stochastic, again: a square wave vs the same square wave plus noise. ECG, radar return, any measured voltage. Additive white Gaussian noise (AWGN): \(x(t)=s(t)+n(t)\), \(n\) additive, white, Gaussian, usually zero-mean, so \(x(t)\sim\mathcal{N}(s(t),\sigma_n^2)\). Gaussian by the CLT (many electrons). White means a flat PSD, next sections.
At each time, look across the ensemble: that average is \(\mu_X[n]\). Autocorrelation: how a sample co-varies with another sample at a different time (ensemble product). Real or complex; drop \(*\) if real.
$$ \begin{aligned} \mu_X[n]&=E[X[n]],\\ \sigma_X^2[n]&=E\bigl[|X[n]-\mu_X[n]|^2\bigr],\\ C_X[n_1,n_2]&=E\bigl[(X[n_1]-\mu_X[n_1])(X[n_2]-\mu_X[n_2])^*\bigr],\\ r_X[n_1,n_2]&=E\bigl[X[n_1]X^*[n_2]\bigr]. \end{aligned} $$Relation: \(C_X[n_1,n_2]=r_X[n_1,n_2]-\mu_X[n_1]\mu_X^*[n_2]\). Cross versions \(C_{XY}\), \(r_{XY}\) replace the second process. Continuous-time analogues: \(r(t_1,t_2)=E[X(t_1)X(t_2)]\), and VL/examples often write \(R_X(\tau)\). First line of every WSS test: write \(\mu\) and \(r\) from the definition, before simplifying.
Stationary = the statistics do not care what time it is. Strict-sense: the whole distribution of any finite collection of samples is shift-invariant. Wide-sense (WSS, what exams use): weaker — mean constant, autocorrelation depends only on the lag. Strict \(\Rightarrow\) WSS. WSS \(\not\Rightarrow\) strict, except special cases (Gaussian processes: 1st + 2nd order fix the whole law).
WSS, two checks:
Then \(r_X[l]=E[X[n]X^*[n-l]]\). Joint WSS of two processes: each WSS and \(r_{XY}[n_1,n_2]=r_{XY}[l]\).
Product of independent WSS processes (reader Ex. 3.2). Independent WSS \(X(t),Y(t)\). \(W(t)=X(t)Y(t)\).
$$ E[W(t)]=E[X]E[Y]=\mu_X\mu_Y\quad\text{(constant).} $$ $$ R_W(t,\tau)=E[X(t)Y(t)X(t+\tau)Y(t+\tau)]=R_X(\tau)R_Y(\tau) $$by independence, a function of \(\tau\) only, so \(W\) is WSS. Cross: \(R_{WX}(t,\tau)=E[X(t)Y(t)X(t+\tau)]=\mu_Y R_X(\tau)\) \(\Rightarrow\) jointly WSS.
Random constant (SLT2 2025–26 2.2(a)). \(X(t)=A\), \(A\sim U(0,1)\). Each realization is a horizontal line at a random height \(a_i\).
$$ E[X(t)]=E[A]=\tfrac12\quad\text{(constant).} $$ $$ r_X(t_1,t_2)=E[A\cdot A]=E[A^2]=\mathrm{Var}[A]+(E[A])^2=\tfrac1{12}+\tfrac14=\tfrac13, $$independent of \((t_1,t_2)\), hence a (constant) function of \(\tau=t_1-t_2\). WSS. Ergodicity fails; that is the next section.
Fixed-phase random-amplitude cosine (SLT2 2025–26 2.2(b)). \(X_n=A\cos(\omega_0 n)\), \(A\sim\mathcal{N}(0,1)\). Each realization is a cosine of fixed phase, random amplitude.
$$ \mu_X(n)=E[A]\cos(\omega_0 n)=0\quad\text{(constant — first WSS check OK).} $$ $$ r_X(n_1,n_2)=E[A^2]\cos(\omega_0 n_1)\cos(\omega_0 n_2)=\cos(\omega_0 n_1)\cos(\omega_0 n_2). $$Product-to-sum: \(\frac12\cos(\omega_0(n_1-n_2))+\frac12\cos(\omega_0(n_1+n_2))\). The second term depends on \(n_1+n_2\), not only on the lag. Visual: lag = one period, the product is 0 at zero-crossings and \(\approx A^2\) at peaks. Not WSS. If the phase were random uniform on \([0,2\pi]\) and independent of \(A\), the process would be WSS. That is the standard trap.
For a WSS process:
If a computed \(r[l]\) is not even for a real process, an algebra bug. If \(r[1]\gt r[0]\), a bug. Do not write \(r[0]=\sigma^2\) when \(\mu\neq 0\).
Energy = sum of squares (total size). Power = average energy per sample.
$$ E_s=\sum_{n=-\infty}^{\infty}|x[n]|^2,\qquad P_s=\lim_{N\to\infty}\frac1N\sum_{n=0}^{N-1}|x[n]|^2. $$Energy signal: finite \(E_s\). Then \(P_s=0\) if the duration is infinite. Example: a single pulse. Power signal: finite nonzero \(P_s\) (hence infinite energy). Example: any nonzero bounded everlasting signal. Random signals of infinite duration are treated as aperiodic power signals. For WSS processes, \(P_s=r_X[0]=E[|X[n]|^2\). Exam sentence: we talk about PSD not ESD for noise because noise is modelled as a power signal.
Samples uncorrelated with each other; only \(r[0]\) is nonzero. Spectrum is flat (“white light contains all colours equally”). Zero-mean white:
$$ r_X[l]=\sigma_X^2\delta[l]=\begin{cases}\sigma_X^2,& l=0\\ 0,& l\neq 0.\end{cases} $$Wiener–Khinchin \(\Rightarrow\) \(P_X(e^{j\theta})=\sigma_X^2\) for all \(\theta\). If also Gaussian, uncorrelated \(\Rightarrow\) independent (IID sequence). White is a 2nd-order statement; Gaussian is a distribution. White is not IID unless Gaussian (or otherwise specified). Filter an IID sequence \(\Rightarrow\) coloured (nonzero \(r[l]\) for some \(l\neq 0\)), still WSS if the filter is stable LTI.
\(r[l]\) says how much memory in time. PSD says how power is spread over frequency. They are a Fourier pair for WSS signals. Definition:
$$ P_X(e^{j\theta})=\lim_{N\to\infty}\frac{1}{2N+1}E\left[\left|\sum_{n=-N}^N x[n]e^{-jn\theta}\right|^2\right]. $$Wiener–Khinchin: for WSS \(x[n]\),
$$ P_X(e^{j\theta})=\sum_{l=-\infty}^{\infty} r_X[l]e^{-jl\theta},\qquad r_X[l]=\frac{1}{2\pi}\int_{-\pi}^{\pi}P_X(e^{j\theta})e^{jl\theta}\,d\theta. $$Properties: real-valued; periodic in \(\theta\) with period \(2\pi\); if \(x\) is real, \(P\) is even, \(P(e^{j\theta})=P(e^{-j\theta})\); \(P_X(e^{j\theta})\ge 0\). Average power:
$$ \frac{1}{2\pi}\int_{-\pi}^{\pi}P_X(e^{j\theta})\,d\theta=r_X[0]=E[|x[n]|^2]\ge 0. $$Cross-PSD \(P_{XY}(e^{j\theta})=\sum_l r_{XY}[l]e^{-jl\theta}\), and \(P_{XY}(e^{j\theta})=P_{YX}^*(e^{j\theta})\). Do not apply WK to a non-WSS process. Do not forget the \(1/(2\pi)\) on the inverse. White pair: \(\sigma^2\delta[l]\leftrightarrow\sigma^2\).
AR(1) shape (reader Ex. 3.3). Zero-mean WSS, \(r_X[l]=\alpha^{|l|}\), \(|\alpha|\lt 1\). Split the DTFT at 0 and subtract the double-counted \(l=0\) term:
$$ \begin{aligned} P_X(e^{j\theta}) &=\sum_{l=-\infty}^{0}\alpha^{-l}e^{-jl\theta}+\sum_{l=0}^{\infty}\alpha^{l}e^{-jl\theta}-1\\ &=\sum_{p=0}^{\infty}(\alpha e^{j\theta})^p+\sum_{l=0}^{\infty}(\alpha e^{-j\theta})^l-1\\ &=\frac{1}{1-\alpha e^{j\theta}}+\frac{1}{1-\alpha e^{-j\theta}}-1 =\frac{1-\alpha^2}{1+\alpha^2-2\alpha\cos\theta}. \end{aligned} $$“Find \(P\) from \(r\)” = DTFT. “Find \(r\) from \(P\)” = inverse DTFT.
Stationarity says the ensemble looks the same at every time. Ergodicity says one infinitely long realization already contains the ensemble: time averages = ensemble averages. In the lab you have one finite record, so you need this, or you cannot estimate \(\mu,r\).
Ergodicity \(\Rightarrow\) stationarity (course slogan). Stationarity \(\not\Rightarrow\) ergodicity. Strict-sense stationary \(X\) is strict-sense ergodic if time average = ensemble average for statistics of all orders.
Ergodic in the mean (continuous): \(\lim_{T\to\infty}(1/(2T))\int_{-T}^T x(t)\,dt=\mu_X=E[X(t)]\), in the sense that the variance of the time average \(\to 0\). Discrete, finite window:
$$ E\left\{\frac{1}{2N+1}\sum_{n=-N}^{N}x[n]\right\}=\mu_X, \qquad \mathrm{Var}\left\{\frac{1}{2N+1}\sum_{n=-N}^{N}x[n]\right\}\xrightarrow{N\to\infty}0. $$The first says the time-average estimator is unbiased; the second says it is consistent. Both are required. Ergodic in the autocorrelation (zero-mean form in the reader): the time average of \(x(t)x(t+\tau)\) equals \(R_X(\tau)\), again with variance \(\to 0\).
Dice forever. Each realization is an IID sequence of fair-die rolls. Time average along one sequence \(\to 3.5\); ensemble mean at any \(n\) is \(3.5\). Strict-sense ergodic: every realization is statistically a copy of the process.
Random constant / resistor. Pick \(A\sim U(0,1)\) once; \(X(t)=A\) for all \(t\). Ensemble mean \(1/2\). Time average of realization \(i\) is \(a_i\), which is not \(1/2\) except on a null set. Variance of the time average is \(\mathrm{Var}[A]=1/12\not\to 0\). WSS but not ergodic in the mean. You cannot learn the ensemble mean by watching one resistor forever. The fixed-phase cosine of the previous section is not WSS, so this course does not discuss its ergodicity.
SLT2 2025–26 2.2. For each process: is it WSS? If it is WSS, is it ergodic in the mean?
Write \(\mu\) and \(r\) from the definition before concluding.
(a) \(E[X(t)]=E[A]=1/2\) constant. \(r_X(t_1,t_2)=E[A^2]=1/12+1/4=1/3\), independent of time, so a function of the lag. WSS. Time average of realization \(i\) is \(a_i\neq 1/2\), and \(\mathrm{Var}[\hat\mu]=\mathrm{Var}[A]=1/12\not\to 0\). Not ergodic in the mean.
(b) Mean \(E[A]\cos(\omega_0 n)=0\), constant. Autocorrelation \(\cos(\omega_0 n_1)\cos(\omega_0 n_2)=\frac12\cos(\omega_0(n_1-n_2))+\frac12\cos(\omega_0(n_1+n_2))\). The sum-frequency term depends on \(n_1+n_2\). Not WSS. Ergodicity is not asked.
You have \(N\) samples of one ergodic realization. Approximate \(E[\cdot]\) by a time average. For lag \(l\) only \(N-|l|\) products exist. Dividing by \(N\) (too big) biases the estimate toward 0 at large lags. Dividing by \(N-|l|\) is unbiased, but noisy at large \(|l|\) (one product when \(|l|=N-1\)).
$$ \hat\mu_X=\frac1N\sum_{n=0}^{N-1}x[n]. $$Biased ACF — what the course normally uses:
$$ \hat r_X[l]=\frac1N\sum_{n=0}^{N-|l|-1}x[n+|l|]x^*[n] $$and even extension for \(l\lt 0\). Unbiased would replace \(N\) by \(N-|l|\). Reason to prefer biased: the unbiased ACF is not guaranteed positive-semidefinite; its DTFT can go negative and is a bad PSD estimator (Ch. 12). Keep \(|l|\le L\ll N\). The estimated autocorrelation matrix is Toeplitz, symmetric for real signals, and (with the biased estimator) positive semidefinite. “Write the estimator used in practice” \(\to\) divide by \(N\), not \(N-|l|\).
You cannot write the output waveform (it is random), but you can write how \(\mu\), \(r\) and \(P\) transform. Stable LTI: \(\sum|h[k]|\lt\infty\) (all poles inside the unit circle). Input WSS \(\Rightarrow\) output WSS.
$$ \mu_y=\mu_x\sum_k h[k]=\mu_x H(e^{j0}). $$ $$ r_{yx}[l]=h[l]*r_x[l],\qquad r_{xy}[l]=h^*[-l]*r_x[l], $$ $$ r_y[l]=h[l]*h^*[-l]*r_x[l]=r_h[l]*r_x[l]. $$Frequency / \(z\):
$$ P_y(e^{j\theta})=|H(e^{j\theta})|^2 P_x(e^{j\theta}),\qquad P_y(z)=H(z)H^*(1/z^*)P_x(z). $$If \(x\) and \(h\) are real: \(P_y(z)=H(z)H(z^{-1})P_x(z)\). White input \(P_x=\sigma_i^2\):
$$ P_y(e^{j\theta})=\sigma_i^2|H(e^{j\theta})|^2,\qquad r_y[l]=\sigma_i^2\sum_k h[k]h^*[k-l]. $$SLT2 2024–25 2.3 / reader Ex. 3.4. IID zero-mean unit-variance input \(x[n]\) (white). \(y[n]=x[n]+x[n-1]\). Show that \(y\) is WSS and find \(r_y[l]\). Are adjacent outputs independent?
\(E[y[n]]=0\), constant. With \(l=n_1-n_2\) and \(r_x[l]=\delta[l]\),
$$ \begin{aligned} r_y[n_1,n_2] &=E[(x[n_1]+x[n_1-1])(x[n_2]+x[n_2-1])]\\ &=2r_x[l]+r_x[l-1]+r_x[l+1], \end{aligned} $$ $$ r_y[l]=\begin{cases}2-|l|,& l\in\{-1,0,1\}\\ 0,& \text{otherwise.}\end{cases} $$Depends only on \(l\) \(\Rightarrow\) WSS (an MA(1) process). Same lag-structure if the filter is \(x[n]+x[n+1]\). Adjacent outputs share an \(x\)-sample, so they are not independent. Filter view: \(h=[1,1]\), \(r_y[0]=\sum h[k]^2=2\), \(r_y[\pm 1]=1\).
Model a WSS sequence as an LTI driven by white innovations \(i[n]\) of variance \(\sigma_i^2\). Convention \(a_0=b_0=1\).
Pendulum with random kicks = AR(2). Light-bulb production with last month’s stock error = ARMA(1,1).
AR(\(p\)), all-pole:
$$ H(z)=\frac{1}{1+\sum_{k=1}^p a_k z^{-k}}=\frac{1}{A(z)},\qquad x[n]=i[n]-a_1 x[n-1]-\cdots-a_p x[n-p]. $$MA(\(q\)), all-zero:
$$ H(z)=1+\sum_{k=1}^q b_k z^{-k}=B(z),\qquad x[n]=i[n]+b_1 i[n-1]+\cdots+b_q i[n-q]. $$ARMA(\(p,q\)):
$$ H(z)=\frac{B(z)}{A(z)},\qquad x[n]=i[n]+\sum_{k=1}^q b_k i[n-k]-\sum_{k=1}^p a_k x[n-k]. $$Pole inside the unit circle: that mode is causal and stable. Zero inside the unit circle: the inverse is causal and stable too. Minimum-phase system: all poles and all zeros strictly inside \(|z|=1\) (system and inverse both causal and stable). A pole or zero with \(|z|\gt 1\) is not min-phase. On the unit circle: excluded for the usual factorisation. For AR, min-phase of \(H\) reduces to: roots of \(A(z)\) inside the unit circle. Real AR(1): pole at \(z=-a_1\), need \(|a_1|\lt 1\).
AR: infinite, recursive (Yule–Walker). With \(a_0=1\),
$$ \sum_{k=0}^p a_k r_x[l-k]=\sigma_i^2\delta[l], $$ $$ r[l]=\begin{cases} \sigma_i^2-\sum_{k=1}^p a_k r_x[|l|-k],& l=0,\\ -\sum_{k=1}^p a_k r_x[|l|-k],& |l|\gt 0. \end{cases} $$AR(1) fully (reader Ex. 4.3). \(x[n]=i[n]-a_1 x[n-1]\), \(|a_1|\lt 1\), real. Use \(E[i[n]x[n-l]]=\sigma_i^2\delta[l]\) (future/present innovation independent of past \(x\); at lag 0 the new \(i[n]\) is in \(x[n]\)):
$$ r[0]=\sigma_i^2-a_1 r[1],\qquad r[1]=-a_1 r[0] $$(\(r\) even). Substitute:
$$ r[0]=\sigma_i^2-a_1(-a_1 r[0])=\sigma_i^2+a_1^2 r[0] \implies r[0](1-a_1^2)=\sigma_i^2 \implies r[0]=\frac{\sigma_i^2}{1-a_1^2}. $$ $$ r[1]=-a_1\frac{\sigma_i^2}{1-a_1^2},\qquad r[l]=\frac{\sigma_i^2}{1-a_1^2}(-a_1)^{|l|}. $$Same shape as Ex. 3.3 with \(\alpha=-a_1\).
MA: finite support, dies after lag \(q\), with \(b_0=1\):
$$ r_x[l]=\begin{cases}\sigma_i^2\sum_{k=|l|}^q b_k b_{k-|l|},& |l|\le q\\ 0,& |l|\gt q.\end{cases} $$ARMA: MA-like for \(|l|\le q\), then pure AR recursion for \(|l|\gt q\) (modified Yule–Walker). Need impulse-response samples \(h[k]\) in the \(|l|\le q\) equations.
ARMA(1,1) ACF (reader Ex. 4.5). \(H(z)=(1-\frac13 z^{-1})/(1-\frac12 z^{-1})\), \(\sigma_i^2=1\). So \(b_1=-1/3\), \(a_1=-1/2\). Polynomial division: \(H(z)=1+\frac16 z^{-1}+\cdots\), hence \(h[0]=1\), \(h[1]=1/6\). Yule–Walker at lags 0 and 1:
$$ \begin{aligned} r[0]&=b_0 h[0]+b_1 h[1]-a_1 r[1]=1-\tfrac13\cdot\tfrac16+\tfrac12 r[1],\\ r[1]&=b_1 h[0]-a_1 r[0]=-\tfrac13+\tfrac12 r[0]. \end{aligned} $$Solve: \(r[0]=28/27\), \(r[1]=5/27\). For \(|l|\ge 1\), \(r[l]=-a_1 r[|l|-1]=(1/2)r[|l|-1]\), so
$$ r[l]=\begin{cases}28/27,& l=0\\ \frac{5}{27}(1/2)^{|l|-1},& |l|\ge 1.\end{cases} $$Identify the model from the difference equation or from \(H(z)\). AR \(\Leftrightarrow\) infinite ACF, poles, peaks. MA \(\Leftrightarrow\) ACF cutoff at \(q\), zeros, valleys.
Most WSS processes with a continuous PSD can be built as white noise through a causal filter \(L(z)\) (Wold). Many filters share the same \(|H|^2\) (same \(P\)). To pick one: demand minimum-phase and \(l[0]=1\). That unique choice is the innovation filter.
Picture: \(P(e^{j\theta})=\frac54-\cos\theta\) comes from either \(L(z)=1-\frac12 z^{-1}\), \(\sigma_i^2=1\) (zero at \(1/2\), min-phase) or \(L(z)=1-2z^{-1}\), \(\sigma_i^2=1/4\) (zero at 2, not min-phase). Keep the first.
Rational PSD:
$$ P(z)=\sigma_i^2 L(z)L(z^{-1}), $$with \(L(z)\) causal, stable, min-phase (poles and zeros inside \(|z|=1\)), and \(l[0]=1\) fixing the scale in \(\sigma_i^2\). Roots of a valid real PSD come in mirrored pairs: if \(z_k\) is a root, so is \(1/z_k^*\) (and conjugates). Of the assignments, keep those with \(|\text{root}|\lt 1\). On the unit circle, \(L(z)L(z^{-1})\) agrees with \(|L(e^{j\theta})|^2\) for real coefficients; the \(z\)-identity is \(L(z)L(z^{-1})\), not \(L(z)L^*(z)\).
Given \(P(e^{j\theta})\) as a ratio of cosines:
Root method: for every zero/pole \(z_0\) with \(|z_0|\gt 1\), move it to \(1/z_0^*\) and absorb the gain into \(\sigma_i^2\). That is the min-phase / all-pass split: \(H(z)=H_{\min}(z)H_{ap}(z)\), and \(|H_{ap}(e^{j\theta})|=1\) so the PSD is unchanged.
Worked, every step (reader Ex. 4.1).
$$ P(e^{j\theta})=\frac{5-4\cos\theta}{10-6\cos\theta}. $$ $$ P(z)=\frac{5-2(z+z^{-1})}{10-3(z+z^{-1})}=\frac{c_n}{c_d}\frac{1-a z^{-1}}{1-b z^{-1}}\frac{1-a z}{1-b z}. $$Numerator: \(c_n(1+a^2)=5\), \(c_n a=2\).
Denominator: \(c_d(1+b^2)=10\), \(c_d b=3\).
SLT2 2025–26 2.3. Zero-mean process with
$$ P_X(e^{j\omega})=\frac{5+3\cos\omega}{17+8\cos\omega}. $$Find the min-phase innovation filter \(L(z)\) (also written \(H_{\min}(z)\)) and \(\sigma_i^2\). Write the corresponding ARMA difference equation. Keep every root of magnitude \(\lt 1\).
Substitute \(\cos\omega=(z+z^{-1})/2\):
$$ P(z)=\frac{5+\frac32(z+z^{-1})}{17+4(z+z^{-1})} =\frac{10+3(z+z^{-1})}{34+8(z+z^{-1})} =\frac{3z^2+10z+3}{8z^2+34z+8}. $$Factor: \(3z^2+10z+3=3(z+3)(z+1/3)\), \(8z^2+34z+8=8(z+4)(z+1/4)\). In \((1+\alpha z^{-1})(1+\alpha z)\) form,
$$ \bigl(1+\tfrac13 z^{-1}\bigr)\bigl(1+\tfrac13 z\bigr)=\tfrac{10}{9}+\tfrac13(z+z^{-1}); $$times 9 recovers the numerator \(10+3(z+z^{-1})\). Denominator: \(\bigl(1+\tfrac14 z^{-1}\bigr)\bigl(1+\tfrac14 z\bigr)\) times 32 recovers \(34+8(z+z^{-1})\). Thus
$$ P(z)=\frac{9}{32}\frac{(1+\frac13 z^{-1})(1+\frac13 z)}{(1+\frac14 z^{-1})(1+\frac14 z)}. $$Keep the roots of magnitude \(1/3\) and \(1/4\):
$$ L(z)=\frac{1+\frac13 z^{-1}}{1+\frac14 z^{-1}},\qquad \sigma_i^2=\frac{9}{32}. $$Difference equation (\(b_1=1/3\), \(a_1=1/4\)):
$$ x[n]=-\tfrac14 x[n-1]+w[n]+\tfrac13 w[n-1],\qquad \sigma_w^2=\tfrac{9}{32}. $$A length-\(k\) snapshot is a random vector. Same tools as joint densities: integrate over the support. Linear maps, reader §3.2.5: if \(\mathbf{Y}=A\mathbf{X}+b\), then \(\mu_Y=A\mu_X+b\) and \(C_Y=A C_X A^T\). That is the engine behind the Gaussian rotation. Correlation matrix \(R_X=E[\mathbf{X}\mathbf{X}^T]\); covariance \(C_X=R_X-\mu\mu^T\).
SLT2 2024–25 2.2. Support \(0\le x_1\le x_2\le 1\), joint density \(p_{\mathbf{X}}=2\). Compute \(\mu=E[\mathbf{X}]\), the correlation matrix \(R_X=E[\mathbf{X}\mathbf{X}^T]\), and the covariance \(C_X\).
Area of the triangle is \(1/2\), so \(2\cdot 1/2=1\). Integrate \(x_1\) from 0 to \(x_2\), then \(x_2\) from 0 to 1:
$$ E[X_1]=\int_0^1\int_0^{x_2} 2x_1\,dx_1\,dx_2=\int_0^1 x_2^2\,dx_2=\tfrac13, \qquad E[X_2]=\int_0^1 2x_2\cdot x_2\,dx_2=\tfrac23. $$ $$ E[X_1^2]=\int_0^1\tfrac23 x_2^3\,dx_2=\tfrac16,\quad E[X_2^2]=\int_0^1 2x_2^3\,dx_2=\tfrac12,\quad E[X_1 X_2]=\int_0^1 x_2^3\,dx_2=\tfrac14. $$ $$ R_X=\begin{bmatrix}1/6 & 1/4\\ 1/4 & 1/2\end{bmatrix},\qquad C_X=R_X-\mu\mu^T =\begin{bmatrix}1/6 & 1/4\\ 1/4 & 1/2\end{bmatrix} -\begin{bmatrix}1/9 & 2/9\\ 2/9 & 4/9\end{bmatrix} =\begin{bmatrix}1/18 & 1/36\\ 1/36 & 1/18\end{bmatrix}. $$5CTA0 Q1 2026–27 · from the course reader, lectures and official keys.