Task 2.2a, explained from absolute zero
0 · What problem are we solving?
1 · The Gaussian PDF, piece by piece
2 · Calculus refresher (derivatives, chain rule, product rule)
3 · Deriving ∂p/∂μ
4 · Deriving ∂p/∂σ
5 · The sign puzzle: why a rising curve can have a negative derivative
6 · What the Jacobian actually is
7 · Why anyone needs these derivatives (the link to 2.2b)
8 · Watch it move
9 · Check yourself
0 · What problem are we solving?
You have a recording of a heart. Between every two consecutive beats you measure the time gap. That list of gaps is the R-R interval series — roughly 2000 numbers, each around 0.9 seconds, all slightly different.
The first thing you want to know about any list of numbers is what shape is it. So you draw a histogram: chop the horizontal axis (gap length, in seconds) into ~30 bins, and for each bin draw a bar whose height is how many gaps landed in it. The result looks like a lumpy bell.
A bell shape suggests we can describe all 2000 numbers with just two numbers — a centre \(\mu\) and a width \(\sigma\) — if we can find the bell curve that hugs those histogram bars best. That is the entire job of Task 2.2.
Task 2.2a is only the mathematical warm-up: work out how the bell curve's height responds when you nudge \(\mu\) or \(\sigma\). Those responses are two derivatives. You will need them in 2.2b, where the actual fitting loop uses them to correct its guesses. 2.2a does not fit anything. It just prepares the two formulas.
1 · The Gaussian PDF, piece by piece
A probability density function (PDF) is a function \(p(x)\) that answers: "how likely is a value near \(x\)?" Its defining property is that the area under it equals 1 (total probability). The Gaussian (normal) PDF is:
$$p(x)=\frac{1}{\sqrt{2\pi}\,\sigma}\,\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$$Read it as three separate pieces multiplied together:
| Piece | What it does |
|---|---|
| \(\exp\!\big(-\tfrac{(x-\mu)^2}{2\sigma^2}\big)\) | The shape. It equals 1 when \(x=\mu\) (the peak) and shrinks towards 0 as \(x\) moves away from \(\mu\). It shrinks faster when \(\sigma\) is small. |
| \(\dfrac{1}{\sqrt{2\pi}\,\sigma}\) | The height knob. Purely there to force the total area to 1. A narrow bell must be tall (small area covered, so it needs height); a wide bell must be short. |
| \(\mu,\ \sigma\) | The two dials we are fitting. \(\mu\) = where the peak sits. \(\sigma\) = how wide the bell is. \(\sigma^2\) is called the variance. |
The two dials, in words
- Turn \(\mu\) → the whole bell slides sideways. Its shape and height do not change.
- Turn \(\sigma\) → the bell stretches or squeezes. Widening it lowers the peak and lifts the shoulders, because the area has to stay 1.
Everything in Task 2.2a is the answer to two questions: at a fixed point \(x\), how fast does the curve's height change when I turn dial \(\mu\)? and when I turn dial \(\sigma\)?
Two useful facts for the derivations, so you can check them yourself:
- The peak height is \(p(\mu)=\dfrac{1}{\sqrt{2\pi}\,\sigma}\). For \(\sigma=1\) that is \(0.3989\).
- At one \(\sigma\) away from the peak, \(p(\mu\pm\sigma)=\dfrac{e^{-1/2}}{\sqrt{2\pi}\,\sigma}\approx\dfrac{0.6065}{2.5066\,\sigma}\). For \(\sigma=1\): \(\approx 0.2420\).
2 · Calculus refresher (the only tools you need)
Skip this section if derivatives are fresh. If not, everything in Task 2.2a is built from the three rules below.
Derivative = rate of change
\(\dfrac{df}{dx}\) answers: "if I nudge the input by a tiny amount, how much does the output move?" Positive derivative = output rises with input. Negative = falls. Zero = flat spot.
Partial derivative = one dial at a time
Here the function has two inputs: \(p(x;\mu,\sigma)\). We ask about one dial at a time, holding the other frozen. The curly \(\partial\) (instead of \(d\)) is just the notation that says "the others are constant":
$$\frac{\partial p}{\partial \mu}=\text{change in height per unit turn of }\mu,\quad\text{with }\sigma\text{ frozen.}$$Rule 1 — chain rule
For a function wrapped in an exponential, keep the exponential and multiply by the derivative of whatever is inside it:
$$\frac{d}{dt}e^{g(t)}=e^{g(t)}\cdot g'(t)$$That single rule produces both answers we need. It is also the reason every result below comes out as "\(p\) times something" — the exponential never goes away, it stays as the prefactor \(p\).
Rule 2 — power rule (inside the exponent)
$$\frac{d}{dt}\big(t^n\big)=n\,t^{\,n-1}\qquad\text{plus the chain rule if }t\text{ is itself a function}$$We will repeatedly need \(\frac{d}{d\mu}(x-\mu)^2=2(x-\mu)\cdot(-1)=-2(x-\mu)\), because the inside of the square changes by \(-1\) when \(\mu\) goes up by 1.
Rule 3 — product rule
When a product of two things depends on the same dial, differentiate one at a time and add:
$$\frac{d}{dt}(uv)=u'v+uv'$$We need this for the \(\sigma\) dial, because both the height knob \(\frac{1}{\sigma}\) and the exponent contain \(\sigma\).
3 · Deriving ∂p/∂μ
Write the Gaussian with the constant \(\frac{1}{\sqrt{2\pi}}\) pulled out of the way, because it does not contain \(\mu\) or \(\sigma\) at all:
$$p=\frac{1}{\sqrt{2\pi}}\,\sigma^{-1}\,\exp\!\underbrace{\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)}_{\text{call this }g}$$Now differentiate with respect to \(\mu\). The prefactor \(\frac{1}{\sqrt{2\pi}}\sigma^{-1}\) is a constant here, so it just rides along. Only \(g\) depends on \(\mu\):
$$\frac{\partial p}{\partial \mu}=\frac{1}{\sqrt{2\pi}}\,\sigma^{-1}\,e^{g}\cdot\frac{\partial g}{\partial \mu}$$which is exactly \(p\cdot\frac{\partial g}{\partial \mu}\), because \(p\) is that same prefactor times \(e^{g}\). So we only have to differentiate the exponent:
$$\frac{\partial g}{\partial \mu}=\frac{\partial}{\partial \mu}\left[-\frac{(x-\mu)^2}{2\sigma^2}\right] =-\frac{1}{2\sigma^2}\cdot\underbrace{\frac{\partial}{\partial\mu}(x-\mu)^2}_{=\,2(x-\mu)\cdot(-1)} =-\frac{1}{2\sigma^2}\cdot\big(-2(x-\mu)\big)=\frac{x-\mu}{\sigma^2}$$Putting it together, and writing the answer as "\(p\) times something" (the standard clean form):
$$\boxed{\ \frac{\partial p}{\partial \mu}=p\cdot\frac{x-\mu}{\sigma^2}\ }$$Sanity checks on the formula
- At the peak (\(x=\mu\)): the factor is \(0\), so \(\frac{\partial p}{\partial\mu}=0\). Sliding a bell sideways does not change its height at its own summit. ✓
- Right of the peak (\(x>\mu\)): factor positive. Sliding the bell right raises the height at a point that sits on its right flank. ✓
- Left of the peak (\(x<\mu\)): factor negative — the case that confuses people, explained in section 5. ✓
- Numerical: \(\sigma=1\), \(x=\mu+\sigma\). Then \(\frac{\partial p}{\partial\mu}=0.2420\cdot\frac{1}{1}=0.2420\). Notice it equals the height at that point — that is not a coincidence, since \(\frac{x-\mu}{\sigma^2}=1\) there.
4 · Deriving ∂p/∂σ
Now \(\sigma\) appears in two places: the height knob \(\sigma^{-1}\) and the exponent \(g=-\frac{(x-\mu)^2}{2\sigma^2}\). So we must use the product rule. Set \(u=\sigma^{-1}\) and \(v=e^{g}\), with \(p=\frac{1}{\sqrt{2\pi}}uv\).
Step 1 — derivative of the height knob
$$\frac{\partial}{\partial \sigma}\left(\sigma^{-1}\right)=-\sigma^{-2}=-\frac{1}{\sigma^2}$$Step 2 — derivative of the exponent
Rewrite \(g=-\frac{(x-\mu)^2}{2}\sigma^{-2}\) so the power rule applies. Then
$$\frac{\partial g}{\partial \sigma}=-\frac{(x-\mu)^2}{2}\cdot(-2)\sigma^{-3}=\frac{(x-\mu)^2}{\sigma^{3}}$$so by the chain rule \(v'=e^{g}\cdot\frac{(x-\mu)^2}{\sigma^3}=v\cdot\frac{(x-\mu)^2}{\sigma^3}\).
Step 3 — product rule and tidy up
$$\frac{\partial p}{\partial \sigma}=\frac{1}{\sqrt{2\pi}}\Big(u'v+uv'\Big) =\frac{1}{\sqrt{2\pi}}\left(-\frac{1}{\sigma^2}e^{g}+\sigma^{-1}\,e^{g}\,\frac{(x-\mu)^2}{\sigma^3}\right)$$Now factor out \(p=\frac{1}{\sqrt{2\pi}}\sigma^{-1}e^{g}\) from both terms — i.e. divide the whole bracket by \(p\)'s ingredients:
$$\boxed{\ \frac{\partial p}{\partial \sigma}=p\left(\frac{(x-\mu)^2}{\sigma^{3}}-\frac{1}{\sigma}\right)\ }$$Reading the two terms (this is the intuition)
- \(-\dfrac{1}{\sigma}\) comes from the height knob. Widening the bell must lower the whole thing (area stays 1) → this term always pushes the height down.
- \(+\dfrac{(x-\mu)^2}{\sigma^{3}}\) comes from the shape. Widening stretches the bell outwards, which lifts points that sit away from the centre → this term pushes the height up, and it grows the further \(x\) is from \(\mu\).
Which term wins depends on where you look. Near the peak the first wins (height drops). Far out on the flanks the second wins (height rises). They cancel exactly in between.
Sanity checks on the formula
- At the peak (\(x=\mu\)): \(\frac{\partial p}{\partial\sigma}=-p/\sigma<0\). Widening lowers the summit. ✓
- Exactly at one \(\sigma\) away (\(x=\mu\pm\sigma\)): \((x-\mu)^2/\sigma^3=1/\sigma\), so the two terms cancel and \(\frac{\partial p}{\partial\sigma}=0\). Numbers: \(\sigma=1\), \(x=1\), \(p=0.2420\), and \(0.2420\,(1-1)=0\). ✓ So the curve crosses zero at \(\pm1\sigma\).
- Far out (\(x=\mu\pm\sqrt{3}\,\sigma\)): \(p=0.1295\) and \((3-1)=2\), so \(\frac{\partial p}{\partial\sigma}=+0.2590>0\). Widening lifts the shoulders. ✓
A compact form worth memorising
Define the standardised distance from the peak, \(z=\dfrac{x-\mu}{\sigma}\) (how many \(\sigma\)'s away you are). Both answers collapse to:
$$\frac{\partial p}{\partial \mu}=\frac{p\,z}{\sigma},\qquad\qquad \frac{\partial p}{\partial \sigma}=\frac{p\,(z^2-1)}{\sigma}$$That is the whole of Task 2.2a in two short expressions. The first is antisymmetric in \(z\) (flips sign across the peak); the second is symmetric (same value on both flanks) and negative for \(|z|<1\), positive for \(|z|>1\) — the peak-down/shoulders-up behaviour, in one line.
5 · The sign puzzle
A fair objection: on the left flank the yellow curve is rising, so why is \(\frac{\partial p}{\partial\mu}\) negative there?
Because these two quantities answer completely different questions:
- \(\dfrac{\partial p}{\partial x}\) — "walking along the horizontal axis, does the height go up?" On the left flank, yes.
- \(\dfrac{\partial p}{\partial \mu}\) — "standing still at a fixed \(x\), I slide the entire bell to the right. Does the height above my feet go up?"
They are related by a clean identity. Sliding the curve right by a small \(\delta\) is the same as leaving the curve alone and sampling it a little to the left, at \(x-\delta\). So the change in height is \(-\delta\) times the slope:
$$\frac{\partial p}{\partial \mu}=-\,\frac{\partial p}{\partial x}$$The hill analogy. You stand still on the left flank of a hill. The hill itself slides to the right. Because the summit is moving away from you, the ground under your feet goes down — even though the flank you are standing on rises towards the summit. You are not walking uphill; the hill is leaving.
- Standing on the left flank (curve rising) → ground drops → \(\partial p/\partial\mu\) negative.
- Standing on the right flank (curve falling) → more hill arrives under you → \(\partial p/\partial\mu\) positive.
- Standing exactly at the summit → flat spot, nothing changes to first order → zero. This is why the blue curve always crosses zero exactly under the green dashed peak line.
So never read those panels as "the slope of the picture". Read them as "what does turning this one dial do to the picture at this exact spot". That reframing is the point of the task.
6 · What the Jacobian actually is
A "Jacobian" sounds heavy but here it is only a table of the two answers you just derived, evaluated at every histogram bin. If there are \(B\) bins, at bin \(i\) the centre is \(x_i\) and the fitted height is \(p(x_i)\), and we store one row per bin:
$$J=\begin{pmatrix} \dfrac{\partial p}{\partial \mu}(x_1) & \dfrac{\partial p}{\partial \sigma}(x_1)\\[8pt] \vdots & \vdots\\[4pt] \dfrac{\partial p}{\partial \mu}(x_B) & \dfrac{\partial p}{\partial \sigma}(x_B) \end{pmatrix}\in\mathbb{R}^{B\times 2}$$Row \(i\) reads: "at bin \(i\), how sensitive is the curve's height to the \(\mu\) dial (first column) and to the \(\sigma\) dial (second column)?" Nothing more. In code it is exactly two lines, one per column:
dp_dmu = p * (x_bins - mu) / sig**2
dp_dsig = p * ((x_bins - mu)**2 / sig**3 - 1/sig)
J = np.column_stack([dp_dmu, dp_dsig])7 · Why anyone needs these derivatives
Because Task 2.2b has to fit the curve numerically: there is no formula that spits out the best \(\mu,\sigma\) for a histogram in one shot, so the algorithm iterates — guess, measure the error, correct, repeat.
At each iteration you have a current guess and an error vector \(\varepsilon=y-p(x)\), where \(y\) are the histogram heights. The question is: which direction, and how far, should I move the two dials to reduce that error? The Jacobian is precisely the missing information — it says how much each bin's error responds to each dial. The Gauss–Newton update is then one line of linear algebra:
$$\Delta\theta=(J^{\top}J)^{-1}J^{\top}\varepsilon,\qquad \theta\leftarrow\theta+\alpha\,\Delta\theta,\qquad \theta=[\mu,\ \sigma]^{\top}$$In words: \(J^{\top}\varepsilon\) mixes the errors with the sensitivities to find which way helps, and \((J^{\top}J)^{-1}\) converts that into a sensible step size. With hand-derived derivatives the fit converges in about 6 iterations; without them you would be twisting dials blindly. That is the entire reason Task 2.2a exists.
8 · Watch it move
Top: the bell curve with its two dials. Bottom: the same two derivative formulas, plotted against \(x\), as the dials turn.- Yellow, top panel — the Gaussian being fitted. The green dashed line marks the peak \(x=\mu\).
- Blue, bottom panel — \(\partial p/\partial\mu\): the \(\mu\) dial, while \(\sigma\) is frozen. \(\mu\) slides right, then left. Watch it cross zero exactly under the green line, negative on the side the peak is retreating from, positive on the other.
- Red, bottom panel — \(\partial p/\partial\sigma\): the \(\sigma\) dial, while \(\mu\) is frozen. \(\sigma\) widens. Negative at the peak (summit drops), zero at \(\pm1\sigma\), positive on the shoulders (they lift).
Both curves are scaled by 3 so they are visible; the shapes and signs are exact.
9 · Check yourself
If you can do these four, Task 2.2a is yours.
1. Where is \(\partial p/\partial\mu = 0\), and why?
At \(x=\mu\), the peak, because the factor \((x-\mu)\) vanishes: sliding a bell sideways does not change its height at its own summit.
2. Where does \(\partial p/\partial\sigma\) change sign?
At \(x=\mu\pm\sigma\). Inside that range the height-knob term dominates (widening lowers the curve); outside it the shape term dominates (widening lifts it).
3. Redo the \(\partial p/\partial\mu\) derivation on paper without looking.
Write \(p=C\sigma^{-1}e^{g}\) with \(C=1/\sqrt{2\pi}\) and \(g=-(x-\mu)^2/(2\sigma^2)\). Chain rule: \(\partial p/\partial\mu=pe^{g}\)'s version \(=p\,\partial g/\partial\mu\), and \(\partial g/\partial\mu=(x-\mu)/\sigma^2\). Hence \(p(x-\mu)/\sigma^2\).
4. Why do the two answers look like "\(p\) times something"?
Because of the chain rule: differentiating an exponential reproduces the exponential, so the height \(p\) itself never disappears from the result. The "something" is the sensitivity per dial.
Page built for Task 2.2a of the 5CTA0 project. Derivatives are those of the standard Gaussian, plotted with \(\mu\) sliding over \(\pm0.9\) and \(\sigma\in[1.0,1.6]\). Rendered with manim.