In [1, p. 61 exercise 3.6] we are asked to assume that the variance is to be estimated as well as the mean for the conditions of [1, p. 60 exercise 3.4] (see also [2, solution of exercise 3.4]) . We are asked to prove for the vector parameter $\mathbf{\theta}=\left[\mu_x \; \sigma^2_x\right]^T$, that the Fisher information matrix is
 $\mathbf{I}_{\theta}=\left[\begin{array}{cc} \frac{N}{\sigma^2_x} & 0 \\ 0 & \frac{N}{2\sigma^4_x} \end{array}\right]$

Furthermore we are asked to find the CR bound and to determine if the sample mean $\hat{\mu}_x$ is efficient. If additionaly the variance is to be estimated as
 $\hat{\sigma}^2_x=\frac{1}{N-1}\sum\limits_{n=0}^{N-1}(x[n]-\hat{\mu}_x)^2$

then we are asked to determine if this estimator is unbiased and efficient. Hint: We are instructed to use the result that
 $\frac{(N-1)\hat{\sigma}^2_x}{\sigma^2_x} \sim \chi^2_{N-1}$

Solution: We have already obtained the joint pdf $f(\mathbf{x})$ of the $N$ independent samples with normal distribution $N(\mu_x,\sigma^2_x)$ and the natural logarithm of the joint pdf is given by from [3, relation (3) ]:
 $\ln{f(\mathbf{x;\theta})}$ $=$ $-\ln(\sqrt{2\pi}\sigma_x)N-\frac{1}{2}\sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma_x}\right)^2$ $=$ $-\frac{N}{2}\ln(2\pi \sigma^{2}_x)-\frac{1}{2}\sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma_x}\right)^2$

From this relation we can find the gradient in respect to the vector parameter:
 $\nabla_{\mathbf{\theta}} \ln(f(\mathbf{x;\theta}))$ $=$ $\left[ \begin{array}{cc} \frac{\partial \ln(f(\mathbf{x};\theta))}{\partial \mu_{x}} & \frac{\partial \ln(f(\mathbf{x};\theta))}{\partial \sigma^{2}_{x}} \end{array} \right]^{T}$ $=$ $\left[ \begin{array}{cc} \sum_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x}\right) & -\frac{N}{2\sigma_{x}^{2}}+\frac{1}{2} \sum_{i=0}^{N-1}\frac{\left(x[i]-\mu_x\right)^2}{\sigma^{4}_x}\end{array} \right]^{T}$

Thus Fisher’s information matrix is given by [1, p. 47, (3.22) ]:
 $\mathbf{I}_{\theta}$ $=$ $E\left\{ \nabla_{\mathbf{\theta}} \ln(f(\mathbf{x;\theta})) \nabla_{\mathbf{\theta}} \ln(f(\mathbf{x;\theta}))^{T}\right\}$

Considering that the samples $x[i] \;, i=0,..,N$ are independent we obtain the individual elements of the matrix $I_{ij}$ are given by:
 $I_{11}$ $=$ $E\left\{ \left(\sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x}\right) \right)^{2} \right\}$ $=$ $E\left\{ \sum\limits_{i=0}^{N-1}\frac{\left(x[i]-\mu_x\right)^{2}}{\sigma^{4}_x} +\sum\limits_{i=0}^{N-1}\sum\limits_{j=0,j\neq i}^{N-1}\frac{\left(x[i]-\mu_x\right)\left(x[j]-\mu_x\right)}{\sigma^{4}_x} \right\}$ $=$ $\sum\limits_{i=0}^{N-1}\frac{E\left\{ \left(x[i]-\mu_x\right)^{2}\right\}}{\sigma^{4}_x}$  $+\sum\limits_{i=0}^{N-1}\sum\limits_{j=0,j\neq i}^{N-1}\frac{E\left\{\left(x[i]-\mu_x\right)\right\}E\left\{\left(x[j]-\mu_x\right)\right\}}{\sigma^{4}_x}$ $=$ $\frac{N\sigma^{2}_x}{\sigma^{4}_x} +0$ $=$ $\frac{N}{\sigma^{2}_x}$ (1) $I_{12}=I_{21}$ $=$ $E\left\{ \left(\sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x}\right) \right) \left(-\frac{N}{2\sigma_{x}^{2}}+\frac{1}{2} \sum\limits_{i=0}^{N-1}\frac{\left(x[i]-\mu_x\right)^2}{\sigma^{4}_x}\right) \right\}$ $=$ $E\left\{ -\frac{N}{2 \sigma_{x}^{2} } \sum\limits_{i=0}^{N-1} \left(\frac{x[i]-\mu_x}{\sigma^{2}_x}\right)\right\}$  $+E\left\{ \left(\sum\limits_{i=0}^{N-1}\frac{x[i]-\mu_x}{\sigma^{2}_x} \right) \left(\frac{1}{2}\sum\limits_{i=0}^{N-1}\frac{\left( x[i]-\mu_x \right)^2}{\sigma^{4}_x}\right) \right\}$ $=$ $-\frac{N}{2 \sigma_{x}^{2} } \sum\limits_{i=0}^{N-1} \left(\frac{E\left\{x[i]\right\}-\mu_x}{\sigma^{2}_x}\right) +\frac{1}{2}E\left\{ \sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}\right\}$  $+ \frac{1}{2}E\left\{\sum\limits_{i=0}^{N-1} \sum\limits_{j=0,j\neq i}^{N-1}\frac{\left(x[j]-\mu_x\right)\left( x[i]-\mu_x \right)^2}{\sigma^{6}_x} \right\}$ $=$ $0+\frac{1}{2} E\left\{ \sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}\right\}$  $+\frac{1}{2}\sum\limits_{i=0}^{N-1} \sum\limits_{j=0,j\neq i}^{N-1}\frac{ E\left\{\left(x[j]-\mu_x\right)\left( x[i]-\mu_x \right)^2\right\}}{\sigma^{6}_x}$ $=$ $\frac{1}{2}E\left\{ \sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}\right\}$  $+\frac{1}{2}\sum\limits_{i=0}^{N-1} \sum\limits_{j=0,j\neq i}^{N-1}\frac{ E\left\{\left(x[j]-\mu_x\right)\right\}E\left\{\left( x[i]-\mu_x \right)^2\right\}}{\sigma^{6}_x}$ $=$ $\frac{1}{2} E\left\{ \sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}\right\} +0$ $=$ $\frac{1}{2} \sum\limits_{i=0}^{N-1}E\left\{\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3} \right\}$ (2)

We note that $h(x[i])=\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}$ is an odd function about $\mu_{x}$ (that is $h(\mu_{x}+x)=-h(\mu_{x}-x)$) while the gaussian distribution is an even function about $\mu_{x}$. Thus the mean of this function -the integral which is symmetric about $\mu_{x}$ and extends from $-\infty$ to $\infty$ will be equal to zero. This fact can be shown by the following approach:
 $E\left\{\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3} \right\}$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{+\infty}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{x[i]-\mu_{x}}{\sigma_{x}}\right)^{2}} dx$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{\mu_{x}}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{x[i]-\mu_{x}}{\sigma_{x}}\right)^{2}} dx$  $+\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{\mu_{x}}^{+\infty}\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{x[i]-\mu_{x}}{\sigma_{x}}\right)^{2}} dx$

Let $u=\left(x[i]-\mu_{x}\right)$ in the last formula
 $E\left\{\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3} \right\}$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{0}\left(\frac{u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$  $+\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{0}^{+\infty}\left(\frac{u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{0}\left(\frac{u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$  $+\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{0}^{+\infty}-\left(\frac{-u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{0}\left(\frac{u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$  $+\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{0}^{+\infty}\left(\frac{-u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{-u}{\sigma_{x}}\right)^{2}} d(-u)$

If we set $v=-u$ in the second integral, we obtain the following formulas:
 $E\left\{\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3} \right\}$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{0}\left(\frac{u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$  $+\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{0}^{-\infty}\left(\frac{v}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{v}{\sigma_{x}}\right)^{2}} d(v)$ $=$ $\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{0}\left(\frac{u}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{u}{\sigma_{x}}\right)^{2}} du$  $-\frac{1}{\sqrt{2\pi}\sigma_{x}}\int_{-\infty}^{0}\left(\frac{v}{\sigma^{2}_x} \right)^{3}e^{-\left(\frac{v}{\sigma_{x}}\right)^{2}} d(v)$ $E\left\{\left(\frac{x[i]-\mu_x}{\sigma^{2}_x} \right)^{3} \right\}$ $=$ $0$ (3)

Using [2] in conjunction with [3] we obtain
 $I_{12}=I_{21}=0 .$ (4)

Finally it remains to obtain the value for $I_{22}$:
 $I_{22}$ $=$ $E\left\{\left( -\frac{N}{2\sigma_{x}^{2}}+\frac{1}{2} \sum\limits_{i=0}^{N-1}\frac{\left(x[i]-\mu_x\right)^2}{\sigma^{4}_x}\right)^{2}\right\}$ $=$ $\frac{1}{4\sigma_{x}^{4}} E\left\{\left( \sum\limits_{i=0}^{N-1}\left(\frac{x[i]-\mu_x}{\sigma_x}\right)^{2} -N\right)^{2}\right\}$ (5)

We note that $v_{i}=\frac{x[i]-\mu_x}{\sigma_x}$ in (5) is a normalized random gaussian variable and that the squared sum of such variables $\chi^{2}=\sum_{i=0}^{N-1}v_{i}^{2}$ has a chi-square distribution [4, p. 682] with N degrees of freedom with mean $E\left\{\chi^{2}\right\}=N$ and variance equal to $E\left\{\left(\chi^{2}-N\right)^{2}\right\}=2N$. Thus
 $I_{22}=\frac{N}{2\sigma_{x}^{4}}$ (6)

From (1), (4) and (6) we obtain finally that Fisher’s information matrix is equal to:
 $\mathbf{I}_{\theta}=\left[ \begin{array}{cc} \frac{N}{\sigma_{x}^{2}} & 0 \\ 0 &\frac{N}{2\sigma_{x}^{4}} \end{array} \right].$ (7)

We have shown in [2, solution of exercise 3.4] that the mean and the variance [2, solution of exercise 3.4, relation (5) ] of the estimator $\hat{\mu}_{x}$ are given by:
 $E\left\{\hat{\mu}_{x}\right\}$ $=$ $\mu_{x}$ $Var\left\{\hat{\mu}_{x}\right\} =E\left\{\left(\hat{\mu}_{x}- \mu_{x} \right)^{2}\right\}$ $=$ $\frac{\sigma^{2}_{x}}{N}$

The mean and the variance of the estimator $\hat{\sigma}^{2}_{x}$ are given by (always considering the independence of the random variables $x[n], x[k]$ for $k\neq n$):
 $E\left\{\hat{\sigma}^{2}_{x}\right\}$ $=$ $\frac{1}{N-1}\sum\limits_{n=0}^{N-1}E\left\{(x[n]-\hat{\mu}_{x})^{2}\right\}$ $=$ $\frac{1}{N-1}\sum\limits_{n=0}^{N-1}E\left\{((x[n]-\mu_{x})- (\hat{\mu}_{x}-\mu_{x}))^{2}\right\}$ $=$ $\frac{1}{N-1}\sum\limits_{n=0}^{N-1}E\left\{(x[n]-\mu_{x})^{2}\right\}+\frac{1}{N-1}\sum\limits_{n=0}^{N-1} E\left\{(\hat{\mu}_{x}-\mu_{x})^{2}\right\}$  $- \frac{2}{N-1}\sum\limits_{n=0}^{N-1}E\left\{(x[n]-\mu_{x})(\hat{\mu}_{x}-\mu_{x})\right\}$ $=$ $\frac{N\sigma^{2}_{x}}{N-1} + \frac{\sigma^{2}_{x}}{N-1} - \frac{2}{N-1}\sum\limits_{n=0}^{N-1}E\left\{(x[n]-\mu_{x})(\hat{\mu}_{x}-\mu_{x})\right\}$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} - \frac{2}{N-1}\sum\limits_{n=0}^{N-1}E\left\{(x[n]-\mu_{x})(\hat{\mu}_{x}-\mu_{x})\right\}$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} - \frac{2}{N-1}\sum\limits_{n=0}^{N-1}E\left\{(x[n]-\mu_{x})(\frac{1}{N}\sum\limits_{i=0}^{N-1}x[i]-\mu_{x})\right\}$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} - \frac{2}{N-1}\sum\limits_{n=0}^{N-1}E\left\{\frac{1}{N}(x[n]-\mu_{x})(\sum\limits_{i=0}^{N-1}(x[i]-\mu_{x}))\right\}$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} -$  $\frac{2}{N-1}\sum\limits_{n=0}^{N-1}\frac{1}{N}\Bigg( E\left\{(x[n]-\mu_{x})^{2}\right\}$  $\left. +\sum\limits_{i=0,i\neq n}^{N-1}E\left\{(x[n]-\mu_{x})(x[i]-\mu_{x})\right\} \right)$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} -$  $\frac{2}{N-1}\sum\limits_{n=0}^{N-1}\frac{1}{N}\Bigg( \sigma_{x}^{2}$  $+\sum\limits_{i=0,i\neq n}^{N-1}E\left\{(x[n]-\mu_{x})\right\}E\left\{(x[i]-\mu_{x})\right\} \Bigg)$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} - \frac{2}{N-1}\sum\limits_{n=0}^{N-1}\frac{1}{N}(\sigma_{x}^{2}+0)$ $=$ $\frac{(N+1)\sigma^{2}_{x}}{N-1} - \frac{2\sigma_{x}^{2}}{N-1}$

From the previous relation we obtain finally:
 $E\left\{\hat{\sigma}^{2}_{x}\right\}$ $=$ $\sigma^{2}_{x}.$ (8)

Considering that $\frac{(N-1)\hat{\sigma}^2_x}{\sigma^2_x} \sim \chi^2_{N-1}$ we can also obtain the variance of the estimator $\hat{\sigma}_{x}^{2}$:
 $Var\left\{\hat{\sigma}^{2}_{x}\right\}$ $=$ $Var\left\{\frac{\sigma_{x}^{2}}{N-1}\chi^2_{N-1}\right\}$ $=$ $\frac{\sigma_{x}^{4}}{(N-1)^{2}} Var\left\{\chi^2_{N-1} \right \}$ $=$ $\frac{\sigma_{x}^{4}}{(N-1)^{2}} 2 (N-1)$ $=$ $\frac{2 \sigma_{x}^{4}}{N-1}.$ (9)

Because the Cramer Rao bound for the estimator $\hat{\sigma}^{2}_{x}$ is given by the inverse of the diagonal element of Fisher’s information matrix (6) $I_{22}$ we obtain using (9) :
 $Var\left\{\hat{\sigma}^{2}_{x}\right\} \geq I_{22}^{-1}$ $\frac{2 \sigma_{x}^{4}}{N-1} \geq \frac{2\sigma_{x}^{4}}{N}$ (10)

thus the estimator $\hat{\sigma}^{2}_{x}$ is unbiased because of (8) but not efficient because it doesn’t attain the Cramer – Rao bound given by (10). QED.

[1] Steven M. Kay: “Modern Spectral Estimation – Theory and Applications”, Prentice Hall, ISBN: 0-13-598582-X.
[2] Chatzichrisafis: “Solution of exercise 3.4 from Kay’s Modern Spectral Estimation - Theory and Applications”.
[3] Chatzichrisafis: “Solution of exercise 3.5 from Kay’s Modern Spectral Estimation - Theory and Applications”.
[4] Granino A. Korn and Theresa M. Korn: “Mathematical Handbook for Scientists and Engineers”, Dover, ISBN: 978-0-486-41147-7.