In [1, p. 35 exercise 2.15] we are asked to verify the formulas given for the gradient of a quadratic and linear form [1, p. 31 (2.61)]. The corresponding formulas are $\frac{\partial}{\partial \mathbf{x}}(\mathbf{x}^T\mathbf{A}^{\prime}\mathbf{x})=2\mathbf{A}^{\prime}\mathbf{x}$ (1)

and $\frac{\partial}{\partial \mathbf{x}}(\mathbf{b}^{\prime T}\mathbf{x})=\mathbf{b}^{\prime}$ (2)

where $\mathbf{A}^{\prime}$ is a symmetric $n \times n$ matrix with elements $a_{ij}$ and $\mathbf{b}^{\prime}$ is a real $n \times 1$ vector with elements $b_{i}$ and $\frac{\partial}{\partial \mathbf{x}}$ denotes the gradient of a real function in respect to $\mathbf{x}$.
Solution: Let $f(\mathbf{x})$ be the scalar function of the vector variable $\mathbf{x}$ defined by: $f(\mathbf{x})$ $=$ $\mathbf{x}^T\mathbf{A}^{\prime}\mathbf{x}$ $=$ $\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}x_{i}a_{ij}x_j$ $=$ $\sum\limits_{i=1, i\neq j}^{n}\sum\limits_{j=1}^{n}x_{i}a_{ij}x_j +\sum\limits_{i=1}^{n}x_{i}^{2}a_{ii}$

In order to identify the factors to the $l^{th}$ vector component of $\mathbf{x}$ , $x_{l}$ and its powers, we can rewrite the previous equation as: $f(\mathbf{x}) = x_{l}^{2}a_{ll} + \sum\limits_{j=1, j\neq l}^{n}x_{l}a_{lj}x_j +\sum\limits_{i=1, i\neq l}^{n}x_{i}a_{il}x_l$ $+ \sum\limits_{i=1, i\neq l}^{n}\sum\limits_{j=1, j \neq l}^{n}x_{i}a_{ij}x_j$ (3)

The first derivative of (3) in respect to $x_{l}$ is given by: $\frac{\partial f(\mathbf{x})}{\partial x_{l}}$ $=$ $2x_{l}a_{ll} +\sum\limits_{j=1, j\neq l}^{n}a_{lj}x_j +\sum\limits_{i=1, i\neq l}^{n}x_{i}a_{il}$ (4)

and replacing the dummy indexes $i$ and $j$ of the two sums by $k$ we obtain by considering the fact that the matrix is symmetric $a_{kl}=a_{lk}$ the following relation: $\frac{\partial f(\mathbf{x})}{\partial x_{l}}$ $=$ $2x_{l}a_{ll} +\sum\limits_{k=1, k \neq l}^{n}a_{lk}x_k +\sum\limits_{k=1, k\neq l}^{n}x_{k}a_{kl}$ $=$ $2x_{l}a_{ll} +\sum\limits_{k=1, k \neq l}^{n}(a_{lk}+a_{kl})x_{k}$ $=$ $2x_{l}a_{ll} +\sum\limits_{k=1, k \neq l}^{n}2a_{lk}x_{k}$ $=$ $2\sum\limits_{k=1}^{n}a_{lk}x_{k}$

Let $\mathbf{e}_{l}$ be the unit vector along the component $x_{l}$ then the gradient of the matrix $f(\mathbf{x})$ can be written as: $\frac{\partial f(\mathbf{x})}{\partial \mathbf{x}}$ $=$ $2\sum\limits_{l=1}^{n} \sum\limits_{k=1}^{n}a_{lk}x_{k}\mathbf{e}_{l}$ $=$ $2 \mathbf{A}^{\prime}\mathbf{x}$ (5)

which proves the first of the formulas [1, p. 31 (2.61)]. In order to prove the second formula we proceed in a similar way by defining the scalar function $g(\mathbf{x})$ $=$ $\mathbf{b}^{\prime T}\mathbf{x}$ $=$ $\sum\limits_{i=1}^{n}b_{i}x_{i}$ (6)

Thus the derivative in respect to $x_{l}$ is simply: $\frac{\partial g(\mathbf{x})}{\partial x_{l}}$ $=$ $b_{l}$

and the gradient is given by $\frac{\partial g(\mathbf{x})}{\partial \mathbf{x}}$ $=$ $\sum\limits_{i=1}^{n} b_{i}\mathbf{e}_{i}$ $=$ $\mathbf{b}^{\prime}$

The last equation proves the second relation of the formulas [1, p. 31 (2.61)]. QED.

 Steven M. Kay: “Modern Spectral Estimation – Theory and Applications”, Prentice Hall, ISBN: 0-13-598582-X.