In [1, p. 35 exercise 2.15] we are asked to verify the formulas given for the gradient of a quadratic and linear form [1, p. 31 (2.61)]. The corresponding formulas are
\frac{\partial}{\partial \mathbf{x}}(\mathbf{x}^T\mathbf{A}^{\prime}\mathbf{x})=2\mathbf{A}^{\prime}\mathbf{x} (1)

and
\frac{\partial}{\partial \mathbf{x}}(\mathbf{b}^{\prime T}\mathbf{x})=\mathbf{b}^{\prime} (2)

where \mathbf{A}^{\prime} is a symmetric n \times n matrix with elements a_{ij} and \mathbf{b}^{\prime} is a real n \times 1 vector with elements b_{i} and \frac{\partial}{\partial \mathbf{x}} denotes the gradient of a real function in respect to \mathbf{x}.
Solution: Let f(\mathbf{x}) be the scalar function of the vector variable \mathbf{x} defined by:
f(\mathbf{x})=\mathbf{x}^T\mathbf{A}^{\prime}\mathbf{x}
=\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}x_{i}a_{ij}x_j
=\sum\limits_{i=1, i\neq j}^{n}\sum\limits_{j=1}^{n}x_{i}a_{ij}x_j +\sum\limits_{i=1}^{n}x_{i}^{2}a_{ii}

In order to identify the factors to the l^{th} vector component of \mathbf{x} , x_{l} and its powers, we can rewrite the previous equation as:
f(\mathbf{x}) = x_{l}^{2}a_{ll} + \sum\limits_{j=1, j\neq l}^{n}x_{l}a_{lj}x_j +\sum\limits_{i=1, i\neq l}^{n}x_{i}a_{il}x_l
+ \sum\limits_{i=1, i\neq l}^{n}\sum\limits_{j=1, j \neq l}^{n}x_{i}a_{ij}x_j (3)

The first derivative of (3) in respect to x_{l} is given by:
\frac{\partial f(\mathbf{x})}{\partial x_{l}} = 2x_{l}a_{ll} +\sum\limits_{j=1, j\neq l}^{n}a_{lj}x_j +\sum\limits_{i=1, i\neq l}^{n}x_{i}a_{il} (4)

and replacing the dummy indexes i and j of the two sums by k we obtain by considering the fact that the matrix is symmetric a_{kl}=a_{lk} the following relation:
\frac{\partial f(\mathbf{x})}{\partial x_{l}} = 2x_{l}a_{ll} +\sum\limits_{k=1, k \neq l}^{n}a_{lk}x_k +\sum\limits_{k=1, k\neq l}^{n}x_{k}a_{kl}
= 2x_{l}a_{ll} +\sum\limits_{k=1, k \neq l}^{n}(a_{lk}+a_{kl})x_{k}
= 2x_{l}a_{ll} +\sum\limits_{k=1, k \neq l}^{n}2a_{lk}x_{k}
= 2\sum\limits_{k=1}^{n}a_{lk}x_{k}

Let \mathbf{e}_{l} be the unit vector along the component x_{l} then the gradient of the matrix f(\mathbf{x}) can be written as:
\frac{\partial f(\mathbf{x})}{\partial \mathbf{x}}=2\sum\limits_{l=1}^{n} \sum\limits_{k=1}^{n}a_{lk}x_{k}\mathbf{e}_{l}
=2 \mathbf{A}^{\prime}\mathbf{x} (5)

which proves the first of the formulas [1, p. 31 (2.61)]. In order to prove the second formula we proceed in a similar way by defining the scalar function
g(\mathbf{x})=\mathbf{b}^{\prime T}\mathbf{x}
=\sum\limits_{i=1}^{n}b_{i}x_{i} (6)

Thus the derivative in respect to x_{l} is simply:
\frac{\partial g(\mathbf{x})}{\partial x_{l}}=b_{l}

and the gradient is given by
\frac{\partial g(\mathbf{x})}{\partial \mathbf{x}}=\sum\limits_{i=1}^{n} b_{i}\mathbf{e}_{i}
=\mathbf{b}^{\prime}

The last equation proves the second relation of the formulas [1, p. 31 (2.61)]. QED.

[1] Steven M. Kay: “Modern Spectral Estimation – Theory and Applications”, Prentice Hall, ISBN: 0-13-598582-X.