In exercise [1, p. 34 exercise 2.2] we are asked to prove that the rows and columns of a unitary matrix are orthonormal as per [1, p. 21,(2.21)].
Solution: A complex square matrix is unitary if \mathbf{A}^{-1}=\mathbf{A}^H. Let \mathbf{a}_i be the i^{th} column vector of \mathbf{A} then
\mathbf{A}=\left[ \begin{array}{*{20}c} \mathbf{a}_{1}  & \mathbf{a}_{2}  &  \cdots  & \mathbf{a}_{N}   \\ \end{array} \right] (1)

Computing the matrix product of the hermitian transpose and the matrix results in:
\mathbf{A}^H \mathbf{A}=\left[ \begin{array}{*{20}c} \mathbf{a}_1^H   \\
\mathbf{a}_2^H  \\ \vdots   \\ \mathbf{a}_N^H   \\ \end{array} \right]  \left[ \begin{array}{*{20}c} \mathbf{a}_1  & \mathbf{a}_2  &  \cdots  & \mathbf{a}_N   \\ \end{array} \right] (2)
=\left[  \begin{array}{*{20}c} \mathbf{a}_1^H \mathbf{a}_1 & \mathbf{a}_1^H \mathbf{a}_2  &  \cdots  & \mathbf{a}_1^H \mathbf{a}_N  \\  \mathbf{a}_2^H \mathbf{a}_1 & \mathbf{a}_2^H \mathbf{a}_2  &  \cdots  & \mathbf{a}_2^H \mathbf{a}_N   \\   \vdots  &  \vdots  &  \vdots  &  \vdots   \\
\mathbf{a}_N^H \mathbf{a}_1  & \mathbf{a}_N^H \mathbf{a}_2  &  \cdots  & \mathbf{a}_N^H \mathbf{a}_N   \\ \end{array} \right] (3)
={\mathbf I} (4)

Thus  \mathbf{a}_i^{H} \mathbf{a}_j= \delta_{ij} . QED.

[1] Steven M. Kay: “Modern Spectral Estimation – Theory and Applications”, Prentice Hall, ISBN: 0-13-598582-X.