A.kaw Matrix Algebra Preliminary study note 10. Eigenvalues and eigenvectors

"Matrix Algebra Preliminary" (Introduction to Matrix ALGEBRA) course by Prof. A.k.kaw (University of South Florida) is designed and taught.
PDF format Learning note Download (academia.edu)
10th. Download the course handout (PDF)

Summary

• Definition
  If $[a]$ is A $n \times n$ Matrix, then $[x]\neq \vec0$ was an eigenvector of $[a]$ if $$[a][x] = \lambda[x]$$ where $\lambd A$ is a scalar and $[x]\neq0$. The scalar $\lambda$ is called the eigenvalue of $[a]$ and $[x]$ are called the eigenvector corresponding to the eigenvalue $\lambda$.
• finding eigenvalue and eigenvector
  • To find the eigenvalues of a $n \times n$ matrix $[a]$, we have $ $AX =\lambda x$$ $$\rightarrow ax-\lambda x=0$$ $$\rightarr ow (A-\lambda i) x=0$$ for the above set of equations to has A Non-zero solution $$\det (a-\lambda I) = 0$$ the above equat Ion is called the characteristic equation of $[a]$, which gives $$\lambda^n + c_1\lambda^{n-1} + \cdots + c_n=0$$ Hence th Is polynomial has $n $ roots.
  • For example, finding the eigenvalues of the matrix $$[a] = \begin{bmatrix}3& -1.5\\ -0.75& 0.75 \end{bmatrix}$$ We Has $ $A-\lambda I = \begin{bmatrix}3-\lambda & -1.5\\ -0.75& 0.75-\lambda \end{bmatrix}$$ $$\det (A-\lambda I) = (3-\LAMBDA) (0.75-\LAMBDA)-( -0.75) ( -1.5) $$ $$\rightarrow \lambda^2-3.75\lambda + 1.125 =0$$ $$\rightarrow \lambda = {3.75\pm\sqrt{{3.75}^{2}-4.5}\over2} = 3.421165,\ 0.3288354$$, the eigenvalues is 3.421165 and 0.3288354.
  • To find the eigenvectors of the above matrix $[a]$. Let $[x] = \begin{bmatrix}x_1 \ x_2\end{bmatrix}$ and we already have $\lambda_1 = 3.421165$ and $\lambda_2 = 0.3288354$. When $\lambda = 3.421165$, from the definition we have $$ (A-\lambda I) x=0$$ $$\rightarrow \left (\begin{bmatrix}3&-1.5 \ -0.75& 0.75 \end{bmatrix}-\begin{bmatrix}3.421165& 0\\ 0& 3.421165 \end{bmatrix} \right) \begin{bmatrix} x_1 \ X_2\end{bmatrix} = 0$$ $$\rightarrow \begin{bmatrix}-0.421165& -1.5\\ -0.75& -2.671165 \end{bmatrix} \begi N{bmatrix}x_1 \ X_2\end{bmatrix} = \begin{bmatrix}0 \ 0\end{bmatrix}$$ $$\rightarrow-0.421165x_1-1.5x_2 = 0 \Rightarro W x_2 = -0.2807767x_1$$ is, $$[x] = \begin{bmatrix}x_1\\ -0.2807767x_1\end{bmatrix} = x_1\begin{bmatrix}1 \-0.28077 67\end{bmatrix}$$ Hence the eigenvector corresponding to $\lambda_1 = 3.421165$ is $$\begin{bmatrix}1 \ -0.2807767\END{BM atrix}$$ Similarly, we have calculate the eigenvector corresponding to $\lambda_2 = 0.3288354$: $$ \left (\begin{bmatrix}3& -1.5\\ -0.75& 0.75 \end{bmatrix}-\begin{bmatrix}0.3288354& 0\\ 0& 0.3288354 \end{bmatrix} \right) \begin{bmatrix}x_1 \ X_2\end{bmatrix} = 0$$ $$\rightarrow \begin{bmatrix}2.671165& -1.5\\-0. 75& 0.4211646 \end{bmatrix} \begin{bmatrix}x_1 \ X_2\end{bmatrix} = \begin{bmatrix}0 \ 0\end{bmatrix}$$ $$\ RightArrow 2.671165x_1-1.5x_2 = 0 \rightarrow x_2 = 1.780776x_1$$ is, $$[x] = \begin{bmatrix}x_1\\ 1.780776x_1\end{b Matrix} = x_1\begin{bmatrix}1 \ 1.780776\end{bmatrix}$$ Hence the eigenvector corresponding to $\lambda_1 = 0.3288354$ $$\BEGIN{BMATRIX}1 \ 1.780776\end{bmatrix}$$
• Some related theorems
  • If $[a]$ is A $n \times n$ Triangular matrix-upper triangular, lower triangular and diagonal, the eigenvalues of $[a]$ ar E The diagonal entries of $[a]$.
  • $\lambda = 0$ is an eigenvalue of $[a]$ if $[a]$ is A singular (non-invertible) matrix.
  • $[a]$ and $[a]^{t}$ have the same eigenvalues.
  • Eigenvalues of a symmetric matrix is real.
  • Eigenvectors of a symmetric matrix is orthogonal, but only for distinct eigenvalues.
  • $|\det (A) |$ is the product of the absolute values of the eigenvalues of $[a]$.
• Power Method
  • One of the most common methods used for finding eigenvalues and eigenvectors is the power method. It's used to find the largest eigenvalue in a absolute sense. Note if this largest eigenvalues are repeated, this method won't work. Also This eigenvalue needs to be distinct.
  • the method is as follows:
    1. assume a guess $X ^{(0)}$ for the eigenvector in $ $AX =\lambda x$$ equation. One of the entries of $X ^{(0)}$ needs to be unity.
    2. Find $ $Y ^{(1)} = ax^{(0)}$$
    3. scale $Y ^{(1)}$ So, the chosen unity component remains unity. $ $Y ^{(1)} = \lambda^{(1)}x^{(1)}$$
    4. Repeat steps 2 and 3 with $X =x^{(1)}$ to get $X ^{(2)}$.
    5. Repeat steps 2 and 3 until the value of the eigenvalue converges.
  • For example, using the Power method, find the largest eigenvalue and the corresponding eigenvectors of $$[a] = \BEGIN{BMA trix}1.5& 0& 1\\ -0.5& 0.5& -0.5\\ -0.5& 0& 0 \end{bmatrix}$$ given with the initial guess $\begin {bmatrix}1\\ 1\\ 1 \end{bmatrix}$. From the algorithm, we have $ $AX ^{(0)} = \begin{bmatrix}1.5& 0& 1\\ -0.5& 0.5& -0.5\\ -0.5& 0& 0 \ End{bmatrix} \begin{bmatrix}1\\ 1\\ 1 \end{bmatrix} = \begin{bmatrix}2.5\\ -0.5\\ -0.5 \end{bmatrix}$$ $$\Rightarrow Y^{( 1)} = 2.5\begin{bmatrix}1\\ -0.2\\ -0.2 \end{bmatrix} $$ so $\lambda^{(1)} = 2.5$ and $X ^{(1)} = \begin{bmatrix}1\\ -0.2\\ -0.2 \end{bmatrix}$. Note that we choose the first element of $X ^{(0)}$ to be unity. Then $ $AX ^{(1)} = \begin{bmatrix}1.5& 0& 1\\ -0.5& 0.5& -0.5\\ -0.5& 0& 0 \end{bmatrix} \BEGIN{BMA trix}1\\ -0.2\\ -0.2 \end{bmatrix} = \begin{bmatrix}1.3\\ -0.5\\ -0.5 \end{bmatrix}$$ $$\rightarrow Y^{(2)} = 1.3\BEGIN{BM atrix}1\\ -0.3846\\-0.3846 \end{bmatrix}$$ so $\lambda^{(2)} = 1.3$ and $X ^{(2)} = \begin{bmatrix}1\\ -0.3846\\ -0.3846 \end{bmatrix}$. Thus far, the absolute relative approximate error on the eigenvalues is $$|\varepsilon| = \left| {\lambda^{(2)}-\lambda^{(1)}\over \lambda^{(2)}}\right| = \left| {1.3-2.5\over1.3}\right| = 0.9230769$$ Conducting further iterations, the eigenvalue after 5 iterations are 1.02459 and its absolute relative approx Imate error is 0.012441. The exact value of the eigenvalue is $\lambda = 1$ and the corresponding eigenvector is $ $X =\begin{bmatrix}1\\-0.5\\-0.5 \ end{bmatrix}$$
  • R Code This function includes 4 parameters:
    • A is the target matrix;
    • x0 is the initial guess which is a vector;
    • EPS is the tolerance of the "error which can be modified;
    • Maxit is the maximum number of iterations in the process.
    We can calculate the previous example by using this script:

    A = Matrix (c (1.5, -0.5, -0.5, 0, 0.5, 0, 1, -0.5, 0), Ncol = 3) Powereigen (A, x0 = C (1, 1, 1)) converged after Iterat  IONS$VALUE[1] 1$vector     [, 1][1,]  1.0[2,] -0.5[3,]-0.5

Selected problems

1. The eigenvalues $\lambda$ of Matrix $[a]$ is found by solving the equation ().

Solution: $$| A-\lambda i| = 0$$

2. Find the eigenvalues and eigenvectors of $$[a] = \begin{bmatrix} 10& 9\\ 2& 3\end{bmatrix}$$ using the Determin Ant method.

Solution: $$| A-\lambda i| = 0$$ $$\rightarrow \det\left (\begin{bmatrix} 10-\lambda & 9\\ 2 & 3-\lambda\end{bmatrix}\right) = 0$$ $$\Rightarr ow (10-\LAMBDA) (3-\LAMBDA)-18=0$$ $$\rightarrow \lambda^2-13\lambda +12 =0$$ $$\rightarrow \lambda_1=1,\ \lambda_2=12$ $ for $\lambda_1=1$, we have $$\begin{bmatrix} 10-\lambda & 9\\ 2 & 3-\lambda\end{bmatrix} \begin{bmatrix}x_1 \ \ x _2\end{bmatrix} =\begin{bmatrix} 0\\ 0\end{bmatrix} $$ $$\rightarrow \begin{bmatrix} 9 & 9\\ 2 & 2\end{bmatrix}\be Gin{bmatrix}x_1 \ X_2\end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$$ $$\rightarrow x_2 =-x_1$$ $$\Rightarrow X=\beg in{bmatrix}x_1\\-x_1 \end{bmatrix} = x_1\begin{bmatrix} 1\\ -1\end{bmatrix}$$ Thus the eigenvector corresponding to $\lam bda_1=1$ is $\begin{bmatrix} 1\\ -1\end{bmatrix}$. Similarly, we can find the second eigenvector corresponding to $\lambda_2=12$: $$\begin{bmatrix} 10-\lambda & 9\\ 2 &A mp 3-\lambda\end{bmatrix} \begin{bmatrix}x_1 \ X_2\end{bmatrix} =\begin{bmatRIX} 0\\ 0\end{bmatrix} $$ $$\rightarrow \begin{bmatrix}-2 & 9\\ 2 & -9\end{bmatrix}\begin{bmatrix}x_1 \ X_2\end {Bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$$ $$\rightarrow-2x_1+9x_2 = 0 \rightarrow x_2 = {2\over9}x_1$$ $$\Rightar Row x=\begin{bmatrix}x_1\\ {2\over9}x_1 \end{bmatrix} = X_1\begin{bmatrix} 1\\ {2\over9}\end{bmatrix} \Rightarrow \ Begin{bmatrix} 9\\ 2\end{bmatrix}$$ Thus the eigenvector corresponding to $\lambda_2=12$ is $\begin{bmatrix} 9\\ 2\END{BMA trix}$.

3. Find the eigenvalues and eigenvectors of $$[a] = \begin{bmatrix}4& 0& 1\\ -2& 0& 1\\ 2& 0& 1\en d{bmatrix}$$ using the Determinant method.

Solution:

First of all, we can read off for that $\lambda = 0$ are an eigenvalue of the this matrix since it is singular. Then from the definition we have $$| A-\lambda i| = 0$$ $$\rightarrow \det\left (\begin{bmatrix} 4-\lambda & 0 & 1\\-2 &-\lambda & 1\\ 2 & 0& 1-\l Ambda \end{bmatrix}\right) = 0$$ $$\rightarrow (4-\LAMBDA) \left[(-\LAMBDA) (1-\LAMBDA)-0\right]+\left[1\cdot (0+2\ Lambda) \right] =0$$ $$\rightarrow (4-\LAMBDA) (\LAMBDA^2-\LAMBDA) +2\lambda= 0$$ $$\rightarrow \lambda (-\lambda^2+5\ lambda-4+2) =0$$ $$\rightarrow \lambda (\lambda^2-5\lambda+2) =0$$ $$\rightarrow \lambda_1=0,\ \lambda_2 = 4.561553,\ \ lambda_3=0.4384472.$$ for $\lambda_1 =0$, we have $$\begin{bmatrix} 4-\lambda & 0 & 1\\-2 &-\lambda & 1\ \ 2 & 0& 1-\lambda \end{bmatrix}\begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix}0\\ 0\\ 0 \end{bmatri x}$$ $$\rightarrow \begin{bmatrix} 4 & 0 & 1\\-2 & 0 & 1\\ 2 & 0& 1 \end{bmatrix}\begin{bmatrix}x _1\\ x_2\\ X_3 \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}$$ the coefficient matrix is $$\begin{bmatrix} 4& 0& 1\\- 2& 0& 1\\ 2& 0& 1 \end{bmatrix} \Rightarrow \begin{bmatrix} 0& 0& 3\\ -2& 0& 1\\ 0& 0& Amp 2 \end{bmatrix} \rightarrow \begin{bmatrix} 0& 0& 0\\ -2& 0& 0\\ 0& 0& 2 \end{bmatrix}$$ that's, $x _1=x_3=0$ and $x _2$ is arbitrary. Hence the eigenvector corresponding to $\lambda_1=0$ is $\begin{bmatrix}0 \ 1\\ 0 \end{bmatrix}$. For $\lambda_2= 4.561553$, we have $$\begin{bmatrix} 4-\lambda & 0 & 1\\-2 &-\lambda & 1\\ 2 & 0&amp ; 1-\lambda \end{bmatrix} \begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix}0\\ 0\\ 0 \end{bmatrix}$$ $$\ RightArrow \begin{bmatrix} -0.561553 & 0 & 1\\-2 & -4.561553 & 1\\ 2 & 0& -3.561553 \end{bmatrix} \begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}$$ the coefficient matrix is $$\begi N{bmatrix}-0.561553 & 0 & 1\\-2 & -4.561553 & 1\\ 2 & 0& -3.561553 \end{bmatrix} \rightarrow \begin{bmatrix} -0.561553 & 0 & 1\\ 0 & -4.561553 & -2.561553\\ 0 & 0& 0 \end{bmatrix}$$ $$\rightarrow \begin{case s}x_1= 1.780776x_3\\ x_2 = -0.5615528x_3\end{cases}$$ where $x _3$ is arbitrary. Thus the eigenvector corresponding to $\lambda_2=4.561553$ is $\begin{bmatrix}1.780776\\ -0.5615528\\ 1 \end{bmatrix}$. For $\lambda_3= 0.4384472$, we have $$\begin{bmatrix} 4-\lambda & 0 & 1\\-2 &-\lambda & 1\\ 2 & 0&am P 1-\lambda \end{bmatrix} \begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix}0\\ 0\\ 0 \end{bmatrix}$$ $$\ RightArrow \begin{bmatrix} 3.561553 & 0 & 1\\-2 & -0.4384472 & 1\\ 2 & 0& 0.5615528 \end{bmatrix} \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}$$ the coefficient matrix is $$\be Gin{bmatrix} 3.561553 & 0 & 1\\-2 & -0.4384472 & 1\\ 2 &Amp 0& 0.5615528 \end{bmatrix} \rightarrow \begin{bmatrix} 3.561553 & 0 & 1\\ 0 & -0.4384472 & 1.561553\\ 0 & 0& 0 \end{bmatrix}$$ $$\rightarrow \begin{cases}x_1= -0.2807764x_3\\ x_2 = 3.561553x_3\end{cases}$$ where $x _3 $ is arbitrary. Thus the eigenvector corresponding to $\lambda_3= 0.4384472$ are $\begin{bmatrix}-0.2807764\\ 3.561553 \ 1 \end{bmatrix}$.

4. Find the eigenvalues of these matrices by inspection: (A) $\begin{bmatrix}2& 0& 0\\ 0& -3& 0\\ 0& 0 & 6\end{bmatrix}$; (B) $\begin{bmatrix}3& 5& 7\\ 0& -2& 1\\ 0& 0& 0\end{bmatrix}$; (C) $\begin{bmatrix}2& 0& 0\\ 3& 5& 0\\ 2& 1& 6\end{bmatrix}$.

Solution:

The eigenvalues of a triangular matrix is the diagonal entries of the matrix. Thus, (A) $\lambda_1=2,\ \lambda_2=-3,\ \lambda_3=6$. (B) $\lambda_1=3,\ \lambda_2=-2,\ \lambda_3=0$. (C) $\lambda_1=2,\ \lambda_2=5,\ \lambda_3=6$.

5. Find the largest eigenvalue in magnitude and its corresponding vectors by using the Power method $$[a] = \begin{bmatrix} 4& 0& 1\\ -2& 0& 1\\ 2& 0& 1 \end{bmatrix}$$ Start with an initial guess of the eigenvector as $\b egin{bmatrix}1\\ -0.5\\ 0.5 \end{bmatrix}$.

Solution:

We'll use the R script directly,

A = Matrix (c (4,-2, 2, 0, 0, 0, 1, 1, 1), Ncol = 3) Powereigen (A, x0 = C (1, -0.5, -0.5)) converged after  9 Iterations$va LUE[1] 4.561553$vector           [, 1][1,]  1.0000000[2,] -0.3153416[3,]  0.5615528

6. Prove If $\lambda$ is a eigenvalue of $[a]$, then ${1\over\lambda}$ was an eigenvalue of $[a]^{-1}$.

Solution:

We hope to prove that $A ^{-1}x={1\over\lambda}x$ where $AX =\lambda x$. $ $A ^{-1}x=a^{-1} (\lambda \cdot {1\OVER\LAMBDA}) x = {1\over\lambda} A^{-1}\lambda x = {1\OVER\LAMBDA} A^{-1}A x = {1\over\ lambda}x$$

7. Prove that square matrices $[a]$ and $[a]^{t}$ has the same eigenvalues.

Solution:

We hope to prove this $\det (a-\lambda i) = \det (A^{t}-\lambda i) $, and an important result is $\det (A) = \det\left (a^{t}\r ight) $ for $A $ is A square matrix. $$\det (a-\lambda i) = \det\left ((a-\lambda i) ^{t}\right) $$ $$=\det\left (a^{t}-(\lambda i) ^{t}\right) $$ $$=\det\left (A ^{t}-\lambda i\right) $$

8. Show that $|\det (A) |$ is the product of the absolute values of the eigenvalues of $[a]$.

Solution:

We hope to prove that $$|\det (A) | =\prod_{i=1}^{n}|\lambda_i|$$ where $\lambda_i$ is the eigenvalues of the matrix $A $. By the definition we have $$|\det (A-\lambda I) | = |f (\LAMBDA) | =| (\LAMBDA_1-\LAMBDA) (\LAMBDA_2-\LAMBDA) \cdots (\LAMBDA_N-\LAMBDA) | $$ Set $\lambda=0$ (since it is a variable) and we have $$|\det (a) | = |\lambda_1\lambda_2\cdots\lambda_n|= \prod_{i=1}^{n}|\lambda_i|$$

9. What is the eigenvalues of the following matrix? $$\begin{bmatrix}5& 6& 17\\ 0& -19& 23\\ 0& 0& Notoginseng \end{bmatrix}$$

Solution:

This is a upper triangular matrix, hence its eigenvalues is the diagonal elements, which is, 5, -19, and 37.

If $\begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix}$ is an eigenvector of $\begin{bmatrix}8& -4& 2\\ 4& 0& 2\\ 0& -2&-4 \end{bmatrix}$, what's the eigenvalue corresponding to the eigenvector?

Solution:

The definition we have $AX =\lambda x$, which is $$\begin{bmatrix}8& -4& 2\\ 4& 0& 2\\ 0& -2&- 4 \end{bmatrix} \begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix} =\lambda \begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix}$$ $$\Right Arrow \begin{bmatrix}-18\\ -16\\ 4 \end{bmatrix} = \lambda \begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix}$$ Hence $\LAMBDA = 4 $.

The eigenvalues of the following matrix $$\begin{bmatrix}3& 2& 9\\ 7& 5& 13\\ 6& 17& 19\END{BM Atrix}$$ is given by solving the cubic equation ().

Solution: $$| A-\lambda i| =\det\left (\begin{bmatrix}3-\lambda& 2& 9\\ 7& 5-\lambda& 13\\ 6& 17& 19-\lambda\end{bmatrix}\ right) $$ $$= (3-\LAMBDA) \begin{vmatrix}5-\lambda & 13\\ & 19-\lambda\end{vmatrix}-2\begin{vmatrix}7 & 13 \ \ 6 & 19-\lambda\end{vmatrix} + 9\begin{vmatrix}7 & 5-\lambda\\ 6 & 17\end{vmatrix}$$ $$= (3-\LAMBDA) \left ((5 -\LAMBDA) (19-\LAMBDA)-13\times17\right)-2\times \left (7 (9-\LAMBDA)-6 \times \right) + \left (7\times17-6 (5-\l AMBDA) \right) $$ $$=\lambda^3-27\lambda^2-122\lambda-313$$

The eigenvalues of a $4\times4$ matrix $[a]$ is given as 2, -3, 7, and. What is the $|\det (A) |$?

Solution:

Since for a $n \times n$ Matrix $$|\det (a) | = \prod_{i=1}^{n}|\lambda_i|$$ Hence We have $$|\det (A) | = |2\times ( -3) \times13\times7| = 546$$

If one of the eigenvalues of $[a]_{n\times n}$ is zero, it implies ().

Solution:

If an eigenvalue is zero and then its determinant must be zero. Furthermore, this means it is a singular matrix (i.e. non-invertible).

Given that matrix $$[a] = \begin{bmatrix}8& -4& 2\\ 4& 0& 2\\ 0& -2&-3 \end{bmatrix}$$ have an Eigenvalue value of 4 with the corresponding eigenvectors of $[x]=\begin{bmatrix}-4.5\\ -4\\ 1\end{bmatrix}$ s the value of $[a]^{5}[x]$?

Solution:

Firstly, we show that $A ^{m}x=\lambda^{m}x$, where $\lambda$ are an eigenvalue of $[a]$. By mathematical induction, we can read off that $n =1$ are correct.\\ then suppose that $n =m-1$ are correct, that's, $A ^{m-1 }x = \lambda^{m-1}x$ holds. For $n =m$, we have $ $A ^{m}x = Aa^{m-1}x = a\lambda^{m-1}x =\lambda^{m-1}ax = \lambda^{m-1}\lambda X =\lambda^{m}x$$ as des Ired. From this result, we have $ $A ^5x=\lambda^{5}x = 4^5\begin{bmatrix}-4.5\\ -4\\ 1\end{bmatrix} = \begin{bmatrix}-4608\\-409 6\\ 1024\end{bmatrix}$$

A.kaw Matrix Algebra Preliminary study note 10. Eigenvalues and eigenvectors