Numerical Analysis by Dr. Anita Pal Assistant Professor Department of Mathematics National

17 Pages·2016·0.89 MB·English

Numerical Analysis
by
Dr. Anita Pal
Assistant Professor
Department of Mathematics
National Institute of Technology Durgapur
Durgapur-713209
email: [email protected]
1
.
Chapter 5
Solution of System of Linear Equations
Module No. 2
Iteration Methods to Solve System of Linear Equations
2.1. Some terminologies.........................................................
Inthepreviousmodule,itismentionedthatapartfromdirectmethodthereisanother
method called iteration method to solve a system of linear equations. Generally, for
large system the iteration methods are used. Theoretically, the direct method gives
the accurate solution, but practically it is not possible due to finite representation of
number in computer. Again, in direct method there is no scope to update the solution
obtained by applying the method. In iteration method, it is possibleto obtain the roots
of a system with a specified accuracy as the limit of the sequence of some vectors. The
process to generate such sequence of roots is known as the iterative process.
A number of iteration methods is available to solve a system of linear equations, viz.
Jacobi’s iteration method, Gauss-Seidal’s iteration method, relaxation method, etc.
The efficiency of any iteration method depends on the choice of the initial vector and
the rate of convergence of the process.
Also, the iteration methods are not applicable for all types of system of equations.
Let us defined some useful terms.
2.1 Some terminologies
Let us consider a system of n linear equations containing n variables:
a x +a x +···+a x = b
11 1 12 2 1n n 1
a x +a x +···+a x = b (2.1)
21 1 22 2 2n n 2
··························· ··· ···
a x +a x +···+a x = b .
n1 1 n2 2 nn n n
A set of values of the variables x ,x ,...,x which satisfies the above equations is
1 2 n
called the roots of the system. But, such values cannot be determined in one execution.
The values obtained in a particular iteration are to be updated in the next iteration.
This process continues until we get the desired accuracy.
(k)
Let x , i = 1,2,...,n be the kth (k = 1,2,...) iterated value of the variable x and
i i
x(k) = (x(k),x(k),...,x(k))t be the solution vector obtained at the kth iteration.
1 2 n
The sequence of numbers {x(k)}, k = 1,2,... is said to converge to a vector x =
(x ,x ,...,x )t if for each i(= 1,2,...,n)
1 2 n
(k)
x −→ x as k −→ ∞. (2.2)
i i
1
...................Iteration Methods to Solve System of Linear Equations
Let ξ = (ξ ,ξ ,...,ξ )t be the exact solution of the given system of linear equations.
1 2 n
(k)
Then the error ε of the ith variable x occurred at the kth iteration is given by
i i
(k) (k)
ε = ξ −x . (2.3)
i i i
The errors occurred in all variables at the kth iteration is denoted by the vector ε(k),
i.e.
ε(k) = (ε(k),ε(k),...,ε(k))t. (2.4)
1 2 n
The difference of the errors e(k) at two consecutive iterations is given by
e(k) = x(k+1)−x(k) = ε(k) −ε(k+1), (2.5)
(k) (k+1) (k)
where e = x −x .
i i i
Therate of convergence of a system of equations dependson the method. Therate of
convergence for a method applied on a system of linear equations is defined as follows.
An iteration method is said to be of order p ≥ 1 if there exists a positive constant A
such that for all k
kε(k+1)k ≤ Akε(k)kp. (2.6)
Now, we discussed a simple iteration method to solve a system of linear equations.
2.2 Jacobi’s iteration method
Let us consider the following system of linear equations containing n variables and n
equations.
a x +a x +···+a x = b
11 1 12 2 1n n 1
a x +a x +···+a x = b (2.7)
21 1 22 2 2n n 2
··························· ··· ···
a x +a x +···+a x = b .
n1 1 n2 2 nn n n
For this iteration method, we assume that the coefficients matrix is diagonally dom-
inant, i.e. either
2
2.1. Some terminologies.........................................................
n
|a | < |a | for all i = 1,2,...,n
ij ii
Xj=1
or
n
|a |< |a | for all j = 1,2,...,n.
ij jj
Xi=1
Now, we rewrite the equations (2.7) as
1
x = (b −a x −a x −···−a x )
1 1 12 2 13 3 1n n
a
11
1
x = (b −a x −a x −···−a x ) (2.8)
2 2 21 1 23 3 2n n
a
22
··· ··· ·······································
1
xn = (bn−an1x1−an2x2−···−ann−1xn−1).
a
nn
(0) (0) (0)
Let x ,x ,...,x be the initial guess/solution of the system. That is, x =
1 2 n 1
(0) (0) (0)
x ,x = x ,...,x = x be the initial solution. In some cases, the initial solution
1 2 2 n n
may be taken as the zero vector, i.e. (0,0,...,0).
This initial solution is substitute to the right hand side of the system of equations
(2.8). This gives the first approximate roots of the given system of equations. Let
x(1),x(1),...,x(1) be the first approximate roots, and these are given by
1 2 n
1
x(1) = (b −a x(0)−a x(0)−···−a x(0))
1 a 1 12 2 13 3 1n n
11
1
x(1) = (b −a x(0)−a x(0)−···−a x(0)) (2.9)
2 a 2 21 1 23 3 2n n
22
··· ··· ·······································
1
x(n1) = a (bn −an1x(10)−an2x(20)−···−ann−1x(n0−)1).
nn
(1) (1) (1)
Again, substitute x ,x ,...,x to the right hand side of (2.8) and obtained the
1 2 n
(2) (2) (2)
second approximate roots x ,x ,...,x .
1 2 n
(k) (k) (k)
In general, if x ,x ,...,x be the kth approximate roots, then the (k +1) ap-
1 2 n
3
...................Iteration Methods to Solve System of Linear Equations
proximate roots are given by
1
x(k+1) = (b −a x(k)−a x(k)−···−a x(k))
1 a 1 12 2 13 3 1n n
11
1
x(k+1) = (b −a x(k)−a x(k)−···−a x(k)) (2.10)
2 a 2 21 1 23 3 2n n
22
··· ··· ·······································
1
x(nk+1) = a (bn−an1x(1k)−an2x(2k)−···−ann−1x(nk−)1).
nn
k = 0,1,2,....
This process is repeated until all the roots converge to the required number of sig-
nificant figures. This process if iteration is called Jacobi’s iteration or simply the
method of iteration.
But, there is a limitation of this method. Jacobi’s iteration is not applicable for all
systems of linear equations. The sufficient condition for convergence of this method is
discussed below.
2.2.1 Convergence of Gauss-Jacobi’s iteration
The(k+1)thiteratedvalueofthevariablex obtainedbytheGauss-Jacobi’siteration
i
method is given by
n
1
(k+1) (k)
x = b − a x ,i = 1,2,...,n. (2.11)
i a (cid:18) i ij j (cid:19)
ii Xj=1
j6=i
If ξi be the exact solution for variable xi, then
n
1
ξ = b − a ξ . (2.12)
i i ij j
a (cid:18) (cid:19)
ii Xj=1
j6=i
Thus, the difference between exact value and the (k+1)th iterated value of the ith
variable is
n
1
(k+1) (k)
ξ −x = − a ξ −x
i i a ij j j
ii Xj=1 (cid:16) (cid:17)
j6=i
n
1
(k+1) (k)
or ε = − a ε .
i a ij j
ii Xj=1
j6=i
4
2.1. Some terminologies.........................................................
That is,
n n
1 1
kε(k+1)k ≤ |a | kε(k)k ≤ |a | kε(k)k.
i |a | ij j |a | ij
ii Xj=1 ii Xj=1
j6=i j6=i
n
1
Let M = max |a | .
ij
i n|aii|Xj=1 o
j6=i
Then the above equation reduces to
kε(k+1)k ≤Mkε(k)k. (2.13)
Here, the term kε(k)k is linear, so the rate of convergence of Gauss-Jacobi’s method
is linear.
Again, kε(k+1)k ≤ Mkε(k)k ≤ M2kε(k−1)k ≤ ··· ≤ Mk+1 kε(0)k.
That is,
kε(k)k ≤Mkkε(0)k. (2.14)
If M < 1 then Mk → 0 as k → ∞ and consequently kε(k)k → 0 as k → ∞. Thus,
the iteration process converges.
Hence, the sufficient condition for convergent of Gauss-Jacobi’s iteration method is
n
M < 1, i.e. |a | < |a | for all i.
ij ii
Xj=1
j6=i
Inotherwords,thesufficientconditionforconvergence oftheGauss-Jacobi’siteration
method is that the coefficient matrix be diagonally dominant.
The equation (2.13) can also be written as
kε(k+1)k ≤ Mkε(k)k= M ke(k)+ε(k+1)k [by (2.5)]
≤ Mke(k)k+Mkε(k+1)k
M
or, kε(k+1)k ≤ ke(k)k. (2.15)
M −1
From this equation one can estimate the absolute error at the (k+1)th iteration in
terms of the errors at the kth iteration.
5
...................Iteration Methods to Solve System of Linear Equations
Example 2.1 Consider the following system of linear equations
14x−3y+5z = 4
3x+10y−2z = 12
5x+2y+20z = 16.
Solve these equations by Gauss-Jacobi’s method correct up to four decimal places. Also
calculate the upper bound of absolute errors.
Solution. Since
|−3|+|5| < |14|, |3|+|−2| < |10|, |5|+|2| < |20|,
therefore, the system is diagonally dominant and hence there is a convergence iteration
scheme.
Now, such Gauss-Jacobi’s iteration scheme is
1
x(k+1) = 4+3y(k)−5z(k)
14
(cid:16) (cid:17)
1
y(k+1) = 12−3x(k)+2z(k)
10
(cid:16) (cid:17)
1
z(k+1) = 16−5x(k)−2y(k) .
20
(cid:16) (cid:17)
Let the initial solution be (0, 0, 0). The successive iterations are depicted in the
following table.
k x y z
0 0 0 0
1 0.28571 1.20000 0.80000
2 0.25714 1.27429 0.60857
3 0.34143 1.24457 0.60829
4 0.33516 1.21923 0.59019
5 0.33620 1.21749 0.59429
6 0.33436 1.21800 0.59420
7 0.33450 1.21853 0.59461
8 0.33447 1.21857 0.59452
9 0.33451 1.21856 0.59453
10 0.33450 1.21855 0.59452
11 0.33451 1.21855 0.59452
6
2.1. Some terminologies.........................................................
Thus, the solution correct up to four decimal places is
x = 0.3345, y = 1.2186, z = 0.5945.
Here
n
1 8 5 7 4
M = max |a | = max , , = .
ij
i (cid:26)aii Xj=1 (cid:27) (cid:26)14 10 20(cid:27) 7
j6=i
e(10) = (1×10−5,0,0). Therefore, the upper bound of absolute error is
M
kε(10)k ≤ ke(10)k = 1.3333×10−5.
1−M
2.3 Gauss-Seidal’s iteration method
This method is obtained by a simple modification on Jacobi’s iteration method, and
most of the times this new method gives faster convergence.
Let us consider the same system of linear equations with n variables and n equations.
a x +a x +···+a x = b
11 1 12 2 1n n 1
a x +a x +···+a x = b (2.16)
21 1 22 2 2n n 2
··························· ··· ···
a x +a x +···+a x = b .
n1 1 n2 2 nn n n
Also, in this method we assume that the coefficient matrix is diagonally dominant.
If the coefficient matrix is not diagonally dominant, then the above system of equations
are re-arranged in such a way that the above condition holds. If it is not possible, then
the method may or may not give the solution.
Like Jacobi’s method the equations (2.16) are rewritten in the following form:
1
x = (b −a x −a x −···−a x )
1 1 12 2 13 3 1n n
a
11
1
x = (b −a x −a x −···−a x ) (2.17)
2 2 21 1 23 3 2n n
a
22
··· ··· ·······································
1
xn = (bn−an1x1−an2x2−···−ann−1xn−1).
a
nn
Let x(0),x(0),...,x(0) be the initial solution for the variables x ,x ,...,x respec-
2 3 n 2 3 n
tively. Substitute these values to the first equation of the above system and obtain
7
...................Iteration Methods to Solve System of Linear Equations
(1) (1)
the first approximate value of x and it is denoted by x . Next, substitute x for
1 1 1
x and x(0),x(0),...,x(0) for x ,x ,...,x respectively to the second equation of (2.17)
1 3 4 n 3 4 n
(1)
to get the first approximate value of x and it is denoted as x . Then substitute
2 2
(1) (1) (1) (0) (0)
x1 ,x2 ,...,xi−1,xi+1,...,xn for x1,x2,...,xi−1,xi+1,...,xn respectively to the ith
(1)
equation of (2.17) and obtain the first approximation of x and let it be x . Continue
i i
this process for all equations.
In similar way, one can calculate the second approximate values of all variables.
(k)
If x , i = 1,2,...,n be the kth approximate value of x , then the (k+1)th approx-
i i
imate value of x ,x ,...,x are given by
1 2 n
1
x(k+1) = (b −a x(k)−a x(k)−···−a x(k))
1 a 1 12 2 13 3 1n n
11
1
x(k+1) = (b −a x(k+1)−a x(k)−···−a x(k)) (2.18)
2 a 2 21 1 23 3 2n n
22
·············································
1
(k+1) (k+1) (k+1) (k) (k)
xi = a (bi −ai1x1 −···−aii−1xi−1 −aii+1xi+1−···−ann−1xn−1)
ii
·············································
1
x(nk+1) = a (bn−an1x(1k+1)−an2x(2k+1)−···−ann−1x(nk−+11)).
nn
k = 0,1,2,....
In compact form the above equations can be written as,
i−1 n
1
(k+1) (k+1) (k)
x = b − a x − a x , i = 1,2,...,n and k = 0,1,2,....
i a i ij j ij j
ii(cid:16) Xj=1 j=Xi+1 (cid:17)
(k+1) (k)
This iteration process is repeated until |x −x | < ε for all i = 1,2,...,n, for
i i
any pre-assigned number ε > 0. This number is called the error tolerance. This method
is known as Gauss-Seidal’s iteration method.
Note that the very recent values of the variables are used to calculate the value of
the next variable.
8