Table Of Content

Computing the Rank Profile Matrix∗ Jean-Guillaume Dumas† Clément Pernet‡ Ziad Sultan§ 5 1 August 21, 2015 0 2 g u Abstract A The row (resp. column) rank profile of a matrix describes the stair- caseshapeofitsrow(resp. column)echelonform. InanISSAC’13paper, 0 2 we proposed a recursive Gaussian elimination that can compute simulta- neouslytherowandcolumnrankprofilesofamatrixaswellasthoseofall ] of its leading sub-matrices, in the same time as state of the art Gaussian C eliminationalgorithms. HerewefirststudytheconditionsmakingaGaus- S sianeliminationalgorithmrevealthisinformation. Therefore,wepropose . thedefinitionofanewmatrixinvariant, therankprofilematrix,summa- s c rizingallinformationontherowandcolumnrankprofilesofalltheleading [ sub-matrices. We also explore the conditions for a Gaussian elimination algorithmtocomputeallorpartofthisinvariant,throughthecorrespond- 2 v ing PLUQ decomposition. As a consequence, we show that the classical 9 iterative CUP decomposition algorithm can actually be adapted to com- 3 pute the rank profile matrix. Used, in a Crout variant, as a base-case 2 toourISSAC’13implementation,itdeliversasignificantimprovementin 5 efficiency. Second, the row (resp. column) echelon form of a matrix are 0 usually computed via different dedicated triangular decompositions. We 1. showherethat,fromsomePLUQdecompositions,itispossibletorecover 0 the row and column echelon forms of a matrix and of any of its leading 5 sub-matrices thanks to an elementary post-processing algorithm. 1 : v 1 Introduction i X r Triangularmatrixdecompositionsarewidelyusedincomputationallinearalge- a bra. Besides solving linear systems of equations, they are also used to compute other objects more specific to exact arithmetic: computing the rank, sampling a vector from the null-space, computing echelon forms and rank profiles. ∗This work is partly funded by the HPAC project of the French Agence Nationale de la Recherche (ANR 11 BS02 013). †UniversitédeGrenobleAlpes. LaboratoireLJK,umrCNRS.51,av. desMathématiques, F38041Grenoble,France. [email protected], ‡Université de Grenoble Alpes. Laboratoire LIP, Inria, CNRS, UCBL, ENS de Lyon, 46, Alléed’Italie,F69364LyonCedex07France. [email protected], §Université de Grenoble Alpes. Laboratoires LJK and LIG, Inria, CNRS. Inovallée, 655, av. del’Europe,F38334StIsmierCedex,France. [email protected]. 1 The row rank profile (resp. column rank profile) of an m×n matrix A with rankr,denotedbyRowRP(A)(resp.ColRP(A)),isthelexicographicallysmall- est sequence of r indices of linearly independent rows (resp. columns) of A. An m×nmatrixhasgenericrow(resp.column)rankprofileifitsrow(resp.column) rank profile is (1,..,r). Lastly, an m×n matrix has generic rank profile if its r first leading principal minors are non-zero. Note that if a matrix has generic rankprofile, thenitsrowandcolumnrankprofilesaregeneric, buttheconverse (cid:20) (cid:21) 01 is false: the matrix does not have generic rank profile even if its row and 10 column rank profiles are generic. The row support (resp. column support) of a matrix A, denoted by RowSupp(A) (resp. ColSupp(A)), is the subset of indices of its non-zero rows (resp. columns). We recall that the row echelon form of an m × n matrix A is an upper triangular matrix E = TA, for a non-singular matrix T, with the zero rows of E at the bottom and the non-zero rows in stair-case shape: min{j :a 6=0}< i,j min{j :a 6=0}. As T is non singular, the column rank profile of A is that i+1,j ofE,andthereforecorrespondstothecolumnindicesoftheleadingelementsin thestaircase. SimilarlytherowrankprofileofAiscomposedoftherowindices of the leading elements in the staircase of the column echelon form of A. Rank profile and triangular matrix decompositions The rank profiles of a matrix and the triangular matrix decomposition obtained by Gaussian elimination are strongly related. The elimination of matrices with arbitrary rank profiles gives rise to several matrix factorizations and many algorithmic variants. In numerical linear algebra one often uses the PLUQ decomposition, with P and Q permutation matrices, L a lower unit triangular matrix and U an upper triangular matrix. The LSP and LQUP variants of [8] are used to reduce the complexity rank deficient Gaussian elimination to that of matrix multiplication. Many other algorithmic decompositions exist allowing fraction free computations [10], in-place computations [4, 9] or sub-cubic rank-sensitive time complexity [13, 9]. In [5] we proposed a Gaussian elimination algorithm with a recursive splitting of both row and column dimensions, and replacing row and column transpositions by rotations. This elimination can compute simultaneously the row and column rank profile while preserving the sub-cubic rank-sensitive time complexity and keeping the computation in-place. InthispaperwefirststudytheconditionsaPLUQdecompositionalgorithm must satisfy in order to reveal the rank profile structure of a matrix. We introduce in section 2 the rank profile matrix R , a normal form summarizing A all rank profile information of a matrix and of all its leading sub-matrices. We thendecompose,insection3,thepivotingstrategyofanyPLUQalgorithminto two types of operations: the search of the pivot and the permutation used to moveittothemaindiagonal. Weproposeanewsearchandanewpermutation strategyandshowwhatrankprofilesarecomputedusinganypossiblecombina- tion of these operations and the previously used searches and permutations. In particular we show three new pivoting strategy combinations that compute the 2 rankprofilematrixanduseoneofthem,aniterativeCroutCUPwithrotations, to improve the base case and thus the overall performance of exact Gaussian elimination. Second, we show that preserving both the row and column rank profiles, together with ensuring a monotonicity of the associated permutations, allows us to compute faster several other matrix decompositions, such as the LEU and Bruhat decompositions, and echelon forms. In the following, 0 denotes the m×n zero matrix and A denotes m×n i..j,k..l the sub-matrix of A of rows between i and j and columns between k and l. To apermutationσ :{1,...,n}→{1,...,n}wedefinetheassociatedpermutation matrix P , permuting rows by left multiplication: the rows of P A are that of σ σ A permuted by σ. Reciprocally, for a permutation matrix P, we denote by σ P the associated permutation. 2 The rank profile matrix We start by introducing in Theorem 1 the rank profile matrix, that we will use throughoutthisdocumenttosummarizeallinformationontherankprofilesofa matrix. Fromnowon,matricesareoverafieldKandavalidpivotisanon-zero element. Definition 1. An r-sub-permutation matrix is a matrix of rank r with only r non-zero entries equal to one. Lemma 1. An m×n r-sub-permutation matrix has at most one non-zero entry (cid:20) (cid:21) I per rowand per column, and can bewritten P r Q where P and 0 (m−r)×(n−r) Q are permutation matrices. Theorem 1. Let A ∈ Km×n. There exists a unique m × n rank(A)-sub- permutation matrix R of which every leading sub-matrix has the same rank A as the corresponding leading sub-matrix of A. This sub-permutation matrix is called the rank profile matrix of A. Proof. We prove existence by induction on the row dimension of the leading submatrices. If A =0 , setting R(1) =0 satisfies the defining condition. Oth- 1,1..n 1×n 1×n erwise, let j be the index of the leftmost invertible element in A and set 1,1..n R(1) =eT thej-thn-dimensionalrowcanonicalvector,whichsatisfiesthedefin- j ing condition. Now for a given i ∈ {1,...,m}, suppose that there is a unique i × n rank profile matrix R(i) such that rank(A ) = rank(R ) for every 1..i,1..j 1..i,1..j (cid:20)R(i)(cid:21) j ∈ {1..n}. If rank(A ) = rank(A ), then R(i+1) = . Oth- 1..i+1,1..n 1..i,1..n 0 1×n erwise, consider k, the smallest column index such that rank(A ) = 1..i+1,1..k (cid:20)R(i)(cid:21) rank(A )+1 and set R(i+1) = . Any leading sub-matrix of R(i+1) 1..i,1..k eT k has the same rank as the corresponding leading sub-matrix of A: first, for any 3 leadingsubsetofrowsandcolumnswithlessthanirows, thecaseiscoveredby (cid:20) (cid:21) B u the induction; second define =A , where u,v are vectors and x vT x 1..i+1,1..k isascalar. Fromthedefinitionofk,vislinearlydependentwithB andthusany (cid:20) (cid:21) B leading sub-matrix of has the same rank as the corresponding sub-matrix vT of R(i+1). Similarly, from the definition of k, the same reasoning works when considering more than k columns, with a rank increment by 1. Lastly we show that R(i+1) is a r -sub-permutation matrix. Indeed, u is lin- i+1 (cid:2) (cid:3) early dependent with the columns of B: otherwise, rank( B u )=rank(B)+1. (cid:20) (cid:21) B u (cid:2) (cid:3) From the definition of k we then have rank( ) = rank( B u ) + 1 = vT x (cid:20) (cid:21) B rank(B)+2=rank( )+2 which is a contradiction. Consequently, the k-th vT column of R(i) is all zero, and R(i+1) is a r-sub-permutation matrix. To prove uniqueness, suppose there exist two distinct rank profile matrices R(1) and R(2) for a given matrix A and let (i,j) be some coordinates where R(1) 6= R(2) and R(1) = R(2) . Then, rank(A ) = 1..i,1..j 1..i,1..j 1..i−1,1..j−1 1..i−1,1..j−1 1..i,1..j rank(R(1) )6=rank(R(2) )=rank(A ) which is a contradiction. 1..i,1..j 1..i,1..j 1..i,1..j     2030 1000 1000 0010 Example 1. A = 0040 has RA = 0000 for rank profile matrix over 0201 0100 Q. Remark1. ThematrixE introducedinMalaschonok’sLEUdecomposition[12, Theorem 1], is in fact the rank profile matrix. There, the existence of this decomposition was only shown for m = n = 2k, and no connection was made to the relation with ranks and rank profiles. This connection was made in [5, Corollary 1], and the existence of E generalized to arbitrary dimensions m and n. Finally, after proving its uniqueness here, we propose this definition as a new matrix normal form. The rank profile matrix has the following properties: Lemma 2. Let A be a matrix. 1. R is diagonal if A has generic rank profile. A 2. R is a permutation matrix if A is invertible A 3. RowRP(A)=RowSupp(R ); ColRP(A)=ColSupp(R ). A A Moreover, for all 1≤i≤m and 1≤j ≤n, we have: 4. RowRP(A )=RowSupp((R ) ) 1..i,1..j A 1..i,1..j 5. ColRP(A )=ColSupp((R ) ), 1..i,1..j A 1..i,1..j These properties show how to recover the row and column rank profiles of A and of any of its leading sub-matrix. 4 3 Ingredients of a PLUQ decomposition algorithm Over a field, the LU decomposition generalizes to matrices with arbitrary rank profiles, using row and column permutations (in some cases such as the CUP, or LSP decompositions, the row permutation is embedded in the structure of the C or S matrices). However such PLUQ decompositions are not unique and not all of them will necessarily reveal rank profiles and echelon forms. We will characterize the conditions for a PLUQ decomposition algorithm to reveal the row or column rank profile or the rank profile matrix. WeconsiderthefourtypesofoperationsofaGaussianeliminationalgorithm in the processing of the k-th pivot: Pivot search: finding an element to be used as a pivot, Pivot permutation: moving the pivot in diagonal position (k,k) by column and/or row permutations, Update: applying the elimination at position (i,j): a ←a −a a−1a , i,j i,j i,k k,k k,j Normalization: dividing the k-th row (resp. column) by the pivot. Choosing how each of these operation is done, and when they are scheduled results in an elimination algorithm. Conversely, any Gaussian elimination algo- rithmcomputingaPLUQdecompositioncanbeviewedasasetofspecializations of each of these operations together with a scheduling. The choice of doing the normalization on rows or columns only determines which of U or L will be unit triangular. The scheduling of the updates vary dependingonthetypeofalgorithmused: iterative,recursive,slabortiledblock splitting, with right-looking, left-looking or Crout variants [2]. Neither the normalization nor the update impact the capacity to reveal rank profiles and we will thus now focus on the pivot search and the permutations. Choosing a search and a permutation strategy fixes the matrices P and Q ofthePLUQdecompositionobtainedand,aswewillsee,determinestheability to recover information on the rank profiles. Once these matrices are fixed, the L and the U factors are unique. We introduce the pivoting matrix. Definition 2. The pivoting matrix of a PLUQ decomposition A = PLUQ of rank r is the r-sub-permutation matrix (cid:20) (cid:21) I Π =P r Q. P,Q 0 (m−r)×(n−r) The r non-zero elements of Π are located at the initial positions of the P,Q pivots in the matrix A. Thus Π summarizes the choices made in the search P,Q and permutation operations. Pivot search The search operation vastly differs depending on the field of application. In numerical dense linear algebra, numerical stability is the main criterion for the selection of the pivot. In sparse linear algebra, the pivot is 5 chosen so as to reduce the fill-in produced by the update operation. In order to reveal some information on the rank profiles, a notion of precedence has to be used: a usual way to compute the row rank profile is to search in a given row forapivotandonlymovetothenextrowifthecurrentrowwasfoundtobeall zeros. This guarantees that each pivot will be on the first linearly independent row, and therefore the row support of Π will be the row rank profile. The P,Q precedence here is that the pivot’s coordinates must minimize the order for the first coordinate (the row index). As a generalization, we consider the most commonpreordersofthecartesianproduct{1,...m}×{1,...n}inheritedfrom the natural orders of each of its components and describe the corresponding search strategies, minimizing this preorder: Row order: (i ,j ) (cid:22) (i ,j ) iff i ≤ i : search for any invertible element 1 1 row 2 2 1 2 in the first non-zero row. Column order: (i ,j ) (cid:22) (i ,j ) iff j ≤ j . search for any invertible ele- 1 1 col 2 2 1 2 ment in the first non-zero column. Lexicographic order: (i ,j ) (cid:22) (i ,j ) iff i < i or i = i and j ≤ j : 1 1 lex 2 2 1 2 1 2 1 2 search for the leftmost non-zero element of the first non-zero row. Reverse lexicographic order: (i ,j ) (cid:22) (i ,j ) iff j < j or j = j 1 1 revlex 2 2 1 2 1 2 and i ≤i : search for the topmost non-zero element of the first non-zero 1 2 column. Product order: (i ,j ) (cid:22) (i ,j ) iff i ≤ i and j ≤ j : search for any 1 1 prod 2 2 1 2 1 2 non-zero element at position (i,j) being the only non-zero of the leading (i,j) sub-matrix.   0 0 0 a b 0 c d e f Example 2. Considerthematrix , whereeachliteralisanon-zero g h i j k l mn o p element. The minimum non-zero elements for each preorder are the following: Row order a,b Column order g,l Lexicographic order a Reverse lexic. order g Product order a,c,g Pivot permutation The pivot permutation moves a pivot from its initial position to the leading diagonal. Besides this constraint all possible choices are left for the remaining values of the permutation. Most often, it is done by row or column transpositions, as it clearly involves a small amount of data movement. However, these transpositions can break the precedence relations in the set of rows or columns, and can therefore prevent the recovery of the rank profile information. A pivot permutation that leaves the precedence relations unchanged will be called k-monotonically increasing. Definition 3. A permutation of σ ∈S is called k-monotonically increasing if n its last n−k values form a monotonically increasing sequence. 6 In particular, the last n−k rows of the associated row-permutation matrix P are in row echelon form. For example, the cyclic shift between indices k and σ i, with k < i defined as R = (1,...,k−1,i,k,k+1,...,i−1,i+1,...,n), k,i that we will call a (k,i)-rotation, is an elementary k-monotonically increasing permutation. Example 3. The (1,4)-rotation R =(4,1,2,3) is a 1-monotonically increas- 1,4   0 1 1  ing permutation. Its row permutation matrix is  . In fact, any (k,i)-  1  10 rotation is a k-monotonically increasing permutation. MonotonicallyincreasingpermutationscanbecomposedasstatedinLemma3. Lemma 3. If σ ∈ S is a k -monotonically increasing permutation and σ ∈ 1 n 1 2 S ×S a k -monotonically increasing permutation with k < k then the k1 n−k1 2 1 2 permutation σ ◦σ is a k -monotonically increasing permutation. 2 1 2 Proof. Thelastn−k valuesofσ ◦σ aretheimageofasub-sequenceofn−k 2 2 1 2 values from the last n−k values of σ through the monotonically increasing 1 1 function σ . 2 Therefore an iterative algorithm, using rotations as elementary pivot permutations, maintains the property that the permutation matrices P and Q at any step k are k-monotonically increasing. A similar property also applies with recursive algorithms. 4 How to reveal rank profiles A PLUQ decomposition reveals the row (resp. column) rank profile if it can be readfromthefirstrvaluesofthepermutationmatrixP (resp.Q). Equivalently, by Lemma 2, this means that the row (resp. column) support of the pivoting matrix Π equals that of the rank profile matrix. P,Q Definition 4. The decomposition A=PLUQ reveals: 1. the row rank profile if RowSupp(Π )=RowSupp(R ), P,Q A 2. the col. rank profile if ColSupp(Π )=ColSupp(R ), P,Q A 3. the rank profile matrix if Π =R . P,Q A     2030 1000 1000 0010 Example 4. A = 0040 has RA = 0000 for rank profile matrix over 0201 0100 Q. Now the pivoting matrix obtained from a PLUQ decomposition with a pivot search operation following the row order (any column, first non-zero row) could 7   0010 1000 be the matrix ΠP,Q =0000. As these matrices share the same row support, 0100 the matrix Π reveals the row rank profile of A. P,Q Remark 2. Example 4, suggests that a pivot search strategy minimizing row and column indices could be a sufficient condition to recover both row and column rank profiles at the same time, regardless the pivot permutation. However, this is unfortunately not the case. Consider for example a search based on the lexicographic order (first non-zero column of the first non-zero row) with trans- (cid:20) (cid:21) 001 position permutations, run on the matrix: A= . Its rank profile matrix 230 (cid:20) (cid:21) (cid:20) (cid:21) 001 001 is R = whereas the pivoting matrix could be Π = , which A 100 P,Q 010 does not reveal the column rank profile. This is due to the fact that the column transposition performed for the first pivot changes the order in which the columns will be inspected in the search for the second pivot. We will show that if the pivot permutations preserve the order in which the still unprocessed columns or rows appear, then the pivoting matrix will equal the rank profile matrix. This is achieved by the monotonically increasing permutations. Theorem 2 shows how the ability of a PLUQ decomposition algorithm to recover the rank profile information relates to the use of monotonically increasing permutations. More precisely, it considers an arbitrary step in a PLUQ decomposition where k pivots have been found in the elimination of an ‘×p leading sub-matrix A of the input matrix A. 1 Theorem 2. Consider a partial PLUQ decomposition of an m×n matrix A: (cid:20) (cid:21)(cid:20) (cid:21) L U V A=P 1 1 1 Q 1 M I H 1 1 m−k (cid:20) (cid:21) where L1 ism×k lowertriangularand(cid:2)U V (cid:3)isk×nuppertriangular, and M 1 1 1 let A be some ‘×p leading sub-matrix of A, for ‘,p≥k. Let H =P L U Q 1 2 2 2 2 be a PLUQ decomposition of H. Consider the PLUQ decomposition (cid:20)I (cid:21)(cid:20) L (cid:21)(cid:20)U V QT(cid:21)(cid:20)I (cid:21) A=P k 1 1 1 2 k Q . 1 P PTM L U Q 1 2 2 1 2 2 2 | {z }| {z }| {z }| {z } P L U Q Consider the following clauses: (i) RowRP(A )=RowSupp(Π ) 1 P1,Q1 (ii) ColRP(A )=ColSupp(Π ) 1 P1,Q1 (iii) R =Π A1 P1,Q1 (iv) RowRP(H)=RowSupp(Π ) P2,Q2 8 (v) ColRP(H)=ColSupp(Π ) P2,Q2 (vi) R =Π H P2,Q2 (vii) PT is k-monotonically increasing or (PT is ‘-monotonically increasing 1 1 and p=n) (viii) QT is k-monotonically increasing or (QT is p-monotonically increasing 1 1 and ‘=m) Then, (cid:20) (cid:21) 0 ∗ (a) if (i) or (ii) or (iii) then H = (‘−k)×(p−k) ∗ ∗ (b) if (vii) then ((i) and (iv)) ⇒RowRP(A)=RowSupp(Π ); P,Q (c) if (viii) then ((ii) and (v)) ⇒ColRP(A)=ColSupp(Π ); P,Q (d) if (vii) and (viii) then (iii) and (vi) ⇒R =Π . A P,Q (cid:20) (cid:21) Proof. Let P =(cid:2)P E (cid:3) and Q = Q11 where E is m×(m−k) and F is 1 11 1 1 F 1 1 1 (n−k)×n. On one hand we have (cid:20) (cid:21) (cid:20) (cid:21) A=(cid:2)P E (cid:3) L1 (cid:2)U V (cid:3) Q11 +E HF . (1) 11 1 M 1 1 F 1 1 1 1 | {z } B On the other hand, (cid:20) (cid:21)(cid:20) (cid:21)(cid:20) (cid:21) I I I Π =P k r k Q P,Q 1 P 0 Q 1 2 (m−r)×(n−r) 2 (cid:20) (cid:21) I =P k Q =Π +E Π F . 1 Π 1 P1,Q1 1 P2,Q2 1 P2,Q2 (cid:20) (cid:21) A 0 Let A = 1 and denote by B the ‘×p leading sub-matrix 1 0 0 1 (m−‘)×(n−p) of B. (a) Theclause(i)or(ii)or(iii)impliesthatallkpivotsofthepartialelimination were found within the ‘×p sub-matrix A . Hence rank(A ) = k and we 1 1 (cid:20) (cid:21) (cid:20) (cid:21) P Q 0 can write P = 11 E and Q = 11 k×(n−p) , and the matrix 1 0 1 1 F (m−‘)×k 1 A writes 1 (cid:20) (cid:21) (cid:20) (cid:21) A =(cid:2)I 0(cid:3)A Ip =B +(cid:2)I 0(cid:3)E HF Ip . (2) 1 ‘ 0 1 ‘ 1 1 0 Now rank(B )=k as a sub-matrix of B of rank k and since 1   B1 =(cid:2)P11 (cid:2)I‘ 0(cid:3)·E1(cid:3)(cid:20)ML1(cid:21)(cid:2)U1 V1(cid:3)F Q·(cid:20)11Ip(cid:21) 1 1 0 (cid:20) (cid:21) =P L U Q +(cid:2)I 0(cid:3)E M (cid:2)U V (cid:3)Q Ip 11 1 1 11 ‘ 1 1 1 1 1 0 9 where the first term, P L U Q , has rank k and the second term has a 11 1 1 11 disjoint row support. (cid:20) (cid:21) Finally, consider the term (cid:2)I 0(cid:3)E HF Ip of equation (2). As its row ‘ 1 1 0 supportisdisjointwiththatofthepivotrowsofB ,ithastobecomposedof 1 rowslinearlydependentwiththepivotrowsofB toensurethatrank(A )= 1 1 k. As its column support is disjoint with that of the pivot columns of B , we conclude that it must be the zero matrix. Therefore the leading 1 (‘−k)×(p−k) sub-matrix of E HF is zero. 1 1 (b) From (a) we know that A =B . Thus RowRP(B)=RowRP(A ). Recall 1 1 1 that A=B+E HF . No pivot row of B can be made linearly dependent 1 1 by adding rows of E HF , as the column position of the pivot is always 1 1 zero in the latter matrix. For the same reason, no pivot row of E HF can 1 1 be made linearly dependent by adding rows of B. From (i), the set of pivot rows of B is RowRP(A ), which shows that 1 RowRP(A)=RowRP(A )∪RowRP(E HF ). (3) 1 1 1 Letσ :{1..m−k}→{1..m}bethemaprepresentingthesub-permutation E1 E (i.e. suchthatE [σ (i),i]=1∀i). IfPT isk-monotonicallyincreasing, 1 1 E1 1 the matrix E has full column rank and is in column echelon form, which 1 implies that RowRP(E HF )=σ (RowRP(HF )) 1 1 E1 1 =σ (RowRP(H)), (4) E1 sinceF hasfullrowrank. IfPT is‘monotonicallyincreasing,wecanwrite 1 1 (cid:2) (cid:3) E = E E , where the m×(m−‘) matrix E is in column echelon 1 11 12 12 (cid:20) (cid:21) 0 form. If p = n, the matrix H writes H = (‘−k)×(n−k) . Hence we have H 2 E HF =E H F which also implies 1 1 12 2 1 RowRP(E HF )=σ (RowRP(H)). 1 1 E1 Fromequation(2),therowsupportofΠ isthatofΠ +E Π F , P,Q P1,Q1 1 P2,Q2 1 which is the union of the row support of these two terms as they are disjoint. Under the conditions of point (b), this row support is the union of RowRP(A ) and σ (RowRP(H)), which is, from (4) and (3), RowRP(A). 1 E1 (c) Similarly as for point (b). (d) From (a) we have still A = B . Now since rank(B) = rank(B ) = 1 1 1 rank(A ) = k, there is no other non-zero element in R than those in 1 B R and R = R . The row and column support of R and that of A1 B A1 B E HF are disjoint. Hence 1 1 R =R +R . (5) A A1 E1HF1 If both PT and QT are k-monotonically increasing, the matrix E is in 1 1 1 columnechelonformandthematrixF inrowechelonform. Consequently, 1 10