Many numerical simulations end up on a problem of linear algebra involving an operator which is expressed after discretization by a very large sparse matrix. Typically, the problems include linear system solving, computation of eigenvalues and corresponding eigenvectors, and application of a function of the matrix on a given vector. To solve such problems, the methods based on Krylov subspaces have the advantage of not requiring a transformation of the matrix since they only use the matrix as an operator, i.e. through the multiplication of the matrix by a vector. The classical procedure to run these methods is the Arnoldi process which iteratively builds an orthonormal basis of the Krylov subspace. Unfortunately, this procedure has a limited potential for parallelism. To get rid of the bottleneck of the Gram-Schmidt procedure which is the heart of the Arnoldi process, non-orthonormal bases of Krylov subspaces are considered. The difficulty is then to avoid construction of too ill-conditioned bases. In this talk, we propose two types of three-term recurrences to generate such bases. In the last part and as an illustration, we present GPREMS, a parallel GMRES method preconditioned by a Multiplicative block-Schwarz iteration.