Many problems in scientific computing involving a large sparse square matrix $A$ are solved by Krylov subspace methods. This includes methods for the solution of large linear systems of equations with $A$, for the computation of a few eigenvalues and associated eigenvectors of $A$, and for the approximation of nonlinear matrix functions of $A$. When the matrix $A$ is non-Hermitian, the Arnoldi process commonly is used to compute an orthonormal basis for a Krylov subspace associated with $A$. The Arnoldi process often is implemented with the aid of the modified Gram-Schmidt method. It is well known that the latter constitutes a bottleneck in parallel computing environments, and to some extent also on sequential computers. Several approaches to circumvent orthogonalization by the modified Gram-Schmidt method have been described in the literature, including the generation of Krylov subspace bases with the aid of suitably chosen Chebyshev or Newton polynomials. We review these schemes and describe new ones. Numerical examples are presented.