The accurate solution of the forward electrostatic problem is an essential first step before solving the inverse problem of magneto- and electro-encephalography (MEG/EEG). The symmetric Galerkin boundary element method is accurate but is difficule to use for very large problems because of its computational complexity and memory requirements. We describe a fast multipole-based acceleration for the symmetric BEM with complexity. It creates a hierarchical structure of the elements and approximates far interactions using spherical harmonics expansions. The accelerated method is shown to be as accurate as the direct method, yet for large problems it is both faster and more economical in terms of memory consumption.