It is assumed that complex perceptual or sensori-motor tasks are the result of neural network dynamics and are expressed by spike trains containing the neural code. Hence, in this context two main questions are (i) How to characterize the statistical properties of sequences the spikes trains produced by neuronal networks and (ii) What are the effects of synaptic plasticity on these statistics? Using methods from dynamical systems theory and statistical physics, we introduce a framework which applies for very general forms of spike train properties allowing to characterize miscellaneous forms of neural code (rank coding, synchronizations, correlations of different orders, etc.). In this framework, spike trains are associated with a coding of membrane potential trajectories constituting a symbolic coding in important modeling examples. On this basis, we use the thermodynamic formalism from ergodic theory to show how Gibbs distributions are natural probability measures to describe the statistics of spike trains, given the data of known empirical averages. Finally, we show that Gibbs distribution naturally arise when considering slow synaptic plasticity rules, where the only requirement is that characteristic time for synapse adaptation must be quite a bit longer that the characteristic time for neurons dynamics. We include some simulations results applying this framework on recurrent neural network with discrete time current based dynamics under the action of an STDP rule (see Figures 1 and 2). Numerical results are in accord with theory establishing that the topological pressure is a variation quantity with a minimum given by a Gibbs distribution (Figure 3).