We consider a firing rate model of a neural network of excitatory and inhibitory populations with an excitatory feedforward connectivity. We analyze traveling wave solutions and determine the conditions for their existence and stability. Our study demonstrates the role of inhibition in stable pulse propagation. In a purely excitatory network, pulse waves are unstable because of the existence of stable front wave and back wave with different velocities. Pulse waves can propagate stably in the network where excitation is appropriately balanced by inhibition. Analytical results on the wave speeds and the shape of waves are obtained.