We consider a nonzero-sum $N$-player Markov game on an abstract measurable state space with compact metric action spaces. The payoff functions are bounded Caratheodory functions and the transitions of the system are assumed to have a density function satisfying some continuity conditions. The optimality criterion of the players is given by a total expected payoff on an infinite discrete-time horizon. Under the condition that the game model is absorbing, we establish the existence of Markov strategies that are a noncooperative equilibrium in the family of all history-dependent strategies of the players for both the constrained and the unconstrained problems. We obtain, as a particular case of results, the existence of Nash equilibria for discounted constrained and unconstrained game models.