Reinforcement Learning intends to ease and possibly to perform automatically the design of systems such as software or robot agents. An important aspect is the ability of learning agents to adapt to their environment and to the task they have to accomplish. This kind of learning is unfortunately restrained by problems like combinatorial explosion of the state space that limits the number of sensors or objects an agent can reasonably deal with, especially in the case of Multi-Agent Systems. Considering Markov Decision Processes, different solutions exist to overcome the difficulties related to large state spaces: hierarchical structures \cite{Parr98} or factored representations \cite{Kearns99} of the agent's behavior for example. Nevertheless, these tools require manual preparations before going through the learning step, and result in an agent designed for a specific task. The work presented here intends to define agents able to be efficient in several complex situations, reusing prior knowledge (see also \cite{Dixon00}). The design is based on the use of many basic behaviors which the agent will have to manage, each behavior corresponding to a different motivation. For now, Reinforcement Learning is mainly employed for the ``recording'' of these basic behaviors. Nevertheless there is place for improvements of our framework through other uses of Reinforcement Learning.