Voice over IP (VoIP) has become a major paradigm for providing flexible telecommunication services and reducing operational costs. The large-scale deployment of VoIP has been leveraged by the high-speed broadband access to the Internet and the standardization of dedicated protocols. However, VoIP faces multiple security issues including vulnerabilities inherited from the IP layer as well as specific ones. Our objective is to design, implement and validate new models and architectures for performing proactive defense, monitoring and intrusion detection in VoIP networks. Our work combines two domains: network security and artificial intelligence. We reinforce existent security mechanisms by working on three axes: a machine learning approach for VoIP signaling traffic monitoring, a VoIP specific honeypot and a security event correlation model for intrusion detection.