Innovation in the physical layer of communication systems has traditionally been achieved by breaking down the transceivers into sets of processing blocks, each optimized independently based on mathematical models. This approach is now challenged by the ever-growing demand for wireless connectivity and the increasingly diverse set of devices and use-cases. Conversely, deep learning (DL)-based systems are able to handle increasingly complex tasks for which no tractable models are available. By learning from the data, these systems could be trained to embrace the undesired effects of practical hardware and channels instead of trying to cancel them. This thesis aims at comparing different approaches to unlock the full potential of DL in the physical layer. First, we describe a neural network (NN)-based block strategy, where an NN is optimized to replace one or multiple block(s) in a communication system. We apply this strategy to introduce a multi-user multiple-input multiple-output (MU-MIMO) detector that builds on top of an existing DL-based architecture. The key motivation is to replace the need for retraining on each new channel realization by a hypernetwork that generates optimized sets of parameters for the underlying DL detector. Second, we detail an end-to-end strategy, in which the transmitter and receiver are modeled as NNs that are jointly trained to maximize an achievable information rate. This approach allows for deeper optimizations, as illustrated with the design of waveforms that achieve high throughputs while satisfying peak-to-average power ratio (PAPR) and adjacent channel leakage ratio (ACLR) constraints. Lastly, we propose a hybrid strategy, where multiple DL components are inserted into a traditional architecture but trained to optimize the end-to-end performance. To demonstrate its benefits, we propose a DL-enhanced MU-MIMO receiver that both enable lower bit error rates (BERs) compared to a conventional receiver and remains scalable to any number of users. Each approach has its own strengths and shortcomings. While the first one is the easiest to implement, its individual block optimization does not ensure the overall system optimality. On the other hand, systems designed with the second approach are computationally complex and do not comply with current standards, but allow the emergence of new opportunities such as high-dimensional constellations and pilotless transmissions. Finally, even if the block-based architecture of the third approach prevents deeper optimizations, the combined flexibility and end-to-end performance gains motivate its use for short-term practical implementations