Neural networks are considered as naturally parallel computing models. Their very fine-grain parallelism uses many information exchanges, so that hardware implementations are more likely to fit neural computations. Configurable hardware devices such as FPGAs offer a compromise between the hardware efficiency of ASICs and the flexibility of a simple software-like handling. Several works show that FPGAs are a real opportunity for flexible hardware implementations of neural networks. And yet the implementation of standard neural models raises some specific problems. Such difficulties may strongly limit the size and the architecture of the neural networks that can be mapped onto FPGAs. Moreover, these problems may prevent neural implementations on FPGAs from exploiting the fast FPGA technology improvements. This paper discusses the use of FPGAs for neural implementations. Both assets and obstacles are described, and the various solutions are outlined. The definition of hardware-adapted neural computation paradigms is to be favoured.