The scattering transform is a hierarchical signal transformation that has been designed to be robust to signal deformations. It can be used to compute representations with invariance or tolerance to any transformation group, such as translations, rotations or scaling. In image analysis, going beyond edge detection, its second layer captures higher order features, providing a fine-grain dissection of the signal. Here we use the output coefficients to fit blood oxygen level dependent (BOLD) signal in visual areas using functional magnetic resonance imag- ing. Significant improvement in the prediction accuracy is shown when using the second layer in addition to the first, suggesting biological relevance of the features extracted in layer two or linear combinations thereof.