The capacity to perceive and interpret highly complex visual patterns such as body movements and face gestures, is remarkably efficient in humans and many other species. Among others tasks, the classification of visual sequences without context is a key problem to understand both the coding and the retrieval of spatiotemporal patterns in the human brain. In this work we present a model able to discriminate visual sequences. This model is based on asymmetric neural fields. We apply it to the classification of synthetic sequences. Our model takes into account several properties exhibited by experimental psychophysics and physiology. The presented model shows that sparse spatial coding of spatiotemporal sequences could be sufficient to explain some of these properties, such as classification with partial sequence information and tolerance to time-warping. We are also able to code a temporal sequence with a single population of units, without the need of explicit “snapshots” at each time instant.