Skip to Main content Skip to Navigation
New interface
Conference papers

Video-based Behavior Understanding of Children for Objective Diagnosis of Autism

Abstract : One of the major diagnostic criteria for Autism Spectrum Disorder (ASD) is the recognition of stereotyped behaviors. However, it primarily relies on parental interviews and clinical observations, which result in a prolonged diagnosis cycle preventing ASD children from timely treatment. To help clinicians speed up the diagnosis process, we propose a computer-vision-based solution. First, we collected and annotated a novel dataset for action recognition tasks in videos of children with ASD in an uncontrolled environment. Second, we propose a multi-modality fusion network based on 3D CNNs. In the first stage of our method, we pre-process the RGB videos to get the ROI (child) using Yolov5 and DeepSORT algorithms. For optical flow extraction, we use the RAFT algorithm. In the second stage, we perform extensive experiments on different deep learning frameworks to propose a baseline. In the last stage, a multi-modality-based late fusion network is proposed to classify and evaluate performance of ASD children. The results revealed that the multi-modality fusion network achieves the best accuracy as compared to other methods. The baseline results also demonstrate the potential of an action-recognition-based system to assist clinicians in a reliable, accurate, and timely diagnosis of ASD disorder.
Document type :
Conference papers
Complete list of metadata
Contributor : Abid Ali Connect in order to contact the contributor
Submitted on : Wednesday, November 24, 2021 - 4:14:54 PM
Last modification on : Saturday, June 25, 2022 - 11:54:02 PM
Long-term archiving on: : Friday, February 25, 2022 - 7:34:50 PM


Publisher files allowed on an open archive


  • HAL Id : hal-03447060, version 1


Abid Ali, Farhood F Negin, Francois F Bremond, Susanne Thümmler. Video-based Behavior Understanding of Children for Objective Diagnosis of Autism. VISAPP 2022 - 17th International Conference on Computer Vision Theory and Applications, Feb 2022, Online, France. ⟨hal-03447060⟩



Record views


Files downloads