Skip to Main content Skip to Navigation
Journal articles

Rain Rendering for Evaluating and Improving Robustness to Bad Weather

Abstract : Rain fills the atmosphere with water particles, which breaks the common assumption that light travels unaltered from the scene to the camera. While it is well-known that rain affects computer vision algorithms, quantifying its impact is difficult. In this context, we present a rain rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain. We present three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentation combines a physical particle simulator and accurate rain photometric modeling. We validate our rendering methods with a user study, demonstrating our rain is judged as much as 73% more realistic than the state-of-theart. Using our generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a thorough evaluation of object detection, semantic segmentation, and depth estimation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection, 60% for semantic segmentation, and 6-fold increase in depth estimation error. Finetuning on our augmented synthetic data results in improvements of 21% on object detection, 37% on semantic segmentation, and 8% on depth estimation.
Document type :
Journal articles
Complete list of metadata

https://hal.inria.fr/hal-03133284
Contributor : Raoul de Charette Connect in order to contact the contributor
Submitted on : Friday, February 5, 2021 - 8:36:42 PM
Last modification on : Friday, January 21, 2022 - 3:18:30 AM

Links full text

Identifiers

Collections

Citation

Maxime Tremblay, Shirsendu Sukanta Halder, Raoul de Charette, Jean-François Lalonde. Rain Rendering for Evaluating and Improving Robustness to Bad Weather. International Journal of Computer Vision, Springer Verlag, 2020, ⟨10.1007/s11263-020-01366-3⟩. ⟨hal-03133284⟩

Share

Metrics

Les métriques sont temporairement indisponibles