Skip to Main content Skip to Navigation
Reports

A Hadoop distribution for engineering simulation

Benoit Lange 1 Toan Nguyen 1
1 OPALE - Optimization and control, numerical algorithms and integration of complex multidiscipline systems governed by PDE
CRISAM - Inria Sophia Antipolis - Méditerranée , JAD - Laboratoire Jean Alexandre Dieudonné : UMR6621
Abstract : — In this paper, we discuss on the VELaSSCo project (Visualization for Extremely LArge-Scale Scientific Computing). This project aims to develop a specific platform to store scientific data for FEM (Finite Element Method) and DEM (Discrete Element Method) simulations. Both of these simulations are used by the engineering community to evaluate the behavior of a 3D object (for example fluid simulation in a silo). These simulations produce large files, which are composed of different time steps of a simulation. But the amount of produced data is too big to fit into a single node. Some strategies decompose data between nodes, but after several time-steps, some data has to be scratched to free memory. In this project, we aim to develop a platform, which enables the scientific community to store huge amounts of data on any kind of IT systems. We target to store data on any IT systems because most of scientists have access to modern computation nodes and not huge storage nodes. Our platform will try to fill the gap between both worlds. In this paper, we give an overview of the VELaSSCo project, and we detail our platform and deployment software. This platform can be deployed on any kind of IT system (dedicated storage nodes, HPC nodes, etc.). This platform is specially designed to store data from DEM and FEM simulations. In this paper, we present a performance analysis of our deployment tool compared to the well-defined myHadoop tool. With our tool we are able to increase computation capabilities with containers and virtualization.
Document type :
Reports
Complete list of metadata

Cited literature [23 references]  Display  Hide  Download

https://hal.inria.fr/hal-01130630
Contributor : Benoit Lange <>
Submitted on : Thursday, March 12, 2015 - 10:07:24 AM
Last modification on : Monday, October 12, 2020 - 2:28:04 PM
Long-term archiving on: : Monday, April 17, 2017 - 11:05:15 AM

File

Lange-2014-RR2.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01130630, version 1

Citation

Benoit Lange, Toan Nguyen. A Hadoop distribution for engineering simulation. [Research Report] INRIA Grenoble - Rhône-Alpes. 2014. ⟨hal-01130630⟩

Share

Metrics

Record views

358

Files downloads

488