Abstract : Network traffic monitoring has become fundamental to obtaining insights about a network and its activities. This knowledge allows network administrators to detect anomalies, identify faulty hardware, and make informed decisions. The increase of the number of connected devices and the consequent volume of traffic poses a serious challenge to carrying out the task of network monitoring. Such a task requires techniques that process traffic in an efficient and timely manner. Moreover, it is crucial to be able to store network traffic for forensic purposes for as long a period of time as possible.In this paper, we propose CompactFlow, a hybrid binary format for efficient storage and processing of network flow data. Our solution offers a trade-off between the space required and query performance via an optimized binary representation of flow records and optional indexing. We experimentally assess the efficiency of CompactFlow by comparing it to a wide range of binary flow storage formats. We show that CompactFlow format improves the state of the art by reducing the size required to store network flows by more than 24%.
https://hal.inria.fr/hal-03173900
Contributor : Hal Ifip <>
Submitted on : Thursday, March 18, 2021 - 5:38:28 PM Last modification on : Thursday, March 18, 2021 - 5:51:03 PM
File
Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed
until : 2023-01-01
Michal Piskozub, Riccardo Spolaor, Ivan Martinovic. CompactFlow: A Hybrid Binary Format for Network Flow Data. 13th IFIP International Conference on Information Security Theory and Practice (WISTP), Dec 2019, Paris, France. pp.185-201, ⟨10.1007/978-3-030-41702-4_12⟩. ⟨hal-03173900⟩