Skip to Main content Skip to Navigation
Conference papers

CompactFlow: A Hybrid Binary Format for Network Flow Data

Abstract : Network traffic monitoring has become fundamental to obtaining insights about a network and its activities. This knowledge allows network administrators to detect anomalies, identify faulty hardware, and make informed decisions. The increase of the number of connected devices and the consequent volume of traffic poses a serious challenge to carrying out the task of network monitoring. Such a task requires techniques that process traffic in an efficient and timely manner. Moreover, it is crucial to be able to store network traffic for forensic purposes for as long a period of time as possible.In this paper, we propose CompactFlow, a hybrid binary format for efficient storage and processing of network flow data. Our solution offers a trade-off between the space required and query performance via an optimized binary representation of flow records and optional indexing. We experimentally assess the efficiency of CompactFlow by comparing it to a wide range of binary flow storage formats. We show that CompactFlow format improves the state of the art by reducing the size required to store network flows by more than 24%.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03173900
Contributor : Hal Ifip <>
Submitted on : Thursday, March 18, 2021 - 5:38:28 PM
Last modification on : Thursday, March 18, 2021 - 5:51:03 PM

File

 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2023-01-01

Please log in to resquest access to the document

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Michal Piskozub, Riccardo Spolaor, Ivan Martinovic. CompactFlow: A Hybrid Binary Format for Network Flow Data. 13th IFIP International Conference on Information Security Theory and Practice (WISTP), Dec 2019, Paris, France. pp.185-201, ⟨10.1007/978-3-030-41702-4_12⟩. ⟨hal-03173900⟩

Share

Metrics

Record views

9