Machine Learning-based Data Compression
A. Gallen*,
J. Smith and
for the Baler collaboration*: corresponding author
Pre-published on:
December 17, 2024
Published on:
April 29, 2025
Abstract
With the rise of novel computing techniques such as big data and artificial intelligence, many scientific and industrial disciplines are faced with exponentially increasing demands for data storage and compute resources. Traditional data compression algorithms are either generically applicable but lossless, limiting performance (e.g. zip), or lossy but designed for specific applications (e.g. jpg, mp3). Machine learning can be deployed to identify the most significant features of any given dataset and favour these features in a compression algorithm. Baler is a novel framework for developing, testing and deploying autoencoder-based data compression algorithms. In this talk, we report on recent developments to the Baler framework, including the implementation of Baler on FPGAs and how Baler has been used to compress data from atomic physics (Mössbauer imaging).
DOI: https://doi.org/10.22323/1.476.1005
How to cite
Metadata are provided both in
article format (very
similar to INSPIRE)
as this helps creating very compact bibliographies which
can be beneficial to authors and readers, and in
proceeding format which
is more detailed and complete.