Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data

arXiv:2307.04216v2 Announce Type: replace
Abstract: Lossy compression has become an important technique to reduce data size in many domains. This type of compression is especially valuable for large-scale scientific data, whose size ranges up to several petabytes. Although Autoencoder-based models have been successfully leveraged to compress images and videos, such neural networks have not widely gained attention in the scientific data domain. Our work presents a neural network that not only significantly compresses large-scale scientific data, but also maintains high …

abstract arxiv autoencoder become compression cs.ai cs.lg data domains eess.iv hierarchical images petabytes reduce resolution scale scientific type videos