Dynamic Data Layout Optimization with Worst-case Guarantees

arXiv:2405.04984v1 Announce Type: cross
Abstract: Many data analytics systems store and process large datasets in partitions containing millions of rows. By mapping rows to partitions in an optimized way, it is possible to improve query performance by skipping over large numbers of irrelevant partitions during query processing. This mapping is referred to as a data layout. Recent works have shown that customizing the data layout to the anticipated query workload greatly improves query performance, but the performance benefits may disappear …

abstract analytics arxiv case cs.db cs.ds cs.lg data data analytics datasets dynamic large datasets mapping numbers optimization performance process processing query query processing store systems type