Superior Parallel Big Data Clustering through Competitive Stochastic Sample Size Optimization in Big-means

arXiv:2403.18766v1 Announce Type: new
Abstract: This paper introduces a novel K-means clustering algorithm, an advancement on the conventional Big-means methodology. The proposed method efficiently integrates parallel processing, stochastic sampling, and competitive optimization to create a scalable variant designed for big data applications. It addresses scalability and computation time challenges typically faced with traditional techniques. The algorithm adjusts sample sizes dynamically for each worker during execution, optimizing performance. Data from these sample sizes are continually analyzed, facilitating the identification of the …

abstract advancement algorithm applications arxiv big big data clustering clustering algorithm computation cs.ai cs.dc cs.ir cs.lg data data applications k-means methodology novel optimization paper processing sample sampling scalability scalable stochastic through type