Toward a Push-based Stream Programming Model with AIMSS: An Active In-Memory Storage System Approach

Today's passive (on-disk and/or in-memory, employing a pull-based data access approach) storage architectures are performance- and energy- insufficient for handling the data-intensive demands of tomorrow's exascale machine learning and artificial intelligence (ML/AI) workloads. Industry projections forecast beyond-exascale clusters consuming energy between 500 MW and 1 TW, highlighting the need for a paradigm shift in data movement and processing, necessitating novel solutions that can improve performance, reduce energy consumption, and simplify application development and deployment. We believe exascale computing will require in-memory storage systems with a global perspective on I/O and processing, strategically positioned between traditional disk-based storage systems and CPU-GPU compute engines.
We present the vision for an Active In-Memory Storage System (AIMSS), a novel architecture that shifts data movement management, such as source/sink handling and data shuffling, from ML/AI applications and big data streaming engines, directly to AIMSS. Operating on a log-structured in-memory storage framework, leveraging immutable data access patterns, and facilitating efficient real-time data movement, the AIMSS architecture will be deployed on tens of thousands of large many-core CPU-GPU nodes, harnessing their memory and ensuring efficient and transparent communication with traditional disk-based file storage systems.
We propose a push-based streaming execution model enabling AIMSS to cost-effectively harness application-specific data (such as consumer/producer offsets and data access patterns including read, write, and shuffle) and thereby enable a set of optimizations such as scalable data movement partitioning algorithms, faster stream storage recovery, mitigation of application stragglers, mitigating power fluctuation issues during large-scale ML/AI training by efficiently leveraging idle GPU resources for other computing tasks, and minimizing I/O interference in multi-CPU-GPU setups for multiple applications sharing an exascale high-performance computing infrastructure.
Through its global view of I/O enabled by a push-based in-memory computing approach, AIMSS promises significant performance improvements for data-intensive applications by actively handling data movement, while eliminating the need for manual tuning and inefficient application-based data management.