Towards Unified Data Ingestion and Transfer for the Computing Continuum

The computing continuum can enable new, novel big
data use cases across the edge-cloud-supercomputer spectrum.
Fast and high-volume data movement workflows rely on state-of-
the-art architectures built on top of stream ingestion and
file transfer open-source tools. Unfortunately, users struggle
when faced with dealing with such diverse architectures: stream
ingestion was designed for small-size datasets and low latency,
while file transfer was designed for large-size datasets and high
throughput. In this paper, we propose to unify ingestion and
transfer, while introducing architectural design principles and
discussing future implementation challenges.