Skip to content Skip to navigation

echofs: A Scheduler-guided Temporary Filesystem to Leverage Node-local NVMs

Authors: 
Alberto Miranda, Ramon Nou, Toni Cortes

The growth in data-intensive scientific applications poses strong demands on the HPC storage subsystem, as data needs to be copied from compute nodes to I/O nodes and vice versa for jobs to run. The emerging trend of adding denser, NVM-based burst buffers to compute nodes, however, offers the possibility of using these resources to build temporary filesystems with specific I/O optimizations for a batch job. In this work, we present echofs, a temporary filesystem that coordinates with the job scheduler to preload a job’s input files into node-local burst buffers. We present the results measured with NVM emulation, and different FS backends with DAX/FUSE on a local node, to show the benefits of our proposal and such coordination.

 Conference: 30th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD 2018)