Toward integration of multi-SPMD programming model and

advanced cyberinfrastructure platform

Miwako TsujiRIKEN Center for Computational Sceince

Agenda : In this paper, we introduce a multiSPMD (mSPMD) programming model, which com-bines a workflow paradigm and a distributed parallelprogramming model. Then, we discuss about currentissues in the mSPMD regarding data transfer. At theend, we describe future plan to integrate the mSPMDand advanced cyberinfrastructure platform (ACP).

A multi-SPMD programming model

Figure 1: Overview of the multi SPMD programmingmodel

In order to address reliability, fault tolerance, andscaling problems in future large scale systems, wehave proposed a multi SPMD (mSPMD) program-ming model and have been developing a developmentand execution environment for the mSPMD program-ming model[4, 3]. The mSPMD programming modelcombines a workflow paradigm and a distributed par-allel programming model for future large scale sys-tems, which would be highly hierarchical architecturewith nodes of many-core processors, accelerators andother different architectures.

Figure 1 shows the overview of the mSPMD pro-gramming model. Each task in a workflow can bea distributed parallel program. A PGAS languagecalled XcalableMP [5] has been supported to describetasks in a workflow. To describe and manage depen-dency between tasks, we adopt YML[1] workflow de-velopment and execution environment.

Since tasks in a workflow can be executed inparallel in the mSPMD programming model, some“heavy” tasks can be executed in parallel to speedup the workflow. Another advantage of the mSPMD

is that a huge distributed parallel program can be de-composed into several moderate sized sub-programsbased on the workflow paradigm to avoid the com-munication overhead.

Current “BigData” issues in mSPMDprogramming model

One of important pieces that the mSPMD program-ming model is missing is an intelligent implementa-tion of the data transfer method between tasks. Asshown in Figure 2, our current implementation of thedata transfer between tasks strongly relies on a net-work file system (NFS) and MPI-IO functions. Aftera task writes a data to a NFS, the other tasks whichuse the data are started and read the data from theNFS. The advantage of using NFS and MPI-IO are(1) portability (2) auto check-pointing and (3) easeof use for application developers since the MPI-IOfunction calls can be generated automatically basedon XcalableMP declarations.

Task computationwritten in XMPprocessed in parallel

Data import w/ MPI-IO

<Task 1>

node0 node1

node2 node3

Data export w/ MPI-IO

MPI_File_write()

Task computationwritten in XMPprocessed in parallel

Data import w/ MPI-IO

<Task 2>

node0 node1

node2 node3

Data export w/ MPI-IO

File System

A[N][N] distributed on 4 nodes

MPI_File

_read

()

Figure 2: Data transfer between tasks

The disadvantages of using NFS are speed andperformance instability. To overcome this, we areinvestigating the combination of file-IO and dataservers[2]. Additionally, as future works, we will in-vestigate advanced software and hardware infrastruc-tures such as ADIOS library, data-compression hard-ware, burst buffer.

1

Toward integration of multi-SPMDprogramming model and advanced cy-berinfrastructure platform

In addition to the advantages described above, themSPMD can combine several parallel libraries andexisting parallel programs easily to compose a com-plex application for a heterogeneous architecture.

Figure 3: Integration of mSPMD and ACP

The mSPMD may provide a new method for dataprocessing and data logistic among different systems.So far, we have focused on the data processing inan HPC cluster and the data dependencies betweenthem. However, we consider the workflow paradigmis also useful to manage data dependencies in/fromcyber-physical systems. An unified workflow ap-proach to describe dependencies among data gener-ations, data processings, HPC simulations to updatedata, etc... should be important to develop compli-cated applications. As future works, we will integratethe mSPMD programming model and ACP. As shownin Figure 3, our workflow paradigm will orchestratedata dependencies not only between traditional dis-tributed parallel programs, but also from/to varioussensing and processing devices.

References

[1] Olivier Delannoy, Nahid Emad, and Serge Peti-ton. Workflow global computing with yml. InThe 7th IEEE/ACM International Conference onGrid Computing, pages 25–32, 2006.

[2] Thomas Dufaud, Miwako Tsuji, and MitsuhisaSato. Design of data management for multi-spmd workflow programming model. In Pro-ceedings of the 4th International Workshop onExtreme Scale Programming Models and Middle-ware, SC18. ACM, 2018.

[3] Miwako Tsuji, Serge Petiton, and MitsuhisaSato. Fault tolerance features of a new multi-spmd programming/execution environment. InProceedings of the First International Work-shop on Extreme Scale Programming Mod-els and Middleware, SC15, pages pp.20–27doi:10.1145/2832241.2832243. ACM, 2015.

[4] Miwako Tsuji, Mitsuhisa Sato, Maxime Hugues,and Serge Petiton. Multiple-SPMD programmingenvironment based on PGAS and workflow to-ward post-petascale computing. In Proceedingsof the 2013 International Conference on ParallelProcessing (ICPP-2013), pages 480–485. IEEE,2013.

[5] XcalableMP. http://www.xcalablemp.org/.

2