TY - GEN
T1 - One optimized I/O configuration per HPC application
T2 - 2nd Asia-Pacific Workshop on Systems, APSys'11
AU - Liu, Mingliang
AU - Zhai, Jidong
AU - Zhai, Yan
AU - Ma, Xiaosong
AU - Chen, Wenguang
PY - 2011
Y1 - 2011
N2 - There is a trend to migrate HPC (High Performance Computing) applications to cloud platforms, such as the Amazon EC2 Cluster Compute Instances (CCIs). While existing research has mainly focused on the performance impact of virtualized environments and interconnect technologies on parallel programs, we suggest that the configurability enabled by clouds is another important dimension to explore. Unlike on traditional HPC platforms, on a cloud-resident virtual cluster it is easy to change the I/O configurations, such as the choice of file systems, the number of I/O nodes, and the types of virtual disks, to fit the I/O requirements of different applications. In this paper, we discuss how cloud platforms can be employed to form customized and balanced I/O subsystems for individual I/O-intensive MPI applications. Through our preliminary evaluation, we demonstrate that different applications will benefit from individually tailored I/O system configurations. For a given I/O-intensive application, different I/O settings may lead to significant overall application performance or cost difference (up to 2.5-fold). Our exploration indicates that customized system configuration for HPC applications in the cloud is important and non-trivial.
AB - There is a trend to migrate HPC (High Performance Computing) applications to cloud platforms, such as the Amazon EC2 Cluster Compute Instances (CCIs). While existing research has mainly focused on the performance impact of virtualized environments and interconnect technologies on parallel programs, we suggest that the configurability enabled by clouds is another important dimension to explore. Unlike on traditional HPC platforms, on a cloud-resident virtual cluster it is easy to change the I/O configurations, such as the choice of file systems, the number of I/O nodes, and the types of virtual disks, to fit the I/O requirements of different applications. In this paper, we discuss how cloud platforms can be employed to form customized and balanced I/O subsystems for individual I/O-intensive MPI applications. Through our preliminary evaluation, we demonstrate that different applications will benefit from individually tailored I/O system configurations. For a given I/O-intensive application, different I/O settings may lead to significant overall application performance or cost difference (up to 2.5-fold). Our exploration indicates that customized system configuration for HPC applications in the cloud is important and non-trivial.
UR - https://www.scopus.com/pages/publications/84863173594
U2 - 10.1145/2103799.2103818
DO - 10.1145/2103799.2103818
M3 - Conference contribution
AN - SCOPUS:84863173594
SN - 9781450311793
T3 - Proceedings of the 2nd Asia-Pacific Workshop on Systems, APSys'11
BT - Proceedings of the 2nd Asia-Pacific Workshop on Systems, APSys'11
Y2 - 11 July 2011 through 12 July 2011
ER -