Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere

Jieyang Chen; Qiang Guan; Xin Liang; Paul Bryant; Patricia Grubel; Allen McPherson; Li Ta Lo; Timothy Randles; Zizhong Chen; James Paul Ahrens

doi:10.1109/BigData.2018.8622572

Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere

Jieyang Chen, Qiang Guan, Xin Liang, Paul Bryant, Patricia Grubel, Allen McPherson, Li Ta Lo, Timothy Randles, Zizhong Chen, James Paul Ahrens

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.

Original language	English
Title of host publication	Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018
Editors	Yang Song, Bing Liu, Kisung Lee, Naoki Abe, Calton Pu, Mu Qiao, Nesreen Ahmed, Donald Kossmann, Jeffrey Saltz, Jiliang Tang, Jingrui He, Huan Liu, Xiaohua Hu
Pages	1737-1746
Number of pages	10
ISBN (Electronic)	9781538650356
DOIs	https://doi.org/10.1109/BigData.2018.8622572
State	Published - Jan 22 2019
Event	2018 IEEE International Conference on Big Data, Big Data 2018 - Seattle, United States Duration: Dec 10 2018 → Dec 13 2018

Publication series

Name	Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018

Conference

Conference	2018 IEEE International Conference on Big Data, Big Data 2018
Country/Territory	United States
City	Seattle
Period	12/10/18 → 12/13/18

Bibliographical note

Funding Information:
This work was funded by the the US Government contract DE-AC52-06NA25396 for Los Alamos National Laboratory, operated by Los Alamos National Security, LLC, for the US Department of Energy. This work was also supported by NSF Award No. 1513201. Results presented in this paper were obtained using the Chameleon Cloud sponsored by the National Science Foundation. The publication has been assigned the LANL identifier LA-UR-18-27912.

Publisher Copyright:
© 2018 IEEE.

Keywords

cloud computing
container
high performance computing

ASJC Scopus subject areas

Computer Science Applications
Information Systems

Access to Document

10.1109/BigData.2018.8622572

Cite this

Chen, J., Guan, Q., Liang, X., Bryant, P., Grubel, P., McPherson, A., Lo, L. T., Randles, T., Chen, Z., & Ahrens, J. P. (2019). Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. In Y. Song, B. Liu, K. Lee, N. Abe, C. Pu, M. Qiao, N. Ahmed, D. Kossmann, J. Saltz, J. Tang, J. He, H. Liu, & X. Hu (Eds.), Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018 (pp. 1737-1746). Article 8622572 (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018). https://doi.org/10.1109/BigData.2018.8622572

Chen, Jieyang ; Guan, Qiang ; Liang, Xin et al. / Build and Execution Environment (BEE) : An Encapsulated Environment Enabling HPC Applications Running Everywhere. Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018. editor / Yang Song ; Bing Liu ; Kisung Lee ; Naoki Abe ; Calton Pu ; Mu Qiao ; Nesreen Ahmed ; Donald Kossmann ; Jeffrey Saltz ; Jiliang Tang ; Jingrui He ; Huan Liu ; Xiaohua Hu. 2019. pp. 1737-1746 (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018).

@inproceedings{17a443089b644fcdb8eb133745c5f1c8,

title = "Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere",

abstract = "Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.",

keywords = "cloud computing, container, high performance computing",

author = "Jieyang Chen and Qiang Guan and Xin Liang and Paul Bryant and Patricia Grubel and Allen McPherson and Lo, {Li Ta} and Timothy Randles and Zizhong Chen and Ahrens, {James Paul}",

note = "Funding Information: This work was funded by the the US Government contract DE-AC52-06NA25396 for Los Alamos National Laboratory, operated by Los Alamos National Security, LLC, for the US Department of Energy. This work was also supported by NSF Award No. 1513201. Results presented in this paper were obtained using the Chameleon Cloud sponsored by the National Science Foundation. The publication has been assigned the LANL identifier LA-UR-18-27912. Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 IEEE International Conference on Big Data, Big Data 2018 ; Conference date: 10-12-2018 Through 13-12-2018",

year = "2019",

month = jan,

day = "22",

doi = "10.1109/BigData.2018.8622572",

language = "English",

series = "Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018",

pages = "1737--1746",

editor = "Yang Song and Bing Liu and Kisung Lee and Naoki Abe and Calton Pu and Mu Qiao and Nesreen Ahmed and Donald Kossmann and Jeffrey Saltz and Jiliang Tang and Jingrui He and Huan Liu and Xiaohua Hu",

booktitle = "Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018",

}

Chen, J, Guan, Q, Liang, X, Bryant, P, Grubel, P, McPherson, A, Lo, LT, Randles, T, Chen, Z & Ahrens, JP 2019, Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. in Y Song, B Liu, K Lee, N Abe, C Pu, M Qiao, N Ahmed, D Kossmann, J Saltz, J Tang, J He, H Liu & X Hu (eds), Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018., 8622572, Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018, pp. 1737-1746, 2018 IEEE International Conference on Big Data, Big Data 2018, Seattle, United States, 12/10/18. https://doi.org/10.1109/BigData.2018.8622572

Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. / Chen, Jieyang; Guan, Qiang; Liang, Xin et al.
Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018. ed. / Yang Song; Bing Liu; Kisung Lee; Naoki Abe; Calton Pu; Mu Qiao; Nesreen Ahmed; Donald Kossmann; Jeffrey Saltz; Jiliang Tang; Jingrui He; Huan Liu; Xiaohua Hu. 2019. p. 1737-1746 8622572 (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Build and Execution Environment (BEE)

T2 - 2018 IEEE International Conference on Big Data, Big Data 2018

AU - Chen, Jieyang

AU - Guan, Qiang

AU - Liang, Xin

AU - Bryant, Paul

AU - Grubel, Patricia

AU - McPherson, Allen

AU - Lo, Li Ta

AU - Randles, Timothy

AU - Chen, Zizhong

AU - Ahrens, James Paul

N1 - Funding Information: This work was funded by the the US Government contract DE-AC52-06NA25396 for Los Alamos National Laboratory, operated by Los Alamos National Security, LLC, for the US Department of Energy. This work was also supported by NSF Award No. 1513201. Results presented in this paper were obtained using the Chameleon Cloud sponsored by the National Science Foundation. The publication has been assigned the LANL identifier LA-UR-18-27912. Publisher Copyright: © 2018 IEEE.

PY - 2019/1/22

Y1 - 2019/1/22

N2 - Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.

AB - Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.

KW - cloud computing

KW - container

KW - high performance computing

UR - http://www.scopus.com/inward/record.url?scp=85062637378&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062637378&partnerID=8YFLogxK

U2 - 10.1109/BigData.2018.8622572

DO - 10.1109/BigData.2018.8622572

M3 - Conference contribution

AN - SCOPUS:85062637378

T3 - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018

SP - 1737

EP - 1746

BT - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018

A2 - Song, Yang

A2 - Liu, Bing

A2 - Lee, Kisung

A2 - Abe, Naoki

A2 - Pu, Calton

A2 - Qiao, Mu

A2 - Ahmed, Nesreen

A2 - Kossmann, Donald

A2 - Saltz, Jeffrey

A2 - Tang, Jiliang

A2 - He, Jingrui

A2 - Liu, Huan

A2 - Hu, Xiaohua

Y2 - 10 December 2018 through 13 December 2018

ER -

Chen J, Guan Q, Liang X, Bryant P, Grubel P, McPherson A et al. Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. In Song Y, Liu B, Lee K, Abe N, Pu C, Qiao M, Ahmed N, Kossmann D, Saltz J, Tang J, He J, Liu H, Hu X, editors, Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018. 2019. p. 1737-1746. 8622572. (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018). doi: 10.1109/BigData.2018.8622572

Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this