Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere

Jieyang Chen; Qiang Guan; Xin Liang; Paul Bryant; Patricia Grubel; Allen McPherson; Li Ta Lo; Timothy Randles; Zizhong Chen; James Paul Ahrens

doi:10.1109/BigData.2018.8622572

Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere

Jieyang Chen
, Qiang Guan
, Xin Liang
, Paul Bryant
, Patricia Grubel
, Allen McPherson
, Li Ta Lo
, Timothy Randles
, Zizhong Chen
, James Paul Ahrens

Producción científica: Conference contribution › revisión exhaustiva

7 Citas (Scopus)

Resumen

Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.

Idioma original	English
Título de la publicación alojada	Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018
Editores	Naoki Abe, Huan Liu, Calton Pu, Xiaohua Hu, Nesreen Ahmed, Mu Qiao, Yang Song, Donald Kossmann, Bing Liu, Kisung Lee, Jiliang Tang, Jingrui He, Jeffrey Saltz
Páginas	1737-1746
Número de páginas	10
ISBN (versión digital)	9781538650356
DOI	https://doi.org/10.1109/BigData.2018.8622572
Estado	Published - jul 2 2018
Evento	2018 IEEE International Conference on Big Data, Big Data 2018 - Seattle, United States Duración: dic 10 2018 → dic 13 2018

Serie de la publicación

Nombre	Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018

Conference

Conference	2018 IEEE International Conference on Big Data, Big Data 2018
País/Territorio	United States
Ciudad	Seattle
Período	12/10/18 → 12/13/18

Nota bibliográfica

Publisher Copyright:
© 2018 IEEE.

Financiación

This work was funded by the the US Government contract DE-AC52-06NA25396 for Los Alamos National Laboratory, operated by Los Alamos National Security, LLC, for the US Department of Energy. This work was also supported by NSF Award No. 1513201. Results presented in this paper were obtained using the Chameleon Cloud sponsored by the National Science Foundation. The publication has been assigned the LANL identifier LA-UR-18-27912.

Financiadores	Número del financiador
US Department of Energy
US Department of Veterans Affairs or the US government	DE-AC52-06NA25396
U.S. Department of Energy Chinese Academy of Sciences Guangzhou Municipal Science and Technology Project Oak Ridge National Laboratory Extreme Science and Engineering Discovery Environment National Science Foundation National Energy Research Scientific Computing Center National Natural Science Foundation of China	1513201
U.S. Department of Energy Chinese Academy of Sciences Guangzhou Municipal Science and Technology Project Oak Ridge National Laboratory Extreme Science and Engineering Discovery Environment National Science Foundation National Energy Research Scientific Computing Center National Natural Science Foundation of China

ASJC Scopus subject areas

Computer Science Applications
Information Systems

Acceder al documento

10.1109/BigData.2018.8622572

Otros archivos y enlaces

Citar esto

Chen, J., Guan, Q., Liang, X., Bryant, P., Grubel, P., McPherson, A., Lo, L. T., Randles, T., Chen, Z., & Ahrens, J. P. (2018). Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. En N. Abe, H. Liu, C. Pu, X. Hu, N. Ahmed, M. Qiao, Y. Song, D. Kossmann, B. Liu, K. Lee, J. Tang, J. He, & J. Saltz (Eds.), Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018 (pp. 1737-1746). Artículo 8622572 (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018). https://doi.org/10.1109/BigData.2018.8622572

Chen, Jieyang ; Guan, Qiang ; Liang, Xin et al. / Build and Execution Environment (BEE) : An Encapsulated Environment Enabling HPC Applications Running Everywhere. Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018. editor / Naoki Abe ; Huan Liu ; Calton Pu ; Xiaohua Hu ; Nesreen Ahmed ; Mu Qiao ; Yang Song ; Donald Kossmann ; Bing Liu ; Kisung Lee ; Jiliang Tang ; Jingrui He ; Jeffrey Saltz. 2018. pp. 1737-1746 (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018).

@inproceedings{17a443089b644fcdb8eb133745c5f1c8,

title = "Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere",

abstract = "Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.",

keywords = "cloud computing, container, high performance computing",

author = "Jieyang Chen and Qiang Guan and Xin Liang and Paul Bryant and Patricia Grubel and Allen McPherson and Lo, \{Li Ta\} and Timothy Randles and Zizhong Chen and Ahrens, \{James Paul\}",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 IEEE International Conference on Big Data, Big Data 2018 ; Conference date: 10-12-2018 Through 13-12-2018",

year = "2018",

month = jul,

day = "2",

doi = "10.1109/BigData.2018.8622572",

language = "English",

series = "Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018",

pages = "1737--1746",

editor = "Naoki Abe and Huan Liu and Calton Pu and Xiaohua Hu and Nesreen Ahmed and Mu Qiao and Yang Song and Donald Kossmann and Bing Liu and Kisung Lee and Jiliang Tang and Jingrui He and Jeffrey Saltz",

booktitle = "Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018",

}

Chen, J, Guan, Q, Liang, X, Bryant, P, Grubel, P, McPherson, A, Lo, LT, Randles, T, Chen, Z & Ahrens, JP 2018, Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. En N Abe, H Liu, C Pu, X Hu, N Ahmed, M Qiao, Y Song, D Kossmann, B Liu, K Lee, J Tang, J He & J Saltz (eds.), Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018., 8622572, Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018, pp. 1737-1746, 2018 IEEE International Conference on Big Data, Big Data 2018, Seattle, United States, 12/10/18. https://doi.org/10.1109/BigData.2018.8622572

Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. / Chen, Jieyang; Guan, Qiang; Liang, Xin et al.
Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018. ed. / Naoki Abe; Huan Liu; Calton Pu; Xiaohua Hu; Nesreen Ahmed; Mu Qiao; Yang Song; Donald Kossmann; Bing Liu; Kisung Lee; Jiliang Tang; Jingrui He; Jeffrey Saltz. 2018. p. 1737-1746 8622572 (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018).

Producción científica: Conference contribution › revisión exhaustiva

TY - GEN

T1 - Build and Execution Environment (BEE)

T2 - 2018 IEEE International Conference on Big Data, Big Data 2018

AU - Chen, Jieyang

AU - Guan, Qiang

AU - Liang, Xin

AU - Bryant, Paul

AU - Grubel, Patricia

AU - McPherson, Allen

AU - Lo, Li Ta

AU - Randles, Timothy

AU - Chen, Zizhong

AU - Ahrens, James Paul

PY - 2018/7/2

Y1 - 2018/7/2

N2 - Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.

AB - Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Linux container technologies such as Docker and Charliecloud bring great benefits to the application development, build and deployment processes. While cloud platforms already widely support containers, HPC systems still have non-uniform support of container technologies. In this work, we propose a unified runtime framework - Build and Execution Environment (BEE) across both HPC and cloud platforms that allows users to run their containerized HPC applications across all supported platforms without modification. We design four BEE backends for four different classes of HPC or cloud platform so that together they cover the majority of mainstream computing platforms for HPC users. Evaluations show that BEE provides an easy-to-use unified user interface, execution environment, and comparable performance.

KW - cloud computing

KW - container

KW - high performance computing

UR - https://www.scopus.com/pages/publications/85062637378

UR - https://www.scopus.com/pages/publications/85062637378#tab=citedBy

U2 - 10.1109/BigData.2018.8622572

DO - 10.1109/BigData.2018.8622572

M3 - Conference contribution

AN - SCOPUS:85062637378

T3 - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018

SP - 1737

EP - 1746

BT - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018

A2 - Abe, Naoki

A2 - Liu, Huan

A2 - Pu, Calton

A2 - Hu, Xiaohua

A2 - Ahmed, Nesreen

A2 - Qiao, Mu

A2 - Song, Yang

A2 - Kossmann, Donald

A2 - Liu, Bing

A2 - Lee, Kisung

A2 - Tang, Jiliang

A2 - He, Jingrui

A2 - Saltz, Jeffrey

Y2 - 10 December 2018 through 13 December 2018

ER -

Chen J, Guan Q, Liang X, Bryant P, Grubel P, McPherson A et al. Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere. En Abe N, Liu H, Pu C, Hu X, Ahmed N, Qiao M, Song Y, Kossmann D, Liu B, Lee K, Tang J, He J, Saltz J, editores, Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018. 2018. p. 1737-1746. 8622572. (Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018). doi: 10.1109/BigData.2018.8622572

Build and Execution Environment (BEE): An Encapsulated Environment Enabling HPC Applications Running Everywhere

Resumen

Serie de la publicación

Conference

Nota bibliográfica

Financiación

ASJC Scopus subject areas

Acceder al documento

Otros archivos y enlaces

Huella

Citar esto