| |
 |
| Paper: |
Docker-based Implementation for an Astronomical Data Analysis Cloud Service |
| Volume: |
522, Astronomical Data Analysis Software and Systems XXVII |
| Page: |
323 |
| Authors: |
Diaz, M.; Araya, M.; Jauregui, C.; Valenzuela, C.; Pizarro, L.; Osorio, M.; Solar, M. |
| Abstract: |
The data deluge problem that astronomy research confronts not only require developing new algorithms and computing infrastructure, but also fostering modern software and services.
For that matter, the cloud computing paradigm allows to perform the processing of the data where it is, minimizing the transfer to the end-user and taking advantage of the high-performance computing infrastructure of the datacenters.
However, new challenges arise when these services are deployed under the high-availability principle in order to compete with the convenience of local processing.
Between them we found multi-user support, load balancing, resource usage optimization, securing data and network security.
In this paper we report the architecture and caveats of the deployment of JOVIAL: a notebook-based cloud service specifically designed for astronomy using the JupyterHub platform.
This platform allows users to run arbitrary code, which is is at the same time its main advantage and menace.
Therefore, we used Docker containers to spawn a Jupyter server on demand for each user, which increased both the protection of the user data and the protection of the hosts from users.
Following the high-availability principle, the containers have to be orchestrated making good use of the high-performance infrastructure.
For achieving this, we combined our user-based docker containers with the Kubernetes system, providing on-demand growth and load balancing between the hosts.
We are currently exploring supporting high-availability of the storage system through LustreFS, and managing the whole infrastructure with Rancher, a software that provides user-friendly orchestration of docker within a Kubernetes deployment. |
|
|
|
|
|
