| |
 |
| Paper: |
JOVIAL: Jupyter OVerrIde for Astronomical Libraries |
| Volume: |
521, Astronomical Data Analysis Software and Systems XXVI |
| Page: |
475 |
| Authors: |
Araya, M.; Valenzuela, C.; Far'ias, H.; Solar, a. M. |
| Abstract: |
Jupyter notebooks have proven to be a powerful tool for teaching science,
because they handle executable code snippets, context environments,
visualization, equations and explanatory text, all within the same
web-interface. We propose extending its usage from teaching activities to
science activities through the integration of astronomical libraries to the
Jupyter environment and its execution as a cloud service. The remote execution
of processing tasks through notebooks not only simplifies the usage of
libraries, but enables Big Data processing on data centers while hiding the
infrastructure and platform-dependent details. This allows moving computations
closer to the data archives, executing tasks in a graphical yet non-blocking
fashion, using high-performance computing routines transparently and working
with very large data files without exhausting local memory. Moreover, a proper
use of the notebooks produces self-documented, exportable and reproducible
pipelines with graphical support. Our prototype uses Jupyter Hub as the base
service, including Astropy, ACALib and CASAC libraries to the Python kernel, and
we are currently porting CUPID (Starlink), MPICASA and ADMIT through suitable
wrappers. The integration with the VO is two-fold: through the data access
layer services via VOTables, and through the application layer via the SAMP
protocol. Besides including more libraries, the next steps are exploring the
integration with Big Data frameworks such as Apache Spark, and the proper
parallelization of the main algorithms of the libraries. To encourage its usage
we also plan to assemble a lite distribution of JOVIAL to be used off-line. |
|
|
|
|
|
