| |
 |
| Paper: |
Gearing the DECam Analysis Pipeline for Multi–Messenger Astronomy Using Pegasus Workflows |
| Volume: |
527, Astronomical Data Analysis Software and Systems XXIX |
| Page: |
631 |
| Authors: |
Vahi, K.; Goldstein, D.; Papadimitriou, G.; Nugent, P.; Deelman., E. |
| Abstract: |
In the era of multi–messenger astronomy it has become paramount to verify gravitational waves (GWs) discoveries with their electromagnetic (EM) optical counterparts in near real-time. With these goals in mind, we have developed the Dark Energy Camera (DECam) analysis pipeline that sifts through large amounts of data and tries to identify interesting optical candidates. The pipeline is setup to run nightly at NERSC supercomputers and compares current sky data against the historical data collected from the Victor Blanco telescope + DECam imager archived at NOAO or at NERSC, the latter as part of the DECam Legacy Survey. Depending on the region of sky being analyzed, images may be needed to be downloaded onto the NERSC shared filesystem on demand from NOAO. This access pattern can result in requesting the same input datasets repeatedly, increasing the traffic on NOAO's servers. In order to streamline access, we have archived much of the dataset (≈500TB) on NERSC's tape storage system, accessible via HPSS. Additionally we are porting the pipeline to Pegasus WMS to take advantage of advanced data management techniques and the ability to use additional computational resources on demand, such as the Open Science Grid or local clusters, when an alert is issued. Pegasus allows us to describe our pipeline in a resource agnostic manner, generates an executable workflow with additional data management tasks that retrieve background FITS files from tape, deploys application containers, automatically remove data no longer required and stage–out generated outputs. |
|
|
|
|
|
