| |
 |
| Paper: |
A Scalable Transient Detection Pipeline for the Australian SKA Pathfinder VAST Survey |
| Volume: |
532, ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS XXX |
| Page: |
333 |
| Authors: |
Pintaldi, S.; Stewart, A.; O'Brien, A.; Kaplan, D.; Murphy, T. |
| Abstract: |
The Australian Square Kilometre Array Pathfinder (ASKAP) collects images of the sky at radio wavelengths with an unprecedented field of view, combined with a high resolution and sub-millijansky sensitivities. The large quantity of data produced is used by the ASKAP Survey for Variables and Slow Transients (VAST) to study the dynamic radio sky, primarily in the image domain. Efficient pipelines are vital in such research, where searches often form a ‘needle in a haystack' type of problem to solve. However, the existing pipelines developed among the radio-transient community are not suitable for the scale of ASKAP datasets.
We have developed the “Vast Pipeline”: a modern and scalable Python-based data pipeline for transient searches, using up-to-date dependencies and methods. The pipeline allows source association to be performed at scale using the Pandas dataframe interface and the well-known Astropy crossmatch functions. The Dask Python framework is used to parallelise operations as well as scale them both vertically and horizontally, by means of a cluster of workers.
A modern web interface for data exploration and querying has also been developed using the latest Django web framework combined with Bootstrap.
In this paper we will give an overview of the pipeline features and architecture. |
|
|
|
|
|
