| |
 |
| Paper: |
QuartiCal - embarrassingly parallel calibration using Numba and Dask |
| Volume: |
535, Astronomical Data Analysis Software and Systems XXXI |
| Page: |
411 |
| Authors: |
Kenyon, J.; Perkins, S.; Smirnov, O. |
| Abstract: |
Existing radio interferometers, such as the MeerKAT and LOFAR, already produce unprecedented quantities of visibility data, and even they will soon be dwarfed by the SKA and ngVLA. Despite the modernity of these instruments, they still require extensive calibration in order to correct various science-limiting effects. Thus, calibration is and will remain an integral part of radio interferometric data reduction. This has motivated the development of QuartiCal, a Python application that leverages contemporary packages to make calibration scalable, distributable and fast. The first such package is Dask: a library for parallel and distributed computing with Python. The second package is Numba. Numba is a just-in-time compiler for a subset of Python/NumPy that can provide C-like speed without forfeiting the expressiveness and dynamism of Python. QuartiCal uses these packages to convincingly outperform its predecessor, CubiCal, in terms of both wall time and memory footprint. Finally, whilst testing QuartiCal, we have found that the Measurement Set (backed by the Casacore Table Data System) may be a limiting factor when distributing radio interferometric algorithms. |
|
|
|
|
|
