| |
 |
| Paper: |
ML and Next Steps in the DRAO Data Handling Pipelines |
| Volume: |
541, ADASS XXXIII |
| Page: |
226 |
| Authors: |
Dustin Lagoy; Nicholas Bruce; David A. Del Rizzo; Stephen T. Harrison |
| DOI: |
10.26624/IWZB9197 |
| Abstract: |
The Dominion Radio Astrophysical Observatory (DRAO) has several
telescopes whose radio frequency and digital subsystems are currently undergoing (or
have recently completed) major upgrades, enabling new science via expanded capabilities in bandwidth and frequency resolution. Driven by science observation requirements
and the radio frequency interference (RFI) environment each telescope will produce upwards of 400 MBps of spectral data during long-running observations (on the order of
petabytes every year). This high volume of data requires new infrastructure and techniques in the pre-processing, archiving and distributing pipeline as well as re-thinking
some existing paradigms. Before sending data off-site for archiving and distribution,
we aim to perform real-time data-reduction by automatically removing RFI using a
machine-learning (ML) based spectral kurtosis estimator. This new approach will both
significantly reduce the volume of archived data and reduce the effort of individual scientists in removing the RFI by hand. Here we discuss the current status of the data
pipeline and its ongoing development. |
|
|
|
|
|
