| |
 |
| Paper: |
An Original Way to Manage Huge Astronomical Tables |
| Volume: |
541, ADASS XXXIII |
| Page: |
465 |
| Authors: |
G. Landais; F. Pineau; P. Ocvirk; A. Vizier-Fontanesi; A. Lepicier; A. Vanhulle |
| DOI: |
10.26624/JLNJ1155 |
| Abstract: |
We explored a way to wrap in database large astronomical tables stored
in a remote repository and in a suitable format. The architecture is based on ATSS (Astronomical Table Serialisation System) which exploits binary file allowing a powerful
position indexation well suited to intensive use. This binary format is used in the VizieR
catalogue service which duplicates large tables in ATSS and in a PostgreSQL database
to provide an efficient conesearch service and the SQL flexibility needed by the IVOA
Table Access Protocol (TAP). The database architecture requires an important capacity
to deal with datasets exceeding already the billion of records (Gaia, SDSS, GSC, etc).
Some new surveys such as LSST will drive a significant increase in volume which may
require a redesign of the architecture. We explored a cost-effective solution in term of
funds and maintenance that consists in wrapping ATSS binary data as foreign table. We
developed a PostgreSQL extension using the Foreign-Data-Wrapper that maps SQL
with the API of the binary file including its cone search and indexation capabilities.
The solution has been tested with a selection of large tables in a TAP service, we will
discuss the technology, its potential but also the limitation of the current development. |
|
|
|
|
|
