Back to Volume
Paper: Access to Massive Catalogs in the Gaia Archive: a New Paradigm
Volume: 512, Astronomical Data Analysis Software and Systems XXV
Page: 293
Authors: Salgado, J.; González, J.; Gutiérrez, R.; Segovia, J. C.; Nieto, S.; Arviset, C.; Merín, B.
Abstract: New astronomical missions have reinforced the change on the development of archives. Archives, as simple applications to access the data, are evolving into complex data center structures where computing power services are available for users and data mining tools are integrated into the server side. In the case of astronomy science that involves the use of big catalogs, as in Gaia or Euclid, the common ways to work on the data need to be changed to a new paradigm “move code close to the data”, which implies that data mining functionalities are becoming a must to allow the science exploitation. Some massive operations like crossmatch between catalogs, integration of big queries into workflows by serialization of intermediate results in cloud resources like VOSpace, integration of data mining tools in virtualized environments, etc are being integrated into the ESAC Gaia archive. Also, totally new science use cases, e.g. asteroids discovered by Gaia, combine astronomy with small bodies science and the results should be made available in a transparent way to different communities, allowing filtering, clustering and, in general, integrated data mining techniques. We present the tools already available in the Gaia archive for big catalogs manipulation, like crossmatch operations, pre-cooked crossmatch tables with the main astronomical catalogs, and the ongoing work on the publication of a huge variety of data objets, e.g. asteroids, and other add-ons that would allow scientists a new way to produce science.
Back to Volume