| |
 |
| Paper: |
No-SQL Databases: An Efficient Way to Store and Query Heterogeneous Astronomical Data in DACE |
| Volume: |
523, Astronomical Data Analysis Software and Systems XXVIII |
| Page: |
405 |
| Authors: |
Buchschacher, N.; Alesina, F.; Burnier, J. |
| Abstract: |
Data production is growing every day in all domains. Astronomy is particularly concerned with the recent instruments. While SQL databases have proven their performances for decades and still performs in many cases, it is sometimes difficult to store, analyze and combine data produced by different instruments which do not necessarily use the same data model. This is where No-SQL databases can help to solve our requirements: how to efficiently store heterogeneous data in a common infrastructure ?
SQL database management systems can do a lot of powerful operations like filtering, relation between tables, sub-queries etc. The storage is vertically scalable by adding more rows in the tables but the schema has to be very well defined. In the opposite, No-SQL databases are not restrictive. The scalability is horizontal by adding more shards (nodes) and the different storage engines have been designed to easily modify the structure. This is why it is well suited in the big data era.
DACE (Data and Analysis Center for Exoplanets) is a web platform which facilitates data analysis and visualization for the exoplanet research domain. We are collecting a lot of data from different instruments and we regularly need to adapt our database to accept new data sets with different models. We recently decided to opt for NoSQL databases like Cassandra and Solr. This recent change accelerated our queries and we are now ready to accept new data sets from future instruments and combine them with older data to do better science.
DACE is funded by the Swiss National Centre of Competence in Research (NCCR) PlanetS project, federating the Swiss expertise in exoplanet research. |
|
|
|
|
|
