| |
 |
| Paper: |
Batch Query System with Interactive Local Storage for SDSS and the VO |
| Volume: |
314, Astronomical Data Analysis Software and Systems XIII |
| Page: |
372 |
| Authors: |
O'Mullane, W.; Gray, J.; Li, N.; Budavari, T.; Nieto-Santisteban, M.A.; Szalay, A.S. |
| Abstract: |
The Sloan Digital Sky Survey science database is approaching 1TB in size. While the vast majority of queries normally execute in seconds or minutes, this prompt execution time can be disproportionately increased by a small fraction of queries that take hours or days to run either because they require non-index scans of the largest tables or because they request very large result sets. In response to this, a job submission and tracking system has been developed with multiple queues. The transfer of very large result sets from queries over the network is another serious problem. Statistics suggested that much of this data transfer is unnecessary; users would prefer to store results locally in order to allow further cross matching and filtering. To allow local analysis, a system was developed that gives users their own personal database (MYDB) at the portal site. Users may transfer data to their MYDB, then perform further analysis before extracting it to their own machine. |
|
|
|
|
|
