| |
 |
| Paper: |
Spatial Indexing and Visualization of Large Multi-Dimensional Databases |
| Volume: |
376, Astronomical Data Analysis Software and Systems XVI |
| Page: |
629 |
| Authors: |
Dobos, L.; Csabai, I.; Trencseni, M.; Herczegh, G.; Jozsa, P.; Purger, N. |
| Abstract: |
Scientific endeavors such as large astronomical surveys generate databases on the terabyte scale. These usually multi-dimensional databases must be visualized and mined in order to find interesting objects or to extract meaningful and qualitatively new relationships. Many statistical algorithms required for these tasks run reasonably fast when operating on small sets of in-memory data, but take noticeable performance hits when operating on large databases that do not fit into memory. We utilize new software technologies to develop and evaluate fast multi-dimensional, spatial indexing schemes that inherently follow the underlying highly non-uniform distribution of the data: one of them is hierarchical binary space partitioning; the other is sampled flat Voronoi partitioning of the data. Our working database is the 5-dimensional magnitude space of the Sloan Digital Sky Survey with more than 250 million data points. We show that these techniques can dramatically speed up data mining operations such as finding similar objects by example, classifying objects or comparing extensive simulation sets with observations. We are also developing tools to interact with the spatial database and visualize the data real-time at multiple resolutions at different zoom levels in an adaptive manner. |
|
|
|
|
|
