A. Szalay, P. Kunszt, A. Thakar, J. Gray, and D. Slutz. Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey. In SIGMOD  Home/Search   Document Details and Download   Summary   Related Articles   Check

....data is common in scientific and medical databases as well. For example, the Sloan Digital Sky Survey (SDSS) astronomy database will be storing 200 million objects (galaxies, stars and quasars) with mostly numeric attributes (e.g. position, color, shape etc. defining a 100 dimensional space [140]. Astronomers would then run spatial proximity queries, similarity queries, multidimensional range queries etc. on the high dimensional, multi terabyte database. In the medical area, multidimensional features are extracted from medical data (e.g. tumor images in [82] ECG data in [78] which can ....

.... spaces that arise in modern database applications like multimedia retrieval (e.g. 64 d color histograms) data mining OLAP (e.g. 52 d bank data in clustering [2] and time series scientific medical applications (e.g. 20 d feature vectors extracted from Space Shuttle data [79] 100 d SDSS data [140], 64 dimensional ECG data [78] A simple sequential scan through the entire dataset to answer the query is often faster than accessing the data using a spatial access method [15, 16] Figure 1.2 shows the cost of answering a range query on a 70,000 item color histogram database using R tree and ....

[Article contains additional citation context not shown here]

A. Szalay, P. Kunszt, A. Thakar, and J. Gray. Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey. Proc. of SIGMOD, 2000.

....Endowment. Proceedings of the 29th VLDB Conference, Berlin, Germany, 2003 image processing, CAD and VLSI. In the last 20 years, spatial indexing and searching have drawn a great deal of attention from research society, and a number of techniques have been developed [6] With the availability [9, 20, 21] of a huge collection of on line spatial data, there are strong demands for e#ective techniques to support e# cient browsing of large datasets to summarise spatial characteristics. It becomes extremely important in large digital libraries archives to support interactive queries by query preview ....

....best way to partition the object space is not within the scope of the paper. It should be mentioned that the Euler histogram techniques are not only applicable to estimating spatial range query results but may also be immediately applicable to spatial digital libraries to support window browsers [3, 9, 20, 21]. Further, our results are also fundamental to the development of new selectivity estimation techniques in spatial joins with the join predicates, such as contains, intersection, cross over, etc. 6 Performance Evaluation In this section we evaluate the performance of our new techniques, MESA, ....

A.S. Szalay, P. Kunzst, A. Thakar, J. Gray, D. Slutz, and R.J. Brunner. Designing and mining multiterabyte astronomy archieves: The sloan digital sky survey. In SIGMOD 2000.

....Survey (SDSS) database [10] All SDSS data is stored in Objectivity DB and can be retrieved by a special SQL like query interface developed by the astronomers. Our main challenge was to show that bitmap indices can even further improve the already optimised astronomy queries against the database [11]. The astronomy data consists of 6,182,527 objects with 65 attributes each. The total size is 1.9 GB. We created 65 bitmap indices with 100 equi width bins each. The total size of all indices is 5.8 GB. We evaluated the performance of the bitmap index against queries taken from the SDSS query ....

A.Szalay, P. Kunszt, A. Thakar, J. Gray, D. Slutz, Designing and Mining MultiTerabyte Astronomy Archives: The Sloan Digital Sky Survey, Proceedings ACM SIGMOD International Conference on Management of Data, Philadephia, Pennsylvania, USA, June 1999.

....files depending on the desirable degree of flexibility and e#ciency. 4 A Multi Layer Map Warehouse Online generation of geographical, astronomical or biomedical maps generally can consume a large amount of processing and bandwidth resources depending on the size of the datasets involved [5, 10, 30]. In the past, the problem of handling terabyte sized datasets has been kept manageable by i) limiting the accessible number of layers to one, and ii) producing raster images of all the supported resolutions o#ine so that map rendering would be reduced to data retrieval. In that sense, large scale ....

....and archives [6] A centralized catalog allows accessing the metadata through a standard relational schema. The SDSS Science Archive provides access to astronomical object information via a three tier architecture consisting of a user interface, an intelligent query engine, and a data warehouse [30]. A diverse set of I O intensive queries demonstrate several challenges involved in scanning, classifying, or processing in a streamlike fashion the data using scalable clusterbased infrastructure. Extensible File Systems. Heidemann and Popek proposed stackable filing as a layered structure for ....

Szalay, A. S., Kunszt, P. Z., Thakar, A., Gray, J., Slutz, D., and Brunner, R. J. Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey. In ACM SIGMOD (Dallas, TX, May 2000), pp. 451--462.

....effective facilities for file management and archiving. However, database management systems provide many facilities that are recognised as being impor tant to Grid environments, both for managing Grid metadata (e.g. 3, 11] and for supporting the storage and analysis of application data (e.g. [20]) In any distributed environment there are inevitably multiple related data resources, which, for example, provide complementary or alternative capabilities. Where there is more than one database supported within a distributed environment, it is straightforward to envisage higher level services ....

A. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, and D. R. Slut. Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey. In Proc. ACM SIGMOD, pages 451-462. ACM Press, 2000.

....effective facilities for file management and archiving. However, database management systems provide many facilities that are recognised as being impor tant to Grid environments, both for managing Grid metadata (e.g. 3] and for supporting the storage and analysis of application data (e.g. [18]) In any distributed environment there are inevitably multiple related data resources, which, for example, provide complementary or alternative capabilities. Where there is more than one database supported within a distributed environment, it is straightforward to envisage higher level services ....

A. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, and D. R. Slut. Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey. In Proc. ACM SIGMOD, pages 451-462. ACM Press, 2000.

....place e ective facilities for le management and archiving. However, database management systems provide many facilities that are recognised as being important to Grid environments, both for managing Grid metadata (e.g. 3] and for supporting the storage and analysis of application data (e.g. [18]) In any distributed environment there are inevitably multiple related data resources, which, for example, provide complementary or alternative capabilities. Where there is more than one database supported within a distributed environment, it is straightforward to envisage higher level services ....

A. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, and D. R. Slut. Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey. In Proc. ACM SIGMOD, pages 451-462. ACM Press, 2000.

....the search space. In the paper we describe how this approach reduces from hours to seconds the time needed to generate meaningful visualizations of millions of tuples. 1 Introduction Scientific databases storing continuous observations of natural phenomena will soon store petabytes of data [26, 11]. Finding relevant data points is a major chal lenge in these systems. We have faced this problem during the development of HEDC (the HESSI Experimental Data Center) a data repository that will store the observations of the recently launched HESSI satel lite (High Energy Solar Spectroscopic ....

A. S. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, D. Slutz, and R. J. Brunner. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. In Proc. of the 19th Int. Conference on Management of Data, Dallas, USA, pp. 451 462, May 16 18 2000.

....that these numbers do not necessarily depict the best case scenario for DSM. In an environment having relations with large number of attributes the speed up factors could be much more substantial. For example, one of the key tables used for the Sloan Digital Sky Survey has over 400 attributes [18]. 3.4 Synchronising the mirrors Once the mirrors have been created, they have to be kept synchronised through the course of database operations such as inserts, updates and deletes. In traditional mirroring, all such operations are applied directly to both the copies, which is not feasible with ....

A.Szalay et al. Designing and mining multi-terabyte astronomy archives: The Digital Sky Survey. Proceedings of ACM SIGMOD 2000.

....[5] A request to retrieve data on a particular cluster might thus either lead to the retrieval of the requested data from a local or remote database or the scheduling of a computation to produce the data. One of GriPhyN s scientific collaboration partners is the Sloan Digital Sky Survey (SDSS) [6, 7, 25] , a digital imaging survey that will, by the end of 2005, have mapped a quarter of the sky in five colors with a sensitivity two orders of magnitudes greater than previous large sky surveys. The data of the SDSS is being made available online as both a large collection ( 10 TB) of images and a ....

A. S. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, D. Slutz, and R. J. Brunner, "Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey," SIGMOD Record , vol. 29, pp. 451-462, 2000.

....loosely organized collections of files from automated sensors. Astronomical data include spectral measurements, and high dimensional photographic objects with several hundred attributes. The Sloan Digital Sky Survey will collect several terabytes per year to build a searchable research database [30]. Various NASA Earth observing missions have also collected over 250TB of high resolution image data at different wavelengths, with observations arriving in a steady stream. TerraServer s snapshot of the USA alone includes 15TB of raw data. It is estimated that Google s collection of Web data ....

A. S. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, D. Slutz, and R. J. Brunner. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. In W. Chen, J. Naughton, and P. A. Bernstein, editors, Proceedings of the 2000.

....database [61] which is also based on Objectivity. The main part of this work was done during a research visit at the California Institute of Technology. The Sloan Digital Sky Survey (SDSS) digitally maps about half of the Northern sky in five spectral bands from ultraviolet to the near infrared [63]. In total, some 200 million objects are 101 1 2 3 4 5 6 1 2 3 4 5 a) query 1 2 3 4 5 6 1 2 3 4 5 6 b) query verbatim BBC2 Figure 8.4: Response times for queries Q8 to Q13 based on verbatim vs. compressed bitmap indices. expected to be detected. It will also measure the ....

....galaxy is depicted in Figure 8.6. The SDSS is a collaboration between major universities in the USA. Data is taken by a dedicated 2.5 metre telescope at Apache point, New Mexico, USA. The primary targets of observation are galaxies selected by magnitude and surface brightness limit in the r band [63]. The survey will span five to seven years depending on the weather influences. 8.5.1 Data Preparation All SDSS data is stored in Objectivity DB and can be retrieved by a special SQL like query interface developed by the astronomers. Thus, we first ported all the astronomy data to our so called ....

A.Szalay, P. Kunszt, A. Thakar, J. Gray, D. Slutz, Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey, International Conference on Management of Data, Philadephia, Pennsylvania, USA, June 1999.

No context found.

A. Szalay, P. Z. Kunszt, A.Thakar, J. Gray, D. R. Slutz. "Designing and Minin g Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey," Proc. ACM SIGMOD 2000, pp.451-462, 2000

No context found.

A. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, D. R. Slutz. "Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey," Proc. ACM SIGMOD 2000, pp. 451-462, June 2000

No context found.

Szalay, A.S., Kunszt, P.Z., Thakar, A., Gray, J., Slutz, D. and Brunner, R.J., \Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey", SIGMOD Conference 2000: 451-462.

No context found.

A.S. Szalay, J. Gray, P. Kunszt, A. Thakar "Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey ," Proc. ACM SIGMOD, June 2000, pp. 451-462.

No context found.

A. Szalay, P. Kunszt, A. Thakar, J. Gray, and D. Slutz. Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey. In SIGMOD

No context found.

A. S. Szalay, et al.. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. pp. 451--462. 2000.

No context found.

A. S. Szalay, et al.. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. pp. 451--462. 2000.

No context found.

Szalay A, Kunszt P, Thakar A, Gray J. Designing and mining multi-terabyte astronomy archives: The Sloan Digital Sky Survey. Proceedings of the ACM-SIGMOD Conference on Management of Data. ACM Press, 2000; 451--462.

No context found.

A. S. Szalay, et al.. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. pp. 451--462. 2000.

No context found.

A.S. Szalay, P. Kunzst, A. Thakar, J. Gray, D. Slutz, and R.J. Brunner. Designing and mining multiterabyte astronomy archieves: The sloan digital sky survey. In SIGMOD 2000.

No context found.

A. Szalay, P. Kunszt, A. Thakar, J. Gray, and D. Slutz. Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 451--462, 2000.

No context found.

Szalay, A. S., et al. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. In SIGMOD, Dallas, USA (2000).

No context found.

Alexandar Szalay, Peter Z. Kunszt, Ani Thakar, Jim Gray, and Donald R. Slutz. Designing and mining multi-terabyte astronomy archives: The sloan digital sky survey. In Weidong Chen, Jeffrey F. Naughton, and Philip A. Bernstein, editors, SIGMOD 2000, volume 29, pages 451--462. ACM, 2000.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC