Image

Kanishka Bhaduri

Member since: Sep 24, 2010, Mission Critical Technologies Inc

Scalable Distributed Change Detection from Astronomy Data Streams using Local, Asynchronous Eigen Monitoring Algorithms

Shared by Kanishka Bhaduri, updated on Sep 22, 2010

Summary

Author(s) :
Kamalika Das, Kanishka Bhaduri, S. Arora, W. Griffin, K. D. Borne, C. Giannella, H. Kargupta
Abstract

This paper considers the problem of change detection using local
distributed eigen monitoring algorithms for next generation
of astronomy petascale data pipelines such as the Large Synoptic
Survey Telescopes (LSST). This telescope will take repeat images
of the night sky every 20 seconds, thereby generating 30 terabytes of calibrated imagery every night that will need to be coanalyzed with other astronomical data stored at different locations around the world. Change point detection and event classification in such data sets may provide useful insights to unique astronomical phenomenon displaying astrophysically significant variations: quasars, supernovae, variable stars, and potentially hazardous asteroids. However, performing such data mining tasks is a challenging problem for such high-throughput distributed data streams. In this paper we propose a highly scalable and distributed asynchronous algorithm for monitoring the principal components (PC) of such dynamic data streams. We demonstrate the algorithm on a large set of distributed astronomical data to accomplish well-known astronomy tasks such as measuring variations in the fundamental plane of
galaxy parameters. The proposed algorithm is provably correct (i.e. converges to the correct PCs without centralizing any data) and can seamlessly handle changes to the data or the network. Real experiments performed on Sloan Digital Sky Survey (SDSS) catalogue data show the effectiveness of the algorithm.

show more info
Publication Name
Scalable Distributed Change Detection from Astronomy Data Streams using Local, Asynchronous Eigen Monitoring Algorithms
Publication Location
SIAM Data Mining Conference (SDM'08), pp 245-256
Year Published
2009

Files

Astronomy_Eigen.pdf
163.5 KB 370 downloads

Discussions

Add New Comment

Kanishka's Projects (4)

Need help?

Visit our help center