Seminars

Events Calendar

Previous month Previous day Next day Next month
By Year By Month By Week Today Search Jump to month
[Seminars]
Phase Transitions for High Dimensional Clustering and Related Problems
Dr Wang Wanjie
Department of Statistics and Applied Probability, NUS
 
Wednesday 10 October 2018, 03:00pm - 04:00pm
S16-06-118, Seminar Room
 

Clustering, or unsupervised learning, is a major problem in statistics with many applications. In the Big Data era, it faces two main challenges: 1. the number of features is much larger than the sample size; 2. The signals are sparse and weak, masked by large amount of noise. The two problems are very common in high dimensional problems, such as detection and signal recovery problems.

We consider the two-class clustering problem under the proposed asymptotic rare and weak signals model. In the two-dimensional phase space calibrating the rarity and strengths of the signals, we find the precise demarcation for the Region of Impossibility (clustering is impossible with any method) and Region of Possibility (there are successful clustering methods). Especially, we zoom-in an interesting region of this phase diagram, and find a more delicate demarcation as the result of a newly proposed method, IF-PCA. 

The phase diagrams are also studied for the related detection problem and signal recovery problem.