The intrinsic mechanism of feature selection for linear models is correlation.  For instance, invarious sequential methods, the features are selected according to their Pearson’s correlation coefficients with the residual of a current model, in regularized least squares methods, at a fixed value of the penalty parameter, the active set is indeed the set of features whose absolute Pearson’s correlation coefficients with the response exceed a certain threshold.  We refer to this mechanism as the principle of correlation.  In this talk, we explore sequential feature selection procedures in high-dimensional models with complex structures based on the principle of correlation.  Specifically, we consider two such models: (i) a multi-response model where both the response variables and covariates have group structures, and(ii) anuni-response interaction model.  For the first model, we develop a sequential canonical correlation search method.  For the second model, we develop a sequential interaction group selection method.  We provide the asymptotic properties of these methods as well as the simulation studies comparing these methods with other existing approaches. The sequential methods based on the principle of correlation can achieve selection consistency under meld conditions.  The simulation studies demonstrate that they have an edge over the other methods across a comprehensive simulation settings.