Hierarchical Sparse Dictionary Learning

Sparse coding plays a key role in high dimensional data analysis. One critical challenge of sparse coding is to design a dictionary that is both adaptive to the training data and generalizable to data of same type. In this paper, we propose a novel dictionary learning algorithm to build an adaptive dictionary regularized by a priori over-completed dictionary. This hence leads to a hierarchical sparse structure on the a priori dictionary over the learned dictionary, and a hierarchical sparse structure on the learned dictionary over the data, respectively. We demonstrate that this approach reduces overfitting and enhances the generalizability of the learned dictionary. Moreover, the learned dictionary is optimized to adapt thegiven data and results in a more compact dictionary and a more robust sparse representation.We apply the hierarchical sparse dictionary learning approach on both synthetic data and real-world high-dimensional time series data, where the time series dataexhibit high heterogeneity in their nature, e.g., continuous vs discrete, smooth vs non-smooth, etc. The experimental results on a real dataset from a chemical plant process monitoring system demonstrate that the proposed algorithm can successfully characterize the heterogeneity of the given data, and leads to a better and more robust dictionary.
Authors Name: 
Xiao Bian
Xia Ning
Guofei Jiang
S.I. 2014: 
Tuesday, September 8, 2015 - 15:00 to 15:25