Many scientific datasets are of high dimension, and the analysis usually requires retaining the most important structures of data. Many existing methods work only for data with structures that are mathematically formulated by curves, which is quite restrictive for real applications. To get more general graph structures, we develop a novel graph structure learning framework that captures the local information of the underlying graph structure based on reversed graph embedding. A new learning algorithm is developed that learns a set of principal points and a graph structure from data, simultaneously. Experimental results on various synthetic and real world datasets show that the proposed method can uncover the underlying structure correctly.