Abstract: The load of strong motion in Longmen Shan area leads to severely damage of a large number of existing retaining structures. How to do assessment for the damaged retaining walls in the earthquake zone is a problem to be solved urgently. This paper summarizes five types of retaining walls' failure modes through the earthquake damage analysis in the research area. They include sliding failure, settlement failure, toppling failure, material failure, and overtopping failure. Then the influence factors of seismic safety evaluation are qualitatively analyzed and classified on the basis of comprehensive principle, important principle and scientific principle. Combining with the failure modes of retaining wall, we put forward the sensitive factor and some general factors. We take the range of 60 percent damage as a sensitive factor. The general factors are divided into two levels and 10 indexes including the sliding distance, the settlement depth, the inclination angle, the number of cracks, the ratio of crack length and the retaining wall length, the ratio of crack depth and the retaining wall thickness, the crack width, the dislocation distance, the range of retaining wall collapsed, the coverage of rock and soil on the retaining wall. Finally, the seismic safety evaluation system of retaining walls is built with grey correlation analysis and fuzzy mathematic theory. We can take prevention instead of post processing, and to serve the post-earthquake recovery and reconstruction.