Abstract: The instability and failure of coal pillar along gob-side entry is a typical nonlinear process. The cusp catastrophic model is obtained on the basis of the mechanical model of coal pillar system. Then, the formula of critical width for instability and failure of coal pillar is obtained. The results show that the critical width of coal pillar depends on the external factors such as the depth, the coal pillar height, the internal friction angle and cohesion, the stress concentration factor, the lateral pressure coefficient, the entry width as well as supporting intensity of coal rib. Based on the critical width of coal pillar, a safety factor is introduced to determine the rational width of narrow coal pillar. Finally, the feasibility of the method is verified by an engineering case.