Abstract: Instability and failure of coal seam drilling is a very complex and common problem, stability of coal seam drilling is very important significance to seam gas drainage and coal seam water injection. Drilling instability is studied based on the D-P criterion and UDEC numerical simulation method. Research shows that shear displacement around the drilling increases exponentially with the joint density increasing, the drilling occurs tensile failure firstly in the wall of drilling, coal body is gradually shear failure with the coal along the radial transferring to deep hole.