Internal governance is related to the improvement of the governance capability of local sports associations and the realization of governance modernization. In the field of social governance research, "soft law governance" has attracted increasing attention from academic circles and has been acknowledged in practice. How to use "soft law governance" to improve the internal governance of local sports associations and realize the modernization of governance has become an important part of sports governance research. Selecting 29 typical 4A class local sports associations in south Jiangsu, the most representative demonstration area of China's modernization construction, as investigation subjects, this article made an in-depth analysis of the governance difficulties and the reasons from the soft law perspective through the methods of questionnaire, depth interview and field investigation. Based on the theory of soft law governance, this paper put forward feasible paths and specific measures to improve the governance of local sports associations.