The Moderate Resolution Imaging Spectroradiometer (MODIS) currently provides the most extensive aerosol retrievals on a global basis, but validation is limited to a small number of ground stations. This study presents a comprehensive evaluation of Collection 4 and 5 MODIS aerosol products using ground measurements from the Chinese Sun Hazemeter Network (CSHNET). The Collection 5 product uses a new method to determine the surface reflectance and a new aerosol model to retrieve aerosol optical thickness (AOT). Substantial improvement was found in the Collection 5 AOT product relative to the Collection 4 AOT product. Overall, the correlation coefficient of regression with ground measurements increased from 0.66 to 0.84 for all data points. The offset is reduced from 0.179 to 0.047, and the slope is improved from 0.74 to 0.98. At individual sites, the improvement varies with surface and atmospheric conditions. In general, both versions of the MODIS aerosol product tend to overestimate AOT over deserts/semi-deserts, tend to underestimate AOT over forests, and are more accurate over agricultural and suburban sites. The poorest retrievals occur over urban areas. The land cover dependence of aerosol retrievals was traced to the estimation of surface reflectance. The selection of the aerosol type is another major factor contributing to the discrepancies. Errors caused by both factors are subject to considerable variations with season and location due to land cover changes and varying fractions of coarse and fine mode aerosols, as well as the changing amount of scattering and absorbing aerosols.