Identifying factors that influence the land surface temperature (LST) of urban forests can help improve simulations and predictions of spatial patterns of urban cool islands. This requires a quantitative analytical method that combines spatial statistical analysis with multisource observational data. The purpose of this study was to reveal how human activities and ecological factors jointly influence LST in clustering regions (hot or cool spots) of urban forests. Using Xiamen City, China from 1996 to 2006 as a case study, we assumed that human activities interact with specific ecological factors to influence urban forest LST. Thus, the temporal and spatial distributions of LST are similar to those of these ecological factors and can be quantitatively expressed. Population density was selected as a proxy for human activity. We integrated multi-source data (forest inventory, digital elevation models (DEM), population, and remote sensing imagery) to develop a landscape unit on a unified urban scale. An appropriate threshold scale was determined and spatial statistical analyses were conducted to identify hot and cool spots of urban forest LST. The driving mechanism of urban forest LST was revealed through a combination of multi-source spatial data and spatial statistical analysis in clustering regions. The results showed that during rapid urbanization, the degree of spatial clustering of urban forest LST increased. Hot spots were mainly distributed in the urban core and suburbs, whereas cool spots were mainly found in the suburbs and exurbs. The main factors contributing to urban forest LST were dominant tree species and elevation. The interactions between human activity and specific ecological factors linearly or nonlinearly increased LST in urban forests. In conclusion, quantitative studies based on spatial statistics should be conducted in urban areas to reveal interactions between human activities, ecological factors, and LST.