Environmental exposure to arsenic, phthalate esters (PAEs) and perfluorinated compounds (PFCs) has been associated with human semen quality. However, the epidemiological “black-box” of these associations remains poorly uncovered. In this study, based on the association analysis between arsenic, PAE and PFC exposure and semen quality parameters (i.e., semen volume, sperm concentration, sperm count, progressive motility, total motility and normal morphology) in a Chinese male population, we explored the seminal plasma metabolic signatures that may mediate the exposure-outcome relations by using the meet-in-metabolite-analysis (MIMA) approach. As a result, a negative association was found between DMA and sperm concentration, whereas MEHP and PFHxS were positively associated with sperm count and concentration, respectively. Metabolomics analysis revealed that sixteen and twenty-two seminal plasma metabolites were related to sperm concentration and count, respectively, and they are mainly involved in fatty acid, lipid and amino acid metabolism. Moreover, it was further indicated that eicosatetraenoate, carnitines and DHA may impact the inverse association between DMA and sperm concentration, while eicosatetraenoate, carnitines, DHA, PGB2 and tocotrienol are possible mediators of the positive association between PFHxS and sperm concentration. As these metabolic biomarkers are relevant to antioxidation and fatty acid β-oxidation, we suggest that redox balance and energy generation shifts in seminal plasma are involved in the association of human semen quality with environmental DMA and PFHxS exposure.