In acid rain regions of Southwest China, cement-emulsified asphalt mortar (CA mortar), a key structure component in China Railway Track System (CRTS), is reported with more serious damaged signs than other locations. To investigate the effects of acid rain on its durability, the CA mortar specimens were immersed in acid solutions with pH values of 3.0, 4.0, and 5.0 for 21 months. Scanning electron microscope (SEM), energy dispersive spectrometer (EDS), differential thermal analyzer (DTA) and biological microscope were applied to evaluate the deterioration mechanism. Cracking, desquamation, pulverization, and decreased compressive strength were observed in them. The results of SEM and EDS show that a large number of crystals, rich in Ca, S and O, deposited on surfaces of the specimen soaking in the solutions, which is related to gypsum as shown in DTA pattern. Also, DTA patterns show that the decomposition peak of calcium silicate hydrate (C–S–H) gel in CA mortar specimens decreased gradually from the internal to the external surface. The analysis of acid solution based on the biological microscope indicates that large amounts of filamentous fungi and yeast existed in the solution during the long-term soaking, which corresponds to the investigation of the seriously damaged CA mortar in the field. In conclusion, deterioration mechanisms of CA mortar by acid rain includes CaSO4·2H2O crystalline expansion damage, decomposition and dissolution of C–S–H gel and biological corrosion when the alkaline environment was damaged.