Mixed-organic-cation FA1-xMAxPbI3 perovskites are currently attracting intensive attention for their high photovoltaic performance. However, FA1-xMAxPbI3 usually has undesired non-perovskite phase leading to a compromised efficiency, and a facile strategy to achieve pure-phase FA1-xMAxPbI3 is rarely reported. Here, we demonstrate a facile approach to form δ-phase free FA1-xMAxPbI3 using non-stoichiometric precursor solution. It is found that a small amount of excessive methylammonium iodide (MAI) added in the precursor solution has a profound effect on perovskite crystallization during the liquid-to-solid phase transformation. Using an in-situ photoluminescence spectroscopy measurement, it is found that the excessive MAI can promote the formation of FA1-xMAxPbI3 and facilitate the cation-exchange between organic FA+ and MA+ during the film formation. Based on the optimized pure-phase FA1-xMAxPbI3, the perovskite solar cells exhibit an encouraging power conversion efficiency (PCE) of 17.40%. This study demonstrates a facile method to achieve high performance FA dominating mixed-organic-cation perovskite devices while providing insight into the ion-exchange process during perovskite crystallization.