Reduced graphene oxide (rGO) dispersion was obtained by ultrasonication of rGO powder using dodecanethiol (-SH) as an exfoliating agent. Using thiol chemistry, the gold nanoparticles (Au-NPs) were assembled onto the rGO-SH modified glassy carbon electrode (GCE) which showed a strong binding with the surface of the coated electrode. Furthermore, the GCE/rGO-SH/Au-NPs electrode was used to detect mercury (Hg2+) ions in the aqueous solution. When employed as a working electrode, Hg2+ ions get adsorbed on the electrode surface which was later electrochemically oxidized by differential pulse voltammetry (DPV) with the enhanced oxidation current at +0.172 V. Moreover, this sensor platform showed linear response for Hg detection from 1–10 μM in phosphate buffer saline (PBS) solution and the detection limit was found to be 0.2 μM (S/N = 3). The characterization of the Au-NPs and rGO-SH films were studied by Fourier-transform infrared spectroscopy (FT-IR), UV-Visible spectroscopy, transmission electron microscopy (TEM) and field emission scanning electron microscopy (FE-SEM). The application of the prepared sensor was also demonstrated in detecting mercury ions in tap water samples with satisfactory recovery analysis.