We have developed a Brillouin spectrometer based on frequency modulation (FM) spectroscopy in order to enhance the detection sensitivity and to detect separately the real and imaginary parts of the Brillouin spectrum. With this spectrometer, we have measured Brillouin spectra at room temperature in a variety of single crystals including $\mathrm{Si}{\mathrm{O}}_2$ , $\mathrm{Ca}{\mathrm{F}}_2$ , $\mathrm{Li}\mathrm{Nb}{\mathrm{O}}_3$ , deuterated L-arginine phosphate, $\mathrm{Pb}\mathrm{Mo}{\mathrm{O}}_4$ , $\mathrm{Te}{\mathrm{O}}_2$ , langatate $\left({\mathrm{La}}_3{\mathrm{Ta}}_{0.5}{\mathrm{Ga}}_{5.3}{\mathrm{Al}}_{0.2}{\mathrm{O}}_{14}\right)$ , and KRS-5. To determine the Brillouin linewidth and shift from the observed FM spectrum, we have derived a spectral formula for the FM-stimulated Brillouin spectrum by taking into account several contributions from both electrostrictive and absorptive Brillouin scattering and polarization ellipticity of pump or probe waves. This formula reproduces the observed FM Brillouin spectra quite well.