We incorporate newly developed solid-state detector technology into time-resolved laser Raman spectroscopy, demonstrating the ability to distinguish spectra from Raman and fluorescence processes. As a proof of concept, we show fluorescence rejection on highly fluorescent mineral samples willemite and spodumene using a $128\times 128$ single-photon avalanche diode (SPAD) array with a measured photon detection efficiency of 5%. The sensitivity achieved in this new instrument architecture is comparable to the sensitivity of a technically more complicated system using a traditional photocathode-based imager. By increasing the SPAD active area and improving coupling efficiency, we expect further improvements in sensitivity by over an order of magnitude. We discuss the relevance of these results to in situ planetary instruments, where size, weight, power, and radiation hardness are of prime concern. The potential large-scale manufacturability of silicon SPAD arrays makes them prime candidates for future portable and in situ Raman instruments spanning numerous applications where fluorescence interference is problematic.