A simple and robust setup for femtosecond time-resolved imaging interferometry of surfaces is described. The apparatus is capable of measuring both very small phase shifts $(\sim 3\times {10}^{-2}\mathrm{rad})$ and amplitude changes $(\sim 1\%)$ with micrometer spatial resolution $(\sim 1\mu \mathrm{m})$ . Interferograms are processed using a 2D-Fourier transform algorithm. We discuss the image formation and the physical interpretation of the measured interferograms. The technique is applied to measure transient changes of a GaAs surface irradiated with intense femtosecond laser pulses with fluences near the ablation threshold.