We address a certain inverse problem in ultrasound-modulated optical tomography: the recovery of the amplitude of vibration of scatterers [ $p(\mathbf{r})$ ] in the ultrasound focal volume in a diffusive object from boundary measurement of the modulation depth (M) of the amplitude autocorrelation of light [ $\varphi (\mathbf{r},\tau )$ ] traversing through it. Since M is dependent on the stiffness of the material, this is the precursor to elasticity imaging. The propagation of $\varphi (\mathbf{r},\tau )$ is described by a diffusion equation from which we have derived a nonlinear perturbation equation connecting $p(\mathbf{r})$ and refractive index modulation [ $\mathrm{\Delta }n(\mathbf{r})$ ] in the region of interest to M measured on the boundary. The nonlinear perturbation equation and its approximate linear counterpart are solved for the recovery of $p(\mathbf{r})$ . The numerical results reveal regions of different stiffness, proving that the present method recovers $p(\mathbf{r})$ with reasonable quantitative accuracy and spatial resolution.