Diffraction-controlled wavelength dependence of the effective mode area $S_{\mathrm{eff}}\left(\lambda \right)$ in optical fibers can serve as a mechanism limiting the soliton self-frequency shift induced by the Raman effect in materials with retarded nonlinearity. By numerically solving the generalized nonlinear Schrödinger equation modified to include the $S_{\mathrm{eff}}\left(\lambda \right)$ dependence, we show that, as the central wavelength of the soliton increases, the waveguide mode tends to become less compact, slowing down the soliton self-frequency shift. As a result, for optical fibers with a steep $S_{\mathrm{eff}}\left(\lambda \right)$ profile, wavelength uncertainties and the timing jitter of the frequency-shifted soliton induced by input power fluctuations can be substantially reduced compared with fibers with a weak $S_{\mathrm{eff}}\left(\lambda \right)$ dependence.