We demonstrate generation of $48\mathrm{fs}$ pulses with linear chirp using a short ( $27\mathrm{mm}$ ) fiber optical parametric oscillator (FOPO), which is synchronously pumped by a mode-locked ytterbium-doped fiber laser. We also study the pulse quality for both the short- and long-wavelength operation where the fiber length inside of the oscillator varies from 17 to $61\mathrm{mm}$ . The optimal pulse duration is observed only in the short-wavelength operation. Furthermore, we model the FOPO system as a single-pass parametric amplifier including dispersive pulse broadening and walk-off between the pump and output. The optimal condition arises from the minimization of the walk-off and dispersion. When walk-off is large, the parametric amplification process is most efficient over some reduced effective fiber length, leading to an upper limit in the amount of the observed pulse broadening.