We investigate the comb linewidths of self-referenced, fiber-laser-based frequency combs by measuring the heterodyne beat signal between two independent frequency combs that are phase locked to a common cw optical reference. We demonstrate that the optical comb lines can exhibit instrument-limited, subhertz relative linewidths across the comb spectra from $1200\text{to}1720\mathrm{nm}$ with a residual integrated optical phase jitter of $\sim 1\mathrm{rad}$ in a $60\mathrm{mHz}\text{to}500\mathrm{kHz}$ bandwidth. The projected relative pulse timing jitter is $\sim 1\mathrm{fs}$ . This performance approaches that of Ti:sapphire frequency combs.