Visual signal detection in structured backgrounds. IV. Figures of merit for model performance in multiple-alternative forced-choice detection tasks with correlated responses

Miguel P. Eckstein; Craig K. Abbey; François O. Bochud

doi:10.1364/JOSAA.17.000206

Journal of the Optical Society of America A
Vol. 17,
Issue 2,
pp. 206-217
(2000)
•https://doi.org/10.1364/JOSAA.17.000206

Visual signal detection in structured backgrounds. IV. Figures of merit for model performance in multiple-alternative forced-choice detection tasks with correlated responses

Miguel P. Eckstein, Craig K. Abbey, and François O. Bochud

Not Accessible

Your library or personal account may give you access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Miguel P. Eckstein, Craig K. Abbey, and François O. Bochud, "Visual signal detection in structured backgrounds. IV. Figures of merit for model performance in multiple-alternative forced-choice detection tasks with correlated responses," J. Opt. Soc. Am. A 17, 206-217 (2000)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Abstract

Many investigators are currently developing models to predict human performance in detecting a signal embedded in complex backgrounds. A common figure of merit for model performance is $d^{'},$ an index of detectability that can be mathematically related to the proportion correct (Pc) when the responses of the model are Gaussian distributed and statistically independent. However, in many multiple-alternative forced-choice (MAFC) detection tasks, the target appears in one of M different locations within an image. If the image contains slow spatially varying luminance changes (low-pass noise), the pixel luminance values at the possible signal locations are correlated and therefore the model/human responses to the different locations might also be correlated. We investigate the effect of response correlations on model performance and compare different figures of merit for these conditions. Our results show that use of the standard $d^{'}$ index of detectability assuming statistical independence can lead to erroneous underestimates of Pc and misleading comparisons of models. We introduce a novel figure of merit $d_{r}^{'}$ that takes into account response correlations and can be used to accurately estimate Pc. Furthermore, we show that $d_{r}^{'}$ can be readily related to the standard index of detectability $d^{'}$ by $d_{r}^{'} = d^{'} / \sqrt{1 - r},$ where r is the correlation between the responses in any MAFC detection task. We illustrate the use of the theory by computing figures of merit for two linear models detecting a signal in one of four locations within medical image backgrounds.

Full Article | PDF Article

More Like This

Visual signal detection in structured backgrounds. III. Calculation of figures of merit for model observers in statistically nonstationary backgrounds

François O. Bochud, Craig K. Abbey, and Miguel P. Eckstein
J. Opt. Soc. Am. A 17(2) 193-205 (2000)

Derivation of a detectability index for correlated responses in multiple-alternative forced-choice experiments

Craig K. Abbey and Miguel P. Eckstein
J. Opt. Soc. Am. A 17(11) 2101-2104 (2000)

Visual signal detection in structured backgrounds I. Effect of number of possible spatial locations and signal contrast

Miguel P. Eckstein and James S. Whiting
J. Opt. Soc. Am. A 13(9) 1777-1787 (1996)

Previous Article Next Article

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (7)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (3)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (29)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figure of Merit	Definition	Takes into Account Response Correlations
Pc	Proportion of trials where the model correctly detects the target. Obtained by applying the model template to the images on every trial. Makes no assumptions about the internal response statistics [Eq. (1)].	X
$d^{'}$	Index of detectability defined as the mean model response to the signal location minus mean the model response to nonsignal locations divided by the standard deviation of the response [Eq. (2)].
${Pc}^{'} (d^{'}, M)$	Pc prediction obtained from $d^{'}$ assuming that the responses for each location are Gaussian and statistically independent [Eq. (5)].
${Pc}^{'} (d^{'}, r, M)$ through Monte Carlo	Pc prediction obtained from $d^{'}$ and also taking into account the correlation of the responses of the model. Prediction obtained through Monte Carlo simulations (see Appendix B for details).	X
${Pc}^{'} (d_{r}^{'}, M)$	Pc prediction obtained from $d_{r}^{'} .$ Unlike $d^{'}, d_{r}^{'}$ takes into account the correlation of the responses of the model. Pc prediction obtained by using Eq. (5) with $d_{r}^{'} .$	X
$d_{r}^{'}$	Index of detectability corrected for response correlations.	X
$d_{MAFC}$	Transform of Pc to a detectability metric assuming that responses are Gaussian and independent [inverse of Eq. (5)].	X
Relative efficiency $(d^{'})$	Measure to compare performance of one model relative to the other by taking squared ratios of $d^{'} .$
Relative efficiency $(d_{r}^{'})$	Measure to compare performance of one model relative to the other by taking squared ratios of $d_{r}^{'} .$	X
Relative efficiency $(d_{MAFC})$	Measure to compare performance of one model relative to the other by taking squared ratios of $d_{MAFC} .$	X

Figure of Merit	Model
Figure of Merit	NPWE	Fisher–Hotelling
$d^{'}$	$1.44 \pm 0.104$	$2.23 \pm 0.107$
${Pc}^{'} (d^{'}, M)$	$0.681 \pm 0.029$	$0.868 \pm 0.021$
Relative efficiency $(d^{'})$	$0.417 \pm 0.027$

Figure of Merit	Model
Figure of Merit	NPWE	Fisher–Hotelling
r	$0.488 \pm 0.04$	$0.08 \pm 0.02$
Pc	$0.838 \pm 0.011$	$0.878 \pm 0.014$
${Pc}^{'} (d^{'}, r, M)$ (Monte Carlo)	$0.823 \pm 0.03$	$0.872 \pm 0.021$
${Pc}^{'} (d_{r}^{'}, M)$	$0.826 \pm 0.0298$	$0.878 \pm 0.0159$
$d_{a}$ (through Pc)	$2.08 \pm 0.061$	$2.31 \pm 0.09$
$d_{r}^{'}$	$2.01 \pm 0.145$	$2.32 \pm 0.106$
Relative efficiency $(d_{a})$	$0.81 \pm 0.052$
Relative efficiency $(d_{r}^{'})$	$0.75 \pm 0.0497$

Figure of Merit	Definition	Takes into Account Response Correlations
Pc	Proportion of trials where the model correctly detects the target. Obtained by applying the model template to the images on every trial. Makes no assumptions about the internal response statistics [Eq. (1)].	X
$d^{'}$	Index of detectability defined as the mean model response to the signal location minus mean the model response to nonsignal locations divided by the standard deviation of the response [Eq. (2)].
${Pc}^{'} (d^{'}, M)$	Pc prediction obtained from $d^{'}$ assuming that the responses for each location are Gaussian and statistically independent [Eq. (5)].
${Pc}^{'} (d^{'}, r, M)$ through Monte Carlo	Pc prediction obtained from $d^{'}$ and also taking into account the correlation of the responses of the model. Prediction obtained through Monte Carlo simulations (see Appendix B for details).	X
${Pc}^{'} (d_{r}^{'}, M)$	Pc prediction obtained from $d_{r}^{'} .$ Unlike $d^{'}, d_{r}^{'}$ takes into account the correlation of the responses of the model. Pc prediction obtained by using Eq. (5) with $d_{r}^{'} .$	X
$d_{r}^{'}$	Index of detectability corrected for response correlations.	X
$d_{MAFC}$	Transform of Pc to a detectability metric assuming that responses are Gaussian and independent [inverse of Eq. (5)].	X
Relative efficiency $(d^{'})$	Measure to compare performance of one model relative to the other by taking squared ratios of $d^{'} .$
Relative efficiency $(d_{r}^{'})$	Measure to compare performance of one model relative to the other by taking squared ratios of $d_{r}^{'} .$	X
Relative efficiency $(d_{MAFC})$	Measure to compare performance of one model relative to the other by taking squared ratios of $d_{MAFC} .$	X

Figure of Merit	Model
Figure of Merit	NPWE	Fisher–Hotelling
$d^{'}$	$1.44 \pm 0.104$	$2.23 \pm 0.107$
${Pc}^{'} (d^{'}, M)$	$0.681 \pm 0.029$	$0.868 \pm 0.021$
Relative efficiency $(d^{'})$	$0.417 \pm 0.027$

Abstract

Cited By

Figures (7)

Tables (3)

Equations (29)

Journal of the Optical Society of America A