Sensor sharpening [J. Opt. Soc. Am. A 11, 1553 (1994)] has been proposed as a method for improving computational color constancy, but it has not been thoroughly tested in practice with existing color constancy algorithms. In this paper we study sensor sharpening in the context of viable color constancy processing, both theoretically and empirically, and on four different cameras. Our experimental findings lead us to propose a new sharpening method that optimizes an objective function that includes terms that minimize negative sensor responses as well as the sharpening error for multiple illuminants instead of a single illuminant. Further experiments suggest that this method is more effective for use with several known color constancy algorithms.
You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
Count of TimesStandard ResultsReplaced Full-SharpResult Due toProblems (UsuallyDue to NegativeComponents)
BEST-LINEAR
none
2.93
2.93
*
0
BEST-DIAGONAL
none
4.25
4.25
0.29
0
BEST-DIAGONAL
ave
9.04
27.66
0.96
0
BEST-DIAGONAL
opt
2.98
21.58
0.31
0
BEST-DIAGONAL
mip
4.09
4.12
0.32
0
ACTUAL
none
4.38
4.38
0.00
0
ACTUAL
ave
21.91
21.91
=
0
ACTUAL
opt
4.92
4.92
=
0
ACTUAL
mip
4.20
4.20
=
0
NOTHING
none
*
*
16.30
0
GW
none
136.57
136.57
8.12
0
GW
ave
379.95
379.95
=
0
GW
opt
1951.78
1951.78
=
0
GW
mip
136.70
136.70
=
0
DB-GW
none
33.66
33.66
6.54
0
DB-GW
ave
203.32
167.33
6.54
0
DB-GW
opt
576.75
349.75
11.39
0
DB-GW
mip
33.70
33.75
6.53
0
SCALE-BY-MAX
none
105.57
105.57
9.43
0
SCALE-BY-MAX
ave
105.09
1014.44
9.56
0
SCALE-BY-MAX
opt
829.38
393.21
10.56
0
SCALE-BY-MAX
mip
103.98
106.12
9.35
0
ECRULE-MV
none
50.28
50.28
5.99
0
ECRULE-MV
ave
50.74
63.18
6.22
1
ECRULE-MV
opt
49.87
261.00
5.98
998
ECRULE-MV
mip
49.98
50.59
6.00
0
ECRULE-ICA
none
37.15
37.15
7.01
0
ECRULE-ICA
ave
39.01
37.76
6.93
0
ECRULE-ICA
opt
36.62
283.50
6.79
997
ECRULE-ICA
mip
37.47
37.05
7.10
8
Assuming that the algorithms provide estimates that are normally distributed around the target values, the uncertainty in these numbers is roughly 1%. An asterisk is used for values that are not relevant or appropriate. An equal sign is used for results which must be the same as the nonsharp result. The results from the optimal database transform (assuming the illuminant is known) are indicated by “opt,” the results from the database transform with the average illuminant by “ave,” and the results from the multiple-illuminant-with-positivity sharpening method by “mip.” Results from using the two error measures are provided. In the case of the rms RGB mapping error, we provide results both for using sharpening throughout the processing (full sharp), and for using sharpening only to correct the image based on an illuminant estimate found in nonsharp space (sharp correction). Sharp correction has no effect on the estimation of the illumination RGB, and the angular error is exactly the same as for no sharpening and thus is not listed. The sharp-correction mapping errors are the same as their full-sharp counterparts in the case of ACTUAL
and GW. Here we simply repeat the numbers.
Table 2
Sharpening Results for the Kodak DCS-460 Digital Camera on Synthetic Dataa
Algorithm
SharpeningMethod
rms RGBDifferencebetween MappedImage and TargetImage
rms RGBDifferencebetween MappedImage and TargetImage (SharpCorrection)
Count of TimesStandard ResultsReplaced Full-SharpResult Due toProblems (UsuallyDue to NegativeComponents)
BEST-LINEAR
none
18.8
18.8
*
0
BEST-DIAGONAL
none
21.2
21.2
1.75
0
BEST-DIAGONAL
ave
20.8
21.8
1.16
0
BEST-DIAGONAL
opt
20.8
21.6
1.41
0
BEST-DIAGONAL
mip
20.7
21.7
1.32
0
ACTUAL
none
25.2
25.2
0.00
0
ACTUAL
ave
25.6
25.6
=
0
ACTUAL
opt
26.1
26.1
=
0
ACTUAL
mip
25.8
25.8
=
0
NOTHING
none
83.7
83.7
7.92
0
GW
none
101.8
101.8
5.72
0
GW
ave
102.3
102.3
=
0
GW
opt
105.2
105.2
=
0
GW
mip
102.8
102.8
=
0
DB-GW
none
77.8
77.8
5.12
0
DB-GW
ave
77.9
77.5
5.16
0
DB-GW
opt
78.5
77.8
5.18
0
DB-GW
mip
78.0
77.5
5.15
0
SCALE-BY-MAX
none
48.1
48.1
2.33
0
SCALE-BY-MAX
ave
45.2
123.2
2.35
0
SCALE-BY-MAX
opt
44.0
81.7
1.91
0
SCALE-BY-MAX
mip
44.6
49.9
2.17
0
ECRULE-MV
none
85.3
85.3
5.55
0
ECRULE-MV
ave
38.2
85.5
2.45
0
ECRULE-MV
opt
38.8
84.8
3.02
0
ECRULE-MV
mip
52.8
85.2
3.93
0
ECRULE-ICA
none
102.7
102.7
5.69
0
ECRULE-ICA
ave
68.7
107.6
2.90
0
ECRULE-ICA
opt
68.8
100.5
3.69
0
ECRULE-ICA
mip
84.0
102.4
4.14
0
See notes for Table 1. The results are the rms of 144 results. Assuming that the variation of the errors for a given algorithm is Gaussian, the error in the results is roughly 5%.
Tables (5)
Table 1
Sharpening Results for the Sony DXC-930 Video Camera on Synthetic Dataa
Algorithm
SharpeningMethod
rms RGBDifferencebetween MappedImage and TargetImage
rms RGBDifferencebetween MappedImage and TargetImage (SharpCorrection)
Count of TimesStandard ResultsReplaced Full-SharpResult Due toProblems (UsuallyDue to NegativeComponents)
BEST-LINEAR
none
2.93
2.93
*
0
BEST-DIAGONAL
none
4.25
4.25
0.29
0
BEST-DIAGONAL
ave
9.04
27.66
0.96
0
BEST-DIAGONAL
opt
2.98
21.58
0.31
0
BEST-DIAGONAL
mip
4.09
4.12
0.32
0
ACTUAL
none
4.38
4.38
0.00
0
ACTUAL
ave
21.91
21.91
=
0
ACTUAL
opt
4.92
4.92
=
0
ACTUAL
mip
4.20
4.20
=
0
NOTHING
none
*
*
16.30
0
GW
none
136.57
136.57
8.12
0
GW
ave
379.95
379.95
=
0
GW
opt
1951.78
1951.78
=
0
GW
mip
136.70
136.70
=
0
DB-GW
none
33.66
33.66
6.54
0
DB-GW
ave
203.32
167.33
6.54
0
DB-GW
opt
576.75
349.75
11.39
0
DB-GW
mip
33.70
33.75
6.53
0
SCALE-BY-MAX
none
105.57
105.57
9.43
0
SCALE-BY-MAX
ave
105.09
1014.44
9.56
0
SCALE-BY-MAX
opt
829.38
393.21
10.56
0
SCALE-BY-MAX
mip
103.98
106.12
9.35
0
ECRULE-MV
none
50.28
50.28
5.99
0
ECRULE-MV
ave
50.74
63.18
6.22
1
ECRULE-MV
opt
49.87
261.00
5.98
998
ECRULE-MV
mip
49.98
50.59
6.00
0
ECRULE-ICA
none
37.15
37.15
7.01
0
ECRULE-ICA
ave
39.01
37.76
6.93
0
ECRULE-ICA
opt
36.62
283.50
6.79
997
ECRULE-ICA
mip
37.47
37.05
7.10
8
Assuming that the algorithms provide estimates that are normally distributed around the target values, the uncertainty in these numbers is roughly 1%. An asterisk is used for values that are not relevant or appropriate. An equal sign is used for results which must be the same as the nonsharp result. The results from the optimal database transform (assuming the illuminant is known) are indicated by “opt,” the results from the database transform with the average illuminant by “ave,” and the results from the multiple-illuminant-with-positivity sharpening method by “mip.” Results from using the two error measures are provided. In the case of the rms RGB mapping error, we provide results both for using sharpening throughout the processing (full sharp), and for using sharpening only to correct the image based on an illuminant estimate found in nonsharp space (sharp correction). Sharp correction has no effect on the estimation of the illumination RGB, and the angular error is exactly the same as for no sharpening and thus is not listed. The sharp-correction mapping errors are the same as their full-sharp counterparts in the case of ACTUAL
and GW. Here we simply repeat the numbers.
Table 2
Sharpening Results for the Kodak DCS-460 Digital Camera on Synthetic Dataa
Algorithm
SharpeningMethod
rms RGBDifferencebetween MappedImage and TargetImage
rms RGBDifferencebetween MappedImage and TargetImage (SharpCorrection)
Count of TimesStandard ResultsReplaced Full-SharpResult Due toProblems (UsuallyDue to NegativeComponents)
BEST-LINEAR
none
18.8
18.8
*
0
BEST-DIAGONAL
none
21.2
21.2
1.75
0
BEST-DIAGONAL
ave
20.8
21.8
1.16
0
BEST-DIAGONAL
opt
20.8
21.6
1.41
0
BEST-DIAGONAL
mip
20.7
21.7
1.32
0
ACTUAL
none
25.2
25.2
0.00
0
ACTUAL
ave
25.6
25.6
=
0
ACTUAL
opt
26.1
26.1
=
0
ACTUAL
mip
25.8
25.8
=
0
NOTHING
none
83.7
83.7
7.92
0
GW
none
101.8
101.8
5.72
0
GW
ave
102.3
102.3
=
0
GW
opt
105.2
105.2
=
0
GW
mip
102.8
102.8
=
0
DB-GW
none
77.8
77.8
5.12
0
DB-GW
ave
77.9
77.5
5.16
0
DB-GW
opt
78.5
77.8
5.18
0
DB-GW
mip
78.0
77.5
5.15
0
SCALE-BY-MAX
none
48.1
48.1
2.33
0
SCALE-BY-MAX
ave
45.2
123.2
2.35
0
SCALE-BY-MAX
opt
44.0
81.7
1.91
0
SCALE-BY-MAX
mip
44.6
49.9
2.17
0
ECRULE-MV
none
85.3
85.3
5.55
0
ECRULE-MV
ave
38.2
85.5
2.45
0
ECRULE-MV
opt
38.8
84.8
3.02
0
ECRULE-MV
mip
52.8
85.2
3.93
0
ECRULE-ICA
none
102.7
102.7
5.69
0
ECRULE-ICA
ave
68.7
107.6
2.90
0
ECRULE-ICA
opt
68.8
100.5
3.69
0
ECRULE-ICA
mip
84.0
102.4
4.14
0
See notes for Table 1. The results are the rms of 144 results. Assuming that the variation of the errors for a given algorithm is Gaussian, the error in the results is roughly 5%.