Accommodating Multiple Illumination Sources in an Imaging Colorimetry Environment

header for SPIE use
Accommodating Multiple Illumination Sources in an Imaging
Colorimetry Environment
Kenneth W. Tobin*, James S. Goddard, Martin A. Hunt, Kathy W. Hylton, Thomas P. Karnowski,
Marc L. Simpson, Roger K. Richards, and Dale A. Treece
Oak Ridge National Laboratory†, P.O.Box 2008, Oak Ridge, Tennessee 37831-6011
ABSTRACT
Researchers at the Oak Ridge National Laboratory have been developing a method for measuring color quality in
textile products using a tri-stimulus color camera system. Initial results of the Imaging Tristimulus Colorimeter (ITC) were
reported during 1999. These results showed that the projection onto convex sets (POCS) approach to color estimation could
be applied to complex printed patterns on textile products with high accuracy and repeatability. Image-based color sensors
used for on-line measurement are not colorimetric by nature and require a non-linear transformation of the component colors
based on the spectral properties of the incident illumination, imaging sensor, and the actual textile color. Our earlier work
reports these results for a broad-band, smoothly varying D65 standard illuminant. To move the measurement to the on-line
environment with continuously manufactured textile webs, the illumination source becomes problematic. The spectral
content of these light sources varies substantially from the D65 standard illuminant and can greatly impact the measurement
performance of the POCS system. Although absolute color measurements are difficult to make under different illumination,
referential measurements to monitor color drift provide a useful indication of product quality. Modifications to the ITC
system have been implemented to enable the study of different light sources. These results and the subsequent analysis of
relative color measurements will be reported for textile products.
Keywords: imaging colorimetry, tristimulus colorimetry, projection onto convex sets, D65 illumination, metal-halide
illumination, textile products, rotary screen printing
1. INTRODUCTION
Color measurement has many different meanings in the various disciplines and industries that use and exchange
information dealing with color. As stated by Bartleson “there is no single definition of color measurement that is satisfactory

---

Background
The accurate measurement of the color of surfaces using the human tristimulus models has proven to be a difficult
task for researchers [4]. The reasons for this are many with the primary issues being the highly nonlinear conversion between
color spaces, adequate spectral characterization of the illumination and sensor, and uniform perceptual based metrics. Added
difficulties arise when these color measurements are made during the high-speed production of textiles with intricate printed
patterns. Current industry practice requires that a segment of the material be removed (i.e., a “strike-off”) and taken to a lab
containing a colorimeter or spectrophotometer. The time required for this analysis typically is too great to effect the
production process in a timely manner, especially for those many manufacturers who produce short runs of fabric (e.g., 1000
to 3000 yards at 150 yards / minute = 7 to 20 minutes per run). In addition, the standard color measurement equipment
averages all the color in a predefined region, such as a small (e.g., 3/8 in. to 5/8 in.) circle. The ITC system uses a vector-
space and set theoretic approach to address the nonlinear optimization problem of transforming the measured component
colors to the predicted tristimulus responses. Image processing techniques are used to segment regions based on the
geometry of the printed patterns and generate statistical averages of the component colors within these segmented regions.
The use of international standards such as CIELAB and ?ECMC for the color metrics provides a perceptual basis for all
measurements. We address these issues using a machine vision approach. In a machine vision system the electronic camera
and associated filters function as the viewer and the objective is to map the response of these two components to the ideal
human tristimulus response with a standard illuminant. The advantages of using a machine vision solution in combination
with new processing algorithms for measuring online process color variations over a traditional colorimeter or
spectrophotometer are the ability to make repeatable color measurements for an arbitrary geometric shape during on-line
inspection.
Colorimetry Fundamentals
In this section we will briefly review the fundamentals of colorimetry and describe the color metrics adopted for the
ITC system development effort. Several references are available which review this subject in greater depth [1, 5, 6, 7, 8].
Early human color vision research performed by Grassmann and Maxwell theorized that color could be mathematically
specified in terms of three independent components. This theory matched the hypothesis that the human eye contained three
receptors, each with a unique spectral response function. It was shown that any additive color mixture could be matched by
the proper amounts of three primary colors. This is commonly called the tristimulus response. Determining the amount of
these primary colors that are required to match an unknown color can specify the human visual response. The Commission
Internationale de l’Eclairage (CIE) used this fact to establish a standard for numerical specification of color in terms of three
coordinate or tristimulus values (XYZ).
The tristimulus response of the human eye to a solid-colored object is a function of three physical attributes: the spectral
reflectance of the object, the spectral content of the illumination source, and the spectral response of the viewer. The spectral
response of the viewer is defined by the tristimulus response curves determined by the CIE. The tristimulus values are given
by the following equation,
{d , d , d
,
,
or d = XE?,
(1)
1
2
3} =
{x1(?) x2(?) x3(?)}E(?)?(?)
?
d?
where E(?) is the spectral response of the illumination, ?(?) is the spectral reflectance of the object, and {x1(?),x2(?),x3(?)}
are the human tristimulus response curves. A colorimeter instrument has a spectral response that is different from the CIE-
defined human spectral response and the resulting measured tristimulus values are given by,
{c ,c ,c
,
,
or c = RE?,
(2)
1
2
3 } =
{r1(?) r2(?) r3(?)}E(?)?(?)
?
?
d
where {r1(?),r2(?),r3(?)} represents the spectral responses of the three component color channels of the colorimeter. The
primary objective of the nonlinear optimization is to derive a mapping that converts the vector c to d.
Several color spaces have been developed that attempt to define an n-dimensional space such that a given distance in the
space corresponds to a uniform perceivable color difference. These color spaces are specified by formulas that relate the
tristimulus values to coordinates in the new space. The color space used in this work is the CIE 1976 CIELAB for the 10o
observer. A further refinement of the measurement of color difference is defined as ?ECMC [9]. This refinement attempts to
establish a more uniform metric for describing color differences and has been adopted by in the textile industry. ?ECMC is
defined by the following relationships that are used in this study,

---

2
2
2
? ?L*?
? ?C *
H
ab
?
? ?
*
ab
?
?ECMC
=
+
+
(3)
(
)
1
:
2
?
?
?
?
?
?
?
?
?
?
?
?
? lS
cS
S
L ?
?
c
?
?
H
?
where ?L* in the L*a*b* coordinate system is typically defined by,
13
? Y ?
?
Y
L * = 116 f
?16 if
> 0.01
(4)
?
? ??
? Y
Y
o ?
o
or
? Y ?
?
Y
L * =
.
903 3 f
if
? 0.008856
?
? ??
? Y
Y
o ?
o
and ?Cab* and ?Hab* are polar functions of the a* (red/green) and b* (blue/yellow) coordinates defined by,
1
1
1
1
?
? ? X ? 3
? Y ? 3 ??
?
? ? Y ? 3
? Z ? 3 ??
a * = 500 f
? f
,
b * = 200 f
? f
(5)
? ??
?
?
?
? ??
X
Y
?
? ?? ??
?
?
?
?
Y
Z
?
? ? o ?
? o ? ?
? ? o ?
? o ? ?
Projection onto Convex Sets Approach
As stated in the previous section the fundamental task required to achieve an imaging colorimeter is to derive a
mapping that converts the vector c to d, thus matching the CIE human tristimulus response. Many approaches have been
proposed and used for this mapping with the most straightforward being a linear combination. This transform is sufficient in
many cases, especially if the mapping only occurs in a limited region of the color space. For a more accurate mapping, the
nonlinear nature of the transform must be taken into account. The primary methods used today are three-dimensional look-up
tables [10], least-squares polynomials [11, 12], neural networks [13], and set theoretic approaches. In this research we use a
set theoretic approach based on the POCS method. Set theoretic approaches differ from classical estimation in that the
solution is not based on minimizing or maximizing a cost function but rather on finding a solution which satisfies all the
prescribed constraints, i.e., an intersection of sets. In this work we will also use a linear transform method based on a linear
regression algorithm to compare the POCS results in subsequent sections.
The use of set theoretic methods has been employed in several signal-processing applications such as image restoration,
tomographic reconstruction, and spectral analysis. Youla [14] used POCS for image restoration and gives a thorough
description of the mathematics and algorithms used. Trussell [15] used POCS to specify metamers under different
illuminations and to design optimum color filters for scanners.
In this work we have adapted previous work in set theoretic methods through the definition of various applicable constraints.
The fundamentals of the POCS method are based on linear vector space theory. A central principle of vector space theory is
the definition of an inner product. In the development and application of the POCS algorithm the Hilbert space is used as the
working space because it is a complete inner product space (i.e., a finite dimensional vector space with the standard dot
product used for the inner product represents a Hilbert space). With this vector space defined, we can describe how the
constraints are represented as a closed convex set. A convex set is a set C in the Hilbert space such that if f, g ? C and if 0 ?
? ? 1, then h = ? f + (1- ?) g ? C. This definition is shown graphically in two dimensions in Fig. 1. In basic terms, all the
points on a line between any two points in the set must also be in the set. One can also show that the nonempty intersection of
any two closed convex sets is also closed and convex. A solution lies in the intersection of all the constraint sets and is
obtained by sequentially projecting an initial estimate of the solution onto each constraint set. This intersection is achieved
through an iterative algorithm that converges to the intersecting boundary between the defined constraint sets as represented
in Fig. 2. For further implementation details of the POCS algorithm, see [3, 15].

---

initial
estimate
set B
B
A
B
solution
A
set A
set C
Figure 1 – A graphical representation of a convex set (left
and a non-convex set (right).
Figure 2 – A representation of the iterative projection
operation that results in a solution along the boundary of
the intersection of the constraint sets.
The ITC System
The ITC system developed for testing the POCS approach is shown schematically in Fig. 3. The ITC was developed
as an offline system for developing and evaluating the POCS measurement approach. The product under inspection can be
illuminated at various angles and illuminants. The illuminants used for this research were the MacBeth Sol Source D65
simulant based on a quartz-halogen lamp with custom filters, and a Pendant metal halide illuminant with a color temperature
of 6500K.
The reflected energy was imaged through an Integrated Scientific color filter wheel onto a 768x484 CCD sensor in a Pulnix
TM-9701 digital camera. The tristimulus filters in the filter wheel were selected to have the same general characteristics of
the CIE tristimulus response curves and are shown in Fig. 4.
Windows NT 4.0,
90MHz Pentium

---

1.2
The digital Pulnix camera data was processed by a Matrox image
green
processing board in a Windows NT, 90 MHz Pentium computer.
1
A PC2000 plug-in PC spectrometer was used to measure the
se
spectral characteristics of textile samples and calibration tiles. A
0.8
Microsoft Visual basic graphical user interface (GUI) was
blue
red
0.6
developed to provide user access to the filter wheel, PC
spectrometer, camera, and input and output data files. The image
t
i
ve respon
0.4
a
segmentation, data bookkeeping and tracking, and POCS
Rel
algorithm were implemented in Microsoft Visual C++.
0.2
0
2. SEGMENTATION FOR COLOR ANALYSIS
400
500
600
700
W avelength (nm )
Wavelength (nm)
The benefits of an imaging based colorimeter is the ability to
make measurements on arbitrary shaped regions or objects. This
Figure 4 – Tristimulus spectral product (RE) of the
section will describe the image processing algorithms used to
ITC system.
segment and extract component colors (R in Eq. (2)) from regions
within printed textile patterns. Many segmentation approaches exist and most fall into several broad categories such as local
filtering (e.g. edge detection), snake and balloon methods, region growing and merging, and global optimization [16]. In this
work during the first year, we initially took a global approach based on multilevel thresholds (i.e., a maximum entropy
technique) derived from maximizing the contrast in a given color band [17]. While this approach performed reasonably well
on most textile patterns, it did not take into account spatial information from the pattern under test. Consequently we have
adopted a mean shift algorithm that iterates between a color-space clustering of the component colors and the spatial
information across boundaries [18]. This is a simple, non-parametric method for estimating density gradients and can
produce a high-quality edge image for color segmentation. The result is a more robust approach to discriminating between
the various color regions in the image while filtering out border regions where color confusion will occur due to integration
across discrete pixel boundaries or due to achievable tolerances in the rotary screen print process (e.g., overlapping of print).
This procedure is indicated in Fig. 5.
dark green
non-sampled
boundary
light purple
light green
background
(a)
(b)
(c)
Figure 5 – Example of a textile pattern with three dominant colors, dark green (b), light green (c), and
light purple (a). Note the excluded region of pixels between the boundaries in (a).

---

Figure 6 shows a flow diagram of the approach used by the ITC system for color segmentation. The first step is to perform a
flat-field illumination correction and convert the component color of individual images to hue, lightness, saturation (HLS) or
the L*u*v* color space, depending on whether the maximum entropy or mean shift algorithm is employed. This HLS or
L*u*v* representation generates regions that correspond better to perceived boundaries than a solely intensity or raw
component color segmentation. Next either the maximum entropy (first year effort) or mean shift algorithm (current work) is
applied to the three component images. This process results in three labeled images that must then be post processed to
identify and extract summary statistics from distinct regions. The first step in this post processing is to apply morphological
erosion to each labeled region. This operation removes small noise generated regions and removes the boundary area of valid
regions. The next processing step takes the intersection of the three labeled images to generate a composite labeled image.
The composite labeled image is then used to extract
color
information from the original composite color images
RGB output of ITC
image
and provide it to the POCS algorithm. The average
input
and standard deviation of the pixel values within a
conversion to HLS (or L*u*v*)
region are computed to generate the R vector and a
statistical measure of the color uniformity within a
(1) maximum entropy multilevel thresholding
region. In addition the centroid of each region is
(2) mean shift algorithm
computed and used to look-up the initial estimate of
the spectral reflectance, ?, for the region.
connected component labeling
3. ILLUMINATION
erosion
segmented
image
A complimentary rotary screen print inspection
intersection of connected components (blobs)
output
from each of the three bands
system was developed under the American Textile
(AMTEXTM) Partnership for pattern inspection and
grouping of specific colors based on summary
quality assessment as shown in Fig. 7 [19]. The on-
intensity/hue/saturation statistics for each
line version of this system, designed by Sandia
connected component
National Laboratory, uses a Dalsa tri-linear color line
scan camera which requires a high-intensity
Figure 6 – Flow diagram of color region segmentation process.
illumination source (manufactured by Iridis) to
achieve sufficient intensity for the rotary screen print
process. The Iridis system uses a metal halide arc lamp in conjunction with an acrylic light pipe to deliver a focussed line
source of illumination. Since the Iridis illumination system produces a line source, it was not suitable for testing with the
ITC, but testing was required since a future on-line version of the ITC system will require a similar source to achieve
sufficient exposure. Therefore an area illumination source manufactured by Pendant was obtained that had spectral
characteristics similar to the Iridis.
color line-scan
linear illumination
line-scan imaging
camera
source
colorimeter
drying, chemical
print range
treatment, color
textile web
set, ...
pattern quality
color quality
inspection
reporting
inspection
Figure 7 – Schematic representation of the rotary screen print pattern inspection system and imaging
colorimetry system. Also shown are their relative positions in the manufacturing process.

---

The Sol Source (D65), Iridis (metal halide), and Pendant (metal halide) illumination spectrums are shown in Fig. 8. Note that
the metal halide sources differ from the D65 illuminant in several important ways. For instance, the D65 illuminant, although
fairly uniform, is dominated by energy at the blue
end of the spectrum while the metal halide sources
are dominant in the green region. Also, the D65
source is roughly equal in intensity across the
spectrum varying by about 30% on average,
whereas the metal halide sources vary as much as
90% and contain many peaks. The Pendant source
is even more extreme in its variation across the
visible spectrum than the Iridis source, therefore,
we determined that this would be an overly
conservative comparison and that the D65/Iridis
system should compare even more favorably if the
D65/Pendant system proved satisfactory.
One of the issues that needed to be addressed was
related to the sample spacing used during the
previous year’s activities. The smoothly varying
D65 illuminant and the smoothly varying spectral
reflectance associated with textile colorants, required a sample spacing of no more than 10 nm to adequately represent the
signals in the ITC system. If we sample the metal halide source at 10 nm intervals then important spectral detail is not
maintained. We therefore increased the sampling by a factor of 10 (i.e., 1 nm samples) on all data processing in the ITC
system. The improvement in spectrum fidelity
for the metal halide illumination source is
shown in Fig. 9.
4. RESULTS
The goal of the effort described herein
was to determine the efficacy of using the
metal halide illumination source to estimate
the accuracy of a color difference
measurement on textile products. Note that we
seek to estimate color differences (e.g.,
?ECMC), i.e., the variation of a target color over
time from its original value, and not to
compare absolute color measurements. To
accomplish this goal we needed to determine a
measurement technique to compare difference
values obtained using POCS. A common technique for estimating colorimetric values from RGB color systems is to use a
linear regression approach. Since we have a spectrometer embedded in the ITC system, it was possible for us to obtain
standard CIE tristimulus measurements (i.e., XYZ) from a series of PantoneTM color samples independently of the RGB
values put out by the color camera. Recall that the tristimulus vector was given in Eq. (1) by d = XE?, where X contains the
tristimulus response curves for the CIE standard 10o observer, E contains the illumination spectrum, and ? is the spectral
reflectance of the color sample. A linear regression approach to transforming RGB to XYZ involves determining a
transformation matrix, M, such that d = Mc, where c is a three-element vector that contains the RGB values that correspond
to the XYZ values in d. To determine M, a series of color samples are measured such that a series of equations are obtained,
[d0 d1 …dN-1] = M [c0 c1 …cN-1], for N-1

---

blue, brown, tan, rust, cool gray, and warm gray. The results of a comparison between POCS and the linear regression
method are shown in Figs. 10 and 11 for the D65 and metal halide illuminants respectively. Note that each of the
independent values along the bottom axis represent a shade pair, e.g., “green” refers to a pair of green PantoneTM color tiles
of slightly varying shade. The solid line in these graphs represents a spectrometer color difference measurement that was
taken to be the “standard” for this comparison. The standard in this case was a HunterLab Miniscan handheld
spectrophotometer with its own self-contained D65 illuminant source. In Fig. 10, the regression and POCS results were both
derived from the RGB values collected with the ITC under D65 illumination. In Fig. 11, the regression and POCS results
were derived from the metal halide illuminant.
Sol Source Illuminant
8.00
Regression Color Difference
7.00
POCS Color Difference
Spectrometer Color Difference
6.00
5.00
mc
4.00
?
Ec
3.00
2.00
1.00
0.00
e
n
lu
w
st
tan
ay
ay
reen
b
ru
g
ro
gr
gr
b
ol
Color Pair
co
arm
w
Figure 10 – Results of a shade difference comparison between the POCS technique, the
linear regression method, and a standard spectrometer reading for D65 illumination.
Metal Halide Illuminant
8.00
Regression Color Difference
7.00
POCS Color Difference
Spectrometer Color Difference
6.00
5.00
c
4.00
?
Ecm
3.00
2.00
1.00
0.00
Color Pair
Figure 11 - Results of a shade difference comparison between the POCS technique, the linear
regression method, and a standard spectrometer reading for metal halide illumination.

---

A calculation of the average magnitude difference between the spectrometer (standard) and the D65 data shown in Fig. 10
reveals the average error across all color pairs is ± 0.53 ?ECMC units for the regression approach, while the POCS method
resulted in an average error of ± 0.95 ?ECMC units. For the metal halide illumination data shown in Fig. 11, the average error
across all color pairs is ± 0.38 ?ECMC units for the regression approach, while the POCS method resulted in an average error
of ± 0.48 ?ECMC units. Although the POCS results appear slightly less accurate then the regression technique, the difference
between the two approaches is within the statistical significance of the numbers indicating that their performance is similar.
The advantage to POCS comes from the ease of calibration and maintenance of the instrument in the field as will be
described in the conclusions below.
Textile Color Tests
A final set of tests are reported in this section to
determine the difference in colors from the beginning of a textile
beginning of run
run to the end of a run for approximately 1,500 feet of printed
fabric. The quantities of interest in an on-line measurement are
1
2
3
repeat
related to the side-center-side color difference values in the cross-
length
web direction and the beginning-of-run and end-of-run values in
the web direction. Figure 12 shows the layout of positions on the
fabric that were measured for this experiment.
e
e
d
t
er
d
si
si
There were three color samples that were evaluated during testing:
cen
a solid green background that covered large areas of the textile
tion
pattern, and smaller individual pink and blue regions whose object
o
areas where no larger than 0.25 inch in diameter. In the case of
the green background color, a laboratory spectrophotometer was
used to determine a “standard” for comparison. We also used the
PC spectrometer to measure a secondary “standard” that could be
web m
used with the smaller pink and blue regions. The
spectrophotometer has a integrating field of view of approximately
1 inch and therefore encompasses too large an area for the smaller
4
5
6
samples to be accurately measured. The PC spectrometer, on the
other hand, can be focused to view a small region of the surface.
end of run
Figures 13, 14, and 15 show the results of comparing different
Figure 12 – Layout of textile fabric showing the side
positions across the web for the green, pink, and blue color
(1,4) center (2,5), side (3,6), and beginning of run
samples respectively. In Fig. 13, a plot containing both the
(1,2,3), and end of run (4,5,6) positions on the fabric
spectrophotometer and spectrometer are shown. Note the
web
variability across the different detectors that is inherent in the
color measurement process even between two spot detectors.
When attempting to make absolute color difference measurements across different color measuring equipment, accurate
agreement is difficult to obtain. Table I below summarizes the errors across the textile samples for the D65 and metal halide
illuminations when compared to the PC spectrometer standard. For the green sample a comparison is also given for the
spectrophotometer standard. Note overall for the PC spectrometer standard that the average ?ECMC error under D65
illumination is ± 0.29 while this error is ± 0.49 under metal halide illumination.
Table I – Summary of error across all textile samples versus illumination source. Error values are
reported in ?ECMC units.
Spectrometer
Spectrometer
Spectrophoto-
Spectrophoto-
vs. POCS Sol
vs. POCS metal
meter vs. POCS
meter vs. POCS
Source
halide
Sol Source
metal halide
Green (Fig. 13)
± 0.3
± 0.41
± 0.35
± 0.27
Pink (Fig. 14)
± 0.3
± 0.58
N/A
N/A
Blue (Fig. 15)
± 0.27
± 0.39
N/A
N/A

---

Textile Color Comparison (Green)
2
1.5
1
0.5
0
1,2
1,3
1,4
2,5
3,6
4,5
4,6
Color Pairs (position on web)

---