scanner gamma

[Mike Engles]

I am not sure that I see what your difficulty is here. The output of
the scanner is linear and, if there was a linear way to display that,
then there would be no need for gamma, other than to correct for the
perception of quantisation noise throughout the range - which could be
achieved with sufficient bit depth in any case. The gamma compensation
is required because the CRT response is non-linear, not because the
image is too dark without it. If this was just a case of compensating
for image brightness, a linear offset to the levels or gain would be
sufficient. Most of this is explained in a lot more detail the FAQ that
I referenced earlier in the thread.
--
Kennedy
Yes, Socrates himself is particularly missed;
A lovely little thinker, but a bugger when he's pissed.
Python Philosophers (replace 'nospam' with 'kennedym' when replying)

Hello

Exactly

The problem I have is with the idea that this inverse gamma is primarily
for reasons of visual perception and that it is a happy circumstance
that a CRT has the same necessity.

This was also part of your statement and is the thesis of Mr Poynton and
others.

As I say I find that in the first 3 examples I gave no correction to
visual perception is needed, but for some reason when viewing the output
of a scanner that reads light linearly, a correction has to be made for
the reasons other than that the CRT is nonlinear.

I cannot accept using my logic that human visual perception is non
linear,or at least to the extent that is claimed.
Mike Engles

 

[Mike Engles]

That is also my understanding Mike, the CRT output response should be
corrected to be perceived as linear intensity, and not because gamma matches
any eye perception response curve. A CRT is simply not a linear device (a
field effect at the CRT grid), and so this inverse gamma formula is invented
to first oppositely distort the input data so the CRT response will appear
linear, meaning linear output intensity at the tube phosphor, simply to
match the linear pre-encoded input data intensity from slide or scene.


Not wishing to speak for Kennedy, but I read it supposing he added the final
"in the eye" for that same reason... The CRT response should be linear so the
eye can perceive the image reproduction as linear. The eye/brain and the CRT
do have their non-linear ways of working, but from a CRT result point of view,
it doesnt really matter how the eye does that, nor how the CRT does it, so
long as the CRT response is corrected to show linear intensity, meaning it
looks like the slide or scene that created the data.

The human eye is said to have a 1/3 power law response, and inverse gamma of
1/2.2 is coincidentally similar, but this doesnt change WHY the CRT is
corrected to be linear (not to match the eye response). It would not matter if
the two formulas might have been entirely different somehow, linear intensity
is still the correct CRT response for the eye to view (to match slide or
scene). However this similarity surely must work out as convenient to use the
same gamma shape curve (Curve tool for example) to edit an image arbitrarily
brighter or darker in a satisfactory way to look natural to our eye. That
seems great, but in my opinion, that is the extent of it.

Gamma also has the strong side benefit of compressing the digital data so that
8 bits can be sufficient to show it somewhat like the eye/brain can perceive
it. That compression discards the right data that we wouldnt notice anyway
(discards more bright tones of which we have too many to be useful to the
eye/brain), and retains more important sparse dark data steps that we might
notice (the idea that 100 intensity steps differing by 1% are seen). Which is
fantastic, but it is just clutter in explanations, as it also is NOT WHY gamma
is done. Gamma is instead done to correct the CRT response to be linear, and
has been done from the earliest days of analog television shown on CRT.

Hello

So are you in sffect saying that the eye does not need correction, but
the signal fed to the CRT does. If we had a display device that was
linear, would we need to apply inverse gamma to it for reasons of visual
perception.

Mike Engles

 

[Kennedy McEwen]

The problem I have is with the idea that this inverse gamma is primarily
for reasons of visual perception and that it is a happy circumstance
that a CRT has the same necessity.

This was also part of your statement and is the thesis of Mr Poynton and
others.

As I say I find that in the first 3 examples I gave no correction to
visual perception is needed, but for some reason when viewing the output
of a scanner that reads light linearly, a correction has to be made for
the reasons other than that the CRT is nonlinear.

I cannot accept using my logic that human visual perception is non
linear,or at least to the extent that is claimed.

However, using your logic, there is no indication of whether the human
visual system is a linear, logarithmic or some other response curve to
light intensity. All your earlier three examples demonstrate is that
when the light intensity is represented in a similar (ignoring
flattening of the film response at extremes) manner to compensate for
the display instrument response that the human observer sees it in the
same way as the original scene - but the eye could have any response
curve you like and achieve the same effect. Further tests are required
to establish that actual response of the human visual system to light
intensity. To test linearity, you need a linearly adjustable light
source - such as the observation of a fixed source through a linearly
adjustable aperture stop.

So start by assuming that the human eye has a response which could be
any form linear or non-linear - but lets imagine what would happen if
the response was actually in the same direction as the CRT gamma. This
time, the CRT gamma correction would be in the opposite direction to the
linearising function required for the eye. With a continuously variable
source, this would still produce an output which matched the real world
- a linear grey ramp would appear just as linear on the CRT as it does
on paper or in real life, such as the graduation of shadow down a
surface. So far this is, I think, in full agreement with your
expectations - correcting for the CRT merely linearises so that things
look the same as they do in the real world.

However, the next step is to see what happens with noise. An easy
example is quantisation noise, because it is fairly obvious that this is
evenly distributed throughout the linear response of the scanner. But
the correction for CRT gamma means that the noise is shaped - in
highlights the noise is compressed, while in the shadows it is expanded.
For example, applying an unmodified 2.2 gamma function to a 16-bit
linear ramp results in count 1 (ie. 1/65536th of full scale), maps to
count 424, while 2 maps to count 581 etc. In other words, at the
extreme shadows between 0 and 1 on the 16-bit scale, the gamma
correction has increased the quantisation noise by a factor of over 400.

If the eye had a non-linear response then the quantisation noise in the
linear scanner space would have a similar compression and expansion in
perceptual space. In other words, even though quantisation noise is
evenly distributed in linear space it would, and does, appear worse in
shadows and better in highlights or vice versa - depending on the
non-linearity of the eye.

But, because the gamma of the CRT is similar to the inverse of the eye
response, that increased quantisation caused by the gamma correction
does not look too out of place. If, however, the eye response was the
same as the CRT then the appearance of the quantisation noise in linear
space would actually be exaggerated by the gamma correction.
Fortunately, the eye response has a power law in the opposite direction
as the CRT. Even more fortunately, it is *almost* the same, so that the
compression and expansion of noise at either end of the scale to correct
for the gamma of the CRT compensates to a degree for its perceptual
distribution. This is true for other noise sources, not just
quantisation noise, which is why gamma was seen as a good thing long
before the era of digital video signals.

I have the misfortune of having learned this the hard way. Many years
ago I designed an early prototype thermal imager - a camera that
responds to infra-red radiation that is about 20 times the wavelength of
the light that your eyes see. Obviously this is invisible radiation,
and it is emitted by all objects above absolute zero. At the time, I
ignored gamma - not through oversight, but because I could not see its
function. After all, I thought, gamma only corrects the response of the
CRT to that the light output matches the light input to a normal video
camera. Since the thermal imager was responding to invisible light,
there was no input linearity to match - so gamma was essentially an
irrelevant concept. The problem arose when the system first operated.
Thermal imagers usually have control over contrast and brightness to map
the temperatures in the scene to the black and white video levels.
However, no matter how these linear controls were set, the image always
looked as if it had quantised highlights or too dark. The clue came
through inverting the response - changing hot to map to black and cold
to map to white - when exactly the same defect, with exactly the same
polarity was observed. Fortunately, this was only a prototype unit and
designed for ease of modification, so a gamma LUT was rapidly produced
that solved the problem. A production derivative of that prototype
became the world's first helmet mounted thermal imager for firemen, and
your local fire department probably has one in their truck right now,
but it demonstrated to me that even when there is no input image to
perceptually reference to, gamma still plays an important function -
especially when bits are scarce.

 

[Wayne Fulton]

So are you in sffect saying that the eye does not need correction, but
the signal fed to the CRT does. If we had a display device that was
linear, would we need to apply inverse gamma to it for reasons of visual
perception.

Yes, the reason we need the CRT correction factor called gamma is to provide a
linear output reproduction from the non-linear CRT to match the same linear
intensity scale of the original slide or scene (not to say that printers dont
also need some degree of this, but less so).

The human eye is the probable destination, but it is not a necessary part -
any linear brightness measuring detectors would serve as well, film for
example. This is how film recorders work, placing unexposed film up against a
small CRT display to expose it to create a film copy. But that CRT still also
requires gamma correction too. The point is, a human eye is not a necessary
component of the system, it is instead about linearity.

Poynton is perhaps slow careful reading sometimes, but he does not say
different. His #5 What is Gamma certainly does not say different. He
clearly says gamma is to correct the CRT non-linearity.

However, he does continue, elaborating how and why 8 bits is sufficient with
gamma when 8 bits is not otherwise sufficient for a linear display (in video
terminology, the term linear simply usually means before gamma correction is
added). That may be the confusion factor about the eye, because that 8 bit
reason is due to how the eye sees things (1% rule, etc). However the eye is
not the reason for gamma - gamma is to correct CRT non-linarity. Instead he
is just saying due to the eyes characteristics, 8 bits with gamma encoding
works out really well as a fortuitous circumstance, since gamma is required
anyway, to correct the CRT response.

If somehow it hadnt worked out, we would have surely just used 16 bits, but we
would still need gamma for linear output from the CRT.

 

[Kennedy McEwen]

If somehow it hadnt worked out, we would have surely just used 16 bits

That is true, but it would have delayed the introduction of digital
video by over 20 years. Around the late 70's/early 80's when Quantel
Light Boxes were just beginning to perform their magic digital tricks in
studios around the world, the state-of-the-art video rate analogue to
digital converter was something like the TRW TDC1007J. A monolithic
8-bit, 20MHz chip in a huge double width 64-pin pack with an integral
heat sink that got hot enough to fry an egg on when run at full
specification. It was about another 12 years or so till similar speed
12-bit devices hit the streets and it is fairly recently that we have
had 16-bit video rate devices. Poynton doesn't exaggerate when he says
we have been lucky to be designed the opposite way to CRTs. ;-)

 

[Mike Engles]

However, using your logic, there is no indication of whether the human
visual system is a linear, logarithmic or some other response curve to
light intensity. All your earlier three examples demonstrate is that
when the light intensity is represented in a similar (ignoring
flattening of the film response at extremes) manner to compensate for
the display instrument response that the human observer sees it in the
same way as the original scene - but the eye could have any response
curve you like and achieve the same effect. Further tests are required
to establish that actual response of the human visual system to light
intensity. To test linearity, you need a linearly adjustable light
source - such as the observation of a fixed source through a linearly
adjustable aperture stop.

So start by assuming that the human eye has a response which could be
any form linear or non-linear - but lets imagine what would happen if
the response was actually in the same direction as the CRT gamma. This
time, the CRT gamma correction would be in the opposite direction to the
linearising function required for the eye. With a continuously variable
source, this would still produce an output which matched the real world
- a linear grey ramp would appear just as linear on the CRT as it does
on paper or in real life, such as the graduation of shadow down a
surface. So far this is, I think, in full agreement with your
expectations - correcting for the CRT merely linearises so that things
look the same as they do in the real world.

However, the next step is to see what happens with noise. An easy
example is quantisation noise, because it is fairly obvious that this is
evenly distributed throughout the linear response of the scanner. But
the correction for CRT gamma means that the noise is shaped - in
highlights the noise is compressed, while in the shadows it is expanded.
For example, applying an unmodified 2.2 gamma function to a 16-bit
linear ramp results in count 1 (ie. 1/65536th of full scale), maps to
count 424, while 2 maps to count 581 etc. In other words, at the
extreme shadows between 0 and 1 on the 16-bit scale, the gamma
correction has increased the quantisation noise by a factor of over 400.

If the eye had a non-linear response then the quantisation noise in the
linear scanner space would have a similar compression and expansion in
perceptual space. In other words, even though quantisation noise is
evenly distributed in linear space it would, and does, appear worse in
shadows and better in highlights or vice versa - depending on the
non-linearity of the eye.

But, because the gamma of the CRT is similar to the inverse of the eye
response, that increased quantisation caused by the gamma correction
does not look too out of place. If, however, the eye response was the
same as the CRT then the appearance of the quantisation noise in linear
space would actually be exaggerated by the gamma correction.
Fortunately, the eye response has a power law in the opposite direction
as the CRT. Even more fortunately, it is *almost* the same, so that the
compression and expansion of noise at either end of the scale to correct
for the gamma of the CRT compensates to a degree for its perceptual
distribution. This is true for other noise sources, not just
quantisation noise, which is why gamma was seen as a good thing long
before the era of digital video signals.

I have the misfortune of having learned this the hard way. Many years
ago I designed an early prototype thermal imager - a camera that
responds to infra-red radiation that is about 20 times the wavelength of
the light that your eyes see. Obviously this is invisible radiation,
and it is emitted by all objects above absolute zero. At the time, I
ignored gamma - not through oversight, but because I could not see its
function. After all, I thought, gamma only corrects the response of the
CRT to that the light output matches the light input to a normal video
camera. Since the thermal imager was responding to invisible light,
there was no input linearity to match - so gamma was essentially an
irrelevant concept. The problem arose when the system first operated.
Thermal imagers usually have control over contrast and brightness to map
the temperatures in the scene to the black and white video levels.
However, no matter how these linear controls were set, the image always
looked as if it had quantised highlights or too dark. The clue came
through inverting the response - changing hot to map to black and cold
to map to white - when exactly the same defect, with exactly the same
polarity was observed. Fortunately, this was only a prototype unit and
designed for ease of modification, so a gamma LUT was rapidly produced
that solved the problem. A production derivative of that prototype
became the world's first helmet mounted thermal imager for firemen, and
your local fire department probably has one in their truck right now,
but it demonstrated to me that even when there is no input image to
perceptually reference to, gamma still plays an important function -
especially when bits are scarce.
--
Kennedy
Yes, Socrates himself is particularly missed;
A lovely little thinker, but a bugger when he's pissed.
Python Philosophers (replace 'nospam' with 'kennedym' when replying)

Hello

Thanks for the explantion.

So are saying that even if we had a linear display, showing linear
scanned 16 bit images, we would still need to apply a inverse gamma to
either the image or the display. That is really what I have difficulty
understanding. It seesm to go against common sense.

Mike Engles

 

[Kennedy McEwen]

Thanks for the explantion.

So are saying that even if we had a linear display, showing linear
scanned 16 bit images, we would still need to apply a inverse gamma to
either the image or the display. That is really what I have difficulty
understanding. It seesm to go against common sense.

At 16-bits I doubt that you would probably need it. However at lower
bit depths you certainly would, even with a linear display. In fact,
LCDs are pretty linear display devices, as are the upcoming OLED
systems. Having used early OLED prototypes I can tell you that a form
of gamma correction is very beneficial to getting a good images,
especially of a noisy signal. In general you need a form of gamma at
any bit depth to make best use of the available bits. Of course,
without the CRT gamma to correct, the actual gamma function will be much
less than 2.2-2.5. Poynton actually makes the point (no pun) that gamma
would have needed to be invented even if CRTs had been perfectly linear.
Unfortunately, I haven't seen any of his texts (though I ma sure some
must exist somewhere) that go further than this blanket statement, but
it is to optimise available bit depth.

However, don't get too hung up on this - the main reason for gamma is to
linearise CRTs.

 

[Wayne Fulton]

and it is fairly recently that we have
had 16-bit video rate devices. Poynton doesn't exaggerate when he says
we have been lucky to be designed the opposite way to CRTs. ;-)

But now that the chips have finally become inexpensive, I wonder if 64 bit
Windows will support 16 bit images? Any benefit seems small, and gamers
would probably always imagine a speed problem. PC movie formats already
intentionally give up so much anyway (frame size, lossy compression), that 16
bits seems like a solution without a problem. However probably someone will
do it, and then everyone will have to.

 

[Mike Engles]

At 16-bits I doubt that you would probably need it. However at lower
bit depths you certainly would, even with a linear display. In fact,
LCDs are pretty linear display devices, as are the upcoming OLED
systems. Having used early OLED prototypes I can tell you that a form
of gamma correction is very beneficial to getting a good images,
especially of a noisy signal. In general you need a form of gamma at
any bit depth to make best use of the available bits. Of course,
without the CRT gamma to correct, the actual gamma function will be much
less than 2.2-2.5. Poynton actually makes the point (no pun) that gamma
would have needed to be invented even if CRTs had been perfectly linear.
Unfortunately, I haven't seen any of his texts (though I ma sure some
must exist somewhere) that go further than this blanket statement, but
it is to optimise available bit depth.

However, don't get too hung up on this - the main reason for gamma is to
linearise CRTs.
--
Kennedy
Yes, Socrates himself is particularly missed;
A lovely little thinker, but a bugger when he's pissed.
Python Philosophers (replace 'nospam' with 'kennedym' when replying)

Hello

Thanks for the confirmation. I have always felt this,that there would be
a need for a small correction, but not the huge inverse 2.5 or so that
is used for CRTs. Perhaps it is the difference between inverse 1/3 and
inverse 2.5, about inverse 1.2

Mike Engles

 

[Mike Engles]

Hello

Thanks for the confirmation. I have always felt this,that there would be
a need for a small correction, but not the huge inverse 2.5 or so that
is used for CRTs. Perhaps it is the difference between inverse 1/3 and
inverse 2.5, about inverse 1.2

Mike Engles

Hello

Another question comes to mind. Are these new LCD displays actually
linear, or are the makers tricking about with them so that they emulate
CRTs.
That would mean that the way we work does not have to change much.

Poynton also sorts of implies that.

Mike Engles

 

[Kennedy McEwen]

Another question comes to mind. Are these new LCD displays actually
linear, or are the makers tricking about with them so that they emulate
CRTs.
That would mean that the way we work does not have to change much.

Yes, the displays are linear, but the manufacturers build a LUT into
then that makes them appear to have a similar gamma to a CRT. That
makes them convenient to use, but it is really a sub-optimal solution
because the introduction of gamma results in excessive quantisation at
both ends of the brightness range - at one end when it is introduced and
at the other end when it is corrected. So even though you drive them at
8-bits, they probably have 10-12bits internally just to support this
compatibility.

 

[Mike Engles]

Yes, the displays are linear, but the manufacturers build a LUT into
then that makes them appear to have a similar gamma to a CRT. That
makes them convenient to use, but it is really a sub-optimal solution
because the introduction of gamma results in excessive quantisation at
both ends of the brightness range - at one end when it is introduced and
at the other end when it is corrected. So even though you drive them at
8-bits, they probably have 10-12bits internally just to support this
compatibility.
--
Kennedy
Yes, Socrates himself is particularly missed;
A lovely little thinker, but a bugger when he's pissed.
Python Philosophers (replace 'nospam' with 'kennedym' when replying)

Hello

If it is not too boring for you, I am still puzzled by the necessity for
inverse gamma, when viewing 16 bit images on a truly linear display,
apart from correcting for a not quite ideal exposure.

It seems to me that most scanners apply inverse gamma after the A/D, so
any noise is now digitised. How can inverse gamma in this instant
improve noise or the use of bits? I can see that if inverse gamma is
applied in a analogue domain, then passed through a A/D, there would be
a benefit. It would act like a noise reduction system.

Inverse gamma added to a already digitised image would just make the
noise in the darker end more visible, which is exactly what happens.
This is necessary because most images are not ideal and need further
correction, even though they have been linearised to correct for display
on a CRT.

Mike Engles

 

[Kennedy McEwen]

If it is not too boring for you, I am still puzzled by the necessity for
inverse gamma, when viewing 16 bit images on a truly linear display,
apart from correcting for a not quite ideal exposure.

It seems to me that most scanners apply inverse gamma after the A/D, so
any noise is now digitised. How can inverse gamma in this instant
improve noise or the use of bits? I can see that if inverse gamma is
applied in a analogue domain, then passed through a A/D, there would be
a benefit. It would act like a noise reduction system.

Inverse gamma added to a already digitised image would just make the
noise in the darker end more visible, which is exactly what happens.
This is necessary because most images are not ideal and need further
correction, even though they have been linearised to correct for display
on a CRT.

I thought I had already explained this to your satisfaction Mike, but it
seems you aren't quite convinced yet, so lets take a different approach
and see if that helps you get the principle.

There are a lot of things in nature which can be measured in different
ways and each method produces a different transfer function. In fact,
it is so unusual for a parameter to exhibit the same characteristics
when measured by completely independent means that some people have
spent entire lives trying to find the common link between the methods.
One classic example is something as simple as mass. Isaac Newton
predicted that the gravitational force attracting two objects was a
linear function of their mass and inversely proportional to the square
of the distance between them. So to measure the mass of one object you
could just measure the attractive force towards another at a known
distance - which is effectively what you do every time you step onto the
bathroom scales and weigh yourself: you measure the attractive force
between you and the earth at a known distance from its centre.

But there is another way of determining mass - from inertia. You know
that objects with more mass require more force to get them moving,
change their direction and stop them. This is also one of Newton's
equations, in fact it is Newton's 2nd Law of Motion and defines force in
terms of mass and vice versa: force = mass times acceleration. So you
have the same two properties measured relative to each other, force and
mass, that have an exact linear relationship to each other, and again
have a means of determining mass directly by measuring the force under a
known acceleration - such as in a circular orbit or a centrifuge.

But what connects the mass derived from gravitational force measurements
to the mass derived from inertial force measurements? Why is the mass
exactly the same from both techniques? It suggests that gravity and
inertia are some manifestation of the same underlying principle - as my
old physics lecturer used to call it "the unresolved mystery of
classical mechanics".

It is easy to see why the mass works out to have the same value - in the
inertial mass the connection between force and acceleration is simply
unity because that was the definition of force, so it is measured in
Newtons. But when he came to gravitation he had to 'invent' a
relationship between mass and force and used a constant, Newtons
Gravitational Constant, so that the force and mass were in the correct
scale to each other over the distances he was concerned with, such as
between the sun, earth and moon. Yet that is a fixed constant meaning
that inertial mass and gravitational mass are perfectly linear to each
other over enormous (literally astronomic) scales. Experiments have
shown that they are linear to better than 1 part in a billion over a
range of more than 10 to the power 23. What makes gravitational mass
exactly the same as inertial mass over such huge ranges? I doubt that
even Stephen Hawking can explain it and, as I mentioned earlier, people
have spent their entire lives since Newton's time trying to do just
that.

Even in hard physics it is rare to find parameters that are perfectly
linear when measured in different ways. Temperature, for example, can
be measured in many ways, including the expansion of a gas or liquid, as
in a thermometer. Or it can be measured by the resistance of a
semiconductor, as in an electronic thermometer. Or it can be measured
by the amount of radiation emitted from a black body or numerous other
means. All of these measurement techniques only give the same
temperature over a limited range because temperature has a different
transfer function, a different non-linear curve relating it to each of
the parameters that are actually measured. For example, the expansion
of gas or liquid with temperature becomes extremely non-linear near the
phase transitions of the material, the semiconductor resistance becomes
very non-linear at low absolute temperatures and also again at high
temperatures while emitted radiation has to be corrected by a fourth
power "gamma" to give temperature.

All of this might seem to have nothing to do with the question you ask,
but it has a common underlying question. With inorganic measurements it
is unusual, to say the least, for physically unconnected measurement
techniques to give the same response between parameters. Imagine how
much less likely it is to find such a perfect match between inorganic
and biological methods. Which is where the underlying question comes
in: What is linearity? This isn't just an abstract point of little
importance. Since it is so uncommon for measurement techniques to
provide the same transfer function unless they are somehow related by an
underlying principle, it follows that any two parameters can only have a
linear relationship when measured according to certain rules - any other
measurement of the two parameters will inevitably yield a non-linear
relationship. So which is the "correct" method of measuring two
parameters to determine their mutual linearity? Convention - and it is
the need for that convention and linearity that is behind the constant
redefinition of fundamental parameters as science develops. That is why
a year is no longer 300 days, or 365 days, or 365.242 days or that a
second is no longer 1/86400th of a day - not to be more exact or precise
in scale, but to be more precise in relative terms, in linear and power
relationships with other parameters according to agreed rules for the
measurement system used.

So coming back to your original question about bits and light measured
by a scanner. We say that the scanner is a very linear device because
it produces an output that is linear according to the agreed rules of
the measurement of light in the SI, and other, systems. Within a large
range, the output voltage of the CCD is directly proportional to the
number of photons incident on the CCD pixel. That voltage is then
digitised by another fairly linear device, the analogue to digital
convertor. So the output data from the scanner is a linear function of
the light intensity.

But your eyes don't contain CCDs and ADCs, but biological light
receptors and ionic transfer of the electrical signal to your brain to
produce a sensation of brightness. It would be quite incredible if your
eyes had the same transfer function as the CCD - and we know from
precise measurements that the sensation of brightness is approximately a
logarithmic function of intensity, not a linear one at all.

So, in terms of scanners, displays and eyes, what is meant by "linear"?
We know what is meant in measurement terms, but is that what matters? As
mentioned above, those rules for measurement are only defined by
convention. If human beings were consistent in their perception then
would it not be better to define light intensity as a linear function of
the sensation of brightness perceived by them? After all, it is what
you see in the image and in the scene that matters. Of course if you
define light intensity in this way then the scanner becomes *non-linear*
- and the data produced by the ADC would *not* be equally spaced
intensity levels any longer.

And that really is what is at the crux of your question - the difference
between what you see and what your scanner measures. If you put a
perfectly linear display on the output of your scanner, the output will
look roughly the same as the original image. However, because you
perceive the light in a different way from the scanner, the discrete
light levels produced by the linear quantisation do not appear to be
linearly spaced perceptually. A gamma correction is necessary to make
them appear so. So while the gamma correction can be argued to be
making the quantisation noise worse in certain luminance ranges, such as
the shadows, it is actually making it the same throughout the range as
perceived by your eyes.

 

[Mike Engles]

I thought I had already explained this to your satisfaction Mike, but it
seems you aren't quite convinced yet, so lets take a different approach
and see if that helps you get the principle.

There are a lot of things in nature which can be measured in different
ways and each method produces a different transfer function. In fact,
it is so unusual for a parameter to exhibit the same characteristics
when measured by completely independent means that some people have
spent entire lives trying to find the common link between the methods.
One classic example is something as simple as mass. Isaac Newton
predicted that the gravitational force attracting two objects was a
linear function of their mass and inversely proportional to the square
of the distance between them. So to measure the mass of one object you
could just measure the attractive force towards another at a known
distance - which is effectively what you do every time you step onto the
bathroom scales and weigh yourself: you measure the attractive force
between you and the earth at a known distance from its centre.

But there is another way of determining mass - from inertia. You know
that objects with more mass require more force to get them moving,
change their direction and stop them. This is also one of Newton's
equations, in fact it is Newton's 2nd Law of Motion and defines force in
terms of mass and vice versa: force = mass times acceleration. So you
have the same two properties measured relative to each other, force and
mass, that have an exact linear relationship to each other, and again
have a means of determining mass directly by measuring the force under a
known acceleration - such as in a circular orbit or a centrifuge.

But what connects the mass derived from gravitational force measurements
to the mass derived from inertial force measurements? Why is the mass
exactly the same from both techniques? It suggests that gravity and
inertia are some manifestation of the same underlying principle - as my
old physics lecturer used to call it "the unresolved mystery of
classical mechanics".

It is easy to see why the mass works out to have the same value - in the
inertial mass the connection between force and acceleration is simply
unity because that was the definition of force, so it is measured in
Newtons. But when he came to gravitation he had to 'invent' a
relationship between mass and force and used a constant, Newtons
Gravitational Constant, so that the force and mass were in the correct
scale to each other over the distances he was concerned with, such as
between the sun, earth and moon. Yet that is a fixed constant meaning
that inertial mass and gravitational mass are perfectly linear to each
other over enormous (literally astronomic) scales. Experiments have
shown that they are linear to better than 1 part in a billion over a
range of more than 10 to the power 23. What makes gravitational mass
exactly the same as inertial mass over such huge ranges? I doubt that
even Stephen Hawking can explain it and, as I mentioned earlier, people
have spent their entire lives since Newton's time trying to do just
that.

Even in hard physics it is rare to find parameters that are perfectly
linear when measured in different ways. Temperature, for example, can
be measured in many ways, including the expansion of a gas or liquid, as
in a thermometer. Or it can be measured by the resistance of a
semiconductor, as in an electronic thermometer. Or it can be measured
by the amount of radiation emitted from a black body or numerous other
means. All of these measurement techniques only give the same
temperature over a limited range because temperature has a different
transfer function, a different non-linear curve relating it to each of
the parameters that are actually measured. For example, the expansion
of gas or liquid with temperature becomes extremely non-linear near the
phase transitions of the material, the semiconductor resistance becomes
very non-linear at low absolute temperatures and also again at high
temperatures while emitted radiation has to be corrected by a fourth
power "gamma" to give temperature.

All of this might seem to have nothing to do with the question you ask,
but it has a common underlying question. With inorganic measurements it
is unusual, to say the least, for physically unconnected measurement
techniques to give the same response between parameters. Imagine how
much less likely it is to find such a perfect match between inorganic
and biological methods. Which is where the underlying question comes
in: What is linearity? This isn't just an abstract point of little
importance. Since it is so uncommon for measurement techniques to
provide the same transfer function unless they are somehow related by an
underlying principle, it follows that any two parameters can only have a
linear relationship when measured according to certain rules - any other
measurement of the two parameters will inevitably yield a non-linear
relationship. So which is the "correct" method of measuring two
parameters to determine their mutual linearity? Convention - and it is
the need for that convention and linearity that is behind the constant
redefinition of fundamental parameters as science develops. That is why
a year is no longer 300 days, or 365 days, or 365.242 days or that a
second is no longer 1/86400th of a day - not to be more exact or precise
in scale, but to be more precise in relative terms, in linear and power
relationships with other parameters according to agreed rules for the
measurement system used.

So coming back to your original question about bits and light measured
by a scanner. We say that the scanner is a very linear device because
it produces an output that is linear according to the agreed rules of
the measurement of light in the SI, and other, systems. Within a large
range, the output voltage of the CCD is directly proportional to the
number of photons incident on the CCD pixel. That voltage is then
digitised by another fairly linear device, the analogue to digital
convertor. So the output data from the scanner is a linear function of
the light intensity.

But your eyes don't contain CCDs and ADCs, but biological light
receptors and ionic transfer of the electrical signal to your brain to
produce a sensation of brightness. It would be quite incredible if your
eyes had the same transfer function as the CCD - and we know from
precise measurements that the sensation of brightness is approximately a
logarithmic function of intensity, not a linear one at all.

So, in terms of scanners, displays and eyes, what is meant by "linear"?
We know what is meant in measurement terms, but is that what matters? As
mentioned above, those rules for measurement are only defined by
convention. If human beings were consistent in their perception then
would it not be better to define light intensity as a linear function of
the sensation of brightness perceived by them? After all, it is what
you see in the image and in the scene that matters. Of course if you
define light intensity in this way then the scanner becomes *non-linear*
- and the data produced by the ADC would *not* be equally spaced
intensity levels any longer.

And that really is what is at the crux of your question - the difference
between what you see and what your scanner measures. If you put a
perfectly linear display on the output of your scanner, the output will
look roughly the same as the original image. However, because you
perceive the light in a different way from the scanner, the discrete
light levels produced by the linear quantisation do not appear to be
linearly spaced perceptually. A gamma correction is necessary to make
them appear so. So while the gamma correction can be argued to be
making the quantisation noise worse in certain luminance ranges, such as
the shadows, it is actually making it the same throughout the range as
perceived by your eyes.
--
Kennedy
Yes, Socrates himself is particularly missed;
A lovely little thinker, but a bugger when he's pissed.
Python Philosophers (replace 'nospam' with 'kennedym' when replying)

Hello

Thanks for taking the trouble to write such a detailed answer.
The problem is that it makes me ask more questions.
I can sort of see what you are saying, but there do seem to be some
contradictions.

We have a linear display and a linear sensor.You say that the output
will look roughly the same as the original image.

Question 1
To what and to whom will it look roughly the same.

Question 2
What is the difference between "It will look roughly the same, but we
will not perceive it as such"? What inverse gamma do we need to perceive
linearity? Is it a fixed amount or is it arbitrary?

Question 3
Now it is stated that the eye/brain uses a power law of the order of
inverse 1/3 and that this response is the opposite of that of a CRT
which is 2.5. Does this not mean that the eye/brain already provides its
own inverse gamma, which is greater than that need to linearise a CRT,
with a bit to spare.

Question 4
Does the eye/brain USE or REQUIRE a inverse 1/3.

Question 5
What is the role of a hardware calibrator? Is it a way of creating
absolute linearity or perceptual linearity?

It looks like we need 2 inverse gammas, one to linearise a CRT and one
for perception, but at the moment they have been combined.


I have made a plot of the various gamma curves, as I see them.
Do they make sense to you?


Forgive me if my questions make me look extremely obtuse.

Mike Engles

 

[Mike Engles]

Hello

Thanks for taking the trouble to write such a detailed answer.
The problem is that it makes me ask more questions.
I can sort of see what you are saying, but there do seem to be some
contradictions.

We have a linear display and a linear sensor.You say that the output
will look roughly the same as the original image.

Question 1
To what and to whom will it look roughly the same.

Question 2
What is the difference between "It will look roughly the same, but we
will not perceive it as such"? What inverse gamma do we need to perceive
linearity? Is it a fixed amount or is it arbitrary?

Question 3
Now it is stated that the eye/brain uses a power law of the order of
inverse 1/3 and that this response is the opposite of that of a CRT
which is 2.5. Does this not mean that the eye/brain already provides its
own inverse gamma, which is greater than that need to linearise a CRT,
with a bit to spare.

Question 4
Does the eye/brain USE or REQUIRE a inverse 1/3.

Question 5
What is the role of a hardware calibrator? Is it a way of creating
absolute linearity or perceptual linearity?

It looks like we need 2 inverse gammas, one to linearise a CRT and one
for perception, but at the moment they have been combined.

I have made a plot of the various gamma curves, as I see them.
Do they make sense to you?

Forgive me if my questions make me look extremely obtuse.

Mike Engles

Hello

I forgot to put the link to the curves plot.

http://www.btinternet.com/~mike.engles/mike/Curves.jpg

Mike Engles

 

[Chris Cox]

That is also my understanding Mike, the CRT output response should be
corrected to be perceived as linear intensity, and not because gamma matches
any eye perception response curve. A CRT is simply not a linear device (a
field effect at the CRT grid), and so this inverse gamma formula is invented
to first oppositely distort the input data so the CRT response will appear
linear, meaning linear output intensity at the tube phosphor, simply to
match the linear pre-encoded input data intensity from slide or scene.

You are confusing two different things.

Yes, the image has to be corrected for the display to make the
intensity response linear -- but that may or may not be a gamma curve
(LCD usually is not a gamma curve).

Images are encoded using gamma curves to maximize the usage of the
available bits -- matching the human visual response gets the best use
of the bits and the human visual response is very close to a gamma
curve. But the image encoding has nothing to do with the display.

Chris

 

[Kennedy McEwen]

Thanks for taking the trouble to write such a detailed answer.
The problem is that it makes me ask more questions.
I can sort of see what you are saying, but there do seem to be some
contradictions.

We have a linear display and a linear sensor.You say that the output
will look roughly the same as the original image.

Question 1
To what and to whom will it look roughly the same.

It will have the same brightness range as the original. So peak white
will appear white, peak black will be a good black and the mid tones
will look neither too light nor too dark.

Question 2
What is the difference between "It will look roughly the same, but we
will not perceive it as such"?

The difference is the noise, including the quantisation which will not
be evenly spaced. Thus if we have inadequate bits we will begin to see
posterisation in some levels before it appears in others.

What inverse gamma do we need to perceive
linearity? Is it a fixed amount or is it arbitrary?

To be precise, the correction for the perception function is
logarithmic, and gamma is only an approximation, but it is the same
principle and over a significant range of brightness is adequate.

Question 3
Now it is stated that the eye/brain uses a power law of the order of
inverse 1/3 and that this response is the opposite of that of a CRT
which is 2.5. Does this not mean that the eye/brain already provides its
own inverse gamma, which is greater than that need to linearise a CRT,
with a bit to spare.

Eh? Last time I checked 2.5 was greater than 0.33'. ;-)

Question 4
Does the eye/brain USE or REQUIRE a inverse 1/3.

No - it is approximated by a gamma of roughly that level.

Question 5
What is the role of a hardware calibrator? Is it a way of creating
absolute linearity or perceptual linearity?

Nothing whatsoever to do with perceptual linearity - all it does is
match a device (monitor, scanner, printer etc.) to an abstract standard.

It looks like we need 2 inverse gammas, one to linearise a CRT and one
for perception, but at the moment they have been combined.

Yes, that is why you generally have a gamma of 2.2 for the monitor, not
2.5.

I have made a plot of the various gamma curves, as I see them.
Do they make sense to you?

The perceptual one looks in the wrong direction completely. It is not
an inverse gamma of 1/3, but a gamma of approximately 1/3. The
perceptual gamma is almost the inverse of the CRT.

 

[Mike Engles]

It will have the same brightness range as the original. So peak white
will appear white, peak black will be a good black and the mid tones
will look neither too light nor too dark.


The difference is the noise, including the quantisation which will not
be evenly spaced. Thus if we have inadequate bits we will begin to see
posterisation in some levels before it appears in others.


To be precise, the correction for the perception function is
logarithmic, and gamma is only an approximation, but it is the same
principle and over a significant range of brightness is adequate.

Eh? Last time I checked 2.5 was greater than 0.33'. ;-)

No - it is approximated by a gamma of roughly that level.

Nothing whatsoever to do with perceptual linearity - all it does is
match a device (monitor, scanner, printer etc.) to an abstract standard.

Yes, that is why you generally have a gamma of 2.2 for the monitor, not
2.5.
The perceptual one looks in the wrong direction completely. It is not
an inverse gamma of 1/3, but a gamma of approximately 1/3. The
perceptual gamma is almost the inverse of the CRT.
--
Kennedy
Yes, Socrates himself is particularly missed;
A lovely little thinker, but a bugger when he's pissed.
Python Philosophers (replace 'nospam' with 'kennedym' when replying)

Hello

Thanks for your reply and your time.

I have now made a curve, which I think is a 1/3 or.33 power.

I normalised the values in the curves dialogue, raised it by .33 power
and then multiplied by 255.

So 128/255=.5
..5 to power.33=.8
..8x255=204.

These are the values i have plotted.

24=116
50=149
76=171
102=188
128=204
152=215
177=226
203=236
229=246

The new image is a this link.

http://www.btinternet.com/~mike.engles/mike/Curves2.jpg

It looks very like the previous one.
Are my calculations correct?
If not I would really be grateful if you could provide me with the
correct numbers and how you derived them. I can then plot them.

The reason I used gamma 2.5 for a CRT is because it is usually stated
that the gamma can vary between 2.4 and 2.6.
Apple used about 2.6. Gamma 2.2 is used as a convenience, but does not
full correct a CRT. It is the assumed gamma of a uncalibrated monitor or
TV, if I read Mr Poynton correctly.

Mike Engles

 

[Kennedy McEwen]

Thanks for your reply and your time.

I have now made a curve, which I think is a 1/3 or.33 power.

I normalised the values in the curves dialogue, raised it by .33 power
and then multiplied by 255.

So 128/255=.5
.5 to power.33=.8
.8x255=204.

These are the values i have plotted.

24=116
50=149
76=171
102=188
128=204
152=215
177=226
203=236
229=246

The new image is a this link.

http://www.btinternet.com/~mike.engles/mike/Curves2.jpg

It looks very like the previous one.
Are my calculations correct?

Looks about right to me - and this time you have labelled it as "Eye at
1/3 power", which more correct than last time. It was the label
"inverse" that I was objecting to last time, rather than the precise
data itself, which indicated that this was a precompensation that you
were intending to apply. However, this is really a perceptual response,
rather than just the eye - I don't know that anyone has, in this case,
actually separated out the response of the eye from the interpretation
of the information it produces by the brain. It has been done in other
instances though, such as Campbell's work in determining how much
resolution is produced by the eye and how much is interpreted by the
brain - but that is a different subject. ;-)

One area does look in error though - the blacks of the eye appear to
cross over the inverse gamma of the CRTs (both 2.2 and 1.8) which they
should not. I guess this might just be an misalignment of your screen
grabs used to produce this composite image though.

Anyway, as you can see from your curves, the perceptual response is
almost the inverse of the gamma response of the CRT, so multiplying them
together gives almost a linear perceptual response. Now, obviously,
that would leave the image uncompensated for the CRT, so what your
curves are demonstrating is that by compensating for the gamma of the
CRT you are actually making best use of the available bits in the data.

You can't actually use this "eye gamma" to compensate for anything -
just in case that's where you are heading with this - the CRT pretty
much already achieves that.

If not I would really be grateful if you could provide me with the
correct numbers and how you derived them. I can then plot them.

The reason I used gamma 2.5 for a CRT is because it is usually stated
that the gamma can vary between 2.4 and 2.6.
Apple used about 2.6. Gamma 2.2 is used as a convenience, but does not
full correct a CRT. It is the assumed gamma of a uncalibrated monitor or
TV, if I read Mr Poynton correctly.

Yes, gamma can vary between CRTs - and indeed between colour channels in
the same CRT, though usually by less. However the apparent gamma
changes depending on the light level that the CRT is viewed on. Although
the real CRT is probably around 2.5, when viewed in dimly lit
surroundings, it is preferable for images to have a higher residual
gamma. That is why the "nominal" gamma of a CRT is usually stated as
2.2 - so that it looks right when viewed in a dim room.

 

[Bob Whatsima]

Nikita, I'm not sure you're going to read this far down in the archive, but
thanks for taking the time out to answer my questions. I never did get this
thing right - I guess it may be because of compromises with the drange of a
scanner versus the drange of film.

Locked everything, applying the same gamma curve, etc. the problem seems to
be mostly with dark images, in that they are made darker. Specifically the
quarter tones show up muddy. Anyway my photoshop skills are improving
immensely, so I am not quite as frustrated with the setup as I was a week
ago.

Thanks once again.