An approach for n-bit int. / m-bit ext. scanners (was: Re: Sprintscan 35+, 10 bits vs 8 bits)

[Lorenzo J. Lucchini]

I'm not saying that one should not use an external editor because the
driver-integrated tools are equivalent. They are obviously not, with any
dedicated editor having many more tools.

There you go! So why do you argue against it?

[snip]

Look, we're going around in circles and we'll never get out of this
discussion like this.

I'll try to take another approach, by laying down my point of view in as
formal a way as I can.

You're not really supposed to read carefully through all this, but if you
do, then at least you can now point *very* specifically to the parts you
don't agree with.

If you don't, at this point we can just agree to disagree agreeably, as you
said.

Note that
- an ASSUMPTION is something that was either true in the OP's case or that I
take as reasonable for the user of the scanner to do
- a FACT is something that I take as true if the ASSUMPTIONs are valid; I
suppose most of your disagreement ought to be about FACTs
- a PROBLEM may have a SOLUTION, that only makes sense if the FACTs are true
- a CONCLUSION states what, at the end of the day, can be reasonably
obtained by means of SOLUTIONs; we might disagree on CONCLUSIONs, even
though only empirical tests (or, at best, very rigorous mathematical
demonstrations) can really tell

Lastly, note that what's said here can be applied to setting the exposure
time (instead of setting the blackpoint and whitepoint, as in the OP's
case) as well.
The mere fact that people *do* use their "tiny preview keyholes" to set
their exposure times ought to suggest that it ain't so bad after all... but
I know *you* just go and take multiple exposures




(1) ASSUMPTION: We intend to scan at the scanner's native resolution.

(2) ASSUMPTION: We have a scanner that computes n bits for every sensor
cell, but only outputs m bits to the computer, with (m < n).

(3) ASSUMPTION: The least significant (n - m) bits can carry meaningful
signal over noise.


(4) ASSUMPTION: The scanner allows to perform a levels mapping on the
scanned image, and this levels mapping is performed on the n-bit data

SPECIFICS: the scanner allows to set two variables
blackpoint
whitepoint
Then, every pixel the scanner sends to the computer is calculated as
output_pixel(i) = (input_pixel(i)-blackpoint)/(whitepoint(i)-blackpoint)
where input_pixel(i) is the i-th value from the sensor, and output_pixel(i)
is the i-th value sent to the computer, before quantization to m bits.


(5) DEFINITION:
Let's call max_input the highest value among (input_pixel(i) for every i).
Let's call min_input the lowest value among (input_pixel(i) for every i).
Let's call max_output the highest value among (output_pixel(i) for every i).
Let's call min_output the lowest value among (output_pixel(i) for every i).


(6) FACT: if blackpoint=0 and whitepoint=2^n-1, then max_input=max_output
and min_input=min_output.


(7) FACT: if min_input>0 and max_input<2^n-1, the amount of information that
can be fitted into the m-bit quantized data can be maximized by setting
blackpoint:=min_input and whitepoint:=max_input (*)

(*) This way, it will always hold true that min_output=0 and
max_output=2^n-1, and the full range of values will be available for every
output_pixel(i)


(8) FACT: If it is true that (max_input-min_input)<2^(n-m)-1 (*) , then the
m-bit quantized output data will contain more information by setting
blackpoint:=min_input and whitepoint:=max_input
than by setting
blackpoint:=0 and whitepoint=2^n-1

(*) Above that, the additional (n-m) bits that the scanner has will have
been all mapped 1-to-1 to the m bits of the output)


(9) PROBLEM: min_input and max_input are not known in advance of scanning.


(10) ASSUMPTION: We are prepared to take more than one scan of the same
picture.


(11) SOLUTION: In order to solve (9), set
blackpoint:=0 and whitepoint=2^n-1
then take the first scan ("preview").
Because of (6), it will be true that
min_input=min_output and max_input=max_output
Thus it will now be possible to set
blackpoint:=min_input=min_output and whitepoint:=max_input=max_output
and take another scan ("final").


(12) DEFINITION: Let's call
min_input_preview=min_input for the "preview" scan
max_input_preview=max_input for the "preview" scan
min_input_final=min_input for the "final" scan
max_input_final=max_input for the "final" scan


(13) FACT: It is true that
|min_input_preview-min_input_scan| < epsilon
and
|max_input_preview-max_input_scan| < epsilon
where
epsilon,epsilon << 2^n
since epsilon will only depend on scanner noise and on scanner fluctuations
between the "preview" and the "final" scans.


(14) CONCLUSION: (11) is a valid solution to problem (9), as the error in
the measurement of min_input and max_input has the same order of magnitude
as the errors that will be in every pixel of the output image.


(15) ASSUMPTION: To save time, we intend to take the "preview" scan at less
than the scanner's native resolution.


(16) PROBLEM: Given (15), the error in the measurement of max_input and
min_input may become greater than the above epsilon, as some pixels that
will appear in the "final" scan are ignored in the "preview" scan


(17) ASSUMPTION: At the scanner's native resolution, the scanner and/or the
scanned medium have a point spread function that is non-zero at points
other than the origin (*).

(*) In other words, the scanner's optics and sensor, and/or the film
resolution characteristics (as well as the characteristics of camera, scene
etc.), cause some blur.


(18) FACT: At the scanner's native resolution, a pixel's value is not
independent from the value of neighboring pixels; instead, it may only
differ from them by an amount that is, at most, equal to the maximum
difference between two neighboring sampled points in the above point spread
function.


(19) DEFINITION: Let's call
resolution_final the scanner's optical resolution
resolution_preview the resolution we intend to take the "preview" at
d the maximum possible difference between
max_output_preview and max_output_final (or between
min_output_preview and min_output_final)


(20) FACT: At a resolution of resolution_preview, d is equal to or less than
the maximum value of
|psf(0,0)-psf(i,j)|
for every
(i,j < resolution_final/resolution_preview)
where psf(x,y) is the point spread function discussed in (17) and (18).

(21) SOLUTION: In order to solve (16), take the "preview" scan as in (11).
Then set
blackpoint:=min_output_preview-d
and
whitepoint:=max_output_preview+d
(without letting blackpoint get smaller than 0, or letting whitepoint get
bigger than 2^n-1).
Now take the "final" scan.


(22) CONCLUSION: Solution (21) does not guarantee that min_output_final=0
and max_output_final=2^n-1, as solution (11) does. On the other hand, it
guarantees that no clipping will occur in the "final" scan (except clipping
of high-noise pixels), and thus that the "final" scan will contain at least
as much information as a scan taken with settings
blackpoint:=0 and whitepoint:=n^2-1
Thus, solution (20) is satisfactory, even though solution (11) is
preferable.


(23) PROBLEM: d is not known; even if we know the value d'>=d that can, in
theory, be estimated from the PSF, d' may be greater than the actual upper
limit on the difference between input and output values (*) .

(*) That's because of considerations I've thought of but not written in this
article, which would need some more attention and mathematical knowledge
than I have.


(24) SOLUTION: in order to solve (23), d can be estimated empirically from
test scans, and/or incrementally raised every time it's found to be too low
(i.e. the resulting image clips).


(25) PROBLEM: At this point, some of the "final" scans for which d was
underestimated may clip.


(26) SOLUTION: In order to solve (25), throw away the clipping images, and
scan then again using later, better estimate of d.


(27) CONCLUSION: Especially when a large number of pictures with similar
characteristics (e.g. pictures from the same roll of film and camera) are
going to be scanned (using the same scanner), solution (24) can be a good
approximation of solution (22).
With some heuristic refinements in order to keep the estimated value of d
just high enough to not result in clipping for the majority of the scans,
solution (24) could even approximate solution (11), at the expense of
failures (clipping) with a minority of the scans that will have to be
treated separately as in (26), i.e. re-scanned.


by LjL
(e-mail address removed)

Copyright 2005 Lorenzo J. Lucchini
(reproduction allowed if this copyright notice is left intact)

 

[Don]

I'll try to take another approach, by laying down my point of view in as
formal a way as I can.

I appreciate you taking the time to do that.

You're not really supposed to read carefully through all this,

Uh-oh... Well, that's another contradiction, right there! Before we
even got started!

It's like saying: Here's an "exact" formula but don't take it
literally! :-/

*That's* why we're running around in circles!! I can only repeat: You
can't have it both ways. That's the perennial problem and the root
cause of all the contradictions.

The whole point of a *formal* statement is exactly that it is made
using *precise* language and is therefore essential to read it
*carefully*!

but if you
do, then at least you can now point *very* specifically to the parts you
don't agree with.

I did that already and I do it all the time.

If you don't, at this point we can just agree to disagree agreeably, as you
said.

That's where it will end!

But since you took all the trouble at least you deserve a response.

Note that
- an ASSUMPTION is something that was either true in the OP's case or that I
take as reasonable for the user of the scanner to do

No, an assumption is an unproven assertion based on circumstantial
evidence or, guesswork or, feeling or, phases of the moon or...?

- a FACT is something that I take as true if the ASSUMPTIONs are valid;

Alarm! The moment you use the word "I" all facts disappear and are
replaced by subjective opinion.

I suppose most of your disagreement ought to be about FACTs

Not "ought to", but *are*! Indeed, *all* my disagreements are about
facts only. In such a context I usually have nothing so say about
opinion and even if I do it's clearly indicated as such.

- a PROBLEM may have a SOLUTION, that only makes sense if the FACTs are true

Non sequitur. A solution can make sense even if facts are false.

- a CONCLUSION states what, at the end of the day, can be reasonably
obtained by means of SOLUTIONs; we might disagree on CONCLUSIONs, even
though only empirical tests (or, at best, very rigorous mathematical
demonstrations) can really tell

Non sequitur. As stated, the "conclusion" and the "solution" have
nothing to do with each other.

The mere fact that people *do* use their "tiny preview keyholes" to set
their exposure times ought to suggest that it ain't so bad after all...

No, the *only* thing such a statement *factually* suggests is that
those people have a (very?) low quality threshold. Any other
*interpretation* is subjective guesswork.

(1) ASSUMPTION: We intend to scan at the scanner's native resolution.

(2) ASSUMPTION: We have a scanner that computes n bits for every sensor
cell, but only outputs m bits to the computer, with (m < n).

(3) ASSUMPTION: The least significant (n - m) bits can carry meaningful
signal over noise.


(4) ASSUMPTION: The scanner allows to perform a levels mapping on the
scanned image, and this levels mapping is performed on the n-bit data

(This is a purely semantic digression, but the word I would use in
this context is "premise" not assumption. That's only meant as a hint.
I'd be happy if I spoke Italian half as well as you do English!)

SPECIFICS: the scanner allows to set two variables
blackpoint
whitepoint
Then, every pixel the scanner sends to the computer is calculated as
output_pixel(i) = (input_pixel(i)-blackpoint)/(whitepoint(i)-blackpoint)
where input_pixel(i) is the i-th value from the sensor, and output_pixel(i)
is the i-th value sent to the computer, before quantization to m bits.

....

You've gone through a great deal of trouble to come up with a
procedure to try and maximize scanner output. The problem is all that
is a separate discussion, which has nothing to do with the original
subject matter: Editing images in scanner software degrades data.

That's it!

I've been saying this all along, but you keep ignoring it and keep
going off on a tangent - and this is an example of it - to come up
with ways of minimizing the damage even though that's not the subject.

All that effort is laudable (and I don't mean to minimize it in any
way!) and I applaud you for the meticulous detail and the huge amount
of work (!!!) you put into it but, unfortunately, all that is totally
irrelevant to the above subject matter and totally beside the point.

I'm sorry you went through all the trouble (and I hope at least others
can benefit from the procedure) but in the context of this thread you
are trying to "prove" something which is *not* even at discussion! As
useful and as comprehensive as your procedure may be, in the context
of the thread it's still an unrelated tangential digression.

You're also mixing unrelated things. Black point has nothing to do
with determining correct exposure. Only the white point counts.

Don.

 

[Lorenzo J. Lucchini]

I appreciate you taking the time to do that.


Uh-oh... Well, that's another contradiction, right there! Before we
even got started!

It's like saying: Here's an "exact" formula but don't take it
literally! :-/

I merely meant to say that, since my article was very long, I found it
understandable that you might not be willing to go through all of it point
by point, and rather just preferred to "agree to disagree".

[snip]

Note that
- an ASSUMPTION is something that was either true in the OP's case or that
I take as reasonable for the user of the scanner to do

No, an assumption is an unproven assertion based on circumstantial
evidence or, guesswork or, feeling or, phases of the moon or...?

[snip disagreements on all the terms I defined]

Look, I was almost sure that my lack of knowledge of English (but, really, I
think it's mostly my lack of knowledge of decent scientific terms) would
have made me use incorrect terms.

But that's precisely why I went on defining them in this introductory part
of the article before using them. They're in capitals precisely to convey
the message that they may not correspond with the standard English
definitions of them.

[snip]

The mere fact that people *do* use their "tiny preview keyholes" to set
their exposure times ought to suggest that it ain't so bad after all...

No, the *only* thing such a statement *factually* suggests is that
those people have a (very?) low quality threshold. Any other
*interpretation* is subjective guesswork.

I didn't mean to imply otherwise; in fact, that's why I've worded it as
"ought to suggest that", rather than, say, "demonstrates that".

Anyway, please note that I only used exposure as a comparison (though
apparently not a useful comparison, since you disagree that a preview is
enough to determine it), but *my article was not about exposure!!!*.

I say this because, down below, what you wrote reads like you think I'm
talking about exposure. I'm not! I'm talking about the OP's problem, which
I think is similar to setting exposure, but in and by itself is a different
thing.

[snip ASSUMPTIONs]

(4) ASSUMPTION: The scanner allows to perform a levels mapping on the
scanned image, and this levels mapping is performed on the n-bit data

(This is a purely semantic digression, but the word I would use in
this context is "premise" not assumption. That's only meant as a hint.
I'd be happy if I spoke Italian half as well as you do English!)

In Italian I would have used "hypothesis", but it just though it didn't
sound too good in English, it sounded too pedantic perhaps.

I simply hadn't thought of "premise".

...

You've gone through a great deal of trouble to come up with a
procedure to try and maximize scanner output.

The problem is all that
is a separate discussion, which has nothing to do with the original
subject matter: Editing images in scanner software degrades data.

That's it!

Look, I don't know how to put it anymore.
I DO NOT CARE what the scanner [software] is able to do or is not able to
do.

My article *explains* what the requirements are for "my" procedure to work;
if *one specific scanner [software], or more than one* don't fulfill those
requirements, tough luck.

The SANE driver *I* am using with my scanner allows to do everything that I
described in the article, except for any kind of obscure bugs that it might
have (well, that it *certainly* has, but anyway).

There are two issues:

1) Is in-scanner bp/wp adjustment on an m-bit external / n-bit internal
(m<n) scanner, in and by itself, due to mathematical reasons, flawed?
-- This is what I've addressed in my article, and if you disagree with any
"FACT" or "CONCLUSION" in it, please say where and why.

2) Is a specific scanner's firmware or software too flawed (i.e. has bugs,
or truncates numbers in weird and wrong ways, or such) for what's described
in my article to be applicable to it?
-- This is none of my business at the moment: I'm just concentrating on the
concept, not how specific implementations may not be up to the task. The
only scanner-specific assumption I'm making is that we're talking about a
scanner where m < n (otherwise, the procedure I'm describing wouldn't be
*needed* in the first place).

I've been saying this all along, but you keep ignoring it and keep
going off on a tangent - and this is an example of it - to come up
with ways of minimizing the damage even though that's not the subject.

In what I said, THE ONLY DAMAGE THERE IS TO MINIMIZE IS THAT m < n !
That damage is inherent to the scanner, and the only way to avoid it (rather
than minimize it) is to buy another scanner.

What I'm talking about is how to get the best possible results that are
obtainable *without* getting another scanner.

I'm trying to *maximize the scanner output*, as you just said two paragraphs
above, given the limitation of m being smaller than n.
You seem to imply (well, rather more than imply, actually) that my procedure
*causes* damage, and them does something to minimize the damage caused.
I'm claiming that THAT IS NOT THE CASE.

The damage is caused by the intrinsic characteristics of the scanner, not by
my procedure. My procedure is intended to maximize the output GIVEN the
damage that is intrinsic to an m-bit/n-bit scanner.

All that effort is laudable (and I don't mean to minimize it in any
way!) and I applaud you for the meticulous detail and the huge amount
of work (!!!) you put into it but, unfortunately, all that is totally
irrelevant to the above subject matter and totally beside the point.

The problem is that it definitely isn't, IMHO.
*If* the OP's scanner firmware/software is so terribly flawed and/or bugged
that it doesn't meet the ASSUMPTIONs made in my article, and the OP cannot
decide to use an alternative firmware/software, then the issue becomes
moot.
Otherwise, it's about as relevant as it can get.

I'm sorry you went through all the trouble (and I hope at least others
can benefit from the procedure) but in the context of this thread you
are trying to "prove" something which is *not* even at discussion! As
useful and as comprehensive as your procedure may be, in the context
of the thread it's still an unrelated tangential digression.

Really, I'm not sure, *what* is at discussion in this thread?
I don't know about you, but *I* was discussing about the possibility of
obtaining better output on scanners that respect the "ASSUMPTIONs" I
enumerated (m < n, allow setting "blackpoint" and "whitepoint" variables
that are applied, as described, on the n-bit data, etc).

For all I know, and for all that he described, the OP's scanner may well
respect those "ASSUMPTIONs", even though there is a possibility that it may
not (that's why I first told the OP to try *finding out* whether it does --
but after that, I've reasoned *on the assum^W premise* that it did).

You're also mixing unrelated things. Black point has nothing to do
with determining correct exposure. Only the white point counts.

Yeah, and indeed, it's not exposure that we're talking about.
I just think that, for exposure, a very similar reasoning could be made --
yeah, with some differences, such as the black point isn't involved, as you
say.
But just forget about exposure if you prefer, that's *not* what we're
talking about; I just thought it could represent a useful comparison, which
apparently it does not.


by LjL
(e-mail address removed)

 

[Don]

I didn't mean to imply otherwise; in fact, that's why I've worded it as
"ought to suggest that", rather than, say, "demonstrates that".

Oh, come on Lorenzo, the clear implication from the above statement is
that you're trying to justify a procedure (you yourself advocate!)
simply because some people may use it!

Otherwise you would've said "using the preview ought to suggest some
people have lower standards"?

But you didn't, because you wanted to put a positive spin on using the
preview in this context. Backtracking now is not going to change that.

My article *explains* what the requirements are for "my" procedure to work;
if *one specific scanner [software], or more than one* don't fulfill those
requirements, tough luck.

As good as your procedure may be, that's neither the point nor the
subject. That procedure uses scanner software to edit. That's all we
need to know! What type of edit or procedure is unimportant at this
stage because the subject is: *scanner software edits degrade data*.

That's how this subthread started. You tried all sorts of different
avenues to get around it (some of it good advice, but not pertinent to
the thread). That's why I keep bringing the discussion back to this
point instead of digressing because all that does is makes us run
around in circles.

What I'm talking about is how to get the best possible results that are
obtainable *without* getting another scanner.

I know that's what you want to talk about, and I've been trying to
explain repeatedly that is not the subject!

But instead of addressing this key point (that your procedure is not
the subject) you go into a defense of your procedure (which I am *not*
attacking or even addressing in any way) and digress ever more from
the subject matter.

That's the problem in this thread.

You seem to imply (well, rather more than imply, actually) that my procedure
*causes* damage, and them does something to minimize the damage caused.

That's the key to your misunderstanding. I'm *not* attacking or
addressing your *procedure* in any way, and that's what I'm trying to
make clear (but, apparently, failing).

Your procedure is beside the point. When I say that, it's not a
criticism of your procedure. It says nothing about your procedure. It
makes no qualitative judgment of the procedure. It simply states the
procedure is not pertinent to the subject matter.

Really, I'm not sure, *what* is at discussion in this thread?

OK, let's see if we can agree on this:

The thread started with advice on optimum workflow for the OP's
scanner. NOTE: Optimum, as defined by the OP, meant *both*: maximum
quality *and* minimum time!

Early on, I made a *side note* (!) that any editing in scanner
software is going to cause problems regarding quality. Nothing
Earth-shattering or controversial about that. Plain vanilla.

(Implying, this may not be important to him (given his full context)
in which case just ignore. But, if it's something he does find
important, it's a good thing to make a note of.)

You objected to that (scanner software edits do harm) at which point
the thread *forked*! That fork, then, became the subject!

You devised a complex procedure as proof of your objection. Nothing
wrong with that... per se.

However, the problem is this procedure did not focus on the key
subject (reduced quality) alone but branched out to black point, etc.
(You then brought in the OP and all sorts of other things including
making many contradictory statements which didn't help things.)

That's where I said that the procedure becomes irrelevant because it
digresses. That's *not* a criticism of the procedure! However, you
took it as such and went on defending it only to digress even more!


So, the rational conclusion we can objectively and factually make:

1. Scanner software edits do harm to raw data. That's just a simple,
elementary and axiomatic fact.
2. A procedure can be devised to minimize this harm, but it can't
eliminate it.

Agreed?

If yes: Yippie!
Now, let's go talk about something else...

If not: Let's agree to disagree agreeably!
And then let's go talk about something else...

Don.

 

[Lorenzo J. Lucchini]

Oh, come on Lorenzo, the clear implication from the above statement is
that you're trying to justify a procedure (you yourself advocate!)
simply because some people may use it!

Not "simply" -- I gave many (!) other reasons for it.

I just thought I would try comparing it to something that's been discussed
more often (i.e. exposure), and which people normally do based on preview
scans.
I thought that, maybe, you agreed that it's reasonable to set exposure based
on a preview (at that point, I would have gone on saying why the problem of
setting exposure time is mostly equivalent to the problem we're
discussing).

But I was wrong: you actually don't find it reasonable (at least for your
quality standard) to set exposure based on a preview. Oh well, at least I
tried.

(Of course, you're perfectly entitled to not find it reasonable! even if
"many" people do it. I wasn't trying to say that "many people do it" => "it
is right", just that you might have agreed with the mentioned many people.)

Otherwise you would've said "using the preview ought to suggest some
people have lower standards"?

But you didn't, because you wanted to put a positive spin on using the
preview in this context. Backtracking now is not going to change that.

I'm getting sick and tired now. HECK, I was just trying to relate to
something you MAY have agreed on (on the basis that "many people do it",
yeah). You don't. OK. It was a failed attempt. DON'T MAKE A CASE OF STATE
ABOUT IT.

My article *explains* what the requirements are for "my" procedure to
work; if *one specific scanner [software], or more than one* don't fulfill
those requirements, tough luck.

As good as your procedure may be, that's neither the point nor the
subject. That procedure uses scanner software to edit. That's all we
need to know! What type of edit or procedure is unimportant at this
stage because the subject is: *scanner software edits degrade data*.

That's YOUR subject, because I don't agree on that blanket statement.
*Scanner software edits do not necessarily degrade data*.
My article described *precisely* what's expected from the scanner software,
and I'M SURE IT CAN BE PROVED THAT EDITING AS DESCRIBED IN MY ARTICLE,
USING SCANNER SOFTWARE THAT FULFILLS THE REQUIREMENTS DESCRIBED IN MY
ARTICLE, *DOES NOT DEGRADE DATA*.

THAT is what YOU refuse to understand! You might say that I haven't proved
it well enough (which is perhaps the case), but instead you go on stating
that I'm talking about something else that's irrelevant. *I'm not, and it's
not*.

That's how this subthread started. You tried all sorts of different
avenues to get around it (some of it good advice, but not pertinent to
the thread). That's why I keep bringing the discussion back to this
point instead of digressing because all that does is makes us run
around in circles.

Get around it. Get around *what*? Get around the fact that it's *hard* to
precisely decide what (in-scanner) edits to make based on a preview. True.

BUT WHAT I STATED, AND THEN STATED AGAIN, IS THAT IF YOU DECIDE TO *NOT* DO
ANY IN-SCANNER EDITS AT ALL, THEN YOU'LL END UP WITH AN EVEN *WORSE* RESULT
THAN WOULD HAVE BEEN CAUSED BY THOSE IMPRECISE DECISIONS (caused by preview
limitations).

[snip]

Your procedure is beside the point. When I say that, it's not a
criticism of your procedure. It says nothing about your procedure. It
makes no qualitative judgment of the procedure. It simply states the
procedure is not pertinent to the subject matter.

Yeah, and that's a nice way to avoid attacking my procedure, isn't it?
Problem: my procedure *is* pertinent to the subject matter.

OK, let's see if we can agree on this:

The thread started with advice on optimum workflow for the OP's
scanner. NOTE: Optimum, as defined by the OP, meant *both*: maximum
quality *and* minimum time!

Nah. "Optimum" cannot mean that. You just can't get them both (quality and
speed, or cost and benefit if you prefer) at the same time.
At best, you can make a compromise. But that's up to the OP to decide how to
balance time and quality.

In any case, the procedure I described does attempt to make a sort of
compromise: it guarantees that, in a majority of the cases, you'll only
need to make two scans (a "preview" and a "final" scan), while always
getting a better result (sometimes *much* better, if the scanner is not too
noisy) than by "just scanning".

So, in your (weird) acception of "optimum", I suppose I *am* convinced that
my procedure (or a variation of it) tries to reach that "optimum". So,
apparently, I'm 100% in-topic.

In any case, if a "no compromise" approach is preferred that only favors
quality (over time spent), then it's very easy: still just scan twice, but
have the preview be at the same resolution as the final scan.

In other words:

1) Best quality, most time = scan twice, always at the native resolution
2) Good quality on average, little time on average = scan twice, having the
"preview" scan be at a lower resolution than the "final" scan
3) Bad quality, very little time = just scan once, without doing any
in-scanner edits (as you suggest)

Early on, I made a *side note* (!) that any editing in scanner
software is going to cause problems regarding quality. Nothing
Earth-shattering or controversial about that. Plain vanilla.

Oh, if you say so, then it must be true. I mean, it's not controversial, so
"most people agree on it", so it cannot but be true, right?

Except it's *not* true, not in all cases.
An example of a case when it *is* true is when you have an n-bit external /
n-bit internal scanner.
An example of a case when it can easily not be true is when you have an
m-bit external / n-bit internal scanner (m<n), as in our case.

(Implying, this may not be important to him (given his full context)
in which case just ignore. But, if it's something he does find
important, it's a good thing to make a note of.)

You objected to that (scanner software edits do harm) at which point
the thread *forked*! That fork, then, became the subject!

Well, it sort of forked, but it was the OP himself that was finding it quite
apparent that, on his (supposedly m-bit ext. / n-bit int.) scanner, doing
correct in-scanner editing would have been better than not doing it.

So, I suppose he might have found it quite earth-shattering to learn that
in-scanner editing is 100% evil, while he was thinking that it would have
been 100% good ;-)

His doubt, originally, was whether his scanner *really* was m-bit external /
n-bit internal, and not m-bit external / m-bit internal (specifically,
whether it was 8-bit / 10-bit, as advertized, or really 8-bit / 8-bit). But
in the "fork" of the thread I assumed it was 8-bit / 10-bit, after telling
him that he should check first.

You devised a complex procedure as proof of your objection. Nothing
wrong with that... per se.

However, the problem is this procedure did not focus on the key
subject (reduced quality) alone but branched out to black point, etc.
(You then brought in the OP and all sorts of other things including
making many contradictory statements which didn't help things.)

Bah... "black point, etc." are simply the things I would address to resolve
the "key subject" of reduced quality -- by using them to get better
quality.

About the contradictory statements, well, I might have made some, but I must
say that I think you've made even worse logical fallacies, for instance
incorrectly arguing that my procedure was irrelevant ;-)

That's where I said that the procedure becomes irrelevant because it
digresses. That's *not* a criticism of the procedure! However, you
took it as such and went on defending it only to digress even more!

You've got it completely wrong. I'd *really* like criticism of the
procedure, because *that's* what would make sense.

Just stating that it's irrelevant is *not* something that makes sense, as it
is *not* irrelevant.

So, the rational conclusion we can objectively and factually make:

1. Scanner software edits do harm to raw data. That's just a simple,
elementary and axiomatic fact.

Ok. I suppose we could stop here. You take as an axiom something that I
don't even remotely take as an axiom.
Oh, but let me repeat it once again, I *do* take it as an axiom when an
n-bit / n-bit scanner is involved. Which is not this case.

2. A procedure can be devised to minimize this harm, but it can't
eliminate it.

Maybe such a procedure can be devised, but it's not the procedure I've been
talking about.

Agreed?
No.

If yes: Yippie!
Now, let's go talk about something else...

If not: Let's agree to disagree agreeably!
And then let's go talk about something else...

Guess so.


by LjL
(e-mail address removed)

 

[Roger S.]

"> 1. Scanner software edits do harm to raw data. That's just a simple,
elementary and axiomatic fact. "

I spoke with God about His Natural Law (and implications for scanning)
and got a confirmation that Don's axioms are actually heresy. As I
speak for God, what I say has to be true and there can be no
disagreement. This is a perfectly reasonable position to take and any
disagreement with me is illogical and emotional in nature.

 

[Lorenzo J. Lucchini]

On Wed, 26 Oct 2005 22:44:02 +0200, "Lorenzo J. Lucchini"

[snip]

I thought I can also try another way to put it.

Assume that the OP's scanner driver works this way:

scan --resolution <res> --blackpoint <bp> --whitepoint <wp> <output-image>

And that you can also do
getmax <input-image> (prints the maximum pixel value)
getmin <input-image> (prints the minimum pixel value)

This is actually very similar to the way my driver works, though obviously
only the OP knows whether he can do something equivalent with his driver.

Clearly, the "scan" command, when given <bp> and <wp> different from 0 and
255, "stretches the histogram" to accomodate for them *before* conversion
to 8-bit, while the data are still 10-bit. That goes by itself, and is the
very basic requisite that the OP's scanner *must* have for this to make any
sense.


At this point, you do, for example

scan --resolution 600 --blackpoint 0 --whitepoint 255 preview.tiff
Max=`getmax preview.tiff`
Min=`getmin preview.tiff`
do
# Now play with Min and Max in the ways I described in "my procedure"
scan --resolution 2400 --blackpoint $Min --whitepoint $Max
final.tiff
until `getmin final.tiff` > 0 && `getmax final.tiff` < 255

(The do-until loop cannot really be written that way in a shell, but that's
not the point, it's easier to read the way I've written it)


Now, this is basically an implementation of "my procedure", except that the
real guts of the procedure aren't implemented, but you can read them quite
thoroughly in the other article.
*Here*, however, you can see the rest, i.e. actual commands given to a
hypothetical scanner driver, which has a specified behavior.

So, WHERE IS THE PART THAT MAKES YOU SAY THAT IN-SCANNER (or in-firmware, or
in-driver) EDITING DEGRADES THE DATA?
Unless that part is in "my procedure", but you said it wasn't, and that the
procedure itself was irrelevant.



(All this assumes that we're talking black and white scans for simplicity,
as in the other long post; color would work the same except for having
three times the number of bp's, wp's, min's and max's.)


by LjL
(e-mail address removed)

 

[Don]

....
I thought that, maybe, you agreed that it's reasonable to set exposure based
on a preview (at that point, I would have gone on saying why the problem of
setting exposure time is mostly equivalent to the problem we're
discussing).

But I was wrong: you actually don't find it reasonable (at least for your
quality standard) to set exposure based on a preview. Oh well, at least I
tried.

Again, Lorenzo, you're mixing several things here totally out of
context.

What you fail to understand is that there is no "my" quality standard.
I didn't advocate anything one way or another. I've simply taken the
given context (as specified by the OP!) and then within that context,
stated simple facts. That's all. Just because I state those facts does
not mean I advocate them! It's simply what the facts are, given the
context.

(Of course, you're perfectly entitled to not find it reasonable! even if
"many" people do it. I wasn't trying to say that "many people do it" => "it
is right", just that you might have agreed with the mentioned many people.)

That's not only beside the point, but wrong too. Numbers do not equal
quality!

I can find "many" people who think that web JPG images are "fantastic
quality". Does that mean those web JPGs are high quality in objective
terms?

My article described *precisely* what's expected from the scanner software,
and I'M SURE IT CAN BE PROVED THAT EDITING AS DESCRIBED IN MY ARTICLE,
USING SCANNER SOFTWARE THAT FULFILLS THE REQUIREMENTS DESCRIBED IN MY
ARTICLE, *DOES NOT DEGRADE DATA*.

Yes it does, for all the reasons outlined earlier. Shouting is not
going to improve the quality.

So we'll just have to disagree agreeably.

Nah. "Optimum" cannot mean that.

Again, context, Lorenzo, context!!!

The context is not what's the absolute meaning of "optimum" (your
above ad hoc tangential digression) but what's the OPs requirement (as
I stated above originally). This is what he himself said:

I have a lot of slides to do, so I am trying to get through it in the
fastest way possible, but also allow me to get the best possible
quality.

So, quite clearly he was interested in *both* i.e. a balance.

But you immediately go off on a tangent about absolute meaning of
optimum! That then leads to contradictions and digressions, which are
the perennial problems here.

Please stay focused on the subject!

So, in your (weird) acception of "optimum", I suppose I *am* convinced that
my procedure (or a variation of it) tries to reach that "optimum". So,
apparently, I'm 100% in-topic.

As can be seen from the above quote, not mine, but OP's!

Guess so.

OK then, case closed.

Don.

 

[Don]

I thought I can also try another way to put it.

I really appreciate the thought and effort you are putting into this
Lorenzo. I really do! And I don't mean to make it frustrating for you,
but all I'm trying to point out is that, often, this effort is
misplaced.

But that doesn't mean I'm in any way criticizing or minimizing the
effort. On the contrary, I not only appreciate it but also try to
spare you from spending so much time when it doesn't address the
issue. That's all I mean by "irrelevant".

So, WHERE IS THE PART THAT MAKES YOU SAY THAT IN-SCANNER (or in-firmware, or
in-driver) EDITING DEGRADES THE DATA?

The problem is, Lorenzo, you're taking things out of context.

As I've explained, data degradation is not limited only to *direct*
effects of scanner software (although that usually plays a very large
part) but to *indirect* effects as well! That's the big picture.

All your effort has been to try and "prove" that *direct* negative
effects of scanner software editing can be minimized. However, you've
totally ignored the *indirect* effects.

One of these indirect effect is the fact that scanner software is very
limited in terms of features. So, additional editing will be required
afterwards in all but most trivial cases. And, as we all know,
multiple edits result in cumulative errors.

Ergo, the end result (which is the context and the subject here) will
be lower quality if half the editing is done in scanner software
(regardless of how "perfect" this editing is) and the other half in
external software.

So trying to perfect one half will not change the end result. "A chain
is only as strong as its weakest link." Improving the *strongest*
link, is not going to make the chain any stronger. Or, as I have been
stating it, improving the strongest link is "irrelevant" to the chain
strength. What increases chain strength (i.e. what is relevant) is
improving the *weakest* link! And the weakest link in this case is the
separation of editing, to name just one example.

Unless that part is in "my procedure", but you said it wasn't, and that the
procedure itself was irrelevant.

I didn't say that! That's the key of your misunderstanding.

It is only irrelevant when the context is the resulting quality of the
*whole* process, not just the qulaity of your procedure. In that case,
your procedure, no matter how perfect, is still only one link in the
chain. And trying to improve it, is irrelevant to the end result.

Your procedure is *very* relevant when the context is scanning stage
only. Indeed, then the scanner editing *is* the context! But that's
not the context here which is why I'm saying it's not relevant.

Don.

 

[Don]

I spoke with God

You speak with god? (Lower case intentional.)

And he/she/it talks back to you?

Hmmm!?

Well, that explains all your recent messages here! ;o)

Don, the devout atheist.

P.S. Sorry, couldn't resist. You can go back to venting, now.

 

[Lorenzo J. Lucchini]

On Fri, 28 Oct 2005 14:33:07 +0200, "Lorenzo J. Lucchini"

[snip]

Listen, will you concede that the OP should, at least, measure the color of
the film base (that's easy to measure, you can take as many samples as you
like, as it only needs to be done once for an entire roll of film), and
then use that color as whitepoint in the scanner?

It just doesn't make any sense to waste bit depth for values above the
orange mask's color, and there are no "preview keyhole" issues here.

And note that I'm not thinking of this as a way to remove the orange mask
cast or something (though it might be a good way to achieve that, too), I'm
just thinking about using those damn two bits.

I didn't say that! That's the key of your misunderstanding.

It is only irrelevant when the context is the resulting quality of the
*whole* process, not just the qulaity of your procedure. In that case,
your procedure, no matter how perfect, is still only one link in the
chain. And trying to improve it, is irrelevant to the end result.

Improving one link in a chain is absolutely relevant to the end result,
unless that link isn't the bottleneck, but that link *is* the bottleneck in
the context (I mean, whatever that context is, I cannot make a sense of it
anymore).

Your procedure is *very* relevant when the context is scanning stage
only. Indeed, then the scanner editing *is* the context! But that's
not the context here which is why I'm saying it's not relevant.

Even if the context isn't "scanning stage only", if you lose data at the
scanning stage, there's no means you can recover them later. "My procedure"
tries to minimize the loss at scanning stage, i.e. open up the main
bottleneck as much as possible. What you do *after* the scanning stage can
only benefit from this.

Anyway, at this point I think that those who (are mad enough to) read the
whole thread will have all the elements to make up their minds about this.
I've gotten tired of discussing it now.


by LjL
(e-mail address removed)

 

[Don]

Listen, will you concede that the OP should, at least, measure the color of
the film base (that's easy to measure, you can take as many samples as you
like, as it only needs to be done once for an entire roll of film), and
then use that color as whitepoint in the scanner?

Yes, but that has nothing to do with what we're talking about.

Improving one link in a chain is absolutely relevant to the end result,
unless that link isn't the bottleneck, but that link *is* the bottleneck in
the context (I mean, whatever that context is, I cannot make a sense of it
anymore).

And that's the problem. That link is not the bottleneck.

Even if the context isn't "scanning stage only", if you lose data at the
scanning stage, there's no means you can recover them later. "My procedure"
tries to minimize the loss at scanning stage, i.e. open up the main
bottleneck as much as possible. What you do *after* the scanning stage can
only benefit from this.

Not if subsequent edits destroy any of those benefits.

Anyway, at this point I think that those who (are mad enough to) read the
whole thread will have all the elements to make up their minds about this.

I suspect they just mutter something really rude and click on "Next
Message". ;o)

I've gotten tired of discussing it now.

Finally, we agree! ;o)

Don.

 

[ljlbox]

Don ha scritto:

Yes, but that has nothing to do with what we're talking about.

Why? It's still a matter of doing in-scanner whitepoint and blackpoint
(well, no blackpoint in this case really) adjustments in order to
obtain information from the "latent" two bits of the OP's scanner.

Sure, it's a simpler and safer operation than what we've been talking
about until now.
But it's basically the same idea: take a pixel value above which you
know (or guess) there is no valid information, and have the scanner
"stretch the histogram" accordingly on output.

It's just that the film base color is much easier to find reliably than
the actual image's whitepoint, since there are no "preview keyhole"
issues and so on.

(In what we've been talking about before, the blackpoint was treated
similarly, but it doesn't make much sense in this narrower context, as
the film blackpoint is usually quite close to R0 G0 B0, at least for my
scanner.)


by LjL
(e-mail address removed)

 

[Don]

Why?

Because it doesn't! Two different things. And you know why:

It's just that the film base color is much easier to find reliably than
the actual image's whitepoint, since there are no "preview keyhole"
issues and so on.

There you go!

So, once again, first you challenge and then in the very next
paragraph answer the challenge yourself...

You can continue this discussion quite nicely without me! ;o)

Anyway, I thought you were tired of this?

Don.

 

[Lorenzo J. Lucchini]

Because it doesn't! Two different things. And you know why:

They're *clearly* two different things. It being a different things doesn't
imply it "having nothing to do with what we're talking about".

And you know why: specifically, look above and you'll see me say "Listen,
will you concede [...] at least [...]".

There you go!

So, once again, first you challenge and then in the very next
paragraph answer the challenge yourself...

You can continue this discussion quite nicely without me! ;o)

Anyway, I thought you were tired of this?

Ok, you're really being captious on purpose now. Sorry, I just can't avoid
thinking that.

I very, very clearly stated that I had given up trying to convince you of
the whole thing we've been talking about; now, I simply said: look, since
we can't agree about this, will we at least agree about something narrower
but related (i.e. the orange mask stuff) that doesn't features complicated
issues like the "preview keyhole" and the likes?


Really, I have only one thing left to say now: "bah".


by LjL
(e-mail address removed)

 

[Don]

Ok, you're really being captious on purpose now. Sorry, I just can't avoid
thinking that.

Language barrier.

I was quite serious. There was nothing behind that question. You said
you were tired of this. I agreed. And then you continue...?

So I just posed the obvious question.

Anyway, can we agree to disagree agreeably, now?

Don.

 

[Lorenzo J. Lucchini]

Language barrier.

I was quite serious. There was nothing behind that question. You said
you were tired of this. I agreed. And then you continue...?

So I just posed the obvious question.

Anyway, can we agree to disagree agreeably, now?

It wasn't about that specific question.
Repeat: on the topics we've been talking about, I've already agreed to
disagree agreeably.

I simply thought: well, why not try to agree at least on something similar
and related to the original topic, but simpler and less problematic (so
that it'll probably be easier to agree on)?

Indeed, you did agree on it, but immediately started attacking on the ground
that it had "nothing to do" with the previous topic, and that I was
"contradicting myself again", and so on.

I think that, in reality, you know perfectly that both things are not true:
it is similar and related to, though not the same thing as, the previous
topic, and as such, my statements were not contradictory.


Repeat again (and please take note of this paragraph): it's still a matter
of doing in-scanner whitepoint adjustment to obtain information from the
"latent" 2 bits of the OP's scanner -- and in this respect, it's very
related to the original topic --, but this time, there are no "preview
keyhole" issues involved -- and in this respect, it's different from the
original topic, and probably easier to agree on.


Pretending to not understand this simple concept, and starting to attack me
again instead of just saying something like "yes, now that's right, this
more limited but safer approach would be good to use on the OP's scanner,
and contrary to your previous suggestions, I have no technical objections
to it", is what I call captious.


by LjL
(e-mail address removed)

 

[blumesan]

Listen, will you concede that the OP should, at least, measure the color of
the film base (that's easy to measure, you can take as many samples as you
like, as it only needs to be done once for an entire roll of film), and
then use that color as whitepoint in the scanner?

It just doesn't make any sense to waste bit depth for values above the
orange mask's color, and there are no "preview keyhole" issues here.

It is with great tripidation that I enter this thread. I have nothing
to contribute to the basic subject (or non-subject) of the prior
discussion. I do have a simple question regarding the above quote:
Would not this procedure make sense also for scanners where input bits
= output bits?

Thanks,
Mike

 

[Lorenzo J. Lucchini]

It is with great tripidation that I enter this thread. I have nothing
to contribute to the basic subject (or non-subject) of the prior
discussion. I do have a simple question regarding the above quote:
Would not this procedure make sense also for scanners where input bits
= output bits?

Not much.
It wouldn't because, in such a scanner, you can obtain exactly the same
result (or a better result, as you have some more control) in
post-processing. You don't need to do it in the scanner's firmware or
driver, as there are no bits to lose.

Anyway, note that I was actually intending this as a way *to maximize bit
depth* (on an appropriate scanner), not to remove the orange mask.

In my view, the fact that it also results in removing the orange mask is a
side-effect that can be eliminated later, if desired -- though I suppose
that for most people it would be a *desired* side-effect, and as such there
would be no reason to eliminate it.


Let me put it into another way: imagine that you have a 16bpc image, and
that you want to convert it to 8bpc, but retain as much information as
possible.
What could you do?

Well, you could output a *normalized* 8bpc image, where all values from 0 to
255 have some information. If the original image was already normalized,
you won't gain anything, but if it wasn't, you would.
This is similar to what I was suggesting.

But, this way, you will obviously falsify the correct image colors. What can
you do to avoid it? Well, you can *also* save somewhere the original
whitepoint and blackpoint values.

So, in the end, you will have an 8bpc image *plus* a file containing
something like "Whitepoint Rnnn Gnnn Bnnn, Blackpoint Rnnn Gnnn Bnnn".
Using these two files, you will always be able to get back the original
colors, if desired -- and this is similar to orange mask removal in my
example.


The point is that bit-depth maximization and orange-mask removal (or, in
general, whitepoint adjustment) are conceptually two different things, even
though they may both be obtained by the same process in practice.


by LjL
(e-mail address removed)