Cropping problem, doing Vuescan batch scan-from-disk

[Mendel Leisk]

Note: I've posted this at photo.net, maybe a week back, with zero
responses. I'm hoping some of the Vuescan gurus here may have ideas:

I appreciate this is a pretty esoteric question, but I'm hoping
someone has had a similar experience, and found a solution they could
share. Thanks in advance for any suggestions.

My approach when scanning film with Vuescan is:

1. Scan film, saving raw files only.

2. Determine which are landscape orientated, and rotate these raw
files, using Photoshop.

3. Do Vuescan scan-from-disk, from the raw files (some rotated, some
not), with crop set to "maximum", and rotation set to "none", output
finished gamma tiff files.

With Vuescan versions up to 8.0.8, this has worked fine when I do a
"list" or "all" batch scan, using these raw files. The resulting
finished gamma tiffs are almost always oriented correctly. VERY
occasionally, not even once per roll, I would get a miscropped result,
usually square, with the 2 longer ends cropped. My scans are 35mm film
and the files are roughly the same proportion.

As of Vuescan version 8.0.10, if I do a batch scan-from-disk, I get
this square mis-crop, more or less consistently, as soon as a
landscape oriented raw file is encountered. Once this happens,
landscape oriented raws seem to be consistently mis-cropped to square,
and sometimes portrait oriented raws yield square miscropped results,
as well.

So, I've gone back to version 8.0.8, and problem is gone. I've emailed
Ed Hamrick, and he asked questions and made suggestions. He suspected
my rotating the raws was doing something. He also thought the file's
"dpi in the file header" might have been changed, and that could be
the problem. At that point, we both sort of gave up, and that where it
stands.

After cleaning the raw files of dust and scratches, in Photoshop, I
output a fresh batch of raw files, doing scan-from-disk. These new raw
files are not opened subsequently, in Photoshop. The cropping problem
still happens with these.

I'm concerned that I will be in a Vuescan backwater, missing out on
features in new versions.

 

[Bart van der Wolf]

SNIP

So, I've gone back to version 8.0.8, and problem is gone.
I've emailed Ed Hamrick, and he asked questions and made
suggestions. He suspected my rotating the raws was doing
something.

Would be my assumption as well. How do you save the TIFFs from
Photoshop? As uncompressed, 'plain vanilla' TIFFs?

He also thought the file's "dpi in the file header" might have
been changed, and that could be the problem. At that point,
we both sort of gave up, and that where it stands.

As I understand it, VueScan uses standard TIFF libraries that are
available from a few sources and that can be included in a prrogram
as the libraries are. Therefor it seems unlikely that the issue is
with that, although Ed would know if he used the entire library or
only a sub-set (e.g. leaving out some sort of esoteric compression).

After cleaning the raw files of dust and scratches, in
Photoshop, I output a fresh batch of raw files, doing scan-
from-disk. These new raw files are not opened subsequently,
in Photoshop.

They are not opened? That indicates some strange file corruption,
which is unlikely to happen with a file written by a standard TIFF
library. Just a wild thought, have you checked your hard-disk for bad
sectors?

The cropping problem still happens with these.

Have you tried different Cropping parameters in VueScan, to see if it
might be related to the cropping itself? For instance does Crop|Crop
size Manual or Auto make a difference? And Crop|Preview area Default
or Maximum? If that doesn't produce different results, I'm still
inclined to suspect the TIFF libraries.

Bart

 

[Alex Stols]

Note: I've posted this at photo.net, maybe a week back, with zero
responses. I'm hoping some of the Vuescan gurus here may have ideas:

I appreciate this is a pretty esoteric question, but I'm hoping
someone has had a similar experience, and found a solution they could
share. Thanks in advance for any suggestions.

My approach when scanning film with Vuescan is:

1. Scan film, saving raw files only.

2. Determine which are landscape orientated, and rotate these raw
files, using Photoshop.

3. Do Vuescan scan-from-disk, from the raw files (some rotated, some
not), with crop set to "maximum", and rotation set to "none", output
finished gamma tiff files.

With Vuescan versions up to 8.0.8, this has worked fine when I do a
"list" or "all" batch scan, using these raw files. The resulting
finished gamma tiffs are almost always oriented correctly. VERY
occasionally, not even once per roll, I would get a miscropped
result,
usually square, with the 2 longer ends cropped. My scans are 35mm
film
and the files are roughly the same proportion.

As of Vuescan version 8.0.10, if I do a batch scan-from-disk, I get
this square mis-crop, more or less consistently, as soon as a
landscape oriented raw file is encountered. Once this happens,
landscape oriented raws seem to be consistently mis-cropped to
square,
and sometimes portrait oriented raws yield square miscropped results,
as well.

So, I've gone back to version 8.0.8, and problem is gone. I've
emailed
Ed Hamrick, and he asked questions and made suggestions. He suspected
my rotating the raws was doing something. He also thought the file's
"dpi in the file header" might have been changed, and that could be
the problem. At that point, we both sort of gave up, and that where
it
stands.

After cleaning the raw files of dust and scratches, in Photoshop, I
output a fresh batch of raw files, doing scan-from-disk. These new
raw
files are not opened subsequently, in Photoshop. The cropping problem
still happens with these.

I'm concerned that I will be in a Vuescan backwater, missing out on
features in new versions.

Hello,
I had this type of problem repeatedly and in the end I gave up
rotating some of the images, thus keeping all of them in landscape
(or portrait as the case might be).
Sometimes it helped closing Vuescan and starting it again: the
strange 'square' cropping then had disappeared. I suppose this isn't
of much use to you, but it seems to do something at least.
I didn't notify Ed Hamrich about this as there are still some
unresolved issues with the type of scanner I use that I wanted to be
addressed first
Incidentally: I gave up on Vuescan since version 8.0.10 ... Waiting
patiently for a version that is working properly.

 

[Mendel Leisk]

SNIP

Would be my assumption as well. How do you save the TIFFs from
Photoshop? As uncompressed, 'plain vanilla' TIFFs?

My initial raw file output is compressed tiff. I open the file in
photoshop, and save it. Not sure what that might change.

As I understand it, VueScan uses standard TIFF libraries that are
available from a few sources and that can be included in a prrogram
as the libraries are. Therefor it seems unlikely that the issue is
with that, although Ed would know if he used the entire library or
only a sub-set (e.g. leaving out some sort of esoteric compression).


They are not opened? That indicates some strange file corruption,
which is unlikely to happen with a file written by a standard TIFF
library. Just a wild thought, have you checked your hard-disk for bad
sectors?

I think you mis-understand me. After output a fresh raw file, from the
doctored raw file, I do not open that new file in Photoshop. My main
reason for doing this is I've found with each save of the compressed
file in Photoshop, the compression ratio gets worse (file gets
bigger). Often even bigger than un-compressed! But these files behave
ok. They are openable and viewable, and work fine with Vuescan.

Have you tried different Cropping parameters in VueScan, to see if it
might be related to the cropping itself? For instance does Crop|Crop
size Manual or Auto make a difference? And Crop|Preview area Default
or Maximum? If that doesn't produce different results, I'm still
inclined to suspect the TIFF libraries.

I've been using Crop|Crop Size "maximum" only. As I recollect, this is
the only one that behaves, with rotated raws. I did experiment with
different (I believe I tried all) Crop|Preview Area choices. I have
left in on (default) "default"

Bart

I find it a hassle to have to tell Vuescan over and over and over,
which files are "portrait", and which are "landscape". This is an
unchanging constant, and I don't like having to re-affirm it
constantly. I HAD a solution, up to and including release 8.0.08, by
rotating the raws. Now, with newer releases, something is tripping it
up. It doesn't appear to happen with single scans, especially if I
preview first. But as soon as I do batch scanning without preview,
miscropping occurs, usually with first landscape oriented raw. I
really would have to re-install a higher version, and re-try all the
variations, before I could say conclusively, though.

At this point, I'm thinking to give up on rotating, deal with all my
images in protrait mode, and do the rotations to finished gamma tiffs
in Photoshop. This bothers me in that, it takes Vuescan out of the
loop more and more. I would like the program to be the start-to-finish
software, for the majority of my images.

Anyway Bart, thanks for all your ideas, I knew you would jump in!

Regards,

Mendel Leisk

 

[Bart van der Wolf]

"Bart van der Wolf" <[email protected]> wrote in message

How do you save the TIFFs from Photoshop?
As uncompressed, 'plain vanilla' TIFFs?

My initial raw file output is compressed tiff. I open the
file in photoshop, and save it. Not sure what that might
change.

If your file is 16-bit/channel, the Raw file will be an uncompressed
TIFF.

After output a fresh raw file, from the doctored raw file, I do not

open that new file in Photoshop. My main

reason for doing this is I've found with each save of the
compressed file in Photoshop, the compression ratio
gets worse (file gets bigger). Often even bigger than un-
compressed! But these files behave ok. They are
openable and viewable, and work fine with Vuescan.

I see. Trying to compress a 16-bit/channel (or 15 in the case of
Photoshop), will often result in a larger (!) file. For that reason
VueScan defaults to uncompressed 16-b/ch files. When your files grow
when saving from Photoshop, that suggests they are being saved as
compressed files. The growth results from additional overhead in the
file header when actually little compression can be realized (some
compression algorithms create a kind of library, that needs to be
stored together with the actual data). The file data is usually stored
losslessly in TIFF, but maybe some of the other Save parameters
conflicts with VueScans expectations of a Raw file (thus Ed's question
about PPI).

SNIP

I find it a hassle to have to tell Vuescan over and over
and over, which files are "portrait", and which are
"landscape". This is an unchanging constant, and I don't
like having to re-affirm it constantly. I HAD a solution,
up to and including release 8.0.08, by rotating the raws.
Now, with newer releases, something is tripping it up.

Yes, I can imagine the hassle. I never experienced it because I don't
rotate Raws, and don't run batches, but it should work so it's worth
looking for a solution. Maybe you could look again what else may have
changed in your workflow.
Maybe you upgraded Photoshop, and one of the new TIFF tags conflicts
with a newer release of a TIFF library Ed Hamrick links in (I'm just
speculating).

It doesn't appear to happen with single scans, especially
if I preview first. But as soon as I do batch scanning
without preview, miscropping occurs, usually with first
landscape oriented raw.

I assume this is one of the things you spoke to Ed about? It seems to
point to something in the batch routines that handles the reading of
Raw TIFFs differently.
I

really would have to re-install a higher version, and re-try all the
variations, before I could say conclusively, though.

You do know that you can install multiple versions of VueScan, each in
their own subdirectory/folder, don't you. That way you can benefit
from new features when not batch processing by clicking the shortcut
to the latest release, and do batch processing with the version that
works for you (even at the same time you scan with the other version),
untill the issue is resolved.

At this point, I'm thinking to give up on rotating, deal with
all my images in protrait mode, and do the rotations to
finished gamma tiffs in Photoshop.

I'm also not the quitting type, but there's not much use in wrestling
a pig either, so it is an option. Maybe others can check aswell if
their workflow can reproduce the issue.

Bart

 

[Mendel Leisk]

If your file is 16-bit/channel, the Raw file will be an uncompressed
TIFF.

Not true. In my case, I have Output|Raw file type set to "auto", and
this gives me 16bit rgb tiffs for raw files. I have Output|Raw
compression set to "on", and accordingly, I get compressed tiffs. With
my 2820 ppi scanner (Scan Dual II), uncompressed raws are around
55megs. My compressed raws vary between 35 and 50 megs. Each save in
PS brings this size up. Doing fresh scan-from-disk and outputting a
new raw gets it back down to initial size.

open that new file in Photoshop. My main

I see. Trying to compress a 16-bit/channel (or 15 in the case of
Photoshop), will often result in a larger (!) file. For that reason
VueScan defaults to uncompressed 16-b/ch files. When your files grow
when saving from Photoshop, that suggests they are being saved as
compressed files. The growth results from additional overhead in the
file header when actually little compression can be realized (some
compression algorithms create a kind of library, that needs to be
stored together with the actual data). The file data is usually stored
losslessly in TIFF, but maybe some of the other Save parameters
conflicts with VueScans expectations of a Raw file (thus Ed's question
about PPI).

SNIP

Yes, I can imagine the hassle. I never experienced it because I don't
rotate Raws, and don't run batches, but it should work so it's worth
looking for a solution. Maybe you could look again what else may have
changed in your workflow.
Maybe you upgraded Photoshop, and one of the new TIFF tags conflicts
with a newer release of a TIFF library Ed Hamrick links in (I'm just
speculating).

No, no PS upgrade, recently. Also, cropping problem is gone when I go
back to 8.0.8. For me, all this points to a change in the newer
releases causing the problem, directly or indirectly.

I assume this is one of the things you spoke to Ed about? It seems to
point to something in the batch routines that handles the reading of
Raw TIFFs differently.

Yes, I pointed this out to Ed. I'm not sure if it's my doing preview
before doing a single scan-from-disk. I'd really have to try all the
combinations, and I'm not sure I want/need to spend hours beta
testing. Well, for the price, and with the lifetime free upgrades,
maybe I should. Vuescan is a group effort, I guess.

I

You do know that you can install multiple versions of VueScan, each in
their own subdirectory/folder, don't you. That way you can benefit
from new features when not batch processing by clicking the shortcut
to the latest release, and do batch processing with the version that
works for you (even at the same time you scan with the other version),
untill the issue is resolved.

Good idea.

Only thing I can think, make a text string representing the rotation
for all frames on a roll, stop rotating raws, cut and paste this
string into the Vuescan rotation field. BUT, this is only really
practical when doing "all" scan from disk.

I'm also not the quitting type, but there's not much use in wrestling
a pig either, so it is an option. Maybe others can check aswell if
their workflow can reproduce the issue.

Bart

Well, I have no problem with 8.0.8. I set crop to "maximum", and it
happily cranks out the images, all right way round, sourcing from my
rotated raws. Except I'm stuck with that release.

Thanks again, Bart.

 

[Bart van der Wolf]

"Bart van der Wolf" <[email protected]> wrote in message

Not true. In my case, I have Output|Raw file type set to
"auto", and this gives me 16bit rgb tiffs for raw files. I have
Output|Raw compression set to "on", and accordingly, I get
compressed tiffs.

Okay, if you change the defaults and deliberately select Compression
on, then obviously the files will go through a compression cycle. That
however will not guarantee a smaller file size. That is why the
default (auto) will not try to compress 48/64-bit files. It was added
as an option when people started complaining about their scans growing
in size when compression was selected. Again, it depends on image
content, so use what works best for you.

With my 2820 ppi scanner (Scan Dual II), uncompressed raws
are around 55megs. My compressed raws vary between 35 and
50 megs.

Yes, that is possible, but it is image content dependent (as the
variation in file sizes demonstrates).

Each save in PS brings this size up.

Do you mean that opening, and saving the same file more than once,
will result in different growing file sizes?

Doing fresh scan-from-disk and outputting a new raw gets it
back down to initial size.

Yes, VueScan uses lossless compression, and as long as the image
content doesn't change, the compression cycle will produce identical
results.

Bart

 

[Mendel Leisk]

Okay, if you change the defaults and deliberately select Compression
on, then obviously the files will go through a compression cycle. That
however will not guarantee a smaller file size. That is why the
default (auto) will not try to compress 48/64-bit files. It was added
as an option when people started complaining about their scans growing
in size when compression was selected. Again, it depends on image
content, so use what works best for you.


Yes, that is possible, but it is image content dependent (as the
variation in file sizes demonstrates).

I invariably gain between 5 and 10 megs, with my setup. Actually, if
Vuescan's focus screws-up, the resulting very out-of focus compresses,
REALLY good.

Do you mean that opening, and saving the same file more than once,
will result in different growing file sizes?

Yes, file size increases with each save.

Yes, VueScan uses lossless compression, and as long as the image
content doesn't change, the compression cycle will produce identical
results.

Bart

Anyway, this is kind of wandering away from my real concern, the
cropping. Thanks for the tips, though.

 

[Bart van der Wolf]

SNIP

Yes, file size increases with each save.

That is impossible, well apparently it is possible with your setup,
but that still doesn't make sense.

SNIP

Anyway, this is kind of wandering away from my real concern,
the cropping. Thanks for the tips, though.

Maybe, maybe not. The above mentioned file size growth is not normal
with lossless compression. Maybe there's something wrong there that
somehow triggers the cropping?

Bart

 

[Mendel Leisk]

That is impossible, well apparently it is possible with your setup,
but that still doesn't make sense.

SNIP

Maybe, maybe not. The above mentioned file size growth is not normal
with lossless compression. Maybe there's something wrong there that
somehow triggers the cropping?

Bart

All of these factors (compressed raws, PS editting of raws, etc.) were
present at the point I upgraded from Vuescan 8.0.8 to 8.0.10. Before
upgrade, no cropping problem. After upgrade, constant cropping
problem, with "list" or "all" batch scanning. Wether or not PS
treatment of tiff compression is a factor, it seems readily apparent
that SOMETHING was changed in Vuescan, atleast in 8.0.10 (maybe in
8.0.9, I didn't download that one).

I've emailed Ed on the "big picture" request he made. What I would
like to see is some way legitimately within Vuescan "tagging" the raw
file (or rotating?) it, to eliminate this requirement of having tell
the program wether you want portrait or landscape.

 

[Ed Hamrick]

All of these factors (compressed raws, PS editting of raws, etc.) were
present at the point I upgraded from Vuescan 8.0.8 to 8.0.10. Before
upgrade, no cropping problem. After upgrade, constant cropping
problem, with "list" or "all" batch scanning. Wether or not PS
treatment of tiff compression is a factor, it seems readily apparent
that SOMETHING was changed in Vuescan, atleast in 8.0.10 (maybe in
8.0.9, I didn't download that one).

It would be trivial for me to fix (or figure out what the problem
is with the rotated files) if you'd send me an unrotated raw file
that crops ok and a rotated raw file that doesn't crop ok.

I sent you an e-mail yesterday requesting this, but haven't gotten
this. Did you send it? Don't send it by e-mail if it's larger
than a few MBytes, but instead put the files somewhere I can download
them from.

Regards,
Ed Hamrick

 

[Mendel Leisk]

It would be trivial for me to fix (or figure out what the problem
is with the rotated files) if you'd send me an unrotated raw file
that crops ok and a rotated raw file that doesn't crop ok.

I sent you an e-mail yesterday requesting this, but haven't gotten
this. Did you send it? Don't send it by e-mail if it's larger
than a few MBytes, but instead put the files somewhere I can download
them from.

Regards,
Ed Hamrick

Done Ed, emailed a link.

 

[Erik Krause]

Trying to compress a 16-bit/channel (or 15 in the case of
Photoshop)

Chris Cox writes in the Adobe Photoshop forum that it is actually 16
bit but with a maximum value of 32768 (as you can see if you switch the
info palette to 16 bit values). 15 bit would be a maximum value of
32767.

If you save a file with pure white RGB(32768,32768,32768) and you look
at the resulting file with ImageMagick for example, the maximum values
are (65535,65535,65535). Strange but true...