more to file seizes after 64K (65536) lines

  • Thread starter Thread starter Walter Briscoe
  • Start date Start date
W

Walter Briscoe

%windir%\system32\more.com pages its input to the screen.

If the output is a file, the input is copied through without pagination.

I have just discovered that this mechanism changes after 64K lines.
more then prompts at every succeeding line.

I fell over this by accident on redirecting a .BAT to a file.
(script.bat > file). The script seized after about 6Mb.
I looked at the output file and found the last line was a more prompt.
I supplied a return to script.bat and it immediately asked for another.

This behavior applies to W2K SP3 and XP without service packs.

It looks like a bug to me.
Does it apply in the latest versions?

Demonstration: The following uses the XP more.com

C:\watcom) type source.dir | e:\windows\system32\more.com > output
The process tried to write to a nonexistent pipe.
^C
C:\watcom) tail output
2004/01/05 12:31 1,620 C:\watcom\source\bld\f77\lg86\obj386\fcstruct.obj
2004/01/05 12:31 4,664 C:\watcom\source\bld\f77\lg86\obj386\fcsubpgm.obj
-- More -- 2004/01/08 12:31 20,125 C:\watcom\source\bld\f77\lg86\obj386\fctab.obj
-- More -- 2004/01/05 12:31 3,938 C:\watcom\source\bld\f77\lg86\obj386\fcuchk.obj
-- More -- 2004/01/05 12:31 1,154 C:\watcom\source\bld\f77\lg86\obj386\fdutil.obj
-- More -- 2004/01/05 12:31 1,308 C:\watcom\source\bld\f77\lg86\obj386\fexit.obj
-- More -- 2004/01/05 12:31 1,041 C:\watcom\source\bld\f77\lg86\obj386\fgetcmd.obj
-- More -- 2004/01/05 12:31 1,320 C:\watcom\source\bld\f77\lg86\obj386\fgetenv.obj
-- More -- 2004/01/05 12:31 3,597 C:\watcom\source\bld\f77\lg86\obj386\fgnobj.obj
-- More --
C:\watcom) :: I supplied return seven times
C:\watcom) db e:\windows\system32\more.com
2001/08/23 12:00 15,872 e:\windows\system32\more.com


C:\watcom)
 
In said:
%windir%\system32\more.com pages its input to the screen.

If the output is a file, the input is copied through without
pagination.

I have just discovered that this mechanism changes after 64K
lines. more then prompts at every succeeding line.

I fell over this by accident on redirecting a .BAT to a file.
(script.bat > file). The script seized after about 6Mb.
I looked at the output file and found the last line was a more
prompt. I supplied a return to script.bat and it immediately asked
for another.

This behavior applies to W2K SP3 and XP without service packs.

It looks like a bug to me.
Does it apply in the latest versions?
Click to expand...


Same here on W2K, SP4.

But I have never before intentionally piped typed output through the
more filter and then redirected to a file.
 
But I have never before intentionally piped typed output through the
more filter and then redirected to a file.
Click to expand...

That would ordinarily be considered a weird and strange programming
mistake that only a rank beginner would make ... or somebody who
hadn't slept for several days. If MORE is used, TYPE is usually
redundant; if the output is a file, the process is an extremely
elaborate way of COPYing a file and all of it is questionable. I
don't think MS can be held responsible for something that unreasonable
not working as expected. Actually, since MORE should not be used with
file targets, I would not be surprised if it tried to page every line
- it all depends on what it uses for a default screen height when the
target doesn't have that property.


T.E.D. ([email protected] - e-mail must contain "T.E.D." or my .sig in the body)
 
In message <[email protected]> of Sat, 24 Jan
2004 10:51:14 in alt.msdos.batch.nt, Ted Davis
That would ordinarily be considered a weird and strange programming
mistake that only a rank beginner would make ... or somebody who
hadn't slept for several days. If MORE is used, TYPE is usually
redundant; if the output is a file, the process is an extremely
elaborate way of COPYing a file and all of it is questionable. I
don't think MS can be held responsible for something that unreasonable
not working as expected. Actually, since MORE should not be used with
file targets, I would not be surprised if it tried to page every line
- it all depends on what it uses for a default screen height when the
target doesn't have that property.
Click to expand...

I agree with Mark and Ted that the practice looks strange.
I was modelling the failure rather than showing it.

Let me try and put the failure in context.
DB.BAT is something like DIR. /P is on by default. It is easier for me
to show what it does than explain.
C:\watcom) db /ad
2004/01/24 21:37 <DIR> C:\watcom\.
2004/01/24 21:37 <DIR> C:\watcom\..
2004/01/16 00:23 <DIR> C:\watcom\exec_cmd
2004/01/23 08:04 <DIR> C:\watcom\logs
2003/12/17 09:27 <DIR> C:\watcom\os2tools
2004/01/24 09:36 <DIR> C:\watcom\source
2004/01/19 08:11 <DIR> C:\watcom\source.0
2004/01/12 06:34 <DIR> C:\watcom\source.1
2004/01/12 01:15 <DIR> C:\watcom\source.3


C:\watcom) db /ad > dummy

C:\watcom)

I did db /s > file. This tried to create a 137,322 line file and seized.
I found the reported behavior in more.

As it happens, db /s /-p > file does not seize as it does not use more.

In UNIX Bourne and Korn shells, I could test stdout is connected to a
screen with something like if test -t 1. AFAIK, cmd.exe has no similar
capability. I can obviously produce an isscreen command which takes a
file descriptor number as an argument and has a return code of zero for
a screen and one otherwise. I can't justify the time to do that.

Here is my script which pleases me! It usually serves my purpose well.
It works to an extent with a W9X COMMAND.COM. (That "shell" does not
allow .BAT output redirection.) Some time ago, a much faster version
which used for commands but which does not support /s was shown here. I
saw no easy method of adding /s support and the tool has served well.

@echo %db% off
:: db.bat
::
:: Create a DIR listing without headers or trailers.
::
:: When Who What
:: 2003-06-06 W.Briscoe Original
:: 2003-06-06 W.Briscoe Made portable between W2K and W95
:: 2003-06-06 W.Briscoe Grabbed backslash from directory name.
:: (\\\\ was an unreliable literal backslash.)
:: 2003-06-06 W.Briscoe Slash inconsistent between C:\ and C:\foo
:: 2003-06-07 W.Briscoe Avoid find from path - mismatch with UNIX find.
:: 2003-09-27 W.Briscoe Trimmed 5 spaces from each output line, condensed note_dir
:: 2003-10-11 W.Briscoe Indented each line by two bytes to show usenet wrapping
::
if not "%os%"=="" for %%c in (setlocal "set pfxlen=39" "set wexedir=%windir%\system32") do %%~c
if "%os%"=="" for %%c in ( "set pfxlen=44" "set wexedir=%windir%\command" ) do %%c
set note_dir="/^ *Directory of \(..\(.\).*\)/{s//\1\2/;s:\(.\)\1$:\1:;h;d;}"
set note_dir="/^ *Directory of /{s///;s:[^\]$:&\\\\:;h;d;}"
set no_noise="/^$/d;/^ /d"
set reformat="G;s/\(.\{%pfxlen%\}\)\(.*\)\n\(.*\)/\1\3\2/;s/ //"
set reformat="G;s/\(^.\{17\} *[^ ]* \)\(.*\)\n\(.*\)/\1\3\2/;s/ //"
set reformat="G;s/\(^.\{17\} *[^ ]* *\)\(.*\)\n\(.*\)/\1\3\2/;s/ //"
set args=

:sloop
set args=%args% %1
shift
if not %1.==. goto sloop
set pager=sed -n p
echo %dircmd% %args% | %wexedir%\find.EXE /i "/p" > nul % assume /p not followed by /-p %
if not errorlevel 1 set pager=more % W2K more adds a blank line %
echo %dircmd% %args% | %wexedir%\find.EXE /i "/-p" > nul % assume /p not followed by /-p %
if not errorlevel 1 set pager=sed -n p % W2K more adds a blank line %

dir %args% /-p | sed -e %note_dir% -e %no_noise% -e %reformat% | %pager%
 
In said:
In message <[email protected]> of Sat, 24
Jan 2004 10:51:14 in alt.msdos.batch.nt, Ted Davis
That would ordinarily be considered a weird and strange
programming mistake that only a rank beginner would make ... or
somebody who hadn't slept for several days. If MORE is used, TYPE
is usually redundant; if the output is a file, the process is an
extremely elaborate way of COPYing a file and all of it is
questionable. I don't think MS can be held responsible for
something that unreasonable not working as expected. Actually,
since MORE should not be used with file targets, I would not be
surprised if it tried to page every line - it all depends on what
it uses for a default screen height when the target doesn't have
that property.
Click to expand...

I agree with Mark and Ted that the practice looks strange.
I was modelling the failure rather than showing it.

Let me try and put the failure in context.
DB.BAT is something like DIR. /P is on by default. It is easier
for me to show what it does than explain.
C:\watcom) db /ad
2004/01/24 21:37 <DIR> C:\watcom\.
2004/01/24 21:37 <DIR> C:\watcom\..
2004/01/16 00:23 <DIR> C:\watcom\exec_cmd
2004/01/23 08:04 <DIR> C:\watcom\logs
2003/12/17 09:27 <DIR> C:\watcom\os2tools
2004/01/24 09:36 <DIR> C:\watcom\source
2004/01/19 08:11 <DIR> C:\watcom\source.0
2004/01/12 06:34 <DIR> C:\watcom\source.1
2004/01/12 01:15 <DIR> C:\watcom\source.3


C:\watcom) db /ad > dummy

C:\watcom)

I did db /s > file. This tried to create a 137,322 line file and
seized. I found the reported behavior in more.

As it happens, db /s /-p > file does not seize as it does not use
more.

In UNIX Bourne and Korn shells, I could test stdout is connected
to a screen with something like if test -t 1. AFAIK, cmd.exe has
no similar capability. I can obviously produce an isscreen command
which takes a file descriptor number as an argument and has a
return code of zero for a screen and one otherwise. I can't
justify the time to do that.

Here is my script which pleases me! It usually serves my purpose
well. It works to an extent with a W9X COMMAND.COM. (That "shell"
does not allow .BAT output redirection.) Some time ago, a much
faster version which used for commands but which does not support
/s was shown here. I saw no easy method of adding /s support and
the tool has served well.

@echo %db% off
:: db.bat
::
:: Create a DIR listing without headers or trailers.
::
:: When Who What
:: 2003-06-06 W.Briscoe Original
:: 2003-06-06 W.Briscoe Made portable between W2K and W95
:: 2003-06-06 W.Briscoe Grabbed backslash from directory name.
:: (\\\\ was an unreliable literal
:: backslash.)
:: 2003-06-06 W.Briscoe Slash inconsistent between C:\ and C:\foo
:: 2003-06-07 W.Briscoe Avoid find from path - mismatch with UNIX
:: find. 2003-09-27 W.Briscoe Trimmed 5 spaces from each output
:: line, condensed note_dir 2003-10-11 W.Briscoe Indented each
:: line by two bytes to show usenet wrapping
::
if not "%os%"=="" for %%c in (setlocal "set pfxlen=39" "set
wexedir=%windir%\system32") do %%~c if "%os%"=="" for %%c in
( "set pfxlen=44" "set wexedir=%windir%\command" ) do
%%c set note_dir="/^ *Directory of
\(..\(.\).*\)/{s//\1\2/;s:\(.\)\1$:\1:;h;d;}" set note_dir="/^
*Directory of /{s///;s:[^\]$:&\\\\:;h;d;}" set
no_noise="/^$/d;/^ /d" set
reformat="G;s/\(.\{%pfxlen%\}\)\(.*\)\n\(.*\)/\1\3\2/;s/ //"
set reformat="G;s/\(^.\{17\} *[^ ]* \)\(.*\)\n\(.*\)/\1\3\2/;s/
//" set reformat="G;s/\(^.\{17\} *[^ ]*
*\)\(.*\)\n\(.*\)/\1\3\2/;s/ //" set args=

:sloop
set args=%args% %1
shift
if not %1.==. goto sloop
set pager=sed -n p
echo %dircmd% %args% | %wexedir%\find.EXE /i "/p" > nul % assume
/p not followed by /-p % if not errorlevel 1 set pager=more %
W2K more adds a blank line % echo %dircmd% %args% |
%wexedir%\find.EXE /i "/-p" > nul % assume /p not followed by
/-p % if not errorlevel 1 set pager=sed -n p % W2K more adds a
blank line %

dir %args% /-p | sed -e %note_dir% -e %no_noise% -e %reformat% |
%pager%
Click to expand...

I do not believe that a "page" switch (/P) for an internal command
uses the more filter at all. By that I mean not only is more.com not
invoked directly, but that the paging code must reside inside
cmd.exe. Having said that I _assume_ that the same 64KB limit is
involved and very likely the same code is utilized. I further assume
this is a leftover from earlier DOS days. Curious to hear from
people more familiar with the internal code involved.

One might need to test DIRCMD env. var. and decide to use a "/-d" to
avoid the problem.
 
Back
Top