Redirections and pipes fail in cmd.exe backtick for commands

[Walter Briscoe]

I am running an XP cmd.exe on a W2K system.
I have a script in which the output of one command is fed as an argument
to a second. My command is long and I have hidden some of the details in
foo.bat, bar.bat, and fubar.bat.

) type foo.bat
@echo foo.bat
:: Line in foo.bat containing the word pharlap

) type bar.bat
@echo bar.bat
:: Line in bar.bat containing the word pharlap
:: Does Microsoft have a command to copy standard input to standard output?
@cat

) type fubar.bat
@echo off
foo 2>nul | bar

) foo 2>nul | bar
bar.bat
foo.bat

) fubar
bar.bat
foo.bat

)

I surround long lines in chevron pairs ("<" and ">") to try to persuade
news readers not to split them.

My original command was:
<c:\cygwin\bin\bash.exe -c "grep -n -i -- 'pharlap' `wmake -h -n -d -a -f makefile 2> nul | sed '/^.*Entering file (\([^)]*\).*/!d;s//\1/'`">

The simplified command's operation is:
<) c:\cygwin\bin\bash.exe -c "echo `cmd /c foo 2>nul | cmd /c bar`">
bar.bat foo.bat

<) c:\cygwin\bin\bash.exe -c "grep -n -i -- pharlap `cmd /c foo 2>nul | cmd /c bar`">
bar.bat:2::: Line in bar.bat containing the word pharlap
foo.bat:2::: Line in foo.bat containing the word pharlap

)

Yesterday, I realized I could replace the dependency on bash with
another on xargs and came up with the slightly simpler command:
<wmake -h -n -d -a -f makefile 2>nul | sed "/^.*Entering file (\([^)]*\).*/!d;s//\1/;s:\\\:&&:g" | xargs grep -n -i pharlap>

This simplifies to the following:
) foo 2>nul | bar | xargs grep -n -i pharlap
bar.bat:2::: Line in bar.bat containing the word pharlap
foo.bat:2::: Line in foo.bat containing the word pharlap

)

So much for history! Because xargs is not available in my target
environment, I want to use the backtick or quoted-command facilities of
cmd.exe's for command. However, they seem to give higher priority to ">"
(redirection) and "|" (pipe) than "`" (backtick) and I have no
reasonable way to work around that.
They also produce output in many lines rather than one. I have a
workaround for that for my particular requirement.

) :: backtick and quote have lower priority than redirect and pipe
) for /F "usebackq" %c in (`foo 2>nul | bar`) do @rem
2> was unexpected at this time.
) for /F "usebackq" %c in (`foo | bar`) do @rem
| was unexpected at this time.
) for /F %c in ('foo 2>nul | bar') do @rem
2> was unexpected at this time.
) for /F %c in ('foo | bar') do @rem
| was unexpected at this time.
)
) :: I can use a wrapper batch, but do not want to.
) :: Even if I do, I can not force output onto one line
) for /F "usebackq" %c in (`fubar `) do @echo %c
bar.bat
foo.bat

) :: I think eol=? means ignore characters after ? in text lines
) for /F "usebackq eol=?" %c in (`fubar `) do @echo %c
bar.bat
foo.bat

) :: grep is called once for each file rather than once for all files
<) for /F "usebackq eol=?" %c in (`fubar `) do @grep -n -i -- pharlap %c>
2::: Line in bar.bat containing the word pharlap
2::: Line in foo.bat containing the word pharlap

) :: I can cheat to force grep to put names on its outputs
<) for /F "usebackq eol=?" %c in (`fubar `) do @grep -n -i -- pharlap %c nul>
bar.bat:2::: Line in bar.bat containing the word pharlap
foo.bat:2::: Line in foo.bat containing the word pharlap

) :: This is more economical in characters but aligned for comparison
<) for /F %c in ('fubar ') do @grep -n -i -- pharlap %c nul>
bar.bat:2::: Line in bar.bat containing the word pharlap
foo.bat:2::: Line in foo.bat containing the word pharlap

) :: This is the unaligned version
) for /F %c in ('fubar') do @grep -n -i -- pharlap %c nul
bar.bat:2::: Line in bar.bat containing the word pharlap
foo.bat:2::: Line in foo.bat containing the word pharlap

)

Can somebody please point out how I can do the following in a for
command or confirm my analysis that Microsoft does not yet allow for
them?
1) use ">" (redirection) and "|" (pipe)
2) force output onto a single line

 

[Matthias Tacke]

Walter Briscoe wrote:
Hello Walter,
I didn't understand all aspects of your not so short story
Seems you choose the ) as your prompt char what irritates me.

I am running an XP cmd.exe on a W2K system.
I have a script in which the output of one command is fed as an argument
to a second. My command is long and I have hidden some of the details in
foo.bat, bar.bat, and fubar.bat.

Can somebody please point out how I can do the following in a for
command or confirm my analysis that Microsoft does not yet allow for
them?
1) use ">" (redirection) and "|" (pipe)

Escape with a caret ^|

2) force output onto a single line

Use the caret as the last char on a line for better formatting.
The shell understands that the command proceeds on the next line.

 

[Dean Wells [MVP]]

Short of reading your entire post (I can't bring myself to do it, sorry
... I've taken a look at your closing question and think I
understand some of what you're after. Assuming I do, it appears you're
looking to delay the expansion of the redirection and/or piping symbols.
Let's assume we have a file named foo containing the following 4 lines -

[file]
This is line 1
Now we have line 2
Penultimately, we now have line 3
Finally we've got line 4
[/file]

.... and assume we want to output its content using a for command to the
console -

@echo off
for /f "tokens=*" %%a in (\foo) do echo %%a

.... now we'd like to redirect standard out to a file -

@echo off
for /f "tokens=*" %%a in (\foo) do echo %%a>\bar

.... now we'd like to keep it all on one line (painful syntax) -

@echo off
for /f "tokens=*" %%a in (\foo) do set /p nul=%%a<nul

.... finally, we'd only like to hit lines that contain the word "have" -

@echo off
for /f "tokens=*" %%a in (type \foo ^| 'find /i "have"') do set /p
nul=%%a<nul

I'm hoping that covers some of your requirements.

Dean

 

[Ted Davis]

I am running an XP cmd.exe on a W2K system.

Can somebody please point out how I can do the following in a for
command or confirm my analysis that Microsoft does not yet allow for
them?
1) use ">" (redirection) and "|" (pipe)
2) force output onto a single line

I assume you are talking about something like

for /f %A in ('foo | bar') do baz

That has to be

for /f %A in ('foo ^| bar') do baz

so that the pipe is not interpreted as part of the top level FOR
command but instead is parsed with the secondary command.


T.E.D. ([email protected])
SPAM filter: Messages to this address *must* contain "T.E.D."
somewhere in the body or they will be automatically rejected.

 

[Walter Briscoe]

In message <[email protected]> of Fri, 16 Jul 2004
08:57:53 in alt.msdos.batch.nt, "Dean Wells [MVP]" <[email protected]
ology.com> writes
Thanks to all who, though bored with my original posting, gleaned enough
to make useful suggestions: Matthias and T.E.D. suggested escaping
redirection (>) and pipe (|) with caret (^); Matthias also suggested
escaping end of line to allow a single command to be split on several
physical lines. Both ideas are demonstrated in:

W:\source\bld\posix\misc\pharlap) for /f %a in ('foo 2^>nul ^| ^
More? bar') do @echo %a
bar.bat
foo.bat

W:\source\bld\posix\misc\pharlap)

As Dean's cryptic magic with set /p nul looks like having most potential
to address my remaining requirements, I reply thankfully to his posting:

Short of reading your entire post (I can't bring myself to do it, sorry
... I've taken a look at your closing question and think I
understand some of what you're after. Assuming I do, it appears you're
looking to delay the expansion of the redirection and/or piping symbols.
Let's assume we have a file named foo containing the following 4 lines -

[file]
This is line 1
Now we have line 2
Penultimately, we now have line 3
Finally we've got line 4
[/file]

... and assume we want to output its content using a for command to the
console -

@echo off
for /f "tokens=*" %%a in (\foo) do echo %%a

... now we'd like to redirect standard out to a file -

Where you used "\file", I found it more convenient to use "file".

@echo off
for /f "tokens=*" %%a in (\foo) do echo %%a>\bar

I found that only put the last line in \bar. I think you meant to say
something like
@echo off
if exist \bar del \bar
for /f "tokens=*" %%a in (\foo) do echo %%a>>\bar

... now we'd like to keep it all on one line (painful syntax) -

@echo off
for /f "tokens=*" %%a in (\foo) do set /p nul=%%a<nul

That is powerful magic! Can you point to its discovery?

... finally, we'd only like to hit lines that contain the word "have" -

@echo off
for /f "tokens=*" %%a in (type \foo ^| 'find /i "have"') do set /p
nul=%%a<nul

I'm hoping that covers some of your requirements.

It does cover some of my requirements. I now have enough to make my
original script work. I can't yet combine your ideas. I would be able to
do so if I could generate something like:
echo This is line 1 Now we have line 2 Penultimately, ...

That is, I want to feed a set of lines as a set of parameters to a
command.

In UNIX environments (with Microsoft commands), I would be able to do
this with something like "echo `type foo`" or "echo $(type foo)".
Microsoft provides more of a challenge.

 

[Matthias Tacke]

Walter Briscoe wrote:

That is powerful magic! Can you point to its discovery?

There was a discussion in the past month IIRC it was one of the former
regulars but I dont' remember the name.

Once you know it's easy If you read the help of set /? the key point
is that the question text doesn't advance to the next line, so the input
will be straight behind. This misuse of the command doesn't even need a
(dummy) variable:

@echo off
setlocal
set /p ="This is my first line. "<nul
echo This is the second

The second trick with set /p is to set an environment var with the first
line of a file

set /P Myfirstline=<%~f0

It does cover some of my requirements. I now have enough to make my
original script work. I can't yet combine your ideas. I would be able to
do so if I could generate something like:
echo This is line 1 Now we have line 2 Penultimately, ...

That is, I want to feed a set of lines as a set of parameters to a
command.

Here it gets difficult, I don't know a way to escape a return on the
command line. To group a set of words as one argument you have to quote
them. Inside the the batch you may dequote by using the tilde between
the percent sign and the number.
==screen=copy=========================================================
C:\Test>type test-arg.cmd
::testarg.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo off
setlocal
echo [%~1] - %1
echo [%~2] - %2
C:\Test>test-arg "first group line" for_Test_one_word
[first group line] - "first group line"
[for_Test_one_word] - for_Test_one_word
==screen=copy=========================================================

In UNIX environments (with Microsoft commands), I would be able to do
this with something like "echo `type foo`" or "echo $(type foo)".
Microsoft provides more of a challenge.

It's not that easy to include the output of a command in an other
command. With for /f you can workaround like this example to output the
date variants of a file:
:ate-CWA.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This is a quite slow demonstration batch
@echo off
setlocal
echo Created Written Accessed Name
echo ========== ===== ========== ===== ========== ===== ===========
for /F "tokens=1-3,*" %%A in ('DIR %1 /TC /A-D^|findstr "^[0-9]"') do (
for /F "tokens=1-2" %%E in ('DIR "%%~D" /TW ^|find "%%~D"') do (
for /F "tokens=1-2" %%G in ('DIR "%%~D" /TA ^|find "%%~D"') do (
Echo %%A %%B %%E %%F %%G %%H %%D
)))
:ate-CWA.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::

Your first step to post in ambnt and mpwca was right. My advanced batch
knowledge was gathered in these groups - a thank at this point to all
the other regulars here.

Really new questions are rare - to find the answers use google:
Replace "your+keywords" with your keywords
http://www.google.com/groups?group=alt.msdos.batch.nt&q=your+keywords
http://www.google.com/groups?group=microsoft.public.win2000.cmdprompt.admin&q=your+keywords
or directly open the advanced Group Search page with preentered group:
http://www.google.com/advanced_group_search?group=alt.msdos.batch.nt
http://www.google.com/advanced_group_search?group=microsoft.public.win2000.cmdprompt.admin

HTH

 

[Walter Briscoe]

In message <[email protected]> of Mon, 19 Jul 2004
11:30:18 in alt.msdos.batch.nt, Matthias Tacke <[email protected]>
writes
Thanks for a response which needs a lot of thought before I can respond.
I can do no more than what follows:

Walter Briscoe wrote:

There was a discussion in the past month IIRC it was one of the former
regulars but I dont' remember the name.

Once you know it's easy If you read the help of set /? the key point
is that the question text doesn't advance to the next line, so the input
will be straight behind. This misuse of the command doesn't even need a
(dummy) variable:

@echo off
setlocal
set /p ="This is my first line. "<nul
echo This is the second

The second trick with set /p is to set an environment var with the first
line of a file

set /P Myfirstline=<%~f0

I was unable to make any sense of this. ;(

It does cover some of my requirements. I now have enough to make my
original script work. I can't yet combine your ideas. I would be able to
do so if I could generate something like:
echo This is line 1 Now we have line 2 Penultimately, ...

That is, I want to feed a set of lines as a set of parameters to a
command.

Here it gets difficult, I don't know a way to escape a return on the
command line. To group a set of words as one argument you have to quote
them. Inside the the batch you may dequote by using the tilde between
the percent sign and the number.
==screen=copy=========================================================
C:\Test>type test-arg.cmd
::testarg.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo off
setlocal
echo [%~1] - %1
echo [%~2] - %2
C:\Test>test-arg "first group line" for_Test_one_word
[first group line] - "first group line"
[for_Test_one_word] - for_Test_one_word
==screen=copy=========================================================

I knew that. The example is pedagogical.

In UNIX environments (with Microsoft commands), I would be able to do
this with something like "echo `type foo`" or "echo $(type foo)".
Microsoft provides more of a challenge.

It's not that easy to include the output of a command in an other
command. With for /f you can workaround like this example to output the
date variants of a file:
:ate-CWA.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This is a quite slow demonstration batch
@echo off
setlocal
echo Created Written Accessed Name
echo ========== ===== ========== ===== ========== ===== ===========
for /F "tokens=1-3,*" %%A in ('DIR %1 /TC /A-D^|findstr "^[0-9]"') do (
for /F "tokens=1-2" %%E in ('DIR "%%~D" /TW ^|find "%%~D"') do (
for /F "tokens=1-2" %%G in ('DIR "%%~D" /TA ^|find "%%~D"') do (
Echo %%A %%B %%E %%F %%G %%H %%D
)))
:ate-CWA.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::

That is fine if one knows the expected number of arguments.

I was unable to make a nested for work. The following is presented in
short lines. I tried long ones with similar failure. ;(

set x=for /f ^"tokens=*^" ^%a in (foo) do @set /p nul=^%a ^^^^^^^<nul

echo %x%
for /f "tokens=*" %a in (foo) do @set /p nul=%a ^<nul

for /f "tokens=*" %b in ('%x%') do echo %b
The system cannot find the file 'for /f "tokens=*" %a in (foo.

The following hits my requirement at the cost of an intermediary file.

( for /f "tokens=*" %a in (foo) do @set /p nul=%a <nul ) > bar

for /f "tokens=*" %a in (bar) do @echo %a
This is line 1 Now we have line 2 Penultimately, we now have line 3 Finally we've got line 4

Your first step to post in ambnt and mpwca was right. My advanced batch
knowledge was gathered in these groups - a thank at this point to all
the other regulars here.

Really new questions are rare - to find the answers use google:
Replace "your+keywords" with your keywords

Finding the keywords is the hard part. ;(

http://www.google.com/groups?group=alt.msdos.batch.nt&q=your+keywords
http://www.google.com/groups?group=microsoft.public.win2000.cmdprompt.a
dmin&q=your+keywords
or directly open the advanced Group Search page with preentered group:
http://www.google.com/advanced_group_search?group=alt.msdos.batch.nt
http://www.google.com/advanced_group_search?group=microsoft.public.win2
000.cmdprompt.admin

My news-reader mangles your long URI on replies. Enclosing them in
chevrons ('<' and '>') gives some measure of relief and does not require
their presentation on single lines.

<http://www.google.com/groups?group=alt.msdos.batch.nt&q=your+keywords>
<http://www.google.com/groups?group=microsoft.public.win2000.cmdprompt.a
dmin&q=your+keywords>
or directly open the advanced Group Search page with preentered group:
<http://www.google.com/advanced_group_search?group=alt.msdos.batch.nt>
<http://www.google.com/advanced_group_search?group=microsoft.public.win2
000.cmdprompt.admin>

HTH

It does!

 

[Matthias Tacke]

Thanks for a response which needs a lot of thought before I can respond.
I can do no more than what follows:
I was unable to make any sense of this. ;(

Another demonstration batch, %0 is argument count zero resolving to the
name of the current batch. The ~f dequotes the name and extends to the
full name including the drive and path.
==screen=copy=========================================================
C:\Test>TYPE Walter002.cmd
::First Line of Batch
@echo off
set /P Myfirstline=<%~f0
echo MyOwnFileName=%~f0
echo MyFirstLine=%Myfirstline%

C:\Test>Walter002.cmd
MyOwnFileName=C:\test\Walter002.cmd
MyFirstLine=::First Line of Batch

C:\Test>
==screen=copy=========================================================

That is fine if one knows the expected number of arguments.

I was unable to make a nested for work. The following is presented in
short lines. I tried long ones with similar failure. ;(

set x=for /f ^"tokens=*^" ^%a in (foo) do @set /p nul=^%a ^^^^^^^<nul
echo %x%

set is quite superior to the version in dos but it can't do that, if you
expect anything else than x containing the string following the equal
sign.

The ways to go are different to the bash or whatever shell you are used
to.
X is an envireonment variable which can only hold one string/line, the
return can't be escaped. (I did mention this already.)

for /f "tokens=*" %a in (foo) do @set /p nul=%a ^<nul

for /f "tokens=*" %b in ('%x%') do echo %b
The system cannot find the file 'for /f "tokens=*" %a in (foo.

Expectble since the above doesn't work.

The following hits my requirement at the cost of an intermediary file.

( for /f "tokens=*" %%a in (foo) do @set /p nul=%%a <nul ) > bar

for /f "tokens=*" %a in (bar) do @echo %a
This is line 1 Now we have line 2 Penultimately, we now have line 3 Finally we've got line 4

At the command line this works giving the above output:
for /f "tokens=*" %a in (foo) do @set /p nul=%a <nul

In a batch percent sign have to be doubled (same output)
::Walter003.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo off & setlocal
for /f "tokens=*" %%a in (foo) do set /p nul=%%a <nul
echo/
::Walter003.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::

My news-reader mangles your long URI on replies.

My one gives me a warning but lets me proceed.

Enclosing them in
chevrons ('<' and '>') gives some measure of relief and does not require
their presentation on single lines.

Sorry the url with a break don't work here.

<http://www.google.com/groups?group=microsoft.public.win2000.cmdprompt.a
dmin&q=your+keywords>

X'Post + F'up2 ambn