Process Output and Error Redirection and Synchronization Between Threads (C#)

  • Thread starter Thread starter Damien White
  • Start date Start date
D

Damien White

I am hoping someone can help me with a threading issue... I have created a
program which runs a console program and displays the output in a
RichTextBox on a Windows form. Here is the code I'm using for the process
to redirect the console output

m_Process = new Process();
m_Process.StartInfo.FileName = Path;
m_Process.StartInfo.Arguments = "\"" + ScriptPath.Text + "\"";
m_Process.StartInfo.UseShellExecute = false;
m_Process.StartInfo.CreateNoWindow = true;
m_Process.StartInfo.RedirectStandardOutput = true;
m_Process.StartInfo.RedirectStandardError = true;
m_Process.Start();
I'm using Everything with it works great, but I'm having a thread syncing
problem. The console program I am running is a script interpreter. The
program runs through the script and lists each line out to the console. My
problem occurs when an error happens. I have 2 threads running to capture
each input stream from the console (Output and Error). My issue is the
program runs listing out the Output from the console (using my output
thread). When an error occurs in the script, the error thread fires. The
error thread prints out the error message, but it's not in-sync with the
output buffer. Basically, it looks something like this....

LINE 1: 1> Good Code (from Output buffer)
LINE 2: 2> Good Code (from Output buffer)
LINE 3: Error Message (from Error buffer)
LINE 4: 3> Good Code (from Output buffer)
LINE 5: 4> Good Code (from Output buffer)
LINE 6: 5> Good Code (from Output buffer)
LINE 7: 6> Bad Code (from Output buffer)
LINE 8: 7> Good Code (from Output buffer)

The error message is printed out before the error "occurs". This obviously
has to do with timing between the two threads. The desired result is to
have the error message (LINE 3 in the example) be printed after the
offending code (LINE 7 in the example). I can't seem to figure out the best
way to do this.

Just for completeness, here are my 2 thread loops

private void StreamError()
{
string strLine = m_Process.StandardError.ReadLine();
try
{
do
{
if (strLine.Length != 0)
{
AddText(strLine);
}
strLine = m_Process.StandardError.ReadLine();
}while(strLine != null);
}
catch (Exception ex)
{
AddErrorText("ERROR STREAM");
AddErrorText(ex.Message);
AddErrorText(ex.StackTrace);
}
}
private void StreamOutput()
{
string strLine = m_Process.StandardOutput.ReadLine();
try
{
do
{
if (strLine.Length != 0)
{
AddText(strLine);
}
strLine = m_Process.StandardOutput.ReadLine();
}while(strLine != null);
}
catch (Exception ex)
{
AddErrorText("OUTPUT STREAM");
AddErrorText(ex.Message);
AddErrorText(ex.StackTrace);
}
}

Thanks in advance for the help!
-Damien
 
Hello Damien,

Thanks for posting in the group.

Sorry for late response due to weekend. Now I am looking into this problem
and need to do some test first on my side. I will return with further
information here.


Best regards,
Yanhong Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
!From: "Damien White" <[email protected]>
!Subject: Process Output and Error Redirection and Synchronization Between
Threads (C#)
!Date: Fri, 17 Oct 2003 10:09:52 -0400
!Lines: 87
!X-Priority: 3
!X-MSMail-Priority: Normal
!X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
!X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
!Message-ID: <#[email protected]>
!Newsgroups: microsoft.public.dotnet.framework
!NNTP-Posting-Host: 12.34.213.90
!Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13.phx.gbl
!Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework:56419
!X-Tomcat-NG: microsoft.public.dotnet.framework
!
!I am hoping someone can help me with a threading issue... I have created a
!program which runs a console program and displays the output in a
!RichTextBox on a Windows form. Here is the code I'm using for the process
!to redirect the console output
!
!m_Process = new Process();
!m_Process.StartInfo.FileName = Path;
!m_Process.StartInfo.Arguments = "\"" + ScriptPath.Text + "\"";
!m_Process.StartInfo.UseShellExecute = false;
!m_Process.StartInfo.CreateNoWindow = true;
!m_Process.StartInfo.RedirectStandardOutput = true;
!m_Process.StartInfo.RedirectStandardError = true;
!m_Process.Start();
!I'm using Everything with it works great, but I'm having a thread syncing
!problem. The console program I am running is a script interpreter. The
!program runs through the script and lists each line out to the console. My
!problem occurs when an error happens. I have 2 threads running to capture
!each input stream from the console (Output and Error). My issue is the
!program runs listing out the Output from the console (using my output
!thread). When an error occurs in the script, the error thread fires. The
!error thread prints out the error message, but it's not in-sync with the
!output buffer. Basically, it looks something like this....
!
!LINE 1: 1> Good Code (from Output buffer)
!LINE 2: 2> Good Code (from Output buffer)
!LINE 3: Error Message (from Error buffer)
!LINE 4: 3> Good Code (from Output buffer)
!LINE 5: 4> Good Code (from Output buffer)
!LINE 6: 5> Good Code (from Output buffer)
!LINE 7: 6> Bad Code (from Output buffer)
!LINE 8: 7> Good Code (from Output buffer)
!
!The error message is printed out before the error "occurs". This obviously
!has to do with timing between the two threads. The desired result is to
!have the error message (LINE 3 in the example) be printed after the
!offending code (LINE 7 in the example). I can't seem to figure out the
best
!way to do this.
!
!Just for completeness, here are my 2 thread loops
!
!private void StreamError()
!{
! string strLine = m_Process.StandardError.ReadLine();
! try
! {
! do
! {
! if (strLine.Length != 0)
! {
! AddText(strLine);
! }
! strLine = m_Process.StandardError.ReadLine();
! }while(strLine != null);
! }
! catch (Exception ex)
! {
! AddErrorText("ERROR STREAM");
! AddErrorText(ex.Message);
! AddErrorText(ex.StackTrace);
! }
!}
!private void StreamOutput()
!{
! string strLine = m_Process.StandardOutput.ReadLine();
! try
! {
! do
! {
! if (strLine.Length != 0)
! {
! AddText(strLine);
! }
! strLine = m_Process.StandardOutput.ReadLine();
! }while(strLine != null);
! }
! catch (Exception ex)
! {
! AddErrorText("OUTPUT STREAM");
! AddErrorText(ex.Message);
! AddErrorText(ex.StackTrace);
! }
!}
!
!Thanks in advance for the help!
!-Damien
!
!
!
 
Hello Damien,

I have written a sample to test it and did reproduce the same issue. I used
the following code as the executable console program:
Console.WriteLine ("Standard Output Line 1");
Console.WriteLine ("Standard Output Line 2");
Console.WriteLine ("Standard Output Line 3");
Console.WriteLine ("Standard Output Line 4, next line is SEL 1");
Console.Error.WriteLine ("Standard Error Line 1, former line is SOL 4");
Console.Error.WriteLine ("Standard Error Line 2");
Console.Error.WriteLine ("Standard Error Line 3, next line will be SOL
5");
Console.WriteLine ("Standard Output Line 5, former line is SEL 2");
Console.WriteLine ("Standard Output Line 6");
Console.WriteLine ("Standard Output Line 7");
Console.WriteLine ("Standard Output Line 8");

After that, we could see that the sequence of those output/error output is
random. In fact, I think the problem is caused by the executing of two
threads. In the program, we could see that the two threads read standard
output and error output from stream independently. So they could keep
sequence in their own threads correctly. That is to say, standard output
line 2 is always following standard output line 1, error output line 2 is
always following error output line 1. However, the thread is switched by
the OS, it is impossible for us to control when thread 1 is running and
when thread 2 is running. So the conbined output is random.

To workaround it, I could think of two ways:

1) Use two RichTextBox in the windows forms, one is specially for standard
output, the other is specially for error output. So the sequence of each
RichTextBox is correct.

2) Attach some sequence information into the message, for an example,
adding a int number before the standard/error output, or adding a time
information into the output. In this way, we could sort the output in
RichTextBox by ourselfves.

Does that answer your question? If you have any more concerns on it, please
feel free to post here Thanks very much.

Best regards,
Yanhong Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
!From: "Damien White" <[email protected]>
!Subject: Process Output and Error Redirection and Synchronization Between
Threads (C#)
!Date: Fri, 17 Oct 2003 10:09:52 -0400
!Lines: 87
!X-Priority: 3
!X-MSMail-Priority: Normal
!X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
!X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
!Message-ID: <#[email protected]>
!Newsgroups: microsoft.public.dotnet.framework
!NNTP-Posting-Host: 12.34.213.90
!Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13.phx.gbl
!Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework:56419
!X-Tomcat-NG: microsoft.public.dotnet.framework
!
!I am hoping someone can help me with a threading issue... I have created a
!program which runs a console program and displays the output in a
!RichTextBox on a Windows form. Here is the code I'm using for the process
!to redirect the console output
!
!m_Process = new Process();
!m_Process.StartInfo.FileName = Path;
!m_Process.StartInfo.Arguments = "\"" + ScriptPath.Text + "\"";
!m_Process.StartInfo.UseShellExecute = false;
!m_Process.StartInfo.CreateNoWindow = true;
!m_Process.StartInfo.RedirectStandardOutput = true;
!m_Process.StartInfo.RedirectStandardError = true;
!m_Process.Start();
!I'm using Everything with it works great, but I'm having a thread syncing
!problem. The console program I am running is a script interpreter. The
!program runs through the script and lists each line out to the console. My
!problem occurs when an error happens. I have 2 threads running to capture
!each input stream from the console (Output and Error). My issue is the
!program runs listing out the Output from the console (using my output
!thread). When an error occurs in the script, the error thread fires. The
!error thread prints out the error message, but it's not in-sync with the
!output buffer. Basically, it looks something like this....
!
!LINE 1: 1> Good Code (from Output buffer)
!LINE 2: 2> Good Code (from Output buffer)
!LINE 3: Error Message (from Error buffer)
!LINE 4: 3> Good Code (from Output buffer)
!LINE 5: 4> Good Code (from Output buffer)
!LINE 6: 5> Good Code (from Output buffer)
!LINE 7: 6> Bad Code (from Output buffer)
!LINE 8: 7> Good Code (from Output buffer)
!
!The error message is printed out before the error "occurs". This obviously
!has to do with timing between the two threads. The desired result is to
!have the error message (LINE 3 in the example) be printed after the
!offending code (LINE 7 in the example). I can't seem to figure out the
best
!way to do this.
!
!Just for completeness, here are my 2 thread loops
!
!private void StreamError()
!{
! string strLine = m_Process.StandardError.ReadLine();
! try
! {
! do
! {
! if (strLine.Length != 0)
! {
! AddText(strLine);
! }
! strLine = m_Process.StandardError.ReadLine();
! }while(strLine != null);
! }
! catch (Exception ex)
! {
! AddErrorText("ERROR STREAM");
! AddErrorText(ex.Message);
! AddErrorText(ex.StackTrace);
! }
!}
!private void StreamOutput()
!{
! string strLine = m_Process.StandardOutput.ReadLine();
! try
! {
! do
! {
! if (strLine.Length != 0)
! {
! AddText(strLine);
! }
! strLine = m_Process.StandardOutput.ReadLine();
! }while(strLine != null);
! }
! catch (Exception ex)
! {
! AddErrorText("OUTPUT STREAM");
! AddErrorText(ex.Message);
! AddErrorText(ex.StackTrace);
! }
!}
!
!Thanks in advance for the help!
!-Damien
!
!
!
 
For some reason this didn't seem to post yesterday, I apologize if it is a
duplicate.

Thank you for your response, and suggestions...

The first workaround you suggested would work, but wouldn't be very useful
to me as I need to know the line that the error coresponds with. The second
suggestion doesn't work either... Basically, if I capture the time or use a
counter and attempt to use that, the values are off. For instance in my
Standard Output thread, if I increment a global counter and use this when
printing out my error output, it simply prints the line number which came
before the error line (which isn't the output line in error). Is there some
way of determining which Standard Output line is the one which caused the
error? If there is, this would obviously making sync'ing the two a bit
easier. My other thought may be doing everything in one thread, but I can't
seem to work out a good way to do this.

I really appreciate the help!

Thanks,
-Damien
 
Hello Damien,

Thanks for the response. :)

Doing it in one thread won't help under this situation. It seems that when
the reading thread starts, the standard output and error output has been
filled by the console exe. So the result is almost the same.

In debug mode application, we could use StackTrace class in .NET framework
class to get some info on line number or etc. However, this information
can't be got in release build application since there is no such
information stored in exe file.

For your question, a feasible way is to do some logging code in the console
application. For an example, write to Event log instead of just write to
standard and error output. Then in the winform application, we could read
message from event log sequently. It is more convenient.

A simple code slice on writing to event log is:

// Create the source, if it does not already exist.
if(!EventLog.SourceExists("MySource")){
EventLog.CreateEventSource("MySource", "MyNewLog");
Console.WriteLine("CreatingEventSource");
}

// Create an EventLog instance and assign its source.
EventLog myLog = new EventLog();
myLog.Source = "MySource";

// Write an informational entry to the event log.
myLog.WriteEntry("Writing to event log.");

Does that answer your question? Please feel free to post here if there is
any question on it.

Best regards,
Yanhong Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
!From: "Damien White" <[email protected]>
!References: <#[email protected]>
<[email protected]>
!Subject: Re: Process Output and Error Redirection and Synchronization
Between Threads (C#)
!Date: Tue, 21 Oct 2003 08:33:20 -0400
!Lines: 190
!X-Priority: 3
!X-MSMail-Priority: Normal
!X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
!X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
!Message-ID: <#urnH#[email protected]>
!Newsgroups: microsoft.public.dotnet.framework
!NNTP-Posting-Host: 12.34.212.49
!Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13.phx.gbl
!Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework:56680
!X-Tomcat-NG: microsoft.public.dotnet.framework
!
!For some reason this didn't seem to post yesterday, I apologize if it is a
!duplicate.
!
!Thank you for your response, and suggestions...
!
!The first workaround you suggested would work, but wouldn't be very useful
!to me as I need to know the line that the error coresponds with. The
second
!suggestion doesn't work either... Basically, if I capture the time or use
a
!counter and attempt to use that, the values are off. For instance in my
!Standard Output thread, if I increment a global counter and use this when
!printing out my error output, it simply prints the line number which came
!before the error line (which isn't the output line in error). Is there
some
!way of determining which Standard Output line is the one which caused the
!error? If there is, this would obviously making sync'ing the two a bit
!easier. My other thought may be doing everything in one thread, but I
can't
!seem to work out a good way to do this.
!
!I really appreciate the help!
!
!Thanks,
!-Damien
!
!!> Hello Damien,
!>
!> I have written a sample to test it and did reproduce the same issue. I
!used
!> the following code as the executable console program:
!> Console.WriteLine ("Standard Output Line 1");
!> Console.WriteLine ("Standard Output Line 2");
!> Console.WriteLine ("Standard Output Line 3");
!> Console.WriteLine ("Standard Output Line 4, next line is SEL 1");
!> Console.Error.WriteLine ("Standard Error Line 1, former line is SOL 4");
!> Console.Error.WriteLine ("Standard Error Line 2");
!> Console.Error.WriteLine ("Standard Error Line 3, next line will be SOL
!> 5");
!> Console.WriteLine ("Standard Output Line 5, former line is SEL 2");
!> Console.WriteLine ("Standard Output Line 6");
!> Console.WriteLine ("Standard Output Line 7");
!> Console.WriteLine ("Standard Output Line 8");
!>
!> After that, we could see that the sequence of those output/error output
is
!> random. In fact, I think the problem is caused by the executing of two
!> threads. In the program, we could see that the two threads read standard
!> output and error output from stream independently. So they could keep
!> sequence in their own threads correctly. That is to say, standard output
!> line 2 is always following standard output line 1, error output line 2 is
!> always following error output line 1. However, the thread is switched by
!> the OS, it is impossible for us to control when thread 1 is running and
!> when thread 2 is running. So the conbined output is random.
!>
!> To workaround it, I could think of two ways:
!>
!> 1) Use two RichTextBox in the windows forms, one is specially for
standard
!> output, the other is specially for error output. So the sequence of each
!> RichTextBox is correct.
!>
!> 2) Attach some sequence information into the message, for an example,
!> adding a int number before the standard/error output, or adding a time
!> information into the output. In this way, we could sort the output in
!> RichTextBox by ourselfves.
!>
!> Does that answer your question? If you have any more concerns on it,
!please
!> feel free to post here Thanks very much.
!>
!> Best regards,
!> Yanhong Huang
!> Microsoft Online Partner Support
!>
!> Get Secure! - www.microsoft.com/security
!> This posting is provided "AS IS" with no warranties, and confers no
!rights.
!>
!> --------------------
!> !From: "Damien White" <[email protected]>
!> !Subject: Process Output and Error Redirection and Synchronization
Between
!> Threads (C#)
!> !Date: Fri, 17 Oct 2003 10:09:52 -0400
!> !Lines: 87
!> !X-Priority: 3
!> !X-MSMail-Priority: Normal
!> !X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
!> !X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
!> !Message-ID: <#[email protected]>
!> !Newsgroups: microsoft.public.dotnet.framework
!> !NNTP-Posting-Host: 12.34.213.90
!> !Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13.phx.gbl
!> !Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework:56419
!> !X-Tomcat-NG: microsoft.public.dotnet.framework
!> !
!> !I am hoping someone can help me with a threading issue... I have
created
!a
!> !program which runs a console program and displays the output in a
!> !RichTextBox on a Windows form. Here is the code I'm using for the
!process
!> !to redirect the console output
!> !
!> !m_Process = new Process();
!> !m_Process.StartInfo.FileName = Path;
!> !m_Process.StartInfo.Arguments = "\"" + ScriptPath.Text + "\"";
!> !m_Process.StartInfo.UseShellExecute = false;
!> !m_Process.StartInfo.CreateNoWindow = true;
!> !m_Process.StartInfo.RedirectStandardOutput = true;
!> !m_Process.StartInfo.RedirectStandardError = true;
!> !m_Process.Start();
!> !I'm using Everything with it works great, but I'm having a thread
syncing
!> !problem. The console program I am running is a script interpreter. The
!> !program runs through the script and lists each line out to the console.
!My
!> !problem occurs when an error happens. I have 2 threads running to
!capture
!> !each input stream from the console (Output and Error). My issue is the
!> !program runs listing out the Output from the console (using my output
!> !thread). When an error occurs in the script, the error thread fires.
!The
!> !error thread prints out the error message, but it's not in-sync with the
!> !output buffer. Basically, it looks something like this....
!> !
!> !LINE 1: 1> Good Code (from Output buffer)
!> !LINE 2: 2> Good Code (from Output buffer)
!> !LINE 3: Error Message (from Error buffer)
!> !LINE 4: 3> Good Code (from Output buffer)
!> !LINE 5: 4> Good Code (from Output buffer)
!> !LINE 6: 5> Good Code (from Output buffer)
!> !LINE 7: 6> Bad Code (from Output buffer)
!> !LINE 8: 7> Good Code (from Output buffer)
!> !
!> !The error message is printed out before the error "occurs". This
!obviously
!> !has to do with timing between the two threads. The desired result is to
!> !have the error message (LINE 3 in the example) be printed after the
!> !offending code (LINE 7 in the example). I can't seem to figure out the
!> best
!> !way to do this.
!> !
!> !Just for completeness, here are my 2 thread loops
!> !
!> !private void StreamError()
!> !{
!> ! string strLine = m_Process.StandardError.ReadLine();
!> ! try
!> ! {
!> ! do
!> ! {
!> ! if (strLine.Length != 0)
!> ! {
!> ! AddText(strLine);
!> ! }
!> ! strLine = m_Process.StandardError.ReadLine();
!> ! }while(strLine != null);
!> ! }
!> ! catch (Exception ex)
!> ! {
!> ! AddErrorText("ERROR STREAM");
!> ! AddErrorText(ex.Message);
!> ! AddErrorText(ex.StackTrace);
!> ! }
!> !}
!> !private void StreamOutput()
!> !{
!> ! string strLine = m_Process.StandardOutput.ReadLine();
!> ! try
!> ! {
!> ! do
!> ! {
!> ! if (strLine.Length != 0)
!> ! {
!> ! AddText(strLine);
!> ! }
!> ! strLine = m_Process.StandardOutput.ReadLine();
!> ! }while(strLine != null);
!> ! }
!> ! catch (Exception ex)
!> ! {
!> ! AddErrorText("OUTPUT STREAM");
!> ! AddErrorText(ex.Message);
!> ! AddErrorText(ex.StackTrace);
!> ! }
!> !}
!> !
!> !Thanks in advance for the help!
!> !-Damien
!> !
!> !
!> !
!>
!
!
!
 
Okay, currently wrestling with the same problem as outlined in the
original post here. I have no control over the output from the console
app I'm running as a process and I need to redirect both standard
output and error output from the process to either a file or the
console standard output. Is there any way to synchronize the stdout and
stderr, or maybe just attach both streams directly to the stdout of the
console so the output is printed "real time" as it were, i.e. exactly
in the order it appears as outputted from the console app? Any help
would be appreciated.

Cheers,

Steve
 
My
problem occurs when an error happens. I have 2 threads running to capture
each input stream from the console (Output and Error). My issue is the
program runs listing out the Output from the console (using my output
thread). When an error occurs in the script, the error thread fires. The
error thread prints out the error message, but it's not in-sync with the
output buffer. Basically, it looks something like this....

LINE 1: 1> Good Code (from Output buffer)
LINE 2: 2> Good Code (from Output buffer)
LINE 3: Error Message (from Error buffer)
LINE 4: 3> Good Code (from Output buffer)
LINE 5: 4> Good Code (from Output buffer)
LINE 6: 5> Good Code (from Output buffer)
LINE 7: 6> Bad Code (from Output buffer)
LINE 8: 7> Good Code (from Output buffer)

The error message is printed out before the error "occurs". This obviously
has to do with timing between the two threads. The desired result is to
have the error message (LINE 3 in the example) be printed after the
offending code (LINE 7 in the example). I can't seem to figure out the best
way to do this.
Click to expand...

Hi. I'm currently trying to find a solution to the same problem
mentioned above. I need to somehow redirect output (stdout & stderr)
from a process (running a console app whose output I have no control
over) to the stdout of the main console window. Is there any way I can
synchronize the two output streams of the process (stdout & stderr) to
the console or somehow tie both streams to the console stdout so all
the output is listed in "real time", i.e. in the order it's displayed
by the process app? Any help on this would be appreciated.

Cheers,

Steve

p.s. Apologies if this posts twice, I've never used Google to post
before.
 
The only way to produce proper output for me was to set "cmd.exe" as
FileName in process start options, and construct process Arguments as
string argString = "/c \"" + realCommandName + "\" " + realArgs + " >
" + someFileName + " 2>&1";

Note the extra set of double quotes in asgString to handle paths with
spaces.
The 2>&1 piece redirect program stdout and stderr to the file of your
choice.

HTH
 
Back
Top