Web Application/Service for executing long running tasks

[BK]

Can someone point me to a code sample that illustrates executing long
running tasks (asynchronous) from a web application in ASP.NET? I assume
that Web Services might come into play at some point, but I'm not sure how
to get started.

For example, I have an application that, upon a user initiating through a
button or link click, will go out and generate a bunch of files (this could
take several minutes). This will happen in batches, so I would like to have
the web application kick off the generation of all the jobs and then receive
a notification as each job finishes, at which point I could update the page.
But, obviously, I don't want the web application itself tied up during all
this processing. What is the best way to architect this in .NET? Hopefully
someone has done this before and can point me to some starter code
samples...

Thanks,
BK

 

[Cor Ligthert]

HI BK,

As far as I know, can you not control from the serverside the actions on the
clientside using a real webapplication.

You can of course start your batches from the clientside wherefore you can
use by instance repeated instructions in a way I give you the sample bellow.
Using that you can show every time an updated page.

However in my opinion not assynchronous. Therefore the server needs access
on the client.

Keep always in mind that your client can disconnect for whatever reason.

\\\
Dim scriptString As String = "<script language=JavaScript>" & _
"setclock(); function setclock(){document.images.Image1.src = " & _

"'http://mysite/WebForm1.aspx';setTimeout('setclock()',1000)}</script>"
Page.RegisterStartupScript("setclock", scriptString)
///

An intresting page when you are thinking to do something beside the normal
solutions

http://msdn.microsoft.com/security/default.aspx?pull=/library/en-us/dnwxp/html/xpsp2websites.asp

I hope this helps anyway?

Cor

 

[BK]

Cor,

Thanks. No, I am not looking to do things client side. I want the web
application to kick off these tasks server-side. As you mention, the client
could disconnect at any time, and in that case I would like the tasks to
still complete on the server. There would be no notification of completion
in that case, obviously, but perhaps next time they connected I could
indicate the status of their jobs or even send email. That is out of scope
for what I'm asking though.

I just am trying to figure out how, in .NET, I would execute potentially
long-running tasks from a web application and still allow the user to do
other things while they finish.

 

[Cor Ligthert]

Hi BK,

This would in my opinion not a problem.

Thanks. No, I am not looking to do things client side. I want the web
application to kick off these tasks server-side. As you mention, the client
could disconnect at any time, and in that case I would like the tasks to
still complete on the server. There would be no notification of completion
in that case, obviously, but perhaps next time they connected I could
indicate the status of their jobs or even send email. That is out of scope
for what I'm asking though.

You can keep track of where you are in a database.

However when a user is busy with a webpage he can forever do otherthings.
The point is that he has to react before the session ends. (And you catch it
in a way that he comes back at the proper point.)

Just my thoughts,

Cor

 

[Ken Cox [Microsoft MVP]]

I wonder if this is the kind of thing you would use?

http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2

 

[BK]

No, this link to a control is providing a way of shelling out. That's not at
all what I'm talking about.

I have complete control over all areas of execution. This is an architecture
question. What I'm asking is: from a web application, how would one call
another piece of managed code that could potentially take a long time. A
Windows Service? Web Services? A combination of both? I'm talking about
multiple asynchronous calls from a web application to do work. In this case,
it is generating different sets of files on the server.

I know there are multiple ways of doing it, I was just looking for an
example. Web Application initiates something on the server that might take a
long time, returns to the user so they can do other things, gets the
callback when its finished.

 

[Alvin Bruney [MVP]]

alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}

 

[BK]

Yes, this is getting closer to the answer to my question. How does this
compare to using anychronous web service calls?

alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}


--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

I wonder if this is the kind of thing you would use?

http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2

 

[Alvin Bruney [MVP]]

well, really an async call gets executed on a secondary thread which is
equivalent to spawning a thread from the thread pool or firing off your own
thread or invoking a webservice. theoretically, though a webservice was
designed to facilitate b2b exchange of data. there isn't anything wrong with
using a webservice, except for perhaps there may be more overhead involved
in the webservice call than the threadpool approach. webservices are also
invoked on separate threadpool threads so architecturally, the avenue to
achieve asynchronicity is the same (i'm sure that is not word but it sounds
good)

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

Yes, this is getting closer to the answer to my question. How does this
compare to using anychronous web service calls?

alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}


--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

I wonder if this is the kind of thing you would use?

http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2

Can someone point me to a code sample that illustrates executing long
running tasks (asynchronous) from a web application in ASP.NET? I assume
that Web Services might come into play at some point, but I'm not sure
how
to get started.

For example, I have an application that, upon a user initiating
through a
button or link click, will go out and generate a bunch of files (this
could
take several minutes). This will happen in batches, so I would like to
have
the web application kick off the generation of all the jobs and then
receive
a notification as each job finishes, at which point I could update the
page.
But, obviously, I don't want the web application itself tied up during
all
this processing. What is the best way to architect this in .NET?
Hopefully
someone has done this before and can point me to some starter code
samples...

Thanks,
BK

 

[DM McGowan II]

The answer may depend on the type of client. Notifications are built-in to
Asynchronous Web Service calls provided the client hasn't allowed the call
to go out of scope. The up side to this is that it's totally automatic. The
downside is that it's totally automatic. You don't have as much control.
That might be ok. Can a Java client or a Python based client work using .NET
Asynchronous web architecture? Is that important to you or could it be later
on?

You could still use a Synchronous call and ThreadPool on the server but how
do you notify the client? You could use ThreadPool on the client and signal
with a .NET event.

Yes, this is getting closer to the answer to my question. How does this
compare to using anychronous web service calls?

alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}


--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

I wonder if this is the kind of thing you would use?

http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2 through

a

 

[BK]

Based on the replies, its sounding like the best thing to do is set up
asynchrounous web service calls. In the meantime, I've started to write the
web service and client and have a basic scenario working. The thing that I
haven't figured out, yet, is how to handle multiple "jobs". Currently, I
have something like the following:

private void btnStartWork_Click(...)
{
//...
result = service.BeginMyStuff(...)
// Do some other stuff
// ...
// Stop doing other stuff
result.AsyncWaitHandle.WaitOne();
string mystring = service.EndMyStuff();
}

This all works fine with one call. Now, my next question is, If I just want
to sequentially start N of these things, would I have multiple BeginXXX
statements, and then have a WaitMany() followed by all the EndXXX in another
function somewhere? Somewhere I need to end the page processing, right, and
return control to the user? But what if the server processing isn't
finished, yet (ie the Wait is still waiting)? Also, how do I avoid the page
timing out if I choose to just let the page wait for all processing to
finish (aside from setting the page timeout to some obscene value)?

Thanks again.
BK

well, really an async call gets executed on a secondary thread which is
equivalent to spawning a thread from the thread pool or firing off your own
thread or invoking a webservice. theoretically, though a webservice was
designed to facilitate b2b exchange of data. there isn't anything wrong with
using a webservice, except for perhaps there may be more overhead involved
in the webservice call than the threadpool approach. webservices are also
invoked on separate threadpool threads so architecturally, the avenue to
achieve asynchronicity is the same (i'm sure that is not word but it sounds
good)

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

Yes, this is getting closer to the answer to my question. How does this
compare to using anychronous web service calls?

alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}


--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok
I wonder if this is the kind of thing you would use?

http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2

Can someone point me to a code sample that illustrates executing long
running tasks (asynchronous) from a web application in ASP.NET? I assume
that Web Services might come into play at some point, but I'm not sure
how
to get started.

For example, I have an application that, upon a user initiating
through a
button or link click, will go out and generate a bunch of files (this
could
take several minutes). This will happen in batches, so I would like to
have
the web application kick off the generation of all the jobs and then
receive
a notification as each job finishes, at which point I could update the
page.
But, obviously, I don't want the web application itself tied up during
all
this processing. What is the best way to architect this in .NET?
Hopefully
someone has done this before and can point me to some starter code
samples...

Thanks,
BK

 

[Cor Ligthert]

Hi Alvin,

It is possible that my view on the webservice is wrong.

For me a webservice is a service that gives to (or saves for) the requestor
information in the form of packages and does that asynchronous for many
requestors. However, there should be a request. That is very helpful in a
windowforms application and can be (however is not always needed) helpful in
a webform application.

It is in my eyes not the equivalent from a window service application, which
collects and supplies using controlled queues and remoting information
asynchronously because there is no control over the queues.

So therefore, I am curious what gives your solution more than a normal
standard web solution, which is in my opinion always an asynchronous
application on the server side doing processing for many sessions in one
time.

(A multithreading application can in my idea be usefull by the way when
there are more processors on the serverside, however that is an other
question)

Can you tell me where my idea is wrong?

Cor

well, really an async call gets executed on a secondary thread which is
equivalent to spawning a thread from the thread pool or firing off your own
thread or invoking a webservice. theoretically, though a webservice was
designed to facilitate b2b exchange of data. there isn't anything wrong with
using a webservice, except for perhaps there may be more overhead involved
in the webservice call than the threadpool approach. webservices are also
invoked on separate threadpool threads so architecturally, the avenue to
achieve asynchronicity is the same (i'm sure that is not word but it sounds
good)

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

Yes, this is getting closer to the answer to my question. How does this
compare to using anychronous web service calls?

alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}


--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok
I wonder if this is the kind of thing you would use?

http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2

Can someone point me to a code sample that illustrates executing long
running tasks (asynchronous) from a web application in ASP.NET? I assume
that Web Services might come into play at some point, but I'm not sure
how
to get started.

For example, I have an application that, upon a user initiating
through a
button or link click, will go out and generate a bunch of files (this
could
take several minutes). This will happen in batches, so I would like to
have
the web application kick off the generation of all the jobs and then
receive
a notification as each job finishes, at which point I could update the
page.
But, obviously, I don't want the web application itself tied up during
all
this processing. What is the best way to architect this in .NET?
Hopefully
someone has done this before and can point me to some starter code
samples...

Thanks,
BK

 

[Nick Malik]

Hello BK,

Architecturally, you want to send a single message or a series of messages
to the server and have that message acknowledged. The server adds the
message to a queue and returns an id that can be used to query status.

The client gets the response right away with an id. If no id, then the
message wasn't delivered (read: reliable messaging). You can use MSMQ 3.0
over SOAP if you have Windows 2003, which does essentially the same thing.

Then you have two choices: the client can poll for status every minute or
two, or the client can host a web service where status messages can be
posted by the server (remember to send the post-back address in the initial
message). If your client is something you have tremendous control over, the
latter is more efficient, but would be really rare. The former is BY FAR a
more common approach.

To poll for status, simply set up another web service that will accept the
ID passed back when the command was accepted, and will return the status.
The server side can have multiple processes to do the work, record the
status in a common data store (SQL Server or MSDE are nice for this), and,
of course, monitor and respond to web service requests.

This is a simple EAI pattern.

If you need to orchestrate a lot of processes on the server, you are
starting to enter Biztalk territory.

--- Nick Malik
Biztalk bum
EAI Architect

 

[Alvin Bruney [MVP]]

It's not wrong. There are many approaches to the problem and the *right
answer depends on your design specifications and how much lattitude you have
to implement the solution. Typically, if you are concerned about overhead,
it would be better to stick with the threaded approach. Webservices or
remoting are valid options as well. We don't disagree. I was just pointing
out that there may be extra overhead in the webservice route.

Also, the windows service route is a good option. But the drawbacks of this
approach is that the OS makes no guarantee that your service will be
running. If this provides a critical function, you will need to write a
watch dog application to make sure the service is up and running. Services
fail for all sorts of reasons. Also, a service which isn't used continually,
burns resources and cpu cycles. This is where the webservice or the thread
approach is better.

Bear in mind that building a webservice introduces another tier into your
architecture. The webservice services as a data processing tier which is
separate from the front-end and back-end tiers.

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

Hi Alvin,

It is possible that my view on the webservice is wrong.

For me a webservice is a service that gives to (or saves for) the
requestor
information in the form of packages and does that asynchronous for many
requestors. However, there should be a request. That is very helpful in a
windowforms application and can be (however is not always needed) helpful
in
a webform application.

It is in my eyes not the equivalent from a window service application,
which
collects and supplies using controlled queues and remoting information
asynchronously because there is no control over the queues.

So therefore, I am curious what gives your solution more than a normal
standard web solution, which is in my opinion always an asynchronous
application on the server side doing processing for many sessions in one
time.

(A multithreading application can in my idea be usefull by the way when
there are more processors on the serverside, however that is an other
question)

Can you tell me where my idea is wrong?

Cor

well, really an async call gets executed on a secondary thread which is
equivalent to spawning a thread from the thread pool or firing off your own
thread or invoking a webservice. theoretically, though a webservice was
designed to facilitate b2b exchange of data. there isn't anything wrong with
using a webservice, except for perhaps there may be more overhead
involved
in the webservice call than the threadpool approach. webservices are also
invoked on separate threadpool threads so architecturally, the avenue to
achieve asynchronicity is the same (i'm sure that is not word but it sounds
good)

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok

Yes, this is getting closer to the answer to my question. How does this
compare to using anychronous web service calls?


"Alvin Bruney [MVP]" <vapor at steaming post office> wrote in message
alternatively

you can kick off the long work using a thread from the threadpool
the code would look like this
page_load
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
}

and ThreadProc is a function you would define to do your work like so

static void ThreadProc(Object stateInfo) {
//do what you want here
}


--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok
I wonder if this is the kind of thing you would use?


http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=1821&tabindex=2

Can someone point me to a code sample that illustrates executing long
running tasks (asynchronous) from a web application in ASP.NET? I
assume
that Web Services might come into play at some point, but I'm not sure
how
to get started.

For example, I have an application that, upon a user initiating
through
a
button or link click, will go out and generate a bunch of files (this
could
take several minutes). This will happen in batches, so I would like to
have
the web application kick off the generation of all the jobs and
then
receive
a notification as each job finishes, at which point I could update the
page.
But, obviously, I don't want the web application itself tied up during
all
this processing. What is the best way to architect this in .NET?
Hopefully
someone has done this before and can point me to some starter code
samples...

Thanks,
BK