File copy from AS/400 IFS share by Win2k box starts multiple jobs that dont end

  • Thread starter Thread starter Patrick McClellen
  • Start date Start date
P

Patrick McClellen

We have a Win2k box accessing the IFS to 1) list files 2) copy files
and 3) delete files. When I do the same sequence from Windows
Explorer on the Win2k box, I have no problems.

However, when I do it programmatically (C# Windows Service), it seems
that multiple jobs are being spawned on the AS/400 that never end.
The service polls every x seconds (in this case 5) for new files. It
does that by using the GetFiles method on the DirectoryInfo object.
If files exist, it then copies and deletes them.

The file manipulation logic is in a class that I dereference each
loop. I also call GC.GetTotalMemory(true) to garbage collect any
objects on each loop (not so worried about performance within a
polling loop). I also posted this on the AS/400 newsgroup.

Anyone know why 1) the windows service is causing multiple jobs and 2)
why they are not ending? Or how to force the jobs to end
(programmatically) from a C# Windows Service?

I am using a timer that gets initialized in the service constructor.
It calls a function TransferFiles

The programmatic access is very simple (error checking and such
removed):

private void TransferFiles()
{
GC.GetTotalMemory(true);
this.oFileClass = new FileClass;
this.oFileClass.ProcessSomeFiles();
}

In the FileClass class...

public void ProcessFiles()
{
// Directory object and UNC path
DirectoryInfo sourceDir = new
DirectoryInfo("SomeUNCPathToTheIFShere");

// Create array representing files in the current directory matching
filespec.
FileInfo[] sourceFiles = sourceDir.GetFiles(InboundFileSpec);

// Process each file: copy, delete
foreach (FileInfo fiTemp in sourceFiles)
doFileManip(fiTemp.Name);

private void doFileManip(string InboundFileName)
{
// Do some processing to get inbound and outbound paths
File.Copy(InboundPathAndFile, OutboundPathAndFile);
File.Delete(InboundPathAndFile);
}

I also do some impersonation to set the token correctly... done at the
beginning of TransferFiles and only done once for the life of the
service. That class implements IDisposable and I call Dispose when
the service is stopped.
 
Hi Patrick, I have contacted someone from our C# team to respond to your question as it is somewhat out of the scope of what I have worked with before (I have worked with some
Database issues with AS400 but know nothing about the File system or accessing that<G>)

Thank you for your patience..

Scot Rose, MCSD
Microsoft Visual Basic Developer Support
Email : (e-mail address removed) <Remove word online. from address>

This posting is provided “AS IS”, with no warranties, and confers no rights.

Get Secure!
http://www.microsoft.com/security
http://www.microsoft.com/protect


--------------------
From: (e-mail address removed) (Patrick McClellen)
Newsgroups: microsoft.public.dotnet.languages.csharp
Subject: File copy from AS/400 IFS share by Win2k box starts multiple jobs that dont end
Date: 7 Nov 2003 12:04:04 -0800
Organization: http://groups.google.com
Lines: 59
Message-ID: <[email protected]>
NNTP-Posting-Host: 12.29.89.6
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Trace: posting.google.com 1068235445 9464 127.0.0.1 (7 Nov 2003 20:04:05 GMT)
X-Complaints-To: (e-mail address removed)
NNTP-Posting-Date: Fri, 7 Nov 2003 20:04:05 +0000 (UTC)
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!newsfeed00.sul.t-online.de!t-online.de!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!postnews1.google.com!not-for- mail
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.languages.csharp:197502
X-Tomcat-NG: microsoft.public.dotnet.languages.csharp

We have a Win2k box accessing the IFS to 1) list files 2) copy files
and 3) delete files. When I do the same sequence from Windows
Explorer on the Win2k box, I have no problems.

However, when I do it programmatically (C# Windows Service), it seems
that multiple jobs are being spawned on the AS/400 that never end.
The service polls every x seconds (in this case 5) for new files. It
does that by using the GetFiles method on the DirectoryInfo object.
If files exist, it then copies and deletes them.

The file manipulation logic is in a class that I dereference each
loop. I also call GC.GetTotalMemory(true) to garbage collect any
objects on each loop (not so worried about performance within a
polling loop). I also posted this on the AS/400 newsgroup.

Anyone know why 1) the windows service is causing multiple jobs and 2)
why they are not ending? Or how to force the jobs to end
(programmatically) from a C# Windows Service?

I am using a timer that gets initialized in the service constructor.
It calls a function TransferFiles

The programmatic access is very simple (error checking and such
removed):

private void TransferFiles()
{
GC.GetTotalMemory(true);
this.oFileClass = new FileClass;
this.oFileClass.ProcessSomeFiles();
}

In the FileClass class...

public void ProcessFiles()
{
// Directory object and UNC path
DirectoryInfo sourceDir = new
DirectoryInfo("SomeUNCPathToTheIFShere");

// Create array representing files in the current directory matching
filespec.
FileInfo[] sourceFiles = sourceDir.GetFiles(InboundFileSpec);

// Process each file: copy, delete
foreach (FileInfo fiTemp in sourceFiles)
doFileManip(fiTemp.Name);

private void doFileManip(string InboundFileName)
{
// Do some processing to get inbound and outbound paths
File.Copy(InboundPathAndFile, OutboundPathAndFile);
File.Delete(InboundPathAndFile);
}

I also do some impersonation to set the token correctly... done at the
beginning of TransferFiles and only done once for the life of the
service. That class implements IDisposable and I call Dispose when
the service is stopped.
 
Actually, I have a little more information... I seemed to have
pinpointed that the issue is related to impersonation.

I changed the service to not use impersonation and authenticate against
the domain. Instead, I installed the service and have it running under
the domain user account that has access to the AS/400 IFS. That seems
to prevent the multiple "neverending" jobs from being started.

I would still like to understand why an impersonated user causes the
problem.
 
Hi Patrrick,

Thank you for your question about why your service could cause multiple
processes to run on the AS/400. I'd like to make sure I understand the
exact details of when you see this problem. From the code you posted out
and the details provided, I don't see anything where your code could spawn
a process on the AS/400.

So, here is my understanding of the problem:

Your service runs on W2K in the local system account and impersonates
somebody (a domain account?). It then executes the code you posted--use a
UNC name to connect to a file share on an AS/400, get the list of files in
a directory there, for each file, copy it and then delete it. You do this
once every 5 seconds. When you do, you are seeing processes being started
on the AS/400. Futhermore, you can't kill these processes. But, if you
put your service in a domain account and make it not impersonate somebody,
and don't change any other code, the problem does not occur.


If this is not what you are seeing, please tell me where my description is
wrong. Please tell me exactly what accounts you are using on the W2K
machine for your service and for impersonation. Also please tell me
whether the AS/400 is on the domain and recognizes the account that is
being impersonated. Please confirm where the extra processes are getting
created. Finally, does this occur when connecting between two W2K machines?


If this is what you are seeing, then there is something on the AS/400 box
that is launching the processes in response to the impersonation context
(username and password) that the Windows machine is sending it. There are
two reasons this must be true:

1) Windows network file I/O is done through a protocol called Server
Message Block (SMB). This protocol does not have a way to launch
processes. It merely allows a user to connect and then issue file I/O
commands like open, read, write, close, etc. The Windows client redirector
sends these to the server's SMB service. The SMB service does the actual
I/O operation, and then sends back the results. I'm not sure what provides
the SMB service on AS/400.

2) Applications running on a W2K machine can't directly spawn applications
on a remote machine. If an application wanted to spawn a program on a
remote machine, it would have to talk to a piece of software on the remote
machine and have the remote machine spawn the application. Likewise for
killing applications on a remote machine.


Your best bet is to debug the AS/400 side to see what it is doing when it
receives the file I/O requests (i.e. what processes it starts and why).
You may also want to see what SMB traffic the W2K machine is sending to the
AS/400 machine. You can use Network Monitor (NetMon) to do this. I'd
recommed comparing two different scenarios:

1) When your service impersonates a user and the AS/400 creates the
processes
2) When your service runs in a domain account and the AS/400 doesn't create
the processes.

The main difference should be what username and password was used to make
the connection the the AS/400. If you are seeing different SMB traffic
once you get into the code that fires after every timer
(DirectoryInfo.GetFiles, File.Copy, File.Delete, etc) then you really may
be in different places in your code. Those functions will generate the
same SMBs in both cases.

Sincerely,

Dan Ruder
Microsoft Developer Support

This posting is provided "AS IS" with no warranties, and confers no rights.
 
Thanks for your response, Dan.

You are almost correct about the situation. Let me be a little more
explicative.

When the problem occurs, the service does run under the local system
account and impersonates a domain account that has access on the
Netserver IFS on the AS/400. The Integrated File System (IFS) that the
AS/400 product NetServer enables is so that Windows machines can access
the AS/400 directly as a file server (via SMB) rather than through FTP,
HTTP, etc. Now, NetServer starts pre-start jobs QZLSFILE whenever a
client accesses the IFS. It is suppose to create one QZLSFILE job per
user name/workstation name accessing the AS/400. The same
user/workstation should use the same prestarted job.

In order to get the service working correctly, I removed the
impersonation code (actually, I just removed the calls to the class that
handles the impersonation), and then run the service as an account that
has access to the AS/400. The AS/400 is not on the domain, so it
doesn't really matter if it is a domain account or not, except that the
user name on the AS/400 is set up to be [domain]/user. When I
impersonate that user, the only way to pass that user name (that I know
of) using a primary token is for that user to actually exist on
[domain].

The jobs that are created on the AS/400 can be killed. They just don't
end unless the service is stopped or they are manually removed.

Some more information:
1) Of course the Windows machine doesn't "create" the jobs on the
AS/400. The AS/400 creates jobs to handle the file system requests from
the Windows client. The question is what about the Windows client is
causing the AS/400 to create these multiple jobs ONLY in the scenario
where I'm trying to impersonate a user through code (and using a primary
token)? And, why does it not happen when I dir/copy/delete from the
windows explorer directly to the IFS share via a UNC or mapped path?

2) If I do a WindowsIdentity.Impersonate() and then a
WindowsImpersonationContext.Undo() each pass, I notice a couple of
things. The first is that I get a new primary token in the process (I
use a sysinternals shareware program to watch the process and its
resources) each time. Those primary tokens do not go away (actually,
that is a memory leak). If I start the impersonation when the service
starts, and then issue the Undo() upon stop or error, I only get a
single token. In the first case, I get many times more jobs spawned on
the AS/400 than in the second case. That indicates to me that the
impersonation is somehow being seen as a different user/workstation on
the AS/400. It does seem to pinpoint the issue to authentication from
the Windows client to NetServer.

3) NetServer on the AS/400 does not provide full Windows support. For
example: there is no file save notification event when a file is closed
on the IFS (thus, you cannot say, pick up a file from the IFS in Biztalk
using a file receive function). I don't know what else is not
supported.

Unfortunately, I don't have access to the AS/400 side of the equation
(client production system). I am also not an AS/400 expert, so it would
do me little good if I did.

Do you have any ideas why the impersonation could be seen by the AS/400
as a different user/workstation context each time (assuming the same
single primary token is used for authentication)?

Thanks for the help.

Some NetServer info can be found at:
http://www-1.ibm.com/servers/eserver/iseries/netserver/common/v5r1smb/pd
f/netserver_basic_s02.pdf
 
Hi Patrick,

To make a connection to a remote server, Windows passes the user's
credentials (username, password), the type of access, and the location (UNC
name). The SMB server then has to verify the credentials, determine the
access permissions on the resource, and allow the connection or not.

Windows does not store the plaintext passwords of domain users--ever. It
hashes the passwords and stores the hashes. When a domain user connects to
a domain resource, the resource sends the username and password hash to the
DC and the DC authenticates the user. However, when a domain user connects
to a non-domain resource, the resource will not recognize the password
hash. Therefore, the user must enter a password when connecting to the
resource.

The AS/400 SMB server, IFS, performs these tasks and then starts a QZLSFILE
job for the user. Since your AS/400 isn't part of the domain, it's user
account database must have the same usernames as the domain users who will
be accessing the share. Thus, when we pass the credentials, the AS/400
will recognize the usernames and probably ask about the passwords (unless
the passwords happen to be the hashed passwords sent by Windows) before
allowing access to the share.

Tokens are valid only on the machine in which they're built. A primary
token is built when a user account is logged on, either interactively, or
service logon when a service runs under a user account, or through a call
to LogonUser with the account credentials. We never send a token across
the network. So the AS/400 never sees a token.

Windows never creates a primary access token for impersonation. It only
creates impersonation tokens--these are assigned to the thread that is
doing the impersonation. In your code, the impersonation token would be
assigned to the thread that calls WindowsIdentity.Impersonate. (By the
way, since this is wrapped in a .NET class, it is not leaked; all .NET
objects are subject to garbage collection at some point during runtime.
The GC will choose when to do it.)

When a thread impersonates a user and then connects to a network share, it
will do so as that impersonated user. That is, it will get the user's name
from the token and the password from LSASS then pass these as the
credentials to the AS/400. Other than that, there aren't any differences
in what Windows sends to the AS/400 between impersonating or not
impersonating.

Note that you would only expect to see the IFS start a QZLSFILE job when a
DirectoryInfo object is created. This is when the connection occurs. You
wouldn't see it during a copy or delete because the connection to the
server has already been created.

Perhaps the AS/400 isn't recognizing the credentials at logon and is
therefore allowing anonymous access?

There are a couple ways your customer can diagnose this problem:

1) If IFS has any connection auditing or logging, turn this on to see who
is connecting to the IFS share.

2) Run NetMon between the two machines to see what SMB connection packets
are being sent. Specifically, see what credentials are supplied in the
Connect SMB.

Except for these, you probably won't be able to tell what's happening on
the AS/400 or why it thinks each connection while impersonating is a
different user.

Sincerely,

Dan Ruder
Microsoft Developer Support

This posting is provided "AS IS" with no warranties, and confers no rights.
 
Thanks for all the help Dan. Time permitting, maybe the client will
have time to troubleshoot the AS/400 side. We have a workable solution
now, so it's probably as far as I will take it.

I appreciate the instruction on primary and impersonation tokens.
 
Back
Top