Recover from NullReferenceException in OracleClient.DBObjectPool...

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Hello ALL!

We are getting killed by a customer because of this...

..NET 1.1 SP2
Server 2003 SP1
MS Provider for Oracle
Oracle 9.1.0.4

Program runs fine for days at a time and then intermittently we receive
this exception:

System.NullReferenceException: Object reference not set to an instance
of an object. at Data.OracleClient.DBObjectPool.GetObject(Object
owningObject, Boolean& isInTransaction) at
System.Data.OracleClient.OracleConnectionPoolManager.GetPooledConnection(String
encryptedConnectionString, OracleConnectionString options,
OracleConnection owningObject, Boolean& isInTransaction) ...
System.Data.Common.DbDataAdapter.Fill(DataTable dataTable)

At this point we see one of two things happen.
1. We are able to get connections for a short period and during this
period 'Mem Usage' of our app starts sky rocketing up... starts around
40MB'. Eventually we start getting timeouts from the pool manager. At
this point they need to restart the app, 'Mem Usage' is then usually
well over 100MB

2. We immediately start getting timeouts from the pool manager. At this
point they need to restart the app.

Note: From Oracle side we never show more then 3-4 connections.

Any thoughts here? I have seen many people with same situation...

Our current plan is to dynamically modify the connection string to
create a new connection pool once we trap this exception... This code
is in test preparing to move into production... I will post back if
this works...

All connections are being disposed in a try catch finally block! The
system will run for days at a time not leaking memory or connections!
Handling 1000's of transactions a day. Intermittently we then get this
exception.

Thanks in advance!
 
Clarification...

These exceptions aren't totaly random. This customers Oracle Server is
always on the brink of Overload and these exceptions always occur first
thing in the morning or right after lunch when it seems many people are
logging back onto the business system.

They are currently using "max_shared_servers" with their Oracle Server.


Our new code should be moved to production today so I will post back if
this "dynamic connection" string works. I have seem many people post
this suggestion but have not found anyone saying that it worked. So, I
will let you know!

Thanks!
 
Update!

Once placing code into production environment and encountering the Null
Reference Exception, this time we slightly modify the connect string to
create a new pool and this worked perfectly!

However, now the application starts leaking memory like crazy!! We
started at 48MB and 3 hours after getting exception we where at 210MB!!
Before restarting the app I attached and created a mini-dump and have
started looking at this in WinDbg. One think I noticed right off the
bat was that there where 186 Dead Threads!! Our application will use
~18 threads while everything is running fine. These threads are marked
for Completion Port Pool and Thread Pool worker threads.

We have now dumped the OracleClient code and modified for
OleDBClient... this will go into prod tomorrow so I will update if I
have any good news...

We are getting killed by the customer over this... Any thoughts would
be appreciated...
 
Update, Update!

Ok, with the the OleDB provider we are actually able to recover from
the exception without leaking memory! However, sometimes the connection
will block indefinitely on the Open! From what I understand this is
because the MSDAORA uses the 7.x OCI interface which does not support a
timeout on the connection.

So, we have switched all code to the Oracle ODP.NET which uses 9.x OCI
which does support timeout. This code is in testing now... I will let
you know...

Any thoughts would be appreciated...

Thanks!
 
Back
Top