Manual binary serialization of a dataset is slow for me - why?

  • Thread starter Thread starter hs
  • Start date Start date
H

hs

Hi
I am serializing a dataset using a binary formatter as follows:

IFormatter formater = new BinaryFormatter();
formatter.Serialize(stream, ds); // ds=DataSet, stream=MemoryStream
....
DataSet ds2 = (DataSet)formatter2.Deserialize(stream2);

For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
seconds to deserialize.

After reading some previous archive messages, I decided to try some manual
binary serialization using a custom DataSet (myDataSet). I have slightly
improved on my deserialization time (now takes 0.9 seconds) but my
serialzation has gone mad (takes 10 seconds). All this time is being spent
after the myDataSet.GetObjectData method. What is hapenning after this
method?
any advice on how to improve this time would be appreciated.
thanks

Code
------
IFormatter formater = new BinaryFormatter();
formatter.Serialize(stream, myds); // myds=myDataSet, stream=MemoryStream
....
myDataSet myds2 = (myDataSet)formatter2.Deserialize(stream2);

myDataSet class
----------------------
[Serializable]
public class myDataSet : DataSet, ISerializable
{
public myDataSet() : base()
{}
public myDataSet(string name) : base(name)
{}
protected myDataSet(SerializationInfo sinfo, StreamingContext context)
{
// Extract from the serialization info
ArrayList tables = new ArrayList();
tables = (ArrayList) sinfo.GetValue("Tables", typeof(ArrayList));

// foreach table name in tables create a DataTable and add to table
collection
for(int i=0;i<tables.Count; i++)
{
string tableName = (string)tables;
DataTable dt = new DataTable(tableName);

ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
colNames = (ArrayList) sinfo.GetValue("ColNames_"+tableName,
typeof(ArrayList));
colTypes = (ArrayList) sinfo.GetValue("ColTypes_"+tableName,
typeof(ArrayList));
dataRows = (ArrayList) sinfo.GetValue("DataRows_"+tableName,
typeof(ArrayList));

// Add columns
for(int iCol=0; iCol<colNames.Count; iCol++)
{
DataColumn col = new DataColumn(colNames[iCol].ToString(),
Type.GetType(colTypes[iCol].ToString() ));
dt.Columns.Add(col);
}

// Add rows
for(int iRows=0; iRows<dataRows.Count; iRows++)
{
DataRow row = dt.NewRow();
row.ItemArray = (object[]) dataRows[iRows];
dt.Rows.Add(row);
}

this.AcceptChanges();

// Add table to Tables collection
this.Tables.Add(dt);
}

void ISerializable.GetObjectData(SerializationInfo sinfo,
StreamingContext context)
{
// Add arrays
ArrayList tables = new ArrayList();

foreach(DataTable dt in this.Tables)
{
ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
string tableName = dt.TableName ;

// Insert the table name into worker array
tables.Add(tableName);

// Insert column information (names and types) into worker arrays
for(int iCols=0; iCols<dt.Columns.Count; iCols++)
{
DataColumn Col = dt.Columns[iCols];
colNames.Add(Col.ColumnName);
colTypes.Add(Col.DataType.FullName);
}

for(int iRows=0; iRows<dt.Rows.Count; iRows++)
{
dataRows.Add(dt.Rows[iRows].ItemArray);
}
// Add arrays to the serialization info structure
sinfo.AddValue("ColNames_"+tableName, colNames);
sinfo.AddValue("ColTypes_"+tableName, colTypes);
sinfo.AddValue("DataRows_"+tableName, dataRows);
}
sinfo.AddValue("Tables", tables);
}
}
 
Hi
I add about more than 10000 records in database and use your code to
serialize it.
I met the same problem as you.
My serialization time is 19 seconds and my deserialization time is 3
seconds.

I will do some research and give you some suggestion.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

--------------------
| Reply-To: "hs" <[email protected]>
| From: "hs" <[email protected]>
| Subject: Manual binary serialization of a dataset is slow for me - why?
| Date: Wed, 13 Aug 2003 16:28:43 +0100
| Lines: 119
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
| Message-ID: <OvuRW#[email protected]>
| Newsgroups: microsoft.public.dotnet.languages.csharp
| NNTP-Posting-Host: mailgate.synergy-logistics.co.uk 62.49.130.162
| Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP10.phx.gbl
| Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.languages.csharp:176176
| X-Tomcat-NG: microsoft.public.dotnet.languages.csharp
|
| Hi
| I am serializing a dataset using a binary formatter as follows:
|
| IFormatter formater = new BinaryFormatter();
| formatter.Serialize(stream, ds); // ds=DataSet, stream=MemoryStream
| ...
| DataSet ds2 = (DataSet)formatter2.Deserialize(stream2);
|
| For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
| seconds to deserialize.
|
| After reading some previous archive messages, I decided to try some manual
| binary serialization using a custom DataSet (myDataSet). I have slightly
| improved on my deserialization time (now takes 0.9 seconds) but my
| serialzation has gone mad (takes 10 seconds). All this time is being spent
| after the myDataSet.GetObjectData method. What is hapenning after this
| method?
| any advice on how to improve this time would be appreciated.
| thanks
|
| Code
| ------
| IFormatter formater = new BinaryFormatter();
| formatter.Serialize(stream, myds); // myds=myDataSet, stream=MemoryStream
| ...
| myDataSet myds2 = (myDataSet)formatter2.Deserialize(stream2);
|
| myDataSet class
| ----------------------
| [Serializable]
| public class myDataSet : DataSet, ISerializable
| {
| public myDataSet() : base()
| {}
| public myDataSet(string name) : base(name)
| {}
| protected myDataSet(SerializationInfo sinfo, StreamingContext context)
| {
| // Extract from the serialization info
| ArrayList tables = new ArrayList();
| tables = (ArrayList) sinfo.GetValue("Tables", typeof(ArrayList));
|
| // foreach table name in tables create a DataTable and add to table
| collection
| for(int i=0;i<tables.Count; i++)
| {
| string tableName = (string)tables;
| DataTable dt = new DataTable(tableName);
|
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| colNames = (ArrayList) sinfo.GetValue("ColNames_"+tableName,
| typeof(ArrayList));
| colTypes = (ArrayList) sinfo.GetValue("ColTypes_"+tableName,
| typeof(ArrayList));
| dataRows = (ArrayList) sinfo.GetValue("DataRows_"+tableName,
| typeof(ArrayList));
|
| // Add columns
| for(int iCol=0; iCol<colNames.Count; iCol++)
| {
| DataColumn col = new DataColumn(colNames[iCol].ToString(),
| Type.GetType(colTypes[iCol].ToString() ));
| dt.Columns.Add(col);
| }
|
| // Add rows
| for(int iRows=0; iRows<dataRows.Count; iRows++)
| {
| DataRow row = dt.NewRow();
| row.ItemArray = (object[]) dataRows[iRows];
| dt.Rows.Add(row);
| }
|
| this.AcceptChanges();
|
| // Add table to Tables collection
| this.Tables.Add(dt);
| }
|
| void ISerializable.GetObjectData(SerializationInfo sinfo,
| StreamingContext context)
| {
| // Add arrays
| ArrayList tables = new ArrayList();
|
| foreach(DataTable dt in this.Tables)
| {
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| string tableName = dt.TableName ;
|
| // Insert the table name into worker array
| tables.Add(tableName);
|
| // Insert column information (names and types) into worker arrays
| for(int iCols=0; iCols<dt.Columns.Count; iCols++)
| {
| DataColumn Col = dt.Columns[iCols];
| colNames.Add(Col.ColumnName);
| colTypes.Add(Col.DataType.FullName);
| }
|
| for(int iRows=0; iRows<dt.Rows.Count; iRows++)
| {
| dataRows.Add(dt.Rows[iRows].ItemArray);
| }
| // Add arrays to the serialization info structure
| sinfo.AddValue("ColNames_"+tableName, colNames);
| sinfo.AddValue("ColTypes_"+tableName, colTypes);
| sinfo.AddValue("DataRows_"+tableName, dataRows);
| }
| sinfo.AddValue("Tables", tables);
| }
| }
|
|
|
 
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter();
IFormatter formatter = (IFormatter)fmtobj;
FormatterTypeStyle fts = fmtobj.TypeFormat;
fmtobj.TypeFormat = fts | FormatterTypeStyle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.
 
Back
Top