Importing from a text file to a Dataset

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

I have a large(1.25M), fixed field delimited, text file which I want to read into a DataSet. So, I opened a StreamReader, read a line, assigned the values to a new DataRow, and finally added the DataRow to a DataTable in the DataSet. Seemed to be a staight forward task but boy did it progress at a snail's pace!

So I tried to speed things up by first importing the data from the text file into SQL Server. To my amazement, the text import went like lightning! How can I get lightning fast text import?

Michael
 
Hi,

See this thread
http://groups.google.com/groups?hl=...*adonet*&as_uauthors=paul%20clement&lr=&hl=en
Basically you should use OleDb provider to read from text file.
It should be faster.

--
Miha Markic - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

michael said:
I have a large(1.25M), fixed field delimited, text file which I want to
Click to expand...
read into a DataSet. So, I opened a StreamReader, read a line, assigned the
values to a new DataRow, and finally added the DataRow to a DataTable in the
DataSet. Seemed to be a staight forward task but boy did it progress at a
snail's pace!
So I tried to speed things up by first importing the data from the text
Click to expand...
file into SQL Server. To my amazement, the text import went like lightning!
How can I get lightning fast text import?
 
Hi Keith,

Keith Matzen said:
I have had similar performance problems loading a database from text
Click to expand...
files. But the text files I am usually importing contain headings and
fields that need some kind of conversion. Would it make sense to build a
clean temporary text file and then import it by this method?

It would be faster, yes. But I don't know how faster. You'll have to measure
it by yourself.

Can this method be used on a Stream instead of a file?

I don't think so because it uses OleDb...
 
Back
Top