Fastest way to insert 6MB of strings into 64MB List of strings?

  • Thread starter Thread starter DR
  • Start date Start date
D

DR

It seems to take about 30 seconds to insert only 6MB of strings into
random locations in an array of about 64MB of strings. This is way too
slow, I need to be able to insert objects into a large list much much
faster. Any tips?


static void Main(string[] args)
{
Random rnd = new Random();
List<string> db = new List<string>(6710886);
for (int i = 0; i < 6710886; i++)
{
db.Add(rnd.Next().ToString() + "," +
rnd.Next().ToString());
}
int i23zzrez = 23 + 23;

//////////////////////////////////////////////////////////////////////////////
int cores_per_machine = 8;
int machines = 20;
for (int i = 0; i < 629145 / (cores_per_machine * machines);
i++)
{
int loc = rnd.Next(0, db.Count);
db.Insert(loc, rnd.Next().ToString() + "," +
rnd.Next().ToString());
}
////////////////////////////30 seconds 1
core//////////////////////////////////////////
int i23zzz = 23 + 23;
}
 
DR said:
It seems to take about 30 seconds to insert only 6MB of strings into
random locations in an array of about 64MB of strings. This is way too
slow, I need to be able to insert objects into a large list much much
faster. Any tips?


static void Main(string[] args)
{
Random rnd = new Random();
List<string> db = new List<string>(6710886);
for (int i = 0; i < 6710886; i++)
{
db.Add(rnd.Next().ToString() + "," +
rnd.Next().ToString());
}
int i23zzrez = 23 + 23;

//////////////////////////////////////////////////////////////////////////////
int cores_per_machine = 8;
int machines = 20;
for (int i = 0; i < 629145 / (cores_per_machine * machines);
i++)
{
int loc = rnd.Next(0, db.Count);
db.Insert(loc, rnd.Next().ToString() + "," +
rnd.Next().ToString());
}
////////////////////////////30 seconds 1
core//////////////////////////////////////////
int i23zzz = 23 + 23;
}
.
Click to expand...

The size of the strings does not play in the computation speed. In other
words, the timing will be the same if the strings are just "X".

Based only on what you have here, it's hard to recommend what data structure
might be better for your purposes. Maybe a linkedlist or binarytree?

Mike
 
Back
Top