C# or VB.NET - not exaclty the same performance

  • Thread starter Thread starter Daniel P.
  • Start date Start date
Hi Daniel,

When you search the newsgroup vb.language than you will find someone (not a
regular) who has analysed the test.

I once was working with a bicycle company and the only difference on the
bicycles was the names on the bicycles. They where tested, same results as
you see in this test you are refering to.

They tested probably one uphill and the other downhill.

Cor
 
I wonder was the Visual C++ benchmark performed for unmanaged C++ (plain
Win32 executable) or for Managed Extensions for C++ (managed application
hosted in CLR) ?
 
Daniel,

I don't believe that anyone ever made the claim that VB and C# produce
exactly the same IL. If they did, then they were misinformed, as the tools
definitely will produce different IL.
 
The author shows his lack of understanding of the Microsoft platform with
the following statement in the conclusion on the linked page:

I could also extend the range of languages or variants tested. For example,
testing Visual Basic 6 (the last of the pre-.NET versions of VB) would give
us an idea how much (if any) of a performance hit the CLR adds to VB.

To suggest that testing a benchmark of VB.NET against VB6 would produce any
valuable information at all is laughable. The architecture and models are
entirely different.

Also, the presented benchmarks were designed by this person. They are not
industry standard benchmarks. As he is clearly intelligent, but clearly
uninformed about the .NET platform, I'm having a difficult time seeing the
benchmarks as an appropriate test of comparable capabilities.

Also, as the benchmarks attempt to compare code across languages, a
notoriously difficult thing to do, I fail to see how slight changes in the
code from one language to another would NOT produce differences in the IL
code. In fact, the authors inexperience with .NET languages may be the
primary reason for the slight differences between the VB.NET and C#
performance numbers.

All in all, interesting but not altogether valuable.

--- Nick
 
Also notice the huge disclaimer at the bottom... Second, this article
covers benchmarking not comparing IL Generated code. We know nothing of how
the test was conducted nor the samples that were used in derving these
answers.

Did he run 1 test? Many tests? How many tests to proove this was
"conclusive".

Thrid, the ONLY category that VB.NET and C# really vary on is IO. Which the
reasons for speed decrease could be enormous. Anyone that has experience in
both C# and VB.NET will tell you that sometimes IO just acts funny.

I would take this more seriously if he had a lot more statistical
information rather than a stacked bar graph from excel and a table of single
sig fig numbers. Quantitative Analysis baby!
 
Hi

I copied the text Tom Gurath did send on 10 jan 2004 about this test

You can see C# downhill
VB.net uphill.
----------------------------------------------------------------------------
--------------------------------------
This benchmark tests the Math & File I/O of 9 languages/run-times.
Visual C++ (Version 7 - not managed)
Visual C#
gcc C
Visual Basic.NET
Visual J#
Java 1.3.1
Java 1.4.2
Python/Psyco
Python

In the test (code available from the site), they didn't implement the VB.NET
code fairly. It is not syntactically equivalent. For the other languages,
they used streaming file i/o. With VB.NET, they used FileOpen and PrintLine.
With the other languages, the text written to the file was a literal string.
With VB.NET, they placed the text in a String and wrote that to the file.

Here are the results they posted: (in seconds)

int
math long
math double
math
trig
I/O
TOTAL
Visual C++ 9.6 18.8 6.4 3.5 10.5 48.8
Visual C# 9.7 23.9 17.7 4.1 9.9 65.3
gcc C 9.8 28.8 9.5 14.9 10.0 73.0
Visual Basic 9.8 23.7 17.7 4.1 30.7 85.9
Visual J# 9.6 23.9 17.5 4.2 35.1 90.4
Java 1.3.1 14.5 29.6 19.0 22.1 12.3 97.6
Java 1.4.2 9.3 20.2 6.5 57.1 10.1 103.1
Python/Psyco 29.7 615.4 100.4 13.1 10.5 769.1
Python 322.4 891.9 405.7 47.1 11.9 1679.0


I modified the VB.NET code to use streaming file i/o and a literal string.
Below are the results on my machine: (in milliseconds)
* I ran each test 3 times, rebooting between each language - the averages
are in bold.

C++ Averages

int 6312 6265 6265 6280
dbl 4266 4266 4297 4276
lng 13172 12547 12563 12760
trig 2296 2297 2296 2296
IO 3329 3484 3219 3344
Total 29375 28859 28640 28958

C#

int 6359 6343 6343 6348
dbl 11656 11656 11656 11656
lng 16640 15921 15984 16181
trig 2750 2734 2734 2739
IO 2843 3781 2421 3015
Total 40248 40435 39138 39940

VB - Original

int 6422 6453 6453 6442
dbl 11984 12016 12000 12000
lng 16438 15797 15750 15995
trig 2797 2734 2766 2765
IO 28062 27734 27672 27822
Total 65703 64734 64641 65026

VB - Streaming IO

int 6453 6453 6438 6448
dbl 11984 11953 11953 11963
lng 16516 15719 15750 15995
trig 2766 2734 2734 2744
IO 2922 3328 2906 3052
Total 40641 40187 39781 40203

Java 1.4.2_2 (*Gave me errors - I couldn't test it.)

C:\>javac -g:none benchmark.java

C:\>java -server benchmark.class
Error: no `server' JVM at `C:\Program
Files\Java\j2re1.4.2_01\bin\server\jvm.dll'.

C:\>java benchmark.class
Exception in thread "main" java.lang.NoClassDefFoundError: benchmark/class

As you can see, the VB.NET I/O is about equal to the C# I/O. Both ran faster
than using C++ without the CLR. I will be e-mailing the author of the
article and sending him my source code and results. I was hoping to contact
someone on the VB team at Microsoft, but haven't been able to find addresses
for the right individuals. Hopefully someone there will see this post.
 
Daniel said:
MS or anyone still claims that C# and VB.NET generate the exact same IL
code?
Click to expand...

Is it me, or is VB.NET never mentioned in the benchmark? There's VB, but
not VB.NET
 
Daniel P. said:
MS or anyone still claims that C# and VB.NET generate the exact same IL
code?

http://www.osnews.com/story.php?news_id=5602&page=3
Click to expand...

Of course not! Not exact, anyway. But for the most part they do. There
are a few differences, but none as major as those benchmarks would have you
believe. (I don't believe that MS ever said exact same IL anyway---they
said both compile to the same IL language----it's the exact same language,
not the exact same code).

Anyway, I wouldn't call this guy an expert at benchmarking.

He didn't even know how to compile a VisualC++ project without the managed
extensions, so he ended up using gcc. And he uses C's functions, not C++.

But that's not related to the issue at hand.

Look at his source code. He's not doing apples-to-apples comparison of the
I/O between VB and C#. In C# he's using the .NET Framework's StreamWriter
and StreamReader. In VB.NET he's using the OpenFile which replace BASIC's
old OPEN file FOR mode AS #num. These are wrappers for the .NET I/O
classes, so of course they're going to be slower. If the code had been
written in VB to use StreamWriter and StreamReader, the performance would be
much more similar to that of C#.

Cor was right...he's testing VB going uphill and C# going downhill.

Not to mention that some of his tests do exception handling while others
don't. This is not a fair test, and testing Math or I/O alone does not even
begin to reflect how REAL applications perform.

And if you read the comments, he claims that he used VB's built-in I/O
functions because the help told him how to use those and not the .NET stream
versions.

Yet he manages to use the .NET versions in C#. Amazing. He knows the
languages share class libraries yet he fails to use the same class libraries
in the different .NET languages. I also seriously doubt that the MS
documentation advocates that a VB programmer use OpenFile instead of a
Stream, but I could be wrong. In fact I try to avoid all of the VB-specific
functions when coding in VB.NET in favor of the .NET Libraries. The rest
really seem to be there for compatibility and to ease the transition to
..NET.

I will admit that there are *some* performance differences between VB and
C#, but no major ones (unless Option Strict is off or the program is using
On Error Resume Next or On Error Goto label). There are a few minor details
related to how they optimize IL, but they should only result in minor
performance differences.


Read the comments on the article and you'll find lots of people who agree
with my points.

--Matthew W. Jackson
 
Hi Nicholas,
If they did, then they were misinformed, as the tools
definitely will produce different IL.
- Nicholas Paldino [.NET/C# MVP]
Click to expand...

Do you have some more information about this for us?

Some samples, official Microsoft links (not to child sites please).

I am waiting full intrest

Cor
 
Cor,

You don't need samples. You can just take two simple programs, one in
VB and one in C#, and then compile them. Then, use ILDASM to view the IL.
You will see differences in it.


--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

Cor said:
Hi Nicholas,
If they did, then they were misinformed, as the tools
definitely will produce different IL.
- Nicholas Paldino [.NET/C# MVP]
Click to expand...

Do you have some more information about this for us?

Some samples, official Microsoft links (not to child sites please).

I am waiting full intrest

Cor
Click to expand...
 
Hi Nicholas,
You don't need samples. You can just take two simple programs, one in
VB and one in C#, and then compile them. Then, use ILDASM to view the IL.
You will see differences in it.
Click to expand...

That is as you said it in the syntax not in the context

But forget it we always makes sometimes mistakes

:-)

Cor
 
Cor,

The following C# code:

using System;
namespace ConsoleApplication22
{
class Class1
{
[STAThread]
public static void Main(string[] args)
{
Console.WriteLine("Hello there");
}
}
}

Compiles to the following IL:

..method public hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.custom instance void [mscorlib]System.STAThreadAttribute::.ctor() = ( 01
00 00 00 )
// Code size 11 (0xb)
.maxstack 1
IL_0000: ldstr "Hello there"
IL_0005: call void [mscorlib]System.Console::WriteLine(string)
IL_000a: ret
} // end of method Class1::Main

The following VB code:

Public Class Class1
Public Shared Sub Main(ByVal args As String())
Console.WriteLine("Hello there")
End Sub
End Class

Compiles to the following IL:

..method public static void Main(string[] args) cil managed
{
.entrypoint
.custom instance void [mscorlib]System.STAThreadAttribute::.ctor() = ( 01
00 00 00 )
// Code size 11 (0xb)
.maxstack 8
IL_0000: ldstr "Hello there"
IL_0005: call void [mscorlib]System.Console::WriteLine(string)
IL_000a: ret
} // end of method Class1::Main

Notice the differences. Specifically:

- The difference in the declaration for Main
- The difference in the maxstack.

And that is just one line of code.
 
Cor said:
That is as you said it in the syntax not in the context
Click to expand...

Not sure what you mean here, but there is a *definite* difference
between the IL generated, and that can have a definite difference in
performance.

For instance, in some cases the VB.NET compiler will generate fewer IL
instructions, but those may take up more IL *bytes* than the C#
equivalent. Even if the JITter compiles them to the same native code,
it determines which methods to inline based on their IL size in bytes,
so the difference in IL could make the difference between a method
being inlined and not.

If by a quirk of fate that method is one which your app calls all the
time, really heavily, that could make a very significant difference in
performance.
 
Hi CJ,
What do you mean by this?
Click to expand...

Now it is for me an unanswered question. I assume that Jon and Nicholas see
this too.

The context (I hope this is an English word also) for me from the first
answer in the thread from Nicholas does give me the idea that there is a
significant difference between a compiled program written with C# and
VB.net.

I thought always that a program with no late binding, full managed code,
only using a datagrid, a dataset and a dataadapter would give almost the
same code in both languages. Now I understand from Jon and Nicholas that it
is not true and that there are major differences.

The way C# and VB.net are threaten in both newsgroups has always been saying
the opposite.

It confuses me little bit.

:-)

Cor
 
Nicholas,

Ok, so you showed a difference with using a single line code. I can see now
that there are slight differences in the IL, but nothing I'm going to lose
sleep over.

Second, Cor wasn't focusing on the identical attributes between IL, but the
fact that the VB Program used VB6 methods whereas C# used the Stream Reader,
thats a valid point right tthere (in my prevous point I had never seen the
source code to any of the programs, still haven't).

You show two what are seemingly minor differences. I know Jon Skeet points
out that this could change performance over time, but my question is, how
much? And then, with that amount, is that within tolerable limits?

Now, I'll admit, I am very engrossed with the way .NET works and how it all
compiles down to the IDL. But then, I think why I program .NET, because I'm
trying to get the job done (usually for business intelligence) in the
shortest amount of time possible. If I was that concerned over the
difference of 8 clock cycles in an operation in my program, I would have
written it in assembly.

I just don't think showing a stack initializer with a very small difference
and an extra attribute in the constructor really proves anything. But, I
didn't nor do I program compilers... Hell, this could be like using a long
to store a short value.

None of this was meant to be offensive.

-CJ

Nicholas Paldino said:
Cor,

The following C# code:

using System;
namespace ConsoleApplication22
{
class Class1
{
[STAThread]
public static void Main(string[] args)
{
Console.WriteLine("Hello there");
}
}
}

Compiles to the following IL:

.method public hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.custom instance void [mscorlib]System.STAThreadAttribute::.ctor() = ( 01
00 00 00 )
// Code size 11 (0xb)
.maxstack 1
IL_0000: ldstr "Hello there"
IL_0005: call void [mscorlib]System.Console::WriteLine(string)
IL_000a: ret
} // end of method Class1::Main

The following VB code:

Public Class Class1
Public Shared Sub Main(ByVal args As String())
Console.WriteLine("Hello there")
End Sub
End Class

Compiles to the following IL:

.method public static void Main(string[] args) cil managed
{
.entrypoint
.custom instance void [mscorlib]System.STAThreadAttribute::.ctor() = ( 01
00 00 00 )
// Code size 11 (0xb)
.maxstack 8
IL_0000: ldstr "Hello there"
IL_0005: call void [mscorlib]System.Console::WriteLine(string)
IL_000a: ret
} // end of method Class1::Main

Notice the differences. Specifically:

- The difference in the declaration for Main
- The difference in the maxstack.

And that is just one line of code.

--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

Cor said:
Hi Nicholas,
Click to expand...
one
in

That is as you said it in the syntax not in the context


But forget it we always makes sometimes mistakes

:-)

Cor
Click to expand...
Click to expand...
 
I've just read all of the current messages in this thread, and some
interesting points have been made. For the record, I was curious
about this myself and did my own experiment. I have a program
that uses artificial intelligence, written in C#, and though it works,
it is not the most efficient coding that I've run across. I decided
that this would be an excellent candidate for my experiment, and
translated the entire app to VB.NET, to the best of MY ability, and
did my best to keep the structure of the code intact. After both
versions (C# and VB.NET) are compiled to .exe, they both still do
the exact same thing, however the VB.NET version took a severe
perfomance hit. I was astounded at just how much. If anyone is
interested in seeing this, send me a private email and I'll zip it all
up and send it to you. This app is NOT developed in VS, so it
compiles easily from the command line. I can be reached at:

gwmorris1
hotpop
dot
com
 
Interesting...

I just wanted to make sure that you're using "Option Strict" at the top of
your VB source files.

And you ARE using the same classes/libraries in both the C# and VB.NET
versions? In fact, stick away from the VB namespaces and functions if you
want apples-to-apples comparisions.

And you're using structured exception handling (or nothing at all) in both,
not On Error statements in VB?

That's all I can think of right now. It's possible that in your case the VB
compiler is a bit worse.

--Matthew W. Jackson
 
Back
Top