Edit and Continue

[Jon Skeet [C# MVP]]

How exactly? I run my code and it gives me an error that an sql statement is
incorrect. I'm going to fix the sql statement either way?

If that SQL statement has already been used in that run, you can't tell
whether you've broken what it's already done in E&C. You'll need to
rerun the whole application - making sure you hit every point which
could potentially use that SQL statement.

Doing that manually is time-consuming. Running a set of unit tests is
quick.

Jon

 

[Jon Skeet [C# MVP]]

I agree. I think there is a possibility that it can breed complacency.
I think it can be used in moderation, but as Jon said, only for small
UI or similar situations. You don't want to rely on E&C. Its good that
its been absent for a few years, and I personally don't use it at all.

Jon, got any really good resources on unit testing in .NET ?
They're going to push Nunit on us at work and I want to be completely
prepared.
I have never before developed from a test-driven approach. I understand
the basic concepts of writing testable methods, but only the basics
mind you.
I'm sure there are lots of resources available that a Google would
find, but I want to know what YOU recommend.

Well, I use TestDriven.NET (unsurprisingly at www.testdriven.net) and
EasyMock.NET (www.easymock.net). However, the version of EasyMock.NET
I'm using is a completely rewritten one which I must get round to
committing back to CVS at some stage...

There's another mocking framework available at
http://www.ayende.com/projects/rhino-mocks.aspx

I haven't used it, but it's got better documentation etc than
EasyMock.NET, and it looks like the project is rather more lively too.
(That's my fault really, as the sole person who's actually done
anything on EasyMock.NET in the last 18 months.)

Mocking is really important (IMO) - without mocking, unit testing is
nightmarish, particularly when you have lots of collaborators. With
mocking (and IoC, preferrably), you tend to end up with a more modular
design with looser coupling.

There are a few coverage tools for .NET, but I haven't used any in the
last year, so any comments would almost certainly be way out of date.

 

[Barry Kelly]

Well, I use TestDriven.NET (unsurprisingly at www.testdriven.net) and
EasyMock.NET (www.easymock.net). However, the version of EasyMock.NET
I'm using is a completely rewritten one which I must get round to
committing back to CVS at some stage...

There's another mocking framework available at
http://www.ayende.com/projects/rhino-mocks.aspx

I haven't used it, but it's got better documentation etc than
EasyMock.NET, and it looks like the project is rather more lively too.
(That's my fault really, as the sole person who's actually done
anything on EasyMock.NET in the last 18 months.)

How do you deal with the problem that mocking requires modification to
the classes to support replaceability - e.g. descending from
MarshalByRefObject if a RealProxy technique is used, or using interfaces
/ delegates / virtual methods if Rhino Mocks is used?

How do you get rid of the modifications in release-mode code, or toggle
them safely, in order to avoid the downsides to interfaces & virtual
methods?

It seems like something that aspect-oriented programming or binary
rewriting would be useful for, but that would move what you're testing
further away from what you've actually written.

-- Barry

 

[Michael D. Ober]

E&C is really useful when you are single stepping through the code and find
a one-off error in a loop counter. Fix the error and continue.

Mike Ober.

 

[Alvin Bruney [MVP]]

I believe that situation is changing
You are fooling yourself. For the most part, unit tests sit inside an
application and are never run until code is about to hit production. That is
closer to the truth than "the situation is changing". It's not.

Sure, there's a lot of talk about using unit tests regularly but the reality
is as soon as you turn your back, developers are back to coding habit. for
better or for worse.

The code that I have reviewed all have unit tests attached to them. I'd
wager a finger that during the development cycle these unit tests hardly
broke a sweat. And that's real world experience talking.

--
________________________
Warm regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
Professional VSTO.NET - Wrox/Wiley
The O.W.C. Black Book with .NET
www.lulu.com/owc, Amazon
Blog: http://www.msmvps.com/blogs/alvin

 

[Michael C]

If that SQL statement has already been used in that run, you can't tell
whether you've broken what it's already done in E&C. You'll need to
rerun the whole application - making sure you hit every point which
could potentially use that SQL statement.

I've been fixing it in e&c in vb6 like that for 10 years and it rarely
causes a problem. I don't really see how it could if the sql string is
created on the line before it is used, eg

sql = "DELECT * FROM SomeTable"
rs.Open sql

if the rs.Open line failed then the recordset hasn't opened and going back 1
line is not going to cause any problem. Sure if the sql statement is created
in 1 place and passed into an object and that object goes through all sort
of initialisation then you might get a problem but I'm big and ugly enough
to know when not to use it.

Doing that manually is time-consuming. Running a set of unit tests is
quick.

I really don't see how unit tests are going to cover the full range of
testing. Most of the bugs I get reported from testing and user's wouldn't be
picked up by unit testing.

Michael

 

[Jon Skeet [C# MVP]]

How do you deal with the problem that mocking requires modification to
the classes to support replaceability - e.g. descending from
MarshalByRefObject if a RealProxy technique is used, or using interfaces
/ delegates / virtual methods if Rhino Mocks is used?

When writing new classes, I use interfaces. Testing old code is
significantly harder.

How do you get rid of the modifications in release-mode code, or toggle
them safely, in order to avoid the downsides to interfaces & virtual
methods?

What downsides of interfaces are you thinking of?

It seems like something that aspect-oriented programming or binary
rewriting would be useful for, but that would move what you're testing
further away from what you've actually written.

True.

 

[Jon Skeet [C# MVP]]

You are fooling yourself.

What makes you think you have so much better an idea of the state of
the industry than I have?

For the most part, unit tests sit inside an
application and are never run until code is about to hit production. That is
closer to the truth than "the situation is changing". It's not.

Sure, there's a lot of talk about using unit tests regularly but the reality
is as soon as you turn your back, developers are back to coding habit. for
better or for worse.

The code that I have reviewed all have unit tests attached to them. I'd
wager a finger that during the development cycle these unit tests hardly
broke a sweat. And that's real world experience talking.

So make running the tests part of the automated overnight build. That
way you *know* they're being executed, and you can see how much of the
system is being covered. High coverage certainly doesn't guarantee bug-
free code or even good tests, but it's a start.

When you say "and that's real world experience talking" you sound as if
you're accusing me of making it all up. I'm not. Where I work, we *do*
unit test - and we run them. If I'm given code to review, I review the
unit tests as well. If they're not adequate, I'll require them to be
improved. I make sure the unit tests pass before passing the code - and
I require that the existing unit tests haven't been broken.

None of this is particularly hard, so any talk of "only perfect
developers can do it" is nonsense.

If your real world experience is that unit tests aren't being used
properly, it's up to you to improve the situation - hold people to a
high standard, and over time things get better. Persuading people of
the benefits of unit testing can take a while (partly because the big
benefit is a long-term one) but it's far from impossible.

 

[Jon Skeet [C# MVP]]

I've been fixing it in e&c in vb6 like that for 10 years and it rarely
causes a problem.

Or it causes problems you haven't noticed. There are such things as
subtle bugs.

I don't really see how it could if the sql string is
created on the line before it is used, eg

sql = "DELECT * FROM SomeTable"
rs.Open sql

if the rs.Open line failed then the recordset hasn't opened and going back 1
line is not going to cause any problem. Sure if the sql statement is created
in 1 place and passed into an object and that object goes through all sort
of initialisation then you might get a problem but I'm big and ugly enough
to know when not to use it.

That's a simplistic example - more realistic is the situation where an
off-by-one error is shown in a particular run. You fix it, but that
code has already been executed 100 times in that run, with different
parameters which didn't show the bug. Now your fix can show that the
101st time is okay, but it doesn't say anything about whether it would
have worked for the first 100 times.

I really don't see how unit tests are going to cover the full range of
testing. Most of the bugs I get reported from testing and user's wouldn't be
picked up by unit testing.

Without seeing the actual bugs, it's hard to answer that except to say
that my experience is the reverse - and indeed when a bug has been
reported, the first thing I try to do is write a unit test that fails
due to it. I can then fix it and be confident that the code will stay
fixed, as the unit test will start failing if the bug ever creeps back.

Now, are you really going to tell me that unit testing wouldn't have
picked up your SQL example above? Because if you are, that suggests you
really *aren't* unit testing as effectively as you can.

 

[Michael C]

Or it causes problems you haven't noticed. There are such things as
subtle bugs.

Possible but highly unlikely. The worst thing it's ever done is made me
chase bugs that don't exist and that's been something like 3 times in 10
years.

That's a simplistic example - more realistic is the situation where an

Actually it's a typical example. Most of the time the bug can be fixed on
the line the debugger is pointing to or the one previous. I'll always look
through the appropriate lines to make sure it's not likely to cause side
effects.

off-by-one error is shown in a particular run. You fix it, but that
code has already been executed 100 times in that run, with different
parameters which didn't show the bug. Now your fix can show that the
101st time is okay, but it doesn't say anything about whether it would
have worked for the first 100 times.

So you would obviously take caution in such a situation. It's no big deal.

Without seeing the actual bugs, it's hard to answer that except to say
that my experience is the reverse - and indeed when a bug has been
reported, the first thing I try to do is write a unit test that fails
due to it. I can then fix it and be confident that the code will stay
fixed, as the unit test will start failing if the bug ever creeps back.

The bugs are usually difficult to reproduce and would require the unit test
to have fairly intimate knowledge of the business rules. If it had such
knowledge it would really just be a duplication of the business rule and
probably have the same bug in the test anyway.

Now, are you really going to tell me that unit testing wouldn't have
picked up your SQL example above? Because if you are, that suggests you
really *aren't* unit testing as effectively as you can.

I never said that.

Michael

 

[Mark Wilden]

High coverage certainly doesn't guarantee bug-
free code or even good tests, but it's a start.

Coverage is a damn good start to reducing bugs. I'd wager that 50% of the
bugs I've seen in production environments were due to a line of code never
being executed before. When I was managing a group (before TDD) I would
insist that any changes be stepped through in the debugger, to make sure
that all lines were executed.

I really like how NCover and NCoverExplorer integrate with TestDriven.NET.
With just a click or two of the mouse, I can see at least some of the tests
I still need to write (this is legacy code).

///ark

 

[Mark Wilden]

The bugs are usually difficult to reproduce and would require the unit
test to have fairly intimate knowledge of the business rules.

The unit tests better have more than "initimate" knowledge of the business
rules, else how do you know your code supports those rules?

If [the test] had such knowledge it would really just be a duplication of
the business rule and probably have the same bug in the test anyway.

The test specifies what the result of applying the business rules should be
(think of it as executable documentation). The code under test implements
those business rules. So, yes, there is some duplication. That's inherent in
any verification process. What's the alternative?

 

[Barry Kelly]

What downsides of interfaces are you thinking of?

I tend to write libraries / runtimes / frameworks rather than
applications. That means that if you "ship" interfaces, you get all the
maintenance side-effects of interfaces. A good discussion is in
Framework Design Guidelines by Cwalina and Abrams, section 4.3. Here are
some downsides:

* Brittleness - add a method to an interface and you must modify all the
classes that implement it. If you have exactly one interface per class
to create replaceability, there's nothing to stop other people from
implementing the interface, which paradoxically increases inadvertent
and unwanted coupling.

* Double-sided semantics - this is the same problem as virtual methods.
There is one set of semantics for the user of a given interface, how it
is supposed to manipulate the object. Then, there's another set of
semantics for the implementer of the interface: what sort of method
calls it should expect, what order, etc.

* Lack of constructors - one must introduce factories or some other
indirection, adding complexity where it mightn't otherwise be needed.

* Lack of any visibility other than public - often, to get classes
working well together, members with internal visibility are required.
For example, consider IDataReader, IDbCommand and related interface
implementations. When you call ExecuteReader() on an IDbCommand, the
IDataReader you get back needs a communication channel with the
underlying connection etc. How does one test the individual classes that
make up such a system of classes without exposing the implementation
details to the world, thus losing encapsulation, and possibly even
security?

* Lack of discoverability - if you've got a complicated arrangement of
classes, where there are properties, parameters and collections of
interface types, it can be awkward to figure out which classes are the
appropriate implementations of a given interface. It's fine for common
interfaces like IList and IBindingList, but I remember more complicated
COM scenarios where I was never sure if there was some stock
implementation I was supposed to provide, or whether I was supposed to
implement the interface myself. It was even worse when the interfaces
were parceled out by function, because then there was no help from the
syntax to tell you whether or not you could safely QueryInterface (aka
cast) for another interface.

On a totally unrelated side point, I've had some success with using
IServiceProvider for this QueryInterface-like functionality, especially
for some of the more esoteric kinds of QI like tearoffs or conditionally
implemented interfaces. I have a Cast method which falls back to the
object instance itself if the query for IServiceProvider fails. It's a
nicely dynamic way of reducing coupling as well as providing an avenue
for using composition / mixin-style behaviour, since the instance
returned needn't be the same as the object you're querying.

-- Barry

 

[Jon Skeet [C# MVP]]

The bugs are usually difficult to reproduce and would require the unit test
to have fairly intimate knowledge of the business rules. If it had such
knowledge it would really just be a duplication of the business rule and
probably have the same bug in the test anyway.

You don't need to duplicate the business *logic* in the test - just use
the data which demonstrates what's wrong. If you know that the square
root of 144 should be 12 but your code was telling you it was 11, the
unit test only needs to assert that the code will tell you it's 12 -
you don't need any logic for calculating the square root in your code.

I never said that.

In which case your unit test would have picked it up without you even
having to spend the time you took going through your code to get to
that point in the first place - E&C has no advantage there, as far as I
can see. Unit testing it has the advantage that you can have confidence
that future changes won't screw it up.

 

[Michael C]

You don't need to duplicate the business *logic* in the test - just use
the data which demonstrates what's wrong. If you know that the square
root of 144 should be 12 but your code was telling you it was 11, the
unit test only needs to assert that the code will tell you it's 12 -
you don't need any logic for calculating the square root in your code.

Makes sense, that is actually what I've done in the past.

In which case your unit test would have picked it up without you even
having to spend the time you took going through your code to get to
that point in the first place - E&C has no advantage there, as far as I
can see. Unit testing it has the advantage that you can have confidence
that future changes won't screw it up.

We're going around in circles here. This is great, if you do unit testing.

Michael

 

[Jon Skeet [C# MVP]]

I tend to write libraries / runtimes / frameworks rather than
applications. That means that if you "ship" interfaces, you get all the
maintenance side-effects of interfaces. A good discussion is in
Framework Design Guidelines by Cwalina and Abrams, section 4.3. Here are
some downsides:

<snip>

Yup, I take all those points on board. I think it very much depends on
what space you're working in - in a very "uncontrolled" situation such
as producing public libraries, perhaps the downsides outweigh the
upside for many situations. (I wonder how many of those would be fixed
by the ability to have internal interfaces? Just an aside.)

For my own work, interfaces have been incredibly useful - although I've
seen people go overboard with them too. (They're rarely useful for
data-oriented rather than behaviour-oriented types, IME.)

On a totally unrelated side point, I've had some success with using
IServiceProvider for this QueryInterface-like functionality, especially
for some of the more esoteric kinds of QI like tearoffs or conditionally
implemented interfaces. I have a Cast method which falls back to the
object instance itself if the query for IServiceProvider fails. It's a
nicely dynamic way of reducing coupling as well as providing an avenue
for using composition / mixin-style behaviour, since the instance
returned needn't be the same as the object you're querying.

Ooh, I haven't looked at that. I'll have a look - thanks for the
pointer.

 

[Alvin Bruney [MVP]]

What makes you think you have so much better an idea of the state of

the industry than I have?

You are off topic. keep going.

If your real world experience is that unit tests aren't being used
properly, it's up to you to improve the situation - hold people to a
high standard,

And I do. But I'm also practical. I realize that programmers coming out of
school havent heard of unit tests. Habits are already entrenched. And that's
not a bad thing either. It takes a while to adopt a new testing methodology
and it takes an even longer time if it's being shoved down their throats as
a silver bullet. Unit tests are not a silver bullet, but you make them out
to be. In fact, I'd like you to quote where you ever supported using the
debugger be it for E&C or for regular stepping over unit tests.

I'm not sold on this TDD either but that's another story. You may like
writing a unit tests before even looking at code but i don't to me that is
plain wasteful in the real world and counterintuitive. When I used to write
production code, i'd verify the bug and go to the code, fix the problem and
check the code back in. Before production roll out, i'd run my unit tests.
And rarely, very rarely would i hear from QA that the bug isn't fixed or
some regression error had been introduced. You don't need a unit test to be
thorough. developers have different ways of working and my job right now is
to grow these different approaches in a way that encourages growth without
stifling creativity. I do push the unit tests as tool but not at the expense
of other tools like E&C. They are both tools and it's high time you realize
that each has merit.

--
________________________
Warm regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
Professional VSTO.NET - Wrox/Wiley
The O.W.C. Black Book with .NET
www.lulu.com/owc, Amazon
Blog: http://www.msmvps.com/blogs/alvin

 

[Alvin Bruney [MVP]]

And how come paldino don't get sucked into these arguments? You can't sit on
the fence forever. Say something - take a stand. don't be afraid of jon.

--
________________________
Warm regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
Professional VSTO.NET - Wrox/Wiley
The O.W.C. Black Book with .NET
www.lulu.com/owc, Amazon
Blog: http://www.msmvps.com/blogs/alvin
-------------------------------------------------------

What makes you think you have so much better an idea of the state of
the industry than I have?

You are off topic. keep going.

If your real world experience is that unit tests aren't being used
properly, it's up to you to improve the situation - hold people to a
high standard,

And I do. But I'm also practical. I realize that programmers coming out of
school havent heard of unit tests. Habits are already entrenched. And
that's not a bad thing either. It takes a while to adopt a new testing
methodology and it takes an even longer time if it's being shoved down
their throats as a silver bullet. Unit tests are not a silver bullet, but
you make them out to be. In fact, I'd like you to quote where you ever
supported using the debugger be it for E&C or for regular stepping over
unit tests.

I'm not sold on this TDD either but that's another story. You may like
writing a unit tests before even looking at code but i don't to me that is
plain wasteful in the real world and counterintuitive. When I used to
write production code, i'd verify the bug and go to the code, fix the
problem and check the code back in. Before production roll out, i'd run my
unit tests. And rarely, very rarely would i hear from QA that the bug
isn't fixed or some regression error had been introduced. You don't need a
unit test to be thorough. developers have different ways of working and my
job right now is to grow these different approaches in a way that
encourages growth without stifling creativity. I do push the unit tests as
tool but not at the expense of other tools like E&C. They are both tools
and it's high time you realize that each has merit.

--
________________________
Warm regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
Professional VSTO.NET - Wrox/Wiley
The O.W.C. Black Book with .NET
www.lulu.com/owc, Amazon
Blog: http://www.msmvps.com/blogs/alvin

 

[Mark Wilden]

I'm not sold on this TDD either but that's another story. You may like
writing a unit tests before even looking at code but i don't to me that is
plain wasteful in the real world and counterintuitive.

Have you ever tried it? Because that's similar to how I felt before I became
"test-infected."

Intuition isn't a very useful tool in evaluating software development
techniques.

///ark

 

[Jon Skeet [C# MVP]]

You are off topic. keep going.

I'm not sure I see that, but never mind...

And I do. But I'm also practical. I realize that programmers coming out of
school havent heard of unit tests.

Actually, I'd expect more fresh graduates to have used unit tests than
"old hands" in the industry.

Habits are already entrenched. And that's not a bad thing either.

How can it not be a bad thing to be unwilling to adopt better
practices?

It takes a while to adopt a new testing methodology
and it takes an even longer time if it's being shoved down their throats as
a silver bullet. Unit tests are not a silver bullet, but you make them out
to be.

Intriguing, given that I've explicitly stated in this thread that it's
*not* a silver bullet.

In fact, I'd like you to quote where you ever supported using the
debugger be it for E&C or for regular stepping over unit tests.

Sometimes you need the debugger. It's a sad time when it happens
though, IME - it shows that you aren't completely in control of (or
don't really understand) what's going on. That shows that the code
isn't as clear as it should be.

However, I *have* supported (in this thread) using E&C for improving
UIs where one real judge of whether or not it's working is how it
looks. You can't unit test aesthetics.

I'm not sold on this TDD either but that's another story. You may like
writing a unit tests before even looking at code but i don't to me that is
plain wasteful in the real world and counterintuitive.

I agree it's counterintuitive. Have you tried it wholeheartedly though?
If you haven't, you can't really judge it.

When I used to write production code, i'd verify the bug and go to
the code, fix the problem and check the code back in. Before
production roll out, i'd run my unit tests.

So you'd check the code in before running your unit tests? If so,
you're really not getting full benefit from them. If you've got so many
that they take a very long time to run, you might only run the tests
for the area of code you're changing, but to have them and not to run
them until later isn't a good plan IMO.

And rarely, very rarely would i hear from QA that the bug isn't fixed
or some regression error had been introduced. You don't need a unit
test to be thorough.

True, in the same way that you don't need compiler errors and warnings
to stop you from writing bad code. The compiler could just ignore any
statement it didn't understand. I prefer the safety and early-warning
that a strict compiler gives though - and unit tests are just the same.

developers have different ways of working and my job right now is
to grow these different approaches in a way that encourages growth without
stifling creativity.

It seems very odd to encourage growth while saying that it's not a bad
thing for habits to be entrenched.

I do push the unit tests as tool but not at the expense
of other tools like E&C. They are both tools and it's high time you realize
that each has merit.

While you continue to deny that TDD has merit, of course.

E&C has merit in a very few situations, but when it comes to fixing
bugs, it has *less* merit than unit testing - and its availability
effectively discourages unit testing. It makes the less robust solution
easier in the very short term, which means that that is the course of
action many developers will take. Writing a unit test and making it
pass takes longer *at the time* than just fixing the code, but the
benefit is in the long term. People are generally bad at making
decisions which are good in the long term at slight expense of the
short term, unfortunately.