Motivation of software professionals

[Malcolm McLean]

In other words, we have adequate processes available but tend not to
adopt them. And _then_ because the products are seriously flawed we
disclaim liability because the products are in poor shape.

We need to get pushed from the outside, by purchasers of our software.
Unfortunately that hasn't happened.

Software management is not so stupid. If adequate procedures were
available that could ensure bug-free software, at reasonable cost and
time, they they would have been adopted. Except in a few areas
customers would soon shy away from 'no warrantry including the implied
warrantry of suitability for any particular purpose' products.

The fact is that many many formal methods are in existence. Some of
them might work, to some extent, and in some circumstances. But none
have really proved themselves when it comes to the acid test of
developing real software for non-trivial projects.

 

[John Koy]

It's not a question of bug _free_ software. There aren't any other
fields I can think of where it's possible to get away with no liability
simply by claiming that it's impossible to achieve perfection.

Exactly. Engineering is about measurable outcomes, quantification.
What's the equivalent of "this building can withstand a quake of
magnitude 7.5 for 30 seconds" in software? Can any of us state "this
software will stand all virus attacks for 12 months" or "this software
will not crash for 2 years, and if it does your loss won't exceed 20% of
all digital assets managed by it" ?

We can't even guarantee that it won't crash tomorrow, why? Well, for me,
because the underlying platform (OS/JRE/CLR/VM/etc) does not give me any
guarantees. I cannot build any engineering product on top of that, no
matter what process I employ.

Engineering is not about "during", it's about "after": accountability,
liability, warranties, hence insurability. And these shape how the
process of "during" must be. Without them, it's just some monkey
business, hence SD.

 

[Registered User]

It's also not entirely an issue of software "engineering" being an
infant field. The fact is that there exist adequate processes that apply
to every stage of the software lifecycle, and most software shops only
pay lip service to some or all of them. Doing proper requirements
analysis is not "undeveloped". Doing proper design is not "undeveloped".
Doing proper implementation in your languages of choice is not
"undeveloped". And doing proper testing and QA/GC is not "undeveloped".

In other words, we have adequate processes available but tend not to
adopt them. And _then_ because the products are seriously flawed we
disclaim liability because the products are in poor shape. That whole
mindset is not a problem that's going to be fixed by pushing for a
software engineering profession, because the desire for such a
professional status follows from a mindset that already follows the
basic principles in the first place.

We need to get pushed from the outside, by purchasers of our software.
Unfortunately that hasn't happened.

All good points but the 'we' gives me trouble. Weak project management
is the real 'we'.

<rant>
The SDLC should not be determined by the calendar, don't ask how long
'it' will take after promising a due date plucked out of thin air.
Images of what a web app should look like doesn't qualify as a design.
A design document shouldn't contain contradictory requirements and
business rules shouldn't describe how the UI is to function.
Re-usability doesn't end at the UI. Security shouldn't be an
afterthought. Requirements shouldn't disappear solely because the
completion date is nearing, maybe they shouldn't have been
requirements initially. Success shouldn't be based upon meeting an
arbitrary date. Define "forward compatibility" before making it a
requirement. Understand the differences between requirements, features
and chrome. ... KISS
</rant>

The are many reasons why over-promised and under-delivered appears to
be the de facto standard. Rather than continue to rant, let me point
to this article.
http://www.infoworld.com/d/adventur...thats-train-wreck-waiting-happen-477?page=0,0
or
http://tinyurl.com/it-train-wreck

regards
A.G.

 

[Wojtek]

Arved Sandstrom wrote :

And doing proper testing and QA/GC is not "undeveloped".

This is the equivalent of building a bridge, then driving over it. If
it does not collapse, then proper engineering was used.

If it does collapses, well then we just re-build it and try again.

 

[MarkusSchaber]

Hi,

We can't even guarantee that it won't crash tomorrow, why? Well, for me,
because the underlying platform (OS/JRE/CLR/VM/etc) does not give me any
guarantees. I cannot build any engineering product on top of that, no
matter what process I employ.

And that is the problem with the abstraction levels I mentioned. Even
the OS cannot guarantee anything because our mainstream hardware is
extremely underspecified (e. G. try to get any time guarantees for
hard disk access) and does not guarantee for anything (see the long
errata list of current CPUs, or the statistics about RAM error rates
due to cosmic rays.

 

[Seebs]

And that is the problem with the abstraction levels I mentioned. Even
the OS cannot guarantee anything because our mainstream hardware is
extremely underspecified (e. G. try to get any time guarantees for
hard disk access) and does not guarantee for anything (see the long
errata list of current CPUs, or the statistics about RAM error rates
due to cosmic rays.

Yeah, and if all you have is improvised materials, you can't be sure the
things you build will work. That doesn't mean you're not an engineer. The
sign of being not-an-engineer would be promising that they'll work anyway.

The job of the engineer is to know enough to build something as reliably
as possible with the available tools, and tell you how reliable that is.
Usually, with software, the answer is "not all that reliable". But since
software engineers know that, and tell people that, and give clear evidence
that they've carefully analyzed the components to be able to tell you that,
it sounds to me like they're doing exactly what engineers ought to do.

I think the obsession with whether or not there is liability misses the point;
that's a social convention adopted for fields where the tools are much
simpler, it's nothing to do with the substance of what engineers are doing.

-s

 

[Seebs]

Nobody knows how to build earthquake-immune buildings, yet engineers
give certain guarantees. When those are failed to be met, (s)he is
held liable. Maybe it's about time some "software engineers" were held
liable for their unreliable code in the same way.

Why?

Do you have any evidence to suggest that this kind of liability is actually
reasonable, justified, and/or necessary?

That's absolutely right. But we can't sit and wait for software
development to be refined on its own, we should probably do something
about it. Collectively or whatnot.

I agree. I just don't think that rants about liability or certification
are going to do anything. Neither of those has a thing to do with learning
to write more reliable software.

-s

 

[Lew]

Software management is not so stupid. If adequate procedures were
available that could ensure bug-free software, at reasonable cost and
time, they they would have been adopted. Except in a few areas

Bullshit.

Management doesn't adopt effective practices for a variety of reasons, but the
fact is that far too many projects are managed in fashions contrary to best
practices. It's a combination of application of an inappropriate management
paradigm (factory work vs. talent work), ignorance or disbelief of the
fundamentals, mistrust of the commitment and understanding of the developers,
and a desire to keep the process inefficient in order to collect more money.

The observable fact is that software projects are managed as though management
were stupid. That's why half to two-thirds (depending on whose figures you go
by) of all major software projects never go into production, and so many that
do have major, preventable bugs.

customers would soon shy away from 'no warrantry including the implied
warrantry of suitability for any particular purpose' products.

Adequate procedures are available, or haven't you been studying the subject?

The fact is that many many formal methods are in existence. Some of
them might work, to some extent, and in some circumstances. But none
have really proved themselves when it comes to the acid test of
developing real software for non-trivial projects.

Again, not true. Iterative development alone greatly increases productivity
and quality. Add in other elements of agile programming, or whatever the
rubric is for effective practices these days, and you asymptotically approach
perfection.

The field of software project management is well studied and well documented.
Effective techniques are known and published. And have been for decades.
They simply are not followed.

 

[Volker Borchert]

And within the available budget.

and tell you how reliable that is.

Anyone can build a bridge that stands up. It takes an engineer to build
on that barely stands up.

What about Galloping Gertie?

 

[James Kanze]

Arved Sandstrom wrote:

[...]
Engineering "certification" processes are considerably
better and more comprehensive than anything that most
software developers are ever exposed to. Starting with
education - there's no requirement at all that software
developers have a relevant degree or associate degree, or
indeed any real SD training at all. Try that with
prospective professional engineeers.
It's not just entry-level certification that software
developers lack. It's code of conduct, professional
education, duty to the client, professional discipline and
so forth. These are all standards. In order for software
"engineering" to really be engineering it has to adopt
similar standards.

As long as we disclaim all liability and give no warranties
for the solutions/products we build, SD cannot be an
engineering field and the term "software engineer" remains as
an oxymoron.

But that's really only the case for shrink-wrapped software (and
presumably, it doesn't exclude your legal guarantees). Most of
the projects I've worked on did have guarantees, and contractual
penalties for downtime.

 

[James Kanze]

Arved Sandstrom wrote:

Basically no-one knows how to built bug-free software, so the
lability exclusions are necessary.

Basically no one knows how to build 100% bug-free anything.
Witness Toyota. Globally, in fact, you can probably do better
with software than with most other things. And I've never
worked on a project where there have been liability exclusions
(which probably aren't legal anyway).

They would be commercial suicide in any other field. That
doesn't mean there is no such thing as software engineering,
only that it is new and undeveloped. Boilers used to regularly
explode at the beginning of the industrial revolution, now
such accidents are almost unheard of.

Well written software fails a lot less frequently than
automobile brakes.

 

[James Kanze]

Software management is not so stupid. If adequate procedures
were available that could ensure bug-free software, at
reasonable cost and time, they they would have been adopted.

Guaranteed error-free doesn't exist in any domain. It's not too
difficult to achieve error rates of less than one error per 100
KLoc, however, and at least one shop does less than one error
per million lines of code. Curiously enough, achieving one
error per 100 KLoc typical reduces your development costs. But
for whatever reasons, in many companies, this goes
unnoticed---in the end, if you're selling shrink wrapped
software, development costs are more or less negligible,
compared to marketing costs, so it doesn't matter if you spend a
bit more.

Except in a few areas customers would soon shy away from 'no
warrantry including the implied warrantry of suitability for
any particular purpose' products.

Legally, of course, you have certain rights, regardless of what
the "warrenty" says. It's just that as far as I know, no one
has attempted to enforce them for software.

The fact is that many many formal methods are in existence.
Some of them might work, to some extent, and in some
circumstances. But none have really proved themselves when it
comes to the acid test of developing real software for
non-trivial projects.

Less that one error per million lines of code is achievable.
And I've worked on projects where we had less than one error per
100 KLoc (going into integration).

 

[James Kanze]

Management doesn't adopt effective practices for a variety of
reasons, but the fact is that far too many projects are
managed in fashions contrary to best practices. It's a
combination of application of an inappropriate management
paradigm (factory work vs. talent work), ignorance or
disbelief of the fundamentals, mistrust of the commitment and
understanding of the developers, and a desire to keep the
process inefficient in order to collect more money.

I suspect that it's often a case of management not being able to
be everywhere, and the fact that marketing issues are far more
important than saving a bit of money (while simultaneously
improving quality) in development.

The observable fact is that software projects are managed as
though management were stupid. That's why half to two-thirds
(depending on whose figures you go by) of all major software
projects never go into production, and so many that do have
major, preventable bugs.

I've often wondered where such statistics come from. Over the
last twenty years, only one project I've worked on failed to go
into production, and none of the others had major, preventable
bugs---in at least one, the most serious bug that was found was
a spelling error in a log message, and in one other case, no
errors were found after delivery (but that was a very small
project).

Adequate procedures are available, or haven't you been
studying the subject?

Again, not true. Iterative development alone greatly
increases productivity and quality. Add in other elements of
agile programming, or whatever the rubric is for effective
practices these days, and you asymptotically approach
perfection.

From what I've seen, "agile" programming hasn't really changed
much. In fact, it's often been just a relabeling of the same
old procedures. (That's a common problem with "in"
labels---before agile programming, a lot of projects became "OO"
overnight. With no change in procedures.)

The field of software project management is well studied and
well documented. Effective techniques are known and
published. And have been for decades. They simply are not
followed.

Except where they are. I was a contractor for the last 25 or 30
years, and most of the places I worked did you more or less
effective techniques.

 

[James Kanze]

Exactly. Engineering is about measurable outcomes,
quantification. What's the equivalent of "this building can
withstand a quake of magnitude 7.5 for 30 seconds" in
software?

A contractual indemnity for each second of downtime? (Most of
the projects I've worked on have had such clauses in their
contracts.)

Can any of us state "this software will stand all virus
attacks for 12 months" or "this software will not crash for 2
years, and if it does your loss won't exceed 20% of all
digital assets managed by it" ?

Most of the projects I've worked on have had clauses to the
effect that "we will pay you x Euros per minute downtime". It
seems to be a more or less standard clause.

We can't even guarantee that it won't crash tomorrow, why?
Well, for me, because the underlying platform
(OS/JRE/CLR/VM/etc) does not give me any guarantees. I cannot
build any engineering product on top of that, no matter what
process I employ.

True, the underlying platform is always a risk. But then, you
don't have to push it, and you're normally not the first user of
it, and you've validated it, so you have some confidence that it
won't crash for what you're using it for.

 

[James Kanze]

And within the available budget.

and tell you how reliable that is.

Anyone can build a bridge that stands up. It takes an engineer
to build on that barely stands up.

Or to tell you exactly what it will stand up to, or how much it
will cost before you start building it.

And of course, that's exactly what we do every day when we
develop software. Customers won't give us the contract unless
we can provide concrete guarantees with regards to downtime,
etc., and unless we can specify a guaranteed fixed cost in
advance.

 

[James Kanze]

Arved Sandstrom wrote :

This is the equivalent of building a bridge, then driving over
it. If it does not collapse, then proper engineering was used.

If it does collapses, well then we just re-build it and try
again.

That's the way things were in software thirty or fourty years
ago. But it's true that some people have relabeled the
procedure, and are trying to sell it today. (See TDD.)

 

[James Kanze]

On Feb 10, 12:03 pm, Malcolm McLean <[email protected]>
wrote:> On Feb 10, 1:02 am, John Koy <[email protected]>
wrote:> Arved Sandstrom wrote:

Nobody knows how to build earthquake-immune buildings, yet
engineers give certain guarantees. When those are failed to be
met, (s)he is held liable. Maybe it's about time some
"software engineers" were held liable for their unreliable
code in the same way.

They are. That's why independent contractors have liability
insurance.

 

[James Kanze]

Do you have any evidence to suggest that this kind of
liability is actually reasonable, justified, and/or necessary?

Reasonable, justified or necessary, I don't know. But it's a
fact of life. If you deliver software, and it fails, you're
liable for it. Most of the time, the liability is spelled out
in the contract, but that still doesn't exclude any legal
guarantees the buyer may have.

 

[Arved Sandstrom]

Exactly. Engineering is about measurable outcomes, quantification.
What's the equivalent of "this building can withstand a quake of
magnitude 7.5 for 30 seconds" in software? Can any of us state "this
software will stand all virus attacks for 12 months" or "this software
will not crash for 2 years, and if it does your loss won't exceed 20% of
all digital assets managed by it" ?

We can't even guarantee that it won't crash tomorrow, why? Well, for me,
because the underlying platform (OS/JRE/CLR/VM/etc) does not give me any
guarantees. I cannot build any engineering product on top of that, no
matter what process I employ.

Engineering is not about "during", it's about "after": accountability,
liability, warranties, hence insurability. And these shape how the
process of "during" must be. Without them, it's just some monkey
business, hence SD.

[ SNIP ]

Car and computer and TV manufacturers don't guarantee that their
products are 100% absolutely going to work either - why should we have
to? The point being that with existing and understood software
development methodologies, if those are assiduously applied then we can
safely state that for a given population of application deployments that
such and such a percentage of them will fail badly, another fraction
will encounter serious problems that require dedicated support under
warranty, another fraction will encounter minor problems, and so forth.

It's precisely this kind of statistical knowledge that lets you provide
consumers with certain protections - warranties, support offers, and so
forth. We're already doing it with major applications - we could do this
with the majority if we just bothered to write quality software in the
first place.

Seriously, though, why the insistence on perfection? We don't get
perfection from engineers (or other professionals) either, nor from
manufacturers of tangible goods and structures. Transportation
infrastructure crumbles before its time. We are resigned to consumer
goods that must be regarded as disposable (and not all are _designed_ to
be disposable). We accept that not so long after buying a new car that
we will be regularly repairing it. Sick buildings are common. Tens of
thousands of surgical mistakes are made every year just in North
America. Manufacturers of electronics and electrical equipment make a
mint off people who can't be bothered to return broken stuff, and buy
new replacements instead.

A software engineering profession would not require perfect software any
more than traditional engineers are expected to design perfect equipment
or machinery or structures. All I'm saying is that we can do
considerably better, and we can do that to the extent that we can
provide the same protection to consumers for software as we already for
cars or vacuum cleaners.

AHS

 

[Arved Sandstrom]

Why?

Do you have any evidence to suggest that this kind of liability is actually
reasonable, justified, and/or necessary?


I agree. I just don't think that rants about liability or certification
are going to do anything. Neither of those has a thing to do with learning
to write more reliable software.

Well, yeah, it does. Unless you believe that most software developers
and their employers are going to spend the extra time and money to do
all these things out of the goodness of their hearts. We already know
that the market will not force software quality to improve - it hasn't
happened yet.

AHS