Using ClamAV as a general purpose scanner

[Roger Wilco]

And how many of those viruses are samples that have never been seen in
the wild, including ones that the analyses say things like "can only be
persuaded to replicate with great difficulty"?

If all you are concerned with are the ones that replicate profusely and
can be easily 'detected' visually by the subject lines alone, then you
could boast a pretty high number of detections with little technology.
Having the technology to detect viruses that only replicate under
certain specific conditions may again come in handy when the current
breed of worm writers subsides and gives way to file infectors being
written by the more skilled coders.

The only value these
claims of total number of viruses detected have are to the marketing
department, where it helps to bolster the claims of the big-name
products at the expense of newcomers that are not "in the loop" for the
exchange of virus samples.

So you DO see the lack of definitions (or detections) as relating to a
lack of submitted samples?

Do the commercial AV developers pass new
virus samples to the Clam / Open AV people, like they allegedly do with
each other?

A good question - the answer to which I hope is 'yes'. In addition I
would hope that older samples of the more AV evasive viruses were also
shared so a new AV could attempt to determine a way to reliably detect a
mutation engine that only works successfully a small percentage of the
time in an emulated environment and only a small amount better in a
'real' one.

If I have three viruses and I use the same mutation engine on all three,
it would be sufficient for an AV (OysterAV v1) to detect the engine and
call the detections three instances of the same virus (and block them
using one definition) - but for identification (and possible removal or
'healing') more would be needed and stating that the engine now
identifies three viruses (which happen to use the same mutation engine)
would change the number of viruses now detected to three (OysterAV v2).

A general purpose scanner should be more ... general. ClamAV is being
presented (at least here) as a good AV for, specifically, e-mail vector
malware - plus some capability outside that specific application.
Slapping a gui on it and using it as a GP on-demand or on-access scanner
does not mean it will be as impressive. If ClamWin is using that engine
and definition pool I still think that 30,000 shows a lack - especially
when the pool of malware is virus/worm/trojan.

 

[kurt wismer]

So you disagree then with Dr. Brunnstein at the VTC who claims
that its a auto-switch to heurustic mode:

http://agn-www.informatik.uni-hamburg.de/vtc/AVTests/cheat.htm

as a matter of fact i don't... i even agree with his contention about
what conditions scanner speed tests should be conducted under...

a heuristic-only mode is a good explanation of what findvirus was doing...

Well, then maybe cold hard facts will change your mind. I just ran a
scan using McAfee's command line SCAN.EXE on a small collection
of various malwares. With no external switches set, there were 2,392
alerts. With the /VID switch set, the number of alerts dropped
to 2,294 ... roughly a 9.5% and quite a significant difference.

i stand corrected then...

 

[kurt wismer]

Sorry for being offensive, but in my opinion you are actually rather
limited in your ability to interpret facts. And this is indeed abysmal.

ad hominem attacks don't do a very good job of countering his argument....

 

[kurt wismer]

The 0.x version number should tell anyone that Clam is still a beta
product. Therefore any shortcomings exposed by testing should be viewed
as things that need to be addressed in future releases, not used forever
as a stick to beat it with.

and people should, of course, think twice about relying on beta
products on their production systems...

 

[kurt wismer]

Christoph Cordes wrote:
[snip]

Do you realy believe that the number of signatures tells anything about
the performance of an AV.

an av's job is to detect viruses... one that detects significantly
fewer viruses (less than half the known viruses) definitely does not
perform well at it's job...

[snip]

With 30.000 signatures there is no way to compete with them, but it´s by
far enough to do the job it was made for and even a bit more.

no it's not enough... not by a long shot... the primary job of an
anti-virus is to stop viruses and clamav can stop fewer than half the
known viruses...

OK - what´s next? Maybe we can draw a conclusion by reading the
horoscope of one of the developers?

no, hopefully what happens next is that the people behind it stop
making excuses for their failings and get back to correcting the
problems...

hopefully around the same time, the people who aren't part of the
production team but who are proponents of it anyways recognize that the
rest of us have very good reason to balk at recommendations for an
anti-virus product that detects fewer than half the known viruses...

 

[kurt wismer]

And how many of those viruses are samples that have never been seen in
the wild,

every virus has at one time or another not been seen in the wild...
since no one has perfect knowledge of the future, that argument holds
no water...

further, no one has perfect knowledge of what viruses actually are in
the wild...

including ones that the analyses say things like "can only be
persuaded to replicate with great difficulty"? The only value these
claims of total number of viruses detected have are to the marketing
department, where it helps to bolster the claims of the big-name
products at the expense of newcomers that are not "in the loop" for the
exchange of virus samples. Do the commercial AV developers pass new
virus samples to the Clam / Open AV people, like they allegedly do with
each other?

if they do not then it's for one simple reason - the clam/openav people
have not yet earned their trust... sample exchange does not take place
between companies, it takes place between researchers who know and
trust each other... it is entirely possible for a company to run out of
trusted researchers and then be left high and dry despite being a
mainstream commercial av vendor... the trusted network operates outside
of the context of the companies those people work for...

 

[kurt wismer]

Julian wrote:
[snip]

I think detection of "zoo" virus samples and the sophistication of the
software are two separate matters. Inability to detect samples that
aren't found in the wild may have more to do with not spending time
generating signatures for things that aren't any current threat to
users,

<sarcasm>
oh yes, lets wait for an outbreak before we add detection for a virus...
</sarcasm>

that is not in the public's best interest... while outbreaks do still
happen, it's usually not because av vendors shelved the samples until
the threat became real... sitting on samples and not doing anything
about them simply because those viruses don't appear to currently be a
threat is not the sign of an organization that is serious about
offering protection...

or not wanting to waste memory and CPU time scanning for
something you're unlikely ever to find, something that strikes a chord
in those of us who feel that modern AV products are becoming bloated
resource hogs.

the bloat in modern anti-virus products has little to do with the
number of viruses they detect... it has to do with the bells and
whistles they add in... and it should be noted that only some of them
add in such bells and whistles...

If one of these "zoo" viruses did become prevalent in the
wild, they would be treated like a new virus and added to the database
quickly.

while *real* anti-virus products would have already had the ability to
stop it beforehand...

not dealing with zoo viruses unnecessarily expands the window of
exposure when that virus is no longer confined to the zoo - that is not
a reasonable philosophy for an av developer to take...

 

[Julian]

Shouldn't they wait until Clamav becomes a real alternative to
professional scanners, if it happens someday? Don't you think it's
dangerous to rely on Clamav in its current state?

I have to admit I'm a little surprised that there are people apparently
willing to bet their jobs on the hope that a polymorphic worm like
Magistr.a isn't going to run through their organization.

 

[Julian]

If all you are concerned with are the ones that replicate profusely and
can be easily 'detected' visually by the subject lines alone, then you
could boast a pretty high number of detections with little technology.

That wasn't what I said. I am concerned with all the ones that replicate
under normal conditions without the user doing anything. If an expert
virus analystr has to set up a computer in a specific way to get a
sample to replicate, then it's unlikely to be a threat in the real world.

Having the technology to detect viruses that only replicate under
certain specific conditions may again come in handy when the current
breed of worm writers subsides and gives way to file infectors being
written by the more skilled coders.

Yes, but that hasn't happened yet, so why damn a product that is still
under development and has focussed its efforts on meeting the threats
that are real right now?

So you DO see the lack of definitions (or detections) as relating to a
lack of submitted samples?

That's conjecture on my part.

A general purpose scanner should be more ... general. ClamAV is being
presented (at least here) as a good AV for, specifically, e-mail vector
malware - plus some capability outside that specific application.
Slapping a gui on it and using it as a GP on-demand or on-access scanner
does not mean it will be as impressive. If ClamWin is using that engine
and definition pool I still think that 30,000 shows a lack - especially
when the pool of malware is virus/worm/trojan.

Yes, but the lack is not 30,000 / 150,000 or whatever the number of
viruses is nowadays. Even products that claim to detect > 100,000
viruses still miss some of them. If they didn't, there would be no point
in VB, Hamburg etc. testing them. If those 30,000 are viruses that occur
in the wild then even though Clam still might not perform as well as the
one that detects 150,000, it would be unfair to say it is only one fifth
as effective.

 

[Julian]

every virus has at one time or another not been seen in the wild...
since no one has perfect knowledge of the future, that argument holds
no water...

That doesn't make what I said irrelevant. I have seen analyses of
viruses that state that because of a bug in the virus or difficulty in
achieving the replication conditions the virus represents no threat, and
you probably have too. What is the benefit to the average joe of his
scanner being able to detect such samples?

if they do not then it's for one simple reason - the clam/openav people
have not yet earned their trust... sample exchange does not take place
between companies, it takes place between researchers who know and
trust each other... it is entirely possible for a company to run out of
trusted researchers and then be left high and dry despite being a
mainstream commercial av vendor... the trusted network operates outside
of the context of the companies those people work for...

Well then, we have a chicken / egg situation here. Very convenient,
though, isn't it, for the commercial boys, who have no interest in
seeeing the success of a free virus scanner that would directly compete
with their lucrative corporate server licensed products.

 

[Julian]

I have to admit I'm a little surprised that there are people apparently
willing to bet their jobs on the hope that a polymorphic worm like
Magistr.a isn't going to run through their organization.

Then again, Clam isn't a bad choice right *now*. This, snipped from a
post in the ClamAV users newsgroup: "We also use Trend here ... of
course - it misses a LOT that ClamAV catches."

So, just because a product performs poorly in laboratory tests doesn't
mean it can't be more effective in the real world than one which did
better. The criticisms are still valid, but in practice, it is better to
use the tool that deals most effectively with the problems you face
*now*, than one which is better at dealing with problems you *might*
face tomorrow.

 

[kurt wismer]

Julian wrote:
[snip

Then again, Clam isn't a bad choice right *now*. This, snipped from a
post in the ClamAV users newsgroup: "We also use Trend here ... of
course - it misses a LOT that ClamAV catches."

So, just because a product performs poorly in laboratory tests doesn't
mean it can't be more effective in the real world than one which did
better.

the real world where people report performance without even knowing
whether the anti-virus in question is false alarming or not...

The criticisms are still valid, but in practice, it is better to
use the tool that deals most effectively with the problems you face
*now*, than one which is better at dealing with problems you *might*
face tomorrow.

that's a shortsighted philosophy... in the long term it's best to use a
product that's better at dealing with the most complete set of viruses
possible than one that is optimized for what you're facing *right now*...

 

[kurt wismer]

That wasn't what I said. I am concerned with all the ones that replicate
under normal conditions without the user doing anything. If an expert
virus analystr has to set up a computer in a specific way to get a
sample to replicate, then it's unlikely to be a threat in the real world.

that's not entirely true... there are all sorts of different computer
environments in the wild - it is not unreasonable for a virus analyst
to have to recreate such an environment in the lab... and doing so does
not imply that the virus would not be a threat in the real world....

Yes, but that hasn't happened yet, so why damn a product that is still
under development and has focussed its efforts on meeting the threats
that are real right now?

because *someone* insists on recommending it to people despite the fact
that it's not yet ready for prime-time...

 

[kurt wismer]

That doesn't make what I said irrelevant. I have seen analyses of
viruses that state that because of a bug in the virus or difficulty in
achieving the replication conditions the virus represents no threat, and
you probably have too. What is the benefit to the average joe of his
scanner being able to detect such samples?

viruses that are supposed to replicate but don't because of a bug are
called intended viruses... they are a minority...

Well then, we have a chicken / egg situation here. Very convenient,
though, isn't it, for the commercial boys, who have no interest in
seeeing the success of a free virus scanner that would directly compete
with their lucrative corporate server licensed products.

your tinfoil hat is too tight... it is *not* a chicken/egg situation...
it is not only possible for new people to become part of that trusted
network, it is happening right now as we speak...

 

[=?ISO-8859-1?Q?=BBQ=AB?=]

The 0.x version number should tell anyone that Clam is still a
beta product. Therefore any shortcomings exposed by testing should
be viewed as things that need to be addressed in future releases,
not used forever as a stick to beat it with.

I'm not using their bad record as a stick to beat them, just as a good
reason not to rely on ClamAv as a general purpose scanner. The beta
status, which you point out, is another good reason.

 

[Roger Wilco]

Roger Wilco wrote:

Yes, but that hasn't happened yet, so why damn a product that is still
under development and has focussed its efforts on meeting the threats
that are real right now?

I'm not damning them at all - I'm sure they are coming along quite
nicely.

What I AM saying is that they are not now ready for use as a "general
purpose" AV scanner as your post (and subject line) seem to suggest.

 

[Julian]

I'm not using their bad record as a stick to beat them, just as a good
reason not to rely on ClamAv as a general purpose scanner. The beta
status, which you point out, is another good reason.

Yes, but the tests that give it a "bad record" used an old version of
the product. Because Clam is relatively new, it is improving more
rapidly than established products. Some of the criticisms that were made
may already have been addressed. Therefore it is unfair to make
statements about its ability now, based on the results of these tests.

 

[Julian]

the real world where people report performance without even knowing
whether the anti-virus in question is false alarming or not...

In the real world, most Clam users are server administrators, who should
have enough of a clue to determine whether an alarm is real or not...

that's a shortsighted philosophy... in the long term it's best to use a
product that's better at dealing with the most complete set of viruses
possible than one that is optimized for what you're facing *right now*...

The point I was trying to make is how well a product performs in these
laboratory tests is no indication of how well they perform in the real
world today. Your statement is correct, but you are thinking of "what
if" situations with small probabilities. No test performed today can
guarantee that a product which passes it will catch the virus that
appears tomorrow. So we're all taking a chance, whatever we decide to use.

 

[Julian]

I'm not damning them at all - I'm sure they are coming along quite
nicely.

What I AM saying is that they are not now ready for use as a "general
purpose" AV scanner as your post (and subject line) seem to suggest.

I can't really take issue with that exact statement, Roger, which is
probably a good final summary of this topic.

It's the criticisms made elsewhere in this thread based on outdated
tests that are then used to justify the use of terms like "useless" and
"crap", and other arguments to suggest that Clam is incapable of
improvement (which are pure speculation) that I felt were unfair. I felt
(rightly or wrongly) that some people were out to rubbish Clam
regardless of the efforts its developers put into it.

 

[Art]

I can't really take issue with that exact statement, Roger, which is
probably a good final summary of this topic.

But somehow I doubt we've managed to convince you to abandon
your project .... at least until if and when clam becomes a reaonably
good general purpose scanner as determined by independent
comparative testing.

It's the criticisms made elsewhere in this thread based on outdated
tests that are then used to justify the use of terms like "useless" and
"crap", and other arguments to suggest that Clam is incapable of
improvement (which are pure speculation) that I felt were unfair.

The term "useless" was used by the VTC when it included clam in
its tests. Personally, I made and do not make any speculations as
to future capabilities. Its current capabilites are unknown, just as
current capabilities of all av products are unknown.

felt
(rightly or wrongly) that some people were out to rubbish Clam
regardless of the efforts its developers put into it.

What's unfair is your accusations and expression of suspicion of
ulterior motives on the part of those who have criticized clam. If
you know for a fact that someone here has something to gain
by engaging in clam bashing then be specific and spell it out.
I know of noone who has anything to gain by critqueing any
particular product. We do it as a public service. And then we
get flamed for it by religious fanatics defending their beloved
product. So be it. Such is life on newsgroups, if you can call
that a life


http://home.epix.net/~artnpeg