Using ClamAV as a general purpose scanner

[Julian]

But somehow I doubt we've managed to convince you to abandon
your project .... at least until if and when clam becomes a reaonably
good general purpose scanner as determined by independent
comparative testing.

The jury is still out on that. I should point out that there is
*already* one Windows GUI for Clam, (ClamWin) and another written in
Java which may or may not run under Windows. So if I did decide to go
ahead with this, it won't be the first.

The term "useless" was used by the VTC when it included clam in
its tests. Personally, I made and do not make any speculations as
to future capabilities. Its current capabilites are unknown, just as
current capabilities of all av products are unknown.

Yes, but you quoted it without any qualifying remark that it was applied
to a much older build of the program.

What's unfair is your accusations and expression of suspicion of
ulterior motives on the part of those who have criticized clam. If
you know for a fact that someone here has something to gain
by engaging in clam bashing then be specific and spell it out.
I know of noone who has anything to gain by critqueing any
particular product. We do it as a public service. And then we
get flamed for it by religious fanatics defending their beloved
product. So be it. Such is life on newsgroups, if you can call
that a life

It's just a feeling I got from the way a number of people went to great
lengths not only to point out Clam's failings but give reasons why they
thought they could never be addressed. Clam's developers are public
spirited volunteers, too. Because of this, I think that they at least
deserve to be criticised kindly, not have their work denigrated using
old facts, speculation and half-truths.

 

[Art]

The jury is still out on that. I should point out that there is
*already* one Windows GUI for Clam, (ClamWin) and another written in
Java which may or may not run under Windows. So if I did decide to go
ahead with this, it won't be the first.

I don't think anyone here will be impressed with that justification
for going ahead. I'm sure not.

Right off the bat, its slowness is a real negative. People won't like
having their PCs bogged down. Or are you speculating that will be
fixed in the future as well?

Yes, but you quoted it without any qualifying remark that it was applied
to a much older build of the program.

And I also said and say it's still lousy when viewed in comparison
to real av products. I found the remark by the VTC both accurate
and amusing. Perhaps you should lighten up

It's just a feeling I got from the way a number of people went to great
lengths not only to point out Clam's failings but give reasons why they
thought they could never be addressed. Clam's developers are public
spirited volunteers, too. Because of this, I think that they at least
deserve to be criticised kindly, not have their work denigrated using
old facts, speculation and half-truths.

I've not seen any unfair criticisms. Nick comes down particularly
hard on it, but that's his way and his perogative. Personally, I think
you'd be wise to consider what he has to say in a objective and
unemotional way. His "speculations" concerning the future of
clam are probably right on the mark.


http://home.epix.net/~artnpeg

 

[Julian]

I don't think anyone here will be impressed with that justification
for going ahead. I'm sure not.

I wasn't trying to impress anyone, merely pointing out that the cat is
already out of the bag.

Right off the bat, its slowness is a real negative. People won't like
having their PCs bogged down. Or are you speculating that will be
fixed in the future as well?

I have addressed the speed issue by offering a "smart scan" option that
only scans executable types. Clam actually scans an individual file
faster than loading an instance of F-Prot for DOS to do the same
(probably thanks to being able to use a daemon that runs in the
background, vs loading a fresh instance in a DOS VM each time), so for
on-access scans or checking mail on a server it's quicker and uses less
resources.

The slow scanning is probably due to the fact that Clam apparently scans
the whole file instead of looking just in the place where a virus is
expected. I say this because I can type the Eicar test string into a
Word document, or add it as a resource string in a program, and it is
detected, where other scanners don't. This is probably also the reason
for the high rate of false alarms, so while I'm not speculating on
whether the developers will address this or not, it seems to me that
they could kill (or at least severely injure) two birds with one stone
if they did.

And I also said and say it's still lousy when viewed in comparison
to real av products. I found the remark by the VTC both accurate
and amusing. Perhaps you should lighten up

If you were a sysadmin whose boss comes into work and asks why you are
using a "useless" bit of software to check the mails on the company mail
server, it might not seem such a funny joke. Clam is neither useless nor
lousy at the job most people are using it for right now, and for that
reason alone it is a real AV product.

I've not seen any unfair criticisms. Nick comes down particularly
hard on it, but that's his way and his perogative. Personally, I think
you'd be wise to consider what he has to say in a objective and
unemotional way. His "speculations" concerning the future of
clam are probably right on the mark.

Some interesting points have been made, I accept. But I prefer to keep
an open mind on it, and not write its obituary when it hasn't even
reached maturity yet.

 

[Art]

If you were a sysadmin whose boss comes into work and asks why you are
using a "useless" bit of software to check the mails on the company mail
server, it might not seem such a funny joke. Clam is neither useless nor
lousy at the job most people are using it for right now, and for that
reason alone it is a real AV product.

But you're playing swithcheroony games here. There's quite a
difference between a scanner that's effective in sig detecting
most malware currently in circulation and a good general purpose
av as we know them today.

As I said in the beginning, I'd have no objection to Clam if it was
characterised differently.


http://home.epix.net/~artnpeg

 

[Roger Wilco]

The slow scanning is probably due to the fact that Clam apparently scans
the whole file instead of looking just in the place where a virus is
expected. I say this because I can type the Eicar test string into a
Word document, or add it as a resource string in a program, and it is
detected, where other scanners don't.

If it isn't the only string with or without CR/LF (or at least the first
string) then it shouldn't be detected at all. This is a false positive
detection on EICAR!!! What if I put the EICAR string in the NNTP
client's header - would the on-access scan stop the file every time one
of my posts came up? What if you visit the site to download some
archived EICAR and the scanner kept deleting the temp file because they
have the EICAR string uncovered in HTML?

 

[Frederic Bonroy]

Roger Wilco a écrit :

If it isn't the only string with or without CR/LF (or at least the first
string) then it shouldn't be detected at all. This is a false positive
detection on EICAR!!!

Well, yes - and I thought that its greatest problem was its reliance on
pattern matching. I would at least have expected it to heed the format
of the file it's dealing with.

I wonder if it looks for, say, .com viruses in PE files?

 

[kurt wismer]

Yes, but the tests that give it a "bad record" used an old version of
the product. Because Clam is relatively new, it is improving more
rapidly than established products. Some of the criticisms that were made
may already have been addressed. Therefore it is unfair to make
statements about its ability now, based on the results of these tests.

in that case we can say that it is *unproven* technology... there are
no scientifically rigorous test results that indicate it has improved
to a level where it would be worth considering for production systems...

 

[kurt wismer]

In the real world, most Clam users are server administrators, who should
have enough of a clue to determine whether an alarm is real or not...

you are being naive... that's called false authority syndrome...
there's no reason to believe an arbitrary server administrator knows
anything about how to tell if the item an anti-virus is alarming on is
actually a virus or not... i've met plenty who don't...

The point I was trying to make is how well a product performs in these
laboratory tests is no indication of how well they perform in the real
world today.

they are approximations of how well they perform in the real world...

Your statement is correct, but you are thinking of "what
if" situations with small probabilities.

it remains to be seen what size the probabilities are...

No test performed today can
guarantee that a product which passes it will catch the virus that
appears tomorrow. So we're all taking a chance, whatever we decide to use.

correct, you are taking a chance... but with a program that detects
significantly fewer viruses you're taking a bigger chance than you
would otherwise be taking if you used a program that detects a more
complete subset of the set of all known viruses...

 

[kurt wismer]

Julian wrote:
[snip]

The slow scanning is probably due to the fact that Clam apparently scans
the whole file instead of looking just in the place where a virus is
expected. I say this because I can type the Eicar test string into a
Word document, or add it as a resource string in a program, and it is
detected, where other scanners don't.

and do you know why that's bad?

This is probably also the reason
for the high rate of false alarms,

you've just demonstrated a false alarm...

so while I'm not speculating on
whether the developers will address this or not, it seems to me that
they could kill (or at least severely injure) two birds with one stone
if they did.

if only it were that simple...

scanning for X only where X is supposed to be is a lot easier said than
done...

[snip]

If you were a sysadmin whose boss comes into work and asks why you are
using a "useless" bit of software to check the mails on the company mail
server, it might not seem such a funny joke. Clam is neither useless nor
lousy at the job most people are using it for right now, and for that
reason alone it is a real AV product.

unfortunately, there are no scientifically rigorous independent test
results proving that clam is neither useless nor lousy... you have, at
best, anecdotal evidence...

 

[=?ISO-8859-1?Q?=BBQ=AB?=]

Yes, but the tests that give it a "bad record" used an old version
of the product. Because Clam is relatively new, it is improving
more rapidly than established products.

Maybe it is.

Some of the criticisms that were made may already have been
addressed.

Maybe they have.

Therefore it is unfair to make statements about its ability now,
based on the results of these tests.

ITYM "would be unfair" rather than "is unfair", since no one has
claimed to know that it's currently as bad as it was at the beginning
of 2004.

What would be more unfair would be for someone to give the impression
that users should trust ClamAv as a general purpose scanner before
rigorous testing has shown it to be reliable for such use. Though
it's not explicitly stated at <http://www.clamwin.org>, casual
visitors to that site might easily get the impression that there is
some good reason to trust it as a general purpose scanner for Win32
machines. In fact, they /are/ getting that impression. That can't be
a Good Thing.

 

[kurt wismer]

ITYM "would be unfair" rather than "is unfair", since no one has
claimed to know that it's currently as bad as it was at the beginning
of 2004.

actually, it's not the slightest bit unfair...

why?

because this is *exactly* the same treatment that NAV received in
alt.comp.virus years ago when it too was lacking in the
good-track-record department... it wasn't until after a few years of
performing well in independent tests that historical performance
criticisms stopped...

(and then it went and developed a bad case of bloat)

what would be unfair would be to hold this new anti-virus to a
different standard than all the other anti-virus products are held to...

What would be more unfair would be for someone to give the impression
that users should trust ClamAv as a general purpose scanner before
rigorous testing has shown it to be reliable for such use. Though
it's not explicitly stated at <http://www.clamwin.org>, casual
visitors to that site might easily get the impression that there is
some good reason to trust it as a general purpose scanner for Win32
machines. In fact, they /are/ getting that impression. That can't be
a Good Thing.

more importantly, they're getting that impression from julian himself
because he insists on endorsing it and suggesting it to people who need
real protection... in spite of the fact that, when pressed on the
subject, he acknowledges that it's "still under development" and is
basically in "beta"...

in other words - julian is being part of the problem rather than part
of the solution... pushing an av product that isn't ready for
prime-time yet and has no proven track record onto the unsuspecting
public...

all of a sudden the word "shill" popped into my head...

 

[Julian]

in that case we can say that it is *unproven* technology... there are
no scientifically rigorous test results that indicate it has improved
to a level where it would be worth considering for production systems...

Okay, I'll accept the last part. But tests only prove the ability of
something to pass the tests. They do not necessarily reflect performance
in the real world, and they can't possibly measure a product's ability
to cope with new threats that might appear in the future.

What about the anecdotal evidence from people using it on production
mailservers, who claim that it detects worms as well as or better than
some established products? I'd say that makes it *proven*, at least in
that application.

 

[Julian]

ITYM "would be unfair" rather than "is unfair", since no one has
claimed to know that it's currently as bad as it was at the beginning
of 2004.

But I thought we were discussing whether Clam was suitable for use
*now*, therefore one might easily assume that these comments represented
the poster's opinion in its performance now, and not more than a year ago.

What would be more unfair would be for someone to give the impression
that users should trust ClamAv as a general purpose scanner before
rigorous testing has shown it to be reliable for such use. Though
it's not explicitly stated at <http://www.clamwin.org>, casual
visitors to that site might easily get the impression that there is
some good reason to trust it as a general purpose scanner for Win32
machines. In fact, they /are/ getting that impression. That can't be
a Good Thing.

I tend to agree, although if you are as picky with words as you were
with me just now, nothing stated on the site is actually untrue, and
they aren't suggesting that anyone should trust it as their only virus
scanner.

 

[Julian]

more importantly, they're getting that impression from julian himself
because he insists on endorsing it and suggesting it to people who need
real protection... in spite of the fact that, when pressed on the
subject, he acknowledges that it's "still under development" and is
basically in "beta"...

in other words - julian is being part of the problem rather than part of
the solution... pushing an av product that isn't ready for prime-time
yet and has no proven track record onto the unsuspecting public...

all of a sudden the word "shill" popped into my head...

Could you point me to some statement I have made "pushing" it to people?
In fact, I think I have only mentioned ClamWin specifically in the
context of pointing out that Clam is already available as a general
purpose AV tool for Windows.

All I have argued against is the negative comments based on old test
results or someone's "opinion" that the open source movement is
incapable of addressing the areas of weakness that still remain. I even
published an article on my own website at
http://www.tech-pro.net/clamav.html that pointed out some of these
weaknesses.

I have, however, given it credit for areas in which it appears to be
successful, viz: as a scanner for mail servers. If that constitutes
"endorsement" then I guess the meaning of the word must have changed
while I slept.

 

[Julian]

you are being naive... that's called false authority syndrome... there's
no reason to believe an arbitrary server administrator knows anything
about how to tell if the item an anti-virus is alarming on is actually a
virus or not... i've met plenty who don't...

People who run Linux servers are, by definition, a lot more techno-savvy
than some of the sysadmins on Windows boxes. I doubt if there is anyone
using Clam on a server who doesn't know how to submit a sample for checking.

they are approximations of how well they perform in the real world...

Yes, but the nature of test results is that they make people focus on
the negatives. AV companies are so paranoiac about them because they
know that a product that detects, say, 99.8% of the test samples will be
judged as superior to one that detects, say 97%. Which, technically, it
is. But that missing 2.8% might be things that a user is extremely
unlikely to encounter, and there might be other things, like a better
UI, a lower resource usage or lower cost, that makes the less well
performing scanner actually a better overall product.

By basing one's assessment of Clam solely on its deficiencies, you make
it seem much worse than it actually manages to be in practice.

correct, you are taking a chance... but with a program that detects
significantly fewer viruses you're taking a bigger chance than you
would otherwise be taking if you used a program that detects a more
complete subset of the set of all known viruses...

Agreed. I think Clam, as it stands now, is not suitable for being used
as the *sole* protection method against viruses. It needs the ability to
detect polymorphics, improved macro virus detection, and fewer false
alarms, before it could take on that responsibility. In conjunction with
other tools for detecting possible viruses it may still be capable of
performing a useful role, however, bearing in mind that currently
superior solutions may be inapplicable due to cost or licensing reasons.

 

[Julian]

If it isn't the only string with or without CR/LF (or at least the first
string) then it shouldn't be detected at all. This is a false positive
detection on EICAR!!! What if I put the EICAR string in the NNTP
client's header - would the on-access scan stop the file every time one
of my posts came up? What if you visit the site to download some
archived EICAR and the scanner kept deleting the temp file because they
have the EICAR string uncovered in HTML?

Yes, I'm aware that the EICAR test is only supposed to be detected in
that specific instance, Roger. I think Clam is basically working like
other scanners do with the crap detector enabled.

I just tried putting the Eicar string into an email header and scanning
it, and it was detected. So now you know how to have a little fun with
anyone who uses Clam on their mailserver. ;-)

 

[Julian]

Well, yes - and I thought that its greatest problem was its reliance on
pattern matching. I would at least have expected it to heed the format
of the file it's dealing with.

I wonder if it looks for, say, .com viruses in PE files?

I would have expected it to as well, but isn't the problem with COM
files the fact that they don't start with any identifiable header? It
would have to base its decision on whther a file is a COM file solely on
the extension, which is probably a concept alien to the Linux-oriented
developers.

I mentioned earlier, I added the EICAR string as a text resource to a
Windows executable file, and it was detected. Whether it will detect the
signature of an EXE virus in a file modified to look like a COM file is
something I don't currently have time to investigate.

 

[Julian]

unfortunately, there are no scientifically rigorous independent test
results proving that clam is neither useless nor lousy... you have, at
best, anecdotal evidence...

Well, that's all most of us have about most things. I really would like
to know how well the *current* build of Clam does at detecting viruses
that are ITW now. However, I'm not going to jump to any conclusion based
on the performance of a build that is more than a year old. There have
been a lot of changes since then, as the release history shows.

 

[Roger Wilco]

I would have expected it to as well,

I would expect it NOT to. Why look for specific viruses in places they
won't be, isn't that a waste of time?

but isn't the problem with COM
files the fact that they don't start with any identifiable header?

All should be familiar with linear executables.

It
would have to base its decision on whther a file is a COM file solely on
the extension, which is probably a concept alien to the Linux-oriented
developers.

The extension is only an indicator and cannot be trusted. If a virus
searched for targets by extension, then you can expect to find that
virus in files with that extension - if a virus searches for targets by
examining the files' contents, then that is a different story.

I mentioned earlier, I added the EICAR string as a text resource to a
Windows executable file, and it was detected.

Then they either didn't understand the concept of the EICAR standard
file, or they did and chose to ignore it.

Whether it will detect the
signature of an EXE virus in a file modified to look like a COM file is
something I don't currently have time to investigate.

Forget extensions and focus on filetypes.

 

[kurt wismer]

Okay, I'll accept the last part. But tests only prove the ability of
something to pass the tests. They do not necessarily reflect performance
in the real world,

actually they approximate how the products will perform in the real
world... as is the case with all scientifically controlled experiments...

and they can't possibly measure a product's ability
to cope with new threats that might appear in the future.

can't possibly? one can't possibly test old product versions with new
viruses? how come?

What about the anecdotal evidence from people using it on production
mailservers, who claim that it detects worms as well as or better than
some established products? I'd say that makes it *proven*, at least in
that application.

anecdotal evidence doesn't generally prove anything - that's why it's
distinguished from other types of evidence... as for the admins who
claim it's detecting worms, i say how do we know they were real worms?
how do we know they were still viable? how do we know they weren't
neutered in transit before getting to those mail servers?

answer: we don't...