Using ClamAV as a general purpose scanner

  • Thread starter Thread starter Julian
  • Start date Start date
Roger said:
I see what you mean, however an 'infected' file once modified (after
infection has taken place) - and modified so that an AV doesn't detect
it - is more trojan (accidental trojan?) than virus since it is not the
usual form.
Click to expand...

Yes, but the viral code wasn't modified in any way. It was still present
in the executable, and capable of being executed, therefore the scanner
should have detected it.
Academic as you say - especially since you didn't name the virus.
Click to expand...

After so many years, I can hardly remember it...
The original 'seed' or 'dropper' can vary far outside the realm of where
the virus would be expected to reside. A virus that infects only exe
files can itself be contained in a dropper that is a com file. If the
dropper is common (like a worm) then the AV probably would detect it -
but the less common random natural mutation would go unnoticed until it
itself got recognised as a variant.

Most any programmer could alter a virally infected file so that it
sneaks past the AV - and there are probably many more of those then
there are of xmodem accidental trojans. And since neither of these
modifications turn up in the offspring, why should AV detect them? Your
sample of xmodem modified virus might be the only one in existence, and
you expect it to be detected? IMO that is not very realistic.
Click to expand...

Well, the scenario I described seemed like a pretty plausible one at the
time, when Internet use was far less widespread than it is now and some
people still did use direct connect BBSes. In this case, the executable
code was not altered in any way, and it was not a deliberately
constructed "dropper".

I have had arguments that an AV should be able to detect virus droppers,
because of the ease of constructing them (as you state) but I agree, in
practice it would be difficult unless the scanner created a protected
environment in memory and ran every executable to see what it is going
to do.
 
kurt said:
no, because their scanners *might* not detect them... they obviously
have a lot more control over whether their scanners can detect viruses
than they do over whether their scanners can detect some arbitrary
piece of crud used in a brain-dead magazine review... so yes, if you
were doing a high (enough) profile review, you're going to get
contacted by folks who would prefer that you not use crud...

that doesn't change the fact that scanners have been forced to detect
crud by brain-dead magazine reviewers who use crud in their tests and
therefore scanners cannot be used to weed out crud...



lets look at this a little more analytically, shall we? it wasn't
detected because it had been modified... at best it was a new variant
(did they add detection?) but more likely it was damaged... in any
case, modifications have a habit of making a virus not match it's
signature anymore...

you say the null bytes didn't affect it's ability to replicate - do you
know this for a fact? were you able to get 2nd generation offspring
from it?

if they said "it's not a virus anymore" and you got 2nd generation
offspring from it then it's your duty to name and shame, otherwise
you're blowing smoke...



you claim to have weeded the crud out of your collection by using
scanners - there's little room for any other interpretation... the only
legitimate way to weed crud out of a collection is to replicate the
samples to at least the 2nd generation or, if you happen to be an
expert virus analyst, you may be able to tell simply by looking at the
code (however, 2nd gen. offspring or lack thereof trump that)...



which, more and more, is a perfectly reasonable thing for magazines to
do... they don't have the resources to produce detection tests and even
if they did, the major players are close enough to each other in terms
of raw detection rates that those other properties are now much more
legitimate criteria for deciding which av to use...
Click to expand...

I've nothing significant to add to what I wrote in my reply to Roger.
The sample concerned was a genuine virus that had apparently been
modified by the process of downloading it using a protocol that padded
out the downloaded file to a multiple of 128 bytes, and this was enough
to cause the scanner to fail to detect it. I believe that the company
concerned did modify their scanner so it wasn't tricked by this in the
future, but it was several years ago and I obviously can't recall the
details. I only used it to try to illustrate that what might be crap to
you could be someone else's virus.

BTW, I know some scanners have a switch you can turn on to scan virus
collections, which presumably forces them to detect crap as you call it.
Despite being instructed to do so by some AV company PR people, I never
did so. I always scanned the collection using the "out of the box"
settings that most end users would use. (Except in the case of one
company that obviously didn't want to leave anything to chance, and had
already enabled the "scan crap" setting in the review copy they sent me.)
 
Julian said:
Yes, but the viral code wasn't modified in any way. It was still present
in the executable, and capable of being executed, therefore the scanner
should have detected it.
Click to expand...

capable of being executed and capable of self-replicating are 2
entirely different things...
Well, the scenario I described seemed like a pretty plausible one at the
time, when Internet use was far less widespread than it is now and some
people still did use direct connect BBSes. In this case, the executable
code was not altered in any way, and it was not a deliberately
constructed "dropper".
Click to expand...

y'know, the thought occurs that if this was so long ago, anti-virus
technology at the time would have been much, much less advanced...
I have had arguments that an AV should be able to detect virus droppers,
because of the ease of constructing them (as you state) but I agree, in
practice it would be difficult unless the scanner created a protected
environment in memory and ran every executable to see what it is going
to do.
Click to expand...

even that wouldn't always work... people who think av's should be able
to detect arbitrary virus droppers for known viruses don't understand
the halting problem...
 
Julian wrote:
[snip]
I've nothing significant to add to what I wrote in my reply to Roger.
The sample concerned was a genuine virus that had apparently been
modified by the process of downloading it using a protocol that padded
out the downloaded file to a multiple of 128 bytes, and this was enough
to cause the scanner to fail to detect it. I believe that the company
concerned did modify their scanner so it wasn't tricked by this in the
future, but it was several years ago and I obviously can't recall the
details. I only used it to try to illustrate that what might be crap to
you could be someone else's virus.
Click to expand...

now you're contradicting yourself... it wasn't crap to you and it also
wasn't crap to the anti-virus company or they shouldn't have bothered
to add detection for it...

if it can recursively self-replicate then it's a virus, not crap or
crud or any of the other monikers we have for *non-viruses*...
BTW, I know some scanners have a switch you can turn on to scan virus
collections, which presumably forces them to detect crap as you call it.
Despite being instructed to do so by some AV company PR people, I never
did so. I always scanned the collection using the "out of the box"
settings that most end users would use. (Except in the case of one
company that obviously didn't want to leave anything to chance, and had
already enabled the "scan crap" setting in the review copy they sent me.)
Click to expand...

yes, i seem to recall that too, but my recollection for the reason for
the flag is different than yours... dr. solly's findvirus had a exactly
this kind of switch because normally when it encountered more than X
number of different viruses it fell back into a fast scanning mode in
order to get better speed performance (because it was assumed it was
being used in some sort of anti-virus test)... the /collect flag (if i
recall correctly) was to prevent it from doing that so that the
scanning results would be optimized for accuracy instead of speed...

f-prot also had some interesting flags... /paranoid increased the
probability of getting "may be" alerts as i recall (and "may be" alerts
aren't exactly the same thing as detecting crud), and /guru i believe
turned on exact identification...
 
yes, i seem to recall that too, but my recollection for the reason for
the flag is different than yours... dr. solly's findvirus had a exactly
this kind of switch because normally when it encountered more than X
number of different viruses it fell back into a fast scanning mode in
order to get better speed performance (because it was assumed it was
being used in some sort of anti-virus test)... the /collect flag (if i
recall correctly)
Click to expand...

You don't. There was no external cheat switch. It was/is internal and
automatic. To _disable_ cheating you use the /VID switch (as most all
testers know and do :)). It wasn't so much for speed as it was for
enhanced crud detection. And at the risk of more false positives, it
also makes more "guesses" when it finds itself in a virus collection
being tested. It's probably still in the McAfee command line scanners.
It was the last time I checked, anyway. It's so well known that it
only fools the most naive testers of av scanner products.

The /collect switch is F-Prots external cheat switch which enables
crud detection. Frisk likes to insist that it be used by testers :)
It's actually quite useful for identifying a good deal of crud.


http://home.epix.net/~artnpeg
 
Julian said:
Yes, but the viral code wasn't modified in any way. It was still present
in the executable, and capable of being executed, therefore the scanner
should have detected it.
Click to expand...

Being not of the normal replicative form (a normal iteration) the file
in question could be expected to pass by the AV undetected. If it were a
common enough form, then I agree they should be expected to detect it.
Otherwise it would be best described as a first generation virus (germ,
dropper, seed, whatever...). It is unrealistic to think that viral code
can't hide from scanners - the best you can hope for is that many or
most of the normal forms are detectable.
After so many years, I can hardly remember it...



Well, the scenario I described seemed like a pretty plausible one at the
time, when Internet use was far less widespread than it is now and some
people still did use direct connect BBSes. In this case, the executable
code was not altered in any way, and it was not a deliberately
constructed "dropper".
Click to expand...

This is simplistic, but what if the virus (to avoid repeatedly
reinfecting itself) padded the file length to a multiple of some figure
(or used some other algorthm)? You could expect all program files
'infected' by this virus to have a file length that fits that algorthm
(so the virus 'knows' it's an infected file and passes it over as
"uninfectable"). The AV is supposed to be able to detect 'infected
files' and the modified file no longer fits the mold.

But you can see that the AV can't be expected to 'know' the difference
between a deliberate or accidental modification. They aren't supposed to
protect you from every form - only the ones it knows about and considers
a threat
I have had arguments that an AV should be able to detect virus droppers,
because of the ease of constructing them (as you state) but I agree, in
practice it would be difficult unless the scanner created a protected
environment in memory and ran every executable to see what it is going
to do.
Click to expand...

It is far more than "difficult", it is provably "impossible" at a very
basic level even with near perfect (or even perfect) emulation. But this
is beside the point - your sample would probably be as easy to detect as
the natually occuring specimens with only a slight definition change.
The thing is, this form might not have been submitted to them by enough
peeps.
 
Thanks for your comments and the link, Nick.
Click to expand...

I hate to jump in and feed Art, but I'm running ClamAV on our two MX servers.
We run ClamAV on the MX servers and Symantec on the Exchange server they feed.
Aside from the day that Clam failed due to a bug in the 0.82-1 version of the
milter, SAV hasn't seen a virus, worm, or bit of malware come in via mail.

Clam may not be perfect, but it works.

In production one relies on layered protection. No single AV product is
sufficient.

-- Steve
Chief Technology Officer
<company withheld>

Steve Stern
Manager, WUGNET VirusCentral Forum
http://community.compuserve.com/viruscentral
 
Art said:
You don't. There was no external cheat switch. It was/is internal and
automatic. To _disable_ cheating you use the /VID switch (as most all
testers know and do :)).
Click to expand...

you've misread my post... of course if you had quoted the entire last
sentence instead of chopping it in half you would have seen that our
descriptions of the behaviour are actually in agreement (that the
switch prevents the speed optimization)...

you are right about it being /VID rather than /collect, though... i
knew it as soon as i read it... then i googled it just to refresh my
memory on other things...
It wasn't so much for speed as it was for
enhanced crud detection.
Click to expand...

my googling suggests otherwise... it only stopped findvirus from going
into 'review' mode (where it would say "like" instead of "identified
as" and be less specific about the exact variant) after finding 10
different viruses...
And at the risk of more false positives, it
also makes more "guesses" when it finds itself in a virus collection
being tested.
Click to expand...

my googling suggests it actually did not affect the detection rate at
all...
 
Steven Stern said:
I hate to jump in and feed Art, but I'm running ClamAV on our two MX servers.
We run ClamAV on the MX servers and Symantec on the Exchange server they feed.
Aside from the day that Clam failed due to a bug in the 0.82-1 version of the
milter, SAV hasn't seen a virus, worm, or bit of malware come in via mail.

Clam may not be perfect, but it works.
Click to expand...

It is expected (and accepted) that ClamAV does what it does, well. But
if a am following this thread right - the OP was asking about "general"
AV use, and ClamAV is more used as an e-mail system utility for
detecting known e-mail vector worms. Am I misreading, or is ClamAV
"really" intended for general AV use?

If I wrote a utility (OysterAV) that woked very well at detecting IRC
bots and IRC vector worms, I would try to make it clear to the customer
that it is not recommended for general AV use. The problem would be that
the customer isn't going to know or care what that means.

As much as most peeps like to dis "security through obscurity" I think
it has its place in AV and open source can't compete for this reason.
 
you are right about it being /VID rather than /collect, though... i
knew it as soon as i read it... then i googled it just to refresh my
memory on other things...


my googling suggests otherwise... it only stopped findvirus from going
into 'review' mode (where it would say "like" instead of "identified
as" and be less specific about the exact variant) after finding 10
different viruses...
Click to expand...

So you disagree then with Dr. Brunnstein at the VTC who claims
that its a auto-switch to heurustic mode:

http://agn-www.informatik.uni-hamburg.de/vtc/AVTests/cheat.htm
my googling suggests it actually did not affect the detection rate at
all...
Click to expand...

Well, then maybe cold hard facts will change your mind. I just ran a
scan using McAfee's command line SCAN.EXE on a small collection
of various malwares. With no external switches set, there were 2,392
alerts. With the /VID switch set, the number of alerts dropped
to 2,294 ... roughly a 9.5% and quite a significant difference.


http://home.epix.net/~artnpeg
 
It is expected (and accepted) that ClamAV does what it does, well.
But if a am following this thread right - the OP was asking about
"general" AV use, and ClamAV is more used as an e-mail system
utility for detecting known e-mail vector worms. Am I misreading,
or is ClamAV "really" intended for general AV use?
Click to expand...

I think you are right, though I haven't followed the entire thread.
<http://www.clamav.net/abstract.html> says, "The main purpose of
this software is the integration with mail servers (attachment
scanning)." I think the notion of using it as a general AV solution
is gaining ground because of ClamWin,
<http://www.clamwin.com/index.php?option=content&task=view&id=5>.
People in alt.comp.freeware are now recommending ClamWin left and
right, complete with the sort of "real world" AV testing Nick loves
to rip to shreds.
 
please point your browser to www.clamav.net, download the latest
stable release, read the source code, and stop defaming us.
Click to expand...

I'm not capable of reading the source to be able to tell what ClamAv is
currently doing about macro virus detection, but I take it from your
comments that over the last few months there has been incredible
improvement in that area? I don't mean to defame anyone, but the July
2005 Uni Hamburg test results were that ClamAv detected 0.5% of the
macro viruses in their zoo.
 
» said:
I'm not capable of reading the source to be able to tell what
ClamAv is currently doing about macro virus detection, but I take
it from your comments that over the last few months there has been
incredible improvement in that area? I don't mean to defame
anyone, but the July 2005 Uni Hamburg test results were that
ClamAv detected 0.5% of the macro viruses in their zoo.
Click to expand...

Sorry, I made one typo and one oversight. Those results were
published in July 2004 (obviously not 2005), but the tested products
were submitted in Dec 2003.
 
I'm not capable of reading the source to be able to tell what ClamAv is
currently doing about macro virus detection, but I take it from your
comments that over the last few months there has been incredible
improvement in that area? I don't mean to defame anyone, but the July
2005 Uni Hamburg test results were that ClamAv detected 0.5% of the
macro viruses in their zoo.
Click to expand...

You mean 2004. I see CLAM was rated by the VTC as "useless" :)

http://agn-www.informatik.uni-hamburg.de/vtc/naveng.htm


http://home.epix.net/~artnpeg
 
Art said:
You mean 2004. I see CLAM was rated by the VTC as "useless" :)
Click to expand...


The tested version was CLAM AntiVirus v:0.54 - the current stable
release is 0.82. Please tell me that you didn´t build your opinion on
such an outdated result.The ability to handle macro-viruses was first
implemented in 0.70. Somehow this is getting more and more ridiculous.

The headline of this test was: AntiVirus Scanner Tests July 2004
The 0.70 was released in march 2004. The 0.65 in november 2003.

But i guess as long as it supports the halftruths, speculations and
presumptions, even such a at least strangely outdated test is welcome.
 
The tested version was CLAM AntiVirus v:0.54 - the current stable
release is 0.82. Please tell me that you didn´t build your opinion on
such an outdated result.The ability to handle macro-viruses was first
implemented in 0.70. Somehow this is getting more and more ridiculous.

The headline of this test was: AntiVirus Scanner Tests July 2004
The 0.70 was released in march 2004. The 0.65 in november 2003.

But i guess as long as it supports the halftruths, speculations and
presumptions, even such a at least strangely outdated test is welcome.
Click to expand...

Perhaps you can show us a comparative that will clear up all our
misconceptions then :) Until then, past test results _do not_
constitute half truths, speculations, and presumptions. They are the
_facts_ that we have to go on. And until then, it seems the
proponents of CLAM are the ones dealing in half truths, speculations
and presumptions.


http://home.epix.net/~artnpeg
 
Christoph Cordes a écrit :
The tested version was CLAM AntiVirus v:0.54 - the current stable
release is 0.82. Please tell me that you didn´t build your opinion on
such an outdated result.The ability to handle macro-viruses was first
implemented in 0.70. Somehow this is getting more and more ridiculous.
Click to expand...

Take a look at this:
ftp://agn-www.informatik.uni-hamburg.de/pub/texts/tests/pc-av/2004-07/a2scanls.txt

and then this:
ftp://agn-www.informatik.uni-hamburg.de/pub/texts/tests/pc-av/2004-07/0xecsum.txt

where it reads:
"VTC test '2004-07' was started in June 2003, with tesbeds frozen as
known on December 31, 2003 and products submitted in February 2003 (test
start was delayed as HEUREKA-3 test was completed only in May 2003)."


It looks like the other products were equally outdated (due to time
constraints: you don't do a huge test like this on a rainy sunday
afternoon) and no product got a special treatment.

If no product got a special treatment, then their respective
performances are certainly comparable, and Clam's performance is, well,
abysmal.
 
Frederic Bonroy said:
If no product got a special treatment, then their respective
performances are certainly comparable, and Clam's performance is, well,
abysmal.
Click to expand...

Sorry for being offensive, but in my opinion you are actually rather
limited in your ability to interpret facts. And this is indeed abysmal.
 
Tomasz Kojm a écrit :
Sorry for being offensive, but in my opinion you are actually rather
limited in your ability to interpret facts. And this is indeed abysmal.
Click to expand...

You are actually rather limited in your ability to back up your arguments.
 
The tested version was CLAM AntiVirus v:0.54 - the current stable
release is 0.82. Please tell me that you didnït build your opinion
on such an outdated result. The ability to handle macro-viruses was
first implemented in 0.70. Somehow this is getting more and more
ridiculous.
Click to expand...

AV apps' reputations are built on their testing track records, whether
you find it ridiculous or not. ClamAv has a short and very bad one;
perhaps that will change.
The headline of this test was: AntiVirus Scanner Tests July 2004
The 0.70 was released in march 2004. The 0.65 in november 2003.
Click to expand...

There should be newer results from Hamburg's VTC soon now, and they
should include the newly macro-aware ClamAv.

"The next comparative test will evaluate file, macro (VBA/VBA5) and
script virus and malware detection. This test (started in July 2004,
with testbeds frozen on April 30, 2004 and products submitted mid-June
2004) will be published about February 2005 (results for macro and
script viruses/malware) and summer 2005 (file virus/malware)."
But i guess as long as it supports the halftruths, speculations
and presumptions, even such a at least strangely outdated test is
welcome.
Click to expand...

I see nothing strange (other than my earlier typo) about the dates of
the test.
 
Back
Top