A method of reducing on-demand scanning times

[David W. Hodgins]

As a matter of general interest, I'd like to hear of examples of
malware that have actually been successful in avoiding "recent
modiification date" heuristics. I think others would as well.

Win32.Jeefo.A http://vic.zonelabs.com/tmpl/body/CA/virusDetails.jsp?VId=35519
although it appears to only infect 2k+

http://www.123renamer.com/allspyware/keylogger.htm
which, apparently only backdates it's log files.

See the section on Methods to avoid detection in
http://encyclopedia.laborlawtalk.com/computer_virus#Methods_to_avoid_detection

Given that it's in most definitions of viruses, It makes searching for
specific examples, rather difficult.

No doubt most users should do routine complete scans, just in case.
I'm going to remove the DOS av scanner - batch file aspect from my web
site.

I think what you've developed is a usefull tool. I just think it should
come with an appropriate warning, that relying on the file dates, may
cause some malware to be missed. As long as the end-user understands
that, I think anything that encourages them to run scans, is a good thing.

Regards, Dave Hodgins

 

[Zvi Netiv]

On Wed, 13 Apr 2005 05:40:13 -0300, "Norman L. DeForest"



<pardon my snippage as well>

Very interesting, Norman. I haven't used 4DOS at all, though I once
downloaded, installed it, and played with it a little when it went
free. Right now I don't have it on my machines.

Your loss!

Haven't used anything else for the DOS shell since the late eighties. Although
4DOS will work almost the same under W2K / XP as for Win4x/Me, except few unique
file/directory services, there is also 4NT (same make) that caters for these
newer OS.

To the attention of the command line freaks. ;-)

Regards, Zvi

 

[null]

Win32.Jeefo.A http://vic.zonelabs.com/tmpl/body/CA/virusDetails.jsp?VId=35519
although it appears to only infect 2k+

Good example though of a PE infector that apparently retains the
original modification date and time on infected files. If it's spread
via unpatched internet app vulnerabilities (a "out of the blue" rather
than purposely executed sort of whammy) it's a very good example even
though it may be Win 2K+ only.

http://www.123renamer.com/allspyware/keylogger.htm
which, apparently only backdates it's log files.
Ok.

See the section on Methods to avoid detection in
http://encyclopedia.laborlawtalk.com/computer_virus#Methods_to_avoid_detection

Given that it's in most definitions of viruses, It makes searching for
specific examples, rather difficult.

Don't waste time on it. I just thought some here might have examples
in mind, more or less off the top of their heads. It's been awhile
since I've spent time reading descriptions and following the virus
scene, and I just don't remember ever seeing a description of a virus
or Trojan that uses date-time obfuscation.

I think what you've developed is a usefull tool. I just think it should
come with an appropriate warning, that relying on the file dates, may
cause some malware to be missed. As long as the end-user understands
that, I think anything that encourages them to run scans, is a good thing.

Thanks. I too (obviously) think there is value in recent date
heuristics for saving time in some cases. I can imagine a repair tech
who understands the issues finding my DOS av scanner idea useful.
He/she might prefer to take the (good) chance that a date filtered
quick scan will nail the culprit(s) almost immediately.

As long as the tech doesn't abandon doing full unfiltered scans before
the PC leaves the shop ....

Art

http://home.epix.net/~artnpeg

 

[Roger Wilco]

Why wouldn't it be on the list? Wouldn't a newly created file be on the
list? If something changes the date - you have other issues, no?

OK, I see now it was my 'old school' mindset - people no longer think of
AV scanners as a preventative measure but more like toilet paper.

 

[kurt wismer]

(e-mail address removed) wrote: [snip]

Wrong. That was a entirely different thing. F-RECENT doesn't store
anything. It finds files recently written to disk.

not to put too fine a point on it but that's still got the same
problem... the data you're using to decide what's new or not (and
therefore what to scan) can be easily forged/modified/whatever...

As a matter of general interest, I'd like to hear of examples of
malware that have actually been successful in avoiding "recent
modiification date" heuristics. I think others would as well.

No doubt most users should do routine complete scans, just in case.

I'm going to remove the DOS av scanner - batch file aspect from my web
site.

see, the problem with this reasoning is that as soon as a lot of people
start using this method to reduce their on-demand scanning times the
malware writers will start writing malware to exploit that very problem...

this method can only be successful so long as it remains obscure...
security through obscurity...

 

[kurt wismer]

If the black hats go to that much trouble to obfuscate dates, the
white hats can counter by analyzing erased files. Then the black hats
will be forced to do a defrag as well

yes, *OR* the white hats could say "y'know, it seems to me this is just
a kind of change detection, and we already *have* good methods for
change detection" and not do anything to fix this problem because this
was the wrong way to do change detection in the first place...

 

[null]

yes, *OR* the white hats could say "y'know, it seems to me this is just
a kind of change detection, and we already *have* good methods for
change detection" and not do anything to fix this problem because this
was the wrong way to do change detection in the first place...

You seem to have missed my smiley. I wasn't being serious about that
insane scenario at all

Art

http://home.epix.net/~artnpeg

 

[null]

(e-mail address removed) wrote: [snip]
Wrong. That was a entirely different thing. F-RECENT doesn't store
anything. It finds files recently written to disk.

not to put too fine a point on it but that's still got the same
problem... the data you're using to decide what's new or not (and
therefore what to scan) can be easily forged/modified/whatever...

As a matter of general interest, I'd like to hear of examples of
malware that have actually been successful in avoiding "recent
modiification date" heuristics. I think others would as well.

No doubt most users should do routine complete scans, just in case.

I'm going to remove the DOS av scanner - batch file aspect from my web
site.

see, the problem with this reasoning is that as soon as a lot of people
start using this method to reduce their on-demand scanning times the
malware writers will start writing malware to exploit that very problem...

this method can only be successful so long as it remains obscure...
security through obscurity...

I think the implementaion of my idea is inherently obscure since
typical users aren't about to use DOS av scanners. And my
F-RECENT util only works with FAT 16 and 32 drives.

I'm mainly concerned that the method would be ab-used. Like my
scenario where repair techs begin to "forget" to do complete scans
before a PC leaves the shop ... for one example.

Art

http://home.epix.net/~artnpeg

 

[Norman L. DeForest]

Date: Wed, 13 Apr 2005 17:06:36 GMT
From: (e-mail address removed)
Newsgroups: alt.comp.anti-virus, alt.comp.virus
Subject: Re: A method of reducing on-demand scanning times



Thanks. I too (obviously) think there is value in recent date
heuristics for saving time in some cases. I can imagine a repair tech
who understands the issues finding my DOS av scanner idea useful.
He/she might prefer to take the (good) chance that a date filtered
quick scan will nail the culprit(s) almost immediately.

As long as the tech doesn't abandon doing full unfiltered scans before
the PC leaves the shop ....

One example comes to mind of where finding new files would be useful.
Someone tells me, "I need a free program that does $foo." I search the
Internet and find a couple of dozen programs that claim to do $foo and
download them. Since I don't trust anything, I then want to avoid running
any of them until they are scanned at least a week after downloading (so,
if they use some new exploit there's time for the new virus definition
files to include them). A week later I want to scan all of the new files.
I can ask the antivirus program to scan everything but that could take
hours. Or I could ask it to just scan the newer files -- even if that has
to be done one file at a time from a batch file. To do that, the newer
files have to be identified first.

That's where a utility (or a 4DOS alias) to find newer files comes in
handy.

 

[null]

One example comes to mind of where finding new files would be useful.
Someone tells me, "I need a free program that does $foo." I search the
Internet and find a couple of dozen programs that claim to do $foo and
download them. Since I don't trust anything, I then want to avoid running
any of them until they are scanned at least a week after downloading (so,
if they use some new exploit there's time for the new virus definition
files to include them). A week later I want to scan all of the new files.
I can ask the antivirus program to scan everything but that could take
hours. Or I could ask it to just scan the newer files -- even if that has
to be done one file at a time from a batch file. To do that, the newer
files have to be identified first.

That's where a utility (or a 4DOS alias) to find newer files comes in
handy.

I don't understand why you wouldn't simply have your on-demand
scanner(s) scan all the files in your download folder without
subdirectory recursion???? They will presumably be still sitting there
in their unwrapped (unzipped) state, right? KAVDOS32 in particular
is quite good at scanning all files within SFXs and various archives
.... and displaying on screen what it's doing ... and unpacking many
kinds of runtime packers and multiply packed files by different
packers

On that point, on-demand scanning can be tricky for the unwary. Some
scanners will only appear to have scanned "within" SFXs, and they
annoyingly produce a "ok" message on the .EXE file that they've not
scanned "within" at all. They do the same damn thing with runtime
packers they can't handle.

I sometimes try to unzip SFXs. And if I can't, and it's apparent that
KAV can't get at the archived files, I simply delete the SFX and say
to hell with it. Archives that can't be scanned on-demand are too
risky to fool around with. KAV is quite reliable since it is honest
enough to let you know when it runs into a "unkown" runtime packer
(which is quite rare).

Well, I've digressed enough for one post

Art

http://home.epix.net/~artnpeg

 

[Norman L. DeForest]

On Thu, 14 Apr 2005 (e-mail address removed) wrote:
[snip]

I don't understand why you wouldn't simply have your on-demand
scanner(s) scan all the files in your download folder without
subdirectory recursion???? They will presumably be still sitting there
in their unwrapped (unzipped) state, right? KAVDOS32 in particular
is quite good at scanning all files within SFXs and various archives
... and displaying on screen what it's doing ... and unpacking many
kinds of runtime packers and multiply packed files by different
packers

I have to move stuff out of my download directory as soon as possible for
a couple of reasons:

1. After several days, I can forget which file was downloaded for which
purpose unless I move the files immediately to a dedicated directory
just for that category. That means that I can have files to scan in
several directories, some of which I have forgotten.

2. I have problems with my command interpreter (4DOS) when I try to run
a command while in a directory with too many files. Windows apparently
tries to switch to that directory and sort the files and runs out of
memory[1]. It then has no room to allocate to 4DOS to save a path and
search another directory so none of my utilities can be accessed.
Executing a CD command to switch to a less populated directory appears
to fix things but makes it harder to access stuff in the crowded
directory. I'm currently pushing things with 963 items (files and
directories) in my G:\TXDN\ directory and it's due for another purge,
moving stuff to other directories, when I can find the time.

[1] I ran across a web site some time ago that claimed that Windows 98
can't properly de-allocate memory if a 16-bit application is running.
I got kicked off the 'net (a momentary power failure) before I could
save or bookmark that page and I haven't been able to find it since.

 

[null]

I have to move stuff out of my download directory as soon as possible for
a couple of reasons:

1. After several days, I can forget which file was downloaded for which
purpose unless I move the files immediately to a dedicated directory
just for that category. That means that I can have files to scan in
several directories, some of which I have forgotten.

Now you're hitting on the reason I created F-RECENT. I find myself
using Charles Dye's Locate.com quite often for such purposes, but not
often enough to remember its rather complex set of command line switch
options. I wanted a dedicated recent files finder util with the
simplest possible command line switch option and a "useage" message.

2. I have problems with my command interpreter (4DOS) when I try to run
a command while in a directory with too many files. Windows apparently
tries to switch to that directory and sort the files and runs out of
memory[1]. It then has no room to allocate to 4DOS to save a path and
search another directory so none of my utilities can be accessed.
Executing a CD command to switch to a less populated directory appears
to fix things but makes it harder to access stuff in the crowded
directory. I'm currently pushing things with 963 items (files and
directories) in my G:\TXDN\ directory and it's due for another purge,
moving stuff to other directories, when I can find the time.

[1] I ran across a web site some time ago that claimed that Windows 98
can't properly de-allocate memory if a 16-bit application is running.
I got kicked off the 'net (a momentary power failure) before I could
save or bookmark that page and I haven't been able to find it since.

I Googled a bit using word, phrase and sentence combos including
"dealloctate memory" "memory leak" "DOS" " Windows 98" and didn't hit
on it.

Art

http://home.epix.net/~artnpeg

 

[kurt wismer]

(e-mail address removed) wrote: [snip]

If the black hats go to that much trouble to obfuscate dates, the
white hats can counter by analyzing erased files. Then the black hats
will be forced to do a defrag as well

yes, *OR* the white hats could say "y'know, it seems to me this is just
a kind of change detection, and we already *have* good methods for
change detection" and not do anything to fix this problem because this
was the wrong way to do change detection in the first place...

You seem to have missed my smiley. I wasn't being serious about that
insane scenario at all

i generally don't acknowledge smilies (as iolo said years ago, you
don't need them to tell when something is funny, and if it's not funny
then they don't help), and my response is applicable even if you had
stopped at "If the black hats go to that much trouble to obfuscate
dates"... your technique is vulnerable to date manipulation exploits...
if they're ever used the white hats will very likely recognize that
checking dates is just a change detection technique, and there really
are more reliable methods for doing change detection...

 

[null]

(e-mail address removed) wrote: [snip]
If the black hats go to that much trouble to obfuscate dates, the
white hats can counter by analyzing erased files. Then the black hats
will be forced to do a defrag as well

yes, *OR* the white hats could say "y'know, it seems to me this is just
a kind of change detection, and we already *have* good methods for
change detection" and not do anything to fix this problem because this
was the wrong way to do change detection in the first place...

You seem to have missed my smiley. I wasn't being serious about that
insane scenario at all

i generally don't acknowledge smilies (as iolo said years ago, you
don't need them to tell when something is funny, and if it's not funny
then they don't help),

Unfortunately on usenet, humor, especially a subtle and weird sense of
humor like I have, doesn't get communicated. I suggest that you
lighten up, Kurt. You must have a sense of humor, no?

and my response is applicable even if you had
stopped at "If the black hats go to that much trouble to obfuscate
dates"... your technique is vulnerable to date manipulation exploits...
if they're ever used the white hats will very likely recognize that
checking dates is just a change detection technique, and there really
are more reliable methods for doing change detection...

No need to beat a dead horse. Let me explain how my idea came about.

As I've posted several times in the past, I don't have any malware or
spyware problems. For years now, I've been watching my hard drives
wear out while doing full on-demand scans before backing up to my
cloned backup drives. I've had the "is this trip necessary?" feeling
for a long time, and I've often tried to think of alternatives.

I once mentioned an earlier idea, which was to activate KAV's realtime
scanner while the process of copying new or changed files ( XXCOPY
only copies files that don't exist on the destination or which have
changed in file size or date ) is going on. My test using eicar.com
was successful .... KAV immediately blocked copying and asked me what
I wanted to do .... which is exactly what I wanted _it_ to do.

After designing F-RECENT (for reasons mostly unrelated to malware),
it was natural that I'd consider adding the shortcut mentioned in my
OP on the topic. The batch engages two top notch scanners so I
get at least one more good "second opinion" on files with recent
modification dates. Imperfect, yes, but in my case I'm actually
_adding_ to the scanning that I'd otherwise do.

I still scan with Spybot and AdAware in addition to using several
generic Trojan "sniffer" methods during my weekly maintenance
routine. But the days of wearing out my drives doing full av scans
weekly are long gone. I sometimes at random intervals use KAV on
demand to do a scan of only Windows subdirectories .... just out of
curiosity.

Safe hex and my approach of hardening the OS so I don't require
realtime av or frewall/router has proven to work "perfectly" over the
years. I'd like to do away completely with scanners, but as soon as I
do, some vulnerability in the browsers I use will be exploited. I
don't see any way around the use of scanners before backing up.

And my wife is now using IE on her PC more and more frequently.
I jusr yesterday put KAV realtime on her Startup list and showed her
how to udate it. It's kinda fun keeping an eye on her situation and
maintaining both of our machines.

Art

http://home.epix.net/~artnpeg

 

[kurt wismer]

Unfortunately on usenet, humor, especially a subtle and weird sense of
humor like I have, doesn't get communicated. I suggest that you
lighten up, Kurt. You must have a sense of humor, no?

some people think so... it doesn't change the fact that smilies are a
crutch...

No need to beat a dead horse. Let me explain how my idea came about.

As I've posted several times in the past, I don't have any malware or
spyware problems. For years now, I've been watching my hard drives
wear out while doing full on-demand scans before backing up to my
cloned backup drives. I've had the "is this trip necessary?" feeling
for a long time, and I've often tried to think of alternatives.

uh huh... has it ever occurred to you that the drive clones were
themselves unnecessary? you put a lot of work into making sure they're
clean and up to date, but the only thing they gain you over more
conventional backups is convenience during restoration - and if you
don't have reason to perform frequent restorations that can be a lost
economy... if you do have reason to perform frequent restorations then
possibly your perimeter defenses should be examined...

Safe hex and my approach of hardening the OS so I don't require
realtime av or frewall/router has proven to work "perfectly" over the
years. I'd like to do away completely with scanners, but as soon as I
do, some vulnerability in the browsers I use will be exploited. I
don't see any way around the use of scanners before backing up.

what amazes me is that you so much to back up each week that it's even
a big deal... i definitely do not product that much valuable locally
stored content, nor do i acquire much from the net (certainly not
anything i can't download again anyways)...

 

[null]

some people think so... it doesn't change the fact that smilies are a
crutch...

Fact are unarguable. I'm not going waste time debating your opinion.

uh huh... has it ever occurred to you that the drive clones were
themselves unnecessary? you put a lot of work into making sure they're
clean and up to date, but the only thing they gain you over more
conventional backups is convenience during restoration - and if you
don't have reason to perform frequent restorations that can be a lost
economy... if you do have reason to perform frequent restorations then
possibly your perimeter defenses should be examined...

Cloning remains my preferred method of backup. It goes very quickly
since only files that require copying are copied. It takes no longer
than cumulative backup and has the advantage that I'm not
accummulating files on the backup drive I don't want saved.

To say cloning is unnecessary is as silly as saying cumulative backups
are unnecessary. People do need to understand the difference .... that
files not on the source are deleted from the destination. So the
cloning type of backup isn't something to recommend to typical users
who may confuse it with cumulative backup and wind up losing
valuable data.

what amazes me is that you so much to back up each week that it's even
a big deal... i definitely do not product that much valuable locally
stored content, nor do i acquire much from the net (certainly not
anything i can't download again anyways)...

Again, it all goes very quickly when done fairly often. I like the
idea that when one of our main hard drives fail, I won't have to go
through all the settings and tweaks and downloads of critical patches,
etc. The ability to Restore all our apps, Windows, and data just as
they were would be a huge time and work saver. The last thing I want
to have to do is reinstall Windows and all our apps and have to set
them all up again from scratch. That would be a lot of work that I
most definitely would not enjoy! I spent several days recently doing
all that on a new PC I purchased. Even though the OS was installed, it
took me quite awhile to get everything the way I want.

I probably spend about a half hour per week per machine (maybe a bit
more) doing the whole maintenance thing, including registry cleaning
and defrag. I think that small investment in time and effort is well
worth it, since we both enjoy trouble-free machines.

Art

http://home.epix.net/~artnpeg

 

[cquirke (MVP Windows shell/user)]

On backup theory; restoring a backup seeks to:
- lose all unwanted changes
- retain all wanted changes

How do you scope between wanted and unwanted?

Cloning remains my preferred method of backup.

Cloning is a backup scoped purely by time. As malware often lie
dormant before discovery and/or payload, IMO the role of a "pure
system backup" as malware steamroller is limited at best; it's an
excellent baseline for immediate problems, though.

It goes very quickly since only files that require copying are
copied. It takes no longer than cumulative backup and has
the advantage that I'm not accummulating files on the backup
drive I don't want saved.

That doesn't sound like what I understand by "cloning"

To say cloning is unnecessary is as silly as saying cumulative backups
are unnecessary. People do need to understand the difference .... that
files not on the source are deleted from the destination. So the
cloning type of backup isn't something to recommend to typical users
who may confuse it with cumulative backup and wind up losing
valuable data.

Good point; it's also breaks the classic "son over father" rule.

When done with media solid enough to be sure you won't be left with no
son and no father, the "son over father" rule doesn't apply as much,
i.e. I'd roll those dice when using a HD as the meduim, but I wouldn't
if I were using CDRW or DVDDRW. There, I'd preserve the fater and
save the son over the grandfather instead (i.e. 2 x disk sets)

---------- ----- ---- --- -- - - - -

Gone to bloggery: http://cquirke.blogspot.com

 

[Doug]

Begs the point....what do you do with something like the Kelvir worm?......
masks itself and hides so that Norton and McAffee report false directory
locations.
....courtesy of one of it's dll's......and under xp....it copies itself
everywhere.
If you didn't get rid of the restores....and do the safe mode...it's always
with you.

&D

 

[null]

On backup theory; restoring a backup seeks to:
- lose all unwanted changes
- retain all wanted changes

How do you scope between wanted and unwanted?

No, the question is how avoid wearing out your hard drive(s)
with on-demand scanning. I've mentioned a couple of methods
I use in this thread.

Cloning is a backup scoped purely by time. As malware often lie
dormant before discovery and/or payload, IMO the role of a "pure
system backup" as malware steamroller is limited at best; it's an
excellent baseline for immediate problems, though.

As I've posted, I use a system backup in case of drive failure.
I used it to Restore Windows just once when I screwed it up
with my messing around. I'm far more dangerous than any
malware

That doesn't sound like what I understand by "cloning"

Then you must not have used the command line disk cloners
like XXCOPY which only copy new or modified files. Takes me
but a minute or two to backup in this way. Of course, the initial
cloning takes awhile. But updating time is insignificant.

Art

http://home.epix.net/~artnpeg

 

[James Egan]

Then you must not have used the command line disk cloners
like XXCOPY which only copy new or modified files. Takes me
but a minute or two to backup in this way. Of course, the initial
cloning takes awhile. But updating time is insignificant.

Okay, so there's a /clone switch in xxcopy. Very inaccurate
terminology.

Take a look at robocopy. Much more in line with your Scrooge price
range where /mir (mirror) is close to what you describe. You can get
it for nowt as part of ms win2k3 resource kit tools.

http://www.ss64.com/nt/robocopy.html
http://www.microsoft.com/downloads/...69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en


Jim.