Remapped sectors: Data security

[Ludwig]

Is a defective sector securely erased before it is remapped by the hdd
OS? Or, to pinpoint it: can a hdd, which is "securely" erased by a
erasing tool like blancco, kroll ontrack dataeraser etc. still contain
sensitive data in the remapped sectors?

How can I access (read & write) remapped sectors either on scsi and on
(e)ide hdds to check them?

Any contributions are appreciated.

Ludwig

 

[Al Dykes]

Is a defective sector securely erased before it is remapped by the hdd
OS? Or, to pinpoint it: can a hdd, which is "securely" erased by a
erasing tool like blancco, kroll ontrack dataeraser etc. still contain
sensitive data in the remapped sectors?

How can I access (read & write) remapped sectors either on scsi and on
(e)ide hdds to check them?

Any contributions are appreciated.

Ludwig

I've seen a discussion of computer forensics were this is discussed.
It seems that at the very least you need special drivers, and I'm not
sure it's possible to read remapped sectors on all disk models. It's
easier on the sever-grade SCSI disks. There may be some linux software
around to try this. (linux is being used in the forensics world.)

If you run an encrypted file system (optional in w2k/XP/Linux) the
blocks could be read but would have to be decrypted to be interesting.

 

[Ron Reaugh]

Is a defective sector securely erased before it is remapped by the hdd
OS?

Assume no.

Or, to pinpoint it: can a hdd, which is "securely" erased by a
erasing tool like blancco, kroll ontrack dataeraser etc. still contain
sensitive data in the remapped sectors?
Yes.

How can I access (read & write) remapped sectors either on scsi and on
(e)ide hdds to check them?

Get an 'in' with the HD mfg or have access to national technical resources.

Any contributions are appreciated.

It's a hole that isn't easily plugged. Generally HDs should be physically
shredded or burned if the data is that sensitive.

 

[Arno Wagner]

I've seen a discussion of computer forensics were this is discussed.
It seems that at the very least you need special drivers, and I'm not
sure it's possible to read remapped sectors on all disk models.

I am pretty sure it is possible, but it may require vendor-specific
commands that might not be documented anywere public. I would not be
surprised if this is something data-recovery companies do routinely.
However this is definitely not something a generic standard tool
can do.

It's easier on the sever-grade SCSI disks.

You can find SCSI command information here:

http://www.t10.org/scsi-3.htm

There may be some linux software
around to try this. (linux is being used in the forensics world.)

If you run an encrypted file system (optional in w2k/XP/Linux) the
blocks could be read but would have to be decrypted to be interesting.

That depends. With the usual cipher modes, you can read up to the last
ciper block before the defect (if yo have the key). From the defect on
you can only decrypt if you know where the defect is, even if you have
the key. Of course a single-bit error will not be a problem for
that. But, e.g. an error with 32 bit uncertainity will add that much
effort to decryption.

In an extreme case, all-or-nothing encryption could be used for
sector encryption. Then you need to guess all defects correctly
to get any infromation from an encrypted disk block. Fowever as
far as I am aware this is not done today, as decrypting e.g. AES
without the key is regarded as infeasible for all practical
purposes.

Arno

 

[Ludwig]

Assume no.


Get an 'in' with the HD mfg or have access to national technical resources.

Do the manufacturers' low level formatting tools re-read already
remapped sectors when running again, in other words: could "bad guys"
find out access routines to remapped sectors by disassembling those
tools (wddiag, seatools etc.)?

Ludwig

 

[Ludwig]

I've seen a discussion of computer forensics were this is discussed.
It seems that at the very least you need special drivers, and I'm not
sure it's possible to read remapped sectors on all disk models. It's
easier on the sever-grade SCSI disks. There may be some linux software
around to try this. (linux is being used in the forensics world.)

Any links and src?

If you run an encrypted file system (optional in w2k/XP/Linux) the
blocks could be read but would have to be decrypted to be interesting.

Could be only true, if the encryption algorithm "stretches" over
multiple sectors. If encryption is sector-wise, this wouldn't prevent
decryption 100%

Ludwig

 

[Al Dykes]

I am pretty sure it is possible, but it may require vendor-specific
commands that might not be documented anywere public. I would not be
surprised if this is something data-recovery companies do routinely.
However this is definitely not something a generic standard tool
can do.

I imagine it may actually require flashing new code to the disk, and
if a mass-market IDE disk doesn't have this cabapility you can't read
the bad blocks. server grade disks have flashable code.

The people that have the capability to do this would not spend the
resources on a fishing expedition; the'd have you already identified
you as a probable bad guy. If this is the case you've got lots of
other things to worry about.

 

[Arno Wagner]

I imagine it may actually require flashing new code to the disk, and
if a mass-market IDE disk doesn't have this cabapility you can't read
the bad blocks. server grade disks have flashable code.

I thought today the HDD firmware is actually stored on disk and loaded
on start-up?

The people that have the capability to do this would not spend the
resources on a fishing expedition; the'd have you already identified
you as a probable bad guy. If this is the case you've got lots of
other things to worry about.

Exactly. Like keyboard sniffers TEMPEST attacks, break-ins to
clone/steal the disk, bribes, etc.. In addition, even when secret
information is on a disk, most/almost all individual 512 byte
blocks will still be pretty uninteresting or completely meaningless,
which reduces the possible return-on-investment of this attack
massively, even when only HDDs are targetted that were used for
sensitive infromation.

As a result, most practical applications need not care about
reallocated defective sectors. But still people should be aware
of the mechanism and its implications. In the rare case where
it could be a problem, conventional erasure should be followed
by physical destruction.

As a further remark, encrypted disks do protect against
problems from reallocated defective sectors, but their
primary use is the protection against HDD theft.

Arno

 

[Arno Wagner]

Do the manufacturers' low level formatting tools re-read already
remapped sectors when running again, in other words: could "bad guys"
find out access routines to remapped sectors by disassembling those
tools (wddiag, seatools etc.)?

I doubt it. The tools from Seagate and Maxtor I used in the
recent past just execute a short or long SMART self-test.
The actual test is done by the HDD firmware and not the tool.

In addition, re-certifying defective sectors in a HDD is a bad idea,
since the original problem will likely not have gone away. It is
different for optical storage with defect management (e.g. MOD,
DVD-RAM, Mt.Rainier): There a lot of defects are just temporary
failures, e.g. from dust. But for this reason SCSI (and thereby
ATAPI, since it uses the SCSI command set) has a seperate class
of commands for optical storage, which includes surface
recertification.

Arno

 

[Ludwig]

Exactly. Like keyboard sniffers TEMPEST attacks, break-ins to
clone/steal the disk, bribes, etc.. In addition, even when secret
information is on a disk, most/almost all individual 512 byte
blocks will still be pretty uninteresting or completely meaningless,

I disagree. Obviously those sectors which are heavily used have a
higher probability to get bad. Such sectors usually do not contain
static data (like program code, static media files etc.) but data
generated from user input, from intermediary data. So, these are
sectors which contain "valuable" information with a much higher
probability than all other (static) sectors on a hdd.

As a result, most practical applications need not care about
reallocated defective sectors. But still people should be aware
of the mechanism and its implications. In the rare case where
it could be a problem, conventional erasure should be followed
by physical destruction.

In our daily practice as a refurbishing and remarketing company of
used pcs we are regularily confronted with our clients (those
companies, which wants to remarket their used equipment) concerns,
that ALL data on the hdds will be deleted. If there is no guarantee,
that ALL data will be securely erased, then the hdd would be
physically destroyed for security reasons. These would have massive
environmental and economical impacts, because a pc without a hdd has
almost no value and has to be physically recycled instead on being
used by other (mostly poor) people (more and more in developping
countries like Africa etc). The commercial second hand pc market could
rather break down.

To clarify the problem of remapped sectors we contacted as the
IASG/CESG as different manufacturers of CESG certified "secure"
erasing software tools. From the first we got no senseful answer, from
the others we didn't get any answer. So, we rather doubt, that
remapped sectors are erased by commercial tools.

What could be the solution?

Ludwig

 

[Al Dykes]

I disagree. Obviously those sectors which are heavily used have a
higher probability to get bad. Such sectors usually do not contain
static data (like program code, static media files etc.) but data
generated from user input, from intermediary data. So, these are
sectors which contain "valuable" information with a much higher
probability than all other (static) sectors on a hdd.

An utterly unsupportable generalization. Support this claim. Sectors
go "soft" (require ECC calculation to recover correct data) even it
they were only written one, years ago.

Also, a sector in the badblock has no idea what file it was part of,
before it went "bad". If you do for a read of a block that's been
mapped-out is just 4096 bits with no context, that probably have
failed the ECC calculation, so you can't know that ABCDEF isn't really
ABCDEG when you read it.

In our daily practice as a refurbishing and remarketing company of
used pcs we are regularily confronted with our clients (those
companies, which wants to remarket their used equipment) concerns,
that ALL data on the hdds will be deleted. If there is no guarantee,
that ALL data will be securely erased, then the hdd would be
physically destroyed for security reasons. These would have massive
environmental and economical impacts, because a pc without a hdd has
almost no value and has to be physically recycled instead on being
used by other (mostly poor) people (more and more in developping
countries like Africa etc). The commercial second hand pc market could
rather break down.

Perfect security (or any security) doesn't come cheap. You can't have
it both ways. Security generally comes down to a cost/risk decision.
I've been in jobs where we routinely smashed IDE disks from desktop
systems (with a sledge hamer on a concrete floor) because doing
_anything_ else to sanitize the disk cost more that scraping the disk.

Sorry about not being able to donate equipment. If the company
making the donation values their data as much as you do the'll
appreciate that smashing HDDs is a cost of doing business.

To clarify the problem of remapped sectors we contacted as the
IASG/CESG as different manufacturers of CESG certified "secure"
erasing software tools. From the first we got no senseful answer, from
the others we didn't get any answer. So, we rather doubt, that
remapped sectors are erased by commercial tools.

What could be the solution?

There may not be one that meets your requirements.

 

[Folkert Rienstra]

I disagree. Obviously those sectors which are heavily used have a
higher probability to get bad. Such sectors usually do not contain
static data (like program code, static media files etc.) but data
generated from user input, from intermediary data.

So, these are sectors which contain "valuable" information with a
much higher probability than all other (static) sectors on a hdd.

Yes, but you can't get to them unless you acquire special firmware.
Even when Low Level Formatting may get the sectors back in use
they will be overwritten when doing so.

 

[Folkert Rienstra]

I've seen a discussion of computer forensics were this is discussed.
It seems that at the very least you need special drivers, and I'm not
sure it's possible to read remapped sectors on all disk models.

 

[Arno Wagner]

An utterly unsupportable generalization. Support this claim. Sectors
go "soft" (require ECC calculation to recover correct data) even it
they were only written one, years ago.

Also, a sector in the badblock has no idea what file it was part of,
before it went "bad". If you do for a read of a block that's been
mapped-out is just 4096 bits with no context, that probably have
failed the ECC calculation, so you can't know that ABCDEF isn't really
ABCDEG when you read it.

Actuslly HDD have 512 byte sector size and do not care about
filesystem blocks at all. Defect management is therefore done
in 512 byte units, giving even less data in the remapped sector.

Well, people have a tendency to make stupid demands if they do not
understand the technology they are talking about. Remapped sectors
are never a security risk in practice. HDDs with contents so sensitive
that remapped sectors could be a problem will not be given away but
be destroyed in-house.

Perfect security (or any security) doesn't come cheap.

In fact perfect security is unavailable at this time. However
good security is a lot cheaper than marginally better security
and does the job as well in practice.

You can't have
it both ways. Security generally comes down to a cost/risk decision.
I've been in jobs where we routinely smashed IDE disks from desktop
systems (with a sledge hamer on a concrete floor) because doing
_anything_ else to sanitize the disk cost more that scraping the disk.

Sorry about not being able to donate equipment. If the company
making the donation values their data as much as you do the'll
appreciate that smashing HDDs is a cost of doing business.

There may not be one that meets your requirements.

Looks like it.

Arno

 

[Al Dykes]

Actuslly HDD have 512 byte sector size and do not care about
filesystem blocks at all. Defect management is therefore done
in 512 byte units, giving even less data in the remapped sector.

The last time I checked 4096 bits = 512 bytes.

Well, people have a tendency to make stupid demands if they do not
understand the technology they are talking about. Remapped sectors
are never a security risk in practice. HDDs with contents so sensitive
that remapped sectors could be a problem will not be given away but
be destroyed in-house.


In fact perfect security is unavailable at this time. However
good security is a lot cheaper than marginally better security
and does the job as well in practice.

The US DoD specs I've seen provide for erasure by multiple passes of a
commercial erasure software package for most types of secure data, but
require destruction for the highest security level. I've always
assumed that the advantage of physical destruction was that your boss,
or the security officer could watch the disk going into the shredder.
It's not as easy to be assured that the software is really working.
It seems that degaussing is obsolete because the magnet would have to
be powerfull enough to pull your belt buckle off, and it would erase
sync data imbedded in the tracks that the electronics need to
function, effectively making it impossible to reformat the disk.

 

[Arno Wagner]

[...]

Actuslly HDD have 512 byte sector size and do not care about
filesystem blocks at all. Defect management is therefore done
in 512 byte units, giving even less data in the remapped sector.

The last time I checked 4096 bits = 512 bytes.

Oops, sorry. I was reading too hastily ;-)

[...]

The US DoD specs I've seen provide for erasure by multiple passes of a
commercial erasure software package for most types of secure data, but
require destruction for the highest security level. I've always
assumed that the advantage of physical destruction was that your boss,
or the security officer could watch the disk going into the shredder.

Yes, that is a serious advantage. An (external) tech could easily fake
a normal erasure and then make off with the data on the disk. (More
advanced model: "I am now overwriting your disk with random data"
= "I am now encryting the disk, you will not be able to tell
the difference, but I can get all the data...")

Or if erasure is done off-site, people could lie about it, in an
extreme case just to save time.

Arno

 

[Folkert Rienstra]

Previously Al Dykes <[email protected]> wrote:

[...]

An utterly unsupportable generalization. Support this claim. Sectors
go "soft" (require ECC calculation to recover correct data) even it
they were only written one, years ago.

Also, a sector in the badblock has no idea what file it was part of,
before it went "bad". If you do for a read of a block that's been
mapped-out is just 4096 bits with no context, that probably have
failed the ECC calculation, so you can't know that ABCDEF isn't really
ABCDEG when you read it.

Actually, when that happens you don't get to see that data.
And using other means (reading bits) you probably will read more than
4096 bits anyway, having to find out which are data and which are not.
But then, when you are technically able to do that, you'll probably also
know "what's what".

Oops, sorry. I was reading too hastily ;-)

Nonsense. It's just your eagerness to shoot your mouth off.
Anyone else would reread that sentence more clearly and find that
it actually says something else than it appears to on first glance.
If it says anything at all, that is, as it is very poor english to begin with.

[snip]

 

[mzjunk]

Is a defective sector securely erased before it is remapped by the hdd
OS? Or, to pinpoint it: can a hdd, which is "securely" erased by a
erasing tool like blancco, kroll ontrack dataeraser etc. still contain
sensitive data in the remapped sectors?

How can I access (read & write) remapped sectors either on scsi and on
(e)ide hdds to check them?

Any contributions are appreciated.

Ludwig

Hello Ludwig,

Blancco allready has a working solution. Their software also erases
remapped sectors.
http://www.blancco.com

MZ

 

[Arno Wagner]

Hello Ludwig,

Blancco allready has a working solution. Their software also erases
remapped sectors.
http://www.blancco.com

And how do they claim they do that, since there is no way for the user
to verify their claim?

Arno

 

[mzilenas]

people trust them , that's it.