Unattend over network with USB boot device

[Gerry Hickman]

Hi,

Bored with floppies, and with some machines no longer having them by
default, I decided to make a USB boot device that can load up real-mode
Win98, connect to the network and run WINNT.EXE from there.

However, there's a major problem! The boot device ends up being drive C: I
don't know of any way to change it. When I try to run WINNT from the network
(even with the /t switch), it trashes the boot sector of the USB device and
copies $WIN_NT$.~BT to it along with various other files such as $LDR$ and
NTLDR.

If I unplug the USB device just before running WINNT, setup fails saying
"can't access C drive press 'r' to retry".

It seems this goes back to the old problem of how DOS assigns drive letters.
The USB device is running under "Emulation" and therefore DOS sees it as the
first fixed drive on the first controller! Arrrrh.

 

[Johan Arwidmark]

If you have a Software Assurance (SA) or Enterprrise Agreement (EA)
with Microsoft you can use Windows PE for free as a platform for
deployment instead of a boot disk.

Windows PE (which is a "lite" Windows XP version) can be booted from
CD/DVD/HDD or over the Network using RIS. Windows PE supports USB
devices and gives you more flexibility regarding preparing for the
setup.

regards
Johan Arwidmark

Windows User Group - Nordic
http://www.wug-nordic.net

 

[Patrick J. LoPresti]

Hi, Gerry. I have been planning to play with this, too, but my USB
stick is on back-order. Great minds think alike .

Here are some thoughts.

DOS should only see the drive as C: if it recognizes the partition
type. So one option is to use a file system which DOS does not
recognize. For example, you might try booting DOS 6 from a FAT32
partition.

DOS will need a way to find its own files, of course. But if you use
SYSLINUX+memdisk to boot a virtual floppy, this will be no problem.

My favorite option, and the one I intend to try, is to boot a diskless
Linux system and run winnt.exe under dosemu. Actually, this is now
supported and working in the current release of
<http://unattended.sourceforge.net/>. We provide ISO-9660 and PXE
images, but they are identical except for the boot loader (ISOLINUX
vs. PXELINUX, respectively). The exact same images should boot fine
from a USB stick using SYSLINUX. Once the diskless Linux system is
booted, you are home free.

There are several advantages to this approach:

1) You can partition the drive without rebooting.

2) The dosemu environment, being a virtual machine, is entirely
under your control. So you can make any disk be the "C: drive".

3) The emulated environment accesses the network and disk via Linux,
which is faster than DOS (even with the emulation). I no longer
use smartdrv when installing XP!

4) Using Linux to install Windows is just cool, in a perverse sort
of way.

If you are interested, let me know and I can spell this out in more
detail. It is actually pretty simple...

- Pat

 

[Gerry Hickman]

Hi Patrick,

Excellent ideas!

I can see Linux is way ahead when working the O/S at this level. You are
also right about partitions; at present I have ExtendOEMPartition which
is great for the C: drive, but I have to do the others manually after
Windows is installed. About 3 minutes work per box.

I don't understand booting DOS from a FAT32 partition though? How will
it see the file system upon which it resides? I could format the memory
stick to FAT32, use a DOS floppy with the SYS command to copy the boot
sector across, but then when I try to boot... how will DOS "see" itself?
Let's assume it can "see" itself, surely we're back to the same problem
- the memory stick will become the C drive...

Apparently there's a secret DOS command called "Assign" that can be used
to change drive letter in DOS, but it was pulled from more recent
distributions - does anyone have this file or know more about it?

 

[Patrick J. LoPresti]

For the other partitions, I just script commands like these to run
after OS installation:

echo y | format /v:whatever1 /q /x /fs:ntfs d:
echo y | format /v:whatever2 /q /x /fs:ntfs e:

....and so on.

My suggestion is to boot a "virtual floppy" with DOS from the memory
stick. The SYSLINUX boot loader comes with a generic tool (memdisk)
which hooks the BIOS to let you boot from an in-memory virtual floppy
derived from whatever floppy disk image you want.

So the procedure would be to create a 1.44M (or 2.88M, or even larger)
bootable floppy.img file with DOS and whatever tools you want. Then
you copy both memdisk and floppy.img to the USB stick, install
SYSLINUX as the boot loader, and you are done.

Incidentally, this is how my DOS-based boot disk works. I use the
same floppy images for physical floppy boot, CD-ROM boot (via
ISOLINUX+memdisk), and PXE boot (via PXELINUX+memdisk). The same
trick should work for USB boot, as long as the USB stick does not use
a partition type which DOS recognizes.

(Incidentally, SYSLINUX has nothing to do with Linux. It is actually
just an excellent general-purpose boot loader. See
<http://syslinux.zytor.com/>.)

As for "assign", that is interesting; I had not heard of it. You
might try playing around with FreeDOS, since it has both an "assign"
and something called "swsubst", either of which might be relevant:

http://www.freedos.org/freedos/software/lsm.cgi?q=d&a=base

I can confirm that the current release of FreeDOS is capable of
running winnt.exe.

But I still think the diskless Linux approach is cooler .

- Pat
http://unattended.sourceforge.net/

 

[Gerry Hickman]

Hi Pat

For the other partitions, I just script commands like these to run
after OS installation:

echo y | format /v:whatever1 /q /x /fs:ntfs d:
echo y | format /v:whatever2 /q /x /fs:ntfs e:

I thought you had to make "partitions" before you could format them?
let's say your drive is 36Gb and you want to end up with 3 NTFS
partitions of 12Gb each. The first is easy; 2Gb FAT16 gets extended to
12Gb by unattend.txt, but how can you get the other two - the ones for
the D drive and E drive? Also how can I force the CD-ROM to aquire drive
F at build time?

 

[Gerry Hickman]

So the procedure would be to create a 1.44M (or 2.88M, or even larger)
bootable floppy.img file with DOS and whatever tools you want. Then
you copy both memdisk and floppy.img to the USB stick, install
SYSLINUX as the boot loader, and you are done.

I should have said "how can I do it with Microsoft only tools". The docs
for my boot environments are too big as it is without adding new tools.
There must be a simple way to install text mode windows onto something
other than a C drive? As soon as setup does the first reboot, the D
drive becomes the C drive anyway (no USB this time). In a way, it would
be even better if the USB booter stick could be seen as the A drive.

The 'winnt /t:tempdrive' switch only sets the "temp" drive, it still
puts NTLDR and the $WIN_NT$.~BT stuff onto the C drive, arrhhhh!

 

[Patrick J. LoPresti]

Ah, for that I use a slight hack. I do not know whether it is
documented, but it works on Win2k, WinXP, and Server 2003 at least.

First, create three NTFS partitions of 12G each.

Next, delete the first partition.

Finally, create a 2G FAT16 partition in its place and mark it
"active". (Actually, I recently started using a 4G FAT32 partition to
get less fragmentation. But either will work.)

That's it. ExtendOEMPartition will grow the 2G partition as far as it
can without clobbering the others, so you wind up with three 12G
partitions.

Under DOS, I use FreeDOS fdisk because it is fully scriptable with
command-line switches. But this approach should work fine with
Microsoft-only tools as well.

This will give you disk partitions C:, D:, and E:, with your CD-ROM on
F:. I do not know how to change these assignments off-hand (the need
never came up), but I thought I read once that it was possible in
Win2k.

- Pat
http://unattended.sourceforge.net/

 

[Gerry Hickman]

Hi Patrick,

Your method makes a LOT of sense. I'd never tried it because I thought

"how can I type in the exact number of bytes to get my first partition
to extend all the way up to the next without overwriting it?"

Are you saying I should just enter a number that's higher than the space
available in the first slot? That makes everything much easier!

I don't quite understand where you say about using a FAT32 partition?
Surely setup is going to convert to NTFS and then extend to 12Gb, so
what difference does it make whether it's FAT16 or FAT32? It's only
going to be FAT for about 10 minutes!

Regarding FreeDOS, if it can do scriptable partitions, it's sounds VERY
nice! One reason I've avoided non-MS partition utilities is that I ran
into a major problem where FDISK refused to delete some partitions when
I was rebuilding a PC. I'm pretty sure Norton Ghost had been used to
build the machine originally, and FDISK didn't like it. I had to use
Partition Magic to delete the old partitions in the end.

 

[Patrick J. LoPresti]

Are you saying I should just enter a number that's higher than the
space available in the first slot? That makes everything much easier!

I just use "ExtendOemPartition=1", which extends it as far as
possible. But I suspect any large number would do as well.

I don't quite understand where you say about using a FAT32 partition?
Surely setup is going to convert to NTFS and then extend to 12Gb, so
what difference does it make whether it's FAT16 or FAT32? It's only
going to be FAT for about 10 minutes!

A 2G FAT16 partition uses a 32K cluster size. In other words, every
file actually occupies a multiple of 32K bytes on the disk.

After NTFS conversion, the cluster size will typically be much smaller
(like 512 or 4K bytes), so almost every file will suddenly have some
extra free space just after it. Multiply that by several hundred (or
thousand?) files in the OS install, and that adds up to a lot of
fragmented free space. And it is not easy to defragment since so many
of those files are system files.

The post-NTFS conversion cluster size is a somewhat subtle issue. It
will be 512 bytes unless the FAT clusters are "aligned":

http://www.microsoft.com/whdc/hwdev/tech/storage/ntfs-preinstall.mspx

(Search for "oformat".)

So if you use "oformat /A:8", you will end up with a 4K cluster size
after NTFS conversion, which is the cluster size recommended by
Microsoft. The only catch I found is that oformat refuses to run on
anything earlier than DOS 8 (Windows ME).

I brought this up on the freedos-devel list, and the FreeDOS
developers responded by adding a "/A" switch to the current version of
FreeDOS format.exe. So now I use FreeDOS format.exe and I get 4K
clusters after NTFS conversion.

Now, using FAT32, a 2G partition has a 4K cluster size. Since this is
the same cluster size as after NTFS conversion, there will be no
fragmentation.

Oh, I just remembered... I think this only applies to Windows XP and
Server 2003. With Windows 2000, the post-conversion cluster size is
always 512 bytes. So much for staying on-topic.

Regarding FreeDOS, if it can do scriptable partitions, it's sounds
VERY nice! One reason I've avoided non-MS partition utilities is
that I ran into a major problem where FDISK refused to delete some
partitions when I was rebuilding a PC.

I have never had a problem using FreeDOS fdisk. I usually blow away
the entire existing partition table ("fdisk /clear 1"), then create a
4000M FAT32 partition ("fdisk /pri:4000"), then activate it ("fdisk
/activate:1").

But more interesting schemes are possible. For example, to create a
12G C: drive, a 5G D: drive, and use the "rest of the disk" for E:,
I use:

fdisk /clear 1 (clear partition table)
fdisk /pri:12288 (create 12G FAT32 partition)
fdisk /pri:5120 /spec:7 (create 5G NTFS partition)
fdisk /pri:100,100 /spec:7 (use 100% of remainder for NTFS partition)
fdisk /delete /pri:1 (delete first partition)
fdisk /pri:4000 (re-create first partition as 4G FAT32)
fdisk /activate:1 (activate it)

Incidentally, all of this is in <http://unattended.sourceforge.net/>
if you dig deep enough. I suggest checking it out sometime .

- Pat

 

[Gerry Hickman]

I just use "ExtendOemPartition=1", which extends it as far as
possible. But I suspect any large number would do as well.

Thanks, I thought the number after the "=" had to be in megabytes; this
is useful to know.

After NTFS conversion, the cluster size will typically be much smaller
(like 512 or 4K bytes), so almost every file will suddenly have some
extra free space just after it.

Eeek, this is all totally new to me! I've saved the whole section so I
can read it properly tomorrow.

But more interesting schemes are possible. For example, to create a
12G C: drive, a 5G D: drive, and use the "rest of the disk" for E:,
I use:

[Snip FreeDOS's scriptable FDISK]

Incredible! This is way ahead of Microsoft's DOS.

Incidentally, all of this is in <http://unattended.sourceforge.net/>
if you dig deep enough. I suggest checking it out sometime .

I check it out all the time, but I don't remember seeing ANY of this!
Not the FreeDOS stuff, nor the cluster size details...

I spoke to Dell about the USB and C: drive issue, and they basically
blame the way Microsoft's DOS assigns drive letters. I'm not sure this
is correct though. Surely the BIOS should emulate the USB device as a
"removable" device, not a fixed disk?

Anyway, you have convinced me that I've got to learn SysLinux, FreeDOS
and so on.

 

[Patrick J. LoPresti]

Thanks, I thought the number after the "=" had to be in megabytes;
this is useful to know.

Yeah, normally it is. But "1" is a special case, apparently.

I spoke to Dell about the USB and C: drive issue, and they basically
blame the way Microsoft's DOS assigns drive letters. I'm not sure
this is correct though. Surely the BIOS should emulate the USB
device as a "removable" device, not a fixed disk?

Well, if it were treated as a "removable" device, you would have to
boot from it like a CD-ROM. That is, it would have to be structured
like a bootable CD-ROM, rather than a hard disk. So you would have to
use CD-ROM-like tools to create it, and those probably expect to
"burn" data rather than copying... Basically, you would end up with a
somewhat confusing mess.

When you boot from a hard disk, the BIOS will always make that the
primary disk. I believe this is fundamental to how BIOSes have always
worked.

I suppose the BIOS could treat the USB stick as a giant floppy. Hm.
Aren't there USB floppy devices? Maybe you could trick your BIOS into
thinking your USB stick was one of those?

I wish I had my own USB drive to play with. Maybe I will cancel the
backordered one and buy from somebody else.

Anyway, you have convinced me that I've got to learn SysLinux,
FreeDOS and so on.

Well, FreeDOS is just a clone of MS-DOS, with some additional features
and bugs. You should feel right at home .

SYSLINUX is just a really nice (open source) boot loader. It comes
with lots of files, but that is mostly source code; I only end up
using a tiny number (three or four) of the binaries. The SYSLINUX
author is very smart and responsive, if a little brusque...

- Pat

 

[Gerry Hickman]

Yeah, normally it is. But "1" is a special case, apparently.

Turns out this is in unattend.doc, it says a value of "1" will "extend
to fill the drive". I never really extrapolated this to mean it could
also be used to extend all the way up to the next partition.

Well, if it were treated as a "removable" device, you would have to
boot from it like a CD-ROM.

There's been bootable ZIP drives for years, I wonder if they were alwsys
"C:" too? As you also state, why not see it as an A drive; maybe there's
a limitation in terms of heads, sectors tracks.

primary disk. I believe this is fundamental to how BIOSes have always
worked.

Are you saying the drive letter assignment is BIOS related, as opposed
to O/S related? I was under the impression BIOS's knew nothing about
drive letter assignments. I thought they just saw things as "devices"
and it was the O/S that assigned the letters?

I suppose the BIOS could treat the USB stick as a giant floppy. Hm.
Aren't there USB floppy devices? Maybe you could trick your BIOS into
thinking your USB stick was one of those?

I don't know if USB floppies exist, but there's certainly no options in
the BIOS in relation to this.

I wish I had my own USB drive to play with. Maybe I will cancel the
backordered one and buy from somebody else.

One thing to bear in mind. There's no point getting a big expensive
memory stick for this, you'll only be putting a few Kb of files on it. I
think you can pretty much walk into any shop and buy one nowadays.

 

[Patrick J. LoPresti]

There's been bootable ZIP drives for years, I wonder if they were
alwsys "C:" too?

Good question.

Are you saying the drive letter assignment is BIOS related, as
opposed to O/S related? I was under the impression BIOS's knew
nothing about drive letter assignments. I thought they just saw
things as "devices" and it was the O/S that assigned the letters?

Well, yes.

But DOS has a special relationship with the BIOS, since in effect it
uses the BIOS as its HAL. Every interaction DOS has with disks, from
enumerating them to reading/writing them, happens through the BIOS.

I believe the BIOS will always treat the boot device as "disk 0"
(aka. "disk 80h in the INT13 interface"). So when DOS enumerates the
disks attached to the system, the boot drive will always appear first
on the list. And if the drive has an active FAT partition, that
partition will be C:.

Consequently, if you change the BIOS boot order, you change the C:
drive. But you knew that already .

I don't know if USB floppies exist, but there's certainly no options
in the BIOS in relation to this.

It was a long shot.

One thing to bear in mind. There's no point getting a big expensive
memory stick for this, you'll only be putting a few Kb of files on
it. I think you can pretty much walk into any shop and buy one
nowadays.

I am considering it. The trouble is that most electronics store I
visit do not want to sell anything for less than $30-40 or so. And
for $50 I can get a 256M USB 2.0 stick (mail order) which should last
me for a long time. Except I have to wait for it.

But yeah, maybe I will just buy one locally anyway. I do want to play
with this stuff.

- Pat

 

[Gerry Hickman]

But DOS has a special relationship with the BIOS, since in effect it
uses the BIOS as its HAL. Every interaction DOS has with disks, from
enumerating them to reading/writing them, happens through the BIOS.

I believe the BIOS will always treat the boot device as "disk 0"

Ah, I see what you mean.

(aka. "disk 80h in the INT13 interface"). So when DOS enumerates the
disks attached to the system, the boot drive will always appear first
on the list. And if the drive has an active FAT partition, that
partition will be C:.

Maybe we could write an ASM program to hack the drive letters? Another
option is to simply copy everything from the memory stick to the real C
drive, then reboot. It could then load network drivers, and fire up
WINNT. Unfortunately (even if it worked) it would leave some nasty
legacy files (IO.SYS, COMMAND.COM, network drivers) sitting on the boot
drive.

Here's some info on the DOS ASSIGN command.

http://www.easydos.com/assign.html

for $50 I can get a 256M USB 2.0 stick (mail order) which should last
me for a long time. Except I have to wait for it.

True, but you really need a dedicated stick JUST for booting. I've got
two, small one for booting, and a 256Mb for storing stuff.

 

[Patrick J. LoPresti]

DOS will need a way to find its own files, of course. But if you use
SYSLINUX+memdisk to boot a virtual floppy, this will be no problem.

My favorite option, and the one I intend to try, is to boot a diskless
Linux system and run winnt.exe under dosemu. Actually, this is now
supported and working in the current release of
<http://unattended.sourceforge.net/>. We provide ISO-9660 and PXE
images, but they are identical except for the boot loader (ISOLINUX
vs. PXELINUX, respectively). The exact same images should boot fine
from a USB stick using SYSLINUX. Once the diskless Linux system is
booted, you are home free.

For the record, my USB stick finally arrived and I got this working
today. Basic process is as I described:

1) Put a standard boot sector in the MBR of the USB drive. (I used
FreeDOS).

2) Download SYSLINUX (http://syslinux.zytor.com/), install it on the
boot sector of the main partition of the USB drive, and make sure
the partition is "active".

3) Download unattended-4.0b.zip and unattended-4.0b-linuxboot.zip
from http://unattended.sourceforge.net/
[Well, I did not actually do this, since I already have the
original copy ]

4) Copy unattended-4.0b/linuxboot/tftpboot/bzImage and .../initrd to
the USB stick. Copy
unattended-4.0b/linuxboot/tftpboot/pxelinux.cfg/default to the
USB stick and rename it "syslinux.cfg".

5) Enable "USB emulation" in the BIOS (Dell Precision 360).

6) Plug in the USB memory key, press F12 while booting, and select
the USB stick as the boot device.

From there, installing Windows went off without a hitch.

And you are right, it is kind of cool. Booting from the network is so
TC. (*)

But network boots are still faster. And booting from USB drive is
only supported by new BIOSes. Even my (1 year old) Dell Latitude C640
cannot do it. (I have read that the "D" series Latitudes can.)

Still, it is neat to see it working at all.

- Pat

(*) "Twentieth Century"