Is Itanium the first 64-bit casualty?

[Nick Maclaren]

|> |>
|> > File sizes and physical or virtual addressability are not related. Its a
|> > rare app that *needs* to have a whole file mapped into the adress space,
|> and
|> > if it does then the app isn't intended to handle large files (i.e. more
|> than
|> > tens of megs). It would be folly for a video-stream editor to have to fit
|> > the file into memory.
|>
|> You may very well need to if you're doing a combine and decode in a news
|> reader. Not sure how specific news readers do this but I could see OE for
|> example keeping the entire file in memory until it's fully decoded.

Only if the protocol or application are seriously misdesigned.
Competently designed streaming systems need very little memory.
And video is an inherently streaming data type!

This is precisely where PAE scores, because the file can be kept
in fast memory, and the application just slides a window over it.
So a 'read' is merely a question of fiddling the page tables.
Clean, simple and fast.


Regards,
Nick Maclaren.

 

[Dan Pop]

Here's a list of comments I received in response to suggesting using
Linux :

"It's too expensive !"
"It's unsupported !"
"There's no hardware support !"
"People don't understand it !"
"Customers won't like it !" (Customers wanted gear that worked
appliance style 24x7, no UI)

Seem most of them myself, when budget cuts forced the migration from
RISC systems to Intel PCs. OTOH, the users soon realized that they
can't get their work done on NT and started asking for Linux, loud
enough that the suits had to comply.

Dan

 

[Dan Pop]

I never heard of anyone referring to an Apollo or a Sun as a
Desktop, always a Workstation.

At my previous job, all workstations with a single user, sitting on or
near his desk, were called desktops, just as the PCs and Macs. The ones
in the machine room were called servers, even if they were identical with
those in the first category. The definition was based on how the machine
was used and not on some political/religious issues.

Dan

 

[Dan Pop]

I think it's old enough and dead enough and has had enough money thrown
at it to be called legacy though.

The usual definition of "legacy", in context, is: "obsolete, but we have
nothing to replace it with".

Dan

 

[Yousuf Khan]

That would be very strange as it is supported (using 64 bit PCI
addressing) on older Xeons. But stranger things have happened...
hmm... switch CPU to 32 bit mode, do your 64 bit DMA, then switch
back? Hehe...

I don't think it would need to be switched to 32-bit just to do DMA, more
likely, they will prevent the devices from writing beyond the 4GB barrier.
And if they need to write beyond that barrier, then the OS would have to
take care of repaging the lower 4GB into the upper addresses.

Yousuf Khan

 

[Yousuf Khan]

File sizes and physical or virtual addressability are not related.
Its a rare app that *needs* to have a whole file mapped into the
adress space, and if it does then the app isn't intended to handle
large files (i.e. more than tens of megs). It would be folly for a
video-stream editor to have to fit the file into memory.

They probably did it to get higher performance access to their database
files, but likely they were never envisioning that these databases would get
that large.

Yousuf Khan

 

[Peter Dickerson]

You may very well need to if you're doing a combine and decode in a news
reader. Not sure how specific news readers do this but I could see OE for
example keeping the entire file in memory until it's fully decoded.

Carlo

Indeed MS might do that but it certainly isn't necessary or particularly
sensible. MS Win NT was designed with 64-bit file sizes in mind so I doubt
they would then restrict such apps to creating 32-bit sized files.

Peter

 

[Sander Vesik]

Many news readers are running into the file size limitation when downloading
from binary newsgroups.

What does file size have to do with 32 vs 64bit? The OS I run on my desktop
has been supporting file sizes in excess of 4GB since at least 1994 when I
switched to it, *including* on plain vanilla x86 hardware.

 

[CJT]

You may very well need to if you're doing a combine and decode in a news
reader. Not sure how specific news readers do this but I could see OE for
example keeping the entire file in memory until it's fully decoded.

Carlo

Define "need to." Is your definition specific to OE?

 

[Alexander Grigoriev]

I'd be very surprized. Actually, Windows NT and its children are designed
from the beginning to support 64-bit physical address for DMA.

You don't have to switch CPU to another mode, to do 64-bit DMA. It's
physical address, not virtual.

 

[Yousuf Khan]

What does file size have to do with 32 vs 64bit? The OS I run on my
desktop has been supporting file sizes in excess of 4GB since at
least 1994 when I switched to it, *including* on plain vanilla x86
hardware.

Usually file size and addressability are related when you're using memory
mapped i/o. The advantage of memmaps is that after you've finished the
initial setup of the call, you no longer have to make any more OS calls to
get further pieces of the file, they are just demand-paged in just like
virtual memory. Actually, not _just like_ virtual memory, it _is_ actual
virtual memory. Saves many stages of context switching in between this way.

Many operating systems, such as Solaris, are actually using memmaps natively
even within the OS kernel. The OS standard file i/o calls are all wrappers
around memmaps. When an application uses standard file i/o, it's still
ending up using memmaps, except the OS does the job of memmapping for you;
so an application employing memmaps directly saves a few intermediate steps.

Of course for a memmap to work you have to native support for a large
address space. You can do this either by combining two 32-bit registers
together to form a single 64-bit pointer, or you can have a true 64-bit
register.

Yousuf Khan

 

[Yousuf Khan]

At my previous job, all workstations with a single user, sitting on or
near his desk, were called desktops, just as the PCs and Macs. The
ones
in the machine room were called servers, even if they were identical
with
those in the first category. The definition was based on how the
machine
was used and not on some political/religious issues.

I've always looked at a "workstation" as simply a very expensive "desktop".


Yousuf Khan

 

[Sander Vesik]

Usually file size and addressability are related when you're using memory
mapped i/o. The advantage of memmaps is that after you've finished the

no. They are still not related and you can use any size of files you want.
the *only* relation adressing has to this is the size of teh windows you can
use.

initial setup of the call, you no longer have to make any more OS calls to
get further pieces of the file, they are just demand-paged in just like
virtual memory. Actually, not _just like_ virtual memory, it _is_ actual
virtual memory. Saves many stages of context switching in between this way.

With all the drawbacks of that aswell, however.

Many operating systems, such as Solaris, are actually using memmaps natively
even within the OS kernel. The OS standard file i/o calls are all wrappers
around memmaps. When an application uses standard file i/o, it's still
ending up using memmaps, except the OS does the job of memmapping for you;
so an application employing memmaps directly saves a few intermediate steps.

Of course for a memmap to work you have to native support for a large
address space. You can do this either by combining two 32-bit registers
together to form a single 64-bit pointer, or you can have a true 64-bit
register.

*not* pointer. File offsets are not pointers. BigInts are very easy to handle
and it has basicly almost no extra overhead.

 

[Stephen Sprunk]

I don't think it would need to be switched to 32-bit just to do DMA, more
likely, they will prevent the devices from writing beyond the 4GB barrier.
And if they need to write beyond that barrier, then the OS would have to
take care of repaging the lower 4GB into the upper addresses.

This technique has been used for a decade to handle other types of DMA which
must be in the first 1MB or 16MB of memory; extending the model to handle
32-bit PCI DMA only in the first 4GB of memory would be minor.

Linux has had "bounce buffers" to do these sorts of things for a long time,
if not since the beginning. I recall seeing PCI bounce buffers in the
source to support earlier 64-bit machines such as a the Alpha. Little work
should be required to extend this to amd64, and presumably it's already been
done.

IIRC, DMA writes to physical memory addresses and not logical or linear
addresses, so you can't play tricks with the paging system to get around
this. Either use some trickery to allocate DMA buffers in low memory, or
employ bounce buffers when the specified buffer isn't in low memory itself.

S

 

[Stephen Sprunk]

I'd be very surprized. Actually, Windows NT and its children are designed
from the beginning to support 64-bit physical address for DMA.

You don't have to switch CPU to another mode, to do 64-bit DMA. It's
physical address, not virtual.

Ah, but only 64-bit PCI cards can understand 64-bit physical addresses.
It's quite possible (and in time, likely) to allocate a buffer above the
32-bit limit and drivers for 32-bit PCI cards will need some trickery to get
the data to/from the right place.

S

 

[Robin KAY]

Stephen Sprunk wrote:
[snip]

IIRC, DMA writes to physical memory addresses

[snip]

DMA does but DVMA doesn't. Sun, and presumably all the other RISC
vendors', systems have IOMMUs which translate bus addresses to physical
addresses for DMA transfers. The AMD K8 has an on-chip IOMMU.

 

[George Macdonald]

This technique has been used for a decade to handle other types of DMA which
must be in the first 1MB or 16MB of memory; extending the model to handle
32-bit PCI DMA only in the first 4GB of memory would be minor.

Yeah it was used in DOS Extenders and we were all glad (ecstatic ?) when it
was retired for "doing it right".

Linux has had "bounce buffers" to do these sorts of things for a long time,
if not since the beginning. I recall seeing PCI bounce buffers in the
source to support earlier 64-bit machines such as a the Alpha. Little work
should be required to extend this to amd64, and presumably it's already been
done.

Extend to AMD64?? It's a hack... which AMD64 doesn't need.

IIRC, DMA writes to physical memory addresses and not logical or linear
addresses, so you can't play tricks with the paging system to get around
this. Either use some trickery to allocate DMA buffers in low memory, or
employ bounce buffers when the specified buffer isn't in low memory itself.

Depends on the device - the AGP aperture allows precisely that.

Rgds, George Macdonald

"Just because they're paranoid doesn't mean you're not psychotic" - Who, me??

 

[Stephen Sprunk]

Yeah it was used in DOS Extenders and we were all glad (ecstatic ?) when it
was retired for "doing it right".

And what, pray tell, is "doing it right"?

Extend to AMD64?? It's a hack... which AMD64 doesn't need.

The hack already exists for 16-bit ISA DMA, and there's no obvious way to
remove it or it would have been done by now. Likewise, there's no obvious
way to remove bounce bufferring for 32-bit PCI DMA. The Linux folks may be
religious wackos, but they're not dumb.

If you have a better solution then, as they say, "send patches."

S

 

[daytripper]

Ah, but only 64-bit PCI cards can understand 64-bit physical addresses.
It's quite possible (and in time, likely) to allocate a buffer above the
32-bit limit and drivers for 32-bit PCI cards will need some trickery to get
the data to/from the right place.

Someone obviously never heard about 32b PCI's 64b "Dual Address Cycle"...

 

[Bengt Larsson]

In comp.arch, Rupert Pigott

Not my call. Reminds me a little of the thread about some Intel dude
calling SPARC "proprietry".

Actually, there are only two systems-vendors for SPARC systems (Sun
and Fujitsu) but more for Itanium systems (IBM, HP, Dell, SGI,
NEC...). Raw processors is not what is bought by most people. So the
argument isn't totally stupid.

In theory and legally you can clone SPARC, build Solaris systems to
compete directly with Sun. In practice, forget it. Sun has too much
momentum (Fujitsu has managed by building bigger systems than Sun).

"In Theory, there is no difference between Theory and Practice, in
Practice, there is."