Re: doscall1

  • Thread starter Thread starter Jonathan de Boyne Pollard
  • Start date Start date
J

Jonathan de Boyne Pollard

maybe he can successfully shed some more light on this subject.

Here are some more basics, just so that we're all on the same page:

* In theory one can have multiple I/O APICs in a system. In practice,
on the sorts of systems we're talking about here, there's only going to
be one. It lives in the PCI-to-ISA bridge part of what is variously
called an "I/O Controller Hub" (ICH) or a "PCI ISA IDE Xcelerator"
(PIIX) by Intel or a "South Bridge" by VIA.

* The difference between MPS (Intel's "Multi-Processor Specification")
version 1.1 tables and version 1.4 tables is that the version 1.4 tables
contain various extensions, dealing with things like multiple I/O APICs.
Again, this will be largely irrelevant here.

* For most, perhaps all (certainly all that I have datasheets for),
chipsets, there is a control bit of some form in configuration space
that enables or disables the I/O APIC. If there are no local APICs
enabled on the other end of the APIC bus, however, it doesn't really
matter what state the I/O APICs are in. Nothing will be listening to
their messages. Indeed, in a more extreme case, the APICCLK signal may
be simply tied to ground, and no I/O APIC messages will happen on the
APIC bus at all, because its clock is frozen.

* There are two ways to turn off a local APIC: the hard way and the easy
way. Both involve setting flags in CPU registers. If a local APIC is
turned off the hard way, it cannot be turned back on again without
potentially losing interrupts and confusing the entire APIC bus
arbitration scheme. A local APIC turned off the easy way can be re-enabled.

* There are two ways for firmware to report I/O APIC configuration
information to an operating system: the MPS table, and ACPI tables.
They aren't quite the same. The configuration information reported
states among other things which ISA devices and which PCI IRQ lines are
connected to which I/O APIC inputs. There's no requirement, after all,
that every motherboard manufacturer connect the ISA IRQ #7 signal to I/O
APIC INTIN pin #7.

* In addition to deciding how to program the enable control bits for the
I/O APIC and all of the local APICs, firmware also gets to decide what
it reports in the MPS and ACPI tables. It might decide to lie, for
example, and deny the existence of I/O APICs or local APICs in the
machine. The idea of this would be to force an operating system, whose
primary source of this hardware information is supposed to be the MPS
and ACPI tables, to still "see" an ACPI system, but one that doesn't
have APICs; thereby forcing it to fall back to whatever dual-8259 mode
of operation it has, whilst still retaining other unrelated
ACPI-provided information such as (say) system reset and environment
control capabilities.

* An OS/2 Platform-Specific Driver (PSD) is not a Windows NT Hardware
Abstraction Layer (HAL). A HAL abstracts away quite a lot of the
details of interrupt processing and low-level inter-processor
synchronization and communication. A PSD does not. There are two
particulars of note. First: The OS/2 kernel has fallback code that
knows how to talk to dual 8259s, should a PSD not implement certain
optional capabilities, and operate what is essentially an *asymmetric*
multiprocessor system. Second: An OS/2 PSD has no responsibility for
implementing spinlocks. So a system where the PSD omits the optional
features does not devolve to being identical to a uniprocessor OS/2 system.

* The idea that I/O APICs increase the number of available interrupts is
a bit of a swizz. The number of PIRQ signal lines on the PCI bus
doesn't magically change. Some internal devices, built in to the
southbridge, gain the ability to use extra interrupt signals that don't
exist on the real PCI bus (On some VIA southbridges, for example, the
internal PCI-to-ATA bridge gets to use PCI INT #E.). But, really,
stating that the point of an I/O APIC is to "gain more interrupts" is
mis-selling it. I/O APICs provide better ways to manage and control the
*same* set of PCI and ISA interrupt signals, not more of them.
 
Here are some more basics, just so that we're all on the same page:
Click to expand...


....snip...snip...a ton of absolutely WEALTHY WONDERFUL info...thank you!!!

* An OS/2 Platform-Specific Driver (PSD) is not a Windows NT Hardware
Abstraction Layer (HAL). A HAL abstracts away quite a lot of the
details of interrupt processing and low-level inter-processor
synchronization and communication. A PSD does not. There are two
particulars of note. First: The OS/2 kernel has fallback code that
knows how to talk to dual 8259s, should a PSD not implement certain
optional capabilities, and operate what is essentially an *asymmetric*
multiprocessor system. Second: An OS/2 PSD has no responsibility for
implementing spinlocks. So a system where the PSD omits the optional
features does not devolve to being identical to a uniprocessor OS/2 system.

* The idea that I/O APICs increase the number of available interrupts is
a bit of a swizz. The number of PIRQ signal lines on the PCI bus
doesn't magically change. Some internal devices, built in to the
Click to expand...


OK...so are there any tools currently in existence on our platform that we could
use to really understand the 'setup' that our BIOS has attempted to implement?

I found this interesting IOAPIC note
(http://www.o3one.org/tutorials/apicarticle.txt), it explains the functionality
in accessible terms.

Now, a question on my part as I've been trying to understand the options for the
OS2APIC.PSD driver...what exactly do the following mean (as best as you can
estimate):

/apic
/pic
/nmi
/prec
....and directly from a reference post:

There are three (3) possible entries that can describe a route:
route to the I/O APIC
route to the Local APIC
route described by argument(s) to the PSD

For a typical single user, multi core system, what are the possible
'combinations' of these options to use/try? I understand that this depends
heavily on the hardware itself, but is there an overall direction one should be
trying to pursue while setting up the system?

Thanks again!
 
Hallo,

since OS2APIC.PSD is pretty old I would think that it exclusively follows
Intel's multiprocessor specification Version 1.4 and completely ignores what
the ACPI multiprocessor table has to offer
(which eventually will replace the MP tables):

http://www.intel.com/design/pentium/datashts/242016.htm

Get that document to gain a better understanding. In particular it has some
nice diagrams
that will tell you more than a thousand words.

Then, switches for OS2APIC.PSD:

/apic: set up the system as described in chapter 3.6.2.3 in said spec
Please note that SMP for Warp Server (this is what we use today if
we have a
multiprocessor system) only routes interrupts to CPU1. OS/2 SMP
version 2.1
used to route interrupts to all CPUs. There were reasons to change
this
with SMP for Warp Server (avoid "parallel" paths in device
drivers).

/pic: set up the system as described in chapter 3.6.2.2 of said spec
Note: yes, the system runs in "virtual wire" mode as a "minimum".
It will never run as defined in chapter 3.6.2.1. That also means
OS2APIC.PSD will only work if you have a CPU with a local APIC.
Of course that is not a problem since the first generation Pentiums
came out.
You will need to run in this mode if your system DOES NOT contain
an IO-APIC.

/nmi: basically defines on what local APIC input pin the NMI signal line is
routed across.
You will only need this if the MP tables on your system are broken
or incomplete
If you look at what the MP tables contain, you will see that the
NMI signal line is
routed via the local APIC just as the interrupt lines are.
See chapter 4.3.5 of said spec. See table 4-11 for the types of
"interrupts" that are
routed across a local APIC.

/prec: as far as I understand if the NMI signal line is routed over more
than one local APIC
input, you can choose with this switch what the preferrable route
is. Again you
will only need this if the MP tables on your system are broken

Don't ask me about MSIs (message signalled interrupts).
Maybe it's the "messages" sent via the ICC-bus that the MP spec talks about
but I am not sure.


If I ever find the time, I could write a utility that dumps the MP tables
content (if you have Linux I would
guess that utility already exists). That would also help to find out if your
MP tables are broken or incomplete
(for example, if the NMI is not routed via any local APIC pin, then NMIs
will obviously not work).

But be aware that the MP tables will become obsolete with the multiprocessor
ACPI table that replaces them.
In other words: if your system no longer contains MP tables, then
OS2APIC.PSD will cease to work as it will
be looking for them.


Lars
 
Jonathan said:
* The idea that I/O APICs increase the number of available interrupts is
a bit of a swizz. The number of PIRQ signal lines on the PCI bus
doesn't magically change. Some internal devices, built in to the
southbridge, gain the ability to use extra interrupt signals that don't
exist on the real PCI bus (On some VIA southbridges, for example, the
internal PCI-to-ATA bridge gets to use PCI INT #E.). But, really,
stating that the point of an I/O APIC is to "gain more interrupts" is
mis-selling it. I/O APICs provide better ways to manage and control the
*same* set of PCI and ISA interrupt signals, not more of them.
Click to expand...

My motherboard manual shows the interrupt assignments. It shows 2 PCI
slots using INT #F and #G and the other one sharing #A IIRC.
Dave
 
My motherboard manual shows the interrupt assignments. It shows 2 PCI
slots using INT #F and #G and the other one sharing #A IIRC.
Click to expand...

That's unusual. What chipset does it have?
 
Jonathan said:
That's unusual. What chipset does it have?
Click to expand...

Northbridge: Intel® 865G
Southbridge: Intel® ICH5

One PCI slot is shared with the on board audio on #B, The USB 2
controller and on board LAN share int #H. Nothing uses #E and no mention
is made of the on board Pata, Sata or what I guess are connected to the
ISA bus, mouse, keyboard, serial and parallel.
Dave
 
That's unusual. What chipset does it have?
Northbridge: Intel® 865G

Southbridge: Intel® ICH5
Click to expand...
Was it you who had me looking at the ICH7? The ICH7 is a little more
versatile, but also a little more restricted in its "mobile" flavour, in
that it only supports four PCI interrupt signals internally rather than
eight.
One PCI slot is shared with the on board audio on #B, The USB 2
controller and on board LAN share int #H. Nothing uses #E and no
mention is made of the on board Pata, Sata or what I guess are
connected to the ISA bus, mouse, keyboard, serial and parallel.
Click to expand...

Your mainboard manual is misleading you somewhat. These aren't actual
PCI slots on your motherboard. All of these are internal devices that
exist on the ICH chip itself, at fixed PCI addresses.

What your mainboard manual should be telling you is how the PIRQ A, B,
C, and D signals from the actual PCI slots on the mainboard are wired up
to the PIRQ A, B, C, and D inputs to the ICH5 chip. This is
mainboard-specific. The idea is that every card uses the PIRQ A signal
for preference, and that a wiring matrix on the mainboard distributes
everyone's PIRQ A amongst PIRQs A, B, C, and D by the time that the
signal reaches the ICH chip. Mainboard manufacturers are free to choose
the matrix, and must lay out a table, the so-called "$PIR" table, in
firmware memory that describes it to software (so that software knows
what PCI interrupt a card that reports "I'm using PIRQ A!" is actually
using). The signals after they have been processed through the wiring
matrix are what the ICH5 treats as PIRQ A, B, C, and D, to which it adds
E, F, G, and H.

According to the ICH5 datasheet, the PCI interrupt signals used by the
built-in devices on the ICH itself are as follows:

PIRQ A: USB UHCI Controller #1, USB UHCI Controller #4
PIRQ B: AC ’97 , MC '97, SMbus
PIRQ C: USB UHCI Controller #3, PCI-to-ATA bridge in native mode
PIRQ D: USB UHCI Controller #2
PIRQ E: LAN
PIRQ F: (nothing)
PIRQ G: (nothing)
PIRQ H: USB EHCI Controller

SCI, TCO, and HPETs #0, #1, and #2 are all configurable between PCI IRQs
E, F, G, and H.

As usual for southbridges, these PCI interrupts are both routed directly
to I/O APIC INTIN signals #16 to #24 and routed indirectly, via a
programmable PCI IRQ router that controls what PCI IRQ maps to what INT
pin, to the (emulated) dual 8259s. As I said before, I/O APICs don't
give one more interrupts, as is so often claimed. As you can see, only
two devices (leaving aside the HPETs et al.) even use PIRQs #E to #H.
What using I/O APICs gives is (amongst other things) separation between
PCI and ISA interrupts, so that there's no risk of an edge-triggered
active-high ISA interrupt having to share a signal with a
level-triggered active-low PCI interrupt. They also free the systems
programmer from all of the PCI IRQ steering hooplah, which would be a
fourth layer of mapping and indirection in a mechanism that already has
three such layers for the systems programmer to deal with.

An i8042 keyboard and mouse, UART, parallel port, infrared port, MIDI
interface, environment controller, and 82078 floppy disc controller will
be connected to a separate "Super I/O" chip that is on the LPC bus. (If
you look at your mainboard you'll see it somewhere. It's usually a
WinBond or ITE chip. I'm interested in knowing which particular one you
have.) As such, they issue ISA interrupts, not PCI ones. The ISA
interrupts are, of course, routed in the traditional way to the
(emulated) dual 8259s, and are routed in a somewhat different
arrangement to the I/O APIC INTIN signals #0 to #15. This routing is
hardwired within the ICH5.

The interrupts issued by a PCI-to-ATA bridge are somewhat special. This
is because PCI-to-ATA bridges have two modes: "ISA compatibility" and
"PCI native". (The former makes a PCI-to-ATA bridge behave like an
ISA-to-ATA bridge. The latter makes it behave more like a proper PCI
device should, with reconfigurable base address registers, PCI INT pin
advertisements, and whatnot.) In ISA compatibility mode, the PCI-to-ATA
bridge always uses two interrupt signals, one per channel, and they are
both ISA interrupt signals: ISA IRQs #14 and #15. In PCI native mode,
the PCI-to-ATA bridge issues *one* interrupt signal, for *both*
channels, and it is a PCI interrupt.

Needless to say: That, coupled with the facts that each individual
channel on the bridge is theoretically individually switchable between
ISA compatibility and PCI native mode and that there's no "I'm the one
interrupting you!" flag in certain I/O transaction modes, makes
interrupt handling and Plug-and-Play resource management for Intel
PCI-to-ATA bridges somewhat ... erm ... interesting. (It's less
interesting for VIA PCI-to-ATA bridges, which have an additional "I'm
the one interrupting!" flag that's valid in all I/O modes. Although they
do add quirks of their own that, in compensation, make Plug-and-Play
resource management *more* interesting: In ISA compatibility mode the
ISA IRQs are not fixed as they are for Intel but configurable among ISA
IRQs #10, #11, #14, and #15. However, this uses the configuration
registers of a *different* PCI device, the PCI-to-ISA bridge. Using
someone else's configuration space isn't really in the spirit of PCI.)
 
I found this interesting IOAPIC note
(http://www.o3one.org/tutorials/apicarticle.txt), it explains the functionality
in accessible terms.
Click to expand...

It gets three things quite wrong, however:

* There is a means of querying the firmware for the mainboard's PCI
interrupt wiring matrix. It's the "$PIR" table.

* Many more than just "usually only [...] multiprocessor boards" have
I/O APICs. This is because an I/O APIC is built in to several
southbridges. A VIA VT8235M(CD) has a built-in I/O APIC, for example.
The mainboard manufacturer may well not have wired it up to the APIC
bus, but it's there.

* The swapping of INTIN #0 and INTIN #2 on the I/O APIC relative to IRQ
#0 and IRQ #2 at the dual 8259s isn't universal. It happens for some
southbridges that have built-in I/O APICs. But for southbridges that
use external I/O APIC chips the wiring is up to the mainboard
manufacturer, who can choose whatever wiring xe likes. How the I/O APIC
is wired up is something that has to be read out of the MPS or ACPI
tables, constructed by the mainboard manufacturer and supplied in
firmware. (I'm sure that M. Rieker discovered this when running xyr
program under Bochs. Bochs emulates one of the chipsets that requires
an external I/O APIC chip. It emulates such a chip being present, but
wires it up *without* swapping INTIN #0 and INTIN #2: a fact that system
software has to read the Bochs firmware-supplied MPS table to discover.)
 
I've been trying to understand the options for the OS2APIC.PSD driver...

Don't go reading the copies of its documentation that exist on the WWW,
such as at the EDM/2 CONFIG.SYS Documentation Project for example. They
all seem to have a single source, that was mis-transcribed from the
original documentation. Where the WWW documents say
The argument is "int" or "lint".
Click to expand...

the original document actually said
The argument is "int<n>" or "lint<n>".
Click to expand...

which of course makes more sense when one reads what follows.

For a better description of OS2APIC.PSD's command-line arguments, see
IBM Technical Document #10322931. It has the advantage of being more up
to date than OS/2 version 2.11, as well. (-:
For a typical single user, multi core system, what are the possible
'combinations' of these options to use/try?
Click to expand...

The main use of the /NMI, /PREC, and /PIC options is to compensate for
faulty MPS tables. You've already seen that your firmware's SETUP
utility has the wrong help text attached to the "IOAPIC function"
option. This is not the only way that a mainboard manufacturer can
stuff up building a firmware image. A manufacturer can stuff up the MPS
tables, too.

It was common enough in years gone by that operating system vendors had
to implement workarounds like this. DOS+Windows 9x users wouldn't
notice that MPS tables were wrong. Not even most OS/2 users would.
Only a few people with Windows NT and other SMP operating systems would.
Now that operating systems that actually care about such things are
pretty much in the majority, the situation is somewhat different
(although the operating systems that care also tend to prefer ACPI
information to MPS).

Of course, if your MPS tables are *not* faulty, and correctly describe
how your system is wired up, you won't need to fiddle with these options
at all.
 
Jonathan said:
Was it you who had me looking at the ICH7? The ICH7 is a little more
versatile, but also a little more restricted in its "mobile" flavour, in
that it only supports four PCI interrupt signals internally rather than
eight.
Click to expand...

Yes, I had a ICH7 based computer that unfortunately died

Your mainboard manual is misleading you somewhat. These aren't actual
PCI slots on your motherboard. All of these are internal devices that
exist on the ICH chip itself, at fixed PCI addresses.
Click to expand...

Well, it claims to have 3 PCI buses, so perhaps slots 1 and 2 are
physically tied to #F and #G. The point of the manual was that only in
slot 3 did you have to worry about sharing interrupts.
What your mainboard manual should be telling you is how the PIRQ A, B,
C, and D signals from the actual PCI slots on the mainboard are wired up
to the PIRQ A, B, C, and D inputs to the ICH5 chip. This is
mainboard-specific. The idea is that every card uses the PIRQ A signal
for preference, and that a wiring matrix on the mainboard distributes
everyone's PIRQ A amongst PIRQs A, B, C, and D by the time that the
signal reaches the ICH chip. Mainboard manufacturers are free to choose
the matrix, and must lay out a table, the so-called "$PIR" table, in
firmware memory that describes it to software (so that software knows
what PCI interrupt a card that reports "I'm using PIRQ A!" is actually
using). The signals after they have been processed through the wiring
matrix are what the ICH5 treats as PIRQ A, B, C, and D, to which it adds
E, F, G, and H.
Click to expand...

The one PCI card I have installed is reported by pci.exe to use PCI
bus#2, IRQ#3, INT#A
[...]
Dave
 
Jonathan said:
An i8042 keyboard and mouse, UART, parallel port, infrared port, MIDI
interface, environment controller, and 82078 floppy disc controller will
be connected to a separate "Super I/O" chip that is on the LPC bus. (If
you look at your mainboard you'll see it somewhere. It's usually a
WinBond or ITE chip. I'm interested in knowing which particular one you
have.)
Click to expand...

Forgot to answer this, it is a Winbond, W83627EHG-A
Dave
 
For a typical single user, multi core system, what are the possible
'combinations' of these options to use/try?
Click to expand...

Note that your situation is different to that of M. Yeo. You're getting
different symptoms. M. Yeo's problem may well be a bona fide interrupt
delivery problem. You say that you have a working system for a while,
indicating that all of the interrupt signals that are involved in disk
I/O, keyboard I/O, mouse I/O, and the system heartbeat are being handled
and not misrouted or lost.

Don't get hung up on OS2APIC.PSD just because it is the only tunable
that is easy for you to tune.

For starters, have you tried Mike Greene's procedure from 2004?

http://mgreene.org./wikka/Os2Smp
 
An i8042 keyboard and mouse, UART, parallel port, infrared port, MIDI
Forgot to answer this, it is a Winbond, W83627EHG-A
Click to expand...

I wanted to make sure that I had the datasheet for it. Indeed I have.
Now to check the support in the bus driver. ... (-:
 
Your mainboard manual is misleading you somewhat. These aren't actual
Well, it claims to have 3 PCI buses, so perhaps slots 1 and 2 are
physically tied to #F and #G. The point of the manual was that only in
slot 3 did you have to worry about sharing interrupts.
Click to expand...

3 *root* PCI buses? Or just 3 PCI buses connected through two
PCI-to-PCI bridges? The difference is important. Three root PCI buses
is an unusual configuration, and a fairly unnecessary one for just three
expansion slots in total. Even three buses connected via bridges is
slightly overkill (unless AGP is involved).
The one PCI card I have installed is reported by pci.exe to use PCI
bus#2, IRQ#3, INT#A
Click to expand...

Which rather puts the kibosh on what the manual says about interrupt
sharing, doesn't it? That's sharing with what is normally an ISA
interrupt for a UART.
 
Jonathan said:
3 *root* PCI buses? Or just 3 PCI buses connected through two
PCI-to-PCI bridges? The difference is important. Three root PCI buses
is an unusual configuration, and a fairly unnecessary one for just three
expansion slots in total. Even three buses connected via bridges is
slightly overkill (unless AGP is involved).
Click to expand...

I can't find where I read about the 3 PCI buses, I thought it was in the
manual but now can't find it.
PCI.exe reports all the onboard stuff on PCI BUS [0], the ATI AGP card
on BUS [1] and the network card plugged in as on BUS [2]
I've had problems with this network card previously in another computer
where it stopped working shortly after boot which was fixed by moving it
to another slot.
Which rather puts the kibosh on what the manual says about interrupt
sharing, doesn't it? That's sharing with what is normally an ISA
interrupt for a UART.
Click to expand...

Only one UART here which is using IRQ#4

Dave
 
PCI.exe reports all the onboard stuff on PCI BUS [0], the ATI AGP card
on BUS [1] and the network card plugged in as on BUS [2]
Click to expand...

That's three buses interconnected with PCI-to-PCI and PCI-to-AGP
bridges, and is far more the usual case.
 
The one PCI card I have installed is reported by pci.exe to use PCI
Only one UART here which is using IRQ#4
Click to expand...

A W83627EHG has two UARTs, A and B. Unless you have gone into the
firmware SETUP utility and explicitly disabled UART B, which instructs
the firmware to set the enable bit to zero in the device's configuration
space during POST, you'll have two UARTs.

I've disabled almost all of the devices in the Super I/O chips on my
test machines. But this was not their default firmware configuration.
It's also not the power-on default for the chip itself, before the
firmware gets around to programming it. The power-on default for the
W83627EHG is for both UARTs to have the enable bit set to one. The
power-on default IRQ selection for UART B is IRQ #3.
 
Jonathan said:
A W83627EHG has two UARTs, A and B. Unless you have gone into the
firmware SETUP utility and explicitly disabled UART B, which instructs
the firmware to set the enable bit to zero in the device's configuration
space during POST, you'll have two UARTs.
Click to expand...

The firmware SETUP utility only shows one serial port, which besides the
disabled setting, can be set to any of the usual settings for the first
4 ports. Hardware manager only shows one port as well and there is
physically only one port at the back of the computer.
I'd guess the firmware disables it during post.
Dave
 
The firmware SETUP utility only shows one serial port, which besides the
disabled setting, can be set to any of the usual settings for the first
4 ports. Hardware manager only shows one port as well and there is
physically only one port at the back of the computer.
I'd guess the firmware disables it during post.
Click to expand...

Let's hope; because the lack of a connector in the motherboard's back
panel I/O area means nothing at all with respect to I/O resource usage.
It just means that the UART B I/O pins on the chip aren't wired to a
connector. The chip hasn't changed, and UART B is still there. It's
the actual UART within the Super I/O chip that requires the port and
interrupt resources.

It's a shame in a way that your motherboard vendor has only wired up
UART A on the W83627EHG. UART B has all of the natty infra-red stuff.
(-: But then the evidence is piling up that ASUS did this motherboard
on the cheap. (You read my other message, ne?)

I'm interested to read that both UARTs can operate at various rates up
to 921 kilobits per second. However, thinking about the way that this
is programmed (according to the datasheet) it seems non-trivial to
instrument an otherwise general-purpose ISA UART device driver to know
about the vendor-specific and chip-specific configuration space
registers that enable this.
 
Note that your situation is different to that of M. Yeo. You're getting
different symptoms. M. Yeo's problem may well be a bona fide interrupt
delivery problem. You say that you have a working system for a while,
indicating that all of the interrupt signals that are involved in disk
I/O, keyboard I/O, mouse I/O, and the system heartbeat are being handled
and not misrouted or lost.

Don't get hung up on OS2APIC.PSD just because it is the only tunable
that is easy for you to tune.

For starters, have you tried Mike Greene's procedure from 2004?

http://mgreene.org./wikka/Os2Smp
Click to expand...

I have read that note before...and I think the key difference is that I was NOT
running any of the SMP aware kernels before applying the latest un-official FP6.


I am currently going through the FP in details, understanding what files go
where to determine if I need to upgrade anything in particular.
 
Back
Top