How lock memory so not paged? vs Unix mlock.

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Ive got a time-critical application that runs on Windows XP and Unix. I've
got some timing issues that Ive traced to virtual memory getting paged in and
out. I think I can solve the problem if I can "lock" a certain huge memory
buffer into physical memory, so it is not paged out and in. The rest of my
program (code and other data) can get paged in and out: it is just this one
buffer that is critical.

Unix (some Unixs, at least) have an mlock() function that is working well
for me. Now Id like to do the same thing on Windows.

Question: Is there a technique in Windows XP (Im using Visual C++) to lock
a memory buffer (that my program allocated with malloc() ) into physical
memory so it is never paged out?

Thanks in advance for any replies,
neal
 
I've done some hunting around and found the function
VirtualLock()
that may meet my requriements in Windows.

If anyone has any advice or tips about VirtualLock(), Id appreciate it.

Thanks,
neal
 
noleander said:
I've done some hunting around and found the function
VirtualLock()
that may meet my requriements in Windows.

If anyone has any advice or tips about VirtualLock(), Id appreciate it.
Click to expand...

I wouldn't know how mlock works so I don't know if it will do what you want
exactly.

But, to be clear the function locks some number of pages into the "working
set" of the calling process. (The working set is the collection of pages
that are backed by physical memory). Under memory pressure Windows is free
to swap out your process to make room for something more pressing, locked
pages included.

Regards,
Will
 
Hi William!

But, to be clear the function locks some number of pages into the "working
set" of the calling process.
Click to expand...

AFAIK: Virtual Lock really locks the pages into *physical* memory!
The following will fail on the second call to "SetProcessWorkingSet",
(trimming to 0 is not allowed, because some pages are locked into
virtual memory).

char *data = new char[1024*1024]; // 1 MB
BOOL bRet = SetProcessWorkingSetSize(GetCurrentProcess(),
1024*1024*2, 1024*1024*10);
bRet = VirtualLock(data, 1024*1024); // LOCK into memory!
bRet = SetProcessWorkingSetSize(GetCurrentProcess(), 0, 0); // WILL
FAIL!
while(1) Sleep(1000);

You also can see that the working-set is never under 1 MB...

Of course, this will require "PROCESS_SET_QUOTA" permissions...

--
Greetings
Jochen

My blog about Win32 and .NET
http://blog.kalmbachnet.de/
 
Jochen,
Locking memory pages doesn't prevent paging-out, it only asks the memory
manager to keep these pages in memory as long as possible. Run following
code, and watch the WS getting trimmed to ~40-200Kb (your milleage may vary
depending on OS version and type of application) despite the allocation of
2MB of locked memory.
Another point is that SetProcessWorkingSetSize (user API) and WS trimming by
the memory balance manager are two different things, the balance set
manager can remove a complete process from memory irrespective the number of
locked pages.


int BytesToLock = 2*1024*1024;
char *data = new char[BytesToLock]; // 2 MB
bool bRet = SetProcessWorkingSetSize(GetCurrentProcess(), 1024*1024*2,
1024*1024*10);
bRet = VirtualLock(data,BytesToLock); // LOCK into memory!
bRet = SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1); //
Succeeds
while(1) Sleep(1000);


Willy.

| Hi William!
|
|
| >> VirtualLock()
| >
| > But, to be clear the function locks some number of pages into the
"working
| > set" of the calling process.
|
| AFAIK: Virtual Lock really locks the pages into *physical* memory!
| The following will fail on the second call to "SetProcessWorkingSet",
| (trimming to 0 is not allowed, because some pages are locked into
| virtual memory).
|
| char *data = new char[1024*1024]; // 1 MB
| BOOL bRet = SetProcessWorkingSetSize(GetCurrentProcess(),
| 1024*1024*2, 1024*1024*10);
| bRet = VirtualLock(data, 1024*1024); // LOCK into memory!
| bRet = SetProcessWorkingSetSize(GetCurrentProcess(), 0, 0); // WILL
| FAIL!
| while(1) Sleep(1000);
|
| You also can see that the working-set is never under 1 MB...
|
| Of course, this will require "PROCESS_SET_QUOTA" permissions...
|
| --
| Greetings
| Jochen
|
| My blog about Win32 and .NET
| http://blog.kalmbachnet.de/
 
Hi Willy!
Locking memory pages doesn't prevent paging-out,
Click to expand...


I can´t verify your statement on my machine...
Even with the following code, the working set is always > 2MB!

#include <windows.h>
#include <tchar.h>

int _tmain()
{
char *data = new char[1024*1024*2]; // 2 MB
BOOL bRet = SetProcessWorkingSetSize(GetCurrentProcess(),
1024*1024*4, 1024*1024*10);
bRet = VirtualLock(data, 1024*1024*2);
while(1)
{
// swap-out the whole process:
bRet = SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1);
Sleep(1000);
}
}


So I can´t verify that the working set gets trimmed < 2MB!


And that is also what the documentation of "VirtualLock" is saying...


--
Greetings
Jochen

My blog about Win32 and .NET
http://blog.kalmbachnet.de/
 
Jochen,

Are you sure you aren't looking at the "private bytes" instead of the
"working set"?
Also, you should not include the call to SetProcessWorkingSetSize in the
loop, this will increase the WS because you keep the process in the "active"
list, so the balance set manager keeps more pages in the WS (at least the
pages for Sleep, SetProcessWorkingSetSize and their callee's).

If I run this from the commandline (XP SP2), the WS drops to 132Kb and when
I minimize the cmd window it drops further to 48Kb. (On W2K3 SP1 the values
are resp. 115Kb and 32Kb).

int BytesToLock = 2*1024*1024;
char *data = new char[BytesToLock]; // 1 MB
bool bRet = SetProcessWorkingSetSize(GetCurrentProcess(), 1024*1024*2,
1024*1024*10);

bRet = VirtualLock(data,BytesToLock); // LOCK into memory!
bRet = SetProcessWorkingSetSize(GetCurrentProcess(),-1, -1); // WILL
FAIL!
Sleep(60000); // sleep 1'
return 0;

As far as I know, it's impossible to lock more pages than the "minimum
working set", that means that when you call SetProcessWorkingSetSize (after
VirtualLock) to reduce the WS to a value lower than the locked page value,
the WS will drop and a number of pages will unlock. This can be watched by
using the vadump utility.

Willy.


| Hi Willy!
|
| > Locking memory pages doesn't prevent paging-out,
|
|
| I can´t verify your statement on my machine...
| Even with the following code, the working set is always > 2MB!
|
| #include <windows.h>
| #include <tchar.h>
|
| int _tmain()
| {
| char *data = new char[1024*1024*2]; // 2 MB
| BOOL bRet = SetProcessWorkingSetSize(GetCurrentProcess(),
| 1024*1024*4, 1024*1024*10);
| bRet = VirtualLock(data, 1024*1024*2);
| while(1)
| {
| // swap-out the whole process:
| bRet = SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1);
| Sleep(1000);
| }
| }
|
|
| So I can´t verify that the working set gets trimmed < 2MB!
|
|
| And that is also what the documentation of "VirtualLock" is saying...
|
|
| --
| Greetings
| Jochen
|
| My blog about Win32 and .NET
| http://blog.kalmbachnet.de/
 
Hi Willy!
Are you sure you aren't looking at the "private bytes" instead of the
"working set"?
Yes....

Also, you should not include the call to SetProcessWorkingSetSize in the
loop,
Click to expand...

It just wanted to be sure that the WS will be trimmed to a very small
amount...
But I removed it from the loop and put the sleep to 100000000... still
same effect (XP-SP2). The working-Set is still min. 2MB...

If I run this from the commandline (XP SP2), the WS drops to 132Kb and when
I minimize the cmd window it drops further to 48Kb.
Click to expand...

After starting the WS has 2160 KB and the Private bytes has 2276 KB.
After minimizing it hase 2084 KB and 2276 KB...

The observed behaviour is exactly the documented behaviour...




bool bRet = SetProcessWorkingSetSize(GetCurrentProcess(), 1024*1024*2,
1024*1024*10);
bRet = VirtualLock(data,BytesToLock); // LOCK into memory!
Click to expand...

Are you sure that the adjustment of the min. working set and the
virtual-lock succeeds?

As far as I know, it's impossible to lock more pages than the "minimum
working set",
Click to expand...

Yes. Thats the reason why we adjust the min. working set before we call
VirtualLock...


that means that when you call SetProcessWorkingSetSize (after
VirtualLock) to reduce the WS to a value lower than the locked page value,
the WS will drop and a number of pages will unlock. This can be watched by
using the vadump utility.
Click to expand...

Here is (a small excerpt) of the output from vadump -o

Catagory Total Private Shareable
Pages KBytes KBytes KBytes
Page Table Pages 4 16 16 0
Other System 4 16 16 0
Code/StaticData 0 0 0 0
Heap 0 0 0 0
Stack 0 0 0 0
Teb 0 0 0 0
Mapped Data 0 0 0 0
Other Data 513 2052 2052 0

Total Modules 0 0 0 0
Total Dynamic Data 513 2052 2052 0
Total System 8 32 32 0
Grand Total Working Set 521 2084 2084 0


--
Greetings
Jochen

My blog about Win32 and .NET
http://blog.kalmbachnet.de/
 
Jochen,

Correct, my bad, the VirtualLock failed, the minimum WS must be at least the
locked page size + the minimum WS (I had to set this value to 140Kb + size
of locked memory).
In the posted sample both were equal so the call failed with error 1453. I
had the error check in place when I first tried your code snip but
uncommented it it when I changed the code, something you should never do,
lesson learned ;-)

Willy.



| Hi Willy!
| > Are you sure you aren't looking at the "private bytes" instead of the
| > "working set"?
|
| Yes....
|
| > Also, you should not include the call to SetProcessWorkingSetSize in the
| > loop,
|
| It just wanted to be sure that the WS will be trimmed to a very small
| amount...
| But I removed it from the loop and put the sleep to 100000000... still
| same effect (XP-SP2). The working-Set is still min. 2MB...
|
|
| > If I run this from the commandline (XP SP2), the WS drops to 132Kb and
when
| > I minimize the cmd window it drops further to 48Kb.
|
| After starting the WS has 2160 KB and the Private bytes has 2276 KB.
| After minimizing it hase 2084 KB and 2276 KB...
|
| The observed behaviour is exactly the documented behaviour...
|
|
|
|
|
| > bool bRet = SetProcessWorkingSetSize(GetCurrentProcess(),
1024*1024*2,
| > 1024*1024*10);
| > bRet = VirtualLock(data,BytesToLock); // LOCK into memory!
|
| Are you sure that the adjustment of the min. working set and the
| virtual-lock succeeds?
|
|
| > As far as I know, it's impossible to lock more pages than the "minimum
| > working set",
|
| Yes. Thats the reason why we adjust the min. working set before we call
| VirtualLock...
|
|
|
| > that means that when you call SetProcessWorkingSetSize (after
| > VirtualLock) to reduce the WS to a value lower than the locked page
value,
| > the WS will drop and a number of pages will unlock. This can be watched
by
| > using the vadump utility.
|
| Here is (a small excerpt) of the output from vadump -o
|
| Catagory Total Private Shareable
| Pages KBytes KBytes KBytes
| Page Table Pages 4 16 16 0
| Other System 4 16 16 0
| Code/StaticData 0 0 0 0
| Heap 0 0 0 0
| Stack 0 0 0 0
| Teb 0 0 0 0
| Mapped Data 0 0 0 0
| Other Data 513 2052 2052 0
|
| Total Modules 0 0 0 0
| Total Dynamic Data 513 2052 2052 0
| Total System 8 32 32 0
| Grand Total Working Set 521 2084 2084 0
|
|
| --
| Greetings
| Jochen
|
| My blog about Win32 and .NET
| http://blog.kalmbachnet.de/
 
Jochen Kalmbach said:
AFAIK: Virtual Lock really locks the pages into *physical* memory!
The following will fail on the second call to "SetProcessWorkingSet",
(trimming to 0 is not allowed, because some pages are locked into virtual
memory).
Click to expand...

It could be, but the docs have always said otherwise.

At this link

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dngenlib/html/msdn_virtmm.asp

you'll find this

<quote>
Processes in Windows NT are granted subtle influence into this system
behavior with the VirtualLock and VirtualUnlock functions. Essentially, a
process can establish specific pages to lock into its working set. However,
this does not give the process free reign over its working set. It cannot
affect the number of pages that make up its working set (the system adjusts
the working set for each process routinely), and it cannot control when the
working set is in memory and when it is not.
</quote>

What you demonstrate is that the working set of a process can not be trimmed
down past the limit of what it has locked. I think that docs are pointing
out that the operating system may choose to page out an _entire_
application, locked pages and all, under severe memory pressure.

Now that article goes back to the early days of NT, and the real behavior
may differ. I don't know but it goes against the prevailing wisdom in the
kernel group.

Further, I'd never suggest something at odds with the docs unless I
scrutinized the source and confirmed the anomalous behavior or heard from
someone whom I trust who did.

Regards,
Will
 
message
| | >> But, to be clear the function locks some number of pages into the
| >> "working set" of the calling process.
| >
| > AFAIK: Virtual Lock really locks the pages into *physical* memory!
| > The following will fail on the second call to "SetProcessWorkingSet",
| > (trimming to 0 is not allowed, because some pages are locked into
virtual
| > memory).
|
| It could be, but the docs have always said otherwise.
|
| At this link
|
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dngenlib/html/msdn_virtmm.asp
|
| you'll find this
|
| <quote>
| Processes in Windows NT are granted subtle influence into this system
| behavior with the VirtualLock and VirtualUnlock functions. Essentially, a
| process can establish specific pages to lock into its working set.
However,
| this does not give the process free reign over its working set. It cannot
| affect the number of pages that make up its working set (the system
adjusts
| the working set for each process routinely), and it cannot control when
the
| working set is in memory and when it is not.
| </quote>
|
| What you demonstrate is that the working set of a process can not be
trimmed
| down past the limit of what it has locked. I think that docs are pointing
| out that the operating system may choose to page out an _entire_
| application, locked pages and all, under severe memory pressure.
|
| Now that article goes back to the early days of NT, and the real behavior
| may differ. I don't know but it goes against the prevailing wisdom in the
| kernel group.
|
| Further, I'd never suggest something at odds with the docs unless I
| scrutinized the source and confirmed the anomalous behavior or heard from
| someone whom I trust who did.
|
| Regards,
| Will
|

This docs are still valid, as I said in another reply, there is a
significant difference between a user process initiated WS trimming action
(through SetProcessWorkingSet) and the OS "balance set manager" initiated WS
trimming action. The BSM is the master of the game here and he's allowed to
trim all pages from a process's WS, even the pages that are locked by
VirtualLock. The only pages that never gets removed from memory are those
allocated the non-paged pool, but these do not apply to user processes at
all.

Willy.
 
Back
Top