A Messy Transition: Practical Problems With 32bit Addressing In Windows
by Ryan Smith on July 12, 2007 12:00 PM EST- Posted in
- Software
Now that we've seen what can happen when we reach the 2GB barrier and how easy it can be to pass it, let's talk about what it took to remove it in the first place for Supreme Command. Supreme Commander is unfortunately not compiled as being large address aware and must be modified so that Windows thinks that it is. Microsoft supplies a tool as part of the Visual Studio suite, Editbin, that can do just this by rewriting the file header to report to Windows that it is in fact large address aware.
To the best of our knowledge Supreme Commander was programmed using proper programming practices and can handle the larger address space, and this is merely an issue of turning it on. However on a more pragmatic note this can break future patches, and disturbingly it doesn't set off any sort of multiplayer cheat detection in the game in spite of the fact that we have modified the executable in a very visible way. Out of the changes we need to make to deal with the 2GB barrier though, this is the safer of the two.
Update: Gas Powered Games contacted us and let us know that the modified executable not setting off any cheat detection is intentional. The game code is all in a DLL, and the executable is just a launcher; it's left unchecked because of the various Digital Rights Management systems used change the exectuable.
Changing Windows on the other hand to allocate more of the virtual address space to applications is in practice just as dangerous as we theorized earlier. We initially set our copy of Windows Vista to adjust the split to 1GB kernel mode, 3GB user mode, only for Vista to encounter a BSOD while booting. We had to settle for a 1.4GB/2.6GB split before Vista would boot, and even then Vista still periodically encounters a BSOD upon booting at that allocation or any other allocation other than 2GB/2GB. While what problems may occur and with what values is highly variable from system to system, this is why trying to move the barrier at all can be dangerous.
Having made the above changes, we also used the chance to take a look at system performance both in and outside of Supreme Commander, taking interest in to the effect of allocating more user mode space. As we theorized before, taking space away from the kernel may impact performance, and this is something that needs to be tested. For this we ran a cut-down version of our normal system test suite, with allocations of 1.4GB/2.6GB, 1.7GB/2.3GB, and the default 2GB/2GB.
| Software Test Bed | |
| Processor | AMD Athlon 64 4600+ (2x2.4GHz/512KB Cache, S939) |
| RAM | OCZ EL Platinum DDR-400 (4x512MB) |
| Motherboard | ASUS A8N-SLI Premium (nForce 4 SLI) |
| System Platform Drivers | NV 15.00 |
| Hard Drive | Maxtor MaXLine Pro 500GB SATA |
| Video Cards | 1 x GeForce 8800GTX |
| Video Drivers | NV ForceWare 158.45 |
| Power Supply | OCZ GameXStream 700W |
| Desktop Resolution | 1600x1200 |
| Operating System | Windows Vista Ultimate 32-Bit |
| . | |
Starting with Supreme Commander, the only test here that can even utilize more than 2GB of user space, we do find a minor but consistent variation in performance. Increasing the user space improved Supreme Commander performance by about 1 frame per second, which at around a 3.5% performance improvement is right along the edge of either being significant or a normal variation. We repeated this test several times just to make sure that it wasn't a variation and the results remained consistent, so it doesn't appear to be a variation. With that said the instability caused by adjusting the user space size does not justify the extremely minor performance improvement.
Going down the list of benchmarks, we find that there is no notable change in performance in any of our benchmarks. Since none of these benchmarks are capable of using more user space we weren't expecting an increase, but this puts to rest the idea of a performance decrease. There does not appear to be a performance decrease in adjusting the user space size; if it boots it'll perform well.
Facebook
Twitter
RSS
69 Comments
View All Comments
instant - Saturday, July 21, 2007 - link
Could the writer please detail what, exactly, is wrong with XP-64bit edition?Having run it since it was released and never had a problem with it, I would be interested in knowing what problems I should have experienced.
BUL - Tuesday, July 17, 2007 - link
A few things I've found about /3GB in XP...1) In terms of "consumer" OS's, XP Pro is the only one to support it. (XP Home and W2K Pro DO NOT!) Vista has a different method, outlined in the article.
2) If you change boot.ini, make sure you add a "3GB" boot option to your operating systems, not just change the one entry to /3GB. If a new driver gets installed that doesn't like /3GB, you could be left without the ability to change boot.ini back. (I can't find any info on whether /3GB affects Safe Mode, however safe mode may run in "3GB" mode!) So, unless you have the ability to WRITE to NTFS partitions outside of Windows, you may be stuck if you don't. See http://www.gehrytechnologies.com/catia/catia/catia...">http://www.gehrytechnologies.com/catia/catia/catia... for how to do it...
Now, personally, I feel that a limitation like memory shows the "men from the boys" in terms of programmers. Windows and "endless memory" has allowed for bloatware for far too long and allowed for sloppy, inefficient programming... Just think back how many games & other programs could run on an XT with 640KB of RAM because the programmers wrote highly efficient code. (Lotus 1-2-3 for DOS was written primarily in assembler; Quattro Pro could open even larger spreadsheets than 1-2-3 because it utilized pseudo-virtual memory).
Now we've come full-circle into the world of Windows, and some of these programs now have to become more efficient. That's what programmers are paid to do...
Anorax - Monday, July 16, 2007 - link
On some of the larger maps in AW2 the game will occasional crash to desktop and leave an error window saying it has exceeded the 2GB memory space. Not that helpful but at least it tells you why it crashed.the Chase - Sunday, July 15, 2007 - link
Thank you so much for this article. I had a post awhile back in the forums asking for help on why in Vista I couldn't even run PR for bf2 unless I turned down the texture setting to Medium.Never figured it out though people here did offer some good ideas for fixes. After reading this article I knew this is what was going on. I searched on the Net and found out how to use Visual Studio C++ to modify the Bf2 executable. Problem solved! (I have the 64 bit version of Vista).
And to think I was ready(thinkeng about anyway) going to buy another 2 GB's of memory to try and cure the problem. Thanks again.
P.S. I also did the same fix for XP pro 64 and now PR runs much smoother than before in that OS also.
spookware - Friday, July 13, 2007 - link
Just wanted to mention that the description of how windows uses memory is correct for XP and previous versions but it is not quite how windows vista works.Specifically, vista does not automatically take the upper 2GB of adress space for the kernel any more. It instead grows the kernel adress space downard on demand. Allthough the associated switches (/3G) have the same prupose in vista what they are setting is actually the upper maximum for the kernel adress space.
Magumi - Friday, July 13, 2007 - link
I guess this means that if I want to buy a computer to last me at least three or four years, I need to get 64bit Vista and 8GB of RAM and hope that drivers and incompatibilities get resolved soon.Greyhead - Friday, July 13, 2007 - link
Ryan, great article. Some other posters mentioned the use of the /PAE on servers. I recently discovered that MS recommends NOT using the /3GB switch in conjunction with /PAE. I followed their advice on a large SQL server we have and saw immediate positive results. The problem with /PAE /3GB combination is that when the OS is limited to 1 GB, there is not enough room for the size of the heap needed to support the /PAE option! This can be viewed using performance monitor - selecting "memory" options and then viewing total PTEs available (Page Table Entries). There are MS articles that describe the minimum PTEs needed, and before I changed our server it was way under the minimum. We had stupid errors on the server - blue screens, "not enough memory" errors when transferring files to another machine. Once the change was made, these problems disappeared. 2003 server has a more tunable /3GB switch using the /USERVA switch. There are technet articles which provide guidance on it's usage.Keep up the great work -
-bill
redpriest_ - Friday, July 13, 2007 - link
binary is illegal.Kougar - Friday, July 13, 2007 - link
The moment I saw the title I was thinking of Supreme Commander. I particularly enjoy the insanely huge 8-player games, but it only took about 40 minutes before these would crash... oddly enough they did not always crash either in some situations, making the crashing appear random and only confusing my attempts to troubleshoot this game. It would have been quite appreciated that this issue would have at least been mentioned in the game's readme file, if nothing else.Having (very luckily) stumbled across MadBorris's thread I made the modifications to XP and SC has been running since. I have not run into any instability or issues with XP configured with the /3gb switch for what it is worth. Am I wrong in that users with video cards featuring smaller onboard memory sizes will have an "advantage" with this problem? There is a large difference between a 320mb GTS and 740MB GTX, or heaven forbid a 1GB HD 2900 card? And while on the subject does a dual-GPU configuration (and therefore dual VGA memory) make things even worse?
Ryan Smith - Friday, July 13, 2007 - link
Yes to the first question. To the second question I believe that's also a yes, but I don't have a system configured to test that theory.