A Messy Transition: Practical Problems With 32bit Addressing In Windows
by Ryan Smith on July 12, 2007 12:00 PM EST- Posted in
- Software
Other Problems, Other Solutions
Although for this article we are focusing on Supreme Commander, there are other games and applications that are already known to encounter this exact problem. Company of Heroes developer Relic has warned that this problem can occur, especially in conjunction with using Direct3D 10 mode and STALKER developer GSC Game World has also seen this problem. Both have gone so far as to ship the latest versions of their games with the LARGEADDRESSAWARE flag.
Although not an exhaustive list, we have compiled a list of recent games and applications and if they are flagged or not. Ideally every possible game and application that can run in to the 2GB barrier will be flagged so that it can be worked around without modifying the application directly itself.
| Large Address Aware Status | |
| STALKER: Shadow of Chernobyl | Yes |
| Lost Planet | No |
| Company of Heroes | Yes |
| Supreme Commander | No |
| Enemy Territory: Quake Wars | Yes |
| Battlefield 2142 | No |
| Call of Juarez DX10 BM | No |
| World of Warcraft | No |
| The Elder Scrolls IV: Oblivion | No |
| Photoshop CS3 | Yes |
| Dreamweaver CS3 | No |
| . | |
The importance of the above applications and games being flagged is not just to avoid the need to modify the executable, but also because there is another solution to the issue we haven't talked about so far. 64bit versions of Windows(i.e. XP and Vista) do not suffer from the traditional 2GB barrier, as all the kernel mode addressing is usually moved to well above the confines of the limited 32bit addressing area. As such, these versions of Windows don't need to have their space allocations adjusted for an application to gain access to more addressing space, bypassing the instability and any possible performance problems that occurs as a result of making this adjustment.
However in order to maintain compatibility with older applications, Windows still keeps the artificial 2GB barrier in place to keep from triggering any bugs that result from the extra space. So even if we use a 64bit version of Windows, the offending application must still either be flagged by the developer or modified by the user if we want to get past the 2GB barrier.
This in turn is an interesting proposition for developers who may be looking at ways to deal with the 2GB barrier, but don't want to make the jump to having to program and support 2 versions of their executables and associated compiled code. Although as a 32bit application there is still the 4GB hard limit, flagging executables lets a developer keep a single package and is perfectly safe on both stock 32bit and 64bit installations of Windows. They can effectively buy another 2 years or so before they need to actually offer 64bit executables, and can direct users to install a 64bit version of Windows if they are hitting the 2GB barrier, instead of directing them to make the much riskier user space modification.
Of course this is not all roses. As we covered in our Vista performance guide, there are still some issues with Windows Vista 64bit, and Windows XP 64bit is even worse as a result of having been orphaned quickly after its release. For prospective 64bit Vista users, they will still find that driver support is not as good as with the 32bit version of Vista, and 64bit drivers may not be as stable as the 32bit versions. There are also still lingering concerns over application compatibility and performance.
Things have gotten better since we published our February article, but we're not ready to write them off completely yet. Users getting along with 32bit versions of Windows right now will find themselves in between a rock and a hard place as more applications and games start to hit the barrier - they'll either need to make the user space modification or switch to a 64bit version of Windows when neither of these is a perfect solution. This leaves the less imperfect but also less fun option of simply turning down the settings on any affected games. Lower settings result in lower user space usage and reduced chances of crashing, but in spite of the highest stability and compatibility offered by this option we suspect few users will actually opt for it.
Facebook
Twitter
RSS
69 Comments
View All Comments
BabyBear - Thursday, July 12, 2007 - link
This also happens with Flight Simulator X by the way.
Over on Phil Taylor's blog he makes mention of it awhile back
http://blogs.msdn.com/ptaylor/archive/2007/06/15/f...">Microsoft's Phil Taylors Blog
There was also some talk that the June 07 DirectX Redist. seemed to 'help' with Out of Memory problems when using the /3gb switch.
joetron2030 - Thursday, July 12, 2007 - link
On "Page 4: A Case Study: Supreme Commander", in the paragraph discussing the first screen shot from Sysinternals' Process Explorer, you refer to the "Virtual Size" column as the "Private Size" column (which doesn't exist in the screen shot).Had me briefly confused.
quanta - Thursday, July 12, 2007 - link
Although Windows 2000/2003/Vista server versions aren't exactly designed for gaming, did anyone tested game titles on these server OS to see if these large address aware titles will use the Physical Address Extension feature?Ryan Smith - Thursday, July 12, 2007 - link
The short answer is no.The longer answer is that due to a combination of chipset support, software support, performance, and driver support, it's not really usable outside of a server environment and shouldn't be used on a consumer system.
brink - Thursday, July 12, 2007 - link
Wouldn't matter, WinXP SP2 uses PAE, why would you want to install it on a server OS? Only 2003 is a server OS of the ones you mentioned, and I think only Windows 2003 Enterprise is the only 32-bit OS that has the ability to use more than 4GB of memory (8GB seems to be the limit for what modules/mobos are available right now)TA152H - Thursday, July 12, 2007 - link
I'm just reading the remarks about the 386 pushing the memory from 1 MB, to 4 GB. It's patently untrue, unless you think the 386 succeeded the 8086, and not the 286. The 286 had 24 bit addressing (in 64K segments) for 16 MB of memory. This is not only what OS/2 used, but extended memory as well. So, it's not academic.Ryan Smith - Thursday, July 12, 2007 - link
Just hearing the words "segmented memory" gives me flashbacks - and they're not the good kind. For this article we're talking strictly about flat addressing since I could write a small book on just segmented addressing, but I've updated the article to make this clear.TA152H - Thursday, July 12, 2007 - link
Either way, you're using the wrong values. Even on the 8086 the memory was segmented. Actually, so was the 386, but it could handle segments up to 4 GB, so it wasn't important. So, using 1 MB and saying you're not referring to segmentation is inaccurate; that is segmented memory.Segmentation was not a bad thing, particularly back then. It saved memory space, and back in 1978 that was a big thing. Addresses were given in 16 bits, you didn't have to specify the rest, and the net result was you had shorter code. You would have to change the registers to point to the next segment from time to time, but overall it saved a lot of memory. You also could protect apps from each other, but that wasn't really used.
I was an OS/2 developer, and it was a pain sometimes for sure. By the time the 286 came out, which I think was the best processor Intel ever made considering the timing, and the incredible capabilities it had over it's predecessor (the 8086, not the 80186 which was made in parallel with the 286), the extra memory saving was not worth the nuisance of having to deal with, at most, 64K segments. But it really wasn't possible for Intel to do it any other way, as I'll explain below.
Motorola even added some external memory management unit that added segmentation, which today seems strange. It's widely viewed now as just a bad thing, but back then, it wasn't. Motorola really had a choice though, since the 68K was part 32-bit, and part 16-bit, and part 24-bit (addressing). Although the 286 was a more powerful processor than the 68000, it was pure 16-bit, except for the oddity of the 24-bit addressing. Consequently, allowing flat 24-bit addressing would not have been feasible. So, they dealt with it by just adding more segments. Considering the absolutely incredible improvement in this processor in just about every way, it was not such a bad tradeoff.
BitJunkie - Friday, July 13, 2007 - link
Hiya Bill.jay401 - Thursday, July 12, 2007 - link
Just FYI, SupCom has already exhibited this problem and crashes once it breaks the 2GB barrier, which can happen easily in longer games with a high unit limit.This is mostly due to the poor efficiency in their code and models, something GPG (the developer, Gas Powered Games) reported could not easily be fixed in a patch because they basically have to re-render every unit in the game so they take up less memory. Due to this, it's likely to remain unfixed until the expansion pack (now a standalone game) Forged Alliance comes out in November, if it is fixed at all
One forum member developed a way to increase the addressable memory to around 3GB on 32bit WinXP and Vista, so if you're running 4GB, this provides a sort of band-aid solution in the meantime.