Over the weekend we reported on a statement released by NVIDIA regarding recent concern over unusual VRAM allocation and VRAM bandwidth performance from the GeForce GTX 970. In brief, various GTX 970 owners had observed that the GTX 970 was prone to topping out its reported VRAM allocation at 3.5GB rather than 4GB, and that meanwhile the GTX 980 was reaching 4GB allocated in similar circumstances. All of this was at odds with what we thought we knew about the cards and the underlying GM204 GPU, as the two cards were believed to have identical memory subsystems.

In their statement, NVIDIA confirmed that this unusual behavior existed, and that it was due to the GTX 970 possessing “fewer crossbar resources to the memory system” than the GTX 980. Furthermore due to this, the VRAM on the GTX 970 was segmented in a way it’s not on the GTX 980, with 3.5GB of the 4GB serving as one high priority segment, and the final 512MB serving as a low priority segment. The rest of NVIDIA’s statement then focused on performance, and why they believed this design would not have a significant performance impact versus a GTX 980-style non-segmented memory structure.

However because NVIDIA’s statement was intentionally high-level – it was meant to be accessible to the wider press and their readers, and not just the technical press such as ourselves – this meant that the statement offered few technical details on what was going on under the hood, which makes it difficult for us to validate NVIDIA’s claims and on a more human level we’re always dissatisfied with half of an answer.

As a result we’ve been digging into this issue over the weekend, working to collect the final pieces of the puzzle. To that end NVIDIA reached out to us Sunday evening to fully explain the issue at a level of detail necessary for our level of depth, making available to us a “triage team” of sorts of technical marketing, product management, and engineering/architectural personnel to answer questions and to better explain the issue. While there’s still some benchmarking left to do on our end – we’ve only had a relatively brief Sunday evening to work with NVIDIA’s full explanation – at this point we feel like we have a solid grasp of what’s going on with the GTX 970, how its memory segmenting and allocation work, and how it can impact performance.

Now make no mistake, NVIDIA right now is in full damage control mode due to the negative press this issue has garnered and the pain that’s going to follow. When NVIDIA is getting Senior VPs like Jonah Alben on the phone with us on a weekend night to talk architecture and answer questions, this isn’t normal operating procedure for the company. But at the same time it’s a positive sign for how serious NVIDIA is taking our concerns, and meanwhile an NVIDIA under pressure is an NVIDIA that is more likely to answer our deepest technical questions, giving us more insight than ever before into GM204.

GeForce GTX 970: Correcting the Specifications

To kick things off, we need to start at the very beginning and talk about the specifications of the GTX 970 itself. Everything that follows is derived from how the chip is configured, so we cannot go any farther until we understand the true configuration of the GTX 970.

NVIDIA GPU Specification Comparison
  GTX 980 GTX 970 (Original) GTX 970 (Corrected)
CUDA Cores 2048 1664 1664
Texture Units 128 104 104
ROPs 64 64 56
Core Clock 1126MHz 1050MHz 1050MHz
Boost Clock 1216MHz 1178MHz 1178MHz
Memory Clock 7GHz GDDR5 7GHz GDDR5 7GHz GDDR5
Memory Bus Width 256-bit 256-bit 256-bit
L2 Cache 2MB 2MB 1.75MB
FP64 1/32 FP32 1/32 FP32 1/32 FP32
TDP 165W 145W 145W
GPU GM204 GM204 GM204
Transistor Count 5.2B 5.2B 5.2B
Manufacturing Process TSMC 28nm TSMC 28nm TSMC 28nm
Launch Date 09/18/14 09/18/14 09/18/14
Launch Price $549 $329 $329

When the GTX 980 and GTX 970 were released, NVIDIA provided the above original specifications for the two cards. The launch GTX 900 GPUs would be a standard full/die-harvested card pair, with the GTX 980 using a fully enabled GM204 GPU, while the GTX 970 would be using a die-harvested GPU where one or more SMMs had failed. As a result of this the big differences between the GTX 980 and GTX 970 would be a minor clockspeed difference, the disabling of 3 (of 16) SMMs, and a resulting reduction in power consumption. Most important for the conversation at hand, we were told that both possessed identical memory subsystems: 4GB of 7GHz GDDR5 on a 256-bit bus, split amongst 4 ROP/memory controller partitions. All 4 partitions would be fully active on the GTX 970, with 2MB of L2 cache and 64 ROPs available.

This, as it turns out, was incorrect.

As part of our discussion with NVIDIA, they laid out the fact that the original published specifications for the GTX 970 were wrong, and as a result the “unusual” behavior that users had been seeing from the GTX 970 was in fact expected behavior for a card configured as the GTX 970 was. To get straight to the point then, NVIDIA’s original publication of the ROP/memory controller subsystem was wrong; GTX 970 has a 256-bit memory bus, but 1 of the 4 ROP/memory controller partitions was partially disabled, not fully enabled like we were originally told. As a result GTX 970 only has 56 of 64 ROPs and 1.75MB of 2MB of L2 cache enabled. The memory controllers themselves remain unchanged, with all four controllers active and driving 4GB of VRAM over a combined 256-bit memory bus.

This revelation significantly alters how we perceive the performance of the GTX 970 in certain situations, and is the missing piece of the puzzle in understanding the memory allocation issues that originally brought all of this to light. The ability to “partially disable” a ROP/memory controller partition is new to Maxwell, and we’ll fully explore how that works in a moment, but the important part to take away is that the ROP/MC layout on the GTX 970 is not fully enabled like the GTX 980, and as a result will not behave identically to the GTX 980. All of the behavior from the GTX 970 we’ve seen in light of this correction now makes sense, and it is immediately clear that this is not a hardware or software bug in GTX 970, but rather the planned/intentional behavior of the product.

The biggest and most painful question about all of this then is how did this happen? How did we get fed incorrect specifications? NVIDIA’s explanation, in a nutshell, is that this was completely accidental and that all of this stems from assumptions made by NVIDIA’s technical marketing team.

GTX 970 Reviewer's Guide: Specifications

In short, while NVIDIA does publish the specifications of their cards on the GeForce website and in other places (e.g. partner product pages), those are less-detailed high level specifications suitable for a wider audience, not unlike NVIDIA’s initial statement. The deeper technical specifications we get, such as the number of ROPs, transistor count, and L2 cache size, comes from NVIDIA’s reviewer’s guide, which outlines for us the specifications of the card, the major new features that NVIDIA is focusing on, and some examples of benchmark numbers they’re seeing. This document is compiled by NVIDIA’s technical marketing team (a group no less than extensively knowledgeable about GPUs) based on the information provided to them by the product engineers and the product management team.

The error, as NVIDIA explains it, is that in creating the GTX 970 reviewer’s guide, the technical marketing team was unaware of Maxwell’s aforementioned and new “partial disable” capabilities when they filled out the GTX 970 specification table. They were aware that the GTX 970 would have the full 256-bit memory bus, and unaware of the ability to independently disable ROPs they assumed that all 64 ROPs and the full 2MB of L2 cache was similarly available and wrote the specification table accordingly. This error then made it into the final copy of the guide, not getting caught even after being shared around various groups at NVIDIA, with that information finally diffused by press such as ourselves.

Somewhat murkier is the intervening 4 months between the launch of the GTX 970 and the discovery of this error. As NVIDIA tell us, since the error wasn’t caught in the initial check of the reviewer’s guide there weren’t any further checks being made, especially since this information wasn’t also distributed via a public source (such as the GeForce website). As far as technical marketing knew the product had 64 ROPs, and as far as product management and engineering knew the technical marketing team had done their due diligence. As a result this issue was not caught and identified at NVIDIA until earlier this month when NVIDIA technical marketing started investigating the reports and requests from the press about the memory allocation oddities. Only after running similar tests and running into the same odd results as the press and users did NVIDIA properly identify the issue.

GTX 970 Memory Allocation (Image Courtesy error-id10t of Overclock.net Forums)

Now as NVIDIA is in full damage control mode at this point, consideration must be given as to whether NVIDIA’s story is at all true; NVIDIA would hardly be the first company to lie when painted into a corner by controversy. With that in mind, given the story that NVIDIA has provided, do we believe them? In short, yes we do.

To be blunt, if this was intentional then this would be an incredibly stupid plan, and NVIDIA as a company has not shown themselves to be that dumb. NVIDIA gains nothing by publishing an initially incorrect ROP count for the GTX 970, and if this information had been properly presented in the first place it would have been a footnote in an article extoling the virtues of the GTX 970, rather than the centerpiece of a full-on front page exposé. Furthermore if not by this memory allocation issues then other factors would have ultimately brought these incorrect specifications to light, so NVIDIA would have never been able to keep it under wraps for long if it was part of an intentional deception. Ultimately only NVIDIA can know the complete truth, but given what we’ve been presented we have no reason to doubt NVIDIA’s story.

In any case, going forward NVIDIA has made it clear that they intend to make sure this doesn’t happen again, to make sure that the specifications released to the press and public are better checked to ensure nothing is inaccurate or being assumed. NVIDIA rarely makes the same mistake twice, and certainly they will be under a great deal more scrutiny from here on to make sure their published specifications are correct.

Meanwhile as press we play a role in this as well, as in retrospect we should have seen this sooner. Nebulous metrics such as VRAM allocation aside, NVIDIA’s own tools and our own experiences pointed to something being off. The pre-GM204 version of NVIDIA’s DeviceQuery CUDA application, for example, correctly reports that the GTX 970 only has 1.75MB of L2 cache and the GTX 980 has 2MB of L2 cache, even if it doesn’t fully understand the makeup of the Maxwell 2 architecture.

NVIDIA CUDA DeviceQuery: GTX 970

Furthermore our own initial incompatibility issues with the GTX 970 (and only the GTX 970) pointed to some kind of memory subsystem change that we should have looked into, even after NVIDIA corrected the problem in their drivers. Ultimately other tools such as GPU-Z and AIDA64 weren’t able to catch the difference since they too did not know about Maxwell’s ability to disable the ROPs, so only a limited number of flags were raised, but nonetheless this is something we admittedly should have been able to catch had we noticed what was wrong.

Diving Deeper: The Maxwell 2 Memory Crossbar & ROP Partitions
Comments Locked


View All Comments

  • Kutark - Tuesday, January 27, 2015 - link

    b/c consumers are dumb and if the 970 had 1mb less ram than 4gb it would of decreased sales. There is a reason they moved away from stuff like having 1.2gb or 1.5gb, etc etc. People like big solid numbers.
  • HisDivineOrder - Tuesday, January 27, 2015 - link

    You give lawyers so much credit. Often, lawyers like being one to start such things and don't care much if they manage to finish it.
  • maximumGPU - Wednesday, January 28, 2015 - link

    I'd say both Jarred. Sure, i look at performance first, but performance metrics tell me how good the card is NOW. The next thing i do is look at the specs and try and estimate how future proof my purchase will be.
    A 3GB 970 would show great metrics at 1080p, but i wouldn't buy it because i know ram is ever more important thanks to the consoles catching up.
    Since i game at 1440p, 4GB was my minimum ram threshold. i thought i got that with the 970, but instead got 3.5 + 0.5GB of slow ram. That makes my card less future proof than i thought and could've well affected my buying decision, regardless of its current performance metrics.
  • Ranger101 - Tuesday, January 27, 2015 - link

    No surprises as to Nvidia's behaviour, as a company they are of course a rapacious juggernaut, but Tut Tut Anandtech, what would the great founder have to say?

    Having read many recent articles in the GPU section, I am mostly impressed by the high quality of writing, however those who read between the lines of Mr Smith's Gpu reviews, realise that appearances of impartiality in his writing are misleading and that he is in fact a staunch and unrelenting supporter of camp green. ( Everyone is of course biased, it's just less appropriate to let it shine through in technical website reviews.)

    It should therefore come as no surpise that in his initial review these issues "escaped" his attention, despite the fact that "a limited number of flags were raised" and that in the follow up article, he unashamedly wields the Bastard sword of Nvidia. LOL.

    You must remember these things happen for a reason Ryan and I once again encourage you to temper your bias in forthcoming utterances...AMD still make good cards and a little competition is good....right? :)
  • just4U - Tuesday, January 27, 2015 - link

    I think the fact that Ryan gets accused of being in favor of AMD and Nvidia means that's he's doing a pretty good job of not really being in either camp. If anything I'd simply suggest his expectations on performance are limited and when the cards actually do better.. he tends to point that out. Not really a bad thing considering how underwhelming hardware leaps are these days in most segments. Smaller jumps not the leaps and bounds we were all once used to.
  • OrphanageExplosion - Tuesday, January 27, 2015 - link

    Oh do behave. When Anandtech had the AMD News Center sponsorship all we ever heard from the commentariat was that the site, and Ryan specifically, were AMD biased. I think we all know where the bias is on Anandtech - and it's in the comments, not the editorial.
  • HisDivineOrder - Tuesday, January 27, 2015 - link

    You're talking about the same guy that just took it on AMD's word that Mantle was going to be "virtually identical" to the same low level access API as the Xbox One and that subsequently Mantle was AMD bringing the Xbox One's low level access language to PC gaming.

    Seriously. If the guy is biased toward anything, he's biased toward believing more of AMD's statements than he really ought to, but I've had a hard time really blaming him since AMD had JUST paid for him (and his buddy journalists) to go to Hawaii on a beach trip and vacation under the excuse that it was to present the GPU part called "Hawaii." I mean, if I was tatken to Hawaii, I'd probably be willing to believe anything they told me, too.

    Still, don't mistake the man for an nVidia fanboy. He's clearly not. Lots of other people questioned that AMD party line far more than Anandtech did back in the day and it took a long time before they acknowledged that AMD had hoodwinked them and they never REALLY admitted it wholeheartedly.

    Because AMD suggesting that Mantle was anything but a completely proprietary and locked-in API was a lie and no hardware company has yet to sign up in spite of the fact Intel tried very hard to research the subject and was rebuffed by AMD for months.

    Intel likes to do anything they can do for free and they read all the press (like Ryan's) that suggested Mantle was going to be free and freely available, but as it turned out, that was more hyperbole on the part of AMD.

    Yet I saw nothing of that on Anandtech. No, I don't think there's much evidence of his being "a staunch and unrelenting supporter of camp green" unless you're recalling the heady days of AMD's time as a "green" company.
  • Gothmoth - Tuesday, January 27, 2015 - link

    as if you read or even UNDERSTAND what ROP´s mean before you buy a card.....

    you and all the others are just trolls who have to much time on their hands....
  • Kutark - Tuesday, January 27, 2015 - link

    I honestly don't understand why people are so up in arms over this. At the end of the day the performance figures still stand. The situations in which this news could actually arise and cause any problems are so limited its not even funny. At the resolutions and settings most games operate at don't use anywhere close to 4gb of vram.

    Honestly if i didn't have SLI'd 760's i'd go out and buy a 970 tonight, regardless of any of this information.

    That being said, this is another article that proves why anandtech is easily the best tech website out there. Thorough and honest, unbiased, just, amazing, love it. Sorry for all the commas.
  • Kutark - Tuesday, January 27, 2015 - link

    Meant to say gamers, not games. Regardless.

Log in

Don't have an account? Sign up now