I'd like to request an article/chart of haswell laptops and C-state support much like you made recently for mobile device test suite cheats. It's a lot of work but I think it paints a clear picture for consumers and sends a strong message to OEMs. At the very least it makes choosing a laptop easier.
I can provide charts for the Haswell laptops we still have on hand, but most of the previously tested laptops have already been returned. I'll see about doing a follow-up to this post perhaps later today....
It just occurred to me, as I don't recall seeing this information in your reviews, but when testing battery life, I assume you do so through the desktop on Windows 8(.1)? Or with the desktop open at least? It would be nice to know if the battery tests would come out differently if one stayed entirely in the Modern UI (as that would intuitively be the case given the lack of resource control on the desktop). Staying in Modern doesn't make much sense on a laptop necessarily, but it's something that's nice to know for x86 hybrids (Surface Pro 2 being a prime candidate). :-)
The benchmarks we run do use IE10 (now IE11) in desktop mode. I can try doing a test using the IE Win8 app, but actually when Win8 first appeared I did a battery life test and the non-desktop IE10 delivered worse battery life. I think the Modern UI may actually put a constant (albeit small) load on the iGPU that's more demanding in terms of power than the desktop.
It would certainly be nice to know if that is still the case in IE11 and 8.1, and I would also be interested in knowing if it's the case with something like video playback. Just for accuracy's sake. :-)
It will obviously take a bit of time to test, but I'll look into it. I think it might be a good time to do some pure video testing as well using MPC-HC, VLC, Windows Media Player, and the new Video app to see which does best in terms of battery life. It will take about a week, give or take, before I can test all those....
Jarred: Just as a sidenote, Sony is using heavily modified drivers for their Vaio Laptops and I would therefore take the above posted results with a grain of salt as long as there are no "real" Windows 8.1 drivers for the Sony Vaio Pro 13. Other than that I do agree on your statement that PC OEMs definitely need to step up their drivers and continued support.
I installed the latest Intel drivers on the VAIO Pro 13 for testing before the initial review, which is something we do on all laptops. I don't know if the Sony-provided drivers would have changed things, but it's possible.
Thanks. Was not trying to invalidate your results, it just stems from my experiences with Sony Vaio drivers for my Sony Vaio Series S (SA) under Windows 8.1. Most drivers are not even updated for Windows 8.1.
No worries. Installed drivers right now are 15.33.5.3316 -- I think that's the latest from Intel as of last week. The previous battery life testing was completed with the earlier 15.31.17.3257 driver. Maybe that accounts for the change in the Medium test result, but I'm retesting now just for validation....
That would be very welcome as well. I don't know how far from the release the Modern VLC app is, but it might interesting to compare it to VLC on the desktop.
Jarrod the video windows store app uses entirely different rendering APIs do I imagine the difference would be dramatic.
another thing to keep in mind is desktop and all its baggage is not loaded until you tap the desktop tile. So you might get different results if you never tapped it after boot or if you did
Trying to avoid the desktop is essentially impossible and I seriously doubt it's not loaded until used. Think about all the stuff that's on the desktop: Windows Explorer is one, most of the control panel stuff, any non-Modern app, Task Manager, etc. If you had to wait a few extra seconds for a bunch of "baggage" to load you would know it. Even browsing through a file open dialog in a Modern app is likely to use parts of Windows Explorer, so I would bet the desktop is always present in some form with Windows 8. In fact, testing that I did previously indicated that Modern Apps (Music and IE10 at the time) both were less power efficient than the "baggage" desktop IE10 and Windows Media Player -- and the Modern Apps always seem to take way longer to load than they should.
Jarrod, yes, according to Microsoft the desktop code is not loaded (in its entirety) unless you enter the desktop at least once. See here: "You can use only new apps and never leave them if you want (in which case all of the desktop code is not even loaded.) " http://blogs.msdn.com/b/b8/archive/2012/05/18/crea...
I can attest to this because I have a HDD in my desktop running win 8 and now 8.1 and if I hit the desktop within a few hours after boot it takes a while and hard drive banging to load.
I also know that WMP and the Video modern app use different APIs to render video. You can see this in the task manager. one is called protected video pipeline and the other is something else. Back when I had a radeon 4850 in my desktop I had a driver bug where the modern video apps like netflix, hulu, and xbox video would leak ram and crash hard after a while but windows media player of VLC did not. Later I got a geforce card and all is well. This shows it is tied to WDDM more closely so I would assume better battery life
It's worth mentioning too that this Vaio seems to fly in the face of the running theory that Windows is inherently bad at idle/light load battery life, and proves it's more down to manufacturers.
Oh, it's almost entirely up to the manufacturers in my opinion -- maybe it's a bit more difficult to do with Windows than say OS X, but it *can* be done. Sadly, even laptops from the same manufacturer can have issues -- so the VAIO Pro 13 is really good on battery life, but perhaps another Sony has issues. Sony is usually pretty good about optimizing for battery life, but HP, Dell, ASUS, etc. are highly variable depending on the model. Budget laptops rarely get any TLC in terms of BIOS/firmware optimizations for example.
I bring it up because this codinghorror article is making its rounds around the internet, and everyone seems to just "know" windows is the culprit. But this seems like pretty clear evidence that is not the case.
I'd have to see if Anand used the "Balanced" or "Power Saver" setting for his Surface Pro 2 review, but it may have been Balanced -- which would mean it would a bit closer to the VAIO Pro 13 result on Power Saver. Also, Anand used our tablet battery life test, which at one point was less demanding than our laptop battery life test but perhaps the latest iteration changed things -- and depending on whether he used IE10/IE11 in Win8 mode or desktop mode, he could also see lower battery life.
But there are definitely advantages for the OS X side of things. Someone mentions OS X getting timer coalescing and expecting another 10-15% improvement, but I don't think it will be more than 5%. My bet is OS X is so optimized for the hardware that short of reducing the LCD brightness or something along those lines, it's the maximum we'll see in terms of battery life. Anand ran the MBA13 2013 Haswell model on Win8 as well as OS X and also saw markedly improved battery life with OS X, but that's partly because Apple doesn't have any real need to optimize the Windows side of their laptop story.
This article makes for painful reading - battery life for my own Vaio Pro 13 has taken a real hammering after the Win 8.1 update. The culprit seems to be screen brightness - it immediately stretches out to full brightness, regardless of whether the machine is onw balanced or low-power mode, on mains power or battery, and regardless of what the actual screen brightness setting is. any thoughts on sorting this out? What intel updates did you carry out, Jarred?
I had the display brightness get stuck at one point, but a reboot fixed that. I have the latest Sony BIOS and updates from their VAIO Care software, and I have the latest Intel graphics driver. I don't think I did anything else noteworthy. Battery life is for the Power Saver profile, with the LCD set to 87% (200 nits). Incidentally, I did run a test on Balanced at one point and battery life dropped I think around 10%.
I honestly didn't believe OEM optimizations could carry so much weight. The Vaio Pro, which shares the same chipset, resolution with the Surface Pro, sporting even a slightly smaller battery, simply slaughters the latter. One would assume that by controlling the hardware and software, MS would provide an incredibly tuned machine.
Thanks for your amazing responsiveness Jarred, I'm really looking forward both to the tests that Anand ran on Surface, and the video player comparison you mentioned above.
This is the first I've heard that Win 8.1 was supposed to improve battery life. What were they changing in order to optimize it? Improved low power state support? Thanks.
I don't know -- someone posted a comment in the VAIO Pro 13 review and said, "I've heard 8.1 is supposed to further improve battery life" so I tested it and found that at best, battery life remained static and in at least one test it dropped 10% or more.
"Among other things, Windows 8.1 will ship with support for Intel’s Connected Standby technology for Haswell, Direct3D 11.2, and a revised DPI scaling mechanism that is better suited for driving the high DPI displays that are coming down the pipeline for both Ultrabooks and desktops."
Maybe we still need some chipset-driver from intel? did you install Sony's CPPC driver? (Intel Collaborative Processor Performance Control (CPPC) driver).
I hope there's no feature differences between i5-4200u and i7-4500u nor newly available i3-4010u for SVP13.
Connected standby is only of use in increasing battery life while the system is asleep -- it allows portions of the system to wake up, download updates, then go back to sleep. In our tests, we're always on and never shutting of the display, so connected standby is not applicable.
RealTemp 4.00 reports the C8, C9 and C10 package residency time which some Intel low power CPUs support as long as manufacturers remember to enable these in the bios.
Something simple like one bad program or one poorly written driver can significantly reduce the deeper C State residency percentages. Too many laptops are shipped without enough attention being paid to the C States.
Disabling all of the C States used to be an easy way to improve SSD performance. CrystalDiskMark used to show some significant increases in performance depending on what C States were enabled. That might be the reason why Clevo decided to disable the package C States in the laptop you tested.
And people wonder why Apple commands so much respect among the technical elite class. They have made battery life a top priority and they follow through on these details. Only an idiot thinks that people are willing to pay more for a crappy product just because of massive advertising. If that was true the Surface RT would be a top seller.
Excellent laptop, but Sony VAIO Pro 13 set there are many unnecessary applications and programs that load, price. I recommend to remove pre-installed Windows on a Sony VAIO Pro 13 SVP13, fully formatted SSD, and put a clean Windows 8.1 or the new Windows 10 and install all the drivers and utilities. http://sony-vaio-driver-download.blogspot.com/2015...
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
34 Comments
Back to Article
willis936 - Monday, October 21, 2013 - link
I'd like to request an article/chart of haswell laptops and C-state support much like you made recently for mobile device test suite cheats. It's a lot of work but I think it paints a clear picture for consumers and sends a strong message to OEMs. At the very least it makes choosing a laptop easier.JarredWalton - Monday, October 21, 2013 - link
I can provide charts for the Haswell laptops we still have on hand, but most of the previously tested laptops have already been returned. I'll see about doing a follow-up to this post perhaps later today....CecileWamsley - Monday, October 28, 2013 - link
Gabriella got a nice twelve month old Nissan Sentra by working from a home pc... navigate to this website... http://smal.ly/8wUo2andykins - Monday, October 21, 2013 - link
Great post. Thanks for this.sherlockwing - Monday, October 21, 2013 - link
Can we have battery life test of Surface Pro 2 in future ultrabook reviews for comparison reasons?Rainbird01 - Monday, October 21, 2013 - link
It just occurred to me, as I don't recall seeing this information in your reviews, but when testing battery life, I assume you do so through the desktop on Windows 8(.1)? Or with the desktop open at least?It would be nice to know if the battery tests would come out differently if one stayed entirely in the Modern UI (as that would intuitively be the case given the lack of resource control on the desktop).
Staying in Modern doesn't make much sense on a laptop necessarily, but it's something that's nice to know for x86 hybrids (Surface Pro 2 being a prime candidate). :-)
Otherwise, keep up the awesome work! :D
JarredWalton - Monday, October 21, 2013 - link
The benchmarks we run do use IE10 (now IE11) in desktop mode. I can try doing a test using the IE Win8 app, but actually when Win8 first appeared I did a battery life test and the non-desktop IE10 delivered worse battery life. I think the Modern UI may actually put a constant (albeit small) load on the iGPU that's more demanding in terms of power than the desktop.Rainbird01 - Monday, October 21, 2013 - link
It would certainly be nice to know if that is still the case in IE11 and 8.1, and I would also be interested in knowing if it's the case with something like video playback. Just for accuracy's sake. :-)JarredWalton - Monday, October 21, 2013 - link
It will obviously take a bit of time to test, but I'll look into it. I think it might be a good time to do some pure video testing as well using MPC-HC, VLC, Windows Media Player, and the new Video app to see which does best in terms of battery life. It will take about a week, give or take, before I can test all those....MarcAnton - Monday, October 21, 2013 - link
Jarred: Just as a sidenote, Sony is using heavily modified drivers for their Vaio Laptops and I would therefore take the above posted results with a grain of salt as long as there are no "real" Windows 8.1 drivers for the Sony Vaio Pro 13. Other than that I do agree on your statement that PC OEMs definitely need to step up their drivers and continued support.JarredWalton - Monday, October 21, 2013 - link
I installed the latest Intel drivers on the VAIO Pro 13 for testing before the initial review, which is something we do on all laptops. I don't know if the Sony-provided drivers would have changed things, but it's possible.MarcAnton - Monday, October 21, 2013 - link
Thanks. Was not trying to invalidate your results, it just stems from my experiences with Sony Vaio drivers for my Sony Vaio Series S (SA) under Windows 8.1. Most drivers are not even updated for Windows 8.1.JarredWalton - Monday, October 21, 2013 - link
No worries. Installed drivers right now are 15.33.5.3316 -- I think that's the latest from Intel as of last week. The previous battery life testing was completed with the earlier 15.31.17.3257 driver. Maybe that accounts for the change in the Medium test result, but I'm retesting now just for validation....Rainbird01 - Monday, October 21, 2013 - link
That would be very welcome as well. I don't know how far from the release the Modern VLC app is, but it might interesting to compare it to VLC on the desktop.skiboysteve - Monday, October 21, 2013 - link
Jarrod the video windows store app uses entirely different rendering APIs do I imagine the difference would be dramatic.another thing to keep in mind is desktop and all its baggage is not loaded until you tap the desktop tile. So you might get different results if you never tapped it after boot or if you did
JarredWalton - Tuesday, October 22, 2013 - link
Trying to avoid the desktop is essentially impossible and I seriously doubt it's not loaded until used. Think about all the stuff that's on the desktop: Windows Explorer is one, most of the control panel stuff, any non-Modern app, Task Manager, etc. If you had to wait a few extra seconds for a bunch of "baggage" to load you would know it. Even browsing through a file open dialog in a Modern app is likely to use parts of Windows Explorer, so I would bet the desktop is always present in some form with Windows 8. In fact, testing that I did previously indicated that Modern Apps (Music and IE10 at the time) both were less power efficient than the "baggage" desktop IE10 and Windows Media Player -- and the Modern Apps always seem to take way longer to load than they should.skiboysteve - Tuesday, October 22, 2013 - link
Jarrod, yes, according to Microsoft the desktop code is not loaded (in its entirety) unless you enter the desktop at least once. See here:"You can use only new apps and never leave them if you want (in which case all of the desktop code is not even loaded.) "
http://blogs.msdn.com/b/b8/archive/2012/05/18/crea...
I can attest to this because I have a HDD in my desktop running win 8 and now 8.1 and if I hit the desktop within a few hours after boot it takes a while and hard drive banging to load.
I also know that WMP and the Video modern app use different APIs to render video. You can see this in the task manager. one is called protected video pipeline and the other is something else. Back when I had a radeon 4850 in my desktop I had a driver bug where the modern video apps like netflix, hulu, and xbox video would leak ram and crash hard after a while but windows media player of VLC did not. Later I got a geforce card and all is well. This shows it is tied to WDDM more closely so I would assume better battery life
tipoo - Monday, October 21, 2013 - link
It's worth mentioning too that this Vaio seems to fly in the face of the running theory that Windows is inherently bad at idle/light load battery life, and proves it's more down to manufacturers.JarredWalton - Monday, October 21, 2013 - link
Oh, it's almost entirely up to the manufacturers in my opinion -- maybe it's a bit more difficult to do with Windows than say OS X, but it *can* be done. Sadly, even laptops from the same manufacturer can have issues -- so the VAIO Pro 13 is really good on battery life, but perhaps another Sony has issues. Sony is usually pretty good about optimizing for battery life, but HP, Dell, ASUS, etc. are highly variable depending on the model. Budget laptops rarely get any TLC in terms of BIOS/firmware optimizations for example.tipoo - Monday, October 21, 2013 - link
I bring it up because this codinghorror article is making its rounds around the internet, and everyone seems to just "know" windows is the culprit. But this seems like pretty clear evidence that is not the case.http://www.codinghorror.com/blog/2013/10/why-does-...
JarredWalton - Monday, October 21, 2013 - link
I'd have to see if Anand used the "Balanced" or "Power Saver" setting for his Surface Pro 2 review, but it may have been Balanced -- which would mean it would a bit closer to the VAIO Pro 13 result on Power Saver. Also, Anand used our tablet battery life test, which at one point was less demanding than our laptop battery life test but perhaps the latest iteration changed things -- and depending on whether he used IE10/IE11 in Win8 mode or desktop mode, he could also see lower battery life.But there are definitely advantages for the OS X side of things. Someone mentions OS X getting timer coalescing and expecting another 10-15% improvement, but I don't think it will be more than 5%. My bet is OS X is so optimized for the hardware that short of reducing the LCD brightness or something along those lines, it's the maximum we'll see in terms of battery life. Anand ran the MBA13 2013 Haswell model on Win8 as well as OS X and also saw markedly improved battery life with OS X, but that's partly because Apple doesn't have any real need to optimize the Windows side of their laptop story.
jyotib - Monday, October 21, 2013 - link
This article makes for painful reading - battery life for my own Vaio Pro 13 has taken a real hammering after the Win 8.1 update. The culprit seems to be screen brightness - it immediately stretches out to full brightness, regardless of whether the machine is onw balanced or low-power mode, on mains power or battery, and regardless of what the actual screen brightness setting is. any thoughts on sorting this out? What intel updates did you carry out, Jarred?JarredWalton - Monday, October 21, 2013 - link
I had the display brightness get stuck at one point, but a reboot fixed that. I have the latest Sony BIOS and updates from their VAIO Care software, and I have the latest Intel graphics driver. I don't think I did anything else noteworthy. Battery life is for the Power Saver profile, with the LCD set to 87% (200 nits). Incidentally, I did run a test on Balanced at one point and battery life dropped I think around 10%.ananduser - Monday, October 21, 2013 - link
I honestly didn't believe OEM optimizations could carry so much weight. The Vaio Pro, which shares the same chipset, resolution with the Surface Pro, sporting even a slightly smaller battery, simply slaughters the latter. One would assume that by controlling the hardware and software, MS would provide an incredibly tuned machine.JarredWalton - Monday, October 21, 2013 - link
Note that Anand used slightly different tests -- I will need to try and run his workloads to see what they do on the VAIO Pro 13.teiglin - Monday, October 21, 2013 - link
Thanks for your amazing responsiveness Jarred, I'm really looking forward both to the tests that Anand ran on Surface, and the video player comparison you mentioned above.sorten - Tuesday, October 22, 2013 - link
This is the first I've heard that Win 8.1 was supposed to improve battery life. What were they changing in order to optimize it? Improved low power state support? Thanks.JarredWalton - Tuesday, October 22, 2013 - link
I don't know -- someone posted a comment in the VAIO Pro 13 review and said, "I've heard 8.1 is supposed to further improve battery life" so I tested it and found that at best, battery life remained static and in at least one test it dropped 10% or more.juhatus - Tuesday, October 22, 2013 - link
Thanks for the article :)Ryan said it here:
http://www.anandtech.com/show/7425/windows-81-upda...
"Among other things, Windows 8.1 will ship with support for Intel’s Connected Standby technology for Haswell, Direct3D 11.2, and a revised DPI scaling mechanism that is better suited for driving the high DPI displays that are coming down the pipeline for both Ultrabooks and desktops."
and that links to this about connected power states:
http://www.anandtech.com/show/7047/the-haswell-ult...
Maybe we still need some chipset-driver from intel? did you install Sony's CPPC driver? (Intel Collaborative Processor Performance Control (CPPC) driver).
I hope there's no feature differences between i5-4200u and i7-4500u nor newly available i3-4010u for SVP13.
JarredWalton - Tuesday, October 22, 2013 - link
Connected standby is only of use in increasing battery life while the system is asleep -- it allows portions of the system to wake up, download updates, then go back to sleep. In our tests, we're always on and never shutting of the display, so connected standby is not applicable.uncle29 - Wednesday, October 23, 2013 - link
RealTemp 4.00 reports the C8, C9 and C10 package residency time which some Intel low power CPUs support as long as manufacturers remember to enable these in the bios.http://imageshack.us/a/img850/8943/bynx.png
Something simple like one bad program or one poorly written driver can significantly reduce the deeper C State residency percentages. Too many laptops are shipped without enough attention being paid to the C States.
Disabling all of the C States used to be an easy way to improve SSD performance. CrystalDiskMark used to show some significant increases in performance depending on what C States were enabled. That might be the reason why Clevo decided to disable the package C States in the laptop you tested.
Toblerone - Friday, October 25, 2013 - link
Walton, could you perform a similar test for Lenovo Thinkpad, please?Streamlined - Sunday, October 27, 2013 - link
And people wonder why Apple commands so much respect among the technical elite class. They have made battery life a top priority and they follow through on these details. Only an idiot thinks that people are willing to pay more for a crappy product just because of massive advertising. If that was true the Surface RT would be a top seller.Geryhold - Wednesday, March 11, 2015 - link
Excellent laptop, but Sony VAIO Pro 13 set there are many unnecessary applications and programs that load, price. I recommend to remove pre-installed Windows on a Sony VAIO Pro 13 SVP13, fully formatted SSD, and put a clean Windows 8.1 or the new Windows 10 and install all the drivers and utilities. http://sony-vaio-driver-download.blogspot.com/2015...