Revisiting Linux Part 1: A Look at Ubuntu 8.04
by Ryan Smith on August 26, 2009 12:00 AM EST- Posted in
- Linux
File/Networking Performance
Finally we have file and networking performance. As Ubuntu uses a different file system (ext3 versus Windows’ NTFS) there’s the potential for some significant differences here.
Starting with file performance, we will be using a collection of roughly 1500 files, totaling 380MB.
Even after SP1, file performance has long been a thron in the side of Vista. Here it manages an embarrassing loss to Ubuntu, taking over 50% longer to make a copy of the same folder.
Looking at ZIP compression times, it’s an even larger gap. Vista needs 75% longer to compress the same folder. While compression can be CPU bound, looking at our data this specific test is largely I/O bound. We’ve already established at Windows’ built-in ZIP abilities are pretty bad, but we’ve never figured out why this is.
Decompression is even worse for Windows. It takes nearly 4 times as long to decompress the same archive. It’s not even a contest – Ubuntu wins, if only because it’s the only competent operating system out of the two.
Meanwhile in our network copy tests, we are copying that folder to a server running Windows Server 2003. This gives Windows an advantage since we’re using SMB, but since SMB is the predominant protocol for consumer file server gear, it’s a fair test of such use.
Here it’s nearly a dead heat. Both Ubuntu and Vista need just as long to copy our file collection to the server, meanwhile Windows needs a bit less time to copy that same folder off of the server.
Switching gears, using a 2.6GB ISO we see a clear performance difference. In both copying to and from the server, Ubuntu needs at least 50% longer. Since this test isn’t using a lot of CPU time, our best guess is that Windows is doing some buffering that Ubuntu doesn’t get to do. The transfer rates for Linux are below what the hard drives on either end can manage.
Ultimately for users with lots of local storage, Ubuntu appears to outshine Vista. But for users with lots of remote storage (e.g. a NAS), Vista outshines Ubuntu.
Finally we have the amount of time it takes to start up each operating system, another disk-bound test. Vista is not something I would consider particularly speedy, so I’m a bit surprised that Ubuntu did not manage to outperform it here. The 2 second difference is measurable, but small enough that it won’t make any real impact.
Facebook
Twitter
RSS
195 Comments
View All Comments
ParadigmComplex - Wednesday, August 26, 2009 - link
I concur - while most of the article is quite good, Ryan really seemed to have missed quite a bit here. His analysis of it seemed rather limited if not misleading.Not everything *has* to be a package - I have various scripts strewn around, along with Firefox 3.6a1 and a bunch of other things without having them organized properly as .deb's with APT. The packaging system is convenient if you want to use it, but it is not required.
Additionally, Ryan made it seem as though everything must be installed through Synaptic or Add/Remove and that there where no stand-alone installers along the lines of Windows' .msi files. It's quite easy on Ubuntu to download a .deb file and double-click on it. In fact, it's much simpler then Windows' .msi files - there's no questions or hitting next. You just give it your password and it takes care of everything else.
The one area I agree with Ryan is that there needs to be an standardized, easy, GUI fashion to add a repository (both the address and key) to APT. I have no problems with doing things like >>/etc/apt/sources.list, but I could see where others may. I suspect this could be done through a .deb, but I've never seen it done that way.
Ryan Smith - Wednesday, August 26, 2009 - link
Something I've been fishing for here and have not yet seen much of are requests for benchmarks. Part 2 is going to be 9.04 (no multi-Linux comparisons at this point, maybe later) and I'd like to know what you guys would like to see with respect to performance.We'll have a new i7 rig for 9.04, so I'll be taking a look at a few system level things (e.g. startup time) along side a look at what's new between 8.04 and 9.04. I'll also be taking a quick look at some compiler stuff and GPU-related items.
Beyond that the board is open. Are there specific performance areas or applications that you guys would like to see(no laptops, please)? We're open to suggestions, so here's your chance to help us build a testing suite for future Linux articles.
cyriene - Monday, August 31, 2009 - link
I'd like to see differences between PPD in World Community Grid between various Windows and Linux distros.I never really see AT talk about WCG or other distributed computing, but I figure if I'm gonna OC the crap out of my cpu, I might as well put it to good use.
Eeqmcsq - Thursday, August 27, 2009 - link
Cross platform testing is pretty difficult, considering there are a multitude of different apps to accomplish the same task, some faster, some slower. And then there's the compiler optimizations for the same cross platform app as you mentioned in the article. However, I understand that from an end user's perspective, it's all about doing a "task". So just to throw a few ideas out there involving cross platform apps so that it's a bit more comparable...- Image or video conversion using GIMP or vlc.
- Spreadsheet calculations using the Open Office Calc app.
- Performance tests through VMware.
- How about something java related? Java compiling, a java pi calculator app, or some other java single/multi threaded test app.
- Perl or python script tests.
- FTP transfer tests.
- 802.11 b/g/whatever wireless transfer tests.
- Hard drive tests, AHCI. (I read bad things about AMD's AHCI drivers, and that Windows AHCI drivers were OK. What about in Ubuntu?)
- Linux software RAID vs "motherboard RAID", which is usually only available to Windows.
- Linux fat32/NTFS format time/read/write tests vs Windows
- Wasn't there some thread scheduling issues with AMD Cool and Quiet and Windows that dropped AMD's performance? What about in Linux?
While I'm brainstorming, here's a few tests that's more about functionality and features than performance:
- bluetooth connectivity, ip over bluetooth, etc
- printing, detecting local/network printers
- connected accessories, such as ipods, flash drives, or cameras through usb or firewire
- detecting computers on the local network (Places -> Network)
- multi channel audio, multi monitor video
Just for fun:
- Find a Windows virus/trojan/whatever that deletes files, unleash it in Ubuntu through Wine, see how much damage it does.
Veerappan - Thursday, August 27, 2009 - link
I know you've said in previous comments that using Phoronix Test Suite for benchmarking different OSes (e.g. Ubuntu vs Vista) won't work because PTS doesn't install in Windows, but you could probably use a list of the available tests/suites in PTS as a place to get ideas for commonly available programs in Windows/OSX/Linux.I'm pretty sure that Unigine's Tropics/Sanctuary demos/benchmarks are available in Windows, so those could bench OpenGL/graphics.
Maybe either UT2004 or some version of Quake or Doom 3 would work as gaming benchmarks. It's all going to be OpenGL stuff, but it's better than nothing. You could also do WoW in Wine, or Eve under Wine to test some game compatibility/performance.
Once you get VDPAU working, I'd love to see CPU usage comparisons between windows/linux for media playback of H.264 videos. And also, I guess, a test without VDPAU/VAAPI working. Too bad for ATI that XvBA isn't supported yet... might be worth mentioning that in the article.
You also might want to search around for any available OpenCL demos which exist. Nvidia's newest Linux driver supports OpenCL, so that might give you a common platform/API for programs to test.
I've often felt that DVD Shrink runs faster in Wine than in Windows, so the time to run a DVD rip would be nice, but might have legal implications.
Some sort of multitasking benchmark would be nice, but I'm not sure how you'd do it. Yeah, I can see a way of writing a shell script to automatically launch multiple benchmarks simultaneously (and time them all), but the windows side is a little tougher to me (some sort of batch script might work). Web Browsing + File Copy + Transcoding a video (or similar).
Ooh... Encryption performance benchmarks might be nice. Either a test of how many PGP/GPG signs per second, or copying data between a normal disk partition, and a TrueCrypt partition. The TrueCrypt file copy test would be interesting to me, and would cover both encryption performance and some disk I/O.
One last suggestion: Folding@Home benchmarks. F@H is available at least in CPU-driven form in Windows/Linux, and there's F@H benchmark methodologies already developed by other sites (e.g. techreport.com's CPU articles).
Well, that's enough for now. Take or leave the suggestions as you see fit.
haplo602 - Thursday, August 27, 2009 - link
you are out of luck here ... linux does not compare to windows because they are both different architectures. you already did what you could in the article.especialy in a binary distribution like Ubuntu, compilation speed tests are meaningless (but Gentoo folks would kiss your feet for that).
boot up times are also not usefull. the init scripts and even init mechanisms are different from distro to distro.
compression/filesystem benchmarks are half way usable. on windows you only have NTFS these days. on linux there are like 20 different filesystems that you can use (ext3/4, reiser, jfs and xfs are the most used. also quite many distros offer lvm/evms backends or software raid.
I do not think there is much benchmarking you can do that will help in linux vs windows, even ubuntu vs windows because the same benchmars will differ between ubuntu versions.
the only usable types are wine+game vs windows+game, native linux game vs same windows game (mostly limited to unreal and quake angines), some povray/blender tests and application comparisons (like you did with the firefox javascript speed).
GeorgeH - Wednesday, August 26, 2009 - link
Not really a benchmark per se, but I'd be curious to know how the stereotypes of Windows being bloated and Ubuntu being slim and efficient translate to power consumption. Load and idle would be nice, but if at all possible I’d be much more curious to see a comparison of task energy draw, i.e. not so much how long it takes them to finish various tasks, but how much energy they need to finish them.In know that’d be a very difficult test to perform for what will probably be a boring and indeterminate result, but you asked. :)
ioannis - Wednesday, August 26, 2009 - link
is there some kind of cross platform test that can be done to test memory usage? Maybe Firefox on both platforms? not sure.By "no laptops", I presume you mean, no battery tests (therefore power and as a consequence, heat). That would have been nice though. Maybe for those looking for a 'quiet' setup.
but yes, definitely GPU (including video acceleration) and the GCC vs Visual Studio vs Intel compiler arena (along with some technical explanation why there are such huge differences)
ParadigmComplex - Wednesday, August 26, 2009 - link
If you can, find games that are reported to work well under WINE and benchmark those against those running natively in Windows. It'd be interesting to see how the various differences between the two systems, and WINE itself, could effect benchmarks.Fox5 - Wednesday, August 26, 2009 - link
Number 1 use of Ubuntu is probably going to be for netbooks/low end desktops for people who just wanna browse the net.In that case, the browsing experience (including flash) should be investigated.
Boot up time is important.
Performance with differing memory amounts would be nice to see (say 256MB, 512MB, 1GB, and 2GB or higher). Scaling across cpus would be nice.
Ubuntu as a programming environment versus windows would be good to see, including available IDEs and compiler and runtime performance.
Ubuntu as a media server/HTPC would be good to see. Personally, I have my windows box using DAAP shares since Ubuntu interfaces with it much better than Samba. And as an HTPC, XBMC and Boxee are nice, cross-platform apps.
Finally, how Ubuntu compares for more specific applications. For instance, scientific computing, audio editing, video editing, and image manipulation. Can it (with the addition of apps found through it's add/remove programs app) function well-enough in a variety of areas to be an all-around replacement for OSX or Windows?
Speedwise, how do GIMP and Photoshop compare doing similar tasks? Is there anything even on par with Windows Movie Maker?
What's Linux like game wise? Do flash games take a noticeable performance hit? Are the smattering of id software games and quake/super mario bros/tetris/etc clones any good? How does it handle some of the more popular WINE games?