Hyper Threading CPU vs. Dual, Multi-Core

In 1968, IBM had introduced a new concept that could enable Software Threads at the lowest level transparent to the application developers, this was known as SMT or Simultaneous Multithreading.

Sure, it was a great innovation but several years later was copied and re-branded as Hyper Threading (HT) by Intel #mce_temp_url#in it’s x86 processors. Intel claims it to produce two parallel threads without loss in performance. Well, it’s not true.

AMD has recently talked about why it doesn’t opt for Hyper threading, rather lays multicores as it’s core business strategy.

To understand this, lets go back to basics.

Threads vs. Cores

Cores are physical hardware blocks in the processor that can run applications serially. Threads, on the other hand, aren’t physical –  they are software-generated tasks that can execute
independently. A well threaded program will run itself across multiple cores.

As Intel quotes it:

HT adds circuitry and functionality into a traditional processor to enable one physical processor to appear as two separate logical processors. The added circuitry enables the processor to maintain two separate architectural states and separate APIC which provides multi-processor interrupt management and incorporates both static and dynamic symmetric interrupt distribution across all processors. The shared resources include items such as cache, registers, and execution units to execute two separate programs or two threads simultaneously.

As AMD quotes it:

So – cores are like bikes, threads are the riders. Running more threads increases throughput for applications as long as you have available cores. If you have threads waiting to be scheduled and no available cores – you have a bottleneck.

Think of SMT and HT as a tandem bike.  Yes it can move two riders, but not as quickly or efficiently as two separate bikes.

The hard thing for SMT & HT is that, it forces two threads to share a single  physical core.  When one thread pauses or stops due to a wait, then the second thread can jump in and take over.  But, the second thread needs to unload the cache, load its data, then, when finished, reload the original data for the first thread – this is what bring a big overhead.  This  helped with processors that had long pipelines of request.  Todays’ generation and next gen CPU have shorter pipelines make this type of effort  less important.

To be precise, HT is nothing more than a layer of abstraction added to the “processor->kernel” interaction that lets CPU decide how to handle multitasking rather than letting Operating system doing that.

All in all, HT can give applications as much as an extra 10-25% increase in performance by making better use of CPU, which is like a poor option over adding a new core.

Hyper Threading in the Real world

These, limitations were just theoretical. Practically, it’s even worse. Leaving desktop applications aside, it almost fails everywhere. In the production environments I’ve seen HT causing serious performance degrades and often admins just have to switch it off.

Hyper-Threading will increase the latency of individual requests and trade that for increased throughput; any individual request takes longer to complete, but more requests can be completed in a given time period. Going from uni-processor to multi-processor also adds cache contention and the increased locking complexity also slows things down; so performance will vary depending on what you are doing.

Larger multi-processor/Hyper-Threading systems with more than one physical processor tend to do better because their Hyper-Threading companion processors are more likely to be idle and halted. But as the load increases, those processors wake up and the overall latency in the system increases.

• When your bottleneck is device I/O, whether LAN or Disk, Hyper-Threading does not increase performance.
• When you have a single operation/thread generating high CPU utilization, Hyper-Threading does not increase performance.

I remember the time when we deployed my company’s Supplychain J2ee based applications on unix sandboxes, Java performance degrade of  upto 30% was observed. That was the first time I learned the myth about hyper threading.

Today, almost every production environment has mandated to stay away from it. Here are few examples:

• Novell calls Disabling HT a cool solution.
• Cognos, a leading BI software by IBM, recommends  disabling HyperThreading for better performance, stablity.
• Microsoft recommends turning off HyperThreading when running PeopleSoft applications because “our lab testing has shown little or no improvement.”
• A Microsoft TechNet article recommends disabling Hyper-threading for production Exchange servers and “only enabled if absolutely necessary as a temporary measure to increase CPU capacity until additional hardware can be obtained.”
• Advanced Clustering found when running High Performance Linpack (HPL) that “Using HT on the other hand causes a ~10% drop in performance compared to HT not being used.”

Ofcourse HT can provide a cheaper and power efficient solution for desktop by giving 10-20% performance benefit but as desktop apps and Operating Systems are becoming more capable, intelligent, demanding and hence more Multithreaded, HT loses it’s purpose.

We need more cores. Sun SPARC has 8 cores, AMD and Intel too will have them soon. We need to continue this innovation more than looking at the cheaper alternatives that add little value.

We write on Hardware, Open Source, Programming, Tech news, gadgets,  Get them @taranfx on Twitter