What is SMT and Hyper Threading and how it works

How concurrent multithreading works (SMT – Simultaneous Multi-Threading) known by Intel as Hyper Threading and what do physical cores, logical cores and threads mean? Many people confuse these concepts often, so it’s high time to clarify the most important points that apply to them.

Key concepts

To better describe the operation of SMT technology (which AMD began to use in Ryzen processors), first I have to explain the most important concepts related to processors.

• Physical core (ang. core) is a really existing processor element that has its own individual cycle of operation and cache (L1). In most cases, it performs calculations independently of the other cores, and its parameters are, among others:
  • L1 cache capacity
  • operating frequency in GHz
• The logical core (ang. logical core, thread) is something else. We can define it as a programmatic understanding of the physical core. It depends on the thread on the same physical core (due to resource sharing), but it works independently of threads on other physical cores. Its use is similar to threads run by applications, which is why we call it interchangeably a thread and a logical core.
• Thread (ang. thread), it is simply an invention for the needs of the operating system in the form of a series of tasks that must be dealt with by the processor. Each program consists of one, several, or even hundreds of threads, and any number of threads can appear in the system. The only thing that limits us is the power of our processor, because we can lead to a point where it will not be able to handle all threads simultaneously. This is where Hyper Threading technology takes its toll, helping the physical cores plan their work so that they can cope with the task at hand.

After such explanation, most of you are beginning to understand what the mechanism of concurrent multithreading is, but let’s not rest on our laurels and look at the very genesis of Hyper Threading. It is worth bearing in mind that its variation on AMD Ryzen is simply called SMT.

A dose of history

The basis for the development of this technology was the superscalar architecture, which allowed the execution of several instructions in one processor cycle. In short, it consisted in increasing the independent instructions in the stream (ang. pipeline) by increasing the paths of said instructions. So it can be said that it replaced separated execution units (cores) on executive resources, the number of which could have been much larger and could be responsible for various tasks.

In theory, such a solution increased efficiency, but in practice most of the resources obtained (even the CPU registry!) was simply not used. It was then that the Hyper Threading technology was developed, which allows a single physical core to remain in two states (threads) in an endless loop. It comes down to one being processed and the other waiting (eg for data from memory).

Operation of SMT technology

At any one time, the real core can handle only one of the threads connected to it, because they share its resources. However, this does not affect the logical cores on a different physical core, and the speed at which the processor switches computation on each of them is so fast that we cannot notice it.

Ultimately, Hyper Threading technology (Intel) and SMT can be compared to an employee (physical core), which consists of pens from two parts (application threads). Due to the fact that a single production line (logical core) is not able to deliver both elements on time, the employee has to wait a while and thus loses valuable time. Its efficiency will be highest only when the second tape is triggered, which will allow both elements to be sent at the same time. This is what we call Hyper Threading. This technology enables the physical core to be tasked with enough work to keep it running and never waiting for instructions.

The use of SMT technology

Each program (including games) employs physical and logical cores to work, but not in all cases it uses all of them 100%. Specialized software can make use of a multitude of threads because the tasks it puts on the processor are clearly defined. The CPU knows what type of calculations it has to perform and works on them in parallel, because starting work on task B does not require solving the previous task A. In games, the situation is a bit different, because the work of the processor depends on the player whose movement and decisions cannot be predicted . Due to this, the processor has to complete several tasks one hundred percent before it takes care of the next ones. It’s just like assembling a computer – first we have to insert the processor into the socket, and then mount the cooling.

In games, the processor is responsible, among others for managing artificial intelligence, object physics, responding to signals from the player and partial control of the graphics card. Due to this, most of the calculations have to be done in series, one after the other, so as to maintain a certain order and react appropriately to the player’s movements. But wait a minute … you probably asked why the programmers do not plan the work of all the physical and logical cores so that some of them deal with the main tasks, and the rest (in the background) try to guess the player’s next move. The reason, as always in such cases, is simple – money. It’s extremely problematic to program the game engine to distribute the load on all cores in parallel. Also, working on something that will be used by a small number of players (read with multi-core processors) is a simple waste of time and resources.

When we say that a given game cannot use four cores, what we really mean is that the tasks it puts on the processor take up to four threads (logical cores = computation strings) and it is not possible to divide them. An example of such game optimization can be seen in the case of Intel processors (Core i7) i AMD (Ryzen 5 i 7), which in most cases do not use their power in 100%. This is because these titles require the CPU to calculate impossible to divide, making the performance of systems focused on multitasking worse than those that rely on the power of one core. However, there are exceptions on the market (most often in the form of complex strategic games) that make use of multitasking.

Summary

The logical core, or thread, is a sequence of calculations and instructions invented for the operating system. Each of them is assigned to one of the physical cores whose resources they consume. The SMT solution named by Intel as Hyper Threading (previously, AMD had CMT) allowed to assign to one physical core two logical ones that share the resources of the former and cannot operate in parallel with each other. Their operation is based on the scheduling of the CPU work schedule, i.e. you can say that they ensure that the physical cores can work continuously, which in most cases increases the overall performance of the CPU. However, there are times when multitasking is not usable because the calculations have to be performed one after the other in a specific order. In addition, the implementation of SMT increases the energy consumption and the amount of heat released by the system, but we are talking about really small values.