How The Cache Memory Works

Dynamic RAM vs. Static RAM

There are two types of RAM memory: dynamic (DRAM) and static (SRAM). The RAM memory used on the PC is dynamic. On this kind of memory each bit of data is stored inside the memory chip in a tiny capacitor. Capacitors are very small components, meaning that millions of them can be manufactured on a small area – this is called “high density.” On the other hand capacitors lose their charge after some time, so dynamic memories need a recharge process called refresh, which occurs from time to time. During this period data cannot be read or written. Dynamic memory is also cheaper than static memory and also consumes far less power. But, as we saw, on dynamic RAM data is not readily available and it can’t work as fast as the CPU.
Static memory, on the other hand, can work as fast as the CPU, because each data bit is stored on a circuit called flip-flop, which can also deliver data with zero or very small latency, because flip-flops do not require refresh cycles. The problem is that flip-flops require several transistors to be made, i.e., they are far bigger than a single capacitor. This means that on the same area where on a static memory only one flip-flop exists, on a dynamic memory there are hundreds of capacitors. Thus static memories provide a lower density – chips have lower capacity. The other two problems with static memory: it is more expensive and it consumes more power – thus it heats more.
In the table below we summarize the main differences between dynamic RAM (DRAM) and static RAM (SRAM).

Even though static RAM is faster than dynamic RAM, its disadvantages prevent it from being used as the main RAM circuit.
The solution found to lower the impact of using a RAM memory that is slower than the CPU was using a small amount of static RAM between the CPU and the RAM memory. This technique is called memory cache and nowadays this small amount of static memory is located inside the CPU.
The memory cache copies most recently accessed data from the RAM memory to the static memory and tries to guess what data the CPU will ask next, loading them to the static memory before the CPU actually ask for it. The goal is to make the CPU to access the memory cache instead of accessing directly the RAM memory, since it can retrieve data from the memory cache immediately or almost immediately, while it has to wait when accessing data located on RAM. The more the CPU accesses the memory cache instead of the RAM, the fastest the system will be. We will explain exactly how the memory cache works in just a moment.
By the way, here we are using the terms “data” and “instruction” interchangeably because what is stored inside each memory address doesn’t make any difference to the memory.