[nextpage title=”Introduction”]

FB-DIMM (Fully Buffered DIMM) is a memory module technology, targeted at servers, that was recently developed to increase the memory speed and the maximum memory capacity of a server. In this tutorial, we will explain the main features of FB-DIMM modules as well as the differences between DIMM and FB-DIMM modules.

The main difference between FB-DIMM modules and regular DIMM modules is that on FB-DIMM, the communication between the memory controller (chipset) and the module is serial (e.g. PCI Express), whereas on standard DIMM modules this communication is parallel.

Using serial communication, the number of wires needed to connect the chipset to the memory module is lower, which allows the creation of more memory channels, thereby increasing memory performance. With FB-DIMM technology it is possible to have up to eight modules per channel and up to six memory channels, so this technology increases both memory capacity and speed.

Each extra memory channel that is added to the system increases the memory subsystem transfer rate. For example, if you use a single DDR2-533 channel, the memory transfer rate will be 4,264 MB/s. If you use two DDR2-533 channels, as there are on dual channel configuration available today, the memory transfer rate will be 8,528 MB/s. With four channels, the memory transfer rate jumps to 17,056 MB/s, and so on.

FB-DIMM serial communication uses 10 pairs of wires between the chipset and the memory sockets and 12 or 14 pairs of wires between the memory sockets, and the chipset. Each pair of wires uses differential transmission, i.e., the signal is transmitted on a wire and the same signal, but inverted, is transmitted on the other wire from the pair, using the same idea as on twisted pair networking cables. This is done to ensure that the receiver is getting the data correctly. Read this tutorial to understand how this kind of transmission works.

Another important aspect of FB-DIMM is that it uses different paths for data transmission and data reception. Standard DIMM modules use the same path to both transmit and receive data. The system used on FB-DIMM modules helps to increase the performance of the memory subsystem.

Figure 1: How FB-DIMM modules work

Initially, FB-DIMM modules will use DDR2 memory, migrating to DDR3 when this kind of memory becomes available. It is very important to notice that FB-DIMM modules will have the same physical size as DDR2-DIMM modules and will keep this same size when DDR3 memories are released.

[nextpage title=”Motherboard Design”]

There are several advantages in using FB-DIMM memories on servers. We have already mentioned two of them: greater memory capacity and higher performance. There is also another great advantage: the simplification in motherboard design, since the path between the chipset and the memory sockets uses fewer wires.

Even though FB-DIMM uses standard DDR2-DIMM sockets, which have 240 pins, it actually uses only 69 of these pins, making it easier to make the motherboard routing (design of the wires on the printed circuit board).

In Figure 2, you can see the necessary routing to connect the north bridge to the DDR2 sockets using just one memory channel. A three-layer printed circuit board is needed.

Figure 2: DDR2-DIMM routing, one channel

In Figure 3, you can see the necessary routing to connect the north bridge to the FB-DIMM sockets using two channels. It not only uses fewer wires but also uses only two layers from the printed circuit board, making the motherboard simpler to design. (The motherboard still needs to use a three-layer PCB; the third layer is used for power wires). Notice FB-DIMM using two channels utilizes fewer wires than DDR2-DIMM using just one channel.

Figure 3: FB-DIMM routing, two channels

[nextpage title=”Capacity and Performance”]

In Figure 4, you can find a summary regarding FB-DIMM capacity and performance advantages.

Using regular DDR2 memory modules, the maximum memory subsystem you can build is 8 GB and 10 GB/s transfer rate, if you use four DDR2-DIMM sockets, two channels, and DDR2-800 memories.

With FB-DIMM, the maximum capacity jumps to 192 GB. The transfer rate jumps to 40 GB/s if you use 48 modules, six channels, and eight modules per channel, using the same memories (DDR2-800).

Figure 4: DDR2-DIMM vs. FB-DIMM capacity comparison

The FB-DIMM idea is really interesting and should solve the demand for high RAM capacity servers, especially if we think that new CPUs from both Intel and AMD can access far over 4 GB of RAM when using 64-bit operating systems. So far, several memory manufacturers have already announced that they will manufacture FB-DIMM parts, including Samsung, Elpida, Infineon, Micron, Hynix, Nanya and Kingston.