Hardware Secrets Power Supply Test Methodology

[nextpage title=”Introduction”]

We were one of the first hardware review websites to alert users that the vast majority of power supply reviews posted around the web and even on so-called “specialized” magazines were completely wrong. If you want to understand why, please read our article Why 99% of Power Supply Reviews are Wrong. This is recommended reading to understand why we are adopting the methodology described in the present article.

Our power supply reviews cover the following:

• Completely disassembling the power supply for an in-depth analysis from its internal architecture;
• Load tests to see if the power supply is able to deliver its rated power and if it can deliver even more power than labeled;
• Protection tests to see if protections like over current, over power and short-circuit are working correctly;
• Electrical noise tests to see how clean is each power supply voltage output;
• Efficiency tests to see how much power is wasted by the power supply;
• Stability tests to see if there is any voltage fluctuation on the power supply;
• Power factor readings to test the efficiency of the PFC (Power Factor Correction) circuit (valid for tests published after 06/23/2009 only);
• Temperature readings.

We will explain in details each one of these tests, the methodology we are going to use with each one of them, our criteria to label a power supply as “good” or “bad” and the equipment we are going to use.

The idea of this article is to be a reference for all our power supply reviews, so we won’t need to explain our methodology all over again in each review.

[nextpage title=”Load Tester”]

The heart of our testing is our load tester, also known as ATE (Automatic Test Equipment), a SunMoon SM-268, which can be seen on Figures 1, 2 and 3. The basic function of this equipment is to pull the maximum power possible from the power supply being reviewed, but this machine does much more than that, as we will explain.

Figure 1: SunMoon SM-268.

Figure 2: SunMoon SM-268.

Figure 3: SunMoon SM-268.

This load tester allows us to type in five different load patterns, called I1 through I5 (see these buttons in Figure 3). For each load pattern we can set the current the tester will pull from each individual power supply output (+12V, +5V, +5VSB, +3.3 V and -12 V, which on the machine are labeled VA through VF; VG and VH aren’t used – see the displays in Figure 1).

Here it is important to explain something before people get confused on our reviews. This equipment has two separated +12 V inputs, labeled +12V1 and +12V2, which does not necessarily relate with the power supply multiple rails (+12V1, +12V2, +12V3, etc). All plugs that provide +12 V (main motherboard power connector, peripheral power connectors, video card power connector and EPS12V/ATX12V connector) are connected to the machine +12V1 input. The second input is connected only to a second EPS12V/ATX12V connector that is available.

When testing power supplies with single rail design we don’t need to worry much about how we will connect all the plugs. On power supplies with multiple rails, however, we need to think about load distribution, as the machine has only two +12V inputs and the power supply may have more than two +12V rails. What we will basically do is to connect the EPS12V or ATX12V that is connected to an individual rail to +12V2 and all the rest on +12V1 and put this input to draw more current. This should work fine as the current should split evenly between the several connectors (Kirchoff’s Law #2).

On reviews with power supplies with multiple rails we will have to explain how the power supply was connected to the load tester.

So the first step is to program the load tester with the currents (and thus power, as power is given multiplying the current by the voltage of each output) we want to pull from each output. This will depend on each particular power supply, as each power supply has its own particular specs.

In our methodology we decided to make six load tests. First we will test the power supply with 20%, 40%, 60%, 80%, and 100% of its labeled power. Then we will try to see what the maximum power the unit is capable of delivering is, as some good units can deliver more power than what is labeled. All these six loads are pulled from the power supply immediately, meaning that on our methodology the power supply have to be able to deliver these loads as soon as it is turned on.

Just a real example to clarify. Suppose we are reviewing a 500 W power supply. We will conduct complete tests with 20% load (100 W), 40% load (200 W), 60% load (300 W), 80% load (400 W) and 100% load (500W). Then we will check what the maximum power this power supply can deliver right after being turned on is.

During our tests we will concentrate the load on the 12 V outputs, especially on high wattage units (i.e., above 500 W), in order to reflect a typical power supply usage today, as ATX12V, EPS12V and video card connectors have only 12 V wires. Thus in a high-end PC the most part of the power is pulled from the 12 V outputs.

All our load tests will be conducted with a room temperature between 45° C and 50° C (113° F and 122° F). This is a very important aspect of our reviews. The capacity of semiconductors delivering current (and thus power) drops with temperature, a phenomenon called de-rating (click here to read more about this). Many power supplies are labeled at 25° C (77° F), a temperature that is too low and impossible to achieve inside a computer. Because of that many units labeled at 25° C cannot deliver its rated power when used in a real-world environment. We will talk more about temperature later.

Because of the difference between our methodology and the methodology used by some manufacturers a given power supply not passing our load tests doesn’t necessarily mean that the reviewed power supply is bad. For example if we discover that a given 600 W power supply can only deliver 520 W, this doesn’t necessarily mean that this unit is bad; depending on other factors it can be considered a good 520 W model – if the user knows that he or she is bringing home a unit that in real life delivers less that the label says. Of course if the unit is labeled as 600 W and it can only deliver 200 W then it is a completely different story…

The load tester tests a lot more things besides checking if the power supply can deliver its rated power. By pressing a button on its panel we can immediately see if voltages are within the correct range, i.e., if the outputs are stable. The equipment not only shows the current voltage for each output, but also shows an alert whenever any output is out of range.

In our reviews instead of listing the voltages of each output during each load test, we will simply say whether the power supply passed or not on the voltage stability test; if the power supply fails, then we will report values a
nd talk about them. We will consider a 3% tolerance margin for each output, detailed in the table below. This margin is lower than the standard 5% margin (see second table below), so we will be using a tolerance lower than normal. This will help us to qualify the voltage stability of a power supply: if all outputs are below 3% from their nominal voltages this means that we have an excellent power supply. If they are above 3% but below 5% this means that we have a good power supply, but it could have an even better stability. If the power supply is out of the 5% then we are obviously facing a bad power supply, which can even damage your equipment.

Output	Minimum Voltage (3%)	Maximum Voltage (3%)
+12 V	11.64 V	12.36 V
+5 V and +5VSB	4.85 V	5.15 V
+3.3 V	3.20 V	3.40 V
-12 V	-12.36 V	-11.64 V

Output	Minimum Voltage (5%)	Maximum Voltage (5%)
+12 V	11.4 V	12.6 V
+5 V and +5VSB	4.75 V	5.25 V
+3.3 V	3.135 V	3.465 V
-12 V	-12.6 V	-11.4 V

With the load tester we can also test some of the power supply protections. During our load tests we automatically test two of them: over current and over power protection. If the power supply doesn’t have these two protections it will literally burn when we try to go over its rated specs (bad power supplies will burn even within its specs). The load tester also provides separated short-circuit tests for the +12 V and +5 V outputs by just pressing a button. Of course we will also test this feature.

There is one known "flaw" with our load tester that we should talk about. Each of its 12 V inputs has a 33 A limit, meaning that the maximum we can pull from the +12 V outputs using this machine is 792 W (33 A x 2 x 12). This makes our system not suitable for testing power supplies above 900 W. Power supplies between 900 W and 1,000 W can be tested, but pulling more power from +5 V and +3.3 V outputs than from +12 V, a situation that we don’t like since we try to concentrate power on the +12 V outputs, as explained before.

[nextpage title=”Electrical Noise”]

The outputs of the power supply aren’t perfectly continuous. There is a small oscillation called ripple and on top of this oscillation we have some spikes, called noise. We need to see if ripple and noise are within specs: maximum of 120 mV for the 12V outputs and maximum of 50 mV for the 5V and 3.3 V outputs. These numbers are peak-to-peak voltages and the lower, the better. In fact we always like to see noise and ripple below half of these numbers. This is something that multimeters can’t detect and that is one of the several reasons why reviews based solely on multimeters are flawed. To measure ripple and noise we will use an oscilloscope.

Since ripple and noise aren’t in the range of MHz we can use a cheap PC-based oscilloscope, and in our case we bought a Stingray DS1M12 from USB Instruments. This equipment is simply an analog-to-digital converter (ADC) with a program that collects data sent by the ADC and plots a chart on the screen.

Figure 4: Stingray PC-based oscilloscope.

Our load tester has a BNC connector for installing an oscilloscope, allowing us to monitor any one of the power supply outputs (there is a switch where we can choose which output we will monitor). During our tests we will monitor each power supply output for each load pattern. Whenever possible we will try to bring details of the noise level when the power supply was delivering its maximum power because it is usually under this scenario that we usually find the highest noise level.

In Figure 5, you can see an example of the output presented by the Stingray scope. Here we were monitoring noise from the +12 V output of a power supply and since we were using the 0.01 V/div scale (i.e., the distance between each horizontal line represents 0.01 V or 10 mV) the peak-to-peak voltage is a little bit above 20 mV, well under the maximum admissible noise – which is great, by the way. If you can’t see this the program tells you the peak-to-peak voltage, the RMS voltage and the frequency of the noise in a human-readable format (Figure 6).

Figure 5: Output from the oscilloscope.

Figure 6: Data measured by the oscilloscope program.

One final note. The ATX12V specification states that ripple and noise should be measured with a 0.1 µF ceramic capacitor and a 10 µF electrolytic capacitor attached to the oscilloscope probe. Our load tester has these capacitors behind its panel, so we don’t need to add them. This is another advantage of having a professional load tester.

[nextpage title=”Efficiency Tests”]

For each load test we will measure efficiency. Efficiency is how much power the power supply wastes in the process of converting AC power into DC. For example if a given power supply is delivering 500 W on its outputs but it is pulling 650 W from the power grid, this means that the power supply is wasting 150 W. The problem with this wasted power is that you are paying for it but you are not using it!

In this example, this power supply would have an efficiency of 77% (500 W/650 W). Good power supplies have an efficiency of at least 80%. The higher this number, the better, as it will mean lower electricity bills.

Measuring efficiency is easy. For each load test we know how much power the power supply is delivering, since this number is being shown on our load tester display (this value is published in our reviews as "Total"). Then we have how much power the unit being tested is pulling from the power grid, which is published in our reviews as "AC power". Efficiency is calculated dividing the DC power by the AC power.

AC power is measured by a precision digital meter GWInstek GPM-8212. This is a very expensive equipment, mainly because it provides a 0.2% precision, which can’t be achieved with cheap units like Kill-a-Watt and Brand Electronics. It is important to note that we bought this equipment in June 2009 and before we used a Brand Electronics 4-1850 power meter, which is not so precise. Power measured with this Brand Electronics unit gives higher readings, making efficiency results to be greater than they should be. In other words, the efficiency numbers published on reviews posted before 06/23/2009 are a little bit higher than they should be, and you should keep this in mind when comparing new reviews to old ones. We are re-testing several units but we won’t be able to fix all reviews. 

Figure 7: GWInstek GPM-8212 precision power meter.

Another advantage of this equipment compared to cheaper solutions is that it allows calibration via PC, so from time to time we can recalibrate this unit to make sure it is accurate.

This instrument also measures the AC voltage and the power factor (PF) and we are adding this information on reviews posted after we bought it. Power factor equals to active power divided by apparent power and is a number between 0 and 1. The closer this number is to 1, the better. This result measures the efficiency of the power factor correction (PFC) circuit from the power supply. Click here to understand more about this subject.

[nextpage title=”Temperature”]

As we have already explained, temperature is a very important aspect on true power supply reviews, since semiconductors lose their ability to deliver current (and thus power) as the temperature increases.

Several power supply manufacturers label their products at 25° C, which is an unrealistic temperature. Inside the computer case the temperature is far higher than that. Because of this we will conduct our tests with a room temperature between 45° and 50° C.

We measure temperature with a Fluke 52 II precision digital thermometer, which has accuracy of 0.05% + 0.3° C. This instrument was bought in May 2009 and on reviews posted before that we used a CompuNurse digital thermometer, which isn’t so precise (that is why the Fluke thermometer costs over USD 200 and the CompuNurse costs less than USD 20). This thermometer has two probes. One is used to measure temperature inside our "hot box", while the other is used to measure the temperature on the surface of the power supply, measured on its top side.

Figure 8: Fluke 52 II precision digital thermometer.

Instead of buying a thermal chamber (a.k.a. “incubator”), which would allow us to set the exact temperature we want the power supply to run under, we decided to build our own "hot box". The goal of this "hot box" is to keep temperature around the power supply between 45° C and 50° C (between 113° F and 122° F). To increase the temperature inside the "hot box" we installed a duct connecting the exhaust system from our load tester, which blows hot air, to inside the box. The box itself is constructed in MDF. We built this box on 06/23/2009. On reviews posted before this date we used a cardboard box (we used the box from our Sony home theater receiver), which, despite its amateur appearance, worked just fine.

Figure 9: Our “hot box.”

Figure 10: Duct connecting the exhaust system from our load tester to the "hot box".

Figure 11: Location of the thermometer probe.

Next to our box we have a five pound ABC fire extinguisher (not shown on the picture) for any emergency.

Before starting our load tests we will keep the power supply running until we get the temperature inside the box with at least 45° C. We can increase or decrease the temperature inside the box by opening or closing its front acrylic cover. We also added a 110 V fan connected to a dimmer on one of the sides of the box so we can better manage the temperature inside the box.

Figure 12: Fan and dimmer.

Some may argue that we could install the power supply inside a case. Actually this isn’t a good choice for several reasons. First, the length of the power supply cables wouldn’t allow us to do this. Second, we would need to keep the case open to install the cables from the power supply on the load tester. Third, we would need to have a running system inside the case in order to generate an amount of heat compatible with the one produced by a real PC, and that would be impossible to accomplish as we would need another power supply feeding this system – and where should we install it? So we would need to keep the case open for using this second power supply and since the case is open, the whole idea of using a PC case goes down the drain (we would need the case closed to really simulate a typical PC).

In fact even using a commercial thermal chamber we would be faced with some of the challenges exposed above: the length of the power supply cables and the need to open a hole on the chamber to pass these cables from the power supply to the load tester.

[nextpage title=”Pictures From The Test Bench”]

Here are a couple of pictures of our power supply test bench.

Figure 13: Power supply test bench.

Figure 14: Power supply test bench.

[nextpage title=”Known “Flaws” in Our Methodology”]

Here are some of the things people may criticize on our methodology, but even with the criticism we are very confident that we are using a good methodology:

• Our load tester is limited to 33 A on each +12 V input, meaning that we are can only pull up to 792 W from the +12 V outputs. So we can only test power supplies up to 900 W. Power supplies above that can be tested, but by pulling more power from the +5 V and +3.3 V outputs instead from the +12 V outputs.
• We won’t test the power supply installed on a typical PC. We will test it exclusively with our load tester. We think this is the best approach.
• So far we don’t measure acoustic noise level, but we may change this in the near futu
  re, as we bought a noise level meter but we are still not using it. The main challenge is that we have other items generating noise near the product being reviewed, like the load tester.
• We are not using any AC conditioner or simulator. Some websites go to the extent of simulating spikes on the AC line to see how the power supply reacts. We think that is too much for us.

By the way if you want to compare our methodology to the one used by other leading reviewing websites, here are the links to their methodology:

• AnandTech (USA)
• CanardPC (Canada)
• Clube do Hardware (Brazil)
• HardwareHeaven (UK)
• Extreme Overclocking (USA)
• HardOCP (USA)
• HardwareLogic (USA)
• JonnyGURU (USA)
• Overclock3D (UK)
• PC-Experience (Germany)
• PC-Max (Germany)
• PC Perspective (USA)
• Planet3dnow (Germany)
• SPCR (Canada)
• Sweclockers (Sweden)
• Technic3D (Germany)
• TheLab.gr (Greece)
• The Tech Report (USA)
• Tom’s Hardware Guide (they haven’t published power supply reviews in the past 3 years) (HQ in France)
• VR-Zone (Singapore)
• X-bit labs (USA)

If you know about any other website that uses a load tester to test power supplies please let us know so we can add it on the list above. This list is short because, like we always say, 99% of power supply reviews are wrong.

[nextpage title=”How Much We Have Invested so Far”]

Most publications don’t have a decent methodology for reviewing power supplies for two reasons. The first is lack of knowledge. And the second is not willing to invest money to build a lab for testing power supplies. We decided to break down exactly how much money we have invested to date to build our lab, so you can have an idea of our commitment in reviewing power supplies and bringing you with real information about this hardware part. The costs listed below include tax and shipping, when applicable. We are not including the costs of items like solder, tools like screwdrivers and pliers, the computer necessary for using the oscilloscope, etc. We hope this list also helps other publications to build their own lab. Just don’t forget to mention who was your source!

Equipment	Cost	Obs
SunMoon SM-268 load tester	USD 2,500.00
GWInstek GPM-8212R power meter	USD 761.37
Fluke 52 II thermometer	USD 228.65
Stingray DS1M12 oscilloscope	USD 226.91
Stingray DS1M12 oscilloscope	USD 226.91	Backup
Hakko 808 dessoldering gun	USD 201.38
Brand Electronics 4-1850 power meter	USD 149.42	Retired
Sound level meter	USD 99.90	Not in use yet
Anti-static mat	USD 89.59
Hot box	USD 96.20	96.2
5 Lbs ABC fire extinguisher	USD 53.63
Weller SPG40 soldering iron + WLC100 station	USD 44.95
Kill-a-Watt P4400 power meter	USD 41.41	Backup
CompuNurse digital thermometer	USD 14.99	Broke
Total	USD 4,735.31