Gigabyte MZ01-CE0 AMD EPYC Workstation Motherboard Review

1. Introduction2. MZ01-CE0 (rev. 1.0): Board Layout and Features3. MZ01-CE0 (rev. 1.0): UEFI BIOS4. Testing Methodology and Configuration5. Tests: Maxon Cinebench R156. Tests: SiSoft Sandra7. Tests: Blender Gooseberry8. Tests: SPECviewperf 139. Tests: Disk Performance10. Closing Thoughts11. View All Pages

AMD has really been taking the fight to Intel over the last year. Ryzen has made AMD a viable CPU option for gamers and enthusiasts while Threadripper is outstanding for workstations. But arguably the big money – and the most important battle – is at the highest end of the market. This is where EPYC comes in, and industrial-strength motherboards like the Gigabyte MZ01-CE0 (rev. 1.0) on test here.

Although the EPYC CPUs look similar in format to Threadripper, and the contacts on the bottom appear the same, the sockets are actually different. Threadripper uses TR4, while EPYC uses SP3. The key difference is in the memory controller. Where Threadripper offers quad-channel memory, EPYC provides an incredible eight channels for immense bandwidth. The basis for EPYC, like Threadripper, is four Ryzen 7 CPUs in one package, and with EPYC the dual-channel memory controllers on all four CPUs are active.

At the moment, EPYC is available like Threadripper with up to 32 cores. But the architecture also supports dual sockets, so you can have up to 64 physical cores and 128 virtual ones in one system. The switch to 7nm will also mean that up to eight  CPUs can be packaged together, for 64 cores per socket with the next ‘Rome' iteration.

AMD is really pushing the single-socket option, however, because you can already get 32 cores even with just one CPU. The Gigabyte MZ01-CE0 (rev. 1.0) is aimed at this market. It's more of a server than workstation motherboard, however. Aside from the CPU and eight DIMM slots for the eight-channel memory, you get four LAN connections – two Gigabit, and two 10Gigabit. There's also a dedicated management port that uses the same RJ45 connector.

While there's a slot for a single M.2 NVMe drive, there are actually no SATA connections. Instead, you get four SlimSAS ports, each of which supports up to four SATA drives apiece. So this motherboard could readily form the hub of a hugely capacious storage platform. There are also five PCI Express connectors – four x16-sized and one x8 slot. So this is a serious piece of kit, and at $700, costs a serious amount of money. Let's take a closer look.

Features:

• AMD EPYC 7000-series Socket SP3 processors
• Eight-channel memory up to DDR4 2,666MHz
• Up to 1024GB memory with LRDIMMs
• 4 x PCI Express x16 slots (all at x16); 1 x PCI Express x8 slot (at x8)
• 2 x Gbit LAN; 2 x 10Gbit LAN; 1 x Gbit management LAN
• 4 x SlimSAS for 16 x SATA III 6Gbits/sec ports
• 1 x M.2 NVMe PCI Express Gen3 x4
• Integrated Aspeed AST2500 video and I/O controller
• 3 x USB 3.0 (2 rear, 1 via header), 4 x LAN, 1 x VGA, 1 x MLAN
• Supports: Microsoft Windows 10, Server 2012 R2 64-bit, Server 2016; Red Hat Enterprise Linux 7.4, SUSE Linux Enterprise Server 11.4 and 12.2, Ubuntu 16.04 and 17.04; VMware ESXi 6.5

Price for this motherboard (at the time of writing): $700
We were sent a bare board in a brown box, so we've skipped talking about the bundle and headed straight to the board itself.

This is a standard ATX-sized board, albeit absolutely crammed full of slots and connectors. The eight RAM slots and huge Socket SP3 really dominate their half of the board.

 

The board supports a single 14nm AMD EPYC 7000 series processor, with up to 32 cores, 64 threads and a TDP up to 180W. We were sent an EPYC 7551P for this review. There's a version without the P that is quite a bit more expensive and aimed at dual-socket configurations, but otherwise has virtually the same characteristics.

It would be quite easy to think that Socket SP3 is the same as Socket TR4. It looks the same, and you have the same trio of screws to undo before you can slide the CPU into its little frame, after which you screw them in again to secure, but in the opposite order to when you're opening the socket.

The eight DIMM slots sit four on each side of the CPU socket. These support DDR4 RDIMMs up to 64GB and LRDIMMs up to 128GB, which seems a bit small for a server-oriented motherboard. The maximum frequency is 2,666MHz, and you will want to populate all of them to take advantage of the eight-channel memory controller. So you will probably want to go for the maximum from the word go, as upgrading will require replacement of the entire complement.

Although Threadripper seemed well endowed with 64 PCI Express lanes, EPYC goes even further, offering 128 PCI Express lanes. This gives a clear-advantage with GPU compute configurations on a motherboard like the MZ01-CE0, because it can offer four PCI Express x16 slots that all operate at the full x16, alongside a PCI Express x8 slot that can operate at x8, and PCI Express Gen3 x4 NVMe.

There is only a single M.2 slot, for a boot drive SSD, but you also get the four SlimSAS connectors. You can't hook up a SATA hard disk to one of these directly, however. Instead, you will need to get hold of breakout cables, although one routing a SlimSAS to four SATA III 6Gb/sec connections is only about £15.

Each of the four SlimSAS ports supports up to four SATA drives, so you could have 16 drives in total. Given that SATA hard disks have already reached 14TB each, you could create a 224TB storage monster with this motherboard – assuming you could find a chassis with 16 bays for 3.5in hard drives.

The motherboard is peppered with additional connectors. There are six system fan headers, a USB 3.0 header, two serial COM port headers, one for a TPM (although this isn't installed as standard), a HDD backplane board header, PMBus and IPMB management connectors.

However, the backplane itself doesn't have much connectivity, apart from the plethora of Ethernet ports. One of these LAN ports is dedicated to remote management, above two USB 3.0 ports, with two Gigabit Ethernet ports to the left and two 10Gigabit Ethernet ports on the right. Furthest right is a VGA video output, driven by the ASPEED AST2500 chipset. On the far left are power and reset buttons.

Overall, this board is primarily aimed at server rather than workstation users, and could do particularly well in a Hyper-Converged Infrastructure environment where you need loads of cores and local storage in one box. The hefty networking provisions would lend themselves well to this usage scenario as well.

After the joys of highly graphical and mouse-driven UEFI BIOSes, the MZ01-CE0's text-based BIOS feels like a retro trip back to the 1990s. It also doesn't have built-in support for taking screengrabs, so we had to take photos of the menus on our curved desktop screen, and we apologise for the bowed appearance of the  following images.

The initial screen is informational, telling you about the motherboard model number and build date, MAC Addresses for the four Ethernet ports, and memory speed. The only things you can edit here are the system date and time.

The Advanced section is not full of settings that the average desktop computer BIOS user will be used to seeing given primary prominence. The first entry is for iSCSI, where you can configure initiators for this virtual form of storage connection. Then there are the entries for the four different Ethernet adapters, all of which support VLANs.

Then there's the Trusted Computing option, although this motherboard doesn't have a TPM built in. You can configure the two serial ports provided by the AST2500 chipset.

After a section where you can get the system to wake up at a specific time, and yet more options for using the above serial ports for console redirection, we finally come to the CPU configuration. However, unlike a consumer-grade motherboard, you can't tweak many settings. You can enable/disable CPU virtualisation and memory encryption, but the CPU performance characteristics are merely informational.

Next, you can configure how the lanes are distributed across the PCI Express slots, followed by USB behaviour. Then more networking, and information on the NVMe drive you have attached.

We've broken out the SATA Configuration to show how each SlimSAS port supports up to four drives. But you will need a special adapter to break out the single SlimSAS port to four SATA connections.

The AMD CBS section is actually home to some CPU tweaking settings, under Zen Common Options. This includes enabling/disabling boost mode, C-States and disabling cores. You can also enable disable the L1 and L2 cache prefetchers.

Under Chipset you can enable/disable SMT, configure some more PCI Express behaviour, and view information about the memory DIMMs attached to the North Bridge. Finally, there are some settings regarding error management.

Server Management includes FRB-2 (fault-resistant boot) timing options and an OS Watchdog for when booting up. You can configure system event logging, and view FRU information so you can quickly change out the motherboard for an exact match in the case of a fault. Finally, there are some more network options.

The usual options are available for adding an administrator and a user password. The former protects entry into the BIOS, the latter won't let you boot until you enter the password.

Last but not least are the usual options for choosing boot order, and whether to operate in Legacy or UEFI mode, although it's hard to believe this is actually a UEFI BIOS.

Overall, while there are lots of industrial-grade configuration options here, the design is like something out of a bygone era. Obviously, this isn't a huge issue for a motherboard that should be spending very little time being configured, and lots of time just running smoothly. But it's not always easy to find specific options where, for example, networking settings can be found in at least three different places.

For testing, we installed the Gigabyte MZ01-CE0 into an Armari R64E-GS1280 workstation. This is custom-designed chassis from Armari with features like externally accessible hot-swappable drive bays, cold-swap dual redundant PSUs (although our example only had one), and a tough steel construction.

Our install was based around an EPYC 7551P processor, which is the top of the range specifically aimed at single-socket systems. There is a 7601 for dual-socket systems that runs at a few hundred MHz faster, but is twice the price. The 7551P has 32 cores and 64 threads, with a base 2GHz clock and top 3GHz boost mode, although if you're using all 32 cores the maximum speed is 2.55GHz. Unlike the 32-core Threadripper 2990WX, this is a 180W CPU, so doesn't require such potent cooling.

In order to take advantage of the eight-channel memory controller, we populated all eight DIMM slots with a matched set of eight 16GB 2,666MHz DDR4 modules, for a grand total of 128GB. This is likely to cost around £900 on its own, and the processor around £2,100, so these two components alone are £3,000.

For graphics, we kept things in the AMD family and went for the Radeon Pro WX 9100. This high-end 3D accelerator sports 4,096 Stream Processors running at a base 1,200MHz with a 1,500MHz boost. There's a healthy 16GB of HBM2 memory on a 2,048-bit bus running at 945MHz and delivering a blistering 484GB/sec bandwidth.

Rounding off the test specification was a Samsung PM981 M.2 NVMe SSD with 1TB capacity.

We ran five different tests. First, Maxon Cinebench R15's rendering and OpenGL tests, to assess the base performance with the installed processor and graphics. Then we ran the gruelling Blender Gooseberry render, followed by the memory bandwidth benchmark from SiSoft Sandra. To see the effect of the CPU on graphics performance, we ran SPECviewperf 13, and then finally CrystalDiskMark 6 to test the SSD throughput.

We compared these results with a variety of previous test systems, which we will note alongside the tests as we go along.

Motherboard test specification:

• OS: Windows 10 64-bit
• CPU: AMD EPYC 7551P @2GHz
• Memory: 128GB (8 x 16GB) 2,666MHz DDR4 SDRAM
• Graphics: 16B HBM2 AMD Radeon Pro WX 9100
• Storage: 1TB Samung PM981 M.2 NVMe SSD

Tests:
Maxon Cinebench R15 – all-core CPU benchmark and OpenGL graphics performance
Blender Gooseberry – all-core real-world render test
SiSoft Sandra – memory bandwidth
SPECviewperf 13 – professional graphics performance
Crystal DiskMark 6 – storage transfer rates

Maxon Cinebench R15

CINEBENCH R15 is a cross-platform testing suite that measures hardware performance and is the de facto standard benchmarking tool for leading companies and trade journals for conducting real-world hardware performance tests. With the new Release 15, systems with up to 256 threads can be tested.

CINEBENCH is available for both Windows and OS X and is used by almost all hardware manufacturers and trade journals for comparing CPUs and graphics cards. For this test, we used the Armari Magnetar V25R-RA750G2, Scan 3XS GW-HTX35, Armari Magnetar S16T-RW850G2 and S18X-RD850G2, and Armari Magnetar S32-RD1000G2 for comparison.

Although the EPYC 7551P has the same number of cores as the Threadripper 2990WX in the Armari Magnetar S32T-RD1000G2, they are running at 2.55GHz with the EPYC, and at 4GHz with the (overclocked) 2990WX. This has a clear effect on performance, with a 32 per cent deficit compared to the 2990WX.

In fact, the EPYC is only 17 per cent faster than the 16-core Threadripper 1950X in the Armari Magnetar S16T-RW850G2, and five per cent faster than the 18-core Intel Core i9-7980XE.

The Open GL result is even more disappointing, because this test is very clock-speed sensitive. Because the EPYC 7551P has a maximum single-core frequency of just 3GHz, this holds back the AMD Radeon Pro WX 9100 graphics considerably.

SiSoft Sandra Memory Bandwidth

SiSoft Sandra is a synthetic benchmark, but we wanted to illustrate how much more memory bandwidth you get with the eight-channel controller compared to quad-channel. So we compared the Gigabyte MZ01-CE0 / AMD EPYC 7551P combination to the Armari Magnetar S16T-RW850G2, which includes an AMD Threadripper 1950X.

The EPYC 7551P has more than twice the memory bandwidth of the Threadripper 1950X, despite the former using 2,666MHz memory and the latter 2,933MHz. For applications that read and write a lot of data to and from main memory, the EPYC is going to have a real edge over any architecture that merely offers quad-channel memory.

Blender 2.79b: Gooseberry Production Benchmark

Blender is a free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modelling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation. The latest version at the time of writing, 2.79b, supports rendering on the GPU as well as the CPU. In GPU mode, it will render using OpenCL with AMD graphics cards, and CUDA with NVIDIA graphics cards.

For this test, we used the Gooseberry Production Benchmark. Project Gooseberry is the code name for the Blender Institute's 6th open movie, Cosmos Laundromat — a 10-minute short, the pilot for the planned first-ever free/open source animated feature film. The benchmark renders a single frame from this film in intermediate quality.

For this test, we used the Armari Magnetar S16T-RW850G2 and S18X-RD850G2, and Armari Magnetar S32-RD1000G2 for comparison.

These results are more promising for the EPYC 7551P, because this test takes much longer than the Maxon Cinebench R15 render and the CPU is likely to hit thermal peaks that cause throttling with the more aggressively overclocked systems. Where the EPYC was 32 per cent behind with Cinebench, it'is now just 22 per cent slower, although the Intel Core i9-7980XE isn't that far behind either.

However, in workstation and server multi-threaded workloads, brute performance isn't the only factor to consider. We also monitored power draw during this render. The AMD Threadripper 1950X-equipped Magnetar S16T-RW850G2 was consuming a peak of 470.3W, the Intel Core i9-7980XE-equipped S18-RD850G2 a peak of 458.3W, and the AMD Threadripper 2990WX-equipped S32T-RD1000G2 a peak of 530.1W.

But the Gigabyte MZ01-CE0 and AMD EPYC 7551P combination was maxing out at 238.2W. You're getting a lot more work per Watt than any of the comparison systems, which could save a lot on the power bill. Even the idle power draw is notably less.

SPECviewperf 13

The SPECviewperf 13 benchmark is the worldwide standard for measuring graphics performance based on professional applications. The benchmark measures the 3D graphics performance of systems running under the OpenGL and Direct X application programming interfaces. The benchmark’s workloads, called viewsets, represent graphics content and behaviour from actual applications.

The latest version is SPECviewperf 13, released on May 23, 2018. SPECgpc members at the time of V13 release include AMD, Dell, Fujitsu, HP, Intel, Lenovo, and NVIDIA.

SPECviewperf 13 is a comprehensive upgrade of previous versions of the benchmark. Medical and energy viewsets incorporate new models and raycasting for volume visualization; the Maya viewset features new models based on the SPECapc for Maya 2017 benchmark; and the Creo viewset has been updated with fresh application traces. All other viewsets have been recompiled with minor changes. Results from SPECviewperf 13 are not comparable to those from earlier versions.

For this test, we only compared the Gigabyte setup to the Armari Magnetar S32-RD1000G2, as it was using the same AMD Radeon Pro WX 9100 graphics.

The Gigabyte MZ01-CE0 and AMD Radeon Pro WX 9100 combination is a little behind the Armari Magnetar S32T-RD1000G2 in most viewsets, although the more recent drivers in the MZ01-CE0 have shown improvements in medical-02 and showcase-02. The sw-04 SolidWorks viewset is quite sensitive to clock speed, so the EPYC 7551P is holding back the WX 9100 graphics a little here.

Clearly, the EPYC is not the best workstation platform if you're primarily focused on modelling, but it's also not that far behind its workstation-focused Threadripper sibling.

SSD Performance

The Samsung PM981 M.2 NVMe SSD is a very fast storage device. These results are very similar to the performance of the PM981 on other platforms, implying that the NVMe connection is just as fast as on more regular desktop and workstation motherboards.

The Gigabyte MZ01-CE0 is a hefty piece of kit, and certainly won't be a gateway for enthusiasts hoping to enter the high-end world of EPYC. But it has some great characteristics for certain types of workstation and server applications. If you're looking to create a workstation with huge GPU compute capability, the four full-speed x16 PCI Express slots will be highly beneficial.

For a video editing workstation needing massive storage, the SlimSAS and support for 16 SATA drives will make this easy to achieve. Similarly, the board will be ideal for a storage server or Hyper-Converged Infrastructure virtualisation server. AMD's EPYC, with its 32 cores and eight-channel memory, will be great for running lots of virtual machines too.

Our biggest dislike is the BIOS design, which is uncomfortable to operate now we're used to the highly graphical UEFI interfaces we see with most consumer-grade motherboards. There are lots of features for the intended user type there, but tracking them down can take quite a few keyboard presses.

Overall, though, this is a powerful and capable motherboard. We only managed to find one vendor in the US selling it, but the $700 price is quite reasonable for this kind of product. It's extortionate compared to a consumer, enthusiast or pure workstation motherboard, but is par for the course for a server board with this feature set.

In other words, this is a great platform for our first look at EPYC.

We found the MZ01-CE0 for $700 from GamePC in the US HERE. We don't yet have a buy link for a UK retailer.

Pros:

• EPYC CPUs offer eight-channel memory.
• Four full-speed PCI Express x16 slots.
• Supports up to 16 SATA drives via SlimSAS adapters.
• Two Gigabit Ethernet and two 10Gigabit Ethernet LAN connections.

Cons:

• Old-fashioned, keyboard-only BIOS interface.
• SATA connections require SlimSAS adapters.

KitGuru says: The Gigabyte MZ01-CE0 is a powerful platform for EPYC servers and workstations looking to offer huge amounts of SATA storage or multi-GPU configurations.

Become a Patron!