~6m14:30
Linus Tech Tips

I tried SO HARD to break this… - M1 Mac Mini 10-gig Ethernet

May 13, 2021

Read: ~6m · You save: 9 min

Testing the M1 Mac mini 10-Gig Ethernet: Full-Speed Networking, Shared Bandwidth, and Hidden I/O Limits

Can the M1 Mac mini's 10-gig Ethernet really hit full speed? We test it with displays, SSDs, and hubs to find where bandwidth gets lost and why.

Published May 13, 2021

The M1 Mac mini drew wide praise for its performance and price. It outperformed every previous Intel Mac in single-threaded benchmarks and a Ryzen 5 5600X, one of the fastest single-thread desktop CPUs on the market, at a lower cost. Its multithreaded performance trailed its single-threaded results because it has fewer cores.

Input and output (I/O) has drawn more criticism. The base model includes two USB ports, two Thunderbolt ports, and one gigabit Ethernet port. Users who needed faster networking had two options: a bulky, expensive dongle that occupied a Thunderbolt port, or an Intel platform. Apple had previously offered 10-gigabit Ethernet on the iMac Pro and on Intel-based Mac minis.

This article examines a 10-gigabit M1 Mac mini tested with Iperf in server mode. The tests measured network throughput under increasing load from displays, Thunderbolt and USB storage, and other attached devices, to identify internal bandwidth limits.

Test Objectives

The tests addressed two questions:

  1. Whether Apple delayed the 10-gigabit version of the M1 Mac mini because of performance concerns.
  2. How the M1 Mac mini allocates limited I/O bandwidth across its ports and internal components.

Apple does not publish block diagrams of its platforms, as Intel does. Internal bottlenecks therefore appear only as performance drops during use.

Baseline Network Performance

The first test ran with only network and power cables connected, with no display attached.

Test conditionThroughput (Gbps)
Baseline, no peripherals9.25–9.45

The 10-gigabit line rate is the signaling speed. Actual data throughput is normally lower.

Display Connection

The second test connected a Pro Display XDR. The M1 Mac mini is the first Mac mini to support Display Stream Compression (DSC). DSC leaves enough bandwidth to run the monitor's integrated USB 3 hub at full speed.

Earlier Intel-based Mac minis either did not support the monitor's full 6K resolution or drove it in an uncompressed tiled mode. Some users reported that this mode reduced the USB hub to USB 2 speeds. Uncompressed, the Pro Display XDR requires about 34 Gbps of bandwidth.

Network throughput did not change with the display connected. The test illustrated that the system divides a finite pool of bandwidth among its components.

Thunderbolt Storage Load

The third test added a high-performance Thunderbolt SSD. Anthony adapted a shell script that writes data continuously to an SSD and logs speed at regular intervals.

  • The SSD read speed exceeded 10 Gbps.
  • Apple advertises up to 40 Gbps for each Thunderbolt port.
  • The two Thunderbolt ports did not share bandwidth in this test.

USB-C Storage on the Display Hub

The fourth test connected three USB-C SSDs to the USB hub of the Pro Display XDR and ran the storage script on all three simultaneously.

The SSDs slowed under load, which identified a bottleneck in the display's USB hub. This result did not reveal any internal bottleneck of the M1 Mac mini.

Expanded Load Test

The final load test used five SSDs on the display hub and one additional Thunderbolt 3 SSD on the Thunderbolt daisy chain.

One drive did not appear correctly. Running the script against a missing target increased CPU load. After the drive was recognized, network throughput ranged from 9.3 to 9.42 Gbps with all drives active.

CPU usage rose as additional storage activity was added:

ConditionCPU usage (%)
All drives connected, normal operation18
Additional drive not connected, script running against missing target32–36
Further increase during script runs40
Peak observed77

The script's CPU overhead affected results but did not change the network results. Runaway CPU usage does reduce network transfer performance.

Key Findings

The 10-Gigabit Port Does Not Share Bandwidth

The 10-gigabit Ethernet controller has a dedicated PCI Express Gen 4 lane. It has no switch chip or controller hub in its path. Testing found no bottleneck affecting this port. The test results indicate that Apple's delay was not caused by performance limits.

Storage Throughput Is Limited

  • Theoretical maximum throughput for a single drive tested individually is just over 58 Gbps.
  • Measured real-world throughput peaked near 50 Gbps.
  • Throttling therefore occurred.

The SSDs connected to the Pro Display XDR were limited by its DisplayPort signal, which uses about 13.7 Gbps.

Each Thunderbolt 3 Port Caps at About 20 to 22 Gbps

Each Thunderbolt 3 port reached a ceiling of about 20 to 22 Gbps. The following factors account for the gap between the advertised 40 Gbps and the measured limit:

  • About 7.6 Gbps is reserved for DisplayPort on the Thunderbolt 3 dock.
  • Signal encoding consumes additional bandwidth.
  • The resulting theoretical maximum is about 26 Gbps.

The measured results are close to this figure. The remaining difference is attributed to traffic through busy hubs rather than direct connections. This explains the wavy patterns on the graphs, compared with the steadier results from individual drives.

Recommendations for Latency-Sensitive Work

Applications sensitive to I/O variation, such as audio or real-time data workflows, perform best when devices attach directly to the computer. Where a hub or dock is unavoidable, only one high-speed device should be attached at a time.

For most users, these limits will not affect everyday work.

I/O Architecture

The test results indicate that the M1 Mac mini has four PCI Express lanes allocated to its I/O:

  • Two PCI Express Gen 4 lanes connect to Apple's Thunderbolt controller, which provides two Thunderbolt 3 channels.
  • One Gen 4 lane serves the 10-gigabit Ethernet controller.
  • About 10 Gbps serves the Type-A USB ports. This allocation may share a lane with the Wi-Fi 6 adapter.
  • The HDMI port connects directly to the GPU through an internal DisplayPort-to-HDMI converter.

The macOS terminal command ioreg displays device hierarchies but does not show how devices connect to the main system.

Storage Architecture

Most PC systems provide more than four PCI Express lanes. The M1 does not provide additional lanes for its SSD.

The T2 chip, which served as the storage controller in the final generation of Intel Macs, does not exist in the M1 generation. Reverse-engineering work to run Linux on the M1 indicates that the M1 system-on-chip (SoC) includes the security functions of the former T2 chip. This work also found that the SoC controls the SSD directly through a memory-mapped arrangement rather than through PCI Express.

NVMe is a protocol. It is commonly associated with PCI Express but does not require it.

Bandwidth for the internal GPU has not yet been accounted for in these tests.

Conclusions

  • The M1 Mac mini is probably near the I/O limits of the M1 SoC.
  • The M1 Mac mini has more I/O capacity than the initial analysis indicated.
  • The reason Apple delayed the 10-gigabit model is likely related to logistics or strategy, according to the test authors. The tests did not establish this.

Introduction: Why Port Speed Is Not Guaranteed

Ports do not guarantee full speed. USB, high-speed I/O, and SSD slots often share bandwidth. Apple publishes no block diagrams, so bottlenecks must be found by testing. The video asks whether Apple delayed the 10-gig M1 Mac mini over performance concerns.

  • Presence of a port does not mean it runs at full speed, especially all the time
  • USB speed is shared across multiple ports; high-speed I/O and SSD slots often share resources and slow each other down when used together
  • Early gigabit networking was limited because many motherboards connected ports to the old non-Express PCI bus
  • Intel publishes block diagrams of its platforms; Apple does not
  • Test goals: check whether Apple delayed the 10-gig M1 Mac mini over performance concerns and gain fresh insight into internal bottlenecks

Background: The M1 Mac mini and Its I/O Complaints

The original M1 Mac mini impressed with its single-thread performance and low price but had weak I/O. Users who needed 10-gig networking had to use bulky dongles or Intel platforms.

  • M1 Mac mini beat every previous Intel Mac in single-threaded benchmarks and beat a Ryzen 5 5600X, one of the fastest single-thread desktop CPUs
  • It lacked cores for heavily multithreaded tasks, but this had generated few complaints
  • Mac Pro was offered for users who needed more power
  • Standard I/O was two USB ports, two Thunderbolt ports, and one gigabit Ethernet jack
  • A gigabit limit forced users to use an expensive dongle that consumed a Thunderbolt port or buy a dead-end Intel platform
  • Apple had pushed 10-gig networking with the iMac Pro and then the Mac mini

Baseline Test: Ethernet With Nothing Else Connected

With nothing attached except network and power, the 10-gig port ran stably at about 9.25 to 9.45 Gbps using Iperf in server mode. This is normal, since 10 Gbps is the line speed and real data throughput is lower.

  • Iperf was run with the M1 Mac mini as the server
  • Best-case baseline with nothing plugged in except networking and power, not even a display
  • Measured throughput was 9.25 and 9.45 Gbps
  • 10 gigabit is the line speed; actual data speed is normally somewhat lower
  • The video also included an unrelated merchandise mention (40 ounce water bottles) which is excluded

Display Test: Pro Display XDR and DSC

A Pro Display XDR was connected. The M1 Mac mini is the first mini with Display Stream Compression (DSC), which leaves enough bandwidth for the monitor's USB 3 hub at full speed. Results stayed the same.

  • Pro Display XDR was connected to test the effect of a display on bandwidth
  • The M1 Mac mini is the first mini with DSC (Display Stream Compression)
  • Earlier Intel minis either ran below full 6K resolution or used uncompressed tiled mode, which some users reported knocked the hub back to USB 2 speeds
  • Uncompressed, this monitor would use about 34 Gbps of bandwidth
  • The terminal on the mini matched the SSH session, confirming the test setup
  • Network speed remained exactly the same after the display was connected

Stress Test: SSDs, Thunderbolt, and USB Hub Load

A shell script written by Anthony that constantly writes to an SSD and reports speeds was pointed at a Thunderbolt SSD. Thunderbolt SSD reads exceeded 10 Gbps, but the two Thunderbolt ports did not share bandwidth. Adding USB-C SSDs to the monitor hub caused slowdowns, and a Thunderbolt dock with five SSDs and an extra Thunderbolt 3 drive was used. A drive that did not show up properly caused CPU usage to spike.

  • Anthony's shell script constantly writes to an SSD and outputs speed info at regular intervals for graphing
  • Thunderbolt SSD reads exceeded 10 Gbps, close to the advertised up to 40 Gbps for these ports
  • The two Thunderbolt ports did not share bandwidth with each other
  • Running three USB-C SSDs on the Pro Display XDR hub slowed the SSDs, showing a bottleneck in the hub
  • Five SSDs were plugged into the hub, plus one Thunderbolt 3 SSD on a daisy chain
  • With all drives running, network results were 9.3 to 9.42 Gbps
  • CPU usage was about 18% with all drives plugged in, rising to 32, 36, 40, and 77% when a missing drive was targeted

Analysis: Where Throttling Happens

Analysis shows the 10-gig NIC has its own non-shared Gen 4 PCI Express lane, so no bottleneck is found there. Throttling happens in Thunderbolt 3 and hubs. Each port tops out near 20 to 22 Gbps, and the display signal and encoding overhead reduce the theoretical maximum to about 26 Gbps.

  • The 10-gig M1 port is non-shared, with no switch chips or controller hubs, using a full Gen 4 PCI Express lane
  • Individual drive tests have a theoretical maximum of just over 58 Gbps and reached about 50 Gbps in practice, so throttling did occur
  • Pro Display XDR SSDs were slowed by a 13.7 Gbps DisplayPort signal
  • Each Thunderbolt 3 port appears to cap at about 20 to 22 Gbps
  • About 7.6 Gbps is reserved for the DisplayPort signal in the Thunderbolt 3 dock, and encoding overhead reduces the theoretical maximum to about 26 Gbps
  • Wavy graph patterns are due to busy hubs versus direct attachment
  • Latency-sensitive work such as audio or real-time data should attach directly or use one high-speed device per hub

I/O Architecture: Four PCI Express Lanes and the SSD

The M1 has about four PCI Express lanes of I/O. Two Gen 4 lanes go to Apple's Thunderbolt controller, one goes to the 10-gig NIC, and one may be shared with the Wi-Fi 6 adapter. The HDMI port goes directly to the GPU. The SSD is controlled by the security chip through a memory map instead of PCIe, which explains the limited I/O.

  • Two PCI Express Gen 4 lanes go to Apple's Thunderbolt controller, which steps down to two Thunderbolt 3 channels
  • One Gen 4 lane is used for the 10-gig NIC
  • The Type-A ports share about 10 gigabits and may share a lane with the Wi-Fi 6 adapter
  • HDMI connects directly to the GPU via an internal DisplayPort-to-HDMI converter
  • The ioreg terminal command shows device hierarchies but not how they connect to the main system
  • The M1 SoC contains its security chip, which controls the SSD directly through a memory map rather than PCI Express
  • NVMe is a protocol that does not technically have to run over PCI Express
  • The delay of the 10-gig version was probably related to logistics or strategy