cmc/cleberg.net
My personal web garden & blog.
clone: git clone https://gitbay.org/cmc/cleberg.net.git
main: content/blog/2026-07-30-10gbps.org · raw
1#+date: [2026-07-30 Thu 17:14:00]
2#+title: Upgrading My Homelab to 10 Gbps
3#+description: Moving my homelab off 1 Gbps and onto a 10 Gbps backbone.
4#+slug: 10gbps
5#+filetags: :self-hosting:
6
7* Why 10 Gbps?
8
9The short answer: fiber finally became available at my current residence, so I
10jumped on it immediately.
11
12The slightly longer answer is that I'd previously enjoyed proper 1 Gbps fiber,
13but for a while now I'd been stuck limping along on "1 Gbps" cable from Cox
14that, in reality, often crawled along at 100 Mbps regardless of what I did to
15fix it. New modem, new cables, different ports, calls to support - nothing
16mattered. It ran at whatever speed it felt like that day.
17
18So when [[https://fiber.google.com][Google Fiber]] showed up in my neighborhood, I ordered the 8 Gbps package
19immediately. I'll upgrade to their 10 Gbps package the moment it's released, but
20I'm not exactly suffering in the meantime.
21
22A quick note on the title before we go further: my /internet/ is 8 Gbps for now,
23but this post is really about rebuilding my /internal/ network to 10 Gbps end to
24end. The WAN is just the excuse to upgrade everything.
25
26#+caption: My WAN status over the past month.
27#+attr_html: :alt A chart of my WAN status and activity over the past month.
28https://img.cleberg.net/blog/20260730-10gbps/wan_history.webp
29
30In the screenshot above, you can see that Google Fiber was installed on 7/20 and
31UniFi entered WAN failover mode (the blue bar) and then I unplugged Cox on 7/22,
32resulting in the green Google Fiber bar.
33
34As you can also see, Cox was a /train wreck/ - cycling between yellow
35(disconnections, lag, etc.) and red (no internet connection) for the previous
36month.
37
38* What I Was Running Before
39
40Nothing exotic. My network was built entirely on 1 Gbps gear:
41
42- A 24-port UniFi switch in the rack.
43- Two 8-port UniFi switches serving other rooms.
44- A Cox "1 Gbps" WAN that, as mentioned, frequently delivered closer to 100
45 Mbps.
46
47It worked. For years, it was more than enough - I've written before about how my
48[[https://cleberg.net/blog/homelab-maintenance.html][homelab basically maintains itself]], and the network was no exception. But once I
49had real bandwidth coming into the house, 1 Gbps internally became the
50bottleneck instead of the ISP.
51
52As a result, I installed the hardware I'll list in the next section. You can see
53the 10G connections in blue and the 1G connections in green.
54
55#+caption: My homelab infrastructure.
56#+attr_html: :alt A network diagram of my homelab infrastructure.
57https://img.cleberg.net/blog/20260730-10gbps/infrastructure.webp
58
59* The Hardware
60
61Everything here is UniFi. I'm already knee-deep in the ecosystem, and at this
62point fighting it would cost me more time than it's worth. Staying in one
63platform means everything shows up in a single controller and I don't have to
64think about it - which is exactly how I like my homelab.
65
66#+caption: My homelab rack.
67#+attr_html: :alt A photo of my homelab rack of UniFi gear.
68https://img.cleberg.net/blog/20260730-10gbps/rack.webp
69
70This rack shows:
71- UDM Pro router.
72- 10G 24-port switch.
73- Switch panel with keystone jacks.
74- 1G 24-port switch.
75- My server.
76
77This is *before* the Google Fiber installation, so you can see that the WAN RJ45
78port is in use on the UDM Pro instead of the SFP+ port.
79
80I also removed the old 1G switch after this picture and plan to sell it. I just
81didn't want to box it up quite yet when I took this photo.
82
83** Switch
84
85Two switches did the heavy lifting for the upgrade:
86
87- [[https://store.ui.com/us/en/category/switching-professional-max-xg/products/usw-pro-hd-24-poe?variant=usw-pro-hd-24-poe][USW Pro HD 24 PoE]]: the new core switch in the rack, replacing my old 1 Gbps
88 24-port unit.
89- [[https://store.ui.com/us/en/category/switching-utility/products/usw-pro-xg-8-poe?variant=usw-pro-xg-8-poe][USW Pro XG 8 PoE]]: a smaller 10G switch serving another room, replacing one of
90 the old 8-port units.
91
92** NICs
93
94Neither of my two most important machines shipped with 10G, so both needed a new
95network card:
96
97*Windows Gaming PC*: an Intel X520-DA1, a 10G SFP+ card.
98
99#+caption: X520-DA1 NIC for Windows
100#+attr_html: :alt A photo of the X520-DA1 NIC product image.
101[[https://img.cleberg.net/blog/20260730-10gbps/x520-da1.webp]]
102
103*Server*: an Intel X710-DA2, a dual-port 10G SFP+ card.
104
105#+caption: X710-DA2 NIC for Linux
106#+attr_html: :alt A photo of the X710-DA2 product image.
107[[https://img.cleberg.net/blog/20260730-10gbps/x710-da2.webp]]
108
109The Ubuntu server already had Intel drivers installed, so I wasn't required to
110install anything. I simply slotted the NIC into the PCIe port on the motherboard
111and it worked.
112
113#+caption: The 10G NIC in my server.
114#+attr_html: :alt A photo of the 10G NIC installed in my server.
115https://img.cleberg.net/blog/20260730-10gbps/server_nic.webp
116
117However, the Windows machine was a different story. It did come with an
118installation CD-ROM, but I of course ignored that since I have no disc drive in
119my PC. I tried to find the right drivers online, but Intel's websites are a vast
120wasteland of useless downloads and circular links.
121
122Eventually, I pulled a USB-C disc reader from storage and used the CD, which
123magically installed on the first try and everything started working from there.
124
125That'll teach me (not really).
126
127#+caption: The 10G NIC in my Windows gaming PC.
128#+attr_html: :alt A photo of the 10G NIC installed in my Windows gaming PC.
129https://img.cleberg.net/blog/20260730-10gbps/desktop_nic.webp
130
131** Cabling
132
133This is where 10G gets fiddly, and where most of my shopping list ended up.
134Going 10G means leaving RJ45 behind for SFP+ almost everywhere:
135
136- [[https://store.ui.com/us/en/category/accessories-modules-fiber/collections/accessories-pro-direct-attach-cables/products/uacc-cm-rj45-mg?variant=uacc-cm-rj45-mg][SFP+ to RJ45 Adapter]]: bridges the Google Fiber RJ45 handoff into the SFP+ port
137 on my UDM Pro.
138- [[https://store.ui.com/us/en/category/accessories-cables-dacs/collections/accessories-pro-patch-cables/products/uacc-cable-patch-el?variant=uacc-cable-patch-el-0dot15m-w][UniFi Etherlighting Patch Cable]]: short in-rack runs between the switch, the
139 other rack devices, and the fiber modem.
140- [[https://store.ui.com/us/en/category/accessories-modules-fiber/collections/accessories-pro-direct-attach-cables/products/10gbps-direct-attach-cable?variant=uacc-dac-sfp10-0dot5m][10G Direct Attach Cable (DAC)]]: the short SFP+ runs from the rack switch to the
141 server, and from the second switch to the gaming PC.
142- [[https://store.ui.com/us/en/category/accessories-modules-fiber/collections/accessories-pro-direct-attach-cables/products/10-gbps-active-optical-cable?variant=uacc-aoc-sfp10-20m][10G Active Optical Cable (AOC)]]: the long run between the rack and the 8-port
143 switch in the other room, where a copper DAC won't reach.
144
145** Cost
146
147None of this is cheap, so here's the full damage:
148
149| Item | Qty | Price |
150|---------------------------+-----+-----------|
151| USW Pro HD 24 PoE | 1 | $999.00 |
152| USW Pro XG 8 PoE | 1 | $499.00 |
153| SFP+ to RJ45 Adapter | 1 | $65.00 |
154| UniFi Etherlighting Patch | 27 | $112.97 |
155| Cat6A Keystone Coupler | 2 | $78.00 |
156| 10G Direct Attach Cable | 3 | $43.00 |
157| 10G Long-Range DAC (AOC) | 1 | $55.00 |
158| Intel X520-DA1 NIC | 1 | $45.00 |
159| Intel X710-DA2 NIC | 1 | $145.00 |
160|---------------------------+-----+-----------|
161| *Total* | | $2,041.97 |
162
163* The Upgrade
164
165With the parts in hand, the actual topology is straightforward. Bandwidth flows
166in from the street and fans out through the rack:
167
168#+begin_src
169Google Fiber
170 │ (RJ45 → SFP+ adapter)
171 ▼
172UDM Pro
173 │ (Etherlighting patch)
174 ▼
175USW Pro HD 24 (core switch)
176 ├─ DAC ──────────► Server
177 ├─ AOC ──────────► USW Pro XG 8 (other room)
178 │ └─ DAC ──► Windows Gaming PC
179 └─ ETH ──────────► USW 8-port (1G switch)
180#+end_src
181
182The UDM Pro takes the Google Fiber handoff through the RJ45-to-SFP+ adapter,
183then hands everything down to the new 24-port switch over an Etherlighting patch
184cable. From there, the server hangs directly off the core switch on a short DAC,
185and a long active optical cable carries 10G across the house to the second
186switch, which finally feeds my gaming PC. My surviving 1 Gbps 8-port switch
187still hangs off the core switch too, handling everything that doesn't need 10G.
188
189The physical installation was pretty easy. I needed to replace a drive in my
190server's ZFS pool, so I already needed to get inside the rack. Since I don't use
191rails, I had to unscrew and pull out each device by hand.
192
193From there, the NIC (and HDD) installation in the server was seamless. Just pop
194out one of the metal slots on the back of the server, slot the NIC into the PCIe
195lane, and line up the port with the new opening in the back panel.
196
197All in all, it was easy and simple. No physical snags, only one driver issue on
198the Windows machine (which was my fault for ignoring the provided CD), cabling
199runs were easy, and UniFi updates were minimal.
200
201* Benchmarks
202
203Let's nerd out on the numbers.
204
205** Before
206
207I don't actually have any speed tests from before the upgrade, but the Cox
208connection had degraded even further by the end - down to roughly 70 Mbps for
209the month prior to the upgrade.
210
211Occasionally, I was able to get ~800 Mbps when the connection would work
212properly.
213
214** After
215
216So, here's the catch. I'm not getting the full 8 Gbps currently. When the Google
217tech installed the modem, we confirmed ~ 8500 Mbps together, which means the
218fiber is working through the ONT and the modem.
219
220However, they would not connect directly to my router, which means the Google
221modem is in bridge mode for now.
222
223The speed test below shows an average speed, where I'm getting 5 to 6 Gbps to my
224router. While not the full 8 Gbps, I'm not saturating it anyway, so I haven't
225dug into my UniFi settings to determine what's causing the difference.
226
227#+caption: Speed test from the UniFi controller.
228#+attr_html: :alt A screenshot of a speed test result from the UniFi controller.
229https://img.cleberg.net/blog/20260730-10gbps/speed_unifi.webp
230
231A supplementary test showing speeds directly to my server:
232
233#+caption: Speed test from my server.
234#+attr_html: :alt A screenshot of a speed test result from my server.
235https://img.cleberg.net/blog/20260730-10gbps/speed_server.webp
236
237* Was It Worth It?
238
239Here's the honest part.
240
241Do I /need/ 10 Gbps between my gaming PC and my server? Probably not. The number
242of times a day I move enough data to saturate even 1 Gbps is small, and for most
243of what I do, this changes nothing I'll consciously notice.
244
245But that was never really the point. I finally have an internet connection that
246delivers what I pay for, and a network that won't be the bottleneck for years to
247come. After spending so long fighting a "1 Gbps" connection that never once hit
2481 Gbps, there's something deeply satisfying about watching a transfer actually
249move as fast as the hardware allows.
250
251Instead of constantly monitoring speeds and pausing services so our
252work-from-home connections aren't impacted, I can let my server idle at 400 Mbps
25324/7, not to mention all the other devices in my home.
254
255Was it worth it? For the peace of mind alone: yes.