10GbE networking

[newby_investor]

Any homelabbers here?

I've got a couple of HP microservers, currently in a Docker Swarm cluster with Glusterfs for persistent storage. (though I'm thinking of playing around with other things as well.)

I've been toying with the idea of getting 10GbE networking for them, to speed up the intra-cluster communication. Something like this:

ASUS XG-C100F 10 Gb/s PCI-E Network Adapter with Fiber/DAC SFP+ Port

Buy ASUS XG-C100F 10 Gb/s PCI-E Network Adapter with Fiber/DAC SFP+ Port featuring Up to 10 Gb/s Data Throughput, SFP+ Port Module, Supports Fiber & DAC Cabling, Built-In Aluminum Heatsink, Automatic QoS Bandwidth Prioritization, Standard & Low-Profile PCI Brackets. Review ASUS XG-C100F

www.bhphotovideo.com

Has anyone tried using anything like this at home?

 

[MDKza]

If I was you I would check reviews on that Asus card.
If they are positive, I can't see why it wouldn't work for your usage.

 

[newby_investor]

I mean I've looked around a bit. I was just wondering whether anyone local had tried anything like that.

 

[irBosOtter]

I just used a old 10Gb Intel Card I pulled from a server, plugged it into a desktop and was working fine on 10Gb to a 10Gb capable switch, used a DAC cable

 

[newby_investor]

I just used a old 10Gb Intel Card I pulled from a server, plugged it into a desktop and was working fine on 10Gb to a 10Gb capable switch, used a DAC cable

An SFP+ card? Or 10GBASE-T? What switch did you use?

 

[DrJohnZoidberg]

I've been contemplating it but I have other priorities (like more hard drives).

All my machines are hooked up with dual aggregated 1Gbit links which is fine for my needs.

 

[newby_investor]

fine for my needs.

I mean... what I've got is probably fine for my needs.

But wouldn't it be nice if it were faster? ;-)

 

[DrJohnZoidberg]

I mean... what I've got is probably fine for my needs.

But wouldn't it be nice if it were faster? ;-)

Ha, don't tempt me.

My storage is seriously bursting at the seams and plan to add a new 14TB x 4 pool to my TrueNAS box. Maybe after that I'll look at upgrading the network

 

[newby_investor]

Ha, don't tempt me.

My storage is seriously bursting at the seems and plan to add a new 14TB x 4 pool to my TrueNAS box. Maybe after that I'll look at upgrading the network

I'm not running out of storage anytime soon, but file transfers only get about 200 Mb/s, I suspect it's because of the clustered file-system that I am using. The boxes I have only have a single 1GbE NIC and it's doing some hard work (though only when I make it work, most of the time it sits idle).

I'd prefer for it to be a bit faster. Though more hard drives are also always tempting...

 

[irBosOtter]

An SFP+ card? Or 10GBASE-T? What switch did you use?

SFP+ card and the switch is a Cisco C9300 with a 8x10Gb uplink module

 

[The_Donkey]

Ha, don't tempt me.

My storage is seriously bursting at the seams and plan to add a new 14TB x 4 pool to my TrueNAS box. Maybe after that I'll look at upgrading the network

Damn, what are you guys hoarding?

 

[dualmeister]

Damn, what are you guys hoarding?

Does midget porn take up less space than regular porn

 

[MDKza]

I mean I've looked around a bit. I was just wondering whether anyone local had tried anything like that.

Upgrading network speed will only speed things up to a certain point.
If you have cr*p drives then you will get cr*p speed regardless of the networking.

 

[newby_investor]

Upgrading network speed will only speed things up to a certain point.
If you have cr*p drives then you will get cr*p speed regardless of the networking.

Previously having the drives in a RAID5 configuration, I got pretty good speeds, certainly in excess of 500 Mb/s.

I'm imagining though that extra interconnect will be helpful for more than just file transfers, especially for containers to talk to each other.

 

[MDKza]

I am pretty sure you will see a benefit then, the jump from 1 to 10 is substantial.
If you have the cash do it and give us a review

 

[newby_investor]

I am pretty sure you will see a benefit then, the jump from 1 to 10 is substantial.
If you have the cash do it and give us a review

I don't have the cash immediately, maybe in a month or two. Just finished the process of buying a house so it'll take a bit for my finances to recover

But I will post a thorough review along with benchmarks before and after once I'm done.

 

[MDKza]

Keen to hear your outcome!