Home(lab) for the Holidays!

A few years ago, I wanted a homelab to dig deeper into my virtualization journey and play with things that I wouldn’t normally do in Production. At one point, a fellow vExpert had given me their former lab when they upgraded. It consisted of three Shuttle boxes and some miscellaneous other gear. I attempted to make a VSAN cluster out of it, but the performance was atrocious. I attributed this to my very meager budget and my inability to even buy high end consumer-grade SSDs. I tried a few more times to resurrect those systems as a lab, but ended up scrapping them when ESXi 6.7 was released and the CPUs were no longer supported.

I wanted to be able to use Production-like equipment without the need to make so many exceptions. I wanted to be able to have a VCSA with a responsive UI because it wasn’t starved for resources in a lab. Honestly… I wanted too much. I didn’t have the budget to make those desires real. At that point, I decided that maybe a homelab wasn’t for me.

Fast forward three years… things are definitely looking up on the homelab front! In fact, I’ve just finished with the basic setup of my new lab environment and I couldn’t be more excited to start playing.

The New Gear

So far, I’ve been able to acquire six pieces of equipment for use in my homelab. I’m planning to (eventually) break things into Compute and Management clusters to make things a little more power-efficient.

Compute Cluster

  • One (1) Dell PowerEdge R730
    • 14-core Intel Xeon E5-2697 v3 @ 2.60GHz
    • 64GB DDR4 Memory
    • Quad GbE Daughter Card
    • (8) SAS/SATA Bays
  • One (1) Dell PowerEdge R730
    • 16-core Intel Xeon E5-2698 v3 @ 2.30GHz
    • 64GB DDR4 Memory
    • Quad GbE Daughter Card
    • (8) SAS/SATA Bays

Management Cluster

  • Three (3) SuperMicro SuperServer 5018A-FTN4
    • 8-Core Intel Atom C2758 @ 2.40GHz
    • (1) 8GB, (2) 16GB DDR3 Memory
    • Quad GbE Ports
    • (2) SATA Bays

Other Gear

  • One (1) SuperMicro SuperServer 5018A-FTN4
    • Same specs as above
    • Dedicated pfSense firewall
  • One (1) Cisco SG200-50 50-Port Gigabit Smart Switch
  • One (1) SysRacks PR 12.900 12U Portable Rack Enclosure

 

When I first started to bring this gear home, I had only the two PowerEdge servers, the switch (and a bonus HP ProCurve), and a monitor. I had nowhere to put rack-mountable servers in my apartment. For about a month or so, they sat on a folding table until I could figure out what to do with them.

image6
There was nothing flattering about this setup, but it worked.

I already knew that I was going to be bringing some more equipment to the apartment. I also knew that I wanted a more friendly way to store the lab. I had read online about some people making Ikea furniture into a makeshift rack or sorts, but I didn’t know whether I could trust that with the weight of the two PowerEdge systems. I decided to pick up a SysRacks 12U Portable Rack. I have to admit that I think it was a great purchase!

image5
Not a bad 12U rack. Note the box in the reflection – I wasted no time putting the servers in.

As of this evening, I’ve managed to fully populate the rack with all of the systems, the switch, and the firewall. After a week of (so-so) effort, I’ve finally figured out how to move traffic between my local LAN and the lab in the rack. (Note: The firewall is in the rack and not the LAN itself.)

image2 (1)
Servers mounted in the front. Firewall and switch in the rear.

Future Plans…

Compute Cluster

I’ve got the bare-bones of my lab created. Prior to getting the SuperMicro stuff, I had deployed a VCSA and vRealize Operations onto the PowerEdge systems. I only have a handful of 7200 RPM drives in there – I’m definitely not getting a lot of performance out of them. I didn’t bother with any RAID since I don’t have an abundance of drives. I’m sure I could have squeezed a couple more IOPs out of what’s there.

I’m going to try to deploy a 2-node VSAN cluster using some Crucial SSDs with one 500GB for caching and one 1TB for capacity per node. From there, I want to play a bit with Distributed Power Management and see how VSAN handles being shut down completely and then turned back on with frequency. I’ve not had a lot of experience with VSAN just yet, so I’m really interested to see how this works out.

Long-term goal of the Compute Cluster is that it mostly only runs when I’m actively working on something. That’s not to say that the systems won’t be on for a week or more at a time, but I want to be able to kill them off on occasion if I need to.

Expansion of the Compute Cluster isn’t incredibly likely unless I start to really have a good time with the lab. I think I have additional RAM for the systems already that should bring me up from 64GB to 128GB per host. I may get more as time moves on. I briefly researched getting a second processor for each host and decided against it. Should I need more compute, that is the likely next step.

Management Cluster

The Management Cluster will need some work to make it viable. I need to buy more RAM for the systems in order to run some of the desired workloads. Each of the systems is capable of up to 64GB DDR3 RAM in a SO-DIMM form factor which should be cheap-ish to outfit. Additionally, I’ll either need to use VSAN for this cluster as well or find some other homelab-worthy NAS to use for shared storage.

Because of the CPU, I need to be conscious of what I’m running in the Management Cluster. The intention is for my VCSA, a VSAN Witness Node for the Compute Cluster, and my domain controller. Down the road, I might introduce some Horizon bits so that I can remotely access the lab. That’s much farther into the lab though.

Blog Fodder

One of the reasons I finally decided to dive into this was because of the serendipitous opportunity to acquire some equipment. Another reason was so that I could dig a little deeper into things in the lab that I didn’t necessarily have a need to dig into in Production. Lastly, I wanted to be able to use the lab to create some content!

A handful of ideas that I want to tackle:

  • Setting up pfSense and getting it to work behind a commercial router (because I struggled with it being new to pfSense and altogether pretty weak in networking)
  • A comparison of the crappy consumer-grade storage that I’ve got in the lab – 7k HDD, no-name SSD, and 2-node VSAN.
  • Finishing my AppDefense series! It feels like I started that ages ago – I just couldn’t move forward using bits of “Production” as my lab for that series.
  • Taking a look at vRealize Automation 8 after finally getting the hang of vRA 7.x
  • A ton of other stuff that’ll sit in Draft for a long time before I publish it…

 

The most important part is that I have enough to keep going. It’s time to fall in love with virtualization again!

2019-12-24_22-29-52

 

Final shot of the rack to give a sizing perspective – you can see the table it was sitting on at the edge of the photo. This thing has Active Cooling with two exhaust fans on the top for when it gets toasty in there. It came with its own 8-port PDU. It’s on wheels for easy movement. It’s pretty awesome!

image0 (1)

 

Happy Holidays!

One thought on “Home(lab) for the Holidays!

Leave a Reply