Setting up a full k8s cluster (vanilla k8s, not k3s etc.) and running most of my self-hosted apps in that cluster caused me a lot of headaches but also got me an immense amount of knowledge and experience.

Setting up exchange server cluster with backups, OWA webmail behind reverse proxy doing IPS+SSO+ MFA, setting up DKIM, DMARC and SPF for this server / testdomain.

Windows PKI using offline and issuing CA. Using these certificates for 802.1x auth.

Hardening Windows Active Directory, setting up LAPS, enforcing TLS where possible, restricting service accounts etc.

Using Azure AD for SAML SSO to where possible. Using JIT or SCIM prorvisioning for accounts. Access roles from groups etc.

Setting up Intune managed workstations with device complience policies and using these policies in conditional access policies.

So yeah mostly Windows stuff.

setting up a pfSense router is cool.

you can use docker to run some local services and give them their own domain names with pfsense

if you want to progress further, you can use traefik to give docker/kubernetes services hostnames and get a cloudflare certificate to enable https on everything

Im a little “seasoned” at 45 rotations around the blue ball so YRMV lol

1 - Pirated Movies - XBMC -> Plex (2 bare metal servers with transition to Virtual in progress)

2 - Storage - Freenas -> Synology (300+TB across 6 units)

3 - Networking - Unmanaged -> Managed / Multi Vendor / LACP / WIFI

4 - Compute - ION Based NUC -> ThinCentre / ML360G6

5 - Virtual Envio - Single ESXI Host -> 5 Node Proxmox Cluster with Ceph on 10G

Basically for me it started with Saving private ryan on VCD / CDR back in the day, that led me down the IRC rabbit hole and i got into XBMC for playback. That meant NAS storage for more movies and then from XBMC i moved into Plex so needed more server based compute and such. All the while needed to learn networking to hook it all up and make it work correctly.

I also went from working at Terminix as a pest control operator to doing customer support via email for a .com in the early 2000’s to now leading technical deployments at a forture 10 pharma :)

All cause of Saving Private Ryan on Pirated VCD in CAMMED quality lol

1. Build your own router, segment your network. I suggest OpenWRT. Openwrt is less stream lined, which means you learn more. You’ll learn trunking, VLANs, sub netting, DNS. Do it all through CLI.

2. Reverse proxy, internal and external. Use Traefik or caddy.

3. Encryption keys. Seems simple. But learn and master ssh keys. The Internet works by communicating from point A to point B. And keys help encrypt the traffic. You should be able to type “ssh hostname” to get into any server you want access to, without the need for a password. Bonus points for finding a secure way to set cronjobs to automatically cycle keys, for security practice.

4. Docker machine. Master docker. Learn docker compose. Everything CLI.

5. Proxmox. Put everything on a VM or container. Create a nas, for storage for your VMs. Bonus if it’s strong enough to run many VMs, you can use to host a instance of any software that you are trying to learn. I for I stance am loading windows server 2022 and multiple windows 10 and 11 instances that I can control.

Do everything through CLI. Take notes on what you did (you won’t remember, it’s ok, no one remembers). Practice documentation.

Setup a Plex/Jellyfin server and then oversubscribe it to family and friends.

A Pfsense/opnsense VM taught me how networking works. Before I set it up, all I knew about was port forwarding. I learned about firewall rules, LAN and WAN, VLANs, VPNs, DNS, Dynamic DNS, reverse proxies, bufferbloat, DHCP, etc.

I’m also learning how to make my own CI/CD pipelines with self hosted GitHub Actions as well as dockerizing applications.

For me it isn’t the specific projects that help me understand IT, it is when things don’t work or break that I learn the most.

I work in IT support day job, Microsoft shop but home lab is all Linux so there isn’t much cross over in terms of specific apps, but the troubleshooting approach and techniques are transferable skills.

Countless projects over the years, but two that stand out in my mind as most formative:

1.) Running a Minecraft server back in the day on CentOS and Ubuntu VPS taught me server administration.

2.) Running my own secret, private network in my college dorm for my friends and I (college provided WiFi sucked) with PFSense at its core taught me the basics of networking.

Software developer here and I had to learn k8s for work (small startup, we didn’t have devops people). We manually were provisioning Debian servers to add more api backends, and it took us one hour per instance. Figured there had to be a better way. So in no particular order:

• docker / containerization.
• secure certificates for everything: nginx / reverse proxy / certificates.
• k8s.

Super steep learning curve. Easier to do on the cloud than it is in the homelab. In the homelab:

• segmenting home network into different VLANs, firewall rules.
• Tailscale for multi-site access.
• cloudflare zero-trust tunnel for secure off-site access by friends.
• reverse proxy backed by let’s encrypt TLS for secure private connections.
• getting all the *arrs setup via docker. plex on nuc, media share on NAS, accessible via NFS. Orchestrating so that either restarting is recoverable.
• Prometheus / grafana for monitoring
• setting up alerts for everything

Current project:

• migrate off docker into k3s on top of metallb for ingress, longhorn for persistent storage, helm for charts, argocd for gitops, ansible for automation. I never want to SSH into a server again. And I want to manage all my infrastructure through a git repository. Totally overkill for the homelab but guaranteed to get you multiple offers in Silicon Valley.

• Installing Server 2016 and learning to provision machines and set group policies in active directory.
• Running Wazuh and lerning how to remediate issues and errors across Windows, Mac, and my Linux machines…or at least learning what they are.
• Using Windows, Mac, and Linux.

I remember one interview where the subject of Home Assistant came up and one of the interviewers was having an issue with his set up and I told him how to fix it. I got an offer from that company.

So, in my experience, a general interest in technology and continuos learning…just because you genuinely like it, helps.

I think setting up active directory domain controller with all the DHCP/DNS and group policies is a number one thing to do, if you don’t know how to do that.

Another thing would be running a Linux server and have a website. Learn how apache and Nginx works. And how to use them together.

It also helped to understand networking and virtual networking from non Cisco perspectives. I have a ccna and net+ and setting up opensense+pihole with network services was very weird, it felt completely different compared to ccna and net+ studies.

Well and of course having experience with virtualization. Learning different types of virtual storage and just in general how virtualization works.

The last thing is options but it is something that I decided to do, that can help you with networking(however there are other things you could set up that would be more useful). I would set up the gns3 server. This would help you with networking, especially if you are trying to study for network certs after ccna. But like I said, there are other projects that you can set up, that will be way more useful as a beginner.

Gonna echo some of the other replies on what I’ve used at home that’s helped me out.

1. Media - Used Windows Media Center on Windows XP/7 to start with > XBMC/Kodi > Plex - on several different machines

2. Networking - First wireless router was a Netgear N750, and it was great until the wife spilled some water on it > Netgear Nighthawk R7000 > Ubiquiti Edgerouter ER-X and UAP-AC-LR Access Point > still using the ER-X router but got a U6-Lite AP and then an Engenius controller and ECS-357 AP > ER-X and Aruba AP315/325 converted to be IAP models.

Got a Meraki MS120-8LP switch for POE for my APs. Ended up getting a bunch of Cisco switches and routers of different models to use at home from my current job. Still haven’t setup a working lab with those yet.

3. Compute - This has been the most recent developments due to getting disposal mini desktops from work. Currently have a 3 node Proxmox cluster with 2 Windows server 2022 eval vms. One is a domain controller and the other is going to be setup for MECM(new acronym for SCCM).

I reckon that’s it for now.

Hypervisor cluster, K8s Cluster, routed Vlans. Learn a lot of IT things building clusters and lot of networking things building out a routed vlan network. Before that just hosting websites, network shares, email, setting up postfix/sendmail running DNS servers. The first stepping stone for me was running a hypervisor so I could build the rest of the things in there.

vlan
k8 cluster.
hpc cluster simulation
GPU cluster simulation
proxmox-/vmware install and management.
building general networking and solving mid level networking issues.