• 0 Posts
  • 2 Comments
Joined 11 months ago
cake
Cake day: October 18th, 2023

help-circle
  • I use proxmox with ceph and cephfs.

    I have two ceph pools. One with ssd/nvme, and the other with slow rpm hdd.

    The ceph pool with ssd holds my vhds for vms. I only install the os, on that drive…and any necessary programs…Docker, portainer, etc. Then I mount my two cephfs into each vm.

    I don’t use lxc…they can’t be migrated between proxmox nodes. I just install Ubuntu server and Docker.

    Since you mentioned a windows vm…cephfs can be mounted in Windows as well…no need to setup samba. No need to have data trapped in different vhds. Useful for mapping drives on laptops and desktops in your house too.

    Cephfs is my central storage that all servers and containers have access too. If you want to have a NAS gui for users, just map your container to your existing files. I do this for nextcloud. I also run containers like sftpgo, filebrowser, syncthing, etc…all mapped to the two cephfs pools so I can expose and share single data pool via different methods and not have a single vm running samba that’s a single point of failure.

    Cephfs and proxmox support clustering, so if you need to grow or upgrade on the future. If you use proxmox for the second server, ceph and cephfs can extend the pool to the second server and moving vms is seamless because the storage is one big pool now. Run some vms on the new server and some on the old…with a single storage pool.

    Ceph can grow while online, and is fault tolerant like raid and zfs.

    Proxmox will also provide a web interface and console access to your VMs.

    I group my Docker containers by function. So I’ll have one Ubuntu Docker server for the Starr apps…then I have another with paperless, Nginx ubiquity controller, filebrowser, sftpgo…then another Ubuntu Docker for nextcloud because it spins up like 10 containers on its own. Another for Plex and jellyfin. Just trying to keep the chaos organized and because I have more than one server…I can move the one of the Docker servers to another proxmox node in my cluster to manually balance cpu and compute.

    Then when it’s time to upgrade, I’m not rebuilding everything…I’m just adding a small node with a small cost (small IT budget at home too 😂)