For anyone interested in running a raspberry pi based kubernetes cluster at home, I wrote a tutorial/example repo that deploys the cluster using Talos (minimal and hardened OS for kubernetes) and Terraform for cluster services. I plan on making another version soon using ArgoCD instead of Terraform, possibly other versions in the future as well (e.g. flux, crossplane, etc).
It has all the bells and whistles such as:
- Talos - Minimal and hardened operating system and tools that deploy and manage kubernetes nodes/clusters.
- Virtual (shared) IP address for the talos and Kubernetes endpoints
- MetalLB - Load balancers using virtual/shared IPs
- metrics-server - Provide metrics for Kubernetes autoscaling (e.g. horizontal pod autoscaler)
- cert-manager - Automated TLS certificate management
- Rook-Ceph - Distributed block, object and file storage
- Prometheus - Monitoring and alerting
- Full monitoring of your cluster! We gather metrics from just about every service that has them.
- Loki - Log aggregation
- Grafana - Visualize and explore metrics, logs and other data.
- Since we use the kube-prometheus-stack helm chart a bunch of dashboards are pre-generated for you. We also automatically deploy dashboards for monitoring rook-ceph.
- Alertmanager - Send Prometheus alerts to email, PagerDuty, etc.
- Vertical Pod Autoscaler - Suggest or automatically adjust resource limits and requests for pods.
My test cluster is:
- 3x rpi4-4GB as control plane nodes
- 4x rpi4-8GB as worker nodes, with 128GB flash drives attached for storage (distributed storage using rook-ceph)
- Effectively silent cooling with 2x 120mm fans running at 5V (ignore the Arduino being abused as a 5V power supply, that’s been changed out).
- 3D printed housing, modified from a design I found on Thingiverse (though I plan on doing a complete redesign for a different form-factor).
