This document provides context about the laptop-local-clusters project for AI assistants.
Purpose: Standardize local Kubernetes cluster testing across multiple laptops (macOS and Linux) with optional centralized GitOps management via homelab ArgoCD.
Key Technologies:
- k3d (k3s in Docker) - Lightweight local Kubernetes clusters
- k3s - Lightweight production Kubernetes distribution
- Cilium (optional, eBPF-based CNI) - For ArgoCD-managed clusters
- Flannel (default CNI) - For standalone clusters
- ArgoCD - GitOps continuous delivery
- Docker Desktop / Podman Desktop - Container runtime
1. Standalone Mode
- Single laptop with independent cluster
- Uses k3s default Flannel CNI (fast, simple)
- Optional ArgoCD installed locally
- Full autonomy, no external dependencies
- Ultra-fast setup: ~15 seconds
2. Multi-Cluster GitOps Mode
- Multiple laptop clusters managed by centralized homelab ArgoCD
- Core applications (Cilium with ingress, monitoring, cert-manager) synced from Git
- Testing applications deployed manually per laptop
- Consistent infrastructure across all laptops
- ArgoCD installs Cilium CNI (Flannel disabled)
Homelab Cluster (ArgoCD)
↓ (via VPN/Tailscale)
Git Repository (core-apps)
↓
Laptop Clusters (labeled: environment=laptop)
├─ Core Apps (ArgoCD-managed, synced)
└─ Testing Apps (manual kubectl/helm)
.
├── README.md # Quick start guide
├── CLUSTER_PLAN.md # Detailed feasibility analysis
├── MULTI_CLUSTER_SETUP.md # Multi-cluster setup guide
├── CLAUDE.md # This file
├── create-cluster-k3d.sh # k3d cluster creation (PRIMARY SCRIPT)
├── create-cluster.sh # Legacy Talos script (deprecated)
├── install-argocd.sh # Install ArgoCD locally (standalone mode)
├── cilium-patch.yaml # Legacy Talos config (not used with k3d)
├── scripts/
│ ├── README.md # Scripts documentation
│ └── register-cluster.sh # Register laptop with homelab ArgoCD
├── argocd-apps/
│ ├── README.md # ArgoCD apps documentation
│ ├── bootstrap-laptop-management.yaml # Bootstrap app (apply once to homelab)
│ ├── laptop-clusters-applicationset.yaml # ApplicationSets (managed by bootstrap)
│ ├── hello-world-app.yaml # Example ArgoCD app
│ └── core-apps/ # Core infrastructure apps
│ ├── README.md # Core apps documentation
│ ├── cilium/ # Cilium CNI + Ingress Controller
│ ├── monitoring/ # Prometheus/Grafana
│ └── dev-tools/ # cert-manager
└── test-apps/
└── hello-world/ # Sample testing application
- k3d: Tool to run k3s clusters in Docker containers
- k3s: Lightweight Kubernetes (CNCF-certified distribution)
- Production-ready but optimized for resource-constrained environments
- Perfect for local development and testing
- Fast startup, low memory footprint
- Scripts automatically detect Docker Desktop or Podman Desktop
- Sets
DOCKER_HOST
environment variable if Podman is detected
- Works seamlessly with both runtimes
- ArgoCD feature for managing multiple clusters
- Uses cluster labels for targeting (
environment=laptop
)
- Single ApplicationSet → Multiple Applications (one per cluster)
- Enables consistent core infrastructure across all laptops
- Control deployment order via annotations
- Wave -1: Cilium (must deploy first for networking and ingress)
- Wave 0: Infrastructure (cert-manager)
- Wave 1: Monitoring
- Wave 2+: Applications
- Core Apps: Managed by ArgoCD, always in sync, cannot be modified manually
- Testing Apps: Deployed via kubectl/helm, independent per laptop, not in Git
- Standalone mode: Uses Flannel (k3s default) - fast, simple, battle-tested
- ArgoCD-managed mode: Uses Cilium (installed by ArgoCD) - advanced features, observability
./create-cluster-k3d.sh my-cluster
./install-argocd.sh # optional
# ONE-TIME: Apply bootstrap app to homelab (if not already done)
kubectl apply -f argocd-apps/bootstrap-laptop-management.yaml --context homelab
# Create laptop cluster and register with homelab
./create-cluster-k3d.sh --skip-cilium --register-with-homelab homelab my-laptop
# ArgoCD automatically deploys core apps via ApplicationSets
kubectl config use-context k3d-my-cluster
kubectl apply -f test-apps/hello-world/deployment.yaml
- Create
argocd-apps/core-apps/new-app/
with manifests
- Add ApplicationSet to
laptop-clusters-applicationset.yaml
- Commit and push
- ArgoCD syncs to all registered clusters
k3d cluster delete my-cluster
Main cluster creation script with two modes:
- Default: Creates cluster with Flannel CNI (~15 seconds)
--skip-cilium
: Disables Flannel for ArgoCD to install Cilium--register-with-homelab
: Auto-registers with homelab ArgoCD- Automatically detects Docker/Podman
Bootstrap Application deployed to homelab ArgoCD (App-of-Apps pattern):
- Manages the ApplicationSets in Git
- Auto-syncs changes to ApplicationSets
- Applied ONCE to homelab, then everything is GitOps
- Enables fully declarative multi-cluster management
Defines ALL core applications deployed to laptop clusters:
- Managed by the bootstrap Application
- Contains ApplicationSets for Cilium (with ingress), monitoring, cert-manager
- Uses cluster selector
environment=laptop
for targeting
- Single source of truth for multi-cluster infrastructure
Registers laptop cluster with homelab ArgoCD:
- Creates service account in laptop cluster
- Grants cluster-admin to ArgoCD
- Adds cluster to ArgoCD with
environment=laptop
label
- ApplicationSets detect new cluster and deploy core apps
- Updated to support both k3d and legacy Talos naming conventions
- k3d clusters: Context named
k3d-<cluster-name>
(e.g., k3d-my-laptop
)
- Legacy Talos: Context named
admin@<cluster-name>
(e.g., admin@my-laptop
)
- Registration script handles both conventions automatically
Registered clusters should have:
environment: laptop
(required for ApplicationSet targeting)hostname: <hostname>
(for identification)- Custom labels for per-cluster customization
Core apps use reduced resource limits for local clusters:
- Prometheus: 512Mi-1Gi memory
- Grafana: 128Mi-256Mi memory
- Single replica for most components
- k3d automatically manages networking (no CIDR conflicts)
- Multiple clusters can run simultaneously without configuration
- Services use NodePort (not LoadBalancer) for local clusters
- VPN/Tailscale required for homelab → laptop connectivity
Normal for ArgoCD-managed clusters - nodes stay NotReady until Cilium is installed by ArgoCD.
- Verify VPN/Tailscale connection
- Check cluster API is reachable from homelab
- Test:
kubectl --context homelab exec -it <argocd-pod> -- curl <laptop-cluster-api>
- Check Git repository URL in ApplicationSet
- Verify cluster has
environment=laptop
label
- Force sync:
argocd app sync cluster-name-app-name
- Verify Cilium ApplicationSet has sync-wave: "-1"
- Check pods:
kubectl get pods -n kube-system
- Review logs:
kubectl logs -n kube-system -l k8s-app=cilium
- Confirm Flannel was disabled:
kubectl get pods -A | grep flannel
(should be empty)
- Docker Desktop: Ensure it's running
- Podman Desktop: Ensure Docker socket compatibility is enabled
- Check:
docker ps
or podman ps
- Script shows which runtime is detected
- Speed: 15 seconds vs hours of troubleshooting
- Reliability: Excellent macOS/Podman compatibility
- Simplicity: Fewer moving parts, easier to debug
- Familiarity: Standard Kubernetes, no custom API-driven OS
- Resource Efficiency: Lightweight, perfect for laptops
- Iteration Speed: Create/destroy in seconds
- Modern eBPF-based networking
- Superior observability (Hubble)
- Network policies
- kube-proxy replacement
- Useful for testing production-like scenarios
- Battle-tested, stable
- Low resource usage
- Fast startup
- No configuration needed
- Good enough for local testing
- Consistent core infrastructure across laptops
- Centralized management from homelab
- Flexibility for independent testing apps
- Realistic multi-cluster scenarios
- Core apps need consistency (monitoring, networking)
- Testing apps need flexibility (experimentation, feature branches)
- Separates infrastructure from applications
- Prevents accidental modification of core infrastructure
- Edit files in
argocd-apps/core-apps/
- Update ApplicationSet if needed
- Commit and push
- ArgoCD auto-syncs to all clusters (if automated sync enabled)
- Place in
scripts/
- Make executable:
chmod +x scripts/script.sh
- Update
scripts/README.md
- Follow existing error handling patterns
- Support both Docker and Podman
- README.md: Quick start and common operations
- CLUSTER_PLAN.md: Architecture rationale and design
- MULTI_CLUSTER_SETUP.md: Multi-cluster setup and workflows
- CLAUDE.md: Context for AI assistants (this file)
- Git initialized: Yes
- Primary script:
create-cluster-k3d.sh
- Remote: Should be pushed to GitHub/GitLab
- Branch: main
Why we migrated:
- Talos had persistent Docker socket connection issues on macOS
- Talos clusters took hours to troubleshoot vs 15 seconds with k3d
- k3d has better Podman Desktop compatibility
- Simpler architecture, fewer points of failure
- Same GitOps multi-cluster architecture still works
What was kept:
- Multi-cluster GitOps architecture
- ApplicationSet pattern
- Core apps structure
- Hybrid deployment model
- Documentation structure
What changed:
- Cluster creation tool: Talos → k3d
- Default CNI: None (Talos) → Flannel (k3s default)
- Cluster creation time: Minutes/hours → 15 seconds
- Context naming:
admin@name
→ k3d-name
- Cluster management:
talosctl
→ k3d
- Immediate: Start creating k3d clusters for local testing
- Optional: Push repository to GitHub/GitLab
- Optional: Update repository URLs in ApplicationSet files
- Optional: Apply bootstrap Application to homelab ArgoCD
- Optional: Create and register laptop clusters with homelab
- Optional: Verify core apps deploy automatically via ApplicationSets
- Always: Deploy testing applications manually to laptop clusters
- Secrets: Not managed yet - consider sealed-secrets or external-secrets
- VPN/Tailscale: Required for multi-cluster mode, not covered in detail
- Security: Service account has cluster-admin - reduce in production
- Storage: Uses default storage class or emptyDir
- Podman: Requires
podman-mac-helper
for Docker socket compatibility on macOS