Kubernetes Orchestration

Project Implementation

Kubernetes manages the containerized pipeline, orchestrating Blender jobs, Airflow workflows and also hosts this documentation site, made externally accessible using an Ingress controller + Traefik.

Cluster Setup

• Control Plane: Multi-master setup for high availability
• Worker Nodes: Future GPU-enabled nodes for Blender rendering. Only a simple node currently.
• Storage: Persistent volumes for asset sharing and output storage, using SMB/CIFS protocol.

Key Workloads

The cluster now runs these workloads:

• Perforce sync jobs for asset retrieval
• Blender animation creation
• Hosts this documentation site, exposed at https://homework.everweb.com.br

Storage Solutions

Persistent Volumes

I created a PersistentVolumeClaim (PVC) for Blender assets and for the Blender animation, using SMB storage for shared access across the cluster.

• Storage Classes: Dynamic provisioning
• SMB/NFS: Shared storage for assets
• Local Storage: High-performance workloads, limited by SSD disk speed as the virtual network within Proxmox is NOT a bottleneck.

Volume Management

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: blender-assets-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: smb-storage

Below, some of the Kubernetes cluster management screenshots.

Kubernetes deployment configuration (I had the Portainer agent installed on it as well)

Cluster overview

The service that hosts this documentation site

Secrets for the external registry (individual images and secrets for each task of the pipeline) and for SMB storage access

Persistent Volumes and Claims

The nodes

Improvements planned

• Proper RBAC
• Cluster configuration recurring backups
• Add local DNS names for easier access
• Monitoring and uptime alerts following the same approach used for other services in the homelab.