Solana on Kubernetes
The scheduling, networking, storage, and shutdown decisions that matter more than a basic Kubernetes manifest.
Kubernetes can schedule and supervise a Solana validator, but it does not automatically supply the stable identity, UDP behavior, local high-performance disks, or graceful lifecycle that a validator needs. The default advice to run Solana on bare metal is sensible; Kubernetes can still be appropriate when the platform can deliberately provide those properties. A deployment that only proves a container can start is not enough.
Treat Kubernetes values as deployment inputs to review, especially identity handling, storage classes, node placement, network exposure, and the validator port range.
Networking
Solana uses UDP and a dynamic port range. The Kubernetes reference profile uses 8002-8030; a VM-oriented development profile uses 8000-8020. A single NodePort does not model the whole range well. Exposing only the two TPU ports, 8004 and 8010, must be tested as an integrated host-network, NodePort, firewall, and CNI design rather than assumed to cover all Solana traffic.
hostNetwork: true is often the honest option when the cluster needs predictable UDP behavior and the node is dedicated to the validator. It trades away network isolation and port flexibility, so document that decision and reserve the host accordingly.
The validator configuration should make its dynamic range explicit. The Kubernetes reference profile starts bootstrap and follower validators with --dynamic-port-range=8002-8030 and declares every port in that range for both TCP and UDP. Upstream Agave guidance documents TCP and UDP traffic in the 8000–8030 range by default and permits a chosen free range through --dynamic-port-range. Confirm the final range against the exact Agave release and the infrastructure firewall policy before opening it.
Network test matrix
Run these tests from an external peer and repeat them after a CNI, firewall, or ingress change:
| Test | Success condition |
|---|---|
| Gossip and entrypoint reachability | Node discovers peers and remains connected after restart. |
| UDP range reachability | Every required dynamic port is reachable; no NodePort translation silently drops traffic. |
| RPC health | getHealth succeeds and the reported slot advances. |
| Sustained load | Slot lag and packet loss remain acceptable during application and repair traffic. |
| Reschedule behavior | A planned drain does not lose identity, ledger placement, or the expected recovery path. |
Identity and lifecycle
Use a StatefulSet when a workload needs a stable identity and persistent state. Do not couple validator identity to a disposable Pod filesystem.
Do not assume a short grace period is sufficient. A hard kill can turn an ordinary rollout into ledger recovery or a full rebuild. Set terminationGracePeriodSeconds from a measured graceful-exit time, and document any pre-stop behavior used to keep the node out of a leader period.
A practical readiness gate is: call getHealth, get local slot data, compare it with an approved reference endpoint, and remain unready until the lag stays below the environment threshold for a sustained period. A simple /health liveness probe does not implement a slot-lag readiness gate. Liveness should be conservative; restarting a busy validator because a short RPC probe timed out can make the incident worse.
Storage and placement
Use local, topology-aware NVMe storage for ledger and accounts data. A suitable Kubernetes pattern uses per-node, ReadWriteOnce PVCs and node selectors to pin each validator's volumes to its intended host. The scheduler must keep the workload on a node that owns those volumes; a generic network volume or casual reschedule can create a severe performance or recovery problem.
Decision log
Keep the following with every deployment:
| Decision | Record |
|---|---|
| Network mode | whether hostNetwork was used, the UDP range, and the test result |
| Storage | ledger/accounts/snapshot locations, volume type, and node affinity |
| Identity | where validator and vote authority material lives and how it is mounted |
| Shutdown | grace period, pre-stop behavior, and observed restart/recovery time |
| Readiness | RPC method, allowed slot lag, and failure threshold |
Only publish an “attempted and reverted” decision after it has a reproducible test result and a clear reason for the reversal.
When Kubernetes is the wrong fit
Do not choose Kubernetes merely to standardize an estate. Bare metal is usually the safer choice for a mainnet consensus validator when the cluster cannot guarantee dedicated CPU, direct UDP behavior, topology-pinned NVMe, and an operator who understands the recovery cost of eviction. Containerizing development workloads is different from supporting a production validator in a shared cluster.