⌂ Home

Kubernetes Taints & Tolerations

Node Access Control: Checkpoint System for Pod Scheduling

Repository YAML Files:

k8s/labs/scheduling/tolerations/tolerations-pod.yaml — Pod declaring tolerations so it can schedule onto tainted nodes.
k8s/labs/scheduling/tolerations/no-tolerations-pod.yaml — Pod without tolerations (illustrates blocking on tainted nodes).

How It Works: Police Checkpoint Analogy

Pod
(No Permit)

✖

Taint
Checkpoint

gpu=true:NoSchedule

✖

GPU Node Tainted

Pod
Has Toleration

→

Taint
Checkpoint

gpu=true:NoSchedule

✓

GPU Node Tainted

Taints repel pods unless they have matching tolerations (permits) to override the restriction.

Taint Effects: Three Enforcement Levels

Effect	New Pods	Existing Pods	Use Case
NoSchedule	Blocks scheduling	No impact	Reserve nodes for specific workloads
PreferNoSchedule	Tries to avoid	No impact	Soft preference, not strict
NoExecute	Blocks scheduling	Evicts immediately	Maintenance mode, force evacuation

Matching Logic: How Tolerations Match Taints

✓ Match Example (Pod Schedules)

✓ Taint: gpu=true:NoSchedule

✓ Toleration: key=gpu, value=true, effect=NoSchedule

All three match: key, value, effect

✖ No Match (Pod Blocked)

✖ Taint: gpu=true:NoSchedule

✖ Toleration: key=gpu, value=false, effect=NoSchedule

Value mismatch: true ≠ false

Exists Operator: Using operator: Exists matches the taint key only (ignores value). Example: tolerate any gpu taint regardless of value.

YAML Configuration Examples

NODE Apply Taint to Node (kubectl command)

# Apply NoSchedule taint for GPU node
kubectl taint nodes worker-node-1 \
  gpu=true:NoSchedule

# Apply NoExecute for maintenance
kubectl taint nodes worker-node-2 \
  maintenance=true:NoExecute

# Remove taint (note the minus)
kubectl taint nodes worker-node-1 \
  gpu=true:NoSchedule-

POD Pod with Toleration

apiVersion: v1
kind: Pod
metadata:
  name: ml-workload
spec:
  tolerations:
  - key: "gpu"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
  containers:
  - name: tensorflow
    image: tensorflow/tensorflow:latest-gpu
    resources:
      limits:
        nvidia.com/gpu: 1

Common Use Cases

GPU Nodes Taint: gpu=true:NoSchedule
Only ML/AI workloads with tolerations can access GPU nodes

Dedicated Production Taint: env=prod:NoSchedule
Isolate production workloads from test/dev pods

Maintenance Mode Taint: maintenance=true:NoExecute
Evacuate all pods except critical ones with tolerations

Database Nodes Taint: dedicated=db:NoSchedule
Reserve high-memory nodes for database workloads

Interactive Demo: Test Taint Effects

Click a button to test taint behavior

Quick Reference

Toleration Operators

Equal: Key, value, and effect must match exactly

Exists: Only key needs to match (ignores value)

Debugging Tips

kubectl describe node <name> - View node taints

kubectl describe pod <name> - Check why pod is pending