Kubernetes Pod Lifecycle & States

Pod Phases Overview

⏳

Pending

Waiting for scheduling or image download

✅

Running

Containers are executing successfully

🎉

Succeeded

All containers completed successfully

❌

Failed

One or more containers failed

❓

Unknown

Communication error with node

Key Points

Pods never move backward in phases - they only progress forward
Once a Pod reaches Succeeded or Failed, it is considered terminated
Controllers (like Deployments) can create new Pods to replace failed ones
The Unknown phase indicates a problem with node communication, not the Pod itself
Only one phase is active at any time for a given Pod

Pod Lifecycle State Diagram

💡 State Transition Rules

Forward Only: Pods never move backward in phases (no reverse transitions)
Terminal States: Succeeded and Failed are final states for that Pod instance
Restart Behavior: Restart policies affect containers, not Pod phases
Controller Actions: Deployments create new Pods (not restart old ones)
Grace Period: Default 30 seconds for graceful termination

Container States

Each container in a Pod can be in one of three states at any time

⏸️

Waiting

Container is not yet running and is still performing setup operations like pulling images, applying secrets, or waiting for dependencies.

Status Fields:
• reason: Why container is waiting
• message: Detailed diagnostic info

Common Reasons:
• ContainerCreating
• PodInitializing
• ImagePullBackOff

▶️

Running

Container has been successfully bound to a node and is currently executing. The process inside the container is active.

Status Fields:
• startedAt: Timestamp when container entered Running state

Characteristics:
• Main process is executing
• PostStart hook has completed (if defined)
• Liveness probes are active

⏹️

Terminated

Container has completed execution either successfully or with an error. The container may or may not be restarted based on the restart policy.

Status Fields:
• exitCode: Exit status (0 = success)
• signal: Signal that terminated container
• reason: Human-readable termination reason
• message: Detailed termination message
• startedAt / finishedAt: Timing info

📊 Container State vs Pod Phase

Independence: Each container has its own state, independent of other containers
Pod Phase: Reflects the overall status of all containers combined
Multiple States: A single Pod can have containers in different states simultaneously
Monitoring: Use kubectl describe pod to see individual container states
Restart Tracking: Container restarts increment the restart count, not create new Pods

Pod Lifecycle Timeline

A chronological view of a Pod's journey from creation to termination

1. Pod Created

Pod manifest is submitted to the API server. The Pod object is created with phase Pending. Pod is assigned a unique UID that never changes.

2. Scheduler Assignment

Kubernetes scheduler selects a node based on resource requirements, node affinity, taints/tolerations. Pod condition PodScheduled becomes True.

3. Image Pull

Kubelet on assigned node pulls container images from registry. Init containers (if defined) are pulled first. Container state: Waiting with reason "ContainerCreating".

4. Init Containers Run

If defined, init containers execute sequentially. Each must complete successfully before the next starts. Pod condition Initialized becomes True when all complete.

5. Main Containers Start

Main application containers start in parallel. Pod phase changes to Running. PostStart lifecycle hooks execute. Container state: Running.

6. Readiness Check

Readiness probes determine when container is ready to accept traffic. Pod condition ContainersReady and Ready become True. Pod is added to Service endpoints.

7. Steady State

Pod is fully operational and serving requests. Liveness probes monitor container health. Restart policy applies if containers fail. Pod remains in Running phase.

8. Termination Signal

Deletion request received or node draining initiated. PreStop lifecycle hooks execute (if defined). Pod is removed from Service endpoints. Grace period (default 30s) begins.

9. Graceful Shutdown

SIGTERM signal sent to main process. Application should save state and close connections. After grace period expires, SIGKILL is sent to force termination.

10. Final State

All containers terminated. Pod phase becomes Succeeded (all exit code 0) or Failed (at least one non-zero exit). Container state: Terminated.

Pod Conditions

Conditions provide detailed status information about a Pod's current state

Condition	Description	True When	False When
PodScheduled	Pod has been assigned to a node	Scheduler successfully bound Pod to a node	Pod is waiting for node assignment (Pending)
Initialized	All init containers completed successfully	All init containers exited with code 0, or no init containers defined	Init containers are still running or failed
ContainersReady	All containers in the Pod are ready	All containers passed readiness checks (or no readiness probe defined)	One or more containers not ready or still starting
Ready	Pod is ready to serve requests	All containers ready AND all custom readiness gates passed	Not all readiness conditions satisfied
DisruptionTarget	Pod is being deleted due to disruption	Pod deletion is in progress (eviction, node drain, etc.)	Pod is not being disrupted

✅ Checking Conditions

kubectl get pod <pod-name> -o jsonpath='{.status.conditions[*]}'

kubectl describe pod <pod-name>

📋 Condition Fields

type: Name of the condition
status: True, False, Unknown
lastProbeTime: Last check time
lastTransitionTime: Status change time
reason: Machine-readable cause
message: Human-readable details

💡 Important Notes

Service Integration: Only Pods with Ready=True are added to Service endpoints
Custom Gates: Applications can define custom readiness gates for advanced scenarios
Condition History: Transitions are tracked with timestamps for debugging
Health vs Ready: Liveness checks health; Readiness checks traffic acceptance capability

Container Restart Policies

Restart policies define how Kubernetes should handle container failures

Always

Kubernetes will always restart the container regardless of exit code. This is the default policy.

Use Case:
• Long-running services (web servers, APIs)
• Microservices that should always be available
• Deployments, DaemonSets, StatefulSets
• Default for most production workloads

Behavior: Container restarts even on successful completion (exit 0). Backoff delay: 10s, 20s, 40s... up to 5 minutes.

OnFailure

Kubernetes restarts the container only if it exits with a non-zero status (failure). Successful completions are not restarted.

Use Case:
• Batch Jobs that should retry on failure
• Data processing tasks
• Scripts that might fail but should succeed eventually
• Jobs with limited retry attempts

Behavior: Exit code 0 = Pod completes (Succeeded). Non-zero exit = restart with exponential backoff.

Never

Kubernetes never restarts the container under any circumstances, regardless of exit status or failure reason.

Use Case:
• One-shot tasks or scripts
• Database migrations
• Backup jobs
• Debug containers for troubleshooting
• Jobs where manual intervention is required

Behavior: Container runs once and terminates. Pod phase becomes Succeeded or Failed based on exit code.

Restart Policy Decision Tree

🔄 Restart Backoff Behavior

Exponential Backoff: Delays between restarts: 10s, 20s, 40s, 80s, 160s (capped at 5 minutes)
Reset Timer: If container runs successfully for 10 minutes, backoff resets to 10s
CrashLoopBackOff: Status shown when container repeatedly crashes and is waiting for next restart
Restart Count: Tracked per container and visible in kubectl get pods
Pod Phase: Restart policy affects containers, not Pod phase (Pod stays Running)

YAML Examples for Different States

1. Basic Pod with Restart Policy

apiVersion: v1 kind: Pod metadata: name: nginx-pod labels: app: nginx spec: restartPolicy: Always # Default for long-running services containers: - name: nginx image: nginx:1.21 ports: - containerPort: 80

2. Job with OnFailure Restart Policy

apiVersion: batch/v1 kind: Job metadata: name: data-processing-job spec: template: spec: restartPolicy: OnFailure # Retry on failure containers: - name: processor image: data-processor:v2 command: ["/bin/process-data.sh"]

3. Pod with Liveness and Readiness Probes

apiVersion: v1 kind: Pod metadata: name: webapp-with-probes spec: containers: - name: webapp image: myapp:latest ports: - containerPort: 8080 # Liveness: Is container running? Restart if fails livenessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 15 periodSeconds: 10 failureThreshold: 3 # Readiness: Is container ready for traffic? readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 5 periodSeconds: 5

4. Pod with Init Container

apiVersion: v1 kind: Pod metadata: name: myapp-with-init spec: # Init containers run before main containers initContainers: - name: init-db image: busybox:1.35 command: ['sh', '-c', 'until nslookup db-service; do sleep 2; done'] containers: - name: myapp image: myapp:v1 ports: - containerPort: 80

5. Pod with Lifecycle Hooks

apiVersion: v1 kind: Pod metadata: name: lifecycle-demo spec: containers: - name: nginx image: nginx lifecycle: # PostStart: Runs after container starts postStart: exec: command: ["/bin/sh", "-c", "echo 'Started' > /usr/share/message"] # PreStop: Runs before container terminates preStop: exec: command: ["/bin/sh", "-c", "nginx -s quit; sleep 5"] # Grace period for cleanup terminationGracePeriodSeconds: 30

6. CronJob with Never Restart Policy

apiVersion: batch/v1 kind: CronJob metadata: name: database-backup spec: schedule: "0 2 * * *" # 2 AM daily jobTemplate: spec: template: spec: restartPolicy: Never # Don't retry on failure containers: - name: backup image: backup-tool:v1 command: ["/backup.sh"]

🛠️ Useful kubectl Commands

# Check Pod phase and conditions kubectl get pod <pod-name> -o wide # View detailed Pod status including container states kubectl describe pod <pod-name> # Watch Pod status changes in real-time kubectl get pod <pod-name> -w # Get Pod conditions in JSON format kubectl get pod <pod-name> -o jsonpath='{.status.conditions[*]}' # Check container restart count kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[*].restartCount}' # View logs from previous container instance (after restart) kubectl logs <pod-name> --previous # Get events for troubleshooting kubectl get events --field-selector involvedObject.name=<pod-name>