⌂ Home

DNS Resolution Flow in Kubernetes

Interactive Guide to Understanding Service Discovery and DNS Resolution

DNS Query Resolution Flow

Click on each step to see detailed information about the DNS resolution process.

1
Pod Initiates DNS Query
Application in Pod requests DNS resolution for service nameService names can be short (my-nginx) or fully qualified (my-nginx.default.svc.cluster.local)
Example Query: nslookup my-nginx

Pod's /etc/resolv.conf:
nameserver 10.32.0.10 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5

The kubelet configures each Pod's DNS settings, pointing to CoreDNS cluster IP.

↓
2
Query Sent to CoreDNS
DNS query is forwarded to CoreDNSCoreDNS is the cluster DNS server running in kube-system namespace, replacing the older kube-dns service (kube-dns)
CoreDNS Service:
kubectl get svc kube-dns -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) kube-dns ClusterIP 10.32.0.10 <none> 53/UDP,53/TCP

CoreDNS pods receive the query through the kube-dns service endpoint.

↓
3
CoreDNS Processes Query
CoreDNS expands short names using search domainsSearch domains allow using short names (my-nginx) which expand to FQDNs (my-nginx.default.svc.cluster.local) and queries Kubernetes API
Query Expansion Process:
  • my-nginx → my-nginx.default.svc.cluster.local
  • If not found → my-nginx.svc.cluster.local
  • If not found → my-nginx.cluster.local

CoreDNS uses the Kubernetes plugin to query the API server for Service and Endpoint information.

↓
4
Service Lookup
CoreDNS queries Kubernetes API to find the ServiceServices provide stable DNS names and IP addresses for dynamic sets of Pods matching the DNS name
Service Resolution Types:
  • Normal Service (ClusterIP): Returns the Service's cluster IP address
  • Headless Service (clusterIP: None): Returns all Pod IPs directly
  • ExternalName Service: Returns a CNAME record to external DNS name
# Normal Service my-nginx.default.svc.cluster.local → 10.100.200.50 (ClusterIP) # Headless Service my-nginx.default.svc.cluster.local → 172.17.0.3, 172.17.0.4 (Pod IPs)
↓
5
Endpoint Resolution
For normal Services, EndpointsEndpoints track the IP addresses of Pods backing a Service are checked to find backing Pod IPs
Endpoint Lookup:
kubectl get endpoints my-nginx NAME ENDPOINTS AGE my-nginx 172.17.0.3:80,172.17.0.4:80 5m

While DNS returns the ClusterIP, traffic to that IP is load-balanced by kube-proxy to the Pod IPs listed in Endpoints.

↓
6
DNS Response Returned
CoreDNS returns DNS response with A/AAAA recordA records map names to IPv4 addresses, AAAA records map to IPv6 addresses containing IP address(es)
DNS Response Format:
;; QUESTION SECTION: ;my-nginx.default.svc.cluster.local. IN A ;; ANSWER SECTION: my-nginx.default.svc.cluster.local. 30 IN A 10.100.200.50

The Pod receives the IP address and can now connect to the Service.

↓
7
Connection Established
Pod uses resolved IP to connect; kube-proxykube-proxy manages network rules for Service traffic routing and load balancing routes traffic to healthy Pod
Traffic Flow:
  • Pod connects to ClusterIP (10.100.200.50:80)
  • kube-proxy intercepts traffic via iptables/IPVS rules
  • Traffic is load-balanced to one of the healthy backend Pods
  • Connection established with selected Pod (172.17.0.3:80 or 172.17.0.4:80)

Key Point

DNS resolution is separate from traffic routing. DNS provides the Service IP, while kube-proxy handles the actual Pod selection and load balancing.

DNS Naming Conventions in Kubernetes

FQDN Breakdown

my-nginx.default.svc.cluster.local

my-nginx

Service Name

default

Namespace

svc

Service Indicator

cluster.local

Cluster Domain

Namespace Isolation

Queries without namespace specification are limited to the Pod's namespace. To access services in other namespaces, specify the namespace:

  • data - Only searches in current namespace
  • data.prod - Accesses service in 'prod' namespace
  • data.prod.svc.cluster.local - Fully qualified domain name

Service A/AAAA Records

Format: service-name.namespace.svc.cluster-domain

# Normal Service (ClusterIP) my-nginx.default.svc.cluster.local → Returns: 10.100.200.50 # Headless Service (clusterIP: None) my-nginx.default.svc.cluster.local → Returns: 172.17.0.3, 172.17.0.4, 172.17.0.5

Normal services return a single stable ClusterIP. Headless services return all Pod IPs directly for client-side load balancing.

Service SRV Records

Format: _port-name._protocol.service-name.namespace.svc.cluster-domain

# Named port SRV record _http._tcp.my-nginx.default.svc.cluster.local → Returns: port + domain name # For headless service _http._tcp.my-nginx.default.svc.cluster.local → Returns: Multiple records with Pod hostnames

SRV records enable service discovery for named ports, providing both port numbers and target hostnames.

Pod A/AAAA Records

Format: pod-ip.namespace.pod.cluster-domain

# Pod with IP 172.17.0.3 172-17-0-3.default.pod.cluster.local → Returns: 172.17.0.3 # Service-scoped Pod record 172-17-0-3.my-nginx.default.svc.cluster.local → Returns: 172.17.0.3

Pod DNS records use dashes instead of dots in IP addresses. These are automatically created by CoreDNS.

Pod with Hostname/Subdomain

Format: hostname.subdomain.namespace.svc.cluster-domain

# Pod with hostname and subdomain apiVersion: v1 kind: Pod metadata: name: busybox1 spec: hostname: busybox-1 subdomain: busybox-subdomain ... # Resulting FQDN busybox-1.busybox-subdomain.default.svc.cluster.local

Requires both hostname and subdomain fields. A matching headless Service with the subdomain name must exist.

Service DNS Configuration Example

apiVersion: v1 kind: Service metadata: name: my-nginx namespace: default spec: type: ClusterIP selector: app: nginx ports: - name: http port: 80 targetPort: 80 protocol: TCP # DNS Records Created: # - A/AAAA: my-nginx.default.svc.cluster.local → ClusterIP # - SRV: _http._tcp.my-nginx.default.svc.cluster.local → 80 + hostname

Pod with Custom DNS Configuration Example

apiVersion: v1 kind: Pod metadata: name: busybox-1 labels: app: busybox spec: hostname: busybox-1 subdomain: busybox-service containers: - name: busybox image: busybox:1.28 command: ["sleep", "3600"] # Requires matching headless Service: # apiVersion: v1 # kind: Service # metadata: # name: busybox-service # spec: # clusterIP: None # selector: # app: busybox # Resulting FQDN: busybox-1.busybox-service.default.svc.cluster.local

Important Limitations

  • FQDN Length: When using setHostnameAsFQDN: true, Linux hostname field is limited to 64 characters. Longer FQDNs will cause Pod startup failure.
  • Headless Service Required: For Pod hostname/subdomain DNS to work, a matching headless Service must exist with the same name as the subdomain.
  • Search Domain Order: Queries are tried in order of search domains. This can cause delays if the service is in a later search domain.

DNS Policy Modes in Kubernetes

Repository YAML files — apply directly from k8s/labs/networking/:
  • dnsdefault.yaml — dnsPolicy: Default (uses node's DNS, not cluster DNS)
  • dnspolicy.yaml — hostNetwork: true + dnsPolicy: ClusterFirstWithHostNet
  • dnsconfig.yaml — dnsPolicy: None with custom nameservers, searches, and ndots

The dnsPolicy field in Pod spec determines how DNS resolution is configured. Choose the appropriate policy based on your networking requirements.

ClusterFirst DEFAULT

Uses cluster DNS (CoreDNS) for resolution. Any DNS query that does not match the cluster domain suffix is forwarded to the upstream nameserver inherited from the node.

dnsPolicy: ClusterFirst # /etc/resolv.conf nameserver 10.32.0.10 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5
Use Case: Standard Pods that need to discover Kubernetes Services and resolve external DNS names.

Default

Pod inherits DNS resolution from the node. Bypasses cluster DNS entirely and uses the node's /etc/resolv.conf configuration.

dnsPolicy: Default # Inherits from node's /etc/resolv.conf # (varies by node configuration) nameserver 8.8.8.8 nameserver 8.8.4.4 search example.com
Use Case: Pods that don't need Kubernetes Service discovery and should use standard DNS resolution.

ClusterFirstWithHostNet

For Pods using hostNetwork: true. Ensures cluster DNS is used even when the Pod shares the host's network namespace.

hostNetwork: true dnsPolicy: ClusterFirstWithHostNet # /etc/resolv.conf nameserver 10.32.0.10 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5
Use Case: Pods using host networking that still need to resolve Kubernetes Services (e.g., monitoring agents, network tools).

None

Ignores all DNS settings from the cluster. Requires explicit dnsConfig to specify custom DNS configuration. Provides complete control over DNS settings.

dnsPolicy: "None" dnsConfig: nameservers: - 8.8.8.8 searches: - my-custom.search.suffix options: - name: ndots value: "1"
Use Case: Testing custom DNS resolvers, specific DNS requirements, or integration with external DNS systems.

Example: Pod with hostNetwork and ClusterFirstWithHostNet

apiVersion: v1 kind: Pod metadata: name: busybox-hostnetwork spec: containers: - name: busybox image: busybox:1.28 command: ["sleep", "3600"] hostNetwork: true dnsPolicy: ClusterFirstWithHostNet # This configuration: # - Pod uses host's network stack (shares host IP) # - But still resolves Kubernetes Services via CoreDNS # - Can resolve both cluster Services and external DNS names

Example: Pod with Custom DNS Configuration (dnsPolicy: None)

apiVersion: v1 kind: Pod metadata: name: custom-dns-pod spec: containers: - name: nginx image: nginx dnsPolicy: "None" dnsConfig: nameservers: - 8.8.8.8 - 1.1.1.1 searches: - my-custom.search.suffix - another.search.domain options: - name: ndots value: "1" - name: timeout value: "2" # Resulting /etc/resolv.conf: # nameserver 8.8.8.8 # nameserver 1.1.1.1 # search my-custom.search.suffix another.search.domain # options ndots:1 timeout:2

dnsConfig Options Explained

  • nameservers: List of DNS server IP addresses (maximum 3)
  • searches: DNS search domains for hostname lookups (maximum 6)
  • options: List of DNS resolver options
    • ndots: Minimum number of dots in a name to query as absolute before trying search domains (default: 5)
    • timeout: Timeout for DNS queries in seconds
    • attempts: Number of DNS query attempts
    • edns0: Enable DNS extensions

Comparison Table

DNS Policy Resolves K8s Services Resolves External DNS When to Use
ClusterFirst ✅ Yes ✅ Yes (via upstream) Default - most Pods
Default ❌ No ✅ Yes (from node) No K8s service discovery needed
ClusterFirstWithHostNet ✅ Yes ✅ Yes (via upstream) hostNetwork=true Pods
None Depends on dnsConfig Depends on dnsConfig Custom DNS requirements

DNS Resolution Examples

Compare successful and failed DNS resolution scenarios to understand common issues and best practices.

✅ Successful Resolution

  • Query: nslookup my-nginx
  • Pod Namespace: default
  • Search Domain: default.svc.cluster.local
  • Expanded Query: my-nginx.default.svc.cluster.local
  • Service Found: Yes (in default namespace)
  • ClusterIP: 10.100.200.50
  • Result: Connection successful
Server: 10.32.0.10 Address: 10.32.0.10#53 Name: my-nginx.default.svc.cluster.local Address: 10.100.200.50

❌ Failed Resolution

  • Query: nslookup data
  • Pod Namespace: test
  • Search Domain: test.svc.cluster.local
  • Expanded Query: data.test.svc.cluster.local
  • Service Found: No (service is in prod namespace)
  • Error: NXDOMAIN (Non-Existent Domain)
  • Solution: Use data.prod or full FQDN
Server: 10.32.0.10 Address: 10.32.0.10#53 ** server can't find data: NXDOMAIN

Scenario 1: Cross-Namespace Service Access

# Pod in 'test' namespace trying to access service in 'prod' namespace # ❌ FAILS - searches only in test namespace curl data # ✅ SUCCESS - specifies prod namespace curl data.prod # ✅ SUCCESS - uses full FQDN curl data.prod.svc.cluster.local # DNS Query Expansion: # Query: data # 1. data.test.svc.cluster.local (NOT FOUND) # 2. data.svc.cluster.local (NOT FOUND) # 3. data.cluster.local (NOT FOUND) # Result: NXDOMAIN # Query: data.prod # 1. data.prod.test.svc.cluster.local (NOT FOUND) # 2. data.prod.svc.cluster.local (FOUND!) # Result: Success

Scenario 2: Headless Service Resolution

# Normal Service - returns ClusterIP $ nslookup my-nginx.default.svc.cluster.local Name: my-nginx.default.svc.cluster.local Address: 10.100.200.50 # Headless Service - returns all Pod IPs $ nslookup my-nginx-headless.default.svc.cluster.local Name: my-nginx-headless.default.svc.cluster.local Address: 172.17.0.3 Name: my-nginx-headless.default.svc.cluster.local Address: 172.17.0.4 Name: my-nginx-headless.default.svc.cluster.local Address: 172.17.0.5 # Headless Service YAML: # apiVersion: v1 # kind: Service # metadata: # name: my-nginx-headless # spec: # clusterIP: None # This makes it headless # selector: # app: nginx

Scenario 3: Pod DNS with Hostname/Subdomain

# Pod configuration apiVersion: v1 kind: Pod metadata: name: web-1 spec: hostname: web-1 subdomain: web-service ... # Headless Service (REQUIRED) apiVersion: v1 kind: Service metadata: name: web-service spec: clusterIP: None ... # ✅ DNS Resolution $ nslookup web-1.web-service.default.svc.cluster.local Name: web-1.web-service.default.svc.cluster.local Address: 172.17.0.10 # ❌ Without headless Service - DNS will fail # Pod hostname/subdomain requires matching headless Service

Scenario 4: External DNS Resolution

# With ClusterFirst (default policy) $ nslookup google.com Server: 10.32.0.10 Address: 10.32.0.10#53 Non-authoritative answer: Name: google.com Address: 142.250.185.78 # Flow: # 1. Query sent to CoreDNS (10.32.0.10) # 2. CoreDNS sees it's not a cluster domain # 3. CoreDNS forwards to upstream DNS (from node) # 4. Response returned through CoreDNS to Pod # With Default policy - bypasses CoreDNS $ nslookup google.com Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: Name: google.com Address: 142.250.185.78 # Flow: # 1. Query sent directly to node's DNS (8.8.8.8) # 2. No CoreDNS involvement # 3. Cannot resolve Kubernetes Services

Best Practices for DNS Resolution

  • Use full FQDNs for cross-namespace access: Specify namespace explicitly to avoid ambiguity and reduce DNS query attempts.
  • Prefer short names within the same namespace: Simplifies configuration and leverages search domains efficiently.
  • Use headless Services for StatefulSets: Provides stable, predictable DNS names for each Pod.
  • Monitor CoreDNS performance: Watch for DNS resolution delays which can impact application latency.
  • Configure ndots appropriately: Lower ndots value (e.g., 2) can reduce DNS queries for external domains.

Common DNS Issues and Solutions

  • NXDOMAIN errors: Check namespace, service name spelling, and ensure Service exists.
  • Slow DNS resolution: High ndots value causes many search domain attempts. Consider using FQDNs with trailing dot (e.g., google.com.).
  • Cannot resolve Services with hostNetwork: Use dnsPolicy: ClusterFirstWithHostNet instead of default.
  • Custom DNS not working: Ensure dnsPolicy: "None" is set when using dnsConfig.
  • Pod hostname DNS fails: Verify matching headless Service exists with clusterIP: None.