Resource Management and QoS

← Back to CKA Practice Index

Overview

Kubernetes resource management ensures Pods receive reasonable resource allocation. The CKA exam focuses on requests/limits configuration, QoS class identification, LimitRange, and ResourceQuota.

---

1. Resource Requests and Limits

1.1 Basic Concepts

Parameter	Description	Role
requests	The minimum guaranteed resource amount for a container	The scheduler uses this value to select a node
limits	The maximum resource amount a container can use	Limits resource overuse

1.2 CPU Units

resources:
  requests:
    cpu: "0.5"         # 500m (millicores), equivalent to 500m
    memory: "256Mi"    # 256 Mebibytes
  limits:
    cpu: "1"           # 1 core
    memory: "512Mi"

CPU Unit Quick Reference:

Unit	Equivalent
1	1 core (vCPU)
0.5	500m
100m	0.1 core
1k	1000m (uncommon)

Memory Unit Quick Reference:

Unit	Description
Mi	Mebibytes (2^20, recommended)
M	Megabytes (10^6, not recommended)
Gi	Gibibytes (2^30, recommended)
Ki	Kibibytes (2^10)

1.3 Pod Resource Configuration Example

apiVersion: v1
kind: Pod
metadata:
  name: resource-pod
spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      requests:
        cpu: 250m
        memory: "128Mi"
      limits:
        cpu: 500m
        memory: "256Mi"

1.4 Key Commands

kubectl describe node <node-name>

# View node resource allocation (Requested vs Allocatable)
kubectl describe node <node-name> | grep -A 5 "Allocated resources"

# View resource usage across all nodes (commonly used in exam)
kubectl top node
kubectl top pod

# Set resource limits for a Pod
kubectl run nginx --image=nginx --dry-run=client -o yaml --limits='cpu=500m,memory=256Mi' --requests='cpu=250m,memory=128Mi'

---

2. QoS Classes (Quality of Service)

Kubernetes automatically assigns a QoS class based on a Pod's requests/limits configuration.

2.1 Three QoS Classes

QoS Class	Condition	Eviction Priority
Guaranteed	All containers have requests == limits	Lowest
Burstable	At least one container has requests or limits, but does not meet Guaranteed	Medium
BestEffort	No container has requests/limits set	Highest (evicted first)

2.2 Configuration Examples

Guaranteed (all containers requests == limits):

spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: 500m
        memory: 256Mi
      limits:
        cpu: 500m
        memory: 256Mi

Burstable (requests < limits or only requests set):

spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: 250m
        memory: 128Mi
      limits:
        cpu: 500m
        memory: 256Mi

BestEffort (no resources field):

spec:
  containers:
  - name: app
    image: nginx
    # No resources field

2.3 Checking QoS Class

# View Pod QoS class
kubectl describe pod <pod-name> | grep QoS
kubectl get pod <pod-name> -o yaml | grep qosClass
kubectl get pod <pod-name> -o jsonpath='{.status.qosClass}'

2.4 Impact of QoS on Pod Eviction

When node resources are insufficient, kubelet evicts Pods in the following order:

1. BestEffort: Evicted first
2. Burstable: Pods whose usage exceeds their requests
3. Guaranteed: Only evicted when system processes need resources

Eviction thresholds:

• memory.available falls below threshold
• nodefs.available or nodefs.inodesFree falls below threshold
• imagefs.available falls below threshold

---

3. LimitRange

LimitRange sets default resource requests and limits for Pods/containers at the namespace level.

3.1 Configuration Example

apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-limit-range
  namespace: default
spec:
  limits:
  - default:              # Default limits
      cpu: 500m
      memory: 256Mi
    defaultRequest:       # Default requests
      cpu: 250m
      memory: 128Mi
    max:                  # Maximum limit
      cpu: "2"
      memory: 1Gi
    min:                  # Minimum limit
      cpu: 100m
      memory: 64Mi
    type: Container       # Applies to Container
  - type: Pod
    max:
      cpu: "4"
      memory: 4Gi

3.2 LimitRange Types

Type	Scope
Container	Single container (most common)
Pod	Sum of all containers in the Pod
PersistentVolumeClaim	Storage claim size

3.3 Usage Commands

# Create LimitRange
kubectl apply -f limitrange.yaml

# View
kubectl get limitrange
kubectl get limits
kubectl describe limitrange cpu-limit-range

# Delete
kubectl delete limitrange cpu-limit-range

Note: LimitRange only affects Pods created after it is applied; it does not modify existing Pods.

---

4. ResourceQuota

ResourceQuota limits the total resource usage of a namespace.

4.1 Configuration Example

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: test
spec:
  hard:
    requests.cpu: "4"          # Total CPU requests cap across all Pods
    requests.memory: "8Gi"     # Total memory requests cap across all Pods
    limits.cpu: "8"            # Total CPU limits cap across all Pods
    limits.memory: "16Gi"      # Total memory limits cap across all Pods
    requests.nvidia.com/gpu: 4 # GPU count
    pods: "10"                 # Pod count cap
    services: "5"              # Service count cap
    persistentvolumeclaims: "5" # PVC count cap
    configmaps: "5"            # ConfigMap count cap
    secrets: "5"               # Secret count cap

4.2 Object Count Quota

apiVersion: v1
kind: ResourceQuota
metadata:
  name: object-quota
  namespace: test
spec:
  hard:
    pods: "10"
    services: "5"
    configmaps: "5"
    secrets: "5"
    persistentvolumeclaims: "3"
    replicationcontrollers: "5"
    deployments.apps: "5"
    resourcequotas: "1"

4.3 Best Practice: Set Both LimitRange and ResourceQuota

# When ResourceQuota's requests.cpu is set, Pods created must specify CPU requests
# Therefore, a LimitRange is also needed to provide default values

# Step 1: Create LimitRange to provide defaults
kubectl apply -f limitrange.yaml

# Step 2: Create ResourceQuota
kubectl apply -f resourcequota.yaml

# Step 3: Create a Pod (LimitRange defaults will be applied automatically)
kubectl run nginx --image=nginx

4.4 ResourceQuota Commands

# Create ResourceQuota
kubectl create quota compute-quota -n test \
  --hard=requests.cpu=4,requests.memory=8Gi,limits.cpu=8,limits.memory=16Gi,pods=10

# View ResourceQuota
kubectl get resourcequota -n test
kubectl get quota -n test
kubectl describe quota compute-quota -n test

# View ResourceQuota usage
kubectl get quota -n test -o yaml    # Shows Used and Hard

---

5. Useful Command Summary

# View node resources
kubectl top node
kubectl top pod -n <namespace>

# View node allocatable resources
kubectl describe node <node-name> | grep -E "Capacity|Allocatable"

# View node allocated resources
kubectl describe node <node-name> | grep -A 5 "Allocated resources"

# Create a Pod with resource limits (imperative)
kubectl run nginx --image=nginx --restart=Never \
  --requests='cpu=250m,memory=128Mi' \
  --limits='cpu=500m,memory=256Mi'

# Quickly create LimitRange (--dry-run)
kubectl create limitrange cpu-limit --dry-run=client -o yaml > limitrange.yaml
# Edit to add default/defaultRequest/max/min

# Quickly create ResourceQuota
kubectl create quota my-quota --hard=pods=5,services=3 --dry-run=client -o yaml

# Find Pods with BestEffort QoS class
kubectl get pods -o json | jq '.items[] | select(.status.qosClass=="BestEffort") | .metadata.name'

# Set ResourceQuota for a namespace
kubectl create namespace restricted
kubectl create quota ns-quota -n restricted --hard=requests.cpu=2,requests.memory=4Gi,pods=5

---

6. Common Scenarios

Scenario 1: Create a Namespace and Set Resource Limits

kubectl create namespace team-a
kubectl create quota team-quota -n team-a --hard=requests.cpu=2,requests.memory=4Gi,pods=10
# At this point, creating a Pod requires specifying requests, otherwise an error occurs
# Solution: first create a LimitRange to provide default values

Scenario 2: Pod Cannot Be Scheduled Due to Insufficient Resources

# View Pod events
kubectl describe pod <pod-name> | grep -A 10 Events

# Common error: 0/3 nodes are available: 1 Insufficient cpu, 2 Insufficient memory
# Solution: Check requests settings, or add more nodes

Scenario 3: OOMKilled (Memory Overrun)

# Pod status shows OOMKilled
kubectl get pod <pod-name>
# Output: CrashLoopBackOff or Error

kubectl describe pod <pod-name> | grep -i oom
# Last State: Terminated
#   Reason: OOMKilled
#   Exit Code: 137

# Solution: Increase memory limits or optimize application memory usage

---

🧪 Complete Hands-On Example: Configure Pod Resource Limits and Observe QoS Classes

Scenario

Create three Pods corresponding to the three QoS classes (Guaranteed/Burstable/BestEffort) and observe the differences.

Prerequisites

• A working Kubernetes cluster
• kubectl configured to connect to the cluster

Steps

Step 1: Create Guaranteed QoS Pod (requests == limits)

cat <<'EOF' > pod-guaranteed.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-guaranteed
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: "500m"
        memory: "256Mi"
      limits:
        cpu: "500m"
        memory: "256Mi"
EOF

kubectl apply -f pod-guaranteed.yaml
# Expected output: pod/pod-guaranteed created

Step 2: Create Burstable QoS Pod (requests < limits)

cat <<'EOF' > pod-burstable.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-burstable
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        cpu: "250m"
        memory: "128Mi"
      limits:
        cpu: "500m"
        memory: "256Mi"
EOF

kubectl apply -f pod-burstable.yaml
# Expected output: pod/pod-burstable created

Step 3: Create BestEffort QoS Pod (no resources field)

cat <<'EOF' > pod-besteffort.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-besteffort
spec:
  containers:
  - name: app
    image: nginx
EOF

kubectl apply -f pod-besteffort.yaml
# Expected output: pod/pod-besteffort created

Verification

# Method 1: Use jsonpath to view QoS class
kubectl get pod pod-guaranteed -o jsonpath='{.status.qosClass}'
# Expected output: Guaranteed

kubectl get pod pod-burstable -o jsonpath='{.status.qosClass}'
# Expected output: Burstable

kubectl get pod pod-besteffort -o jsonpath='{.status.qosClass}'
# Expected output: BestEffort

# Method 2: Use describe
kubectl describe pod pod-guaranteed | grep QoS
# Expected output: QoS Class: Guaranteed

kubectl describe pod pod-burstable | grep QoS
# Expected output: QoS Class: Burstable

kubectl describe pod pod-besteffort | grep QoS
# Expected output: QoS Class: BestEffort

# Method 3: View all Pod QoS classes at once
for p in pod-guaranteed pod-burstable pod-besteffort; do
  echo "$p: $(kubectl get pod $p -o jsonpath='{.status.qosClass}')"
done
# Expected output:
# pod-guaranteed: Guaranteed
# pod-burstable: Burstable
# pod-besteffort: BestEffort

# Clean up
kubectl delete pod pod-guaranteed pod-burstable pod-besteffort
# Expected output: pod "pod-guaranteed" deleted
#                  pod "pod-burstable" deleted
#                  pod "pod-besteffort" deleted

Exam Tips

• The condition for Guaranteed is that all containers must have requests and limits exactly equal (both CPU and memory must match)
• In a multi-container Pod, if even one container has no resources set, it is BestEffort; if even one container has requests != limits, it is Burstable
• When node resources are insufficient, the eviction order is: BestEffort > Burstable > Guaranteed
• The CKA exam often requires creating Pods with resource limits and then checking QoS using kubectl get pod -o jsonpath or -o yaml | grep qosClass
• Resource unit notes: CPU can be specified without quotes (cpu: 1), but memory must include units (memory: "256Mi")

---

Official Documentation Links

• Resource Management for Pods and Containers
• Quality of Service (QoS)
• LimitRange
• ResourceQuota
• Configure Default Memory Requests and Limits
• Node-pressure Eviction