Prerequisites: service, clusterip, endpointslice, iptables, daemonset

kube-proxy is the component that, on a normal cluster, keeps the agreement a clusterip stands for. It is the most misunderstood piece of Kubernetes networking, because the name suggests traffic flows through it, and it does not. kube-proxy is a controller. It runs as a daemonset, one copy per node, watches every service and the endpointslice behind it, and writes kernel rules so the kernel itself rewrites packets headed for a ClusterIP to a real backend pod.

You cannot watch kube-proxy on devata, because devata does not run it. So watch it on a throwaway cluster that does. Create one with kind, give it a Service, and read the rules it programmed into iptables:

kind create cluster --name kproxy
kubectl --context kind-kproxy create deployment web --image=nginx
kubectl --context kind-kproxy expose deployment web --port=80
kubectl --context kind-kproxy get svc web
docker exec kproxy-control-plane iptables-save -t nat | grep <the web ClusterIP>

The rule you find DNATs the ClusterIP to the nginx pod. That is the entire job. Now prove what it means by removing it: kubectl --context kind-kproxy -n kube-system delete daemonset kube-proxy, create a second Service, and look for its rule. There is none, because the only thing that would have written it is gone, and a pod reaching that Service’s ClusterIP now sends packets nowhere, even though every node still says Ready. Delete the sandbox with kind delete cluster --name kproxy when done. The full walkthrough is going-cilium-only.

The catch that motivated its replacement: every Service adds more rules, and that pile grows with the cluster. That is the problem cilium solves with ebpf.

Reference: virtual IPs and Service proxies.