cat README.md
Two GKE clusters, identical down to the version, zone, and machine type — save one thing: how they move a packet. The same LoadBalancer Service, fronting three nginx pods, runs on the legacy dataplane (kube-proxy writing iptables) and on Dataplane V2 (managed Cilium, in eBPF). Here they are, traced side by side down every layer — legacy on the left, eBPF on the right.
Most layers come out identical — the dim ones below. The dataplane truly diverges at L4 (DNAT) and L6 (failover); then eBPF adds two things iptables has no answer for — L7 observability and L8 policy.
gcloud compute forwarding-rules list \
--filter="IPAddress=136.64.249.50" NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
ac973ff85436744bfa586378db41283e us-central1 136.64.249.50 TCP us-central1/targetPools/ac973ff85436744bfa586378db41283e
The external VIP is a regional GCE forwarding rule pointing at a target pool.
kubectl -n netflow-test get svc nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx LoadBalancer 10.24.9.186 136.64.249.50 80:31057/TCP 50m
One Service, three IPs: ClusterIP 10.24.9.186 (internal), external VIP 136.64.249.50, NodePort 31057.
kubectl -n netflow-test describe svc nginx Name: nginx
Namespace: netflow-test
Labels: <none>
Annotations: cloud.google.com/neg: {"ingress":true}
networking.gke.io/target-pool: ac973ff85436744bfa586378db41283e
Selector: app=nginx
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.24.9.186
IPs: 10.24.9.186
LoadBalancer Ingress: 136.64.249.50 (VIP)
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 31057/TCP
Endpoints: 10.20.1.13:80,10.20.0.8:80,10.20.0.9:80
Session Affinity: None
External Traffic Policy: Cluster
Internal Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 46m (x4 over 50m) service-controller Ensuring load balancer
Normal EnsuredLoadBalancer 45m (x4 over 49m) service-controller Ensured load balancer
The GKE target-pool annotation and the three backing pod endpoints show up here.
gcloud compute forwarding-rules list \
--filter="IPAddress=34.72.78.182" NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
a433432921d1a44f9b12e84eff8ab128 us-central1 34.72.78.182 TCP us-central1/targetPools/a433432921d1a44f9b12e84eff8ab128
Google's LB frontend is identical regardless of dataplane — the forwarding rule doesn't know or care about eBPF.
kubectl -n netflow-test get svc nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx LoadBalancer 10.24.8.39 34.72.78.182 80:30574/TCP 47s
Same three-IP shape as V1: ClusterIP 10.24.8.39, external VIP 34.72.78.182, NodePort 30574. The Kubernetes objects are identical — only the dataplane below differs.
kubectl -n netflow-test describe svc nginx Name: nginx
Namespace: netflow-test
Labels: <none>
Annotations: cloud.google.com/neg: {"ingress":true}
networking.gke.io/target-pool: a433432921d1a44f9b12e84eff8ab128
Selector: app=nginx
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.24.8.39
IPs: 10.24.8.39
LoadBalancer Ingress: 34.72.78.182 (VIP)
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 30574/TCP
Endpoints: 10.20.1.9:80,10.20.1.10:80,10.20.0.15:80
Session Affinity: None
External Traffic Policy: Cluster
Internal Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 6s (x2 over 47s) service-controller Ensuring load balancer
Normal EnsuredLoadBalancer 1s (x2 over 6s) service-controller Ensured load balancer
Byte-for-byte the same Service spec as V1. The divergence lives at L4, not here.
gcloud compute target-pools describe \
ac973ff85436744bfa586378db41283e --region us-central1 creationTimestamp: '2026-07-01T17:11:48.998-07:00'
description: '{"kubernetes.io/service-name":"netflow-test/nginx"}'
healthChecks:
- https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/global/httpHealthChecks/k8s-d22eb523ac8ee445-node
id: '2121913344195039787'
instances:
- https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/zones/us-central1-c/instances/gke-authlab-gke-primary-c7a308a6-bnmh
- https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/zones/us-central1-c/instances/gke-authlab-gke-primary-c7a308a6-wvbv
kind: compute#targetPool
name: ac973ff85436744bfa586378db41283e
region: https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/regions/us-central1
selfLink: https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/regions/us-central1/targetPools/ac973ff85436744bfa586378db41283e
sessionAffinity: NONE
GKE's L2 advertisement is Google's frontend: the two nodes are registered as target-pool backends behind a node health check.
gcloud compute target-pools describe \
<forwarding-rule-name> --region us-central1 creationTimestamp: '2026-07-02T06:23:25.031-07:00'
description: '{"kubernetes.io/service-name":"netflow-test/nginx"}'
healthChecks:
- https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/global/httpHealthChecks/k8s-8abe6c2aefb4360c-node
id: '368248394460420258'
instances:
- https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/zones/us-central1-c/instances/gke-authlab-gke-primary-a62dd7d5-91t0
- https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/zones/us-central1-c/instances/gke-authlab-gke-primary-a62dd7d5-q8b5
kind: compute#targetPool
name: a433432921d1a44f9b12e84eff8ab128
region: https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/regions/us-central1
selfLink: https://www.googleapis.com/compute/v1/projects/authlab-gke-lab/regions/us-central1/targetPools/a433432921d1a44f9b12e84eff8ab128
sessionAffinity: NONE
Edge advertisement is unchanged: the same two nodes registered as target-pool backends. The dataplane divergence is inside the node, past the edge.
kubectl -n netflow-test get endpoints nginx NAME ENDPOINTS AGE
nginx 10.20.0.8:80,10.20.0.9:80,10.20.1.13:80 50m
The three pod IPs (10.20.x.x) that any of the three entry points ultimately reach.
kubectl -n netflow-test get endpointslices -l \
kubernetes.io/service-name=nginx -o wide NAME ADDRESSTYPE PORTS ENDPOINTS AGE
nginx-66c7s IPv4 80 10.20.1.13,10.20.0.8,10.20.0.9 50m
The EndpointSlice is the source of truth kube-proxy watches to program the dataplane.
kubectl -n netflow-test get svc nginx -o \
jsonpath='{.spec.ports[0].nodePort}' NodePort: 31057
Every node opens :31057, but no process listens on it — it's a DNAT rewrite rule, not a socket.
kubectl -n netflow-test get endpoints nginx NAME ENDPOINTS AGE
nginx 10.20.0.15:80,10.20.1.10:80,10.20.1.9:80 50s
The same three pod IPs — but here Cilium (anetd), not kube-proxy, is the component watching them.
kubectl -n netflow-test get endpointslices -l \
kubernetes.io/service-name=nginx -o wide NAME ADDRESSTYPE PORTS ENDPOINTS AGE
nginx-cpdvm IPv4 80 10.20.1.9,10.20.1.10,10.20.0.15 50s
Same source-of-truth object; the consumer changed from kube-proxy to the Cilium agent.
kubectl -n netflow-test get svc nginx -o \
jsonpath='{.spec.ports[0].nodePort}' NodePort: 30574
Same NodePort — now programmed as an entry in the eBPF service map instead of an iptables rule.
kubectl debug node/<node> --profile=sysadmin \
--image=netshoot -- iptables-save -t nat | grep -E \
'KUBE-SVC|KUBE-SEP' :KUBE-SEP-6WWOHZMG4JZC35KS - [0:0]
:KUBE-SEP-DHOMQKZHY6OP2O7F - [0:0]
:KUBE-SEP-FCUWAI27TURSNJ3V - [0:0]
:KUBE-SEP-NOCIEW2ZPJ2KT5NU - [0:0]
:KUBE-SEP-OM74ABY4QM3JMIN3 - [0:0]
:KUBE-SEP-OU3PBSM4WDVQOGDB - [0:0]
:KUBE-SEP-P5UCONCOPU635JFJ - [0:0]
:KUBE-SEP-PB43UR2RE63FHQ6K - [0:0]
:KUBE-SEP-PZKQCIXFD46UYYKL - [0:0]
:KUBE-SEP-RLIUIU6TARNS75T3 - [0:0]
:KUBE-SEP-SR5EBPYJ4Z6OR2QY - [0:0]
:KUBE-SEP-UIVMLVLGJWJX3OYQ - [0:0]
:KUBE-SEP-UPX3ADRPCRZFTSOK - [0:0]
:KUBE-SEP-WINV7FEMCBPDEZGG - [0:0]
:KUBE-SEP-X4MLFZ67G53NH3PU - [0:0]
:KUBE-SEP-XQZZT3NX54ABB7RG - [0:0]
:KUBE-SEP-ZRNKUAQ5OXIGKDJC - [0:0]
:KUBE-SVC-BRK3P4PPQWCLKOAN - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-FXR4M2CWOGAZGGYD - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-P2CJC57QC736K2NH - [0:0]
:KUBE-SVC-QMWWTXBG7KFJQKLO - [0:0]
:KUBE-SVC-SHZP3UT6BYNSPWES - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:KUBE-SVC-XBBXYMVKK37OV7LG - [0:0]
:KUBE-SVC-XP4WJ6VSLGWALMW5 - [0:0]
:KUBE-SVC-YTPYYJDCMEBC5FLU - [0:0]
-A KUBE-EXT-P2CJC57QC736K2NH -m comment --comment "masquerade traffic for netflow-test/nginx external destinations" -j KUBE-MARK-MASQ
-A KUBE-EXT-P2CJC57QC736K2NH -j KUBE-SVC-P2CJC57QC736K2NH
-A KUBE-EXT-XP4WJ6VSLGWALMW5 -j KUBE-SVC-XP4WJ6VSLGWALMW5
-A KUBE-NODEPORTS -d 127.0.0.0/8 -p tcp -m comment --comment "netflow-test/nginx" -m tcp --dport 31057 -m nfacct --nfacct-name localhost_nps_accepted_pkts -j KUBE-EXT-P2CJC57QC736K2NH
-A KUBE-NODEPORTS -p tcp -m comment --comment "netflow-test/nginx" -m tcp --dport 31057 -j KUBE-EXT-P2CJC57QC736K2NH
-A KUBE-SEP-6WWOHZMG4JZC35KS -s 10.20.0.4/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-6WWOHZMG4JZC35KS -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.20.0.4:53
-A KUBE-SEP-DHOMQKZHY6OP2O7F -s 10.20.1.9/32 -m comment --comment "kube-system/kube-dns-upstream:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-DHOMQKZHY6OP2O7F -p udp -m comment --comment "kube-system/kube-dns-upstream:dns" -m udp -j DNAT --to-destination 10.20.1.9:53
-A KUBE-SEP-FCUWAI27TURSNJ3V -s 10.20.1.10/32 -m comment --comment "kube-system/default-http-backend:http" -j KUBE-MARK-MASQ
-A KUBE-SEP-FCUWAI27TURSNJ3V -p tcp -m comment --comment "kube-system/default-http-backend:http" -m tcp -j DNAT --to-destination 10.20.1.10:8080
-A KUBE-SEP-NOCIEW2ZPJ2KT5NU -s 10.20.1.9/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-NOCIEW2ZPJ2KT5NU -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.20.1.9:53
-A KUBE-SEP-OM74ABY4QM3JMIN3 -s 10.20.0.4/32 -m comment --comment "kube-system/kube-dns-upstream:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-OM74ABY4QM3JMIN3 -p udp -m comment --comment "kube-system/kube-dns-upstream:dns" -m udp -j DNAT --to-destination 10.20.0.4:53
-A KUBE-SEP-OU3PBSM4WDVQOGDB -s 10.20.0.5/32 -m comment --comment "cosign-system/policy-controller-webhook-metrics:metrics" -j KUBE-MARK-MASQ
-A KUBE-SEP-OU3PBSM4WDVQOGDB -p tcp -m comment --comment "cosign-system/policy-controller-webhook-metrics:metrics" -m tcp -j DNAT --to-destination 10.20.0.5:9090
-A KUBE-SEP-P5UCONCOPU635JFJ -s 10.20.1.13/32 -m comment --comment "netflow-test/nginx" -j KUBE-MARK-MASQ
-A KUBE-SEP-P5UCONCOPU635JFJ -p tcp -m comment --comment "netflow-test/nginx" -m tcp -j DNAT --to-destination 10.20.1.13:80
-A KUBE-SEP-PB43UR2RE63FHQ6K -s 10.20.0.4/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-PB43UR2RE63FHQ6K -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.20.0.4:53
-A KUBE-SEP-PZKQCIXFD46UYYKL -s 10.20.1.4/32 -m comment --comment "kube-system/metrics-server" -j KUBE-MARK-MASQ
-A KUBE-SEP-PZKQCIXFD46UYYKL -p tcp -m comment --comment "kube-system/metrics-server" -m tcp -j DNAT --to-destination 10.20.1.4:10250
-A KUBE-SEP-RLIUIU6TARNS75T3 -s 10.20.0.5/32 -m comment --comment "cosign-system/webhook:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-RLIUIU6TARNS75T3 -p tcp -m comment --comment "cosign-system/webhook:https" -m tcp -j DNAT --to-destination 10.20.0.5:8443
-A KUBE-SEP-SR5EBPYJ4Z6OR2QY -s 172.16.0.2/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-SR5EBPYJ4Z6OR2QY -p tcp -m comment --comment "default/kubernetes:https" -m tcp -j DNAT --to-destination 172.16.0.2:443
-A KUBE-SEP-UIVMLVLGJWJX3OYQ -s 10.20.0.4/32 -m comment --comment "kube-system/kube-dns-upstream:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-UIVMLVLGJWJX3OYQ -p tcp -m comment --comment "kube-system/kube-dns-upstream:dns-tcp" -m tcp -j DNAT --to-destination 10.20.0.4:53
-A KUBE-SEP-UPX3ADRPCRZFTSOK -s 10.20.1.6/32 -m comment --comment "gmp-system/gmp-operator:webhook" -j KUBE-MARK-MASQ
-A KUBE-SEP-UPX3ADRPCRZFTSOK -p tcp -m comment --comment "gmp-system/gmp-operator:webhook" -m tcp -j DNAT --to-destination 10.20.1.6:10250
-A KUBE-SEP-WINV7FEMCBPDEZGG -s 10.20.0.8/32 -m comment --comment "netflow-test/nginx" -j KUBE-MARK-MASQ
-A KUBE-SEP-WINV7FEMCBPDEZGG -p tcp -m comment --comment "netflow-test/nginx" -m tcp -j DNAT --to-destination 10.20.0.8:80
-A KUBE-SEP-X4MLFZ67G53NH3PU -s 10.20.0.9/32 -m comment --comment "netflow-test/nginx" -j KUBE-MARK-MASQ
-A KUBE-SEP-X4MLFZ67G53NH3PU -p tcp -m comment --comment "netflow-test/nginx" -m tcp -j DNAT --to-destination 10.20.0.9:80
-A KUBE-SEP-XQZZT3NX54ABB7RG -s 10.20.1.9/32 -m comment --comment "kube-system/kube-dns-upstream:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-XQZZT3NX54ABB7RG -p tcp -m comment --comment "kube-system/kube-dns-upstream:dns-tcp" -m tcp -j DNAT --to-destination 10.20.1.9:53
-A KUBE-SEP-ZRNKUAQ5OXIGKDJC -s 10.20.1.9/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-ZRNKUAQ5OXIGKDJC -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.20.1.9:53
-A KUBE-SERVICES -d 10.24.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES -d 10.24.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES -d 10.24.11.196/32 -p udp -m comment --comment "kube-system/kube-dns-upstream:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-FXR4M2CWOGAZGGYD
-A KUBE-SERVICES -d 10.24.14.136/32 -p tcp -m comment --comment "kube-system/metrics-server cluster IP" -m tcp --dport 443 -j KUBE-SVC-QMWWTXBG7KFJQKLO
-A KUBE-SERVICES -d 10.24.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -d 10.24.9.20/32 -p tcp -m comment --comment "gmp-system/gmp-operator:webhook cluster IP" -m tcp --dport 443 -j KUBE-SVC-XBBXYMVKK37OV7LG
-A KUBE-SERVICES -d 10.24.11.196/32 -p tcp -m comment --comment "kube-system/kube-dns-upstream:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-BRK3P4PPQWCLKOAN
-A KUBE-SERVICES -d 10.24.6.139/32 -p tcp -m comment --comment "cosign-system/policy-controller-webhook-metrics:metrics cluster IP" -m tcp --dport 9090 -j KUBE-SVC-SHZP3UT6BYNSPWES
-A KUBE-SERVICES -d 10.24.2.92/32 -p tcp -m comment --comment "cosign-system/webhook:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-YTPYYJDCMEBC5FLU
-A KUBE-SERVICES -d 10.24.9.186/32 -p tcp -m comment --comment "netflow-test/nginx cluster IP" -m tcp --dport 80 -j KUBE-SVC-P2CJC57QC736K2NH
-A KUBE-SERVICES -d 136.64.249.50/32 -p tcp -m comment --comment "netflow-test/nginx loadbalancer IP" -m tcp --dport 80 -j KUBE-EXT-P2CJC57QC736K2NH
-A KUBE-SERVICES -d 10.24.4.246/32 -p tcp -m comment --comment "kube-system/default-http-backend:http cluster IP" -m tcp --dport 80 -j KUBE-SVC-XP4WJ6VSLGWALMW5
-A KUBE-SVC-BRK3P4PPQWCLKOAN ! -s 10.20.1.0/24 -d 10.24.11.196/32 -p tcp -m comment --comment "kube-system/kube-dns-upstream:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-BRK3P4PPQWCLKOAN -m comment --comment "kube-system/kube-dns-upstream:dns-tcp -> 10.20.0.4:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-UIVMLVLGJWJX3OYQ
-A KUBE-SVC-BRK3P4PPQWCLKOAN -m comment --comment "kube-system/kube-dns-upstream:dns-tcp -> 10.20.1.9:53" -j KUBE-SEP-XQZZT3NX54ABB7RG
-A KUBE-SVC-ERIFXISQEP7F7OF4 ! -s 10.20.1.0/24 -d 10.24.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.20.0.4:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-6WWOHZMG4JZC35KS
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.20.1.9:53" -j KUBE-SEP-NOCIEW2ZPJ2KT5NU
-A KUBE-SVC-FXR4M2CWOGAZGGYD ! -s 10.20.1.0/24 -d 10.24.11.196/32 -p udp -m comment --comment "kube-system/kube-dns-upstream:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-FXR4M2CWOGAZGGYD -m comment --comment "kube-system/kube-dns-upstream:dns -> 10.20.0.4:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-OM74ABY4QM3JMIN3
-A KUBE-SVC-FXR4M2CWOGAZGGYD -m comment --comment "kube-system/kube-dns-upstream:dns -> 10.20.1.9:53" -j KUBE-SEP-DHOMQKZHY6OP2O7F
-A KUBE-SVC-NPX46M4PTMTKRN6Y ! -s 10.20.1.0/24 -d 10.24.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https -> 172.16.0.2:443" -j KUBE-SEP-SR5EBPYJ4Z6OR2QY
-A KUBE-SVC-P2CJC57QC736K2NH ! -s 10.20.1.0/24 -d 10.24.9.186/32 -p tcp -m comment --comment "netflow-test/nginx cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SVC-P2CJC57QC736K2NH -m comment --comment "netflow-test/nginx -> 10.20.0.8:80" -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-WINV7FEMCBPDEZGG
-A KUBE-SVC-P2CJC57QC736K2NH -m comment --comment "netflow-test/nginx -> 10.20.0.9:80" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-X4MLFZ67G53NH3PU
-A KUBE-SVC-P2CJC57QC736K2NH -m comment --comment "netflow-test/nginx -> 10.20.1.13:80" -j KUBE-SEP-P5UCONCOPU635JFJ
-A KUBE-SVC-QMWWTXBG7KFJQKLO ! -s 10.20.1.0/24 -d 10.24.14.136/32 -p tcp -m comment --comment "kube-system/metrics-server cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-QMWWTXBG7KFJQKLO -m comment --comment "kube-system/metrics-server -> 10.20.1.4:10250" -j KUBE-SEP-PZKQCIXFD46UYYKL
-A KUBE-SVC-SHZP3UT6BYNSPWES ! -s 10.20.1.0/24 -d 10.24.6.139/32 -p tcp -m comment --comment "cosign-system/policy-controller-webhook-metrics:metrics cluster IP" -m tcp --dport 9090 -j KUBE-MARK-MASQ
-A KUBE-SVC-SHZP3UT6BYNSPWES -m comment --comment "cosign-system/policy-controller-webhook-metrics:metrics -> 10.20.0.5:9090" -j KUBE-SEP-OU3PBSM4WDVQOGDB
-A KUBE-SVC-TCOU7JCQXEZGVUNU ! -s 10.20.1.0/24 -d 10.24.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.20.0.4:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-PB43UR2RE63FHQ6K
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.20.1.9:53" -j KUBE-SEP-ZRNKUAQ5OXIGKDJC
-A KUBE-SVC-XBBXYMVKK37OV7LG ! -s 10.20.1.0/24 -d 10.24.9.20/32 -p tcp -m comment --comment "gmp-system/gmp-operator:webhook cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-XBBXYMVKK37OV7LG -m comment --comment "gmp-system/gmp-operator:webhook -> 10.20.1.6:10250" -j KUBE-SEP-UPX3ADRPCRZFTSOK
-A KUBE-SVC-XP4WJ6VSLGWALMW5 ! -s 10.20.1.0/24 -d 10.24.4.246/32 -p tcp -m comment --comment "kube-system/default-http-backend:http cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SVC-XP4WJ6VSLGWALMW5 -m comment --comment "kube-system/default-http-backend:http -> 10.20.1.10:8080" -j KUBE-SEP-FCUWAI27TURSNJ3V
-A KUBE-SVC-YTPYYJDCMEBC5FLU ! -s 10.20.1.0/24 -d 10.24.2.92/32 -p tcp -m comment --comment "cosign-system/webhook:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-YTPYYJDCMEBC5FLU -m comment --comment "cosign-system/webhook:https -> 10.20.0.5:8443" -j KUBE-SEP-RLIUIU6TARNS75T3
A packet addressed to the Service's ClusterIP has to be turned into a packet for a real pod — that rewrite is DNAT. kube-proxy implements it as iptables rules: one chain per Service that picks a backend by cumulative probability (1/3, then 1/2 of the rest, then the remainder, so three pods each get an equal share), then jumps to a per-endpoint chain that rewrites the destination to that pod's IP. One Service expands to ~14 rules, and the kernel walks the chain top-to-bottom for every packet — O(N).
kubectl -n kube-system exec <anetd-pod> -c cilium-agent \
-- cilium-dbg service list ID Frontend Service Type Backend
1 10.24.0.10:53/TCP LocalRedirect 1 => 10.20.0.13:53/TCP (active)
2 10.24.0.10:53/UDP LocalRedirect 1 => 10.20.0.13:53/UDP (active)
3 10.24.14.246:53/TCP ClusterIP 1 => 10.20.0.6:53/TCP (active)
2 => 10.20.1.5:53/TCP (active)
4 10.24.14.246:53/UDP ClusterIP 1 => 10.20.0.6:53/UDP (active)
2 => 10.20.1.5:53/UDP (active)
5 10.24.3.59:443/TCP ClusterIP 1 => 10.20.0.7:10250/TCP (active)
6 10.24.3.59:8443/TCP ClusterIP
7 10.24.6.208:19092/TCP ClusterIP
8 10.24.5.43:443/TCP ClusterIP
9 10.24.6.8:443/TCP ClusterIP 1 => 10.20.0.8:10250/TCP (active)
10 10.24.0.1:443/TCP ClusterIP 1 => 172.16.0.2:443/TCP (active)
11 0.0.0.0:32458/TCP NodePort 1 => 10.20.0.3:8080/TCP (active)
13 10.24.10.5:80/TCP ClusterIP 1 => 10.20.0.3:8080/TCP (active)
16 10.24.1.81:9090/TCP ClusterIP 1 => 10.20.1.8:9090/TCP (active)
17 10.24.1.216:443/TCP ClusterIP 1 => 10.20.1.8:8443/TCP (active)
18 0.0.0.0:30574/TCP NodePort 1 => 10.20.0.15:80/TCP (active)
2 => 10.20.1.9:80/TCP (active)
3 => 10.20.1.10:80/TCP (active)
20 10.24.8.39:80/TCP ClusterIP 1 => 10.20.0.15:80/TCP (active)
2 => 10.20.1.9:80/TCP (active)
3 => 10.20.1.10:80/TCP (active)
21 34.72.78.182:80/TCP LoadBalancer 1 => 10.20.0.15:80/TCP (active)
2 => 10.20.1.9:80/TCP (active)
3 => 10.20.1.10:80/TCP (active)
Same job — turn a ClusterIP packet into a pod packet — but Cilium stores the mapping as rows in an in-kernel hash map instead of a rule tree. Each frontend (a Service IP:port) points at a list of backend pods; the eBPF program does a single O(1) lookup per packet and rewrites the destination. Notice nginx appears three times: ClusterIP, NodePort, and LoadBalancer are three frontends resolving to the same backend set. The whole state is one structured query, versus V1's 100+ lines of chains to read by eye.
kubectl -n netflow-test exec <pod> -- wget -qO- \
localhost | head -5 <!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
The pod itself serves the page on localhost — the entry points are all just paths to this process.
kubectl -n netflow-test exec <pod> -- ip addr show eth0; \
kubectl -n netflow-test exec <pod> -- cat /proc/net/tcp Inspecting pod: nginx-bb6b8c496-4d49g
--- Interface (the 'real' IP) ---
2: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP qlen 1000
link/ether a6:b1:ed:39:f3:a4 brd ff:ff:ff:ff:ff:ff
inet 10.20.0.9/24 brd 10.20.0.255 scope global eth0
valid_lft forever preferred_lft forever
--- Listening sockets (/proc/net/tcp) ---
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:0050 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 73957 1 0000000000000000 100 0 0 10 0
eth0 carries the real pod IP (one end of a veth pair); /proc/net/tcp shows nginx listening on :0050 (port 80).
kubectl -n netflow-test exec <pod> -- wget -qO- \
localhost | head -5 <!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
Same nginx, same localhost serve — the ultimate destination never changed.
kubectl -n netflow-test exec <pod> -- ip addr show eth0; \
kubectl -n netflow-test exec <pod> -- cat /proc/net/tcp Inspecting pod: nginx-bb6b8c496-jfd4v
--- Interface (the 'real' IP) ---
2: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP qlen 1000
link/ether be:1b:62:45:aa:0e brd ff:ff:ff:ff:ff:ff
inet 10.20.1.10/24 brd 10.20.1.255 scope global eth0
valid_lft forever preferred_lft forever
--- Listening sockets (/proc/net/tcp) ---
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:0050 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 529092 1 0000000000000000 100 0 0 10 0
The pod side is identical — same veth pair, same eth0 pod IP, nginx on :0050. The dataplane changes how packets reach eth0, never the pod itself.
for i in $(seq 10); do curl -s -o /dev/null -w \
'%{http_code} %{time_total}\n' http://136.64.249.50; \
done # ETP=Cluster === Cluster mode, 3 pods on N nodes ===
Request 1: HTTP 200 (time 0.104455s)
Request 2: HTTP 200 (time 0.103758s)
Request 3: HTTP 200 (time 0.112161s)
Request 4: HTTP 200 (time 0.100225s)
Request 5: HTTP 200 (time 0.113110s)
Request 6: HTTP 200 (time 0.099327s)
Request 7: HTTP 200 (time 0.100386s)
Request 8: HTTP 200 (time 0.104417s)
Request 9: HTTP 200 (time 0.104898s)
Request 10: HTTP 200 (time 0.101342s)
externalTrafficPolicy decides what a node does with external traffic that lands on it. 'Cluster' (the default) lets any node forward to a pod on any node — even hopping to another node — so every request succeeds. The cost is an extra hop and the client's IP being replaced by the node's (SNAT). All ten requests return 200 at ~0.1s.
kubectl -n netflow-test patch svc nginx -p \
'{"spec":{"externalTrafficPolicy":"Local"}}' # then \
probe === Local mode, 3 pods (symmetric) ===
Request 1: HTTP 200 (time 0.100833s)
Request 2: HTTP 200 (time 0.097615s)
Request 3: HTTP 200 (time 0.111295s)
Request 4: HTTP 200 (time 0.102077s)
Request 5: HTTP 200 (time 0.117952s)
Request 6: HTTP 200 (time 0.111171s)
Request 7: HTTP 200 (time 0.101594s)
Request 8: HTTP 200 (time 0.103917s)
Request 9: HTTP 200 (time 0.103953s)
Request 10: HTTP 200 (time 0.103402s)
'Local' mode forwards only to a pod on the receiving node — preserving the real client IP (no SNAT) — and relies on a health check to steer around nodes with no local pod. With a pod on every node (symmetric topology), that trade-off is invisible: this looks exactly like Cluster mode. It only bites when the topology is uneven (next).
for i in $(seq 20); do curl -s -o /dev/null -w \
'%{http_code} %{time_total}\n' http://136.64.249.50; \
done # ETP=Local, 1 pod === Local mode, 1 pod on N nodes (failover window) ===
Request 1: HTTP 000 (time 6.006635s)
Request 2: HTTP 000 (time 6.003048s)
Request 3: HTTP 000 (time 6.006082s)
Request 4: HTTP 000 (time 6.004830s)
Request 5: HTTP 000 (time 6.006421s)
Request 6: HTTP 000 (time 6.006136s)
Request 7: HTTP 000 (time 6.002937s)
Request 8: HTTP 000 (time 6.003155s)
Request 9: HTTP 000 (time 6.004225s)
Request 10: HTTP 000 (time 6.002446s)
Request 11: HTTP 000 (time 6.006199s)
Request 12: HTTP 000 (time 6.002653s)
Request 13: HTTP 000 (time 6.005029s)
Request 14: HTTP 000 (time 6.006825s)
Request 15: HTTP 000 (time 6.004151s)
Request 16: HTTP 000 (time 6.006487s)
Request 17: HTTP 000 (time 6.006770s)
Request 18: HTTP 000 (time 6.005179s)
Request 19: HTTP 000 (time 6.002773s)
Request 20: HTTP 000 (time 6.006374s)
With the load balancer still sending to the pod-less node, V1 (iptables) drops those packets into a black hole — the client waits the full 6-second timeout for every one. Nothing recovers inside this 20-request window: the target-pool health check takes ~130 seconds to remove the empty node. Slow, silent failure.
for i in $(seq 10); do curl -s -o /dev/null -w \
'%{http_code} %{time_total}\n' http://136.64.249.50; \
done # ~130s after scale-to-1 === Local mode, 1 pod — AFTER failover convergence (~130s) ===
Request 1: HTTP 000 (time 6.006455s)
Request 2: HTTP 000 (time 6.006284s)
Request 3: HTTP 200 (time 0.101222s)
Request 4: HTTP 000 (time 6.004579s)
Request 5: HTTP 200 (time 0.110207s)
Request 6: HTTP 200 (time 0.112402s)
Request 7: HTTP 200 (time 0.101518s)
Request 8: HTTP 200 (time 0.101016s)
Request 9: HTTP 200 (time 0.105820s)
Request 10: HTTP 200 (time 0.101330s)
Re-probing ~130 seconds later confirms V1 does eventually recover: once the target-pool health check finally drops the pod-less node, requests settle to steady 200s. The behavior isn't broken — just slow to converge, which is the inherent cost of health-check-driven, iptables-era failover.
kubectl -n netflow-test scale deploy/nginx --replicas=1 Deployment: nginx replicas=1
Pod: nginx-bb6b8c496-j6s5g on node gke-authlab-gke-primary-c7a308a6-bnmh
Service externalTrafficPolicy: Local
Nodes:
node/gke-authlab-gke-primary-c7a308a6-bnmh
node/gke-authlab-gke-primary-c7a308a6-wvbv
To expose Local mode's trade-off, scale to a single pod so one of the two backend nodes now has no local endpoint. The external load balancer keeps sending to both nodes until its health check notices and removes the empty one — and how fast that happens is precisely where the two dataplanes part ways.
for i in $(seq 10); do curl -s -o /dev/null -w \
'%{http_code} %{time_total}\n' http://34.72.78.182; done \
# ETP=Cluster === Cluster mode, 3 pods on N nodes ===
Request 1: HTTP 200 (time 1.105134s)
Request 2: HTTP 200 (time 1.116195s)
Request 3: HTTP 200 (time 5.105150s)
Request 4: HTTP 200 (time 0.099194s)
Request 5: HTTP 200 (time 0.110368s)
Request 6: HTTP 200 (time 0.099820s)
Request 7: HTTP 200 (time 0.096816s)
Request 8: HTTP 200 (time 0.110239s)
Request 9: HTTP 200 (time 0.100141s)
Request 10: HTTP 200 (time 0.102655s)
Cluster mode behaves identically on eBPF at steady state — any node forwards to any pod, every request succeeds at ~0.1s. The externalTrafficPolicy contract is the same; the dataplane difference only surfaces during failover, which we force below.
kubectl -n netflow-test patch svc nginx -p \
'{"spec":{"externalTrafficPolicy":"Local"}}' # then \
probe === Local mode, 3 pods (symmetric) ===
Request 1: HTTP 200 (time 0.111140s)
Request 2: HTTP 200 (time 0.102474s)
Request 3: HTTP 200 (time 0.101278s)
Request 4: HTTP 200 (time 0.101190s)
Request 5: HTTP 200 (time 0.103427s)
Request 6: HTTP 200 (time 0.096703s)
Request 7: HTTP 200 (time 0.100782s)
Request 8: HTTP 200 (time 0.101286s)
Request 9: HTTP 200 (time 0.103870s)
Request 10: HTTP 200 (time 0.110058s)
Same story on eBPF: with a pod on every node, Local mode is indistinguishable from Cluster. Whether the dataplane is iptables or eBPF makes no difference while the topology is symmetric — the divergence appears only once we break it.
for i in $(seq 20); do curl -s -o /dev/null -w \
'%{http_code} %{time_total}\n' http://34.72.78.182; done \
# ETP=Local, 1 pod === Local mode, 1 pod on N nodes (failover window) ===
Request 1: HTTP 000 (time 0.054789s)
Request 2: HTTP 000 (time 0.051956s)
Request 3: HTTP 000 (time 0.052886s)
Request 4: HTTP 200 (time 0.102681s)
Request 5: HTTP 000 (time 0.054262s)
Request 6: HTTP 000 (time 0.054924s)
Request 7: HTTP 200 (time 0.103441s)
Request 8: HTTP 000 (time 0.054988s)
Request 9: HTTP 000 (time 0.054369s)
Request 10: HTTP 000 (time 0.056232s)
Request 11: HTTP 200 (time 0.112161s)
Request 12: HTTP 200 (time 0.101623s)
Request 13: HTTP 000 (time 0.054033s)
Request 14: HTTP 000 (time 0.053848s)
Request 15: HTTP 200 (time 0.102416s)
Request 16: HTTP 000 (time 0.055071s)
Request 17: HTTP 200 (time 0.111644s)
Request 18: HTTP 200 (time 0.113271s)
Request 19: HTTP 200 (time 0.102500s)
Request 20: HTTP 200 (time 0.096925s)
The same scenario on eBPF behaves very differently. Requests to the pod-less node fail in ~50 milliseconds, not 6 seconds — the eBPF program actively rejects when there's no local backend instead of black-holing the packet — and traffic converges to steady success within this same short window. Fast-fail plus quick convergence, versus V1's slow-timeout plus ~130s convergence.
kubectl -n netflow-test scale deploy/nginx --replicas=1 Deployment: nginx replicas=1
Pod: nginx-bb6b8c496-w8kkn on node gke-authlab-gke-primary-a62dd7d5-91t0
Service externalTrafficPolicy: Local
Nodes:
node/gke-authlab-gke-primary-a62dd7d5-91t0
node/gke-authlab-gke-primary-a62dd7d5-q8b5
Same setup on V2: one pod, two backend nodes, so one node has no local endpoint. Identical asymmetry — the interesting part is how differently each dataplane handles the requests that land on the pod-less node while the health check catches up.
iptables has no native flow observability. To trace a flow on the legacy dataplane you reconstruct it by hand — conntrack -L, tcpdump — with no identity, verdict, direction, or L7 context. The eBPF datapath emits all of that for free.
hubble observe --namespace netflow-test -o compact Jul 2 15:03:39.769: 162.195.118.115:60598 (world) -> netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: SYN, ECE, CWR)
Jul 2 15:03:39.769: 162.195.118.115:60598 (world) <- netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-stack FORWARDED (TCP Flags: SYN, ACK, ECE)
Jul 2 15:03:39.820: 162.195.118.115:60598 (world) -> netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK)
Jul 2 15:03:39.823: 162.195.118.115:60598 (world) -> netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Jul 2 15:03:39.824: 162.195.118.115:60598 (world) <- netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-stack FORWARDED (TCP Flags: ACK, PSH)
Jul 2 15:03:39.875: 162.195.118.115:60598 (world) -> netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Jul 2 15:03:39.875: 162.195.118.115:60598 (world) <- netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-stack FORWARDED (TCP Flags: ACK, FIN)
Jul 2 15:03:39.897: 10.10.0.3:60599 (remote-node) -> netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: SYN, ECE, CWR)
Jul 2 15:03:39.897: 10.10.0.3:60599 (remote-node) <- netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-stack FORWARDED (TCP Flags: SYN, ACK, ECE)
Jul 2 15:03:39.927: 162.195.118.115:60598 (world) -> netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK)
Jul 2 15:03:39.949: 10.10.0.3:60599 (remote-node) -> netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK)
Jul 2 15:03:39.952: 10.10.0.3:60599 (remote-node) -> netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Jul 2 15:03:39.952: 10.10.0.3:60599 (remote-node) <- netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-stack FORWARDED (TCP Flags: ACK, PSH)
Jul 2 15:03:40.000: 10.10.0.3:60599 (remote-node) -> netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Jul 2 15:03:40.000: 10.10.0.3:60599 (remote-node) <- netflow-test/nginx-bb6b8c496-2gd6m:80 (ID:18565) to-stack FORWARDED (TCP Flags: ACK, FIN)
Jul 2 15:03:40.021: 10.10.0.4:60600 (remote-node) -> netflow-test/nginx-bb6b8c496-w8kkn:80 (ID:18565) to-endpoint FORWARDED (TCP Flags: SYN, ECE, CWR)
This is what eBPF unlocks that iptables cannot: a record of every flow, essentially for free, because the same in-kernel programs that route the packet also emit it. Each line names the source and destination by IDENTITY (world = external, remote-node = another node, plus the nginx endpoint), a VERDICT (FORWARDED), the direction, and TCP flags. On V1 there is no equivalent — you would reconstruct this by hand from conntrack and tcpdump, with no identity or verdict attached.
hubble status --server localhost:4245 --tls \
--tls-client-cert-file ... (via kubectl port-forward \
svc/hubble-relay) Healthcheck (via localhost:4245): Ok
Current/Max Flows: 126/126 (100.00%)
Flows/s: 53.63
Connected Nodes: 2/2
The relay aggregates flows from the Cilium agent on every node, over mTLS. Status shows it healthy with both nodes connected and ~54 flows/second streaming — the observability pipeline is live before we look at any actual traffic.
kubectl -n gke-managed-dpv2-observability get pods,svc NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/hubble-relay-56b84459c4-cqzqj 3/3 Running 0 13m 10.20.1.13 gke-authlab-gke-primary-a62dd7d5-q8b5 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/hubble-relay ClusterIP 10.24.9.240 <none> 443/TCP 13m k8s-app=hubble-relay
Hubble is Cilium's flow-observability layer. On GKE you don't install it — enabling one flag makes Google deploy the Hubble Relay into a managed namespace for you. That's the 'managed' half of managed Cilium: the capability is there, but you flip a switch instead of running the components yourself (and get a leaner surface than a self-managed cluster would).
Our V1 cluster ran with the network-policy addon OFF — the legacy dataplane enforced nothing, so any pod could reach any pod. Turning it on means the Calico addon: iptables rules matched on pod IP/CIDR, with no notion of workload identity. The eBPF dataplane instead enforces by a numeric identity derived from labels.
kubectl -n netflow-test get networkpolicy \
nginx-allow-frontend -o yaml apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: nginx-allow-frontend
namespace: netflow-test
spec:
podSelector:
matchLabels:
app: nginx
policyTypes: [Ingress]
ingress:
- from:
- podSelector:
matchLabels:
role: frontend
Network policy is a firewall for pod-to-pod traffic. This is a plain Kubernetes NetworkPolicy — note there is no CiliumNetworkPolicy on managed GKE — declaring: pods labeled app=nginx accept ingress only from pods labeled role=frontend, and everything else is denied by default. Next we watch the eBPF dataplane actually enforce it.
kubectl -n netflow-test get ciliumendpoints -o \
custom-columns=POD:..,IDENTITY:..,IP:.. POD IDENTITY IP
client-allowed 51014 10.20.0.18
client-blocked 6385 10.20.1.14
nginx-bb6b8c496-2gd6m 18565 10.20.1.12
nginx-bb6b8c496-vc6pj 18565 10.20.1.11
nginx-bb6b8c496-w8kkn 18565 10.20.0.15
The idea that makes eBPF policy scale: Cilium doesn't reason about pod IPs, it assigns each workload a numeric IDENTITY derived from its labels. All three nginx pods share one identity (18565) because their labels match; the two clients get their own. Enforcement is then identity-to-identity, so it survives pods being recreated with new IPs — the churn that would endlessly rewrite an IP-based iptables ruleset.
kubectl -n netflow-test exec client-{allowed,blocked} -- \
curl -s http://nginx # NetworkPolicy active: nginx accepts ingress only from role=frontend
client-allowed (role=frontend) -> nginx : HTTP 200 (0.006s) ✓ allowed
client-blocked (role=other) -> nginx : HTTP 000 (5.0s timeout) ✗ denied — SYN dropped
The same request, sent from two different identities. The client labeled role=frontend (identity 51014) gets HTTP 200; the client labeled role=other (6385) times out after 5 seconds — its packets are dropped before nginx ever sees them. The policy is working, but the request itself gives no hint as to why. For that, we watch it in Hubble.
hubble observe --namespace netflow-test --verdict \
DROPPED -o compact netflow-test/client-blocked:42222 (ID:6385) <> netflow-test/nginx-bb6b8c496-vc6pj:80 (ID:18565) Policy denied DROPPED (TCP Flags: SYN)
netflow-test/client-blocked:42232 (ID:6385) <> netflow-test/nginx-bb6b8c496-w8kkn:80 (ID:18565) Policy denied DROPPED (TCP Flags: SYN)
netflow-test/client-blocked:42242 (ID:6385) <> netflow-test/nginx-bb6b8c496-w8kkn:80 (ID:18565) Policy denied DROPPED (TCP Flags: SYN)
The drop, made visible. Hubble reports the blocked client (identity 6385) reaching nginx (18565) as 'Policy denied DROPPED' — and it happens at the very first SYN, so the TCP handshake never begins (that's why the curl hung for its whole timeout). Attributing a drop to a specific workload identity and policy decision, in real time, is exactly what the legacy dataplane could never show.
kubectl -n kube-system exec <anetd> -c cilium-agent -- \
cilium-dbg bpf policy get <nginx-endpoint> POLICY DIRECTION LABELS (source:key[=value]) PORT/PROTO PROXY PORT AUTH TYPE BYTES PACKETS PREFIX
Allow Ingress reserved:host ANY NONE disabled - - 0
reserved:remote-node
Allow Ingress k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=netflow-test ANY NONE disabled 1078 14 0
k8s:io.cilium.k8s.policy.cluster=default
k8s:io.cilium.k8s.policy.serviceaccount=default
k8s:io.kubernetes.pod.namespace=netflow-test
k8s:role=frontend
Allow Egress ANY ANY NONE disabled 30444 279 0
And here is the enforcement itself — not magic, just an eBPF allowlist keyed by identity, programmed onto the nginx endpoint. There is an Allow entry for the role=frontend label set (the byte/packet counters prove real traffic matched it) and no entry at all for role=other, so identity 6385 falls through to default-deny. The whole arc in one map: YAML intent → label-derived identity → this allowlist → the Hubble verdict.