How to select HostEndpoints in Network Policy?

Open Source Calico Help Networking
#1

Hi there,

I currently stuck in writing NetworkPolicies that should restrict the communication regarding the metrics server which needs to get accessed by the kube-apiserver on port 4443.

The cluster has the following nodes, which all have one interface with one IPv4 address assigned (ipv6 disabled in the kernel):

  • 3 master nodes (tagged with node-role.kubernetes.io/master and kubernetes-host)
  • 3 worker nodes (tagged withnode-role.kubernetes.io/worker and kubernetes-host)

I’ve set up a few GlobalNetworkPolicies based on auto created HostEndpoints to roughly control what happens between the hosts.

For the kube-system namespace I try to control ingress traffic to the pods.
As soon as I try to restrict ingress for the metrics-server using source it starts failing.
In the iptables log I see that the apiserver tries to connect from the corresponding hosts IP Address to port 4443 and gets dropped by my gnp deny-app-policy.

Is it somehow possible to match the HostEndpoints there?
Writing a GlobalNetworkPolicy seems wrong to me, since the traffic belongs to the same Namespace.
What do I miss? :slight_smile:

regards,
Dennis

The relevant network policies for the namespace kube-system look like that:

---
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-kube-system-egress
  namespace: kube-system
spec:
  order: 10
  egress:
  - action: Allow
    metadata:
      annotations:
        from: kube-system
        to: any
---
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-metrics-server-tcp-4443
  namespace: kube-system
spec:
  order: 10
  selector: k8s-app == "metrics-server"
  ingress:
  - action: Allow
    metadata:
      annotations:
        from: k8s-ctrl-hosts
        to: metrics-server
    # source:
    #   selector: has(kubernetes-host) && has(node-role.kubernetes.io/master)
    protocol: TCP
    destination:
      ports:
        - 4443

And my GlobalNetworkPolicies like that:

---
# global network policy that applies to traffic related to all nodes in the
# cluster
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: all-k8s-hosts-policy
spec:
  order: 10
  selector: has(kubernetes-host)
  # This rule allows all traffic to localhost.
  ingress:
  - action: Allow
    metadata:
      annotations:
        from: kubernetes-host
        to: localhost
    destination:
      nets:
      - 127.0.0.1/32
  # This rule allows the masters to access the kubelet API on other masters
  # (including itself).
  - action: Allow
    protocol: TCP
    metadata:
      annotations:
        from: ctrl-plane
        to: kubelet
    source:
      selector: has(node-role.kubernetes.io/master)
    destination:
      ports:
      - 10250
---
# global network policy that applies to traffic related to all master nodes in the cluster
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: all-ctrl-nodes-policy
spec:
  order: 10
  selector: has(kubernetes-host) && has(node-role.kubernetes.io/master)
  ingress:
  # This rule allows ingress to the Kubernetes API server.
  - action: Allow
    protocol: TCP
    metadata:
      annotations:
        from: any
        to: kube-apiserver
    destination:
      ports:
      - 6443
---
# deny app traffic in general
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: deny-app-policy
spec:
  order: 100
  namespaceSelector: has(projectcalico.org/name)
  types:
  - Ingress
  - Egress
  ingress:
  - action: Log
    metadata:
      annotations:
        deny: any
        log: any
  egress:
  # allow all namespaces outgoing dns requests
  - action: Allow
    protocol: UDP
    destination:
      selector: 'k8s-app == "kube-dns"'
      ports:
      - 53
  - action: Log
    metadata:
      annotations:
        deny: any
        log: any