Hello everyone!
I’m trying to get a manual installation of Calico (without Kubernetes) to get to work using various calico docs; but unfortunately, inter-container communication is not working; nothing works – neither ICMP, nor TCP from within another pod or from another node or the same node as the workload itself. For this I’ve started 2 calico/node daemons as –
docker run -d --rm --privileged --net=host --name=calico -e CLUSTER_TYPE=bgp -e FELIX_DEFAULTENDPOINTTOHOSTACTION=Accept -e CALICO_IPV4POOL_CIDR=10.0.0.0/24 -e AS=4000 -e IP6=none -e NODENAME=node1 -e IP=192.168.3.2 -e CALICO_NETWORKING_BACKEND=bird -e DATASTORE_TYPE=etcdv3 -e ETCD_ENDPOINTS=http://192.168.3.2:2379 -v /var/lib/calico:/var/lib/calico -v /run/docker/plugins:/run/docker/plugins -v /lib/modules:/lib/modules -v /var/run/calico:/var/run/calico -v /etc/pki:/pki -v /var/log/calico:/var/log/calico calico/node:v3.16.5
docker run -d --rm --privileged --net=host --name=calico -e CLUSTER_TYPE=bgp -e FELIX_DEFAULTENDPOINTTOHOSTACTION=Accept -e CALICO_IPV4POOL_CIDR=10.0.1.0/24 -e AS=4200 -e NODENAME=node2 -e IP6=none -e IP=192.168.3.3 -e CALICO_NETWORKING_BACKEND=bird -e DATASTORE_TYPE=etcdv3 -e ETCD_ENDPOINTS=http://192.168.3.2:2379 -v /var/lib/calico:/var/lib/calico -v /run/docker/plugins:/run/docker/plugins -v /lib/modules:/lib/modules -v /var/run/calico:/var/run/calico -v /etc/pki:/pki -v /var/log/calico:/var/log/calico calico/node:v3.16.5
Following are the manifests injected –
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: smallnet
labels:
purpose: internal
spec:
cidr: 10.0.0.0/24
nodeSelector: "name == 'node1'"
---
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: smallnet2
labels:
purpose: node2
spec:
cidr: 10.0.1.0/24
nodeSelector: "name == 'node2'"
---
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: smallnet3
labels:
purpose: node3
spec:
cidr: 10.0.2.0/24
nodeSelector: "name == 'node3'"
---
kind: Node
metadata:
name: node2
labels:
purpose: prod
name: node2
spec:
bgp:
ipv4Address: 192.168.3.3/24
asNumber: 4100
---
apiVersion: projectcalico.org/v3
kind: Node
metadata:
name: node3
labels:
purpose: prod
name: node3
spec:
bgp:
ipv4Address: 192.168.3.4/24
asNumber: 4200
---
apiVersion: projectcalico.org/v3
kind: Node
metadata:
name: node1
labels:
purpose: prod
name: node1
spec:
bgp:
ipv4Address: 192.168.3.2/24
asNumber: 4000
CNI config of the plugin at node 1 –
{
"name": "node1cn2",
"cniVersion": "0.1.0",
"type": "calico",
"nodename": "node1",
"log_level": "DEBUG",
"ipam": {
"type": "calico-ipam",
"assign_ipv4": "true",
"assign_ipv6": "false",
"ipv4_pools": ["smallnet"]
},
"etcd_endpoints": "http://192.168.3.2:2379",
"datastore_type": "etcdv3",
"container_settings": {
"allow_ip_forwarding": true
}
}
CNI config of the plugin at node 2 –
{
"name": "node2cn2",
"cniVersion": "0.1.0",
"type": "calico",
"nodename": "node2",
"log_level": "DEBUG",
"ipam": {
"type": "calico-ipam",
"assign_ipv4": "true",
"assign_ipv6": "false",
"ipv4_pools": ["smallnet2"]
},
"etcd_endpoints": "http://192.168.3.2:2379",
"datastore_type": "etcdv3",
"container_settings": {
"allow_ip_forwarding": true
}
}
Node3 has been turned off for now.
This’s a single node etcd cluster which is on node1.
Here are the logs for calico/node /var/log/calico before the CNI plugin was executed and after some time the container was started (for both the nodes) –