Hi Folks,
Hey Folks, I’ve been tasked with tracking down the recommended Resource Limits for Calico pods (EKS). I can’t seem to find any official/unofficial documentation that outlines this information. Please advise.
Kind Regards,
Chuck
I’m guessing that you’re the same user who asked this in Slack? I’m going to re-quote the answer @fasaxc gave there, for anyone who might find this in future:
Unfortunately, our resource usage depends completely on your use case. Churning pods and policy causes higher CPU usage. Having more pods and/or policy causes higher memory usage.
You need to measure the usage with your workload and feed that back into resource requests. Setting memory limits is the trickiest thing to do; if the memory limit is too small and calico gets OOM killed then it’ll restart and get OOM killed again. At that point you’ve created a problem by setting a limit.
Sadly, this isn’t mentioned in the Calico docs.
Is there a command for measuring the usage
I think the normal way to do it in a k8s cluster is to install the metrics server: GitHub - kubernetes-sigs/metrics-server: Scalable and efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines. , see what it says the calico components are using.
It looks like I’m gathering this for a larger right-sizing project but as Calico isn’t resource intensive and it is mission critical would you go so far to agree that the best thing to do is leave the req/limit config alone? (We’re using v3.20.0)
That’s certainly the Calico team’s default position, yes.
There is a possible downside, of course. If you’re loading your nodes to the very limit to maximise efficiency, and Calico components don’t have a reservation, kubernetes will not keep any capacity available for Calico, possibly leading to CPU/memory starvation. So a reservation might be a sensible idea.