You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
What happened
:
I have enabled ipvs mode on Kube 1.8 and am having issues accessing NodePort Services. I notice if I restart kube-proxy I see the following messages every minute for ~20 minutes before it appears to actually sync the rules
Oct 11 10:33:57 worker-23.test.ibm.com kube-proxy.service[2058581]: I1011 10:33:57.095261 2058581 proxier.go:836] Not syncing ipvs rules until Services and Endpoints have been received from master
Oct 11 10:33:57 worker-23.test.ibm.com kube-proxy.service[2058581]: I1011 10:33:57.095307 2058581 proxier.go:832] syncProxyRules(EndpointsUpdate) took 63.9µs
Oct 11 10:33:57 worker-23.test.ibm.com kube-proxy.service[2058581]: I1011 10:33:57.095342 2058581 config.go:124] Calling handler.OnEndpointsAdd
We have ~35 services, so not a huge amount.
What you expected to happen:
I expect kube-proxy to update the rules immediately, or at least within a few minutes on startup.
How to reproduce it (as minimally and precisely as possible):
Start kube-proxy with the following parameters:
Anything else we need to know?:
I am seeing various other issues with IPVS with it appearing to not sync correctly - i.e if I add/delete a service or delete a pod I regularly see the changes not being reflected in the IPVS rules - it appears kube-proxy is not getting notified about changes.
I compared the iptables proxy code with ipvs code at branch 1.8 and can see a difference in the NewProxier function.
iptables has the following:
Whereas this is missing in ipvs at the 1.8 branch. I notice this code is present in master for ipvs. Could this be causing the sync issues I'm seeing? If so is this change likely to make it into a 1.8.x release in the near future as we would like to progress testing of ipvs.
Environment:
Kubernetes version (use kubectl version): Kubernetes v1.8.0-1+edd813e3ac8824
Cloud provider or hardware configuration**: VSIs running on IBM Softlayer
OS (e.g. from /etc/os-release):Ubuntu 16.04.2 LTS (Xenial)
Kernel (e.g. uname -a):4.4.0-92-generic Update the guestbook sample #115-Ubuntu SMP Thu Aug 10 09:04:33 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Install tools:
Others:
I rebuilt kube-proxy with this commit -> 44afb09 (which includes the change I mentioned above) and it now seems to behave much better, and is syncing on startup and when changes occur.
If someone could confirm this will make it into the net update for V1.8 I'm happy to close this issue
I rebuilt kube-proxy with this commit -> 44afb09 (which includes the change I mentioned above) and it now seems to behave much better, and is syncing on startup and when changes occur.
That's true.
If someone could confirm this will make it into the net update for V1.8 I'm happy to close this issue
I would like to cherry-pick the bug fix into 1.8 but don't have much experience. @cmluciano any consult?