kuryr-k8s may not work properly in a multi-cni cluster

Yes, we also have this requirement.
In a kubernetes ready environment, if we deploy kuryr later when needed, then some existing pods/LB would cause a lot
of error , kuryr controller get incorrect pod/LB info from those already existing pods/LB, then some times we can find kuryr-k8s crashed at last.

@longfei.zhang @kzaitsev

Do you have some idea about what would be acceptabe in this case to determine if we should serve a pod or a service?

I had a couple of ideas:

a) An annotation to inhibit kuryr
b) With multi-network support allow kuryr to be configured as 'on by default' and 'off by default'. In the latter case, it would only act on networks with a kuryr annotation.
c) Implemented in some driver. The driver would raise an exception like "NotKuryrManaged"

I like the 2 option more.

However at this point I'd say that it is safe to downgrade importance to med or low. Is there user-demand for this kind of setup/functionality?

We also face this issue, when kuryr is used along with multus, and a vif might not be requested from kuryr for some pods.

For CNI case, we can annotate/label pods which are mentioned in CNI ADD requests and start processing the pod in controller once such label appears. It does slow down the port request process by a bit, so we might have it as a configurable behaviour.

Fix proposed to branch: master
Review: https://review.opendev.org/696639