This refactor is a preparation for another commit. I want to add instrumentation
around our queues. The code of how queues were handled was spread throughout
the code base, and that made adding such instrumentation nice and complicated.
This centralizes the queue management logic in queue.go, and only requires
the user to provide a (custom) rate limiter, if they want to, a name,
and a handler.
The lease code is moved into its own package to simplify testing, because
the goroutine leak tester was triggering incorrectly if other tests
were running, and it was measuring leaks from those tests.
This also identified buggy behaviour:
wq := workqueue.NewNamedRateLimitingQueue(workqueue.DefaultItemBasedRateLimiter(), "test")
wq.AddRateLimited("hi")
fmt.Printf("Added hi, len: %d\n", wq.Len())
wq.Forget("hi")
fmt.Printf("Forgot hi, len: %d\n", wq.Len())
wq.Done("hi")
fmt.Printf("Done hi, len: %d\n", wq.Len())
---
Prints all 0s because event non-delayed items are delayed. If you call Add
directly, then the last line prints a len of 2.
// Workqueue docs:
// Forget indicates that an item is finished being retried. Doesn't matter whether it's for perm failing
// or for success, we'll stop the rate limiter from tracking it. This only clears the `rateLimiter`, you
// still have to call `Done` on the queue.
^----- Even this seems untrue
This refactors the v1 lease controller. It makes two functional differences
to the lease controller:
* It no longer ties lease updates to node pings or node status updates
* There is no fallback mechanism to status updates
This also moves vk_envtest, allowing for future brown-box testing of the
lease controller with envtest
As described in the issue, if the following sequence happens, we fail to properly
update the pod status in api server:
1. Create pod in k8s
2. Provider creates the pod and syncs its status back
3. Pod in k8s ready/running, all fine
4. Virtual kubelet fails to update node status for some time for whatever reason (e.g. network connectivity issues)
5. Virtual node marked as NotReady with message: Kubelet stopped posting node status
6. kube-controller-manager of k8s, goes and marks all pods as Ready = false:
7. Virtual kubelet never sync's status of pod in provider back to k8s
This uses gobin in the Makefile for golint. Gobin allows for easier
pinning of dependencies that are project specific, as in the actual
gobin command invocation you can specify the version.
We were having issues with golint not properly reporting declaration of functions
without proper documentation (comments). This is due to a config with golangci.
See: https://github.com/golangci/golangci-lint/issues/456
This moves node ping controller to using the new internal lock
API.
The reason for this is twofold:
* The channel approach that was used to notify other
controllers of changes could only be used once (at startup),
and couldn't be used in the future to broadcast node
ping status. The idea idea is here that we could move
to a sync.Cond style API and only wakeup other controllers
on change, as opposed to constantly polling each other
* The problem with sync.Cond is that it's not context friendly.
If we want to do stuff like wait on a sync.cond and use a context
or a timer or similar, it doesn't work whereas this API allows
context cancellations on condition change.
The idea is that as we have more controllers that act as centralized
sources of authority, they can broadcast out their state.
For example:
Provier is a K8s provider, pod created by deployment would be evicted when node is not ready.
If we do not delete pod in K8s, deployment would not create a new one.
Add some tests for updateStatus
This configures the global logger the same as the local logger,
and adds the test name. It also uses the logger with the test
context as the context logger.
This sets KUBEBUILDERA_ASSETS only for envtest. No need to leak
the variable throughout the tests. In addition, it makes it easier
to figure out how to run the tests yourself on command line.
This copies and pastes the loop that used to exist in
func populateEnvironmentVariables(..) {
...
for _, env := range container.Env {
... <--- This code
}
}
Into getEnvironmentVariableValue. getEnvironmentVariableValue
returns val, err, where val is a pointer to a string
to indicate optionality.
I know it's not an impressive test. It just brings up a node, and
makes sure it registers. Let's do more in the future.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
