This allows the use of a built-in provider to do things like mark a node
as ready once all the controllers are spun up.
The e2e tests now use this instead of waiting on the pod that the vk
provider is deployed in to be marked ready (this was waiting on
/stats/summary to be serving, which is racey).
This removes the legacy sync provider interface. All new providers
are expected to implement the async NotifyPods interface.
The legacy sync provider interface creates complexities around
how the deletion flow works, and the mixed sync and async APIs
block us from evolving functionality.
This collapses in the NotifyPods interface into the PodLifecycleHandler
interface.
Allows callers to wait for pod controller exit in addition to readiness.
This means the caller does not have to deal handling errors from the pod
controller running in a gorutine since it can wait for exit via `Done()`
and check the error with `Err()`
* Upgrade k8s.io/api
go get k8s.io/api@kubernetes-1.15.2
* Upgrade k8s.io/apimachinery
go get k8s.io/apimachinery@kubernetes-1.15.2
* Upgrade kubernetes-1.15.2
go get k8s.io/client-go@kubernetes-1.15.2
* Upgrade kk8s.io/kubernetes to v1.15.2
go get k8s.io/kubernetes@v1.15.2
This also locks the the dependency for
github.com/prometheus/client_golang/prometheus due to a golang bug, and to
please the validation scripts.
The replaces were generated by:
go get k8s.io/kubernetes@v1.15.2 2> fail
for i in $(cat fail|grep unknown|cut -f1 -d@|cut -f2 -d" ")
do echo "replace ${i} => ${i} kubernetes-1.15.2"
done
This seems to avoid a race conditions where at pod informer
startup time, the reactor doesn't properly get setup.
It also refactors the root command example to start up
the informers after everything is wired up.
This eliminates all the one-off calls to for specific things like
`Capacity` and other things.
Instead we configure defaults in the CLI and then pass the full node
object to the provider to change as needed.
This makes sure the provider can change whatever they need to without
having to add tons of methods to the provider interface.
This allows `--cluster-domain` to be passed to virtual kubelet like a
traditional kublet, and use this to generate search-domains for
`/etc/resolv.conf`
* Set default `cluster-domain` to `cluster-local` to match current kubelet
Related: #641
Signed-off-by: Graham Hayes <gr@ham.ie>
* Move all but mock provider out of tree
These have all been moved to repos under github.com/virtual-kubelet.
* Introduce a providers.Store
This essentially moves the the old register/ handling into a first class
object that can be controlled from the CLI rather than through build
tags deep in the code.
This actually would have made it a bit easier to build the provider
repos and makes the cmd/ code more re-usable.
We still use it internally, but this does not need to be part of the
public API. Instead just have callers pass us the relevent listers and
we create our own resource manager.
* Move tracing exporter registration
This doesn't belong in the library and should be configured by the
consumer of the opencensus package.
* Rename `vkublet` package to `node`
`vkubelet` does not convey any information to the consumers of the
package.
Really it would be nice to move this package to the root of the repo,
but then you wind up with... interesting... import semantics due to the
repo name... and after thinking about it some, a subpackage is really
not so bad as long as it has a name that convey's some information.
`node` was chosen since this package deals with all the semantics of
operating a node in Kubernetes.
Do not enable leases by default in the node constructor.
Simplify node constructor to not require a lease client when leases may
not even be enabled.
Updates node status update interval to the kubelet default of 10s (was
5s in vk).
This moves the logic for re-creating the a missing node up into the CLI.
We can make this optional, but for now I've just preserved existing
functionality.
This makes the concept of a `Provider` wholely implemented in the cli
implementation in cmd/virtual-kubelet.
It allows us to slim down the interfaces used in vkubelet (and
vkubelet/api) to what is actually used there rather than a huge
interface that is only there to serve the CLI's needs.
* Add the /runningpods/ api endpoint
This adds an API endpoint from the kubelet (/runningpods/). It is
an endpoint on kubelet which is considered a "debug" endpoint, so
it might be worth exposing through the options, but by default
it is exposed in most k8s configs AFAICT.
All of Kubernetes logging is based on klog. Klog currently does
not output any logging information to logrus, so you're flying
somewhat blind to Kubernetes internals.
This exposes the full set of configurables that klog offers,
but decorates (prefixes) the klog configuration with "klog".
This adds two concepts, where one encompasses the other.
Startup timeout
Startup timeout is how long to wait for the entire kubelet
to get into a functional state. Right now, this only waits
for the pod informer cache for the pod controllerto become
in-sync with API server, but this could be extended to other
informers (like secrets informer).
Wait For Startup
This changes the behaviour of the virtual kubelet to wait
for the pod controller to start before registering the node.
It is to avoid the race condition where the node is registered,
but we cannot actually do any pod operations.
Clientset.Coordination is deprecated. We are meant to use the specific
version of the client: CoordinationV1beta1. Clientset.Coordination is
going to be removed in future versions of the client API.
When setting up the http server we return a cancel function to close all
the listeners down.
The issue here is we set the cancel function to nil and thereby cause a
panic when there is an error and the `defer` attempts to call cancel.
This fix just don't set a named return value for the cancel function to
make sure we don't overwrite it with a `return nil, err`.
This ensures that the `defer` can still call `cancel()`.
This cleans up the CLI code significantly.
Also makes some of this re-usable for providers who want to do so.
This also removes the main.go from the top of the tree of the repro,
instead moving it into cmd/virtual-kubelet.
This allows us to better utilize the package namespace (and e.g. mv the
`vkubelet` package to the top of the tree).
* Remove unused lock from the resource manager.
* Add service lister to the resource manager.
This change adds a service lister in the
resource manager.
This will be used to set the service env vars.
Also added a List method to the resource manager
and a simple test to confirm it's a pass through.
This starts the work of having a `NodeProvider` which is responsible for
providing node details.
It splits the responsibilities of node management off to a new
controller.
The primary change here is to add the framework pieces for node
management and move the VK CLI to use this new controller.
It also adds support for node leases where available. This can be
enabled via the command line (disabled by default), but may fall back if
we find that leaess aren't supported on the cluster.
* Add providers subcommand to verify providers
Allows users to check what providers are available
* Fix version output to add new line
This command was totally broken until we moved around the call to
`initConfig()`, this just fixes the output now that it works.
* Flip boolean of provider include tags
All providers are still included by default and fix tags using the old
format.
* Define and use an interface for logging.
This allows alternative implementations to use whatever logging package
they want.
Currently the interface just mimicks what logrus already implements,
with minor modifications to not rely on logrus itself. I think the
interface is pretty solid in terms of logging implementations being able
to do what they need to.
* Make tracing interface to coalesce logging/tracing
Allows us to share data between the tracer and the logger so we can
simplify log/trace handling wher we generally want data to go both
places.
* Don't start things in New
* Move http server handling up to daemon.
This removes the burdern of dealing with listeners, http servers, etc in
the core framework.
Instead provide helpers to attach the appropriate routes to the
caller's serve mux.
With this change, the vkubelet package only helps callers setup HTTP
rather than forcing a specific HTTP config on them.
* vendor: add vendored code
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* controller: use shared informers and a work queue
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* errors: use cpuguy83/strongerrors
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* aci: fix test that uses resource manager
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* readme: clarify skaffold run before e2e
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* cmd: use root context everywhere
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: refactor pod lifecycle management
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* e2e: fix race in test when observing deletions
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* e2e: test pod forced deletion
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* cmd: fix root context potential leak
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: rename metaKey
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: remove calls to HandleError
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* Revert "errors: use cpuguy83/strongerrors"
This reverts commit f031fc6d.
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* manager: remove redundant lister constraint
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: rename the pod event recorder
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: amend misleading comment
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* mock: add tracing
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: add tracing
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* test: observe timeouts
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* trace: remove unnecessary comments
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: limit concurrency in deleteDanglingPods
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: never store context, always pass in calls
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: remove HandleCrash and just panic
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: don't sync succeeded pods
Signed-off-by: Paulo Pires <pjpires@gmail.com>
* sync: ensure pod deletion from kubernetes
Signed-off-by: Paulo Pires <pjpires@gmail.com>
