d8dfun/virtual-kubelet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Importable End-To-End Test Suite (#758)

Browse Source 
* Rename VK to chewong for development purpose

* Rename basic_test.go to basic.go

* Add e2e.go and suite.go

* Disable tests in node.go

* End to end tests are now importable as a testing suite

* Remove 'test' from test files

* Add documentations

* Rename chewong back to virtual-kubelet

* Change 'Testing Suite' to 'Test Suite'

* Add the ability to skip certain testss

* Add unit tests for suite.go

* Add README.md for importable e2e test suite

* VK implementation has to be based on VK v1.0.0

* Stricter checks on validating test functions

* Move certain files back to internal folder

* Add WatchTimeout as a config field

* Add slight modifications
This commit is contained in:
Ernest Wong
2019-09-04 14:25:43 -07:00
committed by Pires
parent da57373abb
commit f10a16aed7
14 changed files with 585 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files

191
test/e2e/README.md Normal file
View File

@@ -0,0 +1,191 @@
# Importable End-To-End Test Suite
Virtual Kubelet (VK) provides an importable end-to-end (E2E) test suite containing a set of common integration tests. As a provider, you can import the test suite and use it to validate your VK implementation.
## Prerequisite
To run the E2E test suite, three things are required:
- A local Kubernetes cluster (we have tested with [Docker for Mac](https://docs.docker.com/docker-for-mac/install/) and [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/));
- Your _kubeconfig_ default context points to the local Kubernetes cluster;
- [skaffold](https://skaffold.dev/docs/getting-started/#installing-skaffold)
> The test suite is based on [VK 1.0](https://github.com/virtual-kubelet/virtual-kubelet/releases/tag/v1.0.0). If your VK implementation is based on legacy VK library (< v1.0.0), you will have to upgrade it to VK 1.0 using [virtual-kubelet/node-cli](https://github.com/virtual-kubelet/node-cli).
### Skaffold Folder
Before running the E2E test suite, you will need to copy the [`./hack`](../../hack) folder containing Skaffold-related files such as Dockerfile, manifests, and certificates to your VK project root. Skaffold essentially helps package your virtual kubelet into a container based on the given [`Dockerfile`](../../hack/skaffold/virtual-kubelet/Dockerfile) and deploy it as a pod (see [`pod.yml`](../../hack/skaffold/virtual-kubelet/pod.yml)) to your Kubernetes test cluster. In summary, you will likely need to modify the VK name in those files, customize the VK configuration file, and the API server certificates (`<vk-name>-crt.pem` and `<vk-name>-key.pem`) before running the test suite.
### Makefile.e2e
Also, you will need to copy [`Makefile.e2e`](../../Makefile.e2e) to your VK project root. It contains necessary `make` commands to run the E2E test suite. Do not forget to add `include Makefile.e2e` in your `Makefile`.
### File Structure
A minimal VK provider should now have a file structure similar to the one below:
```console
.
├── Makefile
├── Makefile.e2e
├── README.md
├── cmd
│   └── virtual-kubelet
│   └── main.go
├── go.mod
├── go.sum
├── hack
│   └── skaffold
│   └── virtual-kubelet
│   ├── Dockerfile
│   ├── base.yml
│   ├── pod.yml
│   ├── skaffold.yml
│   ├── vkubelet-provider-0-cfg.json
│   ├── vkubelet-provider-0-crt.pem
│   └── vkubelet-provider-0-key.pem
├── test
│   └── e2e
│ └── main_test.go # import and run the E2E test suite here
├── provider.go # provider-specific VK implementation
├── provider_test.go # unit test
```
## Importing the Test Suite
The test suite can be easily imported in your test files (e.g. `./test/e2e/main_test.go`) with the following import statement:
```go
import (
vke2e "github.com/virtual-kubelet/virtual-kubelet/test/e2e"
)
```
### Test Suite Customization
The test suite allows providers to customize the test suite using `EndToEndTestSuiteConfig`:
```go
// EndToEndTestSuiteConfig is the config passed to initialize the testing framework and test suite.
type EndToEndTestSuiteConfig struct {
// Kubeconfig is the path to the kubeconfig file to use when running the test suite outside a Kubernetes cluster.
Kubeconfig string
// Namespace is the name of the Kubernetes namespace to use for running the test suite (i.e. where to create pods).
Namespace string
// NodeName is the name of the virtual-kubelet node to test.
NodeName string
// WatchTimeout is the duration for which the framework watch a particular condition to be satisfied (e.g. watches a pod becoming ready)
WatchTimeout time.Duration
// Setup is a function that sets up provider-specific resource in the test suite
Setup suite.SetUpFunc
// Teardown is a function that tears down provider-specific resources from the test suite
Teardown suite.TeardownFunc
// ShouldSkipTest is a function that determines whether the test suite should skip certain tests
ShouldSkipTest suite.ShouldSkipTestFunc
}
```
> `Setup()` is invoked before running the E2E test suite, and `Teardown()` is invoked after all the E2E tests are finished.
You will need an `EndToEndTestSuiteConfig` to create an `EndToEndTestSuite` using `NewEndToEndTestSuite`. After that, invoke `Run` from `EndToEndTestSuite` to start the test suite. The code snippet below is a minimal example of how to import and run the test suite in your test file.
```go
package e2e
import (
"time"
vke2e "github.com/virtual-kubelet/virtual-kubelet/test/e2e"
)
var (
kubeconfig string
namespace string
nodeName string
)
// Read the following variables from command-line flags
func init() {
flag.StringVar(&kubeconfig, "kubeconfig", "", "path to the kubeconfig file to use when running the test suite outside a kubernetes cluster")
flag.StringVar(&namespace, "namespace", defaultNamespace, "the name of the kubernetes namespace to use for running the test suite (i.e. where to create pods)")
flag.StringVar(&nodeName, "node-name", defaultNodeName, "the name of the virtual-kubelet node to test")
flag.Parse()
}
func setup() error {
fmt.Println("Setting up end-to-end test suite...")
return nil
}
func teardown() error {
fmt.Println("Tearing down end-to-end test suite...")
return nil
}
func shouldSkipTest(testName string) bool {
// Skip the test 'TestGetStatsSummary'
return testName == "TestGetStatsSummary"
}
func TestEndToEnd(t *testing.T) {
config := vke2e.EndToEndTestSuiteConfig{
Kubeconfig: kubeconfig,
Namespace: namespace,
NodeName: nodeName,
Setup: setup,
Teardown: teardown,
ShouldSkipTest: shouldSkipTest,
WaitTimeout: 5 * time.Minute,
}
ts := vke2e.NewEndToEndTestSuite(config)
ts.Run(t)
}
```
## Running the Test Suite
Since our CI uses Minikube, we describe below how to run E2E on top of it.
To create a Minikube cluster, run the following command after [installing Minikube](https://github.com/kubernetes/minikube#installation):
```bash
minikube start
```
To run the E2E test suite, you can run the following command:
```bash
make e2e
```
You can see from the console output whether the tests in the test suite pass or not.
```console
...
=== RUN TestEndToEnd
=== RUN TestEndToEnd/TestCreatePodWithMandatoryInexistentConfigMap
=== RUN TestEndToEnd/TestCreatePodWithMandatoryInexistentSecrets
=== RUN TestEndToEnd/TestCreatePodWithOptionalInexistentConfigMap
=== RUN TestEndToEnd/TestCreatePodWithOptionalInexistentSecrets
=== RUN TestEndToEnd/TestGetStatsSummary
=== RUN TestEndToEnd/TestNodeCreateAfterDelete
=== RUN TestEndToEnd/TestPodLifecycleForceDelete
=== RUN TestEndToEnd/TestPodLifecycleGracefulDelete
--- PASS: TestEndToEnd (21.93s)
--- PASS: TestEndToEnd/TestCreatePodWithMandatoryInexistentConfigMap (0.03s)
--- PASS: TestEndToEnd/TestCreatePodWithMandatoryInexistentSecrets (0.03s)
--- PASS: TestEndToEnd/TestCreatePodWithOptionalInexistentConfigMap (0.55s)
--- PASS: TestEndToEnd/TestCreatePodWithOptionalInexistentSecrets (0.99s)
--- PASS: TestEndToEnd/TestGetStatsSummary (0.80s)
--- PASS: TestEndToEnd/TestNodeCreateAfterDelete (9.63s)
--- PASS: TestEndToEnd/TestPodLifecycleForceDelete (2.05s)
basic.go:158: Created pod: nginx-testpodlifecycleforcedelete-jz84g
basic.go:164: Pod nginx-testpodlifecycleforcedelete-jz84g ready
basic.go:197: Force deleted pod: nginx-testpodlifecycleforcedelete-jz84g
basic.go:214: Pod ended as phase: Running
--- PASS: TestEndToEnd/TestPodLifecycleGracefulDelete (1.04s)
basic.go:87: Created pod: nginx-testpodlifecyclegracefuldelete-r84v7
basic.go:93: Pod nginx-testpodlifecyclegracefuldelete-r84v7 ready
basic.go:120: Deleted pod: nginx-testpodlifecyclegracefuldelete-r84v7
PASS
...
```

362
test/e2e/basic.go Normal file
View File

@@ -0,0 +1,362 @@
package e2e
import (
"fmt"
"testing"
"time"
"github.com/virtual-kubelet/virtual-kubelet/node"
"gotest.tools/assert"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/kubernetes/pkg/kubelet/apis/stats/v1alpha1"
)
const (
// deleteGracePeriodForProvider is the maximum amount of time we allow for the provider to react to deletion of a pod
// before proceeding to assert that the pod has been deleted.
deleteGracePeriodForProvider = 1 * time.Second
)
// TestGetStatsSummary creates a pod having two containers and queries the /stats/summary endpoint of the virtual-kubelet.
// It expects this endpoint to return stats for the current node, as well as for the aforementioned pod and each of its two containers.
func (ts *EndToEndTestSuite) TestGetStatsSummary(t *testing.T) {
// Create a pod with prefix "nginx-" having three containers.
pod, err := f.CreatePod(f.CreateDummyPodObjectWithPrefix(t.Name(), "nginx", "foo", "bar", "baz"))
if err != nil {
t.Fatal(err)
}
// Delete the "nginx-0-X" pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
// Wait for the "nginx-" pod to be reported as running and ready.
if _, err := f.WaitUntilPodReady(pod.Namespace, pod.Name); err != nil {
t.Fatal(err)
}
// Grab the stats from the provider.
stats, err := f.GetStatsSummary()
if err != nil {
t.Fatal(err)
}
// Make sure that we've got stats for the current node.
if stats.Node.NodeName != f.NodeName {
t.Fatalf("expected stats for node %s, got stats for node %s", f.NodeName, stats.Node.NodeName)
}
// Make sure the "nginx-" pod exists in the slice of PodStats.
idx, err := findPodInPodStats(stats, pod)
if err != nil {
t.Fatal(err)
}
// Make sure that we've got stats for all the containers in the "nginx-" pod.
desiredContainerStatsCount := len(pod.Spec.Containers)
currentContainerStatsCount := len(stats.Pods[idx].Containers)
if currentContainerStatsCount != desiredContainerStatsCount {
t.Fatalf("expected stats for %d containers, got stats for %d containers", desiredContainerStatsCount, currentContainerStatsCount)
}
}
// TestPodLifecycleGracefulDelete creates a pod and verifies that the provider has been asked to create it.
// Then, it deletes the pods and verifies that the provider has been asked to delete it.
// These verifications are made using the /stats/summary endpoint of the virtual-kubelet, by checking for the presence or absence of the pods.
// Hence, the provider being tested must implement the PodMetricsProvider interface.
func (ts *EndToEndTestSuite) TestPodLifecycleGracefulDelete(t *testing.T) {
// Create a pod with prefix "nginx-" having a single container.
podSpec := f.CreateDummyPodObjectWithPrefix(t.Name(), "nginx", "foo")
podSpec.Spec.NodeName = f.NodeName
pod, err := f.CreatePod(podSpec)
if err != nil {
t.Fatal(err)
}
// Delete the pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
t.Logf("Created pod: %s", pod.Name)
// Wait for the "nginx-" pod to be reported as running and ready.
if _, err := f.WaitUntilPodReady(pod.Namespace, pod.Name); err != nil {
t.Fatal(err)
}
t.Logf("Pod %s ready", pod.Name)
// Grab the pods from the provider.
pods, err := f.GetRunningPods()
assert.NilError(t, err)
// Check if the pod exists in the slice of PodStats.
assert.NilError(t, findPodInPods(pods, pod))
podCh := make(chan error)
var podLast *v1.Pod
go func() {
// Close the podCh channel, signaling we've observed deletion of the pod.
defer close(podCh)
var err error
podLast, err = f.WaitUntilPodDeleted(pod.Namespace, pod.Name)
if err != nil {
// Propagate the error to the outside so we can fail the test.
podCh <- err
}
}()
// Gracefully delete the "nginx-" pod.
if err := f.DeletePod(pod.Namespace, pod.Name); err != nil {
t.Fatal(err)
}
t.Logf("Deleted pod: %s", pod.Name)
// Wait for the delete event to be ACKed.
if err := <-podCh; err != nil {
t.Fatal(err)
}
time.Sleep(deleteGracePeriodForProvider)
// Give the provider some time to react to the MODIFIED/DELETED events before proceeding.
// Grab the pods from the provider.
pods, err = f.GetRunningPods()
assert.NilError(t, err)
// Make sure the pod DOES NOT exist in the provider's set of running pods
assert.Assert(t, findPodInPods(pods, pod) != nil)
// Make sure we saw the delete event, and the delete event was graceful
assert.Assert(t, podLast != nil)
assert.Assert(t, podLast.ObjectMeta.GetDeletionGracePeriodSeconds() != nil)
assert.Assert(t, *podLast.ObjectMeta.GetDeletionGracePeriodSeconds() > 0)
}
// TestPodLifecycleForceDelete creates one podsand verifies that the provider has created them
// and put them in the running lifecycle. It then does a force delete on the pod, and verifies the provider
// has deleted it.
func (ts *EndToEndTestSuite) TestPodLifecycleForceDelete(t *testing.T) {
podSpec := f.CreateDummyPodObjectWithPrefix(t.Name(), "nginx", "foo")
// Create a pod with prefix having a single container.
pod, err := f.CreatePod(podSpec)
if err != nil {
t.Fatal(err)
}
// Delete the pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
t.Logf("Created pod: %s", pod.Name)
// Wait for the "nginx-" pod to be reported as running and ready.
if _, err := f.WaitUntilPodReady(pod.Namespace, pod.Name); err != nil {
t.Fatal(err)
}
t.Logf("Pod %s ready", pod.Name)
// Grab the pods from the provider.
pods, err := f.GetRunningPods()
assert.NilError(t, err)
// Check if the pod exists in the slice of Pods.
assert.NilError(t, findPodInPods(pods, pod))
// Wait for the pod to be deleted in a separate goroutine.
// This ensures that we don't possibly miss the MODIFIED/DELETED events due to establishing the watch too late in the process.
// It also makes sure that in light of soft deletes, we properly handle non-graceful pod deletion
podCh := make(chan error)
var podLast *v1.Pod
go func() {
// Close the podCh channel, signaling we've observed deletion of the pod.
defer close(podCh)
var err error
// Wait for the pod to be reported as having been deleted.
podLast, err = f.WaitUntilPodDeleted(pod.Namespace, pod.Name)
if err != nil {
// Propagate the error to the outside so we can fail the test.
podCh <- err
}
}()
time.Sleep(deleteGracePeriodForProvider)
// Forcibly delete the pod.
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil {
t.Logf("Last saw pod in state: %+v", podLast)
t.Fatal(err)
}
t.Log("Force deleted pod: ", pod.Name)
// Wait for the delete event to be ACKed.
if err := <-podCh; err != nil {
t.Logf("Last saw pod in state: %+v", podLast)
t.Fatal(err)
}
// Give the provider some time to react to the MODIFIED/DELETED events before proceeding.
time.Sleep(deleteGracePeriodForProvider)
// Grab the pods from the provider.
pods, err = f.GetRunningPods()
assert.NilError(t, err)
// Make sure the "nginx-" pod DOES NOT exist in the slice of Pods anymore.
assert.Assert(t, findPodInPods(pods, pod) != nil)
t.Logf("Pod ended as phase: %+v", podLast.Status.Phase)
}
// TestCreatePodWithOptionalInexistentSecrets tries to create a pod referencing optional, inexistent secrets.
// It then verifies that the pod is created successfully.
func (ts *EndToEndTestSuite) TestCreatePodWithOptionalInexistentSecrets(t *testing.T) {
// Create a pod with a single container referencing optional, inexistent secrets.
pod, err := f.CreatePod(f.CreatePodObjectWithOptionalSecretKey(t.Name()))
if err != nil {
t.Fatal(err)
}
// Delete the pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
// Wait for the pod to be reported as running and ready.
if _, err := f.WaitUntilPodReady(pod.Namespace, pod.Name); err != nil {
t.Fatal(err)
}
// Wait for an event concerning the missing secret to be reported on the pod.
if err := f.WaitUntilPodEventWithReason(pod, node.ReasonOptionalSecretNotFound); err != nil {
t.Fatal(err)
}
// Grab the pods from the provider.
pods, err := f.GetRunningPods()
assert.NilError(t, err)
// Check if the pod exists in the slice of Pods.
assert.NilError(t, findPodInPods(pods, pod))
}
// TestCreatePodWithMandatoryInexistentSecrets tries to create a pod referencing inexistent secrets.
// It then verifies that the pod is not created.
func (ts *EndToEndTestSuite) TestCreatePodWithMandatoryInexistentSecrets(t *testing.T) {
// Create a pod with a single container referencing inexistent secrets.
pod, err := f.CreatePod(f.CreatePodObjectWithMandatorySecretKey(t.Name()))
if err != nil {
t.Fatal(err)
}
// Delete the pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
// Wait for an event concerning the missing secret to be reported on the pod.
if err := f.WaitUntilPodEventWithReason(pod, node.ReasonMandatorySecretNotFound); err != nil {
t.Fatal(err)
}
// Grab the pods from the provider.
pods, err := f.GetRunningPods()
assert.NilError(t, err)
// Check if the pod exists in the slice of PodStats.
assert.Assert(t, findPodInPods(pods, pod) != nil)
}
// TestCreatePodWithOptionalInexistentConfigMap tries to create a pod referencing optional, inexistent config map.
// It then verifies that the pod is created successfully.
func (ts *EndToEndTestSuite) TestCreatePodWithOptionalInexistentConfigMap(t *testing.T) {
// Create a pod with a single container referencing optional, inexistent config map.
pod, err := f.CreatePod(f.CreatePodObjectWithOptionalConfigMapKey(t.Name()))
if err != nil {
t.Fatal(err)
}
// Delete the pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
// Wait for the pod to be reported as running and ready.
if _, err := f.WaitUntilPodReady(pod.Namespace, pod.Name); err != nil {
t.Fatal(err)
}
// Wait for an event concerning the missing config map to be reported on the pod.
if err := f.WaitUntilPodEventWithReason(pod, node.ReasonOptionalConfigMapNotFound); err != nil {
t.Fatal(err)
}
// Grab the pods from the provider.
pods, err := f.GetRunningPods()
assert.NilError(t, err)
// Check if the pod exists in the slice of PodStats.
assert.NilError(t, findPodInPods(pods, pod))
}
// TestCreatePodWithMandatoryInexistentConfigMap tries to create a pod referencing inexistent secrets.
// It then verifies that the pod is not created.
func (ts *EndToEndTestSuite) TestCreatePodWithMandatoryInexistentConfigMap(t *testing.T) {
// Create a pod with a single container referencing inexistent config map.
pod, err := f.CreatePod(f.CreatePodObjectWithMandatoryConfigMapKey(t.Name()))
if err != nil {
t.Fatal(err)
}
// Delete the pod after the test finishes.
defer func() {
if err := f.DeletePodImmediately(pod.Namespace, pod.Name); err != nil && !apierrors.IsNotFound(err) {
t.Error(err)
}
}()
// Wait for an event concerning the missing config map to be reported on the pod.
if err := f.WaitUntilPodEventWithReason(pod, node.ReasonMandatoryConfigMapNotFound); err != nil {
t.Fatal(err)
}
// Grab the pods from the provider.
pods, err := f.GetRunningPods()
assert.NilError(t, err)
// Check if the pod exists in the slice of PodStats.
assert.Assert(t, findPodInPods(pods, pod) != nil)
}
// findPodInPodStats returns the index of the specified pod in the .pods field of the specified Summary object.
// It returns an error if the specified pod is not found.
func findPodInPodStats(summary *v1alpha1.Summary, pod *v1.Pod) (int, error) {
for i, p := range summary.Pods {
if p.PodRef.Namespace == pod.Namespace && p.PodRef.Name == pod.Name && string(p.PodRef.UID) == string(pod.UID) {
return i, nil
}
}
return -1, fmt.Errorf("failed to find pod \"%s/%s\" in the slice of pod stats", pod.Namespace, pod.Name)
}
// findPodInPodStats returns the index of the specified pod in the .pods field of the specified PodList object.
// It returns error if the pod doesn't exist in the podlist
func findPodInPods(pods *v1.PodList, pod *v1.Pod) error {
for _, p := range pods.Items {
if p.Namespace == pod.Namespace && p.Name == pod.Name && string(p.UID) == string(pod.UID) {
return nil
}
}
return fmt.Errorf("failed to find pod \"%s/%s\" in the slice of pod list", pod.Namespace, pod.Name)
}

61
test/e2e/node.go Normal file
View File

@@ -0,0 +1,61 @@
package e2e
import (
"context"
"testing"
"time"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"
watchapi "k8s.io/apimachinery/pkg/watch"
)
// TestNodeCreateAfterDelete makes sure that a node is automatically recreated
// if it is deleted while VK is running.
func (ts *EndToEndTestSuite) TestNodeCreateAfterDelete(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
podList, err := f.KubeClient.CoreV1().Pods(f.Namespace).List(metav1.ListOptions{
FieldSelector: fields.OneTermEqualSelector("spec.nodeName", f.NodeName).String(),
})
assert.NilError(t, err)
assert.Assert(t, is.Len(podList.Items, 0), "Kubernetes does not allow node deletion with dependent objects (pods) in existence: %v")
chErr := make(chan error, 1)
originalNode, err := f.GetNode()
assert.NilError(t, err)
ctx, cancel = context.WithTimeout(ctx, time.Minute)
defer cancel()
go func() {
wait := func(e watchapi.Event) (bool, error) {
err = ctx.Err()
// Our timeout has expired
if err != nil {
return true, err
}
if e.Type == watchapi.Deleted || e.Type == watchapi.Error {
return false, nil
}
return originalNode.ObjectMeta.UID != e.Object.(*v1.Node).ObjectMeta.UID, nil
}
chErr <- f.WaitUntilNodeCondition(wait)
}()
assert.NilError(t, f.DeleteNode())
select {
case result := <-chErr:
assert.NilError(t, result, "Did not observe new node object created after deletion")
case <-ctx.Done():
t.Fatal("Test timed out while waiting for node object to be deleted / recreated")
}
}

104
test/e2e/suite.go Normal file
View File

@@ -0,0 +1,104 @@
package e2e
import (
"testing"
"time"
"github.com/virtual-kubelet/virtual-kubelet/internal/test/e2e/framework"
"github.com/virtual-kubelet/virtual-kubelet/internal/test/suite"
)
const defaultWatchTimeout = 2 * time.Minute
// f is a testing framework that is accessible across the e2e package
var f *framework.Framework
// EndToEndTestSuite holds the setup, teardown, and shouldSkipTest functions for a specific provider
type EndToEndTestSuite struct {
setup suite.SetUpFunc
teardown suite.TeardownFunc
shouldSkipTest suite.ShouldSkipTestFunc
}
// EndToEndTestSuiteConfig is the config passed to initialize the testing framework and test suite.
type EndToEndTestSuiteConfig struct {
// Kubeconfig is the path to the kubeconfig file to use when running the test suite outside a Kubernetes cluster.
Kubeconfig string
// Namespace is the name of the Kubernetes namespace to use for running the test suite (i.e. where to create pods).
Namespace string
// NodeName is the name of the virtual-kubelet node to test.
NodeName string
// WatchTimeout is the duration for which the framework watch a particular condition to be satisfied (e.g. watches a pod becoming ready)
WatchTimeout time.Duration
// Setup is a function that sets up provider-specific resource in the test suite
Setup suite.SetUpFunc
// Teardown is a function that tears down provider-specific resources from the test suite
Teardown suite.TeardownFunc
// ShouldSkipTest is a function that determines whether the test suite should skip certain tests
ShouldSkipTest suite.ShouldSkipTestFunc
}
// Setup runs the setup function from the provider and other
// procedures before running the test suite
func (ts *EndToEndTestSuite) Setup() {
if err := ts.setup(); err != nil {
panic(err)
}
// Wait for the virtual kubelet (deployed as a pod) to become fully ready
if _, err := f.WaitUntilPodReady(f.Namespace, f.NodeName); err != nil {
panic(err)
}
}
// Teardown runs the teardown function from the provider and other
// procedures after running the test suite
func (ts *EndToEndTestSuite) Teardown() {
if err := ts.teardown(); err != nil {
panic(err)
}
}
// ShouldSkipTest returns true if a provider wants to skip running a particular test
func (ts *EndToEndTestSuite) ShouldSkipTest(testName string) bool {
return ts.shouldSkipTest(testName)
}
// Run runs tests registered in the test suite
func (ts *EndToEndTestSuite) Run(t *testing.T) {
suite.Run(t, ts)
}
// NewEndToEndTestSuite returns a new EndToEndTestSuite given a test suite configuration,
// setup, and teardown functions from provider
func NewEndToEndTestSuite(cfg EndToEndTestSuiteConfig) *EndToEndTestSuite {
if cfg.Namespace == "" {
panic("Empty namespace")
} else if cfg.NodeName == "" {
panic("Empty node name")
}
if cfg.WatchTimeout == time.Duration(0) {
cfg.WatchTimeout = defaultWatchTimeout
}
f = framework.NewTestingFramework(cfg.Kubeconfig, cfg.Namespace, cfg.NodeName, cfg.WatchTimeout)
emptyFunc := func() error { return nil }
if cfg.Setup == nil {
cfg.Setup = emptyFunc
}
if cfg.Teardown == nil {
cfg.Teardown = emptyFunc
}
if cfg.ShouldSkipTest == nil {
// This will not skip any test in the test suite
cfg.ShouldSkipTest = func(_ string) bool { return false }
}
return &EndToEndTestSuite{
setup: cfg.Setup,
teardown: cfg.Teardown,
shouldSkipTest: cfg.ShouldSkipTest,
}
}