d8dfun/virtual-kubelet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Remove VIC provider code.

Browse Source 
The VIC provider is stale and there are no developers working on this
anymore.  Removing the provider from the repo.
This commit is contained in:
sflxn
2019-05-06 15:57:24 -07:00
parent cfa37871ab
commit 4feab78b76
1113 changed files with 3 additions and 246759 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
providers/register/provider_vic.go
View File

@@ -1,16 +0,0 @@
// +build linux,vic_provider
package register
import (
"github.com/virtual-kubelet/virtual-kubelet/providers"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic"
)
func init() {
register("vic", initVic)
}
func initVic(cfg InitConfig) (providers.Provider, error) {
return vic.NewVicProvider(cfg.ConfigPath, cfg.ResourceManager, cfg.NodeName, cfg.OperatingSystem)
}

80
providers/vic/README.md
View File

@@ -1,80 +0,0 @@
# vSphere Integrated Containers virtual kubelet provider
This is a very early preview of the [VMware vSphere Integrated Containers](https://github.com/vmware/vic) virtual kubelet provider.
## Introduction to vSphere Integrated Containers
For those who are unfamiliar with vSphere Integrated Containers, it is a open source project that enables users to create a CaaS (container as a service) endpoint on a vSphere cluster. There are a few advantages of using vSphere Integrated Containers over using standard Docker in a VM:
1. Containers are wrapped in VMs and provides stronger isolation than a standard OS containers.
1. Use of vSphere Integrated Containers removes the need to deploy a host VM, OS, and docker software. It also removes the requirement to maintain that stack. An administrator can simply deploy a VCH (virtual container host) and get back a docker endpoint. With this endpoint, users can deploy containers. vSphere manages the endpoint.
1. The VCH can span a vCenter cluster, providing a much larger virtual host than any VM or physical machine on which users can deploy their containers. Users can deploy containers across the entire vCenter cluster using this single virtual host.
## Virtual Kubelet in vSphere Integrated Containers
Within a deployed VCH, there is a daemon that provides a Docker compatible endpoint. We refer to this daemon as a Docker personality server. With the vSphere integrated Container virtual kubelet provider, we are introducing a second personality server that provides a Kubelet compatible endpoint for the VCH. With this endpoint, the VCH can join a kubernetes cluster, providing a robust virtual node for pods. Just as the VCH provides a CaaS endpoint for Docker users, it will now provide a Pod-as-a-Service endpoint for kubernetes users. vSphere manages the availability of the pods and remove the need for administrators to maintain the uptime of a normal node, it's software, and the pods running on it. For the user, it provides a much larger virtual node than any single VM or physical machine.
![VIC with virtual kubelet](./vic_kubelet_design.png)
## Pod VM in vSphere Integrated Containers
With the vSphere integrated Container virtual kubelet provider, we are also introducing a new isolation concept called the pod vm. With the original vSphere Integrated Containers, we introduced stronger container isolation through our container vm concept. Interested parties may go to the vSphere Integrated Containers github project page to read all about it. A standard pod in a standard kubernetes node is just a collection of containers with a parent container. The containers are isolated by the OS, but the pod is not. With vSphere Integrated Containers, pods are strongly isolated within a pod vm. Each pod runs within it's own pod vm.
![Pod VM Basic Design](./podvm_basic_design.png)
## Project Status
This is an early preview. The project relies on a feature branch within the vSphere Integrated Containers project. Currently, pods can be created and removed. Some node statuses can be queried. This provider reuses vSphere Integrated Containers to provision pods. As such, there is a dependency between this provider and the vSphere Integrated Containers project. All changes to that project are currently being done on a feature branch, named wolfpack.
There are a lot of kubelet features still undefined in the virtual-kubelet project, such as networking, volumes, exec, logging, etc. As these are defined, we will add this support into our provider and into vSphere Integrated Containers.
## Building vSphere Integrated Containers with virtual kubelet support
This project and the associated vSphere Integrated Containers project are in active development. The simplest way to utilize this virtual kubelet provider, is to build the kubelet project and the wolfpack feature branch of vSphere Integrated Containers. The following is a step by step instructions on building both. There is an asciinema playback of this process below. The current instructions assumes your development environment is Linux.
First, ensure golang 1.8.x and git are installed on your machine. The vSphere Integrated Containers project checks for go 1.8. Next, perform the following steps
1. make sure you have go 1.8.x and prepare your GOPATH
* $> `go version`
* $> `mkdir -p go/src/github.com/virtual-kubelet`
1. get and build the virtual kubelet
* $> `cd go/src/github.com/virtual-kubelet`
* $> `git clone https://github.com/virtual-kubelet/virtual-kubelet.git`
* $> `cd virtual-kubelet`
* $> `go build .`
1. get wolfpack feature branch of vSphere Integrated Containers
* $> `cd`
* $> `mkdir -p go/src/github.com/vmware`
* $> `cd $GOPATH/src/github.com/vmware`
* $> `git clone https://github.com/vmware/vic`
* $> `cd vic`
* $> `git checkout feature/wolfpack`
1. build vSphere Integrated Containers **as root**
* $> `cd $GOPATH/src/github.com/vmware/vic`
* $> `sudo su`
* $> `export GOPATH=/home/[user]/go`
* $> `export PATH=$PATH:/usr/local/go/bin:/home/[user]/go/bin`
* $> `export VIRTUAL_KUBELET_PATH=$GOPATH/src/github.com/virtual-kubelet/virtual-kubelet/virtual-kubelet`
* $> `make most-vkubelet`
* $> `chown -R user:user *`
* $> `exit` (exit root)
You should now have a **bin** folder in the vic folder. This contains all the necessary assets to deploy a VCH using the vic-machine CLI.
[Asciinema recording of the build steps](https://asciinema.org/a/oeGbhPmKWqVgWeQxCOPLHCcYN)
## Usage
To get started with the vSphere Integrated Containers' virtual kubelet, users should first familiarize themselves with deploying vSphere Integrated Containers and using the VCH as a docker endpoint. We're leveraging the same deployment model with the virtual kubelet. Administrators deploy a VCH, using the vic-machine CLI or [VIC Appliance](https://github.com/vmware/vic-product), specifying the kubernetes cluster that the VCH should join.
Virtual Container Hosts should be deployed with the proper CLI flags to enable virtual kubelet, including both `--k8s-server-address` and `--k8s-config`.
Once the VCH is fully started, it automatically joins the cluster. Administrators can then apply taints or labels to the virtual node to create node affinity. Pods created with the right toleration or nodeSelector will then be deployed onto the VCH.
1. Get IP or FQDN of your kubernetes master node
2. Get location of your kube config you use for kubectl (usually in $HOME/.kube/config)
3. Deploy a virtual container host with vic-machine.
4. $> `kubectl get nodes`
5. Deploy a pod
[Asciinema recording of an example usage](https://asciinema.org/a/nArPOJSKWJwx09UsJVUFiI2y7)

162
providers/vic/cache/pod_cache.go vendored
View File

@@ -1,162 +0,0 @@
package cache
import (
"fmt"
"sync"
"github.com/vmware/vic/pkg/trace"
vicpod "github.com/virtual-kubelet/virtual-kubelet/providers/vic/pod"
)
type PodCache interface {
Rehydrate(op trace.Operation) error
Get(op trace.Operation, namespace, name string) (*vicpod.VicPod, error)
GetAll(op trace.Operation) []*vicpod.VicPod
Add(op trace.Operation, namespace, name string, pod *vicpod.VicPod) error
Delete(op trace.Operation, namespace, name string) error
}
type VicPodCache struct {
cache map[string]*vicpod.VicPod
lock sync.Mutex
}
type CacheError string
func (c CacheError) Error() string { return string(c) }
const (
PodCachePodNameError = CacheError("PodCache called with empty pod name")
PodCacheNilPodError = CacheError("PodCache called with nil pod")
)
func NewVicPodCache() PodCache {
v := &VicPodCache{}
v.cache = make(map[string]*vicpod.VicPod, 0)
return v
}
// Rehydrate replenishes the cache in the event of a virtual kubelet restart.
// NOT YET IMPLEMENTED
//
// arguments:
// op operation trace logger
// returns:
// error
func (v *VicPodCache) Rehydrate(op trace.Operation) error {
return nil
}
// Get returns the pod definition for a running pod
//
// arguments:
// op operation trace logger
// namespace namespace of the pod. Empty namespace assumes default.
// name name of the pod
// returns:
// error
func (v *VicPodCache) Get(op trace.Operation, namespace, name string) (*vicpod.VicPod, error) {
defer trace.End(trace.Begin(name, op))
if name == "" {
op.Errorf(PodCachePodNameError.Error())
return nil, PodCachePodNameError
}
//TODO: handle namespaces
pod, ok := v.cache[name]
if !ok {
err := fmt.Errorf("Pod %s not found in cache", name)
op.Info(err)
return nil, err
}
return pod, nil
}
// GetAll returns the pod definitions for all running pods
//
// arguments:
// op operation trace logger
// returns:
// error
func (v *VicPodCache) GetAll(op trace.Operation) []*vicpod.VicPod {
defer trace.End(trace.Begin("", op))
defer v.lock.Unlock()
v.lock.Lock()
list := make([]*vicpod.VicPod, 0)
for _, vp := range v.cache {
list = append(list, vp)
}
return list
}
// Add saves the pod definition of a running pod
//
// arguments:
// op operation trace logger
// namespace namespace of the pod. Empty namespace assumes default.
// name name of the pod
// pod pod definition
// returns:
// error
func (v *VicPodCache) Add(op trace.Operation, namespace, name string, pod *vicpod.VicPod) error {
defer trace.End(trace.Begin(name, op))
defer v.lock.Unlock()
v.lock.Lock()
if name == "" {
op.Errorf(PodCachePodNameError.Error())
return PodCachePodNameError
}
if pod == nil {
op.Errorf(PodCacheNilPodError.Error())
return PodCacheNilPodError
}
//TODO: handle namespaces
_, ok := v.cache[name]
if ok {
err := fmt.Errorf("Pod %s already cached. Duplicate pod.", name)
op.Error(err)
return err
}
v.cache[name] = pod
return nil
}
// Delete removes a pod definition from the cache. It does not stop/delete the
// actual pod.
//
// arguments:
// op operation trace logger
// namespace namespace of the pod. Empty namespace assumes default.
// name name of the pod
// returns:
// error
func (v *VicPodCache) Delete(op trace.Operation, namespace, name string) error {
defer trace.End(trace.Begin(name, op))
defer v.lock.Unlock()
v.lock.Lock()
if name == "" {
op.Errorf(PodCachePodNameError.Error())
return PodCachePodNameError
}
//TODO: handle namespaces
delete(v.cache, name)
return nil
}

139
providers/vic/cache/pod_cache_test.go vendored
View File

@@ -1,139 +0,0 @@
package cache
import (
"context"
"testing"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/pod"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
v1 "k8s.io/api/core/v1"
"github.com/vmware/vic/pkg/trace"
)
var (
testpod *v1.Pod
)
func init() {
testpod = &v1.Pod{}
}
func setup(t *testing.T, op trace.Operation) PodCache {
c := NewVicPodCache()
assert.Check(t, c != nil, "NewPod did not return a valid cache")
//populate with dummy data
vp := pod.VicPod{
ID: "123",
Pod: testpod,
}
c.Add(op, "namespace1", "testpod1a", &vp)
c.Add(op, "namespace1", "testpod1b", &vp)
c.Add(op, "namespace2", "testpod2a", &vp)
c.Add(op, "namespace2", "testpod2b", &vp)
return c
}
func TestRehydrate(t *testing.T) {
op := trace.NewOperation(context.Background(), "")
c := NewVicPodCache()
assert.Check(t, c != nil, "NewPod did not return a valid cache")
err := c.Rehydrate(op)
assert.Check(t, err, "PodCache.Rehydrate failed with error: %s", err)
}
func TestAdd(t *testing.T) {
var err error
op := trace.NewOperation(context.Background(), "")
c := NewVicPodCache()
assert.Check(t, c != nil, "NewPod did not return a valid cache")
//populate with dummy data
vp := pod.VicPod{
ID: "123",
Pod: testpod,
}
// Positive cases
err = c.Add(op, "namespace1", "testpod1a", &vp)
assert.Check(t, err, "PodCache.Add failed with error: %s", err)
// Negative cases
err = c.Add(op, "namespace1", "", &vp)
assert.Check(t, err != nil, "PodCache.Add expected error for empty name")
assert.Check(t, is.DeepEqual(err, PodCachePodNameError))
err = c.Add(op, "namespace1", "test2", nil)
assert.Check(t, err != nil, "PodCache.Add expected error for nil pod")
assert.Check(t, is.DeepEqual(err, PodCacheNilPodError))
}
func TestGet(t *testing.T) {
var err error
var vpod *pod.VicPod
op := trace.NewOperation(context.Background(), "")
c := setup(t, op)
// Positive cases
vpod, err = c.Get(op, "namespace1", "testpod1a")
assert.Check(t, err, "PodCache.Get failed with error: %s", err)
assert.Check(t, vpod != nil, "PodCache.Get expected to return non-nil pod but received nil")
vpod, err = c.Get(op, "namespace2", "testpod2a")
assert.Check(t, err, "PodCache.Get failed with error: %s", err)
assert.Check(t, vpod != nil, "PodCache.Get expected to return non-nil pod but received nil")
// Negative cases
vpod, err = c.Get(op, "namespace1", "")
assert.Check(t, is.DeepEqual(err, PodCachePodNameError))
assert.Check(t, is.Nil(vpod), "PodCache.Get expected to return nil pod but received non-nil")
//TODO: uncomment out once namespace support added to cache
//vpod, err = c.Get(op, "namespace1", "testpod2a")
//assert.NotNil(t, err, "PodCache.Get did not respect namespace: %s", err)
//vpod, err = c.Get(op, "", "testpod1a")
//assert.NotNil(t, err, "PodCache.Get did not respect namespace: %s", err)
}
func TestGetAll(t *testing.T) {
op := trace.NewOperation(context.Background(), "")
c := setup(t, op)
vps := c.GetAll(op)
assert.Check(t, vps != nil, "PodCache.GetAll returned nil slice")
assert.Check(t, is.Len(vps, 4), "PodCache.Get did not return all pod definitions. Returned %d pods.", len(vps))
}
func TestDelete(t *testing.T) {
var err error
op := trace.NewOperation(context.Background(), "")
c := setup(t, op)
// Positive cases
err = c.Delete(op, "namespace1", "testpod1a")
assert.Check(t, err, "PodCache.Delete failed with error: %s", err)
vps := c.GetAll(op)
assert.Check(t, is.Len(vps, 3), "PodCache.Delete did not delete pod.")
// Negative cases
err = c.Delete(op, "namespace2", "")
assert.Check(t, is.DeepEqual(err, PodCachePodNameError))
//TODO: uncomment the tests below once namespace support added to cache
//vps = c.GetAll(op)
//currCount := len(vps)
//err = c.Delete(op, "", "testpod1b")
//assert.NotNil(t, err, "PodCache.Delete expected to return error but received nil")
//vps = c.GetAll(op)
//assert.Len(t, vps, currCount, "PodCache.Delete ignored namespace")
}

77
providers/vic/config.go
View File

@@ -1,77 +0,0 @@
package vic
import (
"context"
"fmt"
"io/ioutil"
"os"
"gopkg.in/yaml.v2"
"strings"
"github.com/vmware/vic/pkg/trace"
)
type VicConfig struct {
PersonaAddr string `yaml:"persona-server"`
PortlayerAddr string `yaml:"portlayer-server"`
HostUUID string `yaml:"host-uuid"`
}
const (
personaAddrEnv = "PERSONA_ADDR"
portlayerAddrEnv = "PORTLAYER_ADDR"
hostUUIDEnv = "HOST_UUID"
localVirtualKubelet = "LOCAL_VIRTUAL_KUBELET"
)
func LocalInstance() bool {
value := strings.ToLower(os.Getenv(localVirtualKubelet))
if value == "1" || value == "t" || value == "true" {
return true
}
return false
}
func NewVicConfig(op trace.Operation, configFile string) VicConfig {
var config VicConfig
if configFile == "" {
config.loadConfigFromEnv()
} else {
config.loadConfigFile(configFile)
}
return config
}
func (v *VicConfig) loadConfigFile(configFile string) error {
op := trace.NewOperation(context.Background(), "LoadConfigFile - %s", configFile)
defer trace.End(trace.Begin("", op))
contents, err := ioutil.ReadFile(configFile)
if err != nil {
return err
}
var config VicConfig
err = yaml.Unmarshal(contents, &config)
if err != nil {
err = fmt.Errorf("Unable to unmarshal vic virtual kubelet configfile: %s", err.Error())
op.Error(err)
return err
}
*v = config
return nil
}
func (v *VicConfig) loadConfigFromEnv() {
v.PersonaAddr = os.Getenv(personaAddrEnv)
v.PortlayerAddr = os.Getenv(portlayerAddrEnv)
v.HostUUID = os.Getenv(hostUUIDEnv)
}

13
providers/vic/constants/constants.go
View File

@@ -1,13 +0,0 @@
package constants
const (
// DefaultCPUs - the default number of container VM CPUs
DefaultCPUs = 2
DefaultMemory = 512
DummyImage = "f6e427c148a766d2d6c117d67359a0aa7d133b5bc05830a7ff6e8b64ff6b1d1d" //busybox
DummyLayerID = "02d3847f0b0fb7acd4419040cc53febf91cb112db2451d9b27a245dee5b227c0" //busybox
DummyRepoName = "busybox"
HostName = "test-kubelet"
)

113
providers/vic/docker_client.go
View File

@@ -1,113 +0,0 @@
package vic
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"time"
"github.com/vmware/vic/pkg/trace"
)
type CreateResponse struct {
Id string `json:"Id"`
Warnings string `json:"Warnings"`
}
// Super simplistic docker client for the virtual kubelet to perform some operations
type DockerClient interface {
Ping(op trace.Operation) error
CreateContainer(op trace.Operation, config string) error
PullImage(op trace.Operation, image string) error
}
type VicDockerClient struct {
serverAddr string
}
func NewVicDockerClient(personaAddr string) DockerClient {
return &VicDockerClient{
serverAddr: personaAddr,
}
}
func (v *VicDockerClient) Ping(op trace.Operation) error {
personaServer := fmt.Sprintf("http://%s/v1.35/info", v.serverAddr)
resp, err := http.Get(personaServer)
if err != nil {
op.Errorf("Ping failed: error = %s", err.Error())
return err
}
if resp.StatusCode >= 300 {
op.Errorf("Ping failed: status = %d", resp.StatusCode)
return fmt.Errorf("Server Error")
}
return nil
}
func (v *VicDockerClient) CreateContainer(op trace.Operation, config string) error {
personaServer := fmt.Sprintf("http://%s/v1.35/containers/create", v.serverAddr)
reader := bytes.NewBuffer([]byte(config))
resp, err := http.Post(personaServer, "application/json", reader)
if err != nil {
op.Errorf("Error from from docker create: error = %s", err.Error())
return err
}
if resp.StatusCode >= 300 {
op.Errorf("Error from from docker create: status = %d", resp.StatusCode)
return fmt.Errorf("Image not found")
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
op.Infof("Response from docker create: status = %d", resp.StatusCode)
op.Infof("Response from docker create: body = %s", string(body))
var createResp CreateResponse
err = json.Unmarshal(body, &createResp)
if err != nil {
op.Errorf("Failed to unmarshal response from container create post")
return err
}
startContainerUrl := fmt.Sprintf("http://%s/v1.35/containers/%s/start", v.serverAddr, createResp.Id)
op.Infof("Starting container with request - %s", startContainerUrl)
_, err = http.Post(startContainerUrl, "", nil)
if err != nil {
op.Errorf("Failed to start container %s", createResp.Id)
return err
}
return nil
}
func (v *VicDockerClient) PullImage(op trace.Operation, image string) error {
pullClient := &http.Client{Timeout: 60 * time.Second}
personaServer := fmt.Sprintf("http://%s/v1.35/images/create?fromImage=%s", v.serverAddr, image)
op.Infof("POST %s", personaServer)
reader := bytes.NewBuffer([]byte(""))
resp, err := pullClient.Post(personaServer, "application/json", reader)
if err != nil {
op.Errorf("Error from docker pull: error = %s", err.Error())
return err
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
msg := fmt.Sprintf("Error from docker pull: status = %d", resp.StatusCode)
op.Errorf(msg)
return fmt.Errorf(msg)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
msg := fmt.Sprintf("Error reading docker pull response: error = %s", err.Error())
op.Errorf(msg)
return fmt.Errorf(msg)
}
op.Infof("Response from docker pull: body = %s", string(body))
return nil
}

268
providers/vic/operations/common_test.go
View File

@@ -1,268 +0,0 @@
package operations
import (
"context"
"testing"
vicpod "github.com/virtual-kubelet/virtual-kubelet/providers/vic/pod"
"github.com/vmware/vic/lib/metadata"
"github.com/vmware/vic/pkg/trace"
"k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"fmt"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/cache"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
proxymocks "github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy/mocks"
)
var (
pod v1.Pod
imgConfig metadata.ImageConfig
busyboxIsoConfig proxy.IsolationContainerConfig
alpineIsoConfig proxy.IsolationContainerConfig
vicPod vicpod.VicPod
)
const (
podID = "123"
podName = "busybox-sleep"
podHandle = "fakehandle"
fakeEP = "fake-endpoint"
stateRunning = "Running"
stateStarting = "Starting"
stateStopping = "Stopping"
stateStopped = "Stopped"
stateRemoving = "Removing"
stateRemoved = "Removed"
)
func createMocks(t *testing.T) (*proxymocks.ImageStore, *proxymocks.IsolationProxy, cache.PodCache, trace.Operation) {
store := &proxymocks.ImageStore{}
ip := &proxymocks.IsolationProxy{}
cache := cache.NewVicPodCache()
op := trace.NewOperation(context.Background(), "tests")
return store, ip, cache, op
}
func fakeError(myErr string) error {
return fmt.Errorf("fake error: %s", myErr)
}
func initPod() {
pod = v1.Pod{
//TypeMeta: v1.TypeMeta{},
ObjectMeta: metav1.ObjectMeta{
Name: podName,
GenerateName: "",
Namespace: "default",
SelfLink: "/api/v1/namespaces/default/pods/busybox-sleep",
UID: "b1fc6e1b-499b-11e8-946c-000c29479092",
ResourceVersion: "10338145",
Generation: 0,
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: map[string]string{},
Annotations: map[string]string{},
OwnerReferences: nil,
Initializers: nil,
Finalizers: nil,
ClusterName: "",
},
Spec: v1.PodSpec{
Volumes: []v1.Volume{
{
Name: "default-token-9q9lr",
VolumeSource: v1.VolumeSource{
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: &v1.SecretVolumeSource{
SecretName: "default-token-9q9lr",
Items: nil,
Optional: nil,
},
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: nil,
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
},
},
},
InitContainers: nil,
Containers: []v1.Container{
{
Name: "busybox-container",
Image: "busybox",
Command: []string{"/bin/sleep"},
Args: []string{"2m"},
WorkingDir: "",
Ports: nil,
EnvFrom: nil,
Env: nil,
Resources: v1.ResourceRequirements{},
VolumeMounts: []v1.VolumeMount{
{
Name: "default-token-9q9lr",
ReadOnly: true,
MountPath: "/var/run/secrets/kubernetes.io/serviceaccount",
SubPath: "",
MountPropagation: nil,
},
},
LivenessProbe: nil,
ReadinessProbe: nil,
Lifecycle: nil,
TerminationMessagePath: "/dev/termination-log",
TerminationMessagePolicy: "File",
ImagePullPolicy: "IfNotPresent",
SecurityContext: nil,
Stdin: false,
StdinOnce: false,
TTY: false,
},
{
Name: "alpine-container",
Image: "alpine",
Command: nil,
Args: nil,
WorkingDir: "",
Ports: nil,
EnvFrom: nil,
Env: nil,
Resources: v1.ResourceRequirements{},
VolumeMounts: []v1.VolumeMount{
{
Name: "default-token-9q9lr",
ReadOnly: true,
MountPath: "/var/run/secrets/kubernetes.io/serviceaccount",
SubPath: "",
MountPropagation: nil,
},
},
LivenessProbe: nil,
ReadinessProbe: nil,
Lifecycle: nil,
TerminationMessagePath: "/dev/termination-log",
TerminationMessagePolicy: "File",
ImagePullPolicy: "IfNotPresent",
SecurityContext: nil,
Stdin: false,
StdinOnce: false,
TTY: false,
},
},
RestartPolicy: "Always",
TerminationGracePeriodSeconds: new(int64),
ActiveDeadlineSeconds: nil,
DNSPolicy: "ClusterFirst",
NodeSelector: map[string]string{"affinity": "vmware"},
ServiceAccountName: "default",
DeprecatedServiceAccount: "default",
AutomountServiceAccountToken: nil,
NodeName: "vic-kubelet",
HostNetwork: false,
HostPID: false,
HostIPC: false,
SecurityContext: &v1.PodSecurityContext{},
ImagePullSecrets: nil,
Hostname: "",
Subdomain: "",
Affinity: nil,
SchedulerName: "default-scheduler",
Tolerations: []v1.Toleration{
{
Key: "node.kubernetes.io/not-ready",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: new(int64),
},
{
Key: "node.kubernetes.io/unreachable",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: new(int64),
},
},
HostAliases: nil,
PriorityClassName: "",
Priority: nil,
},
}
busyboxIsoConfig = proxy.IsolationContainerConfig{
ID: "",
ImageID: "",
LayerID: "",
ImageName: "busybox",
Name: "busybox-container",
Namespace: "",
Cmd: []string{"/bin/sleep", "2m"},
Path: "",
Entrypoint: nil,
Env: nil,
WorkingDir: "",
User: "",
StopSignal: "",
Attach: false,
StdinOnce: false,
OpenStdin: false,
Tty: false,
CPUCount: 2,
Memory: 2048,
PortMap: map[string]proxy.PortBinding{},
}
alpineIsoConfig = proxy.IsolationContainerConfig{
ID: "",
ImageID: "",
LayerID: "",
ImageName: "alpine",
Name: "alpine-container",
Namespace: "",
Cmd: nil,
Path: "",
Entrypoint: nil,
Env: nil,
WorkingDir: "",
User: "",
StopSignal: "",
Attach: false,
StdinOnce: false,
OpenStdin: false,
Tty: false,
CPUCount: 2,
Memory: 2048,
PortMap: map[string]proxy.PortBinding{},
}
vicPod = vicpod.VicPod{
ID: podID,
Pod: &pod,
}
}

455
providers/vic/operations/pod_creator.go
View File

@@ -1,455 +0,0 @@
package operations
import (
"fmt"
"strings"
"sync"
"time"
"github.com/kr/pretty"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/cache"
vicpod "github.com/virtual-kubelet/virtual-kubelet/providers/vic/pod"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
vicerrors "github.com/vmware/vic/lib/apiservers/engine/errors"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/lib/metadata"
"github.com/vmware/vic/pkg/trace"
"k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
type PodCreator interface {
CreatePod(op trace.Operation, pod *v1.Pod, start bool) error
}
type VicPodCreator struct {
client *client.PortLayer
imageStore proxy.ImageStore
isolationProxy proxy.IsolationProxy
podCache cache.PodCache
personaAddr string
portlayerAddr string
}
type VicPodCreatorError string
func (e VicPodCreatorError) Error() string { return string(e) }
func NewPodCreatorPullError(image, msg string) VicPodCreatorError {
return VicPodCreatorError(fmt.Sprintf("VicPodCreator failed to get image %s's config from the image store: %s", image, msg))
}
func NewPodCreatorNilImgConfigError(image string) VicPodCreatorError {
return VicPodCreatorError(fmt.Sprintf("VicPodCreator failed to get image %s's config from the image store", image))
}
const (
// MemoryAlignMB is the value to which container VM memory must align in order for hotadd to work
MemoryAlignMB = 128
// MemoryMinMB - the minimum allowable container memory size
MemoryMinMB = 512
// MemoryDefaultMB - the default container VM memory size
MemoryDefaultMB = 2048
// MinCPUs - the minimum number of allowable CPUs the container can use
MinCPUs = 1
// DefaultCPUs - the default number of container VM CPUs
DefaultCPUs = 2
DefaultMemory = 512
MiBytesUnit = 1024 * 1024
defaultEnvPath = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
// Errors
PodCreatorPortlayerClientError = VicPodCreatorError("PodCreator called with an invalid portlayer client")
PodCreatorImageStoreError = VicPodCreatorError("PodCreator called with an invalid image store")
PodCreatorIsolationProxyError = VicPodCreatorError("PodCreator called with an invalid isolation proxy")
PodCreatorPodCacheError = VicPodCreatorError("PodCreator called with an invalid pod cache")
PodCreatorPersonaAddrError = VicPodCreatorError("PodCreator called with an invalid VIC persona addr")
PodCreatorPortlayerAddrError = VicPodCreatorError("PodCreator called with an invalid VIC portlayer addr")
PodCreatorInvalidPodSpecError = VicPodCreatorError("CreatePod called with nil pod")
PodCreatorInvalidArgsError = VicPodCreatorError("Invalid arguments")
)
func NewPodCreator(client *client.PortLayer, imageStore proxy.ImageStore, isolationProxy proxy.IsolationProxy, podCache cache.PodCache, personaAddr string, portlayerAddr string) (PodCreator, error) {
if client == nil {
return nil, PodCreatorPortlayerClientError
} else if imageStore == nil {
return nil, PodCreatorImageStoreError
} else if isolationProxy == nil {
return nil, PodCreatorIsolationProxyError
} else if podCache == nil {
return nil, PodCreatorPodCacheError
}
return &VicPodCreator{
client: client,
imageStore: imageStore,
podCache: podCache,
personaAddr: personaAddr,
portlayerAddr: portlayerAddr,
isolationProxy: isolationProxy,
}, nil
}
// CreatePod creates the pod and potentially start it
//
// arguments:
// op operation trace logger
// pod pod spec
// start start the pod after creation
// returns:
// error
func (v *VicPodCreator) CreatePod(op trace.Operation, pod *v1.Pod, start bool) error {
defer trace.End(trace.Begin("", op))
if pod == nil {
op.Errorf(PodCreatorInvalidPodSpecError.Error())
return PodCreatorInvalidPodSpecError
}
defer trace.End(trace.Begin(pod.Name, op))
// Pull all containers simultaneously
err := v.pullPodContainers(op, pod)
if err != nil {
op.Errorf("PodCreator failed to pull containers: %s", err.Error())
return err
}
// Transform kube container config to docker create config
id, err := v.createPod(op, pod, start)
if err != nil {
op.Errorf("pod_creator failed to create pod: %s", err.Error())
return err
}
vp := &vicpod.VicPod{
ID: id,
Pod: pod.DeepCopy(),
}
err = v.podCache.Add(op, "", pod.Name, vp)
if err != nil {
//TODO: What should we do if pod already exist?
}
if start {
ps, err := NewPodStarter(v.client, v.isolationProxy)
if err != nil {
op.Errorf("Error creating pod starter: %s", err.Error())
return err
}
err = ps.Start(op, id, pod.Name)
if err != nil {
return err
}
now := metav1.NewTime(time.Now())
vp.Pod.Status.StartTime = &now
}
return nil
}
// pullPodContainers simultaneously pulls all containers in a pod
//
// arguments:
// op operation trace logger
// pod pod spec
// returns:
// error
func (v *VicPodCreator) pullPodContainers(op trace.Operation, pod *v1.Pod) error {
defer trace.End(trace.Begin("", op))
if pod == nil || pod.Spec.Containers == nil {
return PodCreatorInvalidPodSpecError
}
var pullGroup sync.WaitGroup
errChan := make(chan error, 2)
for _, c := range pod.Spec.Containers {
pullGroup.Add(1)
go func(img string, policy v1.PullPolicy) {
defer pullGroup.Done()
// Pull image config from VIC's image store if policy allows
var realize bool
if policy == v1.PullIfNotPresent {
realize = true
} else {
realize = false
}
_, err := v.imageStore.Get(op, img, "", realize)
if err != nil {
err = fmt.Errorf("VicPodCreator failed to get image %s's config from the image store: %s", img, err.Error())
op.Error(err)
errChan <- err
}
}(c.Image, c.ImagePullPolicy)
}
pullGroup.Wait()
close(errChan)
for err := range errChan {
if err != nil {
return err
}
}
return nil
}
// createPod creates a pod using the VIC portlayer. Images can be pulled serially if not already present.
//
// arguments:
// op operation trace logger
// pod pod spec
// start start the pod after creation
// returns:
// (pod id, error)
func (v *VicPodCreator) createPod(op trace.Operation, pod *v1.Pod, start bool) (string, error) {
defer trace.End(trace.Begin("", op))
if pod == nil || pod.Spec.Containers == nil {
op.Errorf(PodCreatorInvalidPodSpecError.Error())
return "", PodCreatorInvalidPodSpecError
}
id, h, err := v.isolationProxy.CreateHandle(op)
if err != nil {
return "", err
}
for idx, c := range pod.Spec.Containers {
// Pull image config from VIC's image store if policy allows
var realize bool
if c.ImagePullPolicy == v1.PullIfNotPresent {
realize = true
} else {
realize = false
}
imgConfig, err := v.imageStore.Get(op, c.Image, "", realize)
if err != nil {
err = NewPodCreatorPullError(c.Image, err.Error())
op.Error(err)
return "", err
}
if imgConfig == nil {
err = NewPodCreatorNilImgConfigError(c.Image)
op.Error(err)
return "", err
}
op.Debugf("Receive image config from imagestore = %# v", pretty.Formatter(imgConfig))
// Create the initial config
ic, err := IsolationContainerConfigFromKubeContainer(op, &c, imgConfig, pod)
if err != nil {
return "", err
}
op.Debugf("isolation config %# v", pretty.Formatter(ic))
h, err = v.isolationProxy.AddImageToHandle(op, h, c.Name, imgConfig.V1Image.ID, imgConfig.ImageID, imgConfig.Name)
if err != nil {
return "", err
}
//TODO: We need one task with the container ID as the portlayer uses this to track session. Longer term, we should figure out
// a way to fix this in the portlayer?
if idx == 0 {
h, err = v.isolationProxy.CreateHandleTask(op, h, id, imgConfig.V1Image.ID, ic)
} else {
h, err = v.isolationProxy.CreateHandleTask(op, h, fmt.Sprintf("Container-%d-task", idx), imgConfig.V1Image.ID, ic)
}
if err != nil {
return "", err
}
h, err = v.isolationProxy.AddHandleToScope(op, h, ic)
if err != nil {
return id, err
}
}
// Need both interaction and logging added or we will not be able to retrieve output.log or tether.debug
h, err = v.isolationProxy.AddInteractionToHandle(op, h)
if err != nil {
return "", err
}
h, err = v.isolationProxy.AddLoggingToHandle(op, h)
if err != nil {
return "", err
}
err = v.isolationProxy.CommitHandle(op, h, id, -1)
if err != nil {
return "", err
}
op.Debugf("Created Pod: %s, Handle: %s, ID: %s", pod.Name, h, id)
return id, nil
}
//------------------------------------
// Utility Functions
//------------------------------------
func IsolationContainerConfigFromKubeContainer(op trace.Operation, cSpec *v1.Container, imgConfig *metadata.ImageConfig, pod *v1.Pod) (proxy.IsolationContainerConfig, error) {
defer trace.End(trace.Begin("", op))
if cSpec == nil || imgConfig == nil || pod == nil {
op.Errorf("Invalid args to IsolationContainerConfigFromKubeContainer: cSpec(%#v), imgConfig(%#v), pod(%#v)", cSpec, imgConfig, pod)
return proxy.IsolationContainerConfig{}, PodCreatorInvalidArgsError
}
op.Debugf("** IsolationContainerConfig... imgConfig = %#v", imgConfig)
config := proxy.IsolationContainerConfig{
Name: cSpec.Name,
WorkingDir: cSpec.WorkingDir,
ImageName: cSpec.Image,
Tty: cSpec.TTY,
StdinOnce: cSpec.StdinOnce,
OpenStdin: cSpec.Stdin,
PortMap: make(map[string]proxy.PortBinding, 0),
}
setResourceFromKubeSpec(op, &config, cSpec)
// Overwrite or append the image's config from the CLI with the metadata from the image's
// layer metadata where appropriate
if len(cSpec.Command) > 0 {
config.Cmd = make([]string, len(cSpec.Command))
copy(config.Cmd, cSpec.Command)
config.Cmd = append(config.Cmd, cSpec.Args...)
} else if imgConfig.Config != nil {
config.Cmd = make([]string, len(imgConfig.Config.Cmd))
copy(config.Cmd, imgConfig.Config.Cmd)
}
config.User = ""
if imgConfig.Config != nil {
if imgConfig.Config.User != "" {
config.User = imgConfig.Config.User
}
// set up environment
config.Env = setEnvFromImageConfig(config.Tty, config.Env, imgConfig.Config.Env)
}
op.Debugf("config = %#v", config)
// TODO: Cache the container (so that they are shared with the persona)
return config, nil
}
func setEnvFromImageConfig(tty bool, env []string, imgEnv []string) []string {
// Set PATH in ENV if needed
env = setPathFromImageConfig(env, imgEnv)
containerEnv := make(map[string]string, len(env))
for _, e := range env {
kv := strings.SplitN(e, "=", 2)
var val string
if len(kv) == 2 {
val = kv[1]
}
containerEnv[kv[0]] = val
}
// Set TERM to xterm if tty is set, unless user supplied a different TERM
if tty {
if _, ok := containerEnv["TERM"]; !ok {
env = append(env, "TERM=xterm")
}
}
// add remaining environment variables from the image config to the container
// config, taking care not to overwrite anything
for _, imageEnv := range imgEnv {
key := strings.SplitN(imageEnv, "=", 2)[0]
// is environment variable already set in container config?
if _, ok := containerEnv[key]; !ok {
// no? let's copy it from the image config
env = append(env, imageEnv)
}
}
return env
}
func setPathFromImageConfig(env []string, imgEnv []string) []string {
// check if user supplied PATH environment variable at creation time
for _, v := range env {
if strings.HasPrefix(v, "PATH=") {
// a PATH is set, bail
return env
}
}
// check to see if the image this container is created from supplies a PATH
for _, v := range imgEnv {
if strings.HasPrefix(v, "PATH=") {
// a PATH was found, add it to the config
env = append(env, v)
return env
}
}
// no PATH set, use the default
env = append(env, fmt.Sprintf("PATH=%s", defaultEnvPath))
return env
}
func setResourceFromKubeSpec(op trace.Operation, config *proxy.IsolationContainerConfig, cSpec *v1.Container) error {
if config == nil {
return vicerrors.BadRequestError("invalid config")
}
// Get resource request. If not specified, use the limits. If that's not set, use default VIC values.
config.CPUCount = cSpec.Resources.Requests.Cpu().Value()
if config.CPUCount == 0 {
config.CPUCount = cSpec.Resources.Limits.Cpu().Value()
if config.CPUCount == 0 {
config.CPUCount = DefaultCPUs
}
}
config.Memory = cSpec.Resources.Requests.Memory().Value()
if config.Memory == 0 {
config.Memory = cSpec.Resources.Limits.Memory().Value()
if config.Memory == 0 {
config.Memory = DefaultMemory
}
}
// convert from bytes to MiB for vsphere
memoryMB := config.Memory / MiBytesUnit
if memoryMB == 0 {
memoryMB = MemoryDefaultMB
} else if memoryMB < MemoryMinMB {
memoryMB = MemoryMinMB
}
// check that memory is aligned
if remainder := memoryMB % MemoryAlignMB; remainder != 0 {
op.Warnf("Default container VM memory must be %d aligned for hotadd, rounding up.", MemoryAlignMB)
memoryMB += MemoryAlignMB - remainder
}
config.Memory = memoryMB
op.Debugf("Container memory: %d MB", config.Memory)
return nil
}

467
providers/vic/operations/pod_creator_test.go
View File

@@ -1,467 +0,0 @@
package operations
import (
"fmt"
"testing"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/lib/metadata"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
)
func init() {
initPod()
}
func TestNewPodCreator(t *testing.T) {
var c PodCreator
var err error
store, proxy, cache, _ := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Positive cases
c, err = NewPodCreator(client, store, proxy, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
// Negative cases
c, err = NewPodCreator(nil, store, proxy, cache, persona, portlayer)
assert.Check(t, is.Nil(c), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodCreatorPortlayerClientError))
c, err = NewPodCreator(client, nil, proxy, cache, persona, portlayer)
assert.Check(t, is.Nil(c), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodCreatorImageStoreError))
c, err = NewPodCreator(client, store, nil, cache, persona, portlayer)
assert.Check(t, is.Nil(c), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodCreatorIsolationProxyError))
c, err = NewPodCreator(client, store, proxy, nil, persona, portlayer)
assert.Check(t, is.Nil(c), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodCreatorPodCacheError))
}
func TestCreatePod_NilPod(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Create nil pod
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, nil, true)
assert.Check(t, err != nil, "Expected error from createPod but received '%s'", err)
}
func TestCreatePod_Success(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err, "Expected error from createPod but received '%s'", err)
}
func TestCreatePod_ImageStoreError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
fakeErr := fmt.Errorf("Error getting pod containers")
store.On("Get", op, "alpine", "", true).Return(nil, fakeErr)
store.On("Get", op, "busybox", "", true).Return(nil, fakeErr)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
}
func TestCreatePod_CreateHandleError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake create handle error")
ip.On("CreateHandle", op).Return(podID, podHandle, fakeErr)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_AddImageError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake add image error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, fakeErr)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_CreateHandleTaskError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake create handle task error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, fakeErr)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, fakeErr)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_AddHandleToScopeError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake add handle to scope error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, fakeErr)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, fakeErr)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_AddInteractionError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake add interaction error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, fakeErr)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_AddLoggingError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake add logging error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, fakeErr)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_CommitError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake commit error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(fakeErr)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_HandleError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake handle error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, fakeErr)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_BindScopeError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake bind scope error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, fakeErr)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, nil)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}
func TestCreatePod_SetStateError(t *testing.T) {
store, ip, cache, op := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Setup mocks
fakeErr := fmt.Errorf("fake set state error")
ip.On("CreateHandle", op).Return(podID, podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "busybox-container", "", "", "").Return(podHandle, nil)
ip.On("AddImageToHandle", op, podHandle, "alpine-container", "", "", "").Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, podID, "", busyboxIsoConfig).Return(podHandle, nil)
ip.On("CreateHandleTask", op, podHandle, "Container-1-task", "", alpineIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, busyboxIsoConfig).Return(podHandle, nil)
ip.On("AddHandleToScope", op, podHandle, alpineIsoConfig).Return(podHandle, nil)
ip.On("AddInteractionToHandle", op, podHandle).Return(podHandle, nil)
ip.On("AddLoggingToHandle", op, podHandle).Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("BindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "RUNNING").Return(podHandle, fakeErr)
store.On("Get", op, "busybox", "", true).Return(&metadata.ImageConfig{}, nil)
store.On("Get", op, "alpine", "", true).Return(&metadata.ImageConfig{}, nil)
// The test
c, err := NewPodCreator(client, store, ip, cache, persona, portlayer)
assert.Check(t, c != nil, "Expected not-nil creating a pod creator but received nil")
err = c.CreatePod(op, &pod, true)
assert.Check(t, err != nil, "Expected nil error from createPod")
assert.Check(t, is.Equal(err.Error(), fakeErr.Error()))
}

174
providers/vic/operations/pod_deleter.go
View File

@@ -1,174 +0,0 @@
package operations
import (
"fmt"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/cache"
vicpod "github.com/virtual-kubelet/virtual-kubelet/providers/vic/pod"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
vicerrors "github.com/vmware/vic/lib/apiservers/engine/errors"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/pkg/retry"
"github.com/vmware/vic/pkg/trace"
"k8s.io/api/core/v1"
)
type PodDeleter interface {
DeletePod(op trace.Operation, pod *v1.Pod) error
}
type VicPodDeleter struct {
client *client.PortLayer
imageStore proxy.ImageStore
isolationProxy proxy.IsolationProxy
podCache cache.PodCache
personaAddr string
portlayerAddr string
}
type VicPodDeleterError string
func (e VicPodDeleterError) Error() string { return string(e) }
const (
PodDeleterPortlayerClientError = VicPodDeleterError("PodDeleter called with an invalid portlayer client")
PodDeleterIsolationProxyError = VicPodDeleterError("PodDeleter called with an invalid isolation proxy")
PodDeleterPodCacheError = VicPodDeleterError("PodDeleter called with an invalid pod cache")
PodDeleterPersonaAddrError = VicPodDeleterError("PodDeleter called with an invalid VIC persona addr")
PodDeleterPortlayerAddrError = VicPodDeleterError("PodDeleter called with an invalid VIC portlayer addr")
PodDeleterInvalidPodSpecError = VicPodDeleterError("PodDeleter called with nil pod")
)
type DeleteResponse struct {
Id string `json:"Id"`
Warnings string `json:"Warnings"`
}
func NewPodDeleter(client *client.PortLayer, isolationProxy proxy.IsolationProxy, podCache cache.PodCache, personaAddr string, portlayerAddr string) (PodDeleter, error) {
if client == nil {
return nil, PodDeleterPortlayerClientError
} else if isolationProxy == nil {
return nil, PodDeleterIsolationProxyError
} else if podCache == nil {
return nil, PodDeleterPodCacheError
}
return &VicPodDeleter{
client: client,
podCache: podCache,
personaAddr: personaAddr,
portlayerAddr: portlayerAddr,
isolationProxy: isolationProxy,
}, nil
}
// DeletePod deletes a pod
//
// arguments:
// op operation trace logger
// pod pod spec
// returns:
// error
func (v *VicPodDeleter) DeletePod(op trace.Operation, pod *v1.Pod) error {
if pod == nil {
return PodDeleterInvalidPodSpecError
}
defer trace.End(trace.Begin(pod.Name, op))
// Get pod from cache
vp, err := v.podCache.Get(op, "", pod.Name)
if err != nil {
return err
}
// Stop pod if not already stopped
// Transform kube container config to docker create config
err = v.deletePod(op, vp, true)
if err != nil {
op.Errorf("PodDeleter failed to delete pod: %s", err.Error())
return err
}
op.Infof("PodDeleter deleting from cache, name: %s, ID: %s", pod.Name, vp.ID)
v.podCache.Delete(op, "", pod.Name)
return nil
}
// deletePod deletes a pod
//
// arguments:
// op operation trace logger
// vp VIC pod struct
// force if set to true, the pod will be deleted even if it's still running
// returns:
// error
func (v *VicPodDeleter) deletePod(op trace.Operation, vp *vicpod.VicPod, force bool) error {
defer trace.End(trace.Begin("", op))
if vp == nil {
return PodDeleterInvalidPodSpecError
}
id := vp.ID
name := vp.Pod.Name
running := false
stopper, err := NewPodStopper(v.client, v.isolationProxy)
if err != nil {
return err
}
// Use the force and stop the container first
if force {
if err := stopper.Stop(op, id, name); err != nil {
return err
}
} else {
state, err := v.isolationProxy.State(op, id, name)
if err != nil {
return err
}
switch state {
case "Error":
// force stop if container state is error to make sure container is deletable later
stopper.Stop(op, id, name)
case "Starting":
// if we are starting let the user know they must use the force
return fmt.Errorf("The container is starting. To remove use -f")
case "Running":
running = true
}
handle, err := v.isolationProxy.Handle(op, id, name)
if err != nil {
return err
}
// Unbind the container to the scope
_, ep, err := v.isolationProxy.UnbindScope(op, handle, name)
if err != nil {
return err
}
op.Infof("Scope Unbind returned endpoints %# +v", ep)
}
// Retry remove operation if container is not in running state. If in running state, we only try
// once to prevent retries from degrading performance.
if !running {
operation := func() error {
return v.isolationProxy.Remove(op, id, true)
}
op.Infof("Delete Pod, ID: %s, running: %v", vp.ID, running)
return retry.Do(operation, vicerrors.IsConflictError)
}
err = v.isolationProxy.Remove(op, id, true)
op.Infof("Delete Pod, ID: %s, running: %v err: %v", vp.ID, running, err)
return err
}

212
providers/vic/operations/pod_deleter_test.go
View File

@@ -1,212 +0,0 @@
package operations
import (
"testing"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
)
func init() {
initPod()
}
func TestNewPodDeleter(t *testing.T) {
_, ip, cache, _ := createMocks(t)
client := client.Default
persona := "1.2.3.4"
portlayer := "1.2.3.4"
// Positive Cases
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
// Negative Cases
d, err = NewPodDeleter(nil, ip, cache, persona, portlayer)
assert.Check(t, is.Nil(d), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodDeleterPortlayerClientError))
d, err = NewPodDeleter(client, nil, cache, persona, portlayer)
assert.Check(t, is.Nil(d), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodDeleterIsolationProxyError))
d, err = NewPodDeleter(client, ip, nil, persona, portlayer)
assert.Check(t, is.Nil(d), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodDeleterPodCacheError))
}
func TestDeletePod(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Remove", op, podID, true).Return(nil)
// Add vicPod to the cache
cache.Add(op, "", pod.Name, &vicPod)
// Positive case
err = d.DeletePod(op, &pod)
assert.Check(t, err, "Expected nil")
}
func TestDeletePodErrorHandle(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Remove", op, podID, true).Return(nil)
// Add vicPod to the cache
cache.Add(op, "", pod.Name, &vicPod)
// Failed Handle
fakeErr := fakeError("invalid handle")
ip.On("Handle", op, podID, podName).Return("", fakeErr)
err = d.DeletePod(op, &pod)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected invalid handle error")
}
func TestDeletePodErrorUnbindScope(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Remove", op, podID, true).Return(nil)
// Add vicPod to the cache
cache.Add(op, "", pod.Name, &vicPod)
// Failed UnbindScope
fakeErr := fakeError("failed UnbindScope")
ip.On("UnbindScope", op, podHandle, podName).Return("", nil, fakeErr)
err = d.DeletePod(op, &pod)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed UnbindScope error")
}
func TestDeletePodErrorSetState(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
ip.On("Remove", op, podID, true).Return(nil)
// Add vicPod to the cache
cache.Add(op, "", pod.Name, &vicPod)
// Failed SetState
fakeErr := fakeError("failed SetState")
ip.On("SetState", op, podHandle, podName, "STOPPED").Return("", fakeErr)
err = d.DeletePod(op, &pod)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed SetState error")
}
func TestDeletePodErrorCommitHandle(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("Remove", op, podID, true).Return(nil)
// Add vicPod to the cache
cache.Add(op, "", pod.Name, &vicPod)
// Failed Commit
fakeErr := fakeError("failed Commit")
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(fakeErr)
err = d.DeletePod(op, &pod)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed Commit error")
}
func TestDeletePodErrorRemove(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
// Add vicPod to the cache
cache.Add(op, "", pod.Name, &vicPod)
// Failed Remove
fakeErr := fakeError("failed Remove")
ip.On("Remove", op, podID, true).Return(fakeErr)
err = d.DeletePod(op, &pod)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed Remove error")
}
func TestDeletePodErrorBadArgs(t *testing.T) {
client := client.Default
_, ip, cache, op := createMocks(t)
persona := "1.2.3.4"
portlayer := "1.2.3.4"
d, err := NewPodDeleter(client, ip, cache, persona, portlayer)
assert.Check(t, d != nil, "Expected non-nil creating a pod Deleter but received nil")
// Negative Cases
err = d.DeletePod(op, nil)
assert.Check(t, is.DeepEqual(err, PodDeleterInvalidPodSpecError))
}

91
providers/vic/operations/pod_starter.go
View File

@@ -1,91 +0,0 @@
package operations
import (
"context"
"fmt"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/pkg/trace"
)
type PodStarter interface {
Start(op trace.Operation, id, name string) error
}
type VicPodStarter struct {
client *client.PortLayer
isolationProxy proxy.IsolationProxy
imageStore proxy.ImageStore
}
type VicPodStarterError string
func (e VicPodStarterError) Error() string { return string(e) }
const (
PodStarterPortlayerClientError = VicPodStarterError("PodStarter called with an invalid portlayer client")
PodStarterIsolationProxyError = VicPodStarterError("PodStarter called with an invalid isolation proxy")
PodStarterInvalidPodIDError = VicPodStarterError("PodStarter called with invalid Pod ID")
PodStarterInvalidPodNameError = VicPodStarterError("PodStarter called with invalid Pod name")
)
func NewPodStarter(client *client.PortLayer, isolationProxy proxy.IsolationProxy) (PodStarter, error) {
defer trace.End(trace.Begin("", context.Background()))
if client == nil {
return nil, PodStarterPortlayerClientError
}
if isolationProxy == nil {
return nil, PodStarterIsolationProxyError
}
return &VicPodStarter{
client: client,
isolationProxy: isolationProxy,
}, nil
}
// Start starts up the pod vm
//
// arguments:
// op operation trace logger
// id pod id
// name pod name
// returns:
// error
func (v *VicPodStarter) Start(op trace.Operation, id, name string) error {
defer trace.End(trace.Begin(fmt.Sprintf("id(%s), name(%s)", id, name), op))
h, err := v.isolationProxy.Handle(op, id, name)
if err != nil {
return err
}
// Bind the container to the scope
h, ep, err := v.isolationProxy.BindScope(op, h, name)
if err != nil {
return err
}
op.Debugf("*** Scope bind returned endpoints %#v", ep)
defer func() {
if err != nil {
op.Debugf("Unbinding %s due to error - %s", id, err.Error())
v.isolationProxy.UnbindScope(op, h, name)
}
}()
h, err = v.isolationProxy.SetState(op, h, name, "RUNNING")
if err != nil {
return err
}
// map ports
err = v.isolationProxy.CommitHandle(op, h, id, -1)
return nil
}

33
providers/vic/operations/pod_starter_test.go
View File

@@ -1,33 +0,0 @@
package operations
import (
"testing"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy/mocks"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
)
func TestNewPodStarter(t *testing.T) {
var s PodStarter
var err error
client := client.Default
ip := &mocks.IsolationProxy{}
// Positive Cases
s, err = NewPodStarter(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod starter but received nil")
// Negative Cases
s, err = NewPodStarter(nil, ip)
assert.Check(t, is.Nil(s), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodStarterPortlayerClientError))
s, err = NewPodStarter(client, nil)
assert.Check(t, is.Nil(s), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodStarterIsolationProxyError))
}
//NOTE: The rest of PodStarter tests were handled in PodCreator's tests so there's no need for further tests.

151
providers/vic/operations/pod_status.go
View File

@@ -1,151 +0,0 @@
package operations
import (
"fmt"
"net"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/pkg/trace"
"k8s.io/api/core/v1"
)
type PodStatus interface {
GetStatus(op trace.Operation, namespace string, name string, hostAddress string) (*v1.PodStatus, error)
}
type VicPodStatus struct {
client *client.PortLayer
isolationProxy proxy.IsolationProxy
}
type VicPodStatusError string
func (e VicPodStatusError) Error() string { return string(e) }
const (
PodStatusPortlayerClientError = VicPodStatusError("PodStatus called with an invalid portlayer client")
PodStatusIsolationProxyError = VicPodStatusError("PodStatus called with an invalid isolation proxy")
PodStatusInvalidPodIDError = VicPodStatusError("PodStatus called with invalid PodID")
PodStatusInvalidPodNameError = VicPodStatusError("PodStatus called with invalid PodName")
)
func NewPodStatus(client *client.PortLayer, isolationProxy proxy.IsolationProxy) (PodStatus, error) {
if client == nil {
return nil, PodStatusPortlayerClientError
} else if isolationProxy == nil {
return nil, PodStatusIsolationProxyError
}
return &VicPodStatus{
client: client,
isolationProxy: isolationProxy,
}, nil
}
// Gets pod status does not delete it
//
// arguments:
// op operation trace logger
// id pod id
// name pod name
// returns:
// error
func (v *VicPodStatus) GetStatus(op trace.Operation, id, name string, hostAddress string) (*v1.PodStatus, error) {
defer trace.End(trace.Begin(fmt.Sprintf("id(%s), name(%s)", id, name), op))
return v.getStatus(op, id, name, hostAddress)
}
func (v *VicPodStatus) getStatus(op trace.Operation, id, name string, hostAddress string) (*v1.PodStatus, error) {
defer trace.End(trace.Begin(fmt.Sprintf("id(%s), name(%s)", id, name), op))
// Start out with unknown
phase := v1.PodUnknown
podReady := v1.ConditionUnknown
podInitialized := v1.ConditionUnknown
podScheduled := v1.ConditionUnknown
// Get current state
state, err := v.isolationProxy.State(op, id, name)
if err == nil {
podScheduled = v1.ConditionTrue
switch state {
case "Starting":
// if we are starting let the user know they must use the force
phase = v1.PodPending
podInitialized = v1.ConditionFalse
podReady = v1.ConditionFalse
case "Running":
phase = v1.PodRunning
podInitialized = v1.ConditionTrue
podReady = v1.ConditionTrue
case "Stopping":
phase = v1.PodRunning
podReady = v1.ConditionFalse
podInitialized = v1.ConditionTrue
case "Stopped":
phase = v1.PodSucceeded
podReady = v1.ConditionFalse
podInitialized = v1.ConditionTrue
case "Removing":
phase = v1.PodSucceeded
podReady = v1.ConditionFalse
podInitialized = v1.ConditionTrue
case "Removed":
phase = v1.PodSucceeded
podReady = v1.ConditionFalse
podInitialized = v1.ConditionTrue
}
}
status := &v1.PodStatus{
Phase: phase,
Conditions: []v1.PodCondition{
{
Type: v1.PodInitialized,
Status: podInitialized,
},
{
Type: v1.PodReady,
Status: podReady,
},
{
Type: v1.PodScheduled,
Status: podScheduled,
},
},
}
addresses, err := v.getIPAddresses(op, id, name)
if err == nil && len(addresses) > 0 {
status.HostIP = hostAddress
status.PodIP = addresses[0]
} else {
status.HostIP = "0.0.0.0"
status.PodIP = "0.0.0.0"
}
return status, nil
}
func (v *VicPodStatus) getIPAddresses(op trace.Operation, id, name string) ([]string, error) {
defer trace.End(trace.Begin(id, op))
apAddresses, err := v.isolationProxy.EpAddresses(op, id, name)
if err != nil {
return nil, err
}
IPAddresses := make([]string, 0)
for _, epAddr := range apAddresses {
if epAddr != "" {
ip, _, err := net.ParseCIDR(epAddr)
if err == nil {
IPAddresses = append(IPAddresses, ip.String())
}
}
}
return IPAddresses, err
}

225
providers/vic/operations/pod_status_test.go
View File

@@ -1,225 +0,0 @@
package operations
import (
"testing"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
v1 "k8s.io/api/core/v1"
)
func TestNewPodStatus(t *testing.T) {
_, ip, _, _ := createMocks(t)
client := client.Default
// Positive Cases
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
// Negative Cases
s, err = NewPodStatus(nil, ip)
assert.Check(t, is.Nil(s), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodStatusPortlayerClientError))
s, err = NewPodStatus(client, nil)
assert.Check(t, is.Nil(s), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodStatusIsolationProxyError))
}
func TestStatusPodStarting(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "1.2.3.4"
EndpointAddresses := []string{
"5.6.7.8/24",
}
// Set up the mocks for this test
ip.On("State", op, podID, podName).Return(stateStarting, nil)
ip.On("EpAddresses", op, podID, podName).Return(EndpointAddresses, nil)
// Positive case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, err, "Expected nil")
assert.Check(t, is.Equal(status.Phase, v1.PodPending), "Expected Phase Pending")
verifyConditions(t, status.Conditions, v1.ConditionTrue, v1.ConditionFalse, v1.ConditionFalse)
assert.Check(t, is.Equal(status.HostIP, "1.2.3.4"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "5.6.7.8"), "Expected Pod IP Address")
}
func TestStatusPodRunning(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "1.2.3.4"
EndpointAddresses := []string{
"5.6.7.8/24",
}
// Set up the mocks for this test
ip.On("State", op, podID, podName).Return(stateRunning, nil)
ip.On("EpAddresses", op, podID, podName).Return(EndpointAddresses, nil)
// Pod Running case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, err, "Expected nil")
assert.Check(t, is.Equal(status.Phase, v1.PodRunning), "Expected Phase PodRunning")
verifyConditions(t, status.Conditions, v1.ConditionTrue, v1.ConditionTrue, v1.ConditionTrue)
assert.Check(t, is.Equal(status.HostIP, "1.2.3.4"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "5.6.7.8"), "Expected Pod IP Address")
}
func TestStatusPodStopping(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "1.2.3.4"
EndpointAddresses := []string{
"5.6.7.8/24",
}
// Set up the mocks for this test
ip.On("State", op, podID, podName).Return(stateStopping, nil)
ip.On("EpAddresses", op, podID, podName).Return(EndpointAddresses, nil)
// Pod error case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, is.Equal(status.Phase, v1.PodRunning), "Expected Phase PodFailed")
verifyConditions(t, status.Conditions, v1.ConditionTrue, v1.ConditionTrue, v1.ConditionFalse)
assert.Check(t, is.Equal(status.HostIP, "1.2.3.4"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "5.6.7.8"), "Expected Pod IP Address")
}
func TestStatusPodStopped(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "1.2.3.4"
EndpointAddresses := []string{
"5.6.7.8/24",
}
// Set up the mocks for this test
ip.On("State", op, podID, podName).Return(stateStopped, nil)
ip.On("EpAddresses", op, podID, podName).Return(EndpointAddresses, nil)
// Pod error case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, is.Equal(status.Phase, v1.PodSucceeded), "Expected Phase PodFailed")
verifyConditions(t, status.Conditions, v1.ConditionTrue, v1.ConditionTrue, v1.ConditionFalse)
assert.Check(t, is.Equal(status.HostIP, "1.2.3.4"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "5.6.7.8"), "Expected Pod IP Address")
}
func TestStatusPodRemoving(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "1.2.3.4"
EndpointAddresses := []string{
"5.6.7.8/24",
}
// Set up the mocks for this test
ip.On("State", op, podID, podName).Return(stateRemoving, nil)
ip.On("EpAddresses", op, podID, podName).Return(EndpointAddresses, nil)
// Pod error case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, is.Equal(status.Phase, v1.PodSucceeded), "Expected Phase PodFailed")
verifyConditions(t, status.Conditions, v1.ConditionTrue, v1.ConditionTrue, v1.ConditionFalse)
assert.Check(t, is.Equal(status.HostIP, "1.2.3.4"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "5.6.7.8"), "Expected Pod IP Address")
}
func TestStatusPodRemoved(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "1.2.3.4"
EndpointAddresses := []string{
"5.6.7.8/24",
}
// Set up the mocks for this test
ip.On("State", op, podID, podName).Return(stateRemoved, nil)
ip.On("EpAddresses", op, podID, podName).Return(EndpointAddresses, nil)
// Pod error case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, is.Equal(status.Phase, v1.PodSucceeded), "Expected Phase PodFailed")
verifyConditions(t, status.Conditions, v1.ConditionTrue, v1.ConditionTrue, v1.ConditionFalse)
assert.Check(t, is.Equal(status.HostIP, "1.2.3.4"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "5.6.7.8"), "Expected Pod IP Address")
}
func TestStatusError(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
// Start with arguments
s, err := NewPodStatus(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Status but received nil")
assert.Check(t, err, "Expected nil")
HostAddress := "0.0.0.0"
// Set up the mocks for this test
fakeErr := fakeError("invalid Pod")
ip.On("State", op, podID, podName).Return("", fakeErr)
ip.On("EpAddresses", op, podID, podName).Return(nil, fakeErr)
// Error case
status, err := s.GetStatus(op, podID, podName, HostAddress)
assert.Check(t, err, "Expected nil")
assert.Check(t, is.Equal(status.Phase, v1.PodUnknown), "Expected Phase PodUnknown")
verifyConditions(t, status.Conditions, v1.ConditionUnknown, v1.ConditionUnknown, v1.ConditionUnknown)
assert.Check(t, is.Equal(status.HostIP, "0.0.0.0"), "Expected Host IP Address")
assert.Check(t, is.Equal(status.PodIP, "0.0.0.0"), "Expected Pod IP Address")
}
func verifyConditions(t *testing.T, conditions []v1.PodCondition, scheduled v1.ConditionStatus, initialized v1.ConditionStatus, ready v1.ConditionStatus) {
for _, condition := range conditions {
switch condition.Type {
case v1.PodScheduled:
assert.Check(t, is.Equal(condition.Status, scheduled), "Condition Pod Scheduled")
break
case v1.PodInitialized:
assert.Check(t, is.Equal(condition.Status, initialized), "Condition Pod Initialized")
break
case v1.PodReady:
assert.Check(t, is.Equal(condition.Status, ready), "Condition Pod Ready")
break
}
}
}

95
providers/vic/operations/pod_stopper.go
View File

@@ -1,95 +0,0 @@
package operations
import (
"fmt"
"time"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
vicerrors "github.com/vmware/vic/lib/apiservers/engine/errors"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/pkg/retry"
"github.com/vmware/vic/pkg/trace"
)
type PodStopper interface {
Stop(op trace.Operation, id, name string) error
}
type VicPodStopper struct {
client *client.PortLayer
isolationProxy proxy.IsolationProxy
}
type VicPodStopperError string
func (e VicPodStopperError) Error() string { return string(e) }
const (
PodStopperPortlayerClientError = VicPodStopperError("PodStopper called with an invalid portlayer client")
PodStopperIsolationProxyError = VicPodStopperError("PodStopper called with an invalid isolation proxy")
PodStopperInvalidPodIDError = VicPodStopperError("PodStopper called with invalid PodID")
PodStopperInvalidPodNameError = VicPodStopperError("PodStopper called with invalid PodName")
)
func NewPodStopper(client *client.PortLayer, isolationProxy proxy.IsolationProxy) (PodStopper, error) {
if client == nil {
return nil, PodStopperPortlayerClientError
} else if isolationProxy == nil {
return nil, PodStopperIsolationProxyError
}
return &VicPodStopper{
client: client,
isolationProxy: isolationProxy,
}, nil
}
// Stop stops a pod but does not delete it
//
// arguments:
// op operation trace logger
// id pod id
// name pod name
// returns:
// error
func (v *VicPodStopper) Stop(op trace.Operation, id, name string) error {
defer trace.End(trace.Begin(fmt.Sprintf("id(%s), name(%s)", id, name), op))
operation := func() error {
return v.stop(op, id, name)
}
config := retry.NewBackoffConfig()
config.MaxElapsedTime = 10 * time.Minute
if err := retry.DoWithConfig(operation, vicerrors.IsConflictError, config); err != nil {
return err
}
return nil
}
func (v *VicPodStopper) stop(op trace.Operation, id, name string) error {
defer trace.End(trace.Begin(fmt.Sprintf("id(%s), name(%s)", id, name), op))
h, err := v.isolationProxy.Handle(op, id, name)
if err != nil {
return err
}
// Unbind the container to the scope
h, ep, err := v.isolationProxy.UnbindScope(op, h, name)
if err != nil {
return err
}
op.Infof("Scope Unbind returned endpoints %# +v", ep)
h, err = v.isolationProxy.SetState(op, h, name, "STOPPED")
if err != nil {
return err
}
err = v.isolationProxy.CommitHandle(op, h, id, -1)
op.Infof("Commit handler returned %v", err)
return err
}

133
providers/vic/operations/pod_stopper_test.go
View File

@@ -1,133 +0,0 @@
package operations
import (
"testing"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"gotest.tools/assert"
is "gotest.tools/assert/cmp"
)
func TestNewPodStopper(t *testing.T) {
_, ip, _, _ := createMocks(t)
client := client.Default
// Positive Cases
s, err := NewPodStopper(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Stopper but received nil")
// Negative Cases
s, err = NewPodStopper(nil, ip)
assert.Check(t, is.Nil(s), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodStopperPortlayerClientError))
s, err = NewPodStopper(client, nil)
assert.Check(t, is.Nil(s), "Expected nil")
assert.Check(t, is.DeepEqual(err, PodStopperIsolationProxyError))
}
func TestStopPod(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
// Start with arguments
s, err := NewPodStopper(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Stopper but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
// Positive case
err = s.Stop(op, podID, podName)
assert.Check(t, err, "Expected nil")
}
func TestStopPodErrorHandle(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
// Start with arguments
s, err := NewPodStopper(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Stopper but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
// Failed Handle
fakeErr := fakeError("invalid handle")
ip.On("Handle", op, podID, podName).Return("", fakeErr)
err = s.Stop(op, podID, podName)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected invalid handle error")
}
func TestStopPodErrorUnbindScope(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
// Start with arguments
s, err := NewPodStopper(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Stopper but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
// Failed UnbindScope
fakeErr := fakeError("failed UnbindScope")
ip.On("UnbindScope", op, podHandle, podName).Return("", nil, fakeErr)
err = s.Stop(op, podID, podName)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed UnbindScope error")
}
func TestStopPodErrorSetState(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
// Start with arguments
s, err := NewPodStopper(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Stopper but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(nil)
// Failed SetState
fakeErr := fakeError("failed SetState")
ip.On("SetState", op, podHandle, podName, "STOPPED").Return("", fakeErr)
err = s.Stop(op, podID, podName)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed SetState error")
}
func TestStopPodErrorCommit(t *testing.T) {
client := client.Default
_, ip, _, op := createMocks(t)
// Start with arguments
s, err := NewPodStopper(client, ip)
assert.Check(t, s != nil, "Expected non-nil creating a pod Stopper but received nil")
assert.Check(t, err, "Expected nil")
// Set up the mocks for this test
ip.On("Handle", op, podID, podName).Return(podHandle, nil)
ip.On("UnbindScope", op, podHandle, podName).Return(podHandle, fakeEP, nil)
ip.On("SetState", op, podHandle, podName, "STOPPED").Return(podHandle, nil)
// Failed Commit
fakeErr := fakeError("failed Commit")
ip.On("CommitHandle", op, podHandle, podID, int32(-1)).Return(fakeErr)
err = s.Stop(op, podID, podName)
assert.Check(t, is.Error(err, fakeErr.Error()), "Expected failed Commit error")
}

10
providers/vic/pod/vicpod.go
View File

@@ -1,10 +0,0 @@
package pod
import (
"k8s.io/api/core/v1"
)
type VicPod struct {
ID string
Pod *v1.Pod
}

BIN
providers/vic/podvm_basic_design.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.5 KiB

167
providers/vic/proxy/image_store.go
View File

@@ -1,167 +0,0 @@
package proxy
import (
"bytes"
"fmt"
"io/ioutil"
"net/http"
"time"
"github.com/vmware/vic/lib/apiservers/engine/backends/cache"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/lib/metadata"
"github.com/vmware/vic/pkg/trace"
)
//TODO: This image cache needs to hook into the VIC persona to receive an event when users pulled images
// via docker.
type ImageStore interface {
Get(op trace.Operation, idOrRef, tag string, actuate bool) (*metadata.ImageConfig, error)
GetImages(op trace.Operation) []*metadata.ImageConfig
PullImage(op trace.Operation, image, tag, username, password string) error
}
type VicImageStore struct {
client *client.PortLayer
personaAddr string
portlayerAddr string
}
type ImageStoreError string
func (e ImageStoreError) Error() string { return string(e) }
const (
ImageStorePortlayerClientError = ImageStoreError("ImageStore cannot be created without a valid portlayer client")
ImageStorePersonaAddrError = ImageStoreError("ImageStore cannot be created without a valid VIC persona addr")
ImageStorePortlayerAddrError = ImageStoreError("ImageStore cannot be created without a valid VIC portlayer addr")
ImageStoreContainerIDError = ImageStoreError("ImageStore called with empty container ID")
ImageStoreEmptyUserNameError = ImageStoreError("ImageStore called with empty username")
ImageStoreEmptyPasswordError = ImageStoreError("ImageStore called with empty password")
)
func NewImageStore(plClient *client.PortLayer, personaAddr, portlayerAddr string) (ImageStore, error) {
if plClient == nil {
return nil, ImageStorePortlayerClientError
}
if personaAddr == "" {
return nil, ImageStorePersonaAddrError
}
if portlayerAddr == "" {
return nil, ImageStorePortlayerAddrError
}
err := cache.InitializeImageCache(plClient)
if err != nil {
return nil, err
}
vs := &VicImageStore{
client: plClient,
personaAddr: personaAddr,
portlayerAddr: portlayerAddr,
}
return vs, nil
}
// Get retrieves the VIC ImageConfig data structure. If the config is not cached,
// VicImageStore can request imagec to pull the image if actuate is set to true.
//
// arguments:
// op operation trace logger
// idOrRef docker image id or reference
// tag docker image tag
// realize determines whether the image is pulled if not in the cache
// returns:
// error
func (v *VicImageStore) Get(op trace.Operation, idOrRef, tag string, realize bool) (*metadata.ImageConfig, error) {
defer trace.End(trace.Begin(fmt.Sprintf("Get - %s:%s", idOrRef, tag), op))
if idOrRef == "" {
op.Errorf(ImageStoreContainerIDError.Error())
return nil, ImageStoreContainerIDError
}
c, err := cache.ImageCache().Get(idOrRef)
if err != nil && realize {
err = v.PullImage(op, idOrRef, tag, "", "")
if err == nil {
//TODO: Find a better way to get update imageconfig instead of this hammer
err := cache.InitializeImageCache(v.client)
if err != nil {
return nil, err
}
c, err = cache.ImageCache().Get(idOrRef)
if err != nil {
return nil, err
}
}
}
return c, nil
}
// Get retrieves all the VIC ImageConfig data structure.
//
// arguments:
// op operation trace logger
// returns:
// array of ImageConfig
func (v *VicImageStore) GetImages(op trace.Operation) []*metadata.ImageConfig {
defer trace.End(trace.Begin("", op))
return cache.ImageCache().GetImages()
}
// PullImage makes a request to the VIC persona server (imageC component) to retrieve a container image.
//
// arguments:
// op operation trace logger
// idOrRef docker image id or reference
// tag docker image tag
// username user name for the registry server
// password password for the registry server
// returns:
// array of ImageConfig
func (v *VicImageStore) PullImage(op trace.Operation, image, tag, username, password string) error {
defer trace.End(trace.Begin(fmt.Sprintf("Get - %s:%s", image, tag), op))
if image == "" {
op.Errorf(ImageStoreContainerIDError.Error())
return ImageStoreContainerIDError
}
pullClient := &http.Client{Timeout: 60 * time.Second}
var personaServer string
if tag == "" {
personaServer = fmt.Sprintf("http://%s/v1.35/images/create?fromImage=%s", v.personaAddr, image)
} else {
personaServer = fmt.Sprintf("http://%s/v1.35/images/create?fromImage=%s&tag=%s", v.personaAddr, image, tag)
}
op.Infof("POST %s", personaServer)
reader := bytes.NewBuffer([]byte(""))
resp, err := pullClient.Post(personaServer, "application/json", reader)
if err != nil {
op.Errorf("Error from docker pull: error = %s", err.Error())
return err
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
msg := fmt.Sprintf("Error from docker pull: status = %d", resp.StatusCode)
op.Errorf(msg)
return fmt.Errorf(msg)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
msg := fmt.Sprintf("Error reading docker pull response: error = %s", err.Error())
op.Errorf(msg)
return fmt.Errorf(msg)
}
op.Infof("Response from docker pull: body = %s", string(body))
return nil
}

619
providers/vic/proxy/isolation_proxy.go
View File

@@ -1,619 +0,0 @@
package proxy
import (
"fmt"
"time"
vicerrors "github.com/vmware/vic/lib/apiservers/engine/errors"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/lib/apiservers/portlayer/client/containers"
"github.com/vmware/vic/lib/apiservers/portlayer/client/interaction"
"github.com/vmware/vic/lib/apiservers/portlayer/client/logging"
"github.com/vmware/vic/lib/apiservers/portlayer/client/scopes"
"github.com/vmware/vic/lib/apiservers/portlayer/client/storage"
"github.com/vmware/vic/lib/apiservers/portlayer/client/tasks"
"github.com/vmware/vic/lib/apiservers/portlayer/models"
"github.com/vmware/vic/pkg/trace"
"github.com/vmware/vic/pkg/vsphere/sys"
"github.com/docker/docker/api/types/strslice"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/cache"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/constants"
)
type IsolationProxy interface {
CreateHandle(op trace.Operation) (string, string, error)
AddImageToHandle(op trace.Operation, handle, deltaID, layerID, imageID, imageName string) (string, error)
CreateHandleTask(op trace.Operation, handle, id, layerID string, config IsolationContainerConfig) (string, error)
AddHandleToScope(op trace.Operation, handle string, config IsolationContainerConfig) (string, error)
AddInteractionToHandle(op trace.Operation, handle string) (string, error)
AddLoggingToHandle(op trace.Operation, handle string) (string, error)
CommitHandle(op trace.Operation, handle, containerID string, waitTime int32) error
SetState(op trace.Operation, handle, name, state string) (string, error)
BindScope(op trace.Operation, handle string, name string) (string, interface{}, error)
UnbindScope(op trace.Operation, handle string, name string) (string, interface{}, error)
Handle(op trace.Operation, id, name string) (string, error)
Remove(op trace.Operation, id string, force bool) error
State(op trace.Operation, id, name string) (string, error)
EpAddresses(op trace.Operation, id, name string) ([]string, error)
}
type VicIsolationProxy struct {
client *client.PortLayer
imageStore ImageStore
podCache cache.PodCache
portlayerAddr string
hostUUID string
}
type PortBinding struct {
HostIP string
HostPort string
}
type IsolationContainerConfig struct {
ID string
ImageID string
LayerID string
ImageName string
Name string
Namespace string
Cmd []string
Path string
Entrypoint []string
//Args []string
Env []string
WorkingDir string
User string
StopSignal string
Attach bool
StdinOnce bool
OpenStdin bool
Tty bool
CPUCount int64
Memory int64
PortMap map[string]PortBinding
}
func NewIsolationProxy(plClient *client.PortLayer, portlayerAddr string, hostUUID string, imageStore ImageStore, podCache cache.PodCache) IsolationProxy {
if plClient == nil {
return nil
}
return &VicIsolationProxy{
client: plClient,
imageStore: imageStore,
podCache: podCache,
portlayerAddr: portlayerAddr,
hostUUID: hostUUID,
}
}
// CreateHandle creates a "manifest" that will be used by Commit() to create the actual
// isolation vm.
//
// returns:
// (container/pod id, handle, error)
func (v *VicIsolationProxy) CreateHandle(op trace.Operation) (string, string, error) {
defer trace.End(trace.Begin("CreateHandle", op))
if v.client == nil {
return "", "", vicerrors.NillPortlayerClientError("IsolationProxy")
}
// Call the Exec port layer to create the container
var err error
var hostUUID string
if v.hostUUID != "" {
hostUUID = v.hostUUID
} else {
hostUUID, err = sys.UUID()
}
if err != nil {
return "", "", vicerrors.InternalServerError("IsolationProxy.CreateContainerHandle got unexpected error getting VCH UUID")
}
plCreateParams := initIsolationConfig(op, "", constants.DummyRepoName, constants.DummyImage, constants.DummyLayerID, hostUUID)
createResults, err := v.client.Containers.Create(plCreateParams)
if err != nil {
if _, ok := err.(*containers.CreateNotFound); ok {
cerr := fmt.Errorf("No such image: %s", constants.DummyImage)
op.Errorf("%s (%s)", cerr, err)
return "", "", vicerrors.NotFoundError(cerr.Error())
}
// If we get here, most likely something went wrong with the port layer API server
return "", "", vicerrors.InternalServerError(err.Error())
}
id := createResults.Payload.ID
h := createResults.Payload.Handle
return id, h, nil
}
// Handle retrieves a handle to a VIC container. Handles should be treated as opaque strings.
//
// returns:
// (handle string, error)
func (v *VicIsolationProxy) Handle(op trace.Operation, id, name string) (string, error) {
if v.client == nil {
return "", vicerrors.NillPortlayerClientError("IsolationProxy")
}
resp, err := v.client.Containers.Get(containers.NewGetParamsWithContext(op).WithID(id))
if err != nil {
switch err := err.(type) {
case *containers.GetNotFound:
return "", vicerrors.NotFoundError(name)
case *containers.GetDefault:
return "", vicerrors.InternalServerError(err.Payload.Message)
default:
return "", vicerrors.InternalServerError(err.Error())
}
}
return resp.Payload, nil
}
func (v *VicIsolationProxy) AddImageToHandle(op trace.Operation, handle, deltaID, layerID, imageID, imageName string) (string, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", vicerrors.InternalServerError("IsolationProxy.AddImageToContainer failed to get the portlayer client")
}
var err error
var hostUUID string
if v.hostUUID != "" {
hostUUID = v.hostUUID
} else {
hostUUID, err = sys.UUID()
}
if err != nil {
return "", vicerrors.InternalServerError("IsolationProxy.AddImageToContainer got unexpected error getting VCH UUID")
}
response, err := v.client.Storage.ImageJoin(storage.NewImageJoinParamsWithContext(op).WithStoreName(hostUUID).WithID(layerID).
WithConfig(&models.ImageJoinConfig{
Handle: handle,
DeltaID: deltaID,
ImageID: imageID,
RepoName: imageName,
}))
if err != nil {
return "", vicerrors.InternalServerError(err.Error())
}
handle, ok := response.Payload.Handle.(string)
if !ok {
return "", vicerrors.InternalServerError(fmt.Sprintf("Type assertion failed for %#+v", handle))
}
return handle, nil
}
func (v *VicIsolationProxy) CreateHandleTask(op trace.Operation, handle, id, layerID string, config IsolationContainerConfig) (string, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", vicerrors.InternalServerError("IsolationProxy.CreateContainerTask failed to create a portlayer client")
}
op.Debugf("CreateHandleTask - %#v", config)
plTaskParams := IsolationContainerConfigToTask(op, id, layerID, config)
plTaskParams.Config.Handle = handle
op.Debugf("*** CreateContainerTask - params = %#v", *plTaskParams.Config)
responseJoin, err := v.client.Tasks.Join(plTaskParams)
if err != nil {
op.Errorf("Unable to join primary task to container: %+v", err)
return "", vicerrors.InternalServerError(err.Error())
}
handle, ok := responseJoin.Payload.Handle.(string)
if !ok {
return "", vicerrors.InternalServerError(fmt.Sprintf("Type assertion failed on handle from task join: %#+v", handle))
}
plBindParams := tasks.NewBindParamsWithContext(op).WithConfig(&models.TaskBindConfig{Handle: handle, ID: id})
responseBind, err := v.client.Tasks.Bind(plBindParams)
if err != nil {
op.Errorf("Unable to bind primary task to container: %+v", err)
return "", vicerrors.InternalServerError(err.Error())
}
handle, ok = responseBind.Payload.Handle.(string)
if !ok {
return "", vicerrors.InternalServerError(fmt.Sprintf("Type assertion failed on handle from task bind %#+v", handle))
}
return handle, nil
}
// AddHandleToScope adds a container, referenced by handle, to a scope.
// If an error is return, the returned handle should not be used.
//
// returns:
// modified handle
func (v *VicIsolationProxy) AddHandleToScope(op trace.Operation, handle string, config IsolationContainerConfig) (string, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", vicerrors.NillPortlayerClientError("IsolationProxy")
}
// configure network
netConf := networkConfigFromIsolationConfig(config)
if netConf != nil {
addContRes, err := v.client.Scopes.AddContainer(scopes.NewAddContainerParamsWithContext(op).
WithScope(netConf.NetworkName).
WithConfig(&models.ScopesAddContainerConfig{
Handle: handle,
NetworkConfig: netConf,
}))
if err != nil {
op.Errorf("IsolationProxy.AddContainerToScope: Scopes error: %s", err.Error())
return handle, vicerrors.InternalServerError(err.Error())
}
defer func() {
if err == nil {
return
}
// roll back the AddContainer call
if _, err2 := v.client.Scopes.RemoveContainer(scopes.NewRemoveContainerParamsWithContext(op).WithHandle(handle).WithScope(netConf.NetworkName)); err2 != nil {
op.Warnf("could not roll back container add: %s", err2)
}
}()
handle = addContRes.Payload
}
return handle, nil
}
// AddLoggingToHandle adds logging capability to the isolation vm, referenced by handle.
// If an error is return, the returned handle should not be used.
//
// returns:
// modified handle
func (v *VicIsolationProxy) AddLoggingToHandle(op trace.Operation, handle string) (string, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", vicerrors.NillPortlayerClientError("IsolationProxy")
}
response, err := v.client.Logging.LoggingJoin(logging.NewLoggingJoinParamsWithContext(op).
WithConfig(&models.LoggingJoinConfig{
Handle: handle,
}))
if err != nil {
return "", vicerrors.InternalServerError(err.Error())
}
handle, ok := response.Payload.Handle.(string)
if !ok {
return "", vicerrors.InternalServerError(fmt.Sprintf("Type assertion failed for %#+v", handle))
}
return handle, nil
}
// AddInteractionToContainer adds interaction capabilities to a container, referenced by handle.
// If an error is return, the returned handle should not be used.
//
// returns:
// modified handle
func (v *VicIsolationProxy) AddInteractionToHandle(op trace.Operation, handle string) (string, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", vicerrors.NillPortlayerClientError("IsolationProxy")
}
response, err := v.client.Interaction.InteractionJoin(interaction.NewInteractionJoinParamsWithContext(op).
WithConfig(&models.InteractionJoinConfig{
Handle: handle,
}))
if err != nil {
return "", vicerrors.InternalServerError(err.Error())
}
handle, ok := response.Payload.Handle.(string)
if !ok {
return "", vicerrors.InternalServerError(fmt.Sprintf("Type assertion failed for %#+v", handle))
}
return handle, nil
}
func (v *VicIsolationProxy) CommitHandle(op trace.Operation, handle, containerID string, waitTime int32) error {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return vicerrors.NillPortlayerClientError("IsolationProxy")
}
var commitParams *containers.CommitParams
if waitTime > 0 {
commitParams = containers.NewCommitParamsWithContext(op).WithHandle(handle).WithWaitTime(&waitTime)
} else {
commitParams = containers.NewCommitParamsWithContext(op).WithHandle(handle)
}
_, err := v.client.Containers.Commit(commitParams)
if err != nil {
switch err := err.(type) {
case *containers.CommitNotFound:
return vicerrors.NotFoundError(containerID)
case *containers.CommitConflict:
return vicerrors.ConflictError(err.Error())
case *containers.CommitDefault:
return vicerrors.InternalServerError(err.Payload.Message)
default:
return vicerrors.InternalServerError(err.Error())
}
}
return nil
}
//TODO: I don't think this function should be in here.
// BindNetwork binds the handle to the scope and returns endpoints. Caller of the function does not need
// to interpret the return value. In the event the caller want to unbind,
func (v *VicIsolationProxy) BindScope(op trace.Operation, handle string, name string) (string, interface{}, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", nil, vicerrors.NillPortlayerClientError("IsolationProxy")
}
bindParams := scopes.NewBindContainerParamsWithContext(op).WithHandle(handle)
bindRes, err := v.client.Scopes.BindContainer(bindParams)
if err != nil {
switch err := err.(type) {
case *scopes.BindContainerNotFound:
return "", nil, vicerrors.NotFoundError(name)
case *scopes.BindContainerInternalServerError:
return "", nil, vicerrors.InternalServerError(err.Payload.Message)
default:
return "", nil, vicerrors.InternalServerError(err.Error())
}
}
return bindRes.Payload.Handle, bindRes.Payload.Endpoints, nil
}
func (v *VicIsolationProxy) UnbindScope(op trace.Operation, handle string, name string) (string, interface{}, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", nil, vicerrors.NillPortlayerClientError("IsolationProxy")
}
unbindParams := scopes.NewUnbindContainerParamsWithContext(op).WithHandle(handle)
resp, err := v.client.Scopes.UnbindContainer(unbindParams)
if err != nil {
switch err := err.(type) {
case *scopes.UnbindContainerNotFound:
return "", nil, vicerrors.NotFoundError(name)
case *scopes.UnbindContainerInternalServerError:
return "", nil, vicerrors.InternalServerError(err.Payload.Message)
default:
return "", nil, vicerrors.InternalServerError(err.Error())
}
}
return resp.Payload.Handle, resp.Payload.Endpoints, nil
}
// SetState adds the desire state of the isolation unit once the handle is commited.
//
// returns handle string and error
func (v *VicIsolationProxy) SetState(op trace.Operation, handle, name, state string) (string, error) {
defer trace.End(trace.Begin(handle, op))
if v.client == nil {
return "", vicerrors.NillPortlayerClientError("IsolationProxy")
}
resp, err := v.client.Containers.StateChange(containers.NewStateChangeParamsWithContext(op).WithHandle(handle).WithState(state))
if err != nil {
switch err := err.(type) {
case *containers.StateChangeNotFound:
return "", vicerrors.NotFoundError(name)
case *containers.StateChangeDefault:
return "", vicerrors.InternalServerError(err.Payload.Message)
default:
return "", vicerrors.InternalServerError(err.Error())
}
}
return resp.Payload, nil
}
func (v *VicIsolationProxy) Remove(op trace.Operation, id string, force bool) error {
defer trace.End(trace.Begin(id, op))
if v.client == nil {
return vicerrors.NillPortlayerClientError("IsolationProxy")
}
pForce := force
params := containers.NewContainerRemoveParamsWithContext(op).
WithID(id).
WithForce(&pForce).
WithTimeout(120 * time.Second)
removeOK, err := v.client.Containers.ContainerRemove(params)
op.Debugf("ContainerRemove returned %# +v", removeOK)
return err
}
func (v *VicIsolationProxy) State(op trace.Operation, id, name string) (string, error) {
defer trace.End(trace.Begin(id, op))
payload, err := v.getInfo(op, id, name)
if err != nil {
return "", err
}
state := payload.ContainerConfig.State
return state, nil
}
func (v *VicIsolationProxy) EpAddresses(op trace.Operation, id, name string) ([]string, error) {
defer trace.End(trace.Begin(id, op))
payload, err := v.getInfo(op, id, name)
if err != nil {
return nil, err
}
addresses := make([]string, 0)
for _, ep := range payload.Endpoints {
addresses = append(addresses, ep.Address)
}
return addresses, nil
}
// Private methods
func (v *VicIsolationProxy) getInfo(op trace.Operation, id, name string) (*models.ContainerInfo, error) {
defer trace.End(trace.Begin(id, op))
if v.client == nil {
return nil, vicerrors.NillPortlayerClientError("IsolationProxy")
}
results, err := v.client.Containers.GetContainerInfo(containers.NewGetContainerInfoParamsWithContext(op).WithID(id))
if err != nil {
switch err := err.(type) {
case *containers.GetContainerInfoNotFound:
return nil, vicerrors.NotFoundError(name)
case *containers.GetContainerInfoInternalServerError:
return nil, vicerrors.InternalServerError(err.Payload.Message)
default:
return nil, vicerrors.InternalServerError(fmt.Sprintf("Unknown error from port layer: %s", err))
}
}
return results.Payload, nil
}
//------------------------------------
// Utility Functions
//------------------------------------
// Convert isolation container config to portlayer task param
func IsolationContainerConfigToTask(op trace.Operation, id, layerID string, ic IsolationContainerConfig) *tasks.JoinParams {
config := &models.TaskJoinConfig{}
var path string
var args []string
// we explicitly specify the ID for the primary task so that it's the same as the containerID
config.ID = id
// Set the filesystem namespace this task expects to run in
config.Namespace = layerID
// Expand cmd into entrypoint and args
cmd := strslice.StrSlice(ic.Cmd)
if len(ic.Entrypoint) != 0 {
path, args = ic.Entrypoint[0], append(ic.Entrypoint[1:], cmd...)
} else {
path, args = cmd[0], cmd[1:]
}
// copy the path
config.Path = path
// copy the args
config.Args = make([]string, len(args))
copy(config.Args, args)
// copy the env array
config.Env = make([]string, len(ic.Env))
copy(config.Env, ic.Env)
// working dir
config.WorkingDir = ic.WorkingDir
// user
config.User = ic.User
// attach. Always set to true otherwise we cannot attach later.
// this tells portlayer container is attachable.
config.Attach = true
// openstdin
config.OpenStdin = ic.OpenStdin
// tty
config.Tty = ic.Tty
// container stop signal
config.StopSignal = ic.StopSignal
op.Debugf("dockerContainerCreateParamsToTask = %+v", config)
return tasks.NewJoinParamsWithContext(op).WithConfig(config)
}
// initIsolationConfig returns a default config used to create the isolation unit handle
func initIsolationConfig(op trace.Operation, name, repoName, imageID, layerID, imageStore string) *containers.CreateParams {
config := &models.ContainerCreateConfig{}
config.NumCpus = constants.DefaultCPUs
config.MemoryMB = constants.DefaultMemory
// Layer/vmdk to use
config.Layer = layerID
// Image ID
config.Image = imageID
// Repo Requested
config.RepoName = repoName
//copy friendly name
config.Name = name
// image store
config.ImageStore = &models.ImageStore{Name: imageStore}
// network
config.NetworkDisabled = true
// hostname
config.Hostname = constants.HostName
//// domainname - https://github.com/moby/moby/issues/27067
//config.Domainname = cc.Config.Domainname
op.Debugf("dockerContainerCreateParamsToPortlayer = %+v", config)
return containers.NewCreateParamsWithContext(op).WithCreateConfig(config)
}
func networkConfigFromIsolationConfig(config IsolationContainerConfig) *models.NetworkConfig {
nc := &models.NetworkConfig{
NetworkName: "default",
}
for key, val := range config.PortMap {
nc.Ports = append(nc.Ports, fmt.Sprintf("%s:%s", val.HostPort, key))
}
return nc
}

65
providers/vic/proxy/mocks/ImageStore.go
View File

@@ -1,65 +0,0 @@
// Code generated by mockery v1.0.0. DO NOT EDIT.
package mocks
import metadata "github.com/vmware/vic/lib/metadata"
import mock "github.com/stretchr/testify/mock"
import trace "github.com/vmware/vic/pkg/trace"
// ImageStore is an autogenerated mock type for the ImageStore type
type ImageStore struct {
mock.Mock
}
// Get provides a mock function with given fields: op, idOrRef, tag, actuate
func (_m *ImageStore) Get(op trace.Operation, idOrRef string, tag string, actuate bool) (*metadata.ImageConfig, error) {
ret := _m.Called(op, idOrRef, tag, actuate)
var r0 *metadata.ImageConfig
if rf, ok := ret.Get(0).(func(trace.Operation, string, string, bool) *metadata.ImageConfig); ok {
r0 = rf(op, idOrRef, tag, actuate)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(*metadata.ImageConfig)
}
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string, bool) error); ok {
r1 = rf(op, idOrRef, tag, actuate)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// GetImages provides a mock function with given fields: op
func (_m *ImageStore) GetImages(op trace.Operation) []*metadata.ImageConfig {
ret := _m.Called(op)
var r0 []*metadata.ImageConfig
if rf, ok := ret.Get(0).(func(trace.Operation) []*metadata.ImageConfig); ok {
r0 = rf(op)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]*metadata.ImageConfig)
}
}
return r0
}
// PullImage provides a mock function with given fields: op, image, tag, username, password
func (_m *ImageStore) PullImage(op trace.Operation, image string, tag string, username string, password string) error {
ret := _m.Called(op, image, tag, username, password)
var r0 error
if rf, ok := ret.Get(0).(func(trace.Operation, string, string, string, string) error); ok {
r0 = rf(op, image, tag, username, password)
} else {
r0 = ret.Error(0)
}
return r0
}

318
providers/vic/proxy/mocks/IsolationProxy.go
View File

@@ -1,318 +0,0 @@
// Code generated by mockery v1.0.0. DO NOT EDIT.
package mocks
import mock "github.com/stretchr/testify/mock"
import proxy "github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
import trace "github.com/vmware/vic/pkg/trace"
// IsolationProxy is an autogenerated mock type for the IsolationProxy type
type IsolationProxy struct {
mock.Mock
}
// AddHandleToScope provides a mock function with given fields: op, handle, config
func (_m *IsolationProxy) AddHandleToScope(op trace.Operation, handle string, config proxy.IsolationContainerConfig) (string, error) {
ret := _m.Called(op, handle, config)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, proxy.IsolationContainerConfig) string); ok {
r0 = rf(op, handle, config)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, proxy.IsolationContainerConfig) error); ok {
r1 = rf(op, handle, config)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// AddImageToHandle provides a mock function with given fields: op, handle, deltaID, layerID, imageID, imageName
func (_m *IsolationProxy) AddImageToHandle(op trace.Operation, handle string, deltaID string, layerID string, imageID string, imageName string) (string, error) {
ret := _m.Called(op, handle, deltaID, layerID, imageID, imageName)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string, string, string, string) string); ok {
r0 = rf(op, handle, deltaID, layerID, imageID, imageName)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string, string, string, string) error); ok {
r1 = rf(op, handle, deltaID, layerID, imageID, imageName)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// AddInteractionToHandle provides a mock function with given fields: op, handle
func (_m *IsolationProxy) AddInteractionToHandle(op trace.Operation, handle string) (string, error) {
ret := _m.Called(op, handle)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string) string); ok {
r0 = rf(op, handle)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string) error); ok {
r1 = rf(op, handle)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// AddLoggingToHandle provides a mock function with given fields: op, handle
func (_m *IsolationProxy) AddLoggingToHandle(op trace.Operation, handle string) (string, error) {
ret := _m.Called(op, handle)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string) string); ok {
r0 = rf(op, handle)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string) error); ok {
r1 = rf(op, handle)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// BindScope provides a mock function with given fields: op, handle, name
func (_m *IsolationProxy) BindScope(op trace.Operation, handle string, name string) (string, interface{}, error) {
ret := _m.Called(op, handle, name)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string) string); ok {
r0 = rf(op, handle, name)
} else {
r0 = ret.Get(0).(string)
}
var r1 interface{}
if rf, ok := ret.Get(1).(func(trace.Operation, string, string) interface{}); ok {
r1 = rf(op, handle, name)
} else {
if ret.Get(1) != nil {
r1 = ret.Get(1).(interface{})
}
}
var r2 error
if rf, ok := ret.Get(2).(func(trace.Operation, string, string) error); ok {
r2 = rf(op, handle, name)
} else {
r2 = ret.Error(2)
}
return r0, r1, r2
}
// CommitHandle provides a mock function with given fields: op, handle, containerID, waitTime
func (_m *IsolationProxy) CommitHandle(op trace.Operation, handle string, containerID string, waitTime int32) error {
ret := _m.Called(op, handle, containerID, waitTime)
var r0 error
if rf, ok := ret.Get(0).(func(trace.Operation, string, string, int32) error); ok {
r0 = rf(op, handle, containerID, waitTime)
} else {
r0 = ret.Error(0)
}
return r0
}
// CreateHandle provides a mock function with given fields: op
func (_m *IsolationProxy) CreateHandle(op trace.Operation) (string, string, error) {
ret := _m.Called(op)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation) string); ok {
r0 = rf(op)
} else {
r0 = ret.Get(0).(string)
}
var r1 string
if rf, ok := ret.Get(1).(func(trace.Operation) string); ok {
r1 = rf(op)
} else {
r1 = ret.Get(1).(string)
}
var r2 error
if rf, ok := ret.Get(2).(func(trace.Operation) error); ok {
r2 = rf(op)
} else {
r2 = ret.Error(2)
}
return r0, r1, r2
}
// CreateHandleTask provides a mock function with given fields: op, handle, id, layerID, config
func (_m *IsolationProxy) CreateHandleTask(op trace.Operation, handle string, id string, layerID string, config proxy.IsolationContainerConfig) (string, error) {
ret := _m.Called(op, handle, id, layerID, config)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string, string, proxy.IsolationContainerConfig) string); ok {
r0 = rf(op, handle, id, layerID, config)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string, string, proxy.IsolationContainerConfig) error); ok {
r1 = rf(op, handle, id, layerID, config)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// EpAddresses provides a mock function with given fields: op, id, name
func (_m *IsolationProxy) EpAddresses(op trace.Operation, id string, name string) ([]string, error) {
ret := _m.Called(op, id, name)
var r0 []string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string) []string); ok {
r0 = rf(op, id, name)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]string)
}
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string) error); ok {
r1 = rf(op, id, name)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// Handle provides a mock function with given fields: op, id, name
func (_m *IsolationProxy) Handle(op trace.Operation, id string, name string) (string, error) {
ret := _m.Called(op, id, name)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string) string); ok {
r0 = rf(op, id, name)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string) error); ok {
r1 = rf(op, id, name)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// Remove provides a mock function with given fields: op, id, force
func (_m *IsolationProxy) Remove(op trace.Operation, id string, force bool) error {
ret := _m.Called(op, id, force)
var r0 error
if rf, ok := ret.Get(0).(func(trace.Operation, string, bool) error); ok {
r0 = rf(op, id, force)
} else {
r0 = ret.Error(0)
}
return r0
}
// SetState provides a mock function with given fields: op, handle, name, state
func (_m *IsolationProxy) SetState(op trace.Operation, handle string, name string, state string) (string, error) {
ret := _m.Called(op, handle, name, state)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string, string) string); ok {
r0 = rf(op, handle, name, state)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string, string) error); ok {
r1 = rf(op, handle, name, state)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// State provides a mock function with given fields: op, id, name
func (_m *IsolationProxy) State(op trace.Operation, id string, name string) (string, error) {
ret := _m.Called(op, id, name)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string) string); ok {
r0 = rf(op, id, name)
} else {
r0 = ret.Get(0).(string)
}
var r1 error
if rf, ok := ret.Get(1).(func(trace.Operation, string, string) error); ok {
r1 = rf(op, id, name)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// UnbindScope provides a mock function with given fields: op, handle, name
func (_m *IsolationProxy) UnbindScope(op trace.Operation, handle string, name string) (string, interface{}, error) {
ret := _m.Called(op, handle, name)
var r0 string
if rf, ok := ret.Get(0).(func(trace.Operation, string, string) string); ok {
r0 = rf(op, handle, name)
} else {
r0 = ret.Get(0).(string)
}
var r1 interface{}
if rf, ok := ret.Get(1).(func(trace.Operation, string, string) interface{}); ok {
r1 = rf(op, handle, name)
} else {
if ret.Get(1) != nil {
r1 = ret.Get(1).(interface{})
}
}
var r2 error
if rf, ok := ret.Get(2).(func(trace.Operation, string, string) error); ok {
r2 = rf(op, handle, name)
} else {
r2 = ret.Error(2)
}
return r0, r1, r2
}

179
providers/vic/utils/units.go
View File

@@ -1,179 +0,0 @@
package utils
import (
"fmt"
"strings"
)
const (
BYTE = 1.0 << (10 * iota)
KILOBYTE
MEGABYTE
GIGABYTE
TERABYTE
PETABYTE
)
const (
KILOBYTESUFFIX = "Ki"
MEGABYTESUFFIX = "Mi"
GIGABYTESUFFIX = "Gi"
TERABYTESUFFIX = "Ti"
PETABYTESUFFIX = "Pi"
)
const (
ONE = 1.0
KILO = 1000.0
MEGA = KILO * KILO
GIGA = MEGA * KILO
TERA = GIGA * KILO
PETA = TERA * KILO
)
const (
KILOSUFFIX = "K"
MEGASUFFIX = "M"
GIGASUFFIX = "G"
TERASUFFIX = "T"
PETASUFFIX = "P"
)
const (
MEMORYCUTOVER = 100
FREQUENCYCUTOVER = 10
)
const (
MINPODSIZE = 2 * GIGABYTE
)
func MemsizeToBytesize(size int64, unit string) int64 {
u := strings.ToLower(unit)
var sizeInBytes int64
switch u {
case "b":
sizeInBytes = size
break
case "k":
case "kb":
sizeInBytes = size * KILOBYTE
break
case "m":
case "mb":
sizeInBytes = size * MEGABYTE
break
case "g":
case "gb":
sizeInBytes = size * GIGABYTE
break
case "t":
case "tb":
sizeInBytes = size * TERABYTE
break
case "p":
case "pb":
sizeInBytes = size * PETABYTE
break
default:
return 0
}
return sizeInBytes
}
func MemsizeToDecimalString(size int64, unit string) string {
sizeInBytes := MemsizeToBytesize(size, unit)
var res string
if sizeInBytes >= PETA*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/PETA, PETASUFFIX)
} else if sizeInBytes >= TERA*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/TERA, TERASUFFIX)
} else if sizeInBytes >= GIGA*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/GIGA, GIGASUFFIX)
} else if sizeInBytes >= MEGA*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/MEGA, MEGASUFFIX)
} else if sizeInBytes >= KILO*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/KILO, KILOSUFFIX)
} else {
res = fmt.Sprintf("%d", sizeInBytes)
}
return res
}
func MemsizeToBinaryString(size int64, unit string) string {
sizeInBytes := MemsizeToBytesize(size, unit)
var res string
if sizeInBytes >= PETABYTE*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/PETABYTE, PETABYTESUFFIX)
} else if sizeInBytes >= TERABYTE*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/TERABYTE, TERABYTESUFFIX)
} else if sizeInBytes >= GIGABYTE*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/GIGABYTE, GIGABYTESUFFIX)
} else if sizeInBytes >= MEGABYTE*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/MEGABYTE, MEGABYTESUFFIX)
} else if sizeInBytes >= KILOBYTE*MEMORYCUTOVER {
res = fmt.Sprintf("%d%s", sizeInBytes/KILOBYTE, KILOBYTESUFFIX)
} else {
res = fmt.Sprintf("%d", sizeInBytes)
}
return res
}
func MemsizeToMaxPodCount(size int64, unit string) int64 {
sizeInBytes := MemsizeToBytesize(size, unit)
// Divide by minimum pod size
return sizeInBytes / MINPODSIZE
}
func FrequencyToHertzFrequency(size int64, unit string) int64 {
u := strings.ToLower(unit)
var sizeInBytes int64
switch u {
case "k":
case "khz":
sizeInBytes = size * KILO
break
case "m":
case "mhz":
sizeInBytes = size * MEGA
break
case "g":
case "ghz":
sizeInBytes = size * GIGA
break
default:
return 0
}
return sizeInBytes
}
func CpuFrequencyToString(size int64, unit string) string {
var res string
hertz := FrequencyToHertzFrequency(size, unit)
if hertz >= GIGA*10 {
res = fmt.Sprintf("%d%s", hertz/GIGA, GIGASUFFIX)
} else if hertz >= MEGA*FREQUENCYCUTOVER {
res = fmt.Sprintf("%d%s", hertz/MEGA, MEGASUFFIX)
} else if hertz >= KILO*FREQUENCYCUTOVER {
res = fmt.Sprintf("%d%s", hertz/KILO, KILOSUFFIX)
} else {
res = fmt.Sprintf("%d", hertz)
}
return res
}
func CpuFrequencyToCores(size int64, unit string) int64 {
hertz := FrequencyToHertzFrequency(size, unit)
// Assume 1G per Core
cores := hertz / GIGA
return cores
}

54
providers/vic/utils/units_test.go
View File

@@ -1,54 +0,0 @@
package utils
import (
"testing"
"strings"
"gotest.tools/assert"
)
func TestMemoryConversion(t *testing.T) {
memSize := MemsizeToBinaryString(2, "Gb")
assert.Assert(t, memSize == "2048Mi")
memSize = MemsizeToDecimalString(2, "Gb")
assert.Assert(t, memSize == "2147M")
memSize = MemsizeToBinaryString(2048, "Mb")
assert.Assert(t, memSize == "2048Mi")
memSize = MemsizeToDecimalString(2048, "Mb")
assert.Assert(t, memSize == "2147M")
memSize = MemsizeToBinaryString(2048*1024, "Kb")
assert.Assert(t, memSize == "2048Mi")
memSize = MemsizeToDecimalString(2048*1024, "Kb")
assert.Assert(t, memSize == "2147M")
memSize = MemsizeToBinaryString(MEMORYCUTOVER, "Gb")
assert.Assert(t, memSize == "100Gi")
memSize = MemsizeToBinaryString(MEMORYCUTOVER-1, "Gb")
strings.HasSuffix(memSize, "Mi")
assert.Assert(t, strings.HasSuffix(memSize, "Mi"))
memSize = MemsizeToBinaryString((MEMORYCUTOVER-1)*1024, "Mb")
assert.Assert(t, strings.HasSuffix(memSize, "Mi"))
memSize = MemsizeToDecimalString(MEMORYCUTOVER*1000000000, "b")
assert.Assert(t, memSize == "100G")
memSize = MemsizeToDecimalString((MEMORYCUTOVER-1)*1000000000, "b")
assert.Assert(t, strings.HasSuffix(memSize, "M"))
}
func TestFrequencyConversion(t *testing.T) {
feq := CpuFrequencyToString(FREQUENCYCUTOVER, "Ghz")
assert.Assert(t, feq == "10G")
feq = CpuFrequencyToString(FREQUENCYCUTOVER-1, "Ghz")
assert.Assert(t, feq == "9000M")
feq = CpuFrequencyToString((FREQUENCYCUTOVER-1)*1000, "Mhz")
assert.Assert(t, feq == "9000M")
}

BIN
providers/vic/vic_kubelet_design.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 21 KiB

507
providers/vic/vic_provider.go
View File

@@ -1,507 +0,0 @@
package vic
import (
"fmt"
"os"
"path"
"syscall"
"time"
"golang.org/x/net/context"
log "github.com/Sirupsen/logrus"
"github.com/kr/pretty"
vicerrors "github.com/vmware/vic/lib/apiservers/engine/errors"
vicproxy "github.com/vmware/vic/lib/apiservers/engine/proxy"
"github.com/vmware/vic/lib/apiservers/portlayer/client"
"github.com/vmware/vic/lib/apiservers/portlayer/models"
vicconst "github.com/vmware/vic/lib/constants"
"github.com/vmware/vic/pkg/dio"
viclog "github.com/vmware/vic/pkg/log"
"github.com/vmware/vic/pkg/retry"
"github.com/vmware/vic/pkg/trace"
"github.com/virtual-kubelet/virtual-kubelet/manager"
"github.com/virtual-kubelet/virtual-kubelet/providers"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/cache"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/operations"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/proxy"
"github.com/virtual-kubelet/virtual-kubelet/providers/vic/utils"
"net"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
type VicProvider struct {
resourceManager *manager.ResourceManager
nodeName string
os string
podCount int
config VicConfig
podCache cache.PodCache
client *client.PortLayer
imageStore proxy.ImageStore
isolationProxy proxy.IsolationProxy
systemProxy *vicproxy.VicSystemProxy
}
const (
// Name of filename used in the endpoint vm
LogFilename = "virtual-kubelet"
// PanicLevel level, highest level of severity. Logs and then calls panic with the
// message passed to Debug, Info, ...
PanicLevel uint8 = iota
// FatalLevel level. Logs and then calls `os.Exit(1)`. It will exit even if the
// logging level is set to Panic.
FatalLevel
// ErrorLevel level. Logs. Used for errors that should definitely be noted.
// Commonly used for hooks to send errors to an error tracking service.
ErrorLevel
// WarnLevel level. Non-critical entries that deserve eyes.
WarnLevel
// InfoLevel level. General operational entries about what's going on inside the
// application.
InfoLevel
// DebugLevel level. Usually only enabled when debugging. Very verbose logging.
DebugLevel
)
var (
portlayerUp chan struct{}
)
func NewVicProvider(configFile string, rm *manager.ResourceManager, nodeName, operatingSystem string) (*VicProvider, error) {
initLogger()
op := trace.NewOperation(context.Background(), "VicProvider creation: config - %s", configFile)
defer trace.End(trace.Begin("", op))
config := NewVicConfig(op, configFile)
op.Infof("Provider config = %#v", config)
plClient := vicproxy.NewPortLayerClient(config.PortlayerAddr)
op.Infof("** Wait for VCH servers to start")
if !waitForVCH(op, plClient, config.PersonaAddr) {
msg := "VicProvider timed out waiting for VCH's persona and portlayer servers"
op.Errorf(msg)
return nil, fmt.Errorf(msg)
}
i, err := proxy.NewImageStore(plClient, config.PersonaAddr, config.PortlayerAddr)
if err != nil {
msg := "Couldn't initialize the image store"
op.Error(msg)
return nil, fmt.Errorf(msg)
}
op.Infof("** creating proxy")
p := VicProvider{
config: config,
nodeName: nodeName,
os: operatingSystem,
podCache: cache.NewVicPodCache(),
client: plClient,
resourceManager: rm,
systemProxy: vicproxy.NewSystemProxy(plClient),
}
p.imageStore = i
p.isolationProxy = proxy.NewIsolationProxy(plClient, config.PortlayerAddr, config.HostUUID, i, p.podCache)
op.Infof("** ready to go")
return &p, nil
}
func waitForVCH(op trace.Operation, plClient *client.PortLayer, personaAddr string) bool {
backoffConf := retry.NewBackoffConfig()
backoffConf.MaxInterval = 2 * time.Second
backoffConf.InitialInterval = 500 * time.Millisecond
backoffConf.MaxElapsedTime = 10 * time.Minute
// Wait for portlayer to start up
systemProxy := vicproxy.NewSystemProxy(plClient)
opWaitForPortlayer := func() error {
op.Infof("** Checking portlayer server is running")
if !systemProxy.PingPortlayer(context.Background()) {
return vicerrors.ServerNotReadyError{Name: "Portlayer"}
}
return nil
}
if err := retry.DoWithConfig(opWaitForPortlayer, vicerrors.IsServerNotReady, backoffConf); err != nil {
op.Errorf("Wait for portlayer to be ready failed")
return false
}
// Wait for persona to start up
dockerClient := NewVicDockerClient(personaAddr)
opWaitForPersona := func() error {
op.Infof("** Checking persona server is running")
if err := dockerClient.Ping(op); err != nil {
return vicerrors.ServerNotReadyError{Name: "Persona"}
}
return nil
}
if err := retry.DoWithConfig(opWaitForPersona, vicerrors.IsServerNotReady, backoffConf); err != nil {
op.Errorf("Wait for VIC docker server to be ready failed")
return false
}
return true
}
func initLogger() {
var logPath string
if LocalInstance() {
logPath = path.Join("", ".", LogFilename+".log")
} else {
logPath = path.Join("", vicconst.DefaultLogDir, LogFilename+".log")
}
os.MkdirAll(vicconst.DefaultLogDir, 0755)
// #nosec: Expect file permissions to be 0600 or less
f, err := os.OpenFile(logPath, os.O_RDWR|os.O_CREATE|os.O_APPEND|os.O_SYNC|syscall.O_NOCTTY, 0644)
if err != nil {
detail := fmt.Sprintf("failed to open file for VIC's virtual kubelet provider log: %s", err)
log.Error(detail)
}
// use multi-writer so it goes to both screen and session log
writer := dio.MultiWriter(f, os.Stdout)
logcfg := viclog.NewLoggingConfig()
logcfg.SetLogLevel(DebugLevel)
trace.SetLogLevel(DebugLevel)
trace.Logger.Out = writer
err = viclog.Init(logcfg)
if err != nil {
return
}
trace.InitLogger(logcfg)
}
// CreatePod takes a Kubernetes Pod and deploys it within the provider.
func (v *VicProvider) CreatePod(ctx context.Context, pod *v1.Pod) error {
op := trace.NewOperation(context.Background(), "CreatePod - %s", pod.Name)
defer trace.End(trace.Begin(pod.Name, op))
op.Debugf("Creating %s's pod = %# +v", pod.Name, pretty.Formatter(pod))
pc, err := operations.NewPodCreator(v.client, v.imageStore, v.isolationProxy, v.podCache, v.config.PersonaAddr, v.config.PortlayerAddr)
if err != nil {
return err
}
err = pc.CreatePod(op, pod, true)
if err != nil {
return err
}
//v.resourceManager.AddPod()
op.Debugf("** pod created ok")
return nil
}
// UpdatePod takes a Kubernetes Pod and updates it within the provider.
func (v *VicProvider) UpdatePod(ctx context.Context, pod *v1.Pod) error {
op := trace.NewOperation(context.Background(), "UpdatePod - %s", pod.Name)
defer trace.End(trace.Begin(pod.Name, op))
return nil
}
// DeletePod takes a Kubernetes Pod and deletes it from the provider.
func (v *VicProvider) DeletePod(ctx context.Context, pod *v1.Pod) error {
op := trace.NewOperation(context.Background(), "DeletePod - %s", pod.Name)
defer trace.End(trace.Begin(pod.Name, op))
op.Infof("Deleting %s's pod spec = %#v", pod.Name, pod.Spec)
pd, err := operations.NewPodDeleter(v.client, v.isolationProxy, v.podCache, v.config.PersonaAddr, v.config.PortlayerAddr)
if err != nil {
return err
}
err = pd.DeletePod(op, pod)
return err
}
// GetPod retrieves a pod by name from the provider (can be cached).
func (v *VicProvider) GetPod(ctx context.Context, namespace, name string) (*v1.Pod, error) {
op := trace.NewOperation(context.Background(), "GetPod - %s", name)
defer trace.End(trace.Begin(name, op))
// Look for the pod in our cache of running pods
vp, err := v.podCache.Get(op, namespace, name)
if err != nil {
return nil, err
}
return vp.Pod, nil
}
// GetContainerLogs retrieves the logs of a container by name from the provider.
func (v *VicProvider) GetContainerLogs(ctx context.Context, namespace, podName, containerName string, tail int) (string, error) {
op := trace.NewOperation(context.Background(), "GetContainerLogs - pod[%s], container[%s]", podName, containerName)
defer trace.End(trace.Begin("", op))
return "", nil
}
// Get full pod name as defined in the provider context
// TODO: Implementation
func (p *VicProvider) GetPodFullName(namespace string, pod string) string {
return ""
}
// RunInContainer executes a command in a container in the pod, copying data
// between in/out/err and the container's stdin/stdout/stderr.
func (p *VicProvider) RunInContainer(ctx context.Context, namespace, name, container string, cmd []string, attach providers.AttachIO) error {
log.Printf("receive ExecInContainer %q\n", container)
return nil
}
// GetPodStatus retrieves the status of a pod by name from the provider.
// This function needs to return a status or the reconcile loop will stop running.
func (v *VicProvider) GetPodStatus(ctx context.Context, namespace, name string) (*v1.PodStatus, error) {
op := trace.NewOperation(context.Background(), "GetPodStatus - pod[%s], namespace [%s]", name, namespace)
defer trace.End(trace.Begin("GetPodStatus", op))
now := metav1.NewTime(time.Now())
errorStatus := &v1.PodStatus{
Phase: v1.PodUnknown,
StartTime: &now,
Conditions: []v1.PodCondition{
{
Type: v1.PodInitialized,
Status: v1.ConditionUnknown,
},
{
Type: v1.PodReady,
Status: v1.ConditionUnknown,
},
{
Type: v1.PodScheduled,
Status: v1.ConditionUnknown,
},
},
}
// Look for the pod in our cache of running pods
vp, err := v.podCache.Get(op, namespace, name)
if err != nil {
return errorStatus, err
}
// Instantiate status object
statusReporter, err := operations.NewPodStatus(v.client, v.isolationProxy)
if err != nil {
return errorStatus, err
}
var nodeAddress string
nodeAddresses := v.NodeAddresses(ctx)
if len(nodeAddresses) > 0 {
nodeAddress = nodeAddresses[0].Address
} else {
nodeAddress = "0.0.0.0"
}
status, err := statusReporter.GetStatus(op, vp.ID, name, nodeAddress)
if err != nil {
return errorStatus, err
}
if vp.Pod.Status.StartTime != nil {
status.StartTime = vp.Pod.Status.StartTime
} else {
status.StartTime = &now
}
for _, container := range vp.Pod.Spec.Containers {
status.ContainerStatuses = append(status.ContainerStatuses, v1.ContainerStatus{
Name: container.Name,
Image: container.Image,
Ready: true,
RestartCount: 0,
State: v1.ContainerState{
Running: &v1.ContainerStateRunning{
StartedAt: *status.StartTime,
},
},
})
}
return status, nil
}
// GetPods retrieves a list of all pods running on the provider (can be cached).
func (v *VicProvider) GetPods(ctx context.Context) ([]*v1.Pod, error) {
op := trace.NewOperation(context.Background(), "GetPods")
defer trace.End(trace.Begin("GetPods", op))
vps := v.podCache.GetAll(op)
allPods := make([]*v1.Pod, 0)
for _, vp := range vps {
allPods = append(allPods, vp.Pod)
}
return allPods, nil
}
// Capacity returns a resource list with the capacity constraints of the provider.
func (v *VicProvider) Capacity(ctx context.Context) v1.ResourceList {
op := trace.NewOperation(context.Background(), "VicProvider.Capacity")
defer trace.End(trace.Begin("", op))
if v.systemProxy == nil {
err := NilProxy("VicProvider.Capacity", "SystemProxy")
op.Error(err)
return v1.ResourceList{}
}
info, err := v.systemProxy.VCHInfo(context.Background())
if err != nil {
op.Errorf("VicProvider.Capacity failed to get VCHInfo: %s", err.Error())
}
op.Infof("VCH Config: %# +v\n", pretty.Formatter(info))
return KubeResourcesFromVchInfo(op, info)
}
// NodeConditions returns a list of conditions (Ready, OutOfDisk, etc), which is polled periodically to update the node status
// within Kubernetes.
func (v *VicProvider) NodeConditions(ctx context.Context) []v1.NodeCondition {
// TODO: Make these dynamic and augment with custom ACI specific conditions of interest
return []v1.NodeCondition{
{
Type: "Ready",
Status: v1.ConditionTrue,
LastHeartbeatTime: metav1.Now(),
LastTransitionTime: metav1.Now(),
Reason: "KubeletReady",
Message: "kubelet is ready.",
},
{
Type: "OutOfDisk",
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Now(),
LastTransitionTime: metav1.Now(),
Reason: "KubeletHasSufficientDisk",
Message: "kubelet has sufficient disk space available",
},
{
Type: "MemoryPressure",
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Now(),
LastTransitionTime: metav1.Now(),
Reason: "KubeletHasSufficientMemory",
Message: "kubelet has sufficient memory available",
},
{
Type: "DiskPressure",
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Now(),
LastTransitionTime: metav1.Now(),
Reason: "KubeletHasNoDiskPressure",
Message: "kubelet has no disk pressure",
},
{
Type: "NetworkUnavailable",
Status: v1.ConditionFalse,
LastHeartbeatTime: metav1.Now(),
LastTransitionTime: metav1.Now(),
Reason: "RouteCreated",
Message: "RouteController created a route",
},
}
}
// NodeAddresses returns a list of addresses for the node status
// within Kubernetes.
func (v *VicProvider) NodeAddresses(ctx context.Context) []v1.NodeAddress {
addrs, err := net.InterfaceAddrs()
if err != nil {
return []v1.NodeAddress{}
}
var outAddresses []v1.NodeAddress
for _, addr := range addrs {
ipnet, ok := addr.(*net.IPNet)
if !ok || ipnet.IP.IsLoopback() || ipnet.IP.To4() == nil {
continue
}
outAddress := v1.NodeAddress{
Type: v1.NodeInternalIP,
Address: ipnet.IP.String(),
}
outAddresses = append(outAddresses, outAddress)
}
return outAddresses
}
// NodeDaemonEndpoints returns NodeDaemonEndpoints for the node status
// within Kubernetes.
func (v *VicProvider) NodeDaemonEndpoints(ctx context.Context) *v1.NodeDaemonEndpoints {
return &v1.NodeDaemonEndpoints{
KubeletEndpoint: v1.DaemonEndpoint{
Port: 80,
},
}
}
// OperatingSystem returns the operating system the provider is for.
func (v *VicProvider) OperatingSystem() string {
return v.os
}
//------------------------------------
// Utility Functions
//------------------------------------
// KubeResourcesFromVchInfo returns a K8s node resource list, given the VCHInfo
func KubeResourcesFromVchInfo(op trace.Operation, info *models.VCHInfo) v1.ResourceList {
nr := make(v1.ResourceList)
if info != nil {
cores := utils.CpuFrequencyToCores(info.CPUMhz, "Mhz")
// translate CPU resources. K8s wants cores. We have virtual cores based on mhz.
cpuQ := resource.Quantity{}
cpuQ.Set(cores)
nr[v1.ResourceCPU] = cpuQ
memQstr := utils.MemsizeToBinaryString(info.Memory, "Mb")
// translate memory resources. K8s wants bytes.
memQ, err := resource.ParseQuantity(memQstr)
if err == nil {
nr[v1.ResourceMemory] = memQ
} else {
op.Errorf("KubeResourcesFromVchInfo, cannot parse MEM quantity: %s, err: %s", memQstr, err)
}
}
// Estimate the available pod count, based on memory
podCount := utils.MemsizeToMaxPodCount(info.Memory, "Mb")
containerCountQ := resource.Quantity{}
containerCountQ.Set(podCount)
nr[v1.ResourcePods] = containerCountQ
op.Infof("Capacity Resource Config: %# +v\n", pretty.Formatter(nr))
return nr
}
func NilProxy(caller, proxyName string) error {
return fmt.Errorf("%s: %s not valid", caller, proxyName)
}