VMware vSphere Integrated Containers provider (#206)

* Add Virtual Kubelet provider for VIC

Initial virtual kubelet provider for VMware VIC.  This provider currently
handles creating and starting of a pod VM via the VIC portlayer and persona
server.  Image store handling via the VIC persona server.  This provider
currently requires the feature/wolfpack branch of VIC.

* Added pod stop and delete.  Also added node capacity.

Added the ability to stop and delete pod VMs via VIC.  Also retrieve
node capacity information from the VCH.

* Cleanup and readme file

Some file clean up and added a Readme.md markdown file for the VIC
provider.

* Cleaned up errors, added function comments, moved operation code

1. Cleaned up error handling.  Set standard for creating errors.
2. Added method prototype comments for all interface functions.
3. Moved PodCreator, PodStarter, PodStopper, and PodDeleter to a new folder.

* Add mocking code and unit tests for podcache, podcreator, and podstarter

Used the unit test framework used in VIC to handle assertions in the provider's
unit test.  Mocking code generated using OSS project mockery, which is compatible
with the testify assertion framework.

* Vendored packages for the VIC provider

Requires feature/wolfpack branch of VIC and a few specific commit sha of
projects used within VIC.

* Implementation of POD Stopper and Deleter unit tests (#4)

* Updated files for initial PR
This commit is contained in:
Loc Nguyen
2018-06-04 15:41:32 -07:00
committed by Ria Bhatia
parent 98a111e8b7
commit 513cebe7b7
6296 changed files with 1123685 additions and 8 deletions

View File

@@ -0,0 +1 @@
*.vmdk

View File

@@ -0,0 +1,143 @@
# Vagrant Dev Box
## Overview
This box is an Ubuntu 16.04 VM with the following setup by default:
* Docker daemon with port forwarded to the Fusion/Workstation host at localhost:12375
* Go toolchain
* Additional tools (lsof, strace, etc)
## Requirements
* Vagrant (https://www.vagrantup.com/downloads.html)
* VMware Fusion or Workstation
* Vagrant Fusion or Workstation license (https://www.vagrantup.com/vmware)
## Provisioning
All files matching _provision*.sh_ in this directory will be applied by the Vagrantfile, you can symlink custom scripts
if needed. The scripts are not Vagrant specific and can be applied to a VM running on ESX for example.
## Fusion/Workstation host usage
The following commands can be used from your Fusion or Workstation host.
### Shared Folders
By default your *GOPATH* is shared with the same path as the host. This is useful if your editor runs
on the host, then errors on the guest with filename:line info have the same path. For example, when running the
following command within the top-level project directory:
``` shell
vagrant ssh -- make -C $PWD all
```
### Create the VM
``` shell
vagrant up
```
### SSH Access
``` shell
vagrant ssh
```
### Docker Access
``` shell
DOCKER_HOST=localhost:12375 docker ps
```
### Stop the VM
``` shell
vagrant halt
```
### Restart the VM
``` shell
vagrant reload
```
### Provision
After you've done a `vagrant up`, the provisioning can be applied without reloading via:
``` shell
vagrant provision
```
### Delete the VM
``` shell
vagrant destroy
```
## VM guest usage
To open a bash term in the VM, use `vagrant ssh`.
The following commands can be used from devbox VM guest.
``` shell
cd $GOPATH/src/github.com/vmware/vic
```
### Local Drone CI test
``` shell
drone exec
```
## Devbox on ESX
The devbox can be deployed to ESX, the same provisioning scripts are applied:
``` shell
./deploy-esx.sh
```
### SSH access
``` shell
ssh-add ~/.vagrant.d/insecure_private_key
vmip=$(govc vm.ip $USER-ubuntu-1604)
ssh vagrant@$vmip
```
### Shared folders
You can share your folder by first exporting via NFS:
```
echo "$HOME/vic $(govc vm.ip $USER-ubuntu-1604) -alldirs -mapall=$(id -u):$(id -g)" | sudo tee -a /etc/exports
sudo nfsd restart
```
Then mount within the ubuntu VM:
``` shell
ssh vagrant@$vmip sudo mkdir -p $HOME/vic
ssh vagrant@$vmip sudo mount $(ipconfig getifaddr en1):$HOME/vic $HOME/vic
```
Note that you may need to use enN depending on the type of connection you have - use ifconfig to verify.
Note also that nfs-common is not installed in the box by default.
You can also mount your folder within ESX:
``` shell
govc datastore.create -type nfs -name nfsDatastore -remote-host $(ipconfig getifaddr en1) -remote-path $HOME/vic
esxip=$(govc host.info -json | jq -r '.HostSystems[].Config.Network.Vnic[] | select(.Device == "vmk0") | .Spec.Ip.IpAddress')
ssh root@$esxip mkdir -p $HOME
ssh root@$esxip /vmfs/volumes/nfsDatastore $HOME/vic
```
Add `$esxip` to /etc/exports and restart nfsd again.

View File

@@ -0,0 +1,90 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Deploy Vagrant box to esx
set -e
if [ "$(uname -s)" = "Darwin" ]; then
PATH="/Applications/VMware Fusion.app/Contents/Library:$PATH"
fi
export GOVC_URL=${GOVC_URL-"root:vagrant@localhost:18443"}
export GOVC_DATASTORE=${GOVC_DATASTORE-"datastore1"}
export GOVC_NETWORK=${GOVC_NETWORK-"VM Network"}
export GOVC_INSECURE=1
echo "deploying to $(awk -F@ '{print $2}' <<<"$GOVC_URL"):"
govc about
config="$(git rev-parse --show-toplevel)/Vagrantfile"
box=$(grep vic_dev.vm.box "$config" | awk -F\' '{print $2}')
provider=$(dirname "$box")
name=$(basename "$box")
disk="${name}.vmdk"
pushd "$(dirname "$0")" >/dev/null
if ! govc datastore.ls "${name}/${disk}" 1>/dev/null 2>&1 ; then
if [ ! -e "$disk" ] ; then
src=$(echo ~/.vagrant.d/boxes/"${provider}"-*-"${name}"/*.*.*/vmware_desktop/disk.vmdk)
if [ ! -e "$src" ] ; then
echo "box not found, install via: vagrant box add --provider vmware_desktop $box"
exit 1
fi
echo "converting vagrant box for ESX..."
vmware-vdiskmanager -r "$src" -t 0 "$disk"
fi
echo "importing vmdk to datastore ${GOVC_DATASTORE}..."
govc import.vmdk "$disk" "$name"
fi
vm_name=${VM_NAME-"${USER}-${name}"}
vm_memory=${VM_MEMORY-$(grep memory "$config" | awk -F\= 'FNR == 1 {gsub(/ /, "", $2); print $2}')}
if [ -z "$(govc ls "vm/$vm_name")" ] ; then
echo "creating VM ${vm_name}..."
govc vm.create -m "$vm_memory" -c 2 -g ubuntu64Guest -disk.controller=pvscsi -on=false "$vm_name"
govc vm.disk.attach -vm "$vm_name" -link=true -disk "$name/$disk"
govc device.cdrom.add -vm "$vm_name"
govc vm.power -on "$vm_name"
fi
# An ipv6 is reported by tools when the machine is first booted.
# Wait until we get an ipv4 address from tools.
while true
do
ip=$(govc vm.ip "$vm_name")
ipv4="${ip//[^.]}"
if [ "${#ipv4}" -eq 3 ]
then
break
fi
sleep 1
done
echo "VM ip=$ip"
for script in provision.sh provision-drone.sh; do
echo "Applying $script..."
ssh -i ~/.vagrant.d/insecure_private_key "vagrant@$ip" sudo bash -s - < "$script"
done

View File

@@ -0,0 +1,22 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This depends on docker being on the box and running
docker pull drone/drone
curl http://downloads.drone.io/0.5.0/release/linux/amd64/drone.tar.gz | tar zx
sudo install -t /usr/local/bin drone

View File

@@ -0,0 +1,52 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# add key for docker repo
apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
# add docker apt sources
echo "deb https://apt.dockerproject.org/repo ubuntu-xenial main" > /etc/apt/sources.list.d/docker.list
# https://github.com/mitchellh/vagrant/issues/289
apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
# set GOPATH based on shared folder of vagrant
pro="/home/${BASH_ARGV[0]}/.profile"
echo "export GOPATH=${BASH_ARGV[1]}" >> "$pro"
# add GOPATH/bin to the PATH
echo "export PATH=$PATH:${BASH_ARGV[1]}/bin" >> "$pro"
apt-get -y install curl lsof strace git shellcheck tree mc silversearcher-ag jq htpdate apt-transport-https ca-certificates nfs-common sshpass
function update_go {
(cd /usr/local &&
(curl --silent -L $go_file | tar -zxf -) &&
ln -fs /usr/local/go/bin/* /usr/local/bin/)
}
# install / upgrade go
go_file="https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"
go_version=$(basename $go_file | cut -d. -f1-3)
if [[ ! -d "/usr/local/go" || $(go version | awk '{print $(3)}') != "$go_version" ]] ; then
update_go
fi
# Install docker
apt-get -y install linux-image-extra-$(uname -r)
apt-get -y --allow-downgrades install docker-engine=1.13.1-0~ubuntu-xenial
apt-mark hold docker-engine
usermod -aG docker vagrant
systemctl start docker