VMware vSphere Integrated Containers provider (#206)

* Add Virtual Kubelet provider for VIC

Initial virtual kubelet provider for VMware VIC.  This provider currently
handles creating and starting of a pod VM via the VIC portlayer and persona
server.  Image store handling via the VIC persona server.  This provider
currently requires the feature/wolfpack branch of VIC.

* Added pod stop and delete.  Also added node capacity.

Added the ability to stop and delete pod VMs via VIC.  Also retrieve
node capacity information from the VCH.

* Cleanup and readme file

Some file clean up and added a Readme.md markdown file for the VIC
provider.

* Cleaned up errors, added function comments, moved operation code

1. Cleaned up error handling.  Set standard for creating errors.
2. Added method prototype comments for all interface functions.
3. Moved PodCreator, PodStarter, PodStopper, and PodDeleter to a new folder.

* Add mocking code and unit tests for podcache, podcreator, and podstarter

Used the unit test framework used in VIC to handle assertions in the provider's
unit test.  Mocking code generated using OSS project mockery, which is compatible
with the testify assertion framework.

* Vendored packages for the VIC provider

Requires feature/wolfpack branch of VIC and a few specific commit sha of
projects used within VIC.

* Implementation of POD Stopper and Deleter unit tests (#4)

* Updated files for initial PR
This commit is contained in:
Loc Nguyen
2018-06-04 15:41:32 -07:00
committed by Ria Bhatia
parent 98a111e8b7
commit 513cebe7b7
6296 changed files with 1123685 additions and 8 deletions

16
vendor/github.com/vmware/vic/infra/scripts/README.md generated vendored Normal file
View File

@@ -0,0 +1,16 @@
## header-check
Simple header check for CI jobs, currently checks ".go" files only.
This will be called by the CI system (with no args) to perform checking and
fail the job if headers are not correctly set. It can also be called with the
'fix' argument to automatically add headers to the missing files.
Check if headers are fine:
```
$ ./infra/scripts/header-check.sh
```
Check and fix headers:
```
$ ./infra/scripts/header-check.sh fix
```

View File

@@ -0,0 +1,13 @@
// Copyright 2018 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

View File

@@ -0,0 +1,149 @@
#!/bin/bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.!/bin/bash
OS=$(uname | tr '[:upper:]' '[:lower:]')
tls () {
unset TLS_OPTS
}
no-tls () {
export TLS_OPTS="--no-tls"
}
unset-vic () {
unset MAPPED_NETWORKS NETWORKS IMAGE_STORE DATASTORE COMPUTE VOLUME_STORES IPADDR GOVC_INSECURE TLS THUMBPRINT OPS_CREDS VIC_NAME
}
vic-path () {
echo "${GOPATH}/src/github.com/vmware/vic"
}
vic-create () {
base=$(pwd)
(
cd "$(vic-path)"/bin || return
"$(vic-path)"/bin/vic-machine-"$OS" create --target="$GOVC_URL" "${OPS_CREDS[@]}" --image-store="$IMAGE_STORE" --compute-resource="$COMPUTE" "${TLS[@]}" ${TLS_OPTS} --name="${VIC_NAME:-${USER}test}" "${MAPPED_NETWORKS[@]}" "${VOLUME_STORES[@]}" "${NETWORKS[@]}" ${IPADDR} ${TIMEOUT} --thumbprint="$THUMBPRINT" "$@"
)
unset DOCKER_CERT_PATH DOCKER_TLS_VERIFY
unalias docker 2>/dev/null
envfile=$(vic-path)/bin/${VIC_NAME:-${USER}test}/${VIC_NAME:-${USER}test}.env
if [ -f "$envfile" ]; then
set -a
source "$envfile"
set +a
fi
# Something of a hack, but works for --no-tls so long as that's enabled via TLS_OPTS
if [ -z "${DOCKER_TLS_VERIFY+x}" ] && [ -z "${TLS_OPTS+x}" ]; then
alias docker='docker --tls'
fi
cd "$base" || exit
}
vic-delete () {
"$(vic-path)"/bin/vic-machine-"$OS" delete --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --thumbprint="$THUMBPRINT" --force "$@"
}
vic-inspect () {
"$(vic-path)"/bin/vic-machine-"$OS" inspect --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --thumbprint="$THUMBPRINT" "$@"
}
vic-upgrade () {
"$(vic-path)"/bin/vic-machine-"$OS" upgrade --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --thumbprint="$THUMBPRINT" "$@"
}
vic-ls () {
"$(vic-path)"/bin/vic-machine-"$OS" ls --target="$GOVC_URL" --thumbprint="$THUMBPRINT" "$@"
}
vic-ssh () {
unset keyarg
if [ -e "$HOME"/.ssh/authorized_keys ]; then
keyarg="--authorized-key=$HOME/.ssh/authorized_keys"
fi
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | awk -F"DOCKER_HOST=" '{print $2}' | cut -d ":" -f1 | cut -d "=" -f2)
echo "SSH to ${host}"
sshpass -ppassword ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@"${host}"
}
vic-admin () {
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | sed -n 's/.*DOCKER_HOST=\([^:\s*\).*/\1/p')
open http://"${host}":2378
}
addr-from-dockerhost () {
echo "$DOCKER_HOST" | sed -e 's/:[0-9]*$//'
}
vic-tail-portlayer() {
unset keyarg
if [ -e "$HOME"/.ssh/authorized_keys ]; then
keyarg="--authorized-key=$HOME/.ssh/authorized_keys"
fi
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | awk -F"DOCKER_HOST=" '{print $2}' | cut -d ":" -f1 | cut -d "=" -f2)
sshpass -ppassword ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@"${host}" tail -f /var/log/vic/port-layer.log
}
vic-tail-docker() {
unset keyarg
if [ -e "$HOME"/.ssh/authorized_keys ]; then
keyarg="--authorized-key=$HOME/.ssh/authorized_keys"
fi
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | awk -F"DOCKER_HOST=" '{print $2}' | cut -d ":" -f1 | cut -d "=" -f2)
sshpass -ppassword ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@"${host}" tail -f /var/log/vic/docker-personality.log
}
# import the custom sites
# example entry, actived by typing "example"
# those variales that hold multiple arguments which may contain spaces are arrays to allow for proper quoting
#example () {
# target='https://user:password@host.domain.com/datacenter'
# unset-vic
#
# export GOVC_URL=$target
#
# eval "export THUMBPRINT=$(govc about.cert -k -json | jq -r .ThumbprintSHA1)"
# export COMPUTE=cluster/pool
# export DATASTORE=datastore1
# export IMAGE_STORE=$DATASTORE/image/path
# export TIMEOUT="--timeout=10m"
# export IPADDR="--client-network-ip=vch-hostname.domain.com --client-network-gateway=x.x.x.x/22 --dns-server=y.y.y.y --dns-server=z.z.z.z"
# export VIC_NAME="MyVCH"
#
# TLS=("--tls-cname=vch-hostname.domain.com" "--organization=MyCompany")
# OPS_CREDS=("--ops-user=<user>" "--ops-password=<password>")
# NETWORKS=("--bridge-network=private-dpg-vlan" "--public-network=extern-dpg")
# MAPPED_NETWORKS=("--container-network=VM Network:external" "--container-network=SomeOtherNet:elsewhere")
# VOLUME_STORES=("--volume-store=$DATASTORE:default")
#
# export NETWORKS MAPPED_NETWORKS VOLUME_STORES OPS_CREDS TLS
#}
. ~/.vic

134
vendor/github.com/vmware/vic/infra/scripts/ci-logs.sh generated vendored Executable file
View File

@@ -0,0 +1,134 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Requirements
# ############
# - gcloud SDK: https://cloud.google.com/sdk/docs/
# - drone cli 0.5: https://github.com/drone/drone#from-source
#
# Examples
# ########
# Grab the logs for your current branch:
# $ ./ci-logs.sh
#
# If the build is running you can stream the logs using the '-s' option:
# $ ./ci-logs.sh -s
#
# Grab logs for a specific build:
# $ ./ci-logs.sh 4835
#
# Grab logs for recent failures on the master branch:
# $ drone build list --format {{.Number}}-{{.Branch}}-{{.Status}} vmware/vic | grep master-failure | cut -d- -f1 | xargs -n1 ./ci-logs.sh
#
# Find container log zip files for failed tests:
# $ grep FAIL ci.log | grep Test-Cases.Group | grep :: | awk '{print $1}' | xargs -n1 -I% bash -c "ls %*.zip"
top=$(git rev-parse --show-toplevel)
repo="vmware/$(basename "$top")"
dir="$top/ci-logs"
drone=${DRONE_CLI:-drone}
while getopts s flag
do
case $flag in
d)
dir="$OPTARG"
;;
s)
stream=true
;;
*)
echo "invalid option '$flag'"
exit 1
;;
esac
done
shift $((OPTIND-1))
build="$1"
job="$2"
export DRONE_SERVER=${DRONE_SERVER:-https://ci-vic.vmware.com}
if [ -z "$DRONE_TOKEN" ] ; then
echo "DRONE_TOKEN not set (available at $DRONE_SERVER/settings/profile)"
fi
if [ -z "$build" ] ; then
commit=$(git rev-parse HEAD)
builds=$($drone build list --format "{{.Number}}-{{.Commit}}" "$repo" | grep "$commit")
build=$(cut -d- -f1 <<<"$builds")
fi
if [ -z "$job" ] ; then
job=1
fi
state=$($drone build info "$repo" "$build" --format {{.Status}})
echo "$state"
case "$state" in
running)
if [ -z "$stream" ] ; then
exit
fi
;;
success|failure|killed)
stream=""
;;
pending|error)
exit
;;
esac
if [ ! -d "$dir" ] ; then
mkdir "$dir"
fi
logs="$dir/$build"
mkdir "$logs"
if [ -n "$stream" ] ; then
echo "Streaming CI log..."
curl --silent "$DRONE_SERVER/api/stream/$repo/$build/$job?access_token=$DRONE_TOKEN" | \
grep data: | cut -d: -f2- | grep -v '^$' | tee "$logs/ci.log"
else
echo "Downloading CI log..."
curl --silent "$DRONE_SERVER/api/repos/$repo/logs/$build/$job?access_token=$DRONE_TOKEN" > "$logs/ci.log"
fi
url=$(grep https://console.cloud.google.com/ "$logs/ci.log")
if [ -z "$url" ] ; then
echo "No integration logs link found for build ${build}"
exit
fi
name=$(basename "$url" | cut -d? -f1)
bucket=$(basename "$(dirname "$(dirname "$url")")")
echo "Downloading integration logs ($name)..."
gsutil cp "gs://$bucket/$name" "$logs/$name"
unzip -d "$logs" "$logs/$name" >/dev/null
if [ "$state" = "failure" ] ; then
echo "Container logs for failed tests:"
grep FAIL "$logs/ci.log" | grep Test-Cases.Group | grep :: | awk '{print $1}' | xargs -n1 -I% bash -c "ls $logs/%*.zip" 2>/dev/null
fi

78
vendor/github.com/vmware/vic/infra/scripts/coverage.sh generated vendored Executable file
View File

@@ -0,0 +1,78 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Works around the fact that `go test -coverprofile` does not work
# with multiple packages, see https://code.google.com/p/go/issues/detail?id=6909
#
# Usage: script/coverage [--html]
#
# --html Create HTML report and open it in browser
#
workdir=`git rev-parse --show-toplevel`/.cover
profile="$workdir/cover.out"
dir=$(dirname $0)
mode=count
# list any files (or patterns) to explicitly exclude from coverage
# you should have a pretty good reason before putting items here
exclude_files=(
)
join() { local IFS="$1"; shift; echo "$*"; }
excludes=$(join "|" ${exclude_files[@]} | sed -e 's/\./\\./g')
generate_pkg_cover_data() {
echo "$@"
mkdir -p "$workdir"
for pkg in "$@"; do
f="$workdir/$(echo $pkg | tr / -).cover"
go test -i -covermode="$mode" -coverprofile="$f" "$pkg"
go test -covermode="$mode" -coverprofile="$f" "$pkg"
done
echo "mode: $mode" >"$profile"
if [ -n "$excludes" ]; then
grep -h -v "^mode:" "$workdir"/*.cover | egrep -v "$excludes" >>"$profile"
else
grep -h -v "^mode:" "$workdir"/*.cover >>"$profile"
fi
}
# translate dirs to packages and strip args
dir_to_pkg() {
for dir in $@; do
if test "$dir" == "--html"; then
export html="true"
else
pkgs="$pkgs $(go list $dir/... | grep -v /vendor/)"
fi
done
echo $pkgs
}
show_cover_report() {
go tool cover -${1}="$profile"
}
generate_pkg_cover_data $(dir_to_pkg "$@")
show_cover_report func
if test "$html" == "true"; then
show_cover_report html
fi

43
vendor/github.com/vmware/vic/infra/scripts/focused-test.sh generated vendored Executable file
View File

@@ -0,0 +1,43 @@
#!/bin/bash
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Runs go test on any packages that differ from vmware/master that have
# Go test files.
#
# Usage: infra/scripts/focused-test.sh
#
if [ -z "$1" ]; then
export REMOTE=$(git remote -v | grep "github.com/vmware/vic.git (fetch)" | awk '{print$1;exit}')/master
echo "Using ${REMOTE} as default remote"
else
echo "Using ${REMOTE} as specified remote"
export REMOTE="$1"
fi
echo "Finding modified packages with test files"
PKG=$(git diff --stat ${REMOTE} --name-only | xargs dirname | uniq | while read i; do
for f in ./$i/*_test.go; do
if [ -e $f ]; then
echo -n "./$i ";
fi;
break;
done;
done)
echo "Testing packages: $PKG"
if [ -n "$PKG" ]; then
go test $PKG;
fi

58
vendor/github.com/vmware/vic/infra/scripts/go-deps.sh generated vendored Executable file
View File

@@ -0,0 +1,58 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Lists the non-standard library Go packages the specified package depends
# on.
#
# Usage: script/go-deps.sh pkg
#
# pkg This is github.com/vmware/vic/cmd/imagec for example
#
# If VIC_CACHE_DEPS environment variable is defined, this script will attempt to read
# cached dependencies from disk if those exist. If they are not cached, dependencies will be
# regenerated and cached.
cache_dir=.godeps_cache
pkg=$1
flags=$2
cachedname=`echo .$1.godeps_cache | sed 's/\//_/g'`
if [ -d "$pkg" ]; then
if [[ "$flags" == *d* ]]; then
# Only output if make is given the '-d' flag
echo "Generating deps for $pkg" >&2
fi
if [ -n "$VIC_CACHE_DEPS" ]; then
mkdir -p $cache_dir
if [ ! -f $cache_dir/$cachedname ]; then
go list -f '{{join .Deps "\n"}}' github.com/vmware/vic/"$pkg" 2>/dev/null | \
xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' 2>/dev/null | \
sed -e 's:github.com/vmware/vic/\(.*\)$:\1/*:' > "$cache_dir/$cachedname"
fi
cat "$cache_dir/$cachedname"
else
go list -f '{{join .Deps "\n"}}' github.com/vmware/vic/"$pkg" 2>/dev/null | \
xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' 2>/dev/null | \
sed -e 's:github.com/vmware/vic/\(.*\)$:\1/*:'
fi
else
if [[ "$flags" == *d* ]]
then
echo "$0: package '$pkg' does not exist" >&2
fi
fi

120
vendor/github.com/vmware/vic/infra/scripts/header-check.sh generated vendored Executable file
View File

@@ -0,0 +1,120 @@
#!/bin/bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Simple script that will check files of type .go, .sh, .bash, .robot, or Makefile
# for the copyright header.
#
# This will be called by the CI system (with no args) to perform checking and
# fail the job if headers are not correctly set. It can also be called with the
# 'fix' argument to automatically add headers to the missing files.
#
# Check if headers are fine:
# $ ./scripts/header-check.sh
# Check and fix headers:
# All changes must be committed for fix to work
# $ ./scripts/header-check.sh fix
set -e -o pipefail
# These header variables MUST match the first two lines of the
# VMware-copyright file in the scripts directory.
#
# These will be evaluated as a regex against the target file
HEADER[1]="^\/\/ Copyright [0-9]{4}(-[0-9]{4})? VMware, Inc\. All Rights Reserved\.$"
HEADER[2]="^\/\/$"
# Initialize vars
ERR=false
FAIL=false
all-files() {
git ls-files |\
# Check .go files, Makefile, sh files, bash files, and robot files
grep -e "\.go$" -e "Makefile$" -e "\.sh$" -e "\.bash$" -e "\.robot$" |\
# Ignore vendor/
grep -v vendor/ |\
grep -v lib/apiservers/portlayer/client | grep -v lib/apiservers/portlayer/models
}
for file in $(all-files); do
# get the file extension / type
ext=${file##*.}
# increment line count in certain cases
increment=false
# should we be incrementing the line count
if [[ $ext == "sh" ]]; then
increment=true
fi
for count in $(seq 1 ${#HEADER[@]}); do
if [[ $ext != "go" ]]; then
# if not go code assuming # will suffice
text="${HEADER[$count]/'\/\/'/#}"
else
text=${HEADER[$count]}
fi
if [[ "$increment" = true ]]; then
line=$((count + 1))
else
line=$count
fi
# do we have a header match?
if [[ ! $(sed ${line}q\;d ${file}) =~ ${text} ]]; then
ERR=true
fi
done
if [ $ERR == true ]; then
# is there is a fix argument and are all changes committed
if [[ $# -gt 0 && $1 =~ [[:upper:]fix] ]]; then
# based on file type fix the copyright
case "$ext" in
go)
cat $(dirname $0)/VMware-copyright ${file} > ${file}.new
;;
sh)
head -1 ${file} > ${file}.new
cat $(dirname $0)/VMware-copyright | sed 's/\/\//\#/1' >> ${file}.new
grep -v '#!/bin/bash' ${file} >> ${file}.new
;;
*)
cat $(dirname $0)/VMware-copyright | sed 's/\/\//\#/1' > ${file}.new
cat ${file} >> ${file}.new
;;
esac
if [ "$(uname -s)" = "Darwin" ]; then
permissions=$(stat -f "%OLp" ${file})
else
permissions=$(stat --format '%a' ${file})
fi
mv ${file}.new ${file}
# make permissions the same
chmod $permissions ${file}
echo "$file.. $(tput -T xterm setaf 3)FIXING$(tput -T xterm sgr0)"
ERR=false
else
echo "$file.. $(tput -T xterm setaf 1)FAIL$(tput -T xterm sgr0)"
ERR=false
FAIL=true
fi
fi
done
# If we failed one check, return 1
[ $FAIL == true ] && exit 1 || exit 0

215
vendor/github.com/vmware/vic/infra/scripts/local-ci.sh generated vendored Executable file
View File

@@ -0,0 +1,215 @@
#!/bin/bash
# Copyright 2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
##### defaults
secretsfile=""
docker_test="Group1-Docker-Commands"
target_vch=""
odir="ci-results"
ci_container="gcr.io/eminent-nation-87317/vic-integration-test:1.44"
github_api_key=""
test_url=""
test_datastore=""
test_username=""
test_password=""
BASE_DIR=$(dirname $(readlink -f "$BASH_SOURCE"))
vic_dir=${BASE_DIR}/../../
##### utility functions
function usage() {
echo "Usage: $0 [options]" 1>&2
echo
echo " Options can be provided by commandline argument, environment variable, or a secrets yaml file containing the"
echo " variables. If a secrets file is not provided, this script will attempt to retrieve some info from govc, such as"
echo " TEST_URL, TEST_USERNAME, and TEST_PASSWORD."
echo
echo " options:"
echo " -t DOCKER_TEST (or env var)"
echo " -v TARGET_VCH (or env var) name of VCH"
echo " -f SECRETS_FILE (or env var)"
echo " -g GITHUB_API_KEY (or env var)"
echo " -u TEST_URL (or env var or in secretsfile)"
echo " -s TEST_DATASTORE (or env var or in secretsfile)"
echo " -n TEST_USERNAME (or env var or in secretsfile)"
echo " -p TEST_PASSWORD (or env var or in secretsfile)"
echo " -d debug dumps out all the inputs and results of the resulting options"
echo
echo " example:"
echo " $0 -t Group1-Docker-Commands -v my_vch -s test.secrets.nested -g xxxxxx"
echo " $0 -t 1-01-Docker-Info.robot -v my_vch -s test.secrets.nested -g xxxxxx"
echo
echo " DOCKER_TEST=Group1-Docker-Commands/1-01-Docker-Info.robot TARGET_VCH=my_vch SECRETS_FILE=test.secrets.nested $0"
echo
echo " $0 -s test.secrets.nested (all params defined in secrets file)"
exit 1
}
function GetGovcParamsFromEnv() {
echo "Getting params from GOVC var"
test_username=$(govc env | grep GOVC_USERNAME | cut -d= -f2)
test_password=$(govc env | grep GOVC_PASSWORD | cut -d= -f2)
test_url=$(govc env | grep GOVC_URL | cut -d= -f2)
}
function GetParamsFromSecrets() {
echo "Getting params from the secrets file"
secrets_api_key="$(grep 'GITHUB_AUTOMATION_API_KEY' ${secretsfile} | awk '{ print $2 }')"
secrets_url="$(grep 'TEST_URL_ARRAY' ${secretsfile} | awk '{ print $2 }')"
secrets_datastore="$(grep -E '\s+TEST_DATASTORE' ${secretsfile} | awk '{ print $2 }')"
secrets_username="$(grep 'TEST_USERNAME' ${secretsfile} | awk '{ print $2 }')"
secrets_password="$(grep 'TEST_PASSWORD' ${secretsfile} | awk '{ print $2 }')"
secrets_docker_test="$(grep 'DOCKER_TEST' ${secretsfile} | awk '{ print $2 }')"
secrets_target_vch="$(grep 'TARGET_VCH' ${secretsfile} | awk '{ print $2 }')"
}
function DoWork() {
mkdir -p $odir
testsContainer=$(docker create -it \
-w /vic \
-v "$vic_dir:/vic" \
-e GOVC_URL="$ip" \
-e GOVC_INSECURE=1 \
-e GITHUB_AUTOMATION_API_KEY=${github_api_key}\
-e TEST_URL_ARRAY=${test_url}\
-e TEST_DATASTORE=${test_datastore}\
-e TEST_USERNAME=${test_username}\
-e TEST_PASSWORD=${test_password}\
-e TARGET_VCH=${target_vch}\
-e DEBUG_VCH=1\
${ci_container}\
bash -c "pybot -d /vic/${odir} /vic/tests/test-cases/"$docker_test"")
docker start -ai $testsContainer
}
function DebugInputDump() {
echo "Environment Variables"
echo "---------------------"
echo "SECRETS_FILE="${SECRETS_FILE}
echo "TARGET_VCH="${TARGET_VCH}
echo "DOCKER_TEST="${DOCKER_TEST}
echo "GITHUB_API_KEY="${GITHUB_API_KEY}
echo "TEST_URL="${TEST_URL}
echo "TEST_DATASTORE"=${TEST_DATASTORE}
echo "TEST_USERNAME"=${TEST_USERNAME}
echo "TEST_PASSWORD"=${TEST_PASSWORD}
echo
echo "Arguments"
echo "---------------------"
echo "SECRETS_FILE="$secretsfile
echo "TARGET_VCH="$target_vch
echo "DOCKER_TEST="$docker_Test
echo "GITHUB_API_KEY="$github_api_key
echo "TEST_URL="$test_url
echo "TEST_DATASTORE"=$test_datastore
echo "TEST_USERNAME"=$test_username
echo "TEST_PASSWORD"=$test_password
echo
echo "Secrets file"
echo "---------------------"
echo "TARGET_VCH="$secrets_target_vch
echo "DOCKER_TEST="$secrets_docker_test
echo "GITHUB_API_KEY="$secrets_api_key
echo "TEST_URL="$secrets_url
echo "TEST_DATASTORE"=$secrets_datastore
echo "TEST_USERNAME"=$secrets_username
echo "TEST_PASSWORD"=$secrets_password
}
function DebugDump() {
echo
echo "====================="
echo "SECRETS_FILE="$secretsfile
echo "TARGET_VCH="$target_vch
echo "DOCKER_TEST="$docker_test
echo "GITHUB_API_KEY="$github_api_key
echo "TEST_URL="$test_url
echo "TEST_DATASTORE"=$test_datastore
echo "TEST_USERNAME"=$test_username
echo "TEST_PASSWORD"=$test_password
echo "vic-dir"=$vic_dir
}
##### Get command line arguments
while getopts f:t:v:g:u:s:n:p:d flag
do
case $flag in
f)
secretsfile=$OPTARG
;;
t)
docker_test="$OPTARG"
;;
v)
target_vch=$OPTARG
;;
g)
github_api_key=$OPTARG
;;
u)
test_url=$OPTARG
;;
s)
test_datastore=$OPTARG
;;
n)
test_username=$OPTARG
;;
p)
test_password=$OPTARG
;;
d)
debug_enabled=1
;;
*)
usage
;;
esac
done
##### Preconditions...
# There is a priority in the preconditions. First, environment variable. Second, secrets file. Third, command line argument.
secretsfile=${SECRETS_FILE:-$secretsfile}
if [[ -z $secretsfile ]] ; then
GetGovcParamsFromEnv
else
GetParamsFromSecrets
fi
if [[ ! -z ${debug_enabled} ]] ; then
DebugInputDump
fi
target_vch=${TARGET_VCH:-$secrets_target_vch}
docker_test=${DOCKER_TEST:-$secrets_docker_test}
github_api_key=${GITHUB_API_KEY:-$secrets_api_key}
test_url=${TEST_URL:-$secrets_url}
test_datastore=${TEST_DATASTORE:-$secrets_datastore}
test_username=${TEST_USERNAME:-$secrets_username}
test_password=${TEST_PASSWORD:-$secrets_password}
if [[ -z ${target_vch} ]] || [[ -z "${docker_test}" ]] || [[ -z $github_api_key ]] || [[ -z $test_url ]] || [[ -z $test_datastore ]] || [[ -z $test_password ]] ; then
usage
fi
if [[ ! -z ${debug_enabled} ]] ; then
DebugDump
fi
##### The actual work
DoWork

20
vendor/github.com/vmware/vic/infra/scripts/misspell.sh generated vendored Executable file
View File

@@ -0,0 +1,20 @@
#!/bin/bash
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http:#www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
out=$(misspell -error $(find . -mindepth 1 -maxdepth 1 -type d -not -name vendor) | grep -v -e lib/config/dynamic/admiral/client | grep -v -e lib/config/dynamic/admiral/models | grep -v -e lib/config/dynamic/admiral/swagger.json)
if [[ ! -z "${out// }" ]]; then
echo "$out"
exit 2
fi

View File

@@ -0,0 +1,169 @@
#!/bin/bash
# Copyright 2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
if [ -z "$PS1" ]; then
interactive=1
else
interactive=0
fi
BASE_DIR=$(dirname $(readlink -f "$BASH_SOURCE"))
VIC_DIR=$(dirname $(readlink -f $BASE_DIR/..))
# Run the command given on the VCH instead of locally
function on-vch() {
ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no -i $VIC_KEY root@$VCH_IP -C $@ 2>/dev/null
}
function get-thumbprint() {
govc about.cert -k -thumbprint | awk '{print $NF}'
}
# Determines whether the human VCH name is adequate to continue
function vch-name-is-ambiguous() {
[ $($VIC_DIR/bin/vic-machine-linux \
ls \
--target=$target \
--user=$username \
--password=$password \
--thumbprint=$(get-thumbprint) \
| grep $VIC_NAME | wc -l) -ne 1 ] \
&& return 0 || return 1
}
# Check GOVC vars & print help text if not
function check-govc-vars() {
if [[ ! $(govc ls 2>/dev/null) ]]; then
echo "ERROR:"
echo "GOVC environment variables are required to use this command. Set the necessary variables to allow govc to connect to your vSphere deployment:";
echo "GOVC_USERNAME: username on vSphere target"
echo "GOVC_PASSWORD: password on vSphere target"
echo "GOVC_URL: IP or FQDN of your vSphere target"
echo "GOVC_INSECURE: set to 1 to disable tls verify when using govc to talk to vSphere"
exit 1
fi
}
# Make sure we have at least one of VIC_NAME or VIC_ID
function check-vch-name-or-id () {
if [[ ! -v VIC_NAME ]] && [[ ! -v VIC_ID ]]; then
echo "Please set one of the following environment variables to specify the VCH which you would like to reconfigure:"
echo "VIC_NAME: name of VCH; matches --name argument for vic-machine"
echo "VIC_ID: ID of VCH, as displayed in output of vic-machine ls"
if [[ $interactive -eq 0 ]]; then
exit 1
fi
read -p "Or enter VCH name to continue: " VIC_NAME
fi
}
# Falls back on VIC_ID if possible, bails if not, assumes the VCH name is ambiguous
function check-name-isnt-ambiguous () {
username=$(govc env | grep GOVC_USERNAME | cut -d= -f2)
password=$(govc env | grep GOVC_PASSWORD | cut -d= -f2)
target=$(govc env | grep GOVC_URL | cut -d= -f2)
if [[ ! -v VIC_ID ]] && [[ $(vch-name-is-ambiguous) ]]; then
echo "The provided VIC name is ambiguous; please choose the correct VCH ID from the output below and assign it to the environment variable VIC_ID, e.g., export VIC_ID=12"
$VIC_DIR/bin/vic-machine-linux\
ls \
--target $GOVC_URL \
--user $GOVC_USER \
--password=$GOVC_PASSWORD \
--thumbprint=$(get-thumbprint)
if [[ $interactive -eq 0 ]]; then
exit 1
fi
read -p "Enter unique VIC ID from above: " VIC_ID
fi
}
# Translates VCH name to ID if necessary
function get-vic-id () {
if [[ -z $VIC_ID ]]; then
export VIC_ID="$($VIC_DIR/bin/vic-machine-linux ls --target=$target --user=$username --password=$password --thumbprint=$(get-thumbprint) | grep $VIC_NAME | awk '{print $1}')"
fi
}
function get-ssh-keys() {
if [[ -z $VIC_KEY ]]; then
echo "Variable VIC_KEY not set. Provide the path to your public SSH key below."
if [[ $interactive -eq 0 ]]; then
exit 1
fi
read -p "Path to your public SSH key for access to VCH [/home/$USER/.ssh/id_rsa.pub]: " key
export VIC_KEY=${key:-/home/$USER/.ssh/id_rsa.pub}
fi
}
# Checks environment for required inputs
function sanity-checks () {
check-govc-vars
check-vch-name-or-id
check-name-isnt-ambiguous
get-vic-id
get-ssh-keys
}
# Enables SSH and saves off the VCH IP address
function enable-debug () {
VCH_IP=$($VIC_DIR/bin/vic-machine-linux debug \
--target=$target \
--id=$VIC_ID \
--user=$username \
--password=$password \
--authorized-key=$VIC_KEY \
--thumbprint=$(get-thumbprint) \
| grep -A1 "Published ports" | tail -n1 | awk '{print $NF}')
}
# SCPs the component in $1 to the VCH, plops it in place, and brutally kills the previous running process
function replace-component() {
scp -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no $VIC_DIR/bin/$1 root@$VCH_IP:/tmp/$1 2>/dev/null
pid=$(on-vch ps -e --format='pid,args' \
| grep $1 | grep -v grep | awk '{print $1}')
on-vch chmod 755 /tmp/$1
on-vch mv /tmp/$1 /sbin/$1
if [[ $1 == "vic-init" ]]; then
on-vch systemctl restart vic-init
else
on-vch kill -9 $pid
fi
}
function replace-components () {
if [[ $1 == "" ]]; then # replace everything
services="port-layer-server docker-engine-server vicadmin vic-init"
else
services="$@"
fi
for x in $services; do
echo "Replacing component $x..."
replace-component $x
done
}
sanity-checks
enable-debug
replace-components $@
echo "If you ran make push with no arguments or replaced vic-init, wait a few moments and run this to get the new IP of your VCH:"
echo "$VIC_DIR/bin/vic-machine-linux inspect --target=$target --id=$VIC_ID --user=$username --password=$password --thumbprint=$(get-thumbprint)"

View File

@@ -0,0 +1,39 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
if [[ $# == 1 && $1 == "debug" ]]; then
echo " \
-X github.com/vmware/vic/pkg/version.Version=`git describe --abbrev=0 --tags` \
-X github.com/vmware/vic/pkg/version.BuildNumber=\"${BUILD_NUMBER}\" \
-X github.com/vmware/vic/pkg/version.BuildDate=`date -u +%Y/%m/%d@%H:%M:%S` \
-X github.com/vmware/vic/pkg/version.GitCommit=`git rev-parse --short HEAD` \
-X github.com/vmware/vic/pkg/version.State=` \
if [[ -n $(git ls-files --others --exclude-standard) || \
! $(git diff-files --no-ext-diff --quiet) || \
! $(git diff-index --no-ext-diff --quiet --cached HEAD) \
]]; then echo 'dirty'; fi`"
else
echo "-s -w \
-X github.com/vmware/vic/pkg/version.Version=`git describe --abbrev=0 --tags` \
-X github.com/vmware/vic/pkg/version.BuildNumber=\"${BUILD_NUMBER}\" \
-X github.com/vmware/vic/pkg/version.BuildDate=`date -u +%Y/%m/%d@%H:%M:%S` \
-X github.com/vmware/vic/pkg/version.GitCommit=`git rev-parse --short HEAD` \
-X github.com/vmware/vic/pkg/version.State=` \
if [[ -n $(git ls-files --others --exclude-standard) || \
! $(git diff-files --no-ext-diff --quiet) || \
! $(git diff-index --no-ext-diff --quiet --cached HEAD) \
]]; then echo 'dirty'; fi`"
fi

68
vendor/github.com/vmware/vic/infra/scripts/vic-logs.sh generated vendored Executable file
View File

@@ -0,0 +1,68 @@
#!/bin/bash -e
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Create a VM and boot stateless ESXi via cdrom/iso
set -o pipefail
usage() {
cat <<'EOF'
Usage: $0 [-f] <host> <port-layer.log | docker.log>
GOVC_* environment variables also apply, see https://github.com/vmware/govmomi/tree/master/govc#usage
If GOVC_USERNAME is set, it is used to create an account on the ESX vm. Default is to use the existing root account.
If GOVC_PASSWORD is set, the account password will be set to this value. Default is to use the given ESX_URL password.
EOF
}
if [ $# -lt 2 ] ; then
usage
exit 1
fi
while getopts "f" opt; do
case $opt in
f)
follow="tail/"
;;
\?)
usage
exit 1
;;
esac
done
shift $((OPTIND-1))
host="$1"
file="$2"
username=$GOVC_USERNAME
password=$GOVC_PASSWORD
unset GOVC_USERNAME GOVC_PASSWORD
if [ -z "$password" ] ; then
# extract password from $GOVC_URL
password=$(govc env GOVC_PASSWORD)
fi
if [ -z "$username" ] ; then
username=$(govc env GOVC_USERNAME)
fi
jar=$(mktemp -t cookie-XXXX)
trap 'rm $jar' EXIT
curl -k -c "${jar}" --form username="${username}" --form password="${password}" https://"${host}":2378/authentication
curl -k -b "${jar}" https://"${host}":2378/logs/"${follow}""${file}"

View File

@@ -0,0 +1,35 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
red="\033[1;31m"
color_end="\033[0m"
GLOBIGNORE='vendor'
# will check staged items for whitespace
if [[ $# -gt 0 && $1 =~ [[:upper:]pre] ]]; then
if [[ $(git diff --cached --check) ]]; then
echo -e ${red}whitespace check failed${color_end}
git diff --cached --check
exit 1
fi
else
# check staged and changed
if [[ $(git diff --cached --check) || $(git diff --check) ]]; then
echo -e ${red}whitespace check failed${color_end}
git diff --cached --check
git diff --check
exit 1
fi
fi