d8dfun/virtual-kubelet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

VMware vSphere Integrated Containers provider (#206)

Browse Source 
* Add Virtual Kubelet provider for VIC

Initial virtual kubelet provider for VMware VIC.  This provider currently
handles creating and starting of a pod VM via the VIC portlayer and persona
server.  Image store handling via the VIC persona server.  This provider
currently requires the feature/wolfpack branch of VIC.

* Added pod stop and delete.  Also added node capacity.

Added the ability to stop and delete pod VMs via VIC.  Also retrieve
node capacity information from the VCH.

* Cleanup and readme file

Some file clean up and added a Readme.md markdown file for the VIC
provider.

* Cleaned up errors, added function comments, moved operation code

1. Cleaned up error handling.  Set standard for creating errors.
2. Added method prototype comments for all interface functions.
3. Moved PodCreator, PodStarter, PodStopper, and PodDeleter to a new folder.

* Add mocking code and unit tests for podcache, podcreator, and podstarter

Used the unit test framework used in VIC to handle assertions in the provider's
unit test.  Mocking code generated using OSS project mockery, which is compatible
with the testify assertion framework.

* Vendored packages for the VIC provider

Requires feature/wolfpack branch of VIC and a few specific commit sha of
projects used within VIC.

* Implementation of POD Stopper and Deleter unit tests (#4)

* Updated files for initial PR
This commit is contained in:
Loc Nguyen
2018-06-04 15:41:32 -07:00
committed by Ria Bhatia
parent 98a111e8b7
commit 513cebe7b7
6296 changed files with 1123685 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
vendor/github.com/vmware/vic/infra/dlv/README.md generated vendored Normal file
View File

@@ -0,0 +1,90 @@
# DLV Debugging of VCH servers
## Overview
This notes describes how to set up dlv remote debugging of VCH servers
## Building debug enabled binaries (non stripped)
Set the following environment variable
``` shell
export VIC_DEBUG_BUILD=true
```
this tells the makefile to build non stripped binaries
## Preparing the VCH for debugging
Ssh must be enabled on the VCH. To enabled it run the following command:
``` shell
vic-machine-linux debug --target <TARGET> --thumbprint <THUMBPRINT> --name <vch-name> --enable-ssh --key <PATH-TO-AUTHORIZED-KEYS-FILE>
```
Both scripts: **dlv-setup.sh** and **dlv-ctl.sh** rely on ssh public-key authentication as specified by the command above (authorized_keys file).
The script **dlv-setup.sh**
must be used to set up the VCH to run dlv. It performs several tasks:
* opens the necessary ports in the iptables,
* copies the GO environment necessary to run dlv (from $GOROOT and $GOPATH),
* creates the attach and detach scripts that reside in /usr/local/bin in the VCH
The command requires the address (or FQDN) of the VCH. The environment variables:
``` shell
DLV_TARGET_HOST
```
can be used to pass that information to **dlv-setup.sh**. Alternatively the option __-h__ can be used on the command line.
For instance:
``` shell
dlv-setup.sh -h <target IP address/FQDN>
```
## Launching dlv on the target host
To launch dlv and attach it to one of the VCH server run the command **dlv-ctl.sh**. The following target servers are supported:
* vic-init
* vic-admin
* port-layer
* docker-engine
* vic-machine
The scripts needs the IP address (or the FQDN) of the target VCH host. The same environment variable
and command line options as **dlv-setup.sh** are accepted. The script takes two arguments:
* action: this can be either attach or detach
* target: this can be one of the VCH services listed above
For example:
``` shell
dlv-ctl.sh -h <target IP address/FQDN> attach vic-admin
```
launches dlv in headless mode and attaches it to vic-admin and prints out the port number on which dlv listens.
To detach you can use:
``` shell
dlv-ctl.sh -h <target IP address/FQDN> detach vic-admin
```
The script allows specifying the action through with a couple of additional options __-a__ (for attach) and __-d__ for detach.
For example:
``` shell
dlv-ctl.sh -h <target IP address/FQDN> -a vic-admin
```
Performs an attach. While
``` shell
dlv-ctl.sh -h <target IP address/FQDN> -d vic-admin
```
performs a detach
## Using Goland to perform remote debugging
After dlv is attached to the appropriate server, you can configure Goland to start debugging that process.
On the drop down list with the debugger configurations select: __Edit Configurations__. In the configuration tab
click on the __+__ button to add a new configuration. Select __Go Remote__. Type in the the VCH IP address (or FQDN) and
the port number returned by the **dlv-ctl** attach command. The debugger should be able to connect to the server.
## Timeout issues while debugging
Consider for example the case in which a request is sent to the **port-layer** from **docker-engine**. When the request
is received by the **port-layer** a breakpoint is hit. The developer next steps through the code in the **port-layer**
while the **docker-engine** is waiting for a response. This may cause the **docker-engine** to timeout and abort
or retry the request. Ideally when debugging is enabled all the timeouts should be increased to allow slower
response times. This has not yet been implemented. The current idea is to connect the extension of timeout duration
with the debug level specified at the time of VCH creation.
## Debugging vic-machine
The target __vic-machine__ has been added to debug the vic-machine remotely, in this case everything above applies with
the exception that the __vic-machine__ does not usually run on a VCH host.

117
vendor/github.com/vmware/vic/infra/dlv/dlv-ctl.sh generated vendored Executable file
View File

@@ -0,0 +1,117 @@
#!/bin/bash
# Copyright 2016-2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.!/bin/bash
SSH="ssh -o StrictHostKeyChecking=no"
SCP="scp -o StrictHostKeyChecking=no"
REMOTE_DLV_ATTACH=/usr/local/bin/dlv-attach-headless.sh
REMOTE_DLV_DETACH=/usr/local/bin/dlv-detach-headless.sh
function usage() {
echo "Usage: $0 -h vch-address [-a/-d] [attach/detach] target" >&2
echo "Valid targets are: "
echo " vic-init"
echo " vic-admin"
echo " docker-engine"
echo " port-layer"
echo " vic-machine"
echo " virtual-kubelet"
exit 1
}
while getopts "h:ad" flag
do
case $flag in
h)
# Optional
export DLV_TARGET_HOST="$OPTARG"
;;
a)
export COMMAND="attach"
;;
d)
export COMMAND="detach"
;;
*)
usage
;;
esac
done
shift $((OPTIND-1))
if [[ -z "${COMMAND}" && $# != 2 ]]; then
usage
elif [[ -n "${COMMAND}" && $# != 1 ]]; then
usage
fi
if [ -z "${COMMAND}" ]; then
COMMAND=$1
TARGET=$2
else
TARGET=$1
fi
case ${TARGET} in
vic-init)
PORT=2345
;;
vic-admin)
# Change target to vicadmin
TARGET=vicadmin
PORT=2346
;;
docker-engine)
PORT=2347
;;
port-layer)
PORT=2348
;;
vic-machine)
PORT=2349
;;
virtual-kubelet)
PORT=2350
;;
*)
usage
;;
esac
if [ -z "${DLV_TARGET_HOST}" ]; then
usage
fi
if [ ${COMMAND} == "attach" ]; then
${SSH} root@${DLV_TARGET_HOST} "nohup /usr/local/bin/dlv-attach-headless.sh $TARGET $PORT > /var/tmp/${TARGET}.log 2>&1 &"
elif [ ${COMMAND} == "detach" ]; then
${SSH} root@${DLV_TARGET_HOST} "/usr/local/bin/dlv-detach-headless.sh $PORT"
else
usage
fi
echo $DLV_TARGET_HOST:$PORT

136
vendor/github.com/vmware/vic/infra/dlv/dlv-setup.sh generated vendored Executable file
View File

@@ -0,0 +1,136 @@
#!/bin/bash
# Copyright 2016-2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.!/bin/bash
SSH="ssh -o StrictHostKeyChecking=no"
SCP="scp -q -o StrictHostKeyChecking=no"
function usage() {
echo "Usage: $0 -h vch-address" >&2
exit 1
}
while getopts "h" flag
do
case $flag in
h)
# Optional
export DLV_TARGET_HOST="$OPTARG"
;;
*)
usage
;;
esac
done
shift $((OPTIND-1))
if [ -z "${DLV_TARGET_HOST}" ]; then
usage
fi
DLV_BIN="$GOPATH/bin/dlv"
if [ -z "${GOPATH}" -o -z "${GOROOT}" ]; then
echo GOROOT and GOPATH should be set to point to the current GOLANG enironment
exit 1
fi
# copy dlv binary
echo -n copying dlv binary..
if [ -f ${DLV_BIN} ]; then
${SCP} ${DLV_BIN} root@${DLV_TARGET_HOST}:/usr/local/bin
else
echo $DLV_BIN does not exist. Run \"go get github.com/derekparker/delve/cmd/dlv\"
exit 1
fi
echo done
# copy GOROOT env
echo -n copying GOROOT environment..
${SSH} root@${DLV_TARGET_HOST} "mkdir -p /usr/local/go"
${SCP} -r ${GOROOT}/bin root@${DLV_TARGET_HOST}:/usr/local/go
${SCP} -r ${GOROOT}/api root@${DLV_TARGET_HOST}:/usr/local/go
${SCP} ${GOROOT}/VERSION root@${DLV_TARGET_HOST}:/usr/local/go
${SSH} root@${DLV_TARGET_HOST} "ln -f -s /usr/local/go/bin/go /usr/local/bin/go"
echo done
# open IPTABLES
echo -n fixing ipatables..
${SSH} root@${DLV_TARGET_HOST} "iptables -I INPUT -p tcp -m tcp --dport 2345:2350 -j ACCEPT"
echo done
echo "Iptables changed: run \"iptables -D INPUT 1\" when finished debugging"
# write remote dlv attach script
TEMPFILE=$(mktemp)
cat > ${TEMPFILE} <<EOF
#/bin/bash
if [ \$# != 2 ]; then
echo "\$0 vic-init|vicadmin|docker-engine|port-layer|vic-machine|virtual-kubelet port"
exit 1
fi
NAME=\$1
PORT=\$2
if [ -z "\${NAME}" -o -z "\${PORT}" ]; then
echo "\$0 vic-init|vicadmin|docker-engine|port-layer|vic-machine|virtual-kubelet port"
exit 1
fi
PID=\$(ps -e | grep \${NAME} | grep -v grep | tr -s ' ' | cut -d " " -f 2)
if [ -z "\${PID}" ]; then
echo "\$0: cannot find process \${NAME}"
exit 1
fi
dlv attach \${PID} --api-version 2 --headless --listen=:\${PORT}
EOF
${SCP} ${TEMPFILE} root@${DLV_TARGET_HOST}:/usr/local/bin/dlv-attach-headless.sh
# write dlv detach script
cat > ${TEMPFILE} <<EOF
#/bin/bash
if [ \$# != 1 ]; then
echo "\$0 port-number"
exit 1
fi
PORT=\$1
if [ -z "\${PORT}" ]; then
echo "\$0 port-number"
exit 1
fi
# Find appropriate dlv instance
PID=\$(ps -ef | grep "dlv" | grep "api-version" | grep \${PORT} | grep -v grep | tr -s ' ' | cut -d " " -f 2)
if [ -z "\${PID}" ]; then
echo "\$0: cannot find dlv listening on \${PORT}"
exit 1
fi
kill \${PID}
EOF
${SCP} ${TEMPFILE} root@${DLV_TARGET_HOST}:/usr/local/bin/dlv-detach-headless.sh
${SSH} root@${DLV_TARGET_HOST} 'chmod +x /usr/local/bin/*'
rm ${TEMPFILE}

120
vendor/github.com/vmware/vic/infra/integration-image/Dockerfile generated vendored Normal file
View File

@@ -0,0 +1,120 @@
# Building:
# cp -R /usr/lib/vmware-ovftool/ .
# docker build --no-cache -t vic-test -f Dockerfile .
# docker tag vic-test gcr.io/eminent-nation-87317/vic-integration-test:1.x
# gcloud auth login
# gcloud docker -- push gcr.io/eminent-nation-87317/vic-integration-test:1.x
# download and install harbor certs, then docker login, then:
# docker tag vic-test wdc-harbor-ci.eng.vmware.com/default-project/vic-integration-test:1.x
# docker push wdc-harbor-ci.eng.vmware.com/default-project/vic-integration-test:1.x
FROM golang:1.8
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
RUN curl -sL https://deb.nodesource.com/setup_7.x | bash - && \
apt-get update && apt-get install -y --no-install-recommends \
jq \
bc \
time \
gcc \
python-dev \
libffi-dev \
libssl-dev \
sshpass \
ant \
ant-optional \
openjdk-7-jdk \
rpcbind \
nfs-common \
unzip \
zip \
bzip2 \
nodejs \
parted \
# Add docker in docker support
btrfs-tools \
e2fsprogs \
iptables \
xfsprogs \
dnsutils \
netcat \
# Add headless chrome support
google-chrome-stable \
# Speed up ISO builds with already installed reqs
yum \
yum-utils \
cpio \
rpm \
ca-certificates \
xz-utils \
xorriso \
sendmail && \
# Cleanup
apt-get autoremove -y && \
rm -rf /var/lib/apt/lists/*
RUN wget https://bootstrap.pypa.io/get-pip.py && \
python ./get-pip.py && \
pip install pyasn1 google-apitools==0.5.15 gsutil robotframework robotframework-sshlibrary robotframework-httplibrary requests dbbot robotframework-selenium2library robotframework-pabot --upgrade
# Install docker, docker clients 1.11,1.12 and 1.13
# Also install docker compose 1.13
RUN curl -sSL https://get.docker.com/ | sh && \
curl -fsSLO https://get.docker.com/builds/Linux/x86_64/docker-1.11.2.tgz && \
tar --strip-components=1 -xvzf docker-1.11.2.tgz -C /usr/bin && \
mv /usr/bin/docker /usr/bin/docker1.11 && \
curl -fsSLO https://get.docker.com/builds/Linux/x86_64/docker-1.12.6.tgz && \
tar --strip-components=1 -xvzf docker-1.12.6.tgz -C /usr/bin && \
mv /usr/bin/docker /usr/bin/docker1.12 && \
curl -fsSLO https://get.docker.com/builds/Linux/x86_64/docker-1.13.0.tgz && \
tar --strip-components=1 -xvzf docker-1.13.0.tgz -C /usr/bin && \
mv /usr/bin/docker /usr/bin/docker1.13 && \
ln -s /usr/bin/docker1.13 /usr/bin/docker && \
curl -L https://github.com/docker/compose/releases/download/1.11.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose
COPY vmware-ovftool /usr/lib/vmware-ovftool
RUN ln -s /usr/lib/vmware-ovftool/ovftool /usr/local/bin/ovftool
RUN curl -fsSLO https://releases.hashicorp.com/packer/0.12.2/packer_0.12.2_linux_amd64.zip && \
unzip packer_0.12.2_linux_amd64.zip -d /usr/bin && \
rm packer_0.12.2_linux_amd64.zip
RUN wget https://github.com/drone/drone-cli/releases/download/v0.8.3/drone_linux_amd64.tar.gz && tar zxf drone_linux_amd64.tar.gz && \
install -t /usr/local/bin drone
RUN curl -sSL https://github.com/vmware/govmomi/releases/download/v0.16.0/govc_linux_amd64.gz | gzip -d > /usr/local/bin/govc && \
chmod +x /usr/local/bin/govc
RUN wget https://launchpad.net/ubuntu/+source/wget/1.18-2ubuntu1/+build/10470166/+files/wget_1.18-2ubuntu1_amd64.deb && \
dpkg -i wget_1.18-2ubuntu1_amd64.deb
# Add docker in docker support
# version: docker:1.13-dind
# reference: https://github.com/docker-library/docker/blob/b202ec7e529f5426e2ad7e8c0a8b82cacd406573/1.13/dind/Dockerfile
#
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -x \
&& groupadd --system dockremap \
&& adduser --system --ingroup dockremap dockremap \
&& echo 'dockremap:165536:65536' >> /etc/subuid \
&& echo 'dockremap:165536:65536' >> /etc/subgid
ENV DIND_COMMIT 3b5fac462d21ca164b3778647420016315289034
RUN wget "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind" -O /usr/local/bin/dind \
&& chmod +x /usr/local/bin/dind
# This container needs to be run in privileged mode(run with --privileged option) to make it work
COPY dockerd-entrypoint.sh /usr/local/bin/dockerd-entrypoint.sh
RUN chmod +x /usr/local/bin/dockerd-entrypoint.sh
COPY scripts /opt/vmware/scripts
ENV PATH="${PATH}:/opt/vmware/scripts"
VOLUME /var/lib/docker

21
vendor/github.com/vmware/vic/infra/integration-image/Dockerfile.downstream generated vendored Normal file
View File

@@ -0,0 +1,21 @@
# Building:
# docker build --no-cache -t vic-downstream -f Dockerfile.downstream .
# docker tag vic-downstream gcr.io/eminent-nation-87317/vic-downstream-trigger:1.x
# gcloud auth login
# gcloud docker -- push gcr.io/eminent-nation-87317/vic-downstream-trigger:1.x
# open vpn to CI cluster then run:
# docker tag vic-downstream 192.168.31.15/library/vic-downstream-trigger:1.x
# docker push 192.168.31.15/library/vic-downstream-trigger:1.x
FROM ubuntu
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates
RUN curl http://downloads.drone.io/release/linux/amd64/drone.tar.gz | tar zx && \
install -t /usr/bin drone
RUN echo '#!/bin/bash' >> /usr/bin/trigger
RUN echo 'num=$(drone build list --format "{{.Number}} {{.Status}}" vmware/vic-product | grep -v running | head -n1 | cut -d" " -f1)' >> /usr/bin/trigger
RUN echo 'for i in {1..5}; do drone build start --fork vmware/vic-product $num && break || sleep 15; done' >> /usr/bin/trigger
RUN chmod +x /usr/bin/trigger
ENTRYPOINT ["trigger"]

15
vendor/github.com/vmware/vic/infra/integration-image/Dockerfile.gitclone generated vendored Normal file
View File

@@ -0,0 +1,15 @@
# Building:
# docker build --no-cache -t git-clone -f Dockerfile.gitclone .
# docker tag git-clone gcr.io/eminent-nation-87317/git-clone:1.x
# gcloud auth login
# gcloud docker -- push gcr.io/eminent-nation-87317/git-clone:1.x
# open vpn to CI cluster then run:
# docker tag git-clone 192.168.31.15/library/git-clone:1.x
# docker push 192.168.31.15/library/git-clone:1.x
FROM ubuntu
RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates
ADD git-clone.sh /usr/bin/git-clone.sh
ENTRYPOINT ["git-clone.sh"]

34
vendor/github.com/vmware/vic/infra/integration-image/LocalRepo generated vendored Normal file
View File

@@ -0,0 +1,34 @@
# Building:
# docker build --no-cache -t local-repo -f LocalRepo .
# docker tag local-repo gcr.io/eminent-nation-87317/local-repo:x
# gcloud auth login
# gcloud docker -- push gcr.io/eminent-nation-87317/local-repo:x
# open vpn to CI cluster then run:
# docker tag local-repo 192.168.31.15/library/local-repo:x
# docker push 192.168.31.15/library/local-repo:x
# Running:
# docker run -d -p 80:80 local-repo
FROM fedora:21
RUN yum install yum-plugin-ovl -y && yum install wget createrepo nginx -y
RUN mkdir /usr/share/nginx/html/photon && mkdir /usr/share/nginx/html/photon-updates
ENV EXCLUDE_LIST "index.html*,openjdk*,openjre*,perl*,postgresql*,python-*,\
python3*,ruby*,subversion*,gnome*,NetworkManager*,cloud*,docker*,grub*,ktap*,\
kubernetes*,linux-docs*,linux-sound*,linux-tools*,docbook*,httpd*,go-*,jna*,\
linux-debuginfo*,linux-dev*,linux-docs*,linux-drivers*,linux-oprofile*,linux-sound*,\
linux-tools*,linux-esx-debuginfo*,linux-esx-devel*,linux-esx-docs*,nginx*,sysdig*"
RUN wget -e robots=off -r -nH -nd -np -R $EXCLUDE_LIST https://vmware.bintray.com/photon_release_1.0_x86_64/x86_64/ -P /usr/share/nginx/html/photon
RUN wget -e robots=off -r -nH -nd -np -R $EXCLUDE_LIST https://dl.bintray.com/vmware/photon_updates_1.0_x86_64/x86_64/ -P /usr/share/nginx/html/photon-updates
RUN createrepo /usr/share/nginx/html/photon
RUN createrepo /usr/share/nginx/html/photon-updates
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
EXPOSE 80
CMD [ "/usr/sbin/nginx" ]

39
vendor/github.com/vmware/vic/infra/integration-image/dockerd-entrypoint.sh generated vendored Executable file
View File

@@ -0,0 +1,39 @@
#!/bin/sh
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
set -e
# Only give socker support by default, use bash arguments to add dockerd parameters
# Use unix:///var/run/docker-local.sock to avoid collison with /var/run/docker.sock
# no arguments passed
# or first arg is `-f` or `--some-option`
if [ "$#" -eq 0 -o "${1#-}" != "$1" ]; then
# add our default arguments
set -- dockerd \
--host=unix:///var/run/docker-local.sock \
--storage-driver=vfs \
--pidfile=/var/run/docker-local.pid \
"$@"
fi
if [ "$1" = 'dockerd' ]; then
# if we're running Docker, let's pipe through dind
# (and we'll run dind explicitly with "sh" since its shebang is /bin/bash)
set -- sh "$(which dind)" "$@"
fi
exec "$@"

18
vendor/github.com/vmware/vic/infra/integration-image/git-clone.sh generated vendored Executable file
View File

@@ -0,0 +1,18 @@
#!/bin/bash
# Copyright 2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
sed -i 's|remotes/origin/*|remotes/origin/*\n\tfetch = +refs/pull/*/head:refs/remotes/origin/pr/|g' .git/config
git fetch --tags origin -v refs/pull/$DRONE_PULL_REQUEST/head:refs/remotes/origin/pr/$DRONE_PULL_REQUEST
git checkout pr/$DRONE_PULL_REQUEST

29
vendor/github.com/vmware/vic/infra/integration-image/scripts/README.md generated vendored Normal file
View File

@@ -0,0 +1,29 @@
# scripts
Integration image MUST be rebuilt after updates to scripts in this directory to be included in CI
### ca-test.sh
Generate a root CA and a server certificate
### intermediate-ca-test.sh
Generate a root CA, intermediate CA, and a server certificate
### ubuntu-install-ca.sh
Installs the generated root CA into the system root store
### ubuntu-install-intermediate-ca.sh
Installs the generated intermediate CA into the system root store
### ubuntu-remove-ca.sh
Removes the generated root CA from the system root store
### ubuntu-remove-intermediate-ca.sh
Removes the generated intermediate CA from the system root store

50
vendor/github.com/vmware/vic/infra/integration-image/scripts/bundle-certs.sh generated vendored Executable file
View File

@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Generate a CA and server certificate bundle
set -euf -o pipefail
CA_DIR="/root/ca"
CA_NAME="STARK_ENTERPRISES_ROOT_CA"
OUT_DIR="/root/ca/bundle"
OUT_FILE="/root/ca/cert-bundle.tgz"
SERVER_CERT_CN="starkenterprises.io"
while getopts ":c:d:f:n:o:" opt; do
case $opt in
c) CA_NAME="$OPTARG"
;;
d) CA_DIR="$OPTARG"
;;
f) OUT_FILE="$OPTARG"
;;
n) SERVER_CERT_CN="$OPTARG"
;;
o) OUT_DIR="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
mkdir -p $OUT_DIR
cd $CA_DIR
cp certs/$CA_NAME.crt $OUT_DIR
cp private/${SERVER_CERT_CN}.key.pem $OUT_DIR
cp certs/${SERVER_CERT_CN}.cert.pem $OUT_DIR
dir=$(dirname $OUT_DIR)
bundle_dir=$(basename $OUT_DIR)
tar cvf $OUT_FILE -C $dir $bundle_dir

100
vendor/github.com/vmware/vic/infra/integration-image/scripts/ca-test.sh generated vendored Executable file
View File

@@ -0,0 +1,100 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Generate a root CA and a server certificate
##### Default configuration options
SERVER_CERT_CN="starkenterprises.io"
#####
while getopts ":s:" opt; do
case $opt in
s) SERVER_CERT_CN="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
### Create root CA
mkdir -p /root/ca
cp openssl.cnf /root/ca
cd /root/ca
mkdir certs crl csr newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial
# Generate root CA key
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out private/ca.key.pem 4096
chmod 400 private/ca.key.pem
# Generate root CA CSR
openssl req -config openssl.cnf \
-new -sha256 \
-key private/ca.key.pem \
-out csr/ca.csr.pem \
-extensions v3_ca \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Certificate Authority/CN=Stark Enterprises Global CA"
# Self sign for root CA certificate
openssl x509 -req -extfile openssl.cnf \
-extensions v3_ca \
-days 7300 -in csr/ca.csr.pem -signkey private/ca.key.pem -out certs/ca.cert.pem
chmod 444 certs/ca.cert.pem
openssl x509 -noout -text -in certs/ca.cert.pem
# Output CRT format
openssl x509 -in certs/ca.cert.pem -inform PEM -out certs/ca.cert.crt
### Create server certificate
cd /root/ca
# Generate server key
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out private/${SERVER_CERT_CN}.key.pem 4096
chmod 400 private/${SERVER_CERT_CN}.key.pem
# Generate server CSR
openssl req -config openssl.cnf \
-new -sha256 \
-key private/${SERVER_CERT_CN}.key.pem \
-out csr/${SERVER_CERT_CN}.csr.pem \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Web Services/CN=${SERVER_CERT_CN}"
# Sign CSR with CA key, output server certificate
openssl ca -config openssl.cnf \
-batch \
-extensions server_cert \
-days 365 -notext -md sha256 \
-in csr/${SERVER_CERT_CN}.csr.pem \
-out certs/${SERVER_CERT_CN}.cert.pem
chmod 444 certs/${SERVER_CERT_CN}.cert.pem
openssl x509 -noout -text -in certs/${SERVER_CERT_CN}.cert.pem
# Test certificate
openssl verify -CAfile certs/ca.cert.pem certs/${SERVER_CERT_CN}.cert.pem
### Bundle output
mkdir bundle
cp certs/ca.cert.crt bundle
cp private/${SERVER_CERT_CN}.key.pem bundle
cp certs/${SERVER_CERT_CN}.cert.pem bundle
tar cvf cert-bundle.tgz bundle

66
vendor/github.com/vmware/vic/infra/integration-image/scripts/generate-ca.sh generated vendored Executable file
View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Generate a CA
set -euf -o pipefail
OUTDIR="/root/ca"
CA_NAME="STARK_ENTERPRISES_ROOT_CA"
while getopts ":c:d:" opt; do
case $opt in
c) CA_NAME="$OPTARG"
;;
d) OUTDIR="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
CONF_DIR=`dirname $0`
### Create root CA
mkdir -p $OUTDIR
cp $CONF_DIR/openssl.cnf $OUTDIR
cd $OUTDIR
mkdir certs crl csr newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial
# Generate root CA key
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out private/ca.key.pem 4096
chmod 400 private/ca.key.pem
# Generate root CA CSR
openssl req -config $CONF_DIR/openssl.cnf \
-new -sha256 \
-key private/ca.key.pem \
-out csr/ca.csr.pem \
-extensions v3_ca \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Certificate Authority/CN=Stark Enterprises Global CA"
# Self sign for root CA certificate
openssl x509 -req -extfile $CONF_DIR/openssl.cnf \
-extensions v3_ca \
-days 7300 -in csr/ca.csr.pem -signkey private/ca.key.pem -out certs/ca.cert.pem
chmod 444 certs/ca.cert.pem
openssl x509 -noout -text -in certs/ca.cert.pem
# Output CRT format
openssl x509 -in certs/ca.cert.pem -inform PEM -out certs/$CA_NAME.crt

46
vendor/github.com/vmware/vic/infra/integration-image/scripts/generate-server-key-csr.sh generated vendored Executable file
View File

@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Generate a server key and CSR for specified CN
set -euf -o pipefail
OUTDIR="/root/ca"
SERVER_CERT_CN="starkenterprises.io"
while getopts ":d:n:" opt; do
case $opt in
d) OUTDIR="$OPTARG"
;;
n) SERVER_CERT_CN="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
cd $OUTDIR
# Generate server key
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out private/${SERVER_CERT_CN}.key.pem 4096
chmod 400 private/${SERVER_CERT_CN}.key.pem
# Generate server CSR
openssl req -config openssl.cnf \
-new -sha256 \
-key private/${SERVER_CERT_CN}.key.pem \
-out csr/${SERVER_CERT_CN}.csr.pem \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Web Services/CN=${SERVER_CERT_CN}"

146
vendor/github.com/vmware/vic/infra/integration-image/scripts/intermediate-ca-test.sh generated vendored Executable file
View File

@@ -0,0 +1,146 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Generate a root CA, intermediate CA, and a server certificate
##### Configuration options
SERVER_CERT_CN="starkenterprises.io"
#####
while getopts ":s:" opt; do
case $opt in
s) SERVER_CERT_CN="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
### Create root CA
mkdir -p /root/ca
cp openssl.cnf /root/ca
cd /root/ca
mkdir certs crl csr newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial
# Generate root CA key
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out private/ca.key.pem 4096
chmod 400 private/ca.key.pem
# Generate root CA CSR
openssl req -config openssl.cnf \
-new -sha256 \
-key private/ca.key.pem \
-out csr/ca.csr.pem \
-extensions v3_ca \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Certificate Authority/CN=Stark Enterprises Global CA"
# Self sign for root CA certificate
openssl x509 -req -extfile openssl.cnf \
-extensions v3_ca \
-days 7300 -in csr/ca.csr.pem -signkey private/ca.key.pem -out certs/ca.cert.pem
chmod 444 certs/ca.cert.pem
openssl x509 -noout -text -in certs/ca.cert.pem
# Output CRT format
openssl x509 -in certs/ca.cert.pem -inform PEM -out certs/ca.cert.crt
### Create intermediate CA
cd -
mkdir -p /root/ca/intermediate
cp openssl-intermediate.cnf /root/ca/intermediate
cd /root/ca/intermediate
mkdir certs crl csr newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial
echo 1000 > crlnumber
# Generate intermediate CA key
cd /root/ca
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out intermediate/private/intermediate.key.pem 4096
chmod 400 intermediate/private/intermediate.key.pem
# Generate intermediate CA CSR
openssl req -config intermediate/openssl-intermediate.cnf\
-new -sha256 \
-key intermediate/private/intermediate.key.pem \
-out intermediate/csr/intermediate.csr.pem \
-extensions v3_intermediate_ca \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Certificate Authority/CN=Stark Enterprises Intermediate CA"
# Sign CSR with root CA key
openssl ca -config openssl.cnf \
-batch \
-extensions v3_intermediate_ca \
-days 3650 -notext -md sha256 \
-in intermediate/csr/intermediate.csr.pem \
-out intermediate/certs/intermediate.cert.pem
chmod 444 intermediate/certs/intermediate.cert.pem
openssl x509 -noout -text -in intermediate/certs/intermediate.cert.pem
# Create certificate chain
cat intermediate/certs/intermediate.cert.pem \
certs/ca.cert.pem > intermediate/certs/ca-chain.cert.pem
chmod 444 intermediate/certs/ca-chain.cert.pem
# Output CRT format
openssl x509 -in intermediate/certs/intermediate.cert.pem -inform PEM -out intermediate/certs/intermediate.cert.crt
### Create server certificate
cd /root/ca
# Generate server key
# Private key is not encrypted - use -aes256 to specify a password
openssl genrsa -out intermediate/private/${SERVER_CERT_CN}.key.pem 4096
chmod 400 intermediate/private/${SERVER_CERT_CN}.key.pem
# Generate server CSR
openssl req -config intermediate/openssl-intermediate.cnf \
-new -sha256 \
-key intermediate/private/${SERVER_CERT_CN}.key.pem \
-out intermediate/csr/${SERVER_CERT_CN}.csr.pem \
-subj "/C=US/ST=California/L=Los Angeles/O=Stark Enterprises/OU=Stark Enterprises Web Services/CN=${SERVER_CERT_CN}"
# Sign CSR with intermediate CA key, output server certificate
openssl ca -config intermediate/openssl-intermediate.cnf \
-batch \
-extensions server_cert \
-days 365 -notext -md sha256 \
-in intermediate/csr/${SERVER_CERT_CN}.csr.pem \
-out intermediate/certs/${SERVER_CERT_CN}.cert.pem
chmod 444 intermediate/certs/${SERVER_CERT_CN}.cert.pem
openssl x509 -noout -text -in intermediate/certs/${SERVER_CERT_CN}.cert.pem
# Test certificate
openssl verify -CAfile intermediate/certs/ca-chain.cert.pem intermediate/certs/${SERVER_CERT_CN}.cert.pem
### Bundle output
mkdir bundle
cp certs/ca.cert.crt bundle
cp intermediate/certs/intermediate.cert.crt bundle
cp intermediate/certs/ca-chain.cert.pem bundle
cp intermediate/private/${SERVER_CERT_CN}.key.pem bundle
cp intermediate/certs/${SERVER_CERT_CN}.cert.pem bundle
tar cvf cert-bundle.tgz bundle

132
vendor/github.com/vmware/vic/infra/integration-image/scripts/openssl-intermediate.cnf generated vendored Normal file
View File

@@ -0,0 +1,132 @@
# OpenSSL intermediate CA configuration file.
# Copy to `/root/ca/intermediate/openssl.cnf`.
[ ca ]
# `man ca`
default_ca = CA_default
[ CA_default ]
# Directory and file locations.
dir = /root/ca/intermediate
certs = $dir/certs
crl_dir = $dir/crl
new_certs_dir = $dir/newcerts
database = $dir/index.txt
serial = $dir/serial
RANDFILE = $dir/private/.rand
# The root key and root certificate.
private_key = $dir/private/intermediate.key.pem
certificate = $dir/certs/intermediate.cert.pem
# For certificate revocation lists.
crlnumber = $dir/crlnumber
crl = $dir/crl/intermediate.crl.pem
crl_extensions = crl_ext
default_crl_days = 30
# SHA-1 is deprecated, so use SHA-2 instead.
default_md = sha256
name_opt = ca_default
cert_opt = ca_default
default_days = 375
preserve = no
policy = policy_loose
[ policy_strict ]
# The root CA should only sign intermediate certificates that match.
# See the POLICY FORMAT section of `man ca`.
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
[ policy_loose ]
# Allow the intermediate CA to sign a more diverse range of certificates.
# See the POLICY FORMAT section of the `ca` man page.
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
[ req ]
# Options for the `req` tool (`man req`).
default_bits = 2048
distinguished_name = req_distinguished_name
string_mask = utf8only
# SHA-1 is deprecated, so use SHA-2 instead.
default_md = sha256
# Extension to add when the -x509 option is used.
x509_extensions = v3_ca
[ req_distinguished_name ]
# See <https://en.wikipedia.org/wiki/Certificate_signing_request>.
countryName = Country Name (2 letter code)
stateOrProvinceName = State or Province Name
localityName = Locality Name
0.organizationName = Organization Name
organizationalUnitName = Organizational Unit Name
commonName = Common Name
emailAddress = Email Address
# Optionally, specify some defaults.
countryName_default = US
stateOrProvinceName_default =
localityName_default =
0.organizationName_default =
organizationalUnitName_default =
emailAddress_default =
[ v3_ca ]
# Extensions for a typical CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
[ v3_intermediate_ca ]
# Extensions for a typical intermediate CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
[ usr_cert ]
# Extensions for client certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, email
nsComment = "OpenSSL Generated Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, emailProtection
[ server_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
[ crl_ext ]
# Extension for CRLs (`man x509v3_config`).
authorityKeyIdentifier=keyid:always
[ ocsp ]
# Extension for OCSP signing certificates (`man ocsp`).
basicConstraints = CA:FALSE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, digitalSignature
extendedKeyUsage = critical, OCSPSigning

132
vendor/github.com/vmware/vic/infra/integration-image/scripts/openssl.cnf generated vendored Normal file
View File

@@ -0,0 +1,132 @@
# OpenSSL root CA configuration file.
# Copy to `/root/ca/openssl.cnf`.
[ ca ]
# `man ca`
default_ca = CA_default
[ CA_default ]
# Directory and file locations.
dir = /root/ca
certs = $dir/certs
crl_dir = $dir/crl
new_certs_dir = $dir/newcerts
database = $dir/index.txt
serial = $dir/serial
RANDFILE = $dir/private/.rand
# The root key and root certificate.
private_key = $dir/private/ca.key.pem
certificate = $dir/certs/ca.cert.pem
# For certificate revocation lists.
crlnumber = $dir/crlnumber
crl = $dir/crl/ca.crl.pem
crl_extensions = crl_ext
default_crl_days = 30
# SHA-1 is deprecated, so use SHA-2 instead.
default_md = sha256
name_opt = ca_default
cert_opt = ca_default
default_days = 375
preserve = no
policy = policy_strict
[ policy_strict ]
# The root CA should only sign intermediate certificates that match.
# See the POLICY FORMAT section of `man ca`.
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
[ policy_loose ]
# Allow the intermediate CA to sign a more diverse range of certificates.
# See the POLICY FORMAT section of the `ca` man page.
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
[ req ]
# Options for the `req` tool (`man req`).
default_bits = 2048
distinguished_name = req_distinguished_name
string_mask = utf8only
# SHA-1 is deprecated, so use SHA-2 instead.
default_md = sha256
# Extension to add when the -x509 option is used.
x509_extensions = v3_ca
[ req_distinguished_name ]
# See <https://en.wikipedia.org/wiki/Certificate_signing_request>.
countryName = Country Name (2 letter code)
stateOrProvinceName = State or Province Name
localityName = Locality Name
0.organizationName = Organization Name
organizationalUnitName = Organizational Unit Name
commonName = Common Name
emailAddress = Email Address
# Optionally, specify some defaults.
countryName_default = US
stateOrProvinceName_default =
localityName_default =
0.organizationName_default =
organizationalUnitName_default =
emailAddress_default =
[ v3_ca ]
# Extensions for a typical CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
[ v3_intermediate_ca ]
# Extensions for a typical intermediate CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
[ usr_cert ]
# Extensions for client certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, email
nsComment = "OpenSSL Generated Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, emailProtection
[ server_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
[ crl_ext ]
# Extension for CRLs (`man x509v3_config`).
authorityKeyIdentifier=keyid:always
[ ocsp ]
# Extension for OCSP signing certificates (`man ocsp`).
basicConstraints = CA:FALSE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, digitalSignature
extendedKeyUsage = critical, OCSPSigning

51
vendor/github.com/vmware/vic/infra/integration-image/scripts/sign-csr.sh generated vendored Executable file
View File

@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Sign a CSR with specified CA
set -euf -o pipefail
CA_NAME="STARK_ENTERPRISES_ROOT_CA" # used to verify cert signature
OUTDIR="/root/ca"
SERVER_CERT_CN="starkenterprises.io"
while getopts ":c:d:n:" opt; do
case $opt in
c) CA_NAME="$OPTARG"
;;
d) OUTDIR="$OPTARG"
;;
n) SERVER_CERT_CN="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
CONF_DIR=`dirname $0`
cd $OUTDIR
openssl ca -config $CONF_DIR/openssl.cnf \
-batch \
-extensions server_cert \
-days 365 -notext -md sha256 \
-in csr/${SERVER_CERT_CN}.csr.pem \
-out certs/${SERVER_CERT_CN}.cert.pem
chmod 444 certs/${SERVER_CERT_CN}.cert.pem
openssl x509 -noout -text -in certs/${SERVER_CERT_CN}.cert.pem
# Test certificate
openssl verify -CAfile certs/$CA_NAME.crt certs/${SERVER_CERT_CN}.cert.pem

31
vendor/github.com/vmware/vic/infra/integration-image/scripts/ubuntu-install-ca.sh generated vendored Executable file
View File

@@ -0,0 +1,31 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Install CA into root store
CERT_FILE="/root/ca/certs/STARK_ENTERPRISES_ROOT_CA.crt"
while getopts ":f:" opt; do
case $opt in
f) CERT_FILE="$OPTARG"
;;
\?) echo "Invalid option -$OPTARG" >&2
;;
esac
done
cp $CERT_FILE /usr/local/share/ca-certificates
dpkg-reconfigure --frontend=noninteractive ca-certificates

18
vendor/github.com/vmware/vic/infra/integration-image/scripts/ubuntu-install-intermediate-ca.sh generated vendored Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
cp /root/ca/intermediate/certs/intermediate.cert.crt /usr/local/share/ca-certificates
dpkg-reconfigure --frontend=noninteractive ca-certificates

18
vendor/github.com/vmware/vic/infra/integration-image/scripts/ubuntu-reload-cas.sh generated vendored Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Reload default CAs to remove ALL user installed root certificates
update-ca-certificates --fresh

18
vendor/github.com/vmware/vic/infra/integration-image/scripts/ubuntu-remove-intermediate-ca.sh generated vendored Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
rm /usr/local/share/ca-certificates/intermediate.cert.crt
dpkg-reconfigure --frontend=noninteractive ca-certificates

1
vendor/github.com/vmware/vic/infra/machines/devbox/.gitignore generated vendored Normal file
View File

@@ -0,0 +1 @@
*.vmdk

143
vendor/github.com/vmware/vic/infra/machines/devbox/README.md generated vendored Normal file
View File

@@ -0,0 +1,143 @@
# Vagrant Dev Box
## Overview
This box is an Ubuntu 16.04 VM with the following setup by default:
* Docker daemon with port forwarded to the Fusion/Workstation host at localhost:12375
* Go toolchain
* Additional tools (lsof, strace, etc)
## Requirements
* Vagrant (https://www.vagrantup.com/downloads.html)
* VMware Fusion or Workstation
* Vagrant Fusion or Workstation license (https://www.vagrantup.com/vmware)
## Provisioning
All files matching _provision*.sh_ in this directory will be applied by the Vagrantfile, you can symlink custom scripts
if needed. The scripts are not Vagrant specific and can be applied to a VM running on ESX for example.
## Fusion/Workstation host usage
The following commands can be used from your Fusion or Workstation host.
### Shared Folders
By default your *GOPATH* is shared with the same path as the host. This is useful if your editor runs
on the host, then errors on the guest with filename:line info have the same path. For example, when running the
following command within the top-level project directory:
``` shell
vagrant ssh -- make -C $PWD all
```
### Create the VM
``` shell
vagrant up
```
### SSH Access
``` shell
vagrant ssh
```
### Docker Access
``` shell
DOCKER_HOST=localhost:12375 docker ps
```
### Stop the VM
``` shell
vagrant halt
```
### Restart the VM
``` shell
vagrant reload
```
### Provision
After you've done a `vagrant up`, the provisioning can be applied without reloading via:
``` shell
vagrant provision
```
### Delete the VM
``` shell
vagrant destroy
```
## VM guest usage
To open a bash term in the VM, use `vagrant ssh`.
The following commands can be used from devbox VM guest.
``` shell
cd $GOPATH/src/github.com/vmware/vic
```
### Local Drone CI test
``` shell
drone exec
```
## Devbox on ESX
The devbox can be deployed to ESX, the same provisioning scripts are applied:
``` shell
./deploy-esx.sh
```
### SSH access
``` shell
ssh-add ~/.vagrant.d/insecure_private_key
vmip=$(govc vm.ip $USER-ubuntu-1604)
ssh vagrant@$vmip
```
### Shared folders
You can share your folder by first exporting via NFS:
```
echo "$HOME/vic $(govc vm.ip $USER-ubuntu-1604) -alldirs -mapall=$(id -u):$(id -g)" | sudo tee -a /etc/exports
sudo nfsd restart
```
Then mount within the ubuntu VM:
``` shell
ssh vagrant@$vmip sudo mkdir -p $HOME/vic
ssh vagrant@$vmip sudo mount $(ipconfig getifaddr en1):$HOME/vic $HOME/vic
```
Note that you may need to use enN depending on the type of connection you have - use ifconfig to verify.
Note also that nfs-common is not installed in the box by default.
You can also mount your folder within ESX:
``` shell
govc datastore.create -type nfs -name nfsDatastore -remote-host $(ipconfig getifaddr en1) -remote-path $HOME/vic
esxip=$(govc host.info -json | jq -r '.HostSystems[].Config.Network.Vnic[] | select(.Device == "vmk0") | .Spec.Ip.IpAddress')
ssh root@$esxip mkdir -p $HOME
ssh root@$esxip /vmfs/volumes/nfsDatastore $HOME/vic
```
Add `$esxip` to /etc/exports and restart nfsd again.

90
vendor/github.com/vmware/vic/infra/machines/devbox/deploy-esx.sh generated vendored Executable file
View File

@@ -0,0 +1,90 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Deploy Vagrant box to esx
set -e
if [ "$(uname -s)" = "Darwin" ]; then
PATH="/Applications/VMware Fusion.app/Contents/Library:$PATH"
fi
export GOVC_URL=${GOVC_URL-"root:vagrant@localhost:18443"}
export GOVC_DATASTORE=${GOVC_DATASTORE-"datastore1"}
export GOVC_NETWORK=${GOVC_NETWORK-"VM Network"}
export GOVC_INSECURE=1
echo "deploying to $(awk -F@ '{print $2}' <<<"$GOVC_URL"):"
govc about
config="$(git rev-parse --show-toplevel)/Vagrantfile"
box=$(grep vic_dev.vm.box "$config" | awk -F\' '{print $2}')
provider=$(dirname "$box")
name=$(basename "$box")
disk="${name}.vmdk"
pushd "$(dirname "$0")" >/dev/null
if ! govc datastore.ls "${name}/${disk}" 1>/dev/null 2>&1 ; then
if [ ! -e "$disk" ] ; then
src=$(echo ~/.vagrant.d/boxes/"${provider}"-*-"${name}"/*.*.*/vmware_desktop/disk.vmdk)
if [ ! -e "$src" ] ; then
echo "box not found, install via: vagrant box add --provider vmware_desktop $box"
exit 1
fi
echo "converting vagrant box for ESX..."
vmware-vdiskmanager -r "$src" -t 0 "$disk"
fi
echo "importing vmdk to datastore ${GOVC_DATASTORE}..."
govc import.vmdk "$disk" "$name"
fi
vm_name=${VM_NAME-"${USER}-${name}"}
vm_memory=${VM_MEMORY-$(grep memory "$config" | awk -F\= 'FNR == 1 {gsub(/ /, "", $2); print $2}')}
if [ -z "$(govc ls "vm/$vm_name")" ] ; then
echo "creating VM ${vm_name}..."
govc vm.create -m "$vm_memory" -c 2 -g ubuntu64Guest -disk.controller=pvscsi -on=false "$vm_name"
govc vm.disk.attach -vm "$vm_name" -link=true -disk "$name/$disk"
govc device.cdrom.add -vm "$vm_name"
govc vm.power -on "$vm_name"
fi
# An ipv6 is reported by tools when the machine is first booted.
# Wait until we get an ipv4 address from tools.
while true
do
ip=$(govc vm.ip "$vm_name")
ipv4="${ip//[^.]}"
if [ "${#ipv4}" -eq 3 ]
then
break
fi
sleep 1
done
echo "VM ip=$ip"
for script in provision.sh provision-drone.sh; do
echo "Applying $script..."
ssh -i ~/.vagrant.d/insecure_private_key "vagrant@$ip" sudo bash -s - < "$script"
done

22
vendor/github.com/vmware/vic/infra/machines/devbox/provision-drone.sh generated vendored Executable file
View File

@@ -0,0 +1,22 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This depends on docker being on the box and running
docker pull drone/drone
curl http://downloads.drone.io/0.5.0/release/linux/amd64/drone.tar.gz | tar zx
sudo install -t /usr/local/bin drone

52
vendor/github.com/vmware/vic/infra/machines/devbox/provision.sh generated vendored Executable file
View File

@@ -0,0 +1,52 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# add key for docker repo
apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
# add docker apt sources
echo "deb https://apt.dockerproject.org/repo ubuntu-xenial main" > /etc/apt/sources.list.d/docker.list
# https://github.com/mitchellh/vagrant/issues/289
apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
# set GOPATH based on shared folder of vagrant
pro="/home/${BASH_ARGV[0]}/.profile"
echo "export GOPATH=${BASH_ARGV[1]}" >> "$pro"
# add GOPATH/bin to the PATH
echo "export PATH=$PATH:${BASH_ARGV[1]}/bin" >> "$pro"
apt-get -y install curl lsof strace git shellcheck tree mc silversearcher-ag jq htpdate apt-transport-https ca-certificates nfs-common sshpass
function update_go {
(cd /usr/local &&
(curl --silent -L $go_file | tar -zxf -) &&
ln -fs /usr/local/go/bin/* /usr/local/bin/)
}
# install / upgrade go
go_file="https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz"
go_version=$(basename $go_file | cut -d. -f1-3)
if [[ ! -d "/usr/local/go" || $(go version | awk '{print $(3)}') != "$go_version" ]] ; then
update_go
fi
# Install docker
apt-get -y install linux-image-extra-$(uname -r)
apt-get -y --allow-downgrades install docker-engine=1.13.1-0~ubuntu-xenial
apt-mark hold docker-engine
usermod -aG docker vagrant
systemctl start docker

16
vendor/github.com/vmware/vic/infra/scripts/README.md generated vendored Normal file
View File

@@ -0,0 +1,16 @@
## header-check
Simple header check for CI jobs, currently checks ".go" files only.
This will be called by the CI system (with no args) to perform checking and
fail the job if headers are not correctly set. It can also be called with the
'fix' argument to automatically add headers to the missing files.
Check if headers are fine:
```
$ ./infra/scripts/header-check.sh
```
Check and fix headers:
```
$ ./infra/scripts/header-check.sh fix
```

13
vendor/github.com/vmware/vic/infra/scripts/VMware-copyright generated vendored Normal file
View File

@@ -0,0 +1,13 @@
// Copyright 2018 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

149
vendor/github.com/vmware/vic/infra/scripts/bash-helpers.sh generated vendored Normal file
View File

@@ -0,0 +1,149 @@
#!/bin/bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.!/bin/bash
OS=$(uname | tr '[:upper:]' '[:lower:]')
tls () {
unset TLS_OPTS
}
no-tls () {
export TLS_OPTS="--no-tls"
}
unset-vic () {
unset MAPPED_NETWORKS NETWORKS IMAGE_STORE DATASTORE COMPUTE VOLUME_STORES IPADDR GOVC_INSECURE TLS THUMBPRINT OPS_CREDS VIC_NAME
}
vic-path () {
echo "${GOPATH}/src/github.com/vmware/vic"
}
vic-create () {
base=$(pwd)
(
cd "$(vic-path)"/bin || return
"$(vic-path)"/bin/vic-machine-"$OS" create --target="$GOVC_URL" "${OPS_CREDS[@]}" --image-store="$IMAGE_STORE" --compute-resource="$COMPUTE" "${TLS[@]}" ${TLS_OPTS} --name="${VIC_NAME:-${USER}test}" "${MAPPED_NETWORKS[@]}" "${VOLUME_STORES[@]}" "${NETWORKS[@]}" ${IPADDR} ${TIMEOUT} --thumbprint="$THUMBPRINT" "$@"
)
unset DOCKER_CERT_PATH DOCKER_TLS_VERIFY
unalias docker 2>/dev/null
envfile=$(vic-path)/bin/${VIC_NAME:-${USER}test}/${VIC_NAME:-${USER}test}.env
if [ -f "$envfile" ]; then
set -a
source "$envfile"
set +a
fi
# Something of a hack, but works for --no-tls so long as that's enabled via TLS_OPTS
if [ -z "${DOCKER_TLS_VERIFY+x}" ] && [ -z "${TLS_OPTS+x}" ]; then
alias docker='docker --tls'
fi
cd "$base" || exit
}
vic-delete () {
"$(vic-path)"/bin/vic-machine-"$OS" delete --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --thumbprint="$THUMBPRINT" --force "$@"
}
vic-inspect () {
"$(vic-path)"/bin/vic-machine-"$OS" inspect --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --thumbprint="$THUMBPRINT" "$@"
}
vic-upgrade () {
"$(vic-path)"/bin/vic-machine-"$OS" upgrade --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --thumbprint="$THUMBPRINT" "$@"
}
vic-ls () {
"$(vic-path)"/bin/vic-machine-"$OS" ls --target="$GOVC_URL" --thumbprint="$THUMBPRINT" "$@"
}
vic-ssh () {
unset keyarg
if [ -e "$HOME"/.ssh/authorized_keys ]; then
keyarg="--authorized-key=$HOME/.ssh/authorized_keys"
fi
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | awk -F"DOCKER_HOST=" '{print $2}' | cut -d ":" -f1 | cut -d "=" -f2)
echo "SSH to ${host}"
sshpass -ppassword ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@"${host}"
}
vic-admin () {
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | sed -n 's/.*DOCKER_HOST=\([^:\s*\).*/\1/p')
open http://"${host}":2378
}
addr-from-dockerhost () {
echo "$DOCKER_HOST" | sed -e 's/:[0-9]*$//'
}
vic-tail-portlayer() {
unset keyarg
if [ -e "$HOME"/.ssh/authorized_keys ]; then
keyarg="--authorized-key=$HOME/.ssh/authorized_keys"
fi
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | awk -F"DOCKER_HOST=" '{print $2}' | cut -d ":" -f1 | cut -d "=" -f2)
sshpass -ppassword ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@"${host}" tail -f /var/log/vic/port-layer.log
}
vic-tail-docker() {
unset keyarg
if [ -e "$HOME"/.ssh/authorized_keys ]; then
keyarg="--authorized-key=$HOME/.ssh/authorized_keys"
fi
out=$("$(vic-path)"/bin/vic-machine-"$OS" debug --target="$GOVC_URL" --compute-resource="$COMPUTE" --name="${VIC_NAME:-${USER}test}" --enable-ssh "$keyarg" --rootpw=password --thumbprint="$THUMBPRINT" "$@")
host=$(echo "$out" | grep DOCKER_HOST | awk -F"DOCKER_HOST=" '{print $2}' | cut -d ":" -f1 | cut -d "=" -f2)
sshpass -ppassword ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@"${host}" tail -f /var/log/vic/docker-personality.log
}
# import the custom sites
# example entry, actived by typing "example"
# those variales that hold multiple arguments which may contain spaces are arrays to allow for proper quoting
#example () {
# target='https://user:password@host.domain.com/datacenter'
# unset-vic
#
# export GOVC_URL=$target
#
# eval "export THUMBPRINT=$(govc about.cert -k -json | jq -r .ThumbprintSHA1)"
# export COMPUTE=cluster/pool
# export DATASTORE=datastore1
# export IMAGE_STORE=$DATASTORE/image/path
# export TIMEOUT="--timeout=10m"
# export IPADDR="--client-network-ip=vch-hostname.domain.com --client-network-gateway=x.x.x.x/22 --dns-server=y.y.y.y --dns-server=z.z.z.z"
# export VIC_NAME="MyVCH"
#
# TLS=("--tls-cname=vch-hostname.domain.com" "--organization=MyCompany")
# OPS_CREDS=("--ops-user=<user>" "--ops-password=<password>")
# NETWORKS=("--bridge-network=private-dpg-vlan" "--public-network=extern-dpg")
# MAPPED_NETWORKS=("--container-network=VM Network:external" "--container-network=SomeOtherNet:elsewhere")
# VOLUME_STORES=("--volume-store=$DATASTORE:default")
#
# export NETWORKS MAPPED_NETWORKS VOLUME_STORES OPS_CREDS TLS
#}
. ~/.vic

134
vendor/github.com/vmware/vic/infra/scripts/ci-logs.sh generated vendored Executable file
View File

@@ -0,0 +1,134 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Requirements
# ############
# - gcloud SDK: https://cloud.google.com/sdk/docs/
# - drone cli 0.5: https://github.com/drone/drone#from-source
#
# Examples
# ########
# Grab the logs for your current branch:
# $ ./ci-logs.sh
#
# If the build is running you can stream the logs using the '-s' option:
# $ ./ci-logs.sh -s
#
# Grab logs for a specific build:
# $ ./ci-logs.sh 4835
#
# Grab logs for recent failures on the master branch:
# $ drone build list --format {{.Number}}-{{.Branch}}-{{.Status}} vmware/vic | grep master-failure | cut -d- -f1 | xargs -n1 ./ci-logs.sh
#
# Find container log zip files for failed tests:
# $ grep FAIL ci.log | grep Test-Cases.Group | grep :: | awk '{print $1}' | xargs -n1 -I% bash -c "ls %*.zip"
top=$(git rev-parse --show-toplevel)
repo="vmware/$(basename "$top")"
dir="$top/ci-logs"
drone=${DRONE_CLI:-drone}
while getopts s flag
do
case $flag in
d)
dir="$OPTARG"
;;
s)
stream=true
;;
*)
echo "invalid option '$flag'"
exit 1
;;
esac
done
shift $((OPTIND-1))
build="$1"
job="$2"
export DRONE_SERVER=${DRONE_SERVER:-https://ci-vic.vmware.com}
if [ -z "$DRONE_TOKEN" ] ; then
echo "DRONE_TOKEN not set (available at $DRONE_SERVER/settings/profile)"
fi
if [ -z "$build" ] ; then
commit=$(git rev-parse HEAD)
builds=$($drone build list --format "{{.Number}}-{{.Commit}}" "$repo" | grep "$commit")
build=$(cut -d- -f1 <<<"$builds")
fi
if [ -z "$job" ] ; then
job=1
fi
state=$($drone build info "$repo" "$build" --format {{.Status}})
echo "$state"
case "$state" in
running)
if [ -z "$stream" ] ; then
exit
fi
;;
success|failure|killed)
stream=""
;;
pending|error)
exit
;;
esac
if [ ! -d "$dir" ] ; then
mkdir "$dir"
fi
logs="$dir/$build"
mkdir "$logs"
if [ -n "$stream" ] ; then
echo "Streaming CI log..."
curl --silent "$DRONE_SERVER/api/stream/$repo/$build/$job?access_token=$DRONE_TOKEN" | \
grep data: | cut -d: -f2- | grep -v '^$' | tee "$logs/ci.log"
else
echo "Downloading CI log..."
curl --silent "$DRONE_SERVER/api/repos/$repo/logs/$build/$job?access_token=$DRONE_TOKEN" > "$logs/ci.log"
fi
url=$(grep https://console.cloud.google.com/ "$logs/ci.log")
if [ -z "$url" ] ; then
echo "No integration logs link found for build ${build}"
exit
fi
name=$(basename "$url" | cut -d? -f1)
bucket=$(basename "$(dirname "$(dirname "$url")")")
echo "Downloading integration logs ($name)..."
gsutil cp "gs://$bucket/$name" "$logs/$name"
unzip -d "$logs" "$logs/$name" >/dev/null
if [ "$state" = "failure" ] ; then
echo "Container logs for failed tests:"
grep FAIL "$logs/ci.log" | grep Test-Cases.Group | grep :: | awk '{print $1}' | xargs -n1 -I% bash -c "ls $logs/%*.zip" 2>/dev/null
fi

78
vendor/github.com/vmware/vic/infra/scripts/coverage.sh generated vendored Executable file
View File

@@ -0,0 +1,78 @@
#!/bin/bash -e
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Works around the fact that `go test -coverprofile` does not work
# with multiple packages, see https://code.google.com/p/go/issues/detail?id=6909
#
# Usage: script/coverage [--html]
#
# --html Create HTML report and open it in browser
#
workdir=`git rev-parse --show-toplevel`/.cover
profile="$workdir/cover.out"
dir=$(dirname $0)
mode=count
# list any files (or patterns) to explicitly exclude from coverage
# you should have a pretty good reason before putting items here
exclude_files=(
)
join() { local IFS="$1"; shift; echo "$*"; }
excludes=$(join "|" ${exclude_files[@]} | sed -e 's/\./\\./g')
generate_pkg_cover_data() {
echo "$@"
mkdir -p "$workdir"
for pkg in "$@"; do
f="$workdir/$(echo $pkg | tr / -).cover"
go test -i -covermode="$mode" -coverprofile="$f" "$pkg"
go test -covermode="$mode" -coverprofile="$f" "$pkg"
done
echo "mode: $mode" >"$profile"
if [ -n "$excludes" ]; then
grep -h -v "^mode:" "$workdir"/*.cover | egrep -v "$excludes" >>"$profile"
else
grep -h -v "^mode:" "$workdir"/*.cover >>"$profile"
fi
}
# translate dirs to packages and strip args
dir_to_pkg() {
for dir in $@; do
if test "$dir" == "--html"; then
export html="true"
else
pkgs="$pkgs $(go list $dir/... | grep -v /vendor/)"
fi
done
echo $pkgs
}
show_cover_report() {
go tool cover -${1}="$profile"
}
generate_pkg_cover_data $(dir_to_pkg "$@")
show_cover_report func
if test "$html" == "true"; then
show_cover_report html
fi

43
vendor/github.com/vmware/vic/infra/scripts/focused-test.sh generated vendored Executable file
View File

@@ -0,0 +1,43 @@
#!/bin/bash
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Runs go test on any packages that differ from vmware/master that have
# Go test files.
#
# Usage: infra/scripts/focused-test.sh
#
if [ -z "$1" ]; then
export REMOTE=$(git remote -v | grep "github.com/vmware/vic.git (fetch)" | awk '{print$1;exit}')/master
echo "Using ${REMOTE} as default remote"
else
echo "Using ${REMOTE} as specified remote"
export REMOTE="$1"
fi
echo "Finding modified packages with test files"
PKG=$(git diff --stat ${REMOTE} --name-only | xargs dirname | uniq | while read i; do
for f in ./$i/*_test.go; do
if [ -e $f ]; then
echo -n "./$i ";
fi;
break;
done;
done)
echo "Testing packages: $PKG"
if [ -n "$PKG" ]; then
go test $PKG;
fi

58
vendor/github.com/vmware/vic/infra/scripts/go-deps.sh generated vendored Executable file
View File

@@ -0,0 +1,58 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Lists the non-standard library Go packages the specified package depends
# on.
#
# Usage: script/go-deps.sh pkg
#
# pkg This is github.com/vmware/vic/cmd/imagec for example
#
# If VIC_CACHE_DEPS environment variable is defined, this script will attempt to read
# cached dependencies from disk if those exist. If they are not cached, dependencies will be
# regenerated and cached.
cache_dir=.godeps_cache
pkg=$1
flags=$2
cachedname=`echo .$1.godeps_cache | sed 's/\//_/g'`
if [ -d "$pkg" ]; then
if [[ "$flags" == *d* ]]; then
# Only output if make is given the '-d' flag
echo "Generating deps for $pkg" >&2
fi
if [ -n "$VIC_CACHE_DEPS" ]; then
mkdir -p $cache_dir
if [ ! -f $cache_dir/$cachedname ]; then
go list -f '{{join .Deps "\n"}}' github.com/vmware/vic/"$pkg" 2>/dev/null | \
xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' 2>/dev/null | \
sed -e 's:github.com/vmware/vic/\(.*\)$:\1/*:' > "$cache_dir/$cachedname"
fi
cat "$cache_dir/$cachedname"
else
go list -f '{{join .Deps "\n"}}' github.com/vmware/vic/"$pkg" 2>/dev/null | \
xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' 2>/dev/null | \
sed -e 's:github.com/vmware/vic/\(.*\)$:\1/*:'
fi
else
if [[ "$flags" == *d* ]]
then
echo "$0: package '$pkg' does not exist" >&2
fi
fi

120
vendor/github.com/vmware/vic/infra/scripts/header-check.sh generated vendored Executable file
View File

@@ -0,0 +1,120 @@
#!/bin/bash
# Copyright 2016-2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Simple script that will check files of type .go, .sh, .bash, .robot, or Makefile
# for the copyright header.
#
# This will be called by the CI system (with no args) to perform checking and
# fail the job if headers are not correctly set. It can also be called with the
# 'fix' argument to automatically add headers to the missing files.
#
# Check if headers are fine:
# $ ./scripts/header-check.sh
# Check and fix headers:
# All changes must be committed for fix to work
# $ ./scripts/header-check.sh fix
set -e -o pipefail
# These header variables MUST match the first two lines of the
# VMware-copyright file in the scripts directory.
#
# These will be evaluated as a regex against the target file
HEADER[1]="^\/\/ Copyright [0-9]{4}(-[0-9]{4})? VMware, Inc\. All Rights Reserved\.$"
HEADER[2]="^\/\/$"
# Initialize vars
ERR=false
FAIL=false
all-files() {
git ls-files |\
# Check .go files, Makefile, sh files, bash files, and robot files
grep -e "\.go$" -e "Makefile$" -e "\.sh$" -e "\.bash$" -e "\.robot$" |\
# Ignore vendor/
grep -v vendor/ |\
grep -v lib/apiservers/portlayer/client | grep -v lib/apiservers/portlayer/models
}
for file in $(all-files); do
# get the file extension / type
ext=${file##*.}
# increment line count in certain cases
increment=false
# should we be incrementing the line count
if [[ $ext == "sh" ]]; then
increment=true
fi
for count in $(seq 1 ${#HEADER[@]}); do
if [[ $ext != "go" ]]; then
# if not go code assuming # will suffice
text="${HEADER[$count]/'\/\/'/#}"
else
text=${HEADER[$count]}
fi
if [[ "$increment" = true ]]; then
line=$((count + 1))
else
line=$count
fi
# do we have a header match?
if [[ ! $(sed ${line}q\;d ${file}) =~ ${text} ]]; then
ERR=true
fi
done
if [ $ERR == true ]; then
# is there is a fix argument and are all changes committed
if [[ $# -gt 0 && $1 =~ [[:upper:]fix] ]]; then
# based on file type fix the copyright
case "$ext" in
go)
cat $(dirname $0)/VMware-copyright ${file} > ${file}.new
;;
sh)
head -1 ${file} > ${file}.new
cat $(dirname $0)/VMware-copyright | sed 's/\/\//\#/1' >> ${file}.new
grep -v '#!/bin/bash' ${file} >> ${file}.new
;;
*)
cat $(dirname $0)/VMware-copyright | sed 's/\/\//\#/1' > ${file}.new
cat ${file} >> ${file}.new
;;
esac
if [ "$(uname -s)" = "Darwin" ]; then
permissions=$(stat -f "%OLp" ${file})
else
permissions=$(stat --format '%a' ${file})
fi
mv ${file}.new ${file}
# make permissions the same
chmod $permissions ${file}
echo "$file.. $(tput -T xterm setaf 3)FIXING$(tput -T xterm sgr0)"
ERR=false
else
echo "$file.. $(tput -T xterm setaf 1)FAIL$(tput -T xterm sgr0)"
ERR=false
FAIL=true
fi
fi
done
# If we failed one check, return 1
[ $FAIL == true ] && exit 1 || exit 0

215
vendor/github.com/vmware/vic/infra/scripts/local-ci.sh generated vendored Executable file
View File

@@ -0,0 +1,215 @@
#!/bin/bash
# Copyright 2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
##### defaults
secretsfile=""
docker_test="Group1-Docker-Commands"
target_vch=""
odir="ci-results"
ci_container="gcr.io/eminent-nation-87317/vic-integration-test:1.44"
github_api_key=""
test_url=""
test_datastore=""
test_username=""
test_password=""
BASE_DIR=$(dirname $(readlink -f "$BASH_SOURCE"))
vic_dir=${BASE_DIR}/../../
##### utility functions
function usage() {
echo "Usage: $0 [options]" 1>&2
echo
echo " Options can be provided by commandline argument, environment variable, or a secrets yaml file containing the"
echo " variables. If a secrets file is not provided, this script will attempt to retrieve some info from govc, such as"
echo " TEST_URL, TEST_USERNAME, and TEST_PASSWORD."
echo
echo " options:"
echo " -t DOCKER_TEST (or env var)"
echo " -v TARGET_VCH (or env var) name of VCH"
echo " -f SECRETS_FILE (or env var)"
echo " -g GITHUB_API_KEY (or env var)"
echo " -u TEST_URL (or env var or in secretsfile)"
echo " -s TEST_DATASTORE (or env var or in secretsfile)"
echo " -n TEST_USERNAME (or env var or in secretsfile)"
echo " -p TEST_PASSWORD (or env var or in secretsfile)"
echo " -d debug dumps out all the inputs and results of the resulting options"
echo
echo " example:"
echo " $0 -t Group1-Docker-Commands -v my_vch -s test.secrets.nested -g xxxxxx"
echo " $0 -t 1-01-Docker-Info.robot -v my_vch -s test.secrets.nested -g xxxxxx"
echo
echo " DOCKER_TEST=Group1-Docker-Commands/1-01-Docker-Info.robot TARGET_VCH=my_vch SECRETS_FILE=test.secrets.nested $0"
echo
echo " $0 -s test.secrets.nested (all params defined in secrets file)"
exit 1
}
function GetGovcParamsFromEnv() {
echo "Getting params from GOVC var"
test_username=$(govc env | grep GOVC_USERNAME | cut -d= -f2)
test_password=$(govc env | grep GOVC_PASSWORD | cut -d= -f2)
test_url=$(govc env | grep GOVC_URL | cut -d= -f2)
}
function GetParamsFromSecrets() {
echo "Getting params from the secrets file"
secrets_api_key="$(grep 'GITHUB_AUTOMATION_API_KEY' ${secretsfile} | awk '{ print $2 }')"
secrets_url="$(grep 'TEST_URL_ARRAY' ${secretsfile} | awk '{ print $2 }')"
secrets_datastore="$(grep -E '\s+TEST_DATASTORE' ${secretsfile} | awk '{ print $2 }')"
secrets_username="$(grep 'TEST_USERNAME' ${secretsfile} | awk '{ print $2 }')"
secrets_password="$(grep 'TEST_PASSWORD' ${secretsfile} | awk '{ print $2 }')"
secrets_docker_test="$(grep 'DOCKER_TEST' ${secretsfile} | awk '{ print $2 }')"
secrets_target_vch="$(grep 'TARGET_VCH' ${secretsfile} | awk '{ print $2 }')"
}
function DoWork() {
mkdir -p $odir
testsContainer=$(docker create -it \
-w /vic \
-v "$vic_dir:/vic" \
-e GOVC_URL="$ip" \
-e GOVC_INSECURE=1 \
-e GITHUB_AUTOMATION_API_KEY=${github_api_key}\
-e TEST_URL_ARRAY=${test_url}\
-e TEST_DATASTORE=${test_datastore}\
-e TEST_USERNAME=${test_username}\
-e TEST_PASSWORD=${test_password}\
-e TARGET_VCH=${target_vch}\
-e DEBUG_VCH=1\
${ci_container}\
bash -c "pybot -d /vic/${odir} /vic/tests/test-cases/"$docker_test"")
docker start -ai $testsContainer
}
function DebugInputDump() {
echo "Environment Variables"
echo "---------------------"
echo "SECRETS_FILE="${SECRETS_FILE}
echo "TARGET_VCH="${TARGET_VCH}
echo "DOCKER_TEST="${DOCKER_TEST}
echo "GITHUB_API_KEY="${GITHUB_API_KEY}
echo "TEST_URL="${TEST_URL}
echo "TEST_DATASTORE"=${TEST_DATASTORE}
echo "TEST_USERNAME"=${TEST_USERNAME}
echo "TEST_PASSWORD"=${TEST_PASSWORD}
echo
echo "Arguments"
echo "---------------------"
echo "SECRETS_FILE="$secretsfile
echo "TARGET_VCH="$target_vch
echo "DOCKER_TEST="$docker_Test
echo "GITHUB_API_KEY="$github_api_key
echo "TEST_URL="$test_url
echo "TEST_DATASTORE"=$test_datastore
echo "TEST_USERNAME"=$test_username
echo "TEST_PASSWORD"=$test_password
echo
echo "Secrets file"
echo "---------------------"
echo "TARGET_VCH="$secrets_target_vch
echo "DOCKER_TEST="$secrets_docker_test
echo "GITHUB_API_KEY="$secrets_api_key
echo "TEST_URL="$secrets_url
echo "TEST_DATASTORE"=$secrets_datastore
echo "TEST_USERNAME"=$secrets_username
echo "TEST_PASSWORD"=$secrets_password
}
function DebugDump() {
echo
echo "====================="
echo "SECRETS_FILE="$secretsfile
echo "TARGET_VCH="$target_vch
echo "DOCKER_TEST="$docker_test
echo "GITHUB_API_KEY="$github_api_key
echo "TEST_URL="$test_url
echo "TEST_DATASTORE"=$test_datastore
echo "TEST_USERNAME"=$test_username
echo "TEST_PASSWORD"=$test_password
echo "vic-dir"=$vic_dir
}
##### Get command line arguments
while getopts f:t:v:g:u:s:n:p:d flag
do
case $flag in
f)
secretsfile=$OPTARG
;;
t)
docker_test="$OPTARG"
;;
v)
target_vch=$OPTARG
;;
g)
github_api_key=$OPTARG
;;
u)
test_url=$OPTARG
;;
s)
test_datastore=$OPTARG
;;
n)
test_username=$OPTARG
;;
p)
test_password=$OPTARG
;;
d)
debug_enabled=1
;;
*)
usage
;;
esac
done
##### Preconditions...
# There is a priority in the preconditions. First, environment variable. Second, secrets file. Third, command line argument.
secretsfile=${SECRETS_FILE:-$secretsfile}
if [[ -z $secretsfile ]] ; then
GetGovcParamsFromEnv
else
GetParamsFromSecrets
fi
if [[ ! -z ${debug_enabled} ]] ; then
DebugInputDump
fi
target_vch=${TARGET_VCH:-$secrets_target_vch}
docker_test=${DOCKER_TEST:-$secrets_docker_test}
github_api_key=${GITHUB_API_KEY:-$secrets_api_key}
test_url=${TEST_URL:-$secrets_url}
test_datastore=${TEST_DATASTORE:-$secrets_datastore}
test_username=${TEST_USERNAME:-$secrets_username}
test_password=${TEST_PASSWORD:-$secrets_password}
if [[ -z ${target_vch} ]] || [[ -z "${docker_test}" ]] || [[ -z $github_api_key ]] || [[ -z $test_url ]] || [[ -z $test_datastore ]] || [[ -z $test_password ]] ; then
usage
fi
if [[ ! -z ${debug_enabled} ]] ; then
DebugDump
fi
##### The actual work
DoWork

20
vendor/github.com/vmware/vic/infra/scripts/misspell.sh generated vendored Executable file
View File

@@ -0,0 +1,20 @@
#!/bin/bash
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http:#www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
out=$(misspell -error $(find . -mindepth 1 -maxdepth 1 -type d -not -name vendor) | grep -v -e lib/config/dynamic/admiral/client | grep -v -e lib/config/dynamic/admiral/models | grep -v -e lib/config/dynamic/admiral/swagger.json)
if [[ ! -z "${out// }" ]]; then
echo "$out"
exit 2
fi

169
vendor/github.com/vmware/vic/infra/scripts/replace-running-components.sh generated vendored Executable file
View File

@@ -0,0 +1,169 @@
#!/bin/bash
# Copyright 2018 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eo pipefail
if [ -z "$PS1" ]; then
interactive=1
else
interactive=0
fi
BASE_DIR=$(dirname $(readlink -f "$BASH_SOURCE"))
VIC_DIR=$(dirname $(readlink -f $BASE_DIR/..))
# Run the command given on the VCH instead of locally
function on-vch() {
ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no -i $VIC_KEY root@$VCH_IP -C $@ 2>/dev/null
}
function get-thumbprint() {
govc about.cert -k -thumbprint | awk '{print $NF}'
}
# Determines whether the human VCH name is adequate to continue
function vch-name-is-ambiguous() {
[ $($VIC_DIR/bin/vic-machine-linux \
ls \
--target=$target \
--user=$username \
--password=$password \
--thumbprint=$(get-thumbprint) \
| grep $VIC_NAME | wc -l) -ne 1 ] \
&& return 0 || return 1
}
# Check GOVC vars & print help text if not
function check-govc-vars() {
if [[ ! $(govc ls 2>/dev/null) ]]; then
echo "ERROR:"
echo "GOVC environment variables are required to use this command. Set the necessary variables to allow govc to connect to your vSphere deployment:";
echo "GOVC_USERNAME: username on vSphere target"
echo "GOVC_PASSWORD: password on vSphere target"
echo "GOVC_URL: IP or FQDN of your vSphere target"
echo "GOVC_INSECURE: set to 1 to disable tls verify when using govc to talk to vSphere"
exit 1
fi
}
# Make sure we have at least one of VIC_NAME or VIC_ID
function check-vch-name-or-id () {
if [[ ! -v VIC_NAME ]] && [[ ! -v VIC_ID ]]; then
echo "Please set one of the following environment variables to specify the VCH which you would like to reconfigure:"
echo "VIC_NAME: name of VCH; matches --name argument for vic-machine"
echo "VIC_ID: ID of VCH, as displayed in output of vic-machine ls"
if [[ $interactive -eq 0 ]]; then
exit 1
fi
read -p "Or enter VCH name to continue: " VIC_NAME
fi
}
# Falls back on VIC_ID if possible, bails if not, assumes the VCH name is ambiguous
function check-name-isnt-ambiguous () {
username=$(govc env | grep GOVC_USERNAME | cut -d= -f2)
password=$(govc env | grep GOVC_PASSWORD | cut -d= -f2)
target=$(govc env | grep GOVC_URL | cut -d= -f2)
if [[ ! -v VIC_ID ]] && [[ $(vch-name-is-ambiguous) ]]; then
echo "The provided VIC name is ambiguous; please choose the correct VCH ID from the output below and assign it to the environment variable VIC_ID, e.g., export VIC_ID=12"
$VIC_DIR/bin/vic-machine-linux\
ls \
--target $GOVC_URL \
--user $GOVC_USER \
--password=$GOVC_PASSWORD \
--thumbprint=$(get-thumbprint)
if [[ $interactive -eq 0 ]]; then
exit 1
fi
read -p "Enter unique VIC ID from above: " VIC_ID
fi
}
# Translates VCH name to ID if necessary
function get-vic-id () {
if [[ -z $VIC_ID ]]; then
export VIC_ID="$($VIC_DIR/bin/vic-machine-linux ls --target=$target --user=$username --password=$password --thumbprint=$(get-thumbprint) | grep $VIC_NAME | awk '{print $1}')"
fi
}
function get-ssh-keys() {
if [[ -z $VIC_KEY ]]; then
echo "Variable VIC_KEY not set. Provide the path to your public SSH key below."
if [[ $interactive -eq 0 ]]; then
exit 1
fi
read -p "Path to your public SSH key for access to VCH [/home/$USER/.ssh/id_rsa.pub]: " key
export VIC_KEY=${key:-/home/$USER/.ssh/id_rsa.pub}
fi
}
# Checks environment for required inputs
function sanity-checks () {
check-govc-vars
check-vch-name-or-id
check-name-isnt-ambiguous
get-vic-id
get-ssh-keys
}
# Enables SSH and saves off the VCH IP address
function enable-debug () {
VCH_IP=$($VIC_DIR/bin/vic-machine-linux debug \
--target=$target \
--id=$VIC_ID \
--user=$username \
--password=$password \
--authorized-key=$VIC_KEY \
--thumbprint=$(get-thumbprint) \
| grep -A1 "Published ports" | tail -n1 | awk '{print $NF}')
}
# SCPs the component in $1 to the VCH, plops it in place, and brutally kills the previous running process
function replace-component() {
scp -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no $VIC_DIR/bin/$1 root@$VCH_IP:/tmp/$1 2>/dev/null
pid=$(on-vch ps -e --format='pid,args' \
| grep $1 | grep -v grep | awk '{print $1}')
on-vch chmod 755 /tmp/$1
on-vch mv /tmp/$1 /sbin/$1
if [[ $1 == "vic-init" ]]; then
on-vch systemctl restart vic-init
else
on-vch kill -9 $pid
fi
}
function replace-components () {
if [[ $1 == "" ]]; then # replace everything
services="port-layer-server docker-engine-server vicadmin vic-init"
else
services="$@"
fi
for x in $services; do
echo "Replacing component $x..."
replace-component $x
done
}
sanity-checks
enable-debug
replace-components $@
echo "If you ran make push with no arguments or replaced vic-init, wait a few moments and run this to get the new IP of your VCH:"
echo "$VIC_DIR/bin/vic-machine-linux inspect --target=$target --id=$VIC_ID --user=$username --password=$password --thumbprint=$(get-thumbprint)"

39
vendor/github.com/vmware/vic/infra/scripts/version-linker-flags.sh generated vendored Executable file
View File

@@ -0,0 +1,39 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
if [[ $# == 1 && $1 == "debug" ]]; then
echo " \
-X github.com/vmware/vic/pkg/version.Version=`git describe --abbrev=0 --tags` \
-X github.com/vmware/vic/pkg/version.BuildNumber=\"${BUILD_NUMBER}\" \
-X github.com/vmware/vic/pkg/version.BuildDate=`date -u +%Y/%m/%d@%H:%M:%S` \
-X github.com/vmware/vic/pkg/version.GitCommit=`git rev-parse --short HEAD` \
-X github.com/vmware/vic/pkg/version.State=` \
if [[ -n $(git ls-files --others --exclude-standard) || \
! $(git diff-files --no-ext-diff --quiet) || \
! $(git diff-index --no-ext-diff --quiet --cached HEAD) \
]]; then echo 'dirty'; fi`"
else
echo "-s -w \
-X github.com/vmware/vic/pkg/version.Version=`git describe --abbrev=0 --tags` \
-X github.com/vmware/vic/pkg/version.BuildNumber=\"${BUILD_NUMBER}\" \
-X github.com/vmware/vic/pkg/version.BuildDate=`date -u +%Y/%m/%d@%H:%M:%S` \
-X github.com/vmware/vic/pkg/version.GitCommit=`git rev-parse --short HEAD` \
-X github.com/vmware/vic/pkg/version.State=` \
if [[ -n $(git ls-files --others --exclude-standard) || \
! $(git diff-files --no-ext-diff --quiet) || \
! $(git diff-index --no-ext-diff --quiet --cached HEAD) \
]]; then echo 'dirty'; fi`"
fi

68
vendor/github.com/vmware/vic/infra/scripts/vic-logs.sh generated vendored Executable file
View File

@@ -0,0 +1,68 @@
#!/bin/bash -e
# Copyright 2017 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Create a VM and boot stateless ESXi via cdrom/iso
set -o pipefail
usage() {
cat <<'EOF'
Usage: $0 [-f] <host> <port-layer.log | docker.log>
GOVC_* environment variables also apply, see https://github.com/vmware/govmomi/tree/master/govc#usage
If GOVC_USERNAME is set, it is used to create an account on the ESX vm. Default is to use the existing root account.
If GOVC_PASSWORD is set, the account password will be set to this value. Default is to use the given ESX_URL password.
EOF
}
if [ $# -lt 2 ] ; then
usage
exit 1
fi
while getopts "f" opt; do
case $opt in
f)
follow="tail/"
;;
\?)
usage
exit 1
;;
esac
done
shift $((OPTIND-1))
host="$1"
file="$2"
username=$GOVC_USERNAME
password=$GOVC_PASSWORD
unset GOVC_USERNAME GOVC_PASSWORD
if [ -z "$password" ] ; then
# extract password from $GOVC_URL
password=$(govc env GOVC_PASSWORD)
fi
if [ -z "$username" ] ; then
username=$(govc env GOVC_USERNAME)
fi
jar=$(mktemp -t cookie-XXXX)
trap 'rm $jar' EXIT
curl -k -c "${jar}" --form username="${username}" --form password="${password}" https://"${host}":2378/authentication
curl -k -b "${jar}" https://"${host}":2378/logs/"${follow}""${file}"

35
vendor/github.com/vmware/vic/infra/scripts/whitespace-check.sh generated vendored Executable file
View File

@@ -0,0 +1,35 @@
#!/bin/bash
# Copyright 2016 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
red="\033[1;31m"
color_end="\033[0m"
GLOBIGNORE='vendor'
# will check staged items for whitespace
if [[ $# -gt 0 && $1 =~ [[:upper:]pre] ]]; then
if [[ $(git diff --cached --check) ]]; then
echo -e ${red}whitespace check failed${color_end}
git diff --cached --check
exit 1
fi
else
# check staged and changed
if [[ $(git diff --cached --check) || $(git diff --check) ]]; then
echo -e ${red}whitespace check failed${color_end}
git diff --cached --check
git diff --check
exit 1
fi
fi

940
vendor/github.com/vmware/vic/infra/util/gsml/__gmsl generated vendored Normal file
View File

@@ -0,0 +1,940 @@
# ----------------------------------------------------------------------------
#
# GNU Make Standard Library (GMSL)
#
# A library of functions to be used with GNU Make's $(call) that
# provides functionality not available in standard GNU Make.
#
# Copyright (c) 2005-2014 John Graham-Cumming
#
# This file is part of GMSL
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# Neither the name of the John Graham-Cumming nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# ----------------------------------------------------------------------------
# This is the GNU Make Standard Library version number as a list with
# three items: major, minor, revision
gmsl_version := 1 1 7
__gmsl_name := GNU Make Standard Library
# Used to output warnings and error from the library, it's possible to
# disable any warnings or errors by overriding these definitions
# manually or by setting GMSL_NO_WARNINGS or GMSL_NO_ERRORS
ifdef GMSL_NO_WARNINGS
__gmsl_warning :=
else
__gmsl_warning = $(if $1,$(warning $(__gmsl_name): $1))
endif
ifdef GMSL_NO_ERRORS
__gmsl_error :=
else
__gmsl_error = $(if $1,$(error $(__gmsl_name): $1))
endif
# If GMSL_TRACE is enabled then calls to the library functions are
# traced to stdout using warning messages with their arguments
ifdef GMSL_TRACE
__gmsl_tr1 = $(warning $0('$1'))
__gmsl_tr2 = $(warning $0('$1','$2'))
__gmsl_tr3 = $(warning $0('$1','$2','$3'))
else
__gmsl_tr1 :=
__gmsl_tr2 :=
__gmsl_tr3 :=
endif
# See if spaces are valid in variable names (this was the case until
# GNU Make 3.82)
ifeq ($(MAKE_VERSION),3.82)
__gmsl_spaced_vars := $(false)
else
__gmsl_spaced_vars := $(true)
endif
# Figure out whether we have $(eval) or not (GNU Make 3.80 and above)
# if we do not then output a warning message, if we do then some
# functions will be enabled.
__gmsl_have_eval := $(false)
__gmsl_ignore := $(eval __gmsl_have_eval := $(true))
# If this is being run with Electric Cloud's emake then warn that
# their $(eval) support is incomplete in 1.x, 2.x, 3.x, 4.x and 5.0,
# 5.1, 5.2 and 5.3
ifdef ECLOUD_BUILD_ID
__gmsl_emake_major := $(word 1,$(subst ., ,$(EMAKE_VERSION)))
__gmsl_emake_minor := $(word 2,$(subst ., ,$(EMAKE_VERSION)))
ifneq ("$(findstring $(__gmsl_emake_major),1 2 3 4)$(findstring $(__gmsl_emake_major)$(__gmsl_emake_minor),50 51 52 53)","")
$(warning You are using a version of Electric Cloud's emake which has incomplete $$(eval) support)
__gmsl_have_eval := $(false)
endif
endif
# See if we have $(lastword) (GNU Make 3.81 and above)
__gmsl_have_lastword := $(lastword $(false) $(true))
# See if we have native or and and (GNU Make 3.81 and above)
__gmsl_have_or := $(if $(filter-out undefined, \
$(origin or)),$(call or,$(true),$(false)))
__gmsl_have_and := $(if $(filter-out undefined, \
$(origin and)),$(call and,$(true),$(true)))
ifneq ($(__gmsl_have_eval),$(true))
$(call __gmsl_warning,Your make version $(MAKE_VERSION) does not support $$$$(eval): some functions disabled)
endif
__gmsl_dollar := $$
__gmsl_hash := \#
# ----------------------------------------------------------------------------
# ----------------------------------------------------------------------------
# Function: gmsl_compatible
# Arguments: List containing the desired library version number (maj min rev)
# Returns: $(true) if this version of the library is compatible
# with the requested version number, otherwise $(false)
# ----------------------------------------------------------------------------
gmsl_compatible = $(strip \
$(if $(call gt,$(word 1,$1),$(word 1,$(gmsl_version))), \
$(false), \
$(if $(call lt,$(word 1,$1),$(word 1,$(gmsl_version))), \
$(true), \
$(if $(call gt,$(word 2,$1),$(word 2,$(gmsl_version))), \
$(false), \
$(if $(call lt,$(word 2,$1),$(word 2,$(gmsl_version))), \
$(true), \
$(call lte,$(word 3,$1),$(word 3,$(gmsl_version))))))))
# ###########################################################################
# LOGICAL OPERATORS
# ###########################################################################
# not is defined in gmsl
# ----------------------------------------------------------------------------
# Function: and
# Arguments: Two boolean values
# Returns: Returns $(true) if both of the booleans are true
# ----------------------------------------------------------------------------
ifneq ($(__gmsl_have_and),$(true))
and = $(__gmsl_tr2)$(if $1,$(if $2,$(true),$(false)),$(false))
endif
# ----------------------------------------------------------------------------
# Function: or
# Arguments: Two boolean values
# Returns: Returns $(true) if either of the booleans is true
# ----------------------------------------------------------------------------
ifneq ($(__gmsl_have_or),$(true))
or = $(__gmsl_tr2)$(if $1$2,$(true),$(false))
endif
# ----------------------------------------------------------------------------
# Function: xor
# Arguments: Two boolean values
# Returns: Returns $(true) if exactly one of the booleans is true
# ----------------------------------------------------------------------------
xor = $(__gmsl_tr2)$(if $1,$(if $2,$(false),$(true)),$(if $2,$(true),$(false)))
# ----------------------------------------------------------------------------
# Function: nand
# Arguments: Two boolean values
# Returns: Returns value of 'not and'
# ----------------------------------------------------------------------------
nand = $(__gmsl_tr2)$(if $1,$(if $2,$(false),$(true)),$(true))
# ----------------------------------------------------------------------------
# Function: nor
# Arguments: Two boolean values
# Returns: Returns value of 'not or'
# ----------------------------------------------------------------------------
nor = $(__gmsl_tr2)$(if $1$2,$(false),$(true))
# ----------------------------------------------------------------------------
# Function: xnor
# Arguments: Two boolean values
# Returns: Returns value of 'not xor'
# ----------------------------------------------------------------------------
xnor =$(__gmsl_tr2)$(if $1,$(if $2,$(true),$(false)),$(if $2,$(false),$(true)))
# ###########################################################################
# LIST MANIPULATION FUNCTIONS
# ###########################################################################
# ----------------------------------------------------------------------------
# Function: first (same as LISP's car, or head)
# Arguments: 1: A list
# Returns: Returns the first element of a list
# ----------------------------------------------------------------------------
first = $(__gmsl_tr1)$(firstword $1)
# ----------------------------------------------------------------------------
# Function: last
# Arguments: 1: A list
# Returns: Returns the last element of a list
# ----------------------------------------------------------------------------
ifeq ($(__gmsl_have_lastword),$(true))
last = $(__gmsl_tr1)$(lastword $1)
else
last = $(__gmsl_tr1)$(if $1,$(word $(words $1),$1))
endif
# ----------------------------------------------------------------------------
# Function: rest (same as LISP's cdr, or tail)
# Arguments: 1: A list
# Returns: Returns the list with the first element removed
# ----------------------------------------------------------------------------
rest = $(__gmsl_tr1)$(wordlist 2,$(words $1),$1)
# ----------------------------------------------------------------------------
# Function: chop
# Arguments: 1: A list
# Returns: Returns the list with the last element removed
# ----------------------------------------------------------------------------
chop = $(__gmsl_tr1)$(wordlist 2,$(words $1),x $1)
# ----------------------------------------------------------------------------
# Function: map
# Arguments: 1: Name of function to $(call) for each element of list
# 2: List to iterate over calling the function in 1
# Returns: The list after calling the function on each element
# ----------------------------------------------------------------------------
map = $(__gmsl_tr2)$(strip $(foreach a,$2,$(call $1,$a)))
# ----------------------------------------------------------------------------
# Function: pairmap
# Arguments: 1: Name of function to $(call) for each pair of elements
# 2: List to iterate over calling the function in 1
# 3: Second list to iterate over calling the function in 1
# Returns: The list after calling the function on each pair of elements
# ----------------------------------------------------------------------------
pairmap = $(strip $(__gmsl_tr3)\
$(if $2$3,$(call $1,$(call first,$2),$(call first,$3)) \
$(call pairmap,$1,$(call rest,$2),$(call rest,$3))))
# ----------------------------------------------------------------------------
# Function: leq
# Arguments: 1: A list to compare against...
# 2: ...this list
# Returns: Returns $(true) if the two lists are identical
# ----------------------------------------------------------------------------
leq = $(__gmsl_tr2)$(strip $(if $(call seq,$(words $1),$(words $2)), \
$(call __gmsl_list_equal,$1,$2),$(false)))
__gmsl_list_equal = $(if $(strip $1), \
$(if $(call seq,$(call first,$1),$(call first,$2)), \
$(call __gmsl_list_equal, \
$(call rest,$1), \
$(call rest,$2)), \
$(false)), \
$(true))
# ----------------------------------------------------------------------------
# Function: lne
# Arguments: 1: A list to compare against...
# 2: ...this list
# Returns: Returns $(true) if the two lists are different
# ----------------------------------------------------------------------------
lne = $(__gmsl_tr2)$(call not,$(call leq,$1,$2))
# ----------------------------------------------------------------------------
# Function: reverse
# Arguments: 1: A list to reverse
# Returns: The list with its elements in reverse order
# ----------------------------------------------------------------------------
reverse =$(__gmsl_tr1)$(strip $(if $1,$(call reverse,$(call rest,$1)) \
$(call first,$1)))
# ----------------------------------------------------------------------------
# Function: uniq
# Arguments: 1: A list from which to remove repeated elements
# Returns: The list with duplicate elements removed without reordering
# ----------------------------------------------------------------------------
uniq = $(strip $(__gmsl_tr1) $(if $1,$(firstword $1) \
$(call uniq,$(filter-out $(firstword $1),$1))))
# ----------------------------------------------------------------------------
# Function: length
# Arguments: 1: A list
# Returns: The number of elements in the list
# ----------------------------------------------------------------------------
length = $(__gmsl_tr1)$(words $1)
# ###########################################################################
# STRING MANIPULATION FUNCTIONS
# ###########################################################################
# Helper function that translates any GNU Make 'true' value (i.e. a
# non-empty string) to our $(true)
__gmsl_make_bool = $(if $(strip $1),$(true),$(false))
# ----------------------------------------------------------------------------
# Function: seq
# Arguments: 1: A string to compare against...
# 2: ...this string
# Returns: Returns $(true) if the two strings are identical
# ----------------------------------------------------------------------------
seq = $(__gmsl_tr2)$(if $(subst x$1,,x$2)$(subst x$2,,x$1),$(false),$(true))
# ----------------------------------------------------------------------------
# Function: sne
# Arguments: 1: A string to compare against...
# 2: ...this string
# Returns: Returns $(true) if the two strings are not the same
# ----------------------------------------------------------------------------
sne = $(__gmsl_tr2)$(call not,$(call seq,$1,$2))
# ----------------------------------------------------------------------------
# Function: split
# Arguments: 1: The character to split on
# 2: A string to split
# Returns: Splits a string into a list separated by spaces at the split
# character in the first argument
# ----------------------------------------------------------------------------
split = $(__gmsl_tr2)$(strip $(subst $1, ,$2))
# ----------------------------------------------------------------------------
# Function: merge
# Arguments: 1: The character to put between fields
# 2: A list to merge into a string
# Returns: Merges a list into a single string, list elements are separated
# by the character in the first argument
# ----------------------------------------------------------------------------
merge = $(__gmsl_tr2)$(strip $(if $2, \
$(if $(call seq,1,$(words $2)), \
$2,$(call first,$2)$1$(call merge,$1,$(call rest,$2)))))
ifdef __gmsl_have_eval
# ----------------------------------------------------------------------------
# Function: tr
# Arguments: 1: The list of characters to translate from
# 2: The list of characters to translate to
# 3: The text to translate
# Returns: Returns the text after translating characters
# ----------------------------------------------------------------------------
tr = $(strip $(__gmsl_tr3)$(call assert_no_dollar,$0,$1$2$3) \
$(eval __gmsl_t := $3) \
$(foreach c, \
$(join $(addsuffix :,$1),$2), \
$(eval __gmsl_t := \
$(subst $(word 1,$(subst :, ,$c)),$(word 2,$(subst :, ,$c)), \
$(__gmsl_t))))$(__gmsl_t))
# Common character classes for use with the tr function. Each of
# these is actually a variable declaration and must be wrapped with
# $() or ${} to be used.
[A-Z] := A B C D E F G H I J K L M N O P Q R S T U V W X Y Z #
[a-z] := a b c d e f g h i j k l m n o p q r s t u v w x y z #
[0-9] := 0 1 2 3 4 5 6 7 8 9 #
[A-F] := A B C D E F #
# ----------------------------------------------------------------------------
# Function: uc
# Arguments: 1: Text to upper case
# Returns: Returns the text in upper case
# ----------------------------------------------------------------------------
uc = $(__gmsl_tr1)$(call assert_no_dollar,$0,$1)$(call tr,$([a-z]),$([A-Z]),$1)
# ----------------------------------------------------------------------------
# Function: lc
# Arguments: 1: Text to lower case
# Returns: Returns the text in lower case
# ----------------------------------------------------------------------------
lc = $(__gmsl_tr1)$(call assert_no_dollar,$0,$1)$(call tr,$([A-Z]),$([a-z]),$1)
# ----------------------------------------------------------------------------
# Function: strlen
# Arguments: 1: A string
# Returns: Returns the length of the string
# ----------------------------------------------------------------------------
# This results in __gmsl_tab containing a tab
__gmsl_tab := #
__gmsl_characters := A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
__gmsl_characters += a b c d e f g h i j k l m n o p q r s t u v w x y z
__gmsl_characters += 0 1 2 3 4 5 6 7 8 9
__gmsl_characters += ` ~ ! @ \# $$ % ^ & * ( ) - _ = +
__gmsl_characters += { } [ ] \ : ; ' " < > , . / ? |
# This results in __gmsl_space containing just a space
__gmsl_space :=
__gmsl_space +=
strlen = $(__gmsl_tr1)$(call assert_no_dollar,$0,$1)$(strip $(eval __temp := $(subst $(__gmsl_space),x,$1))$(foreach a,$(__gmsl_characters),$(eval __temp := $$(subst $$a,x,$(__temp))))$(eval __temp := $(subst x,x ,$(__temp)))$(words $(__temp)))
# This results in __gmsl_newline containing just a newline
define __gmsl_newline
endef
# ----------------------------------------------------------------------------
# Function: substr
# Arguments: 1: A string
# 2: Start position (first character is 1)
# 3: End position (inclusive)
# Returns: A substring.
# Note: The string in $1 must not contain a <20>
# ----------------------------------------------------------------------------
substr = $(if $2,$(__gmsl_tr3)$(call assert_no_dollar,$0,$1$2$3)$(strip $(eval __temp := $$(subst $$(__gmsl_space),<2C> ,$$1))$(foreach a,$(__gmsl_characters),$(eval __temp := $$(subst $$a,$$a$$(__gmsl_space),$(__temp))))$(eval __temp := $(wordlist $2,$3,$(__temp))))$(subst <20>,$(__gmsl_space),$(subst $(__gmsl_space),,$(__temp))))
endif # __gmsl_have_eval
# ###########################################################################
# SET MANIPULATION FUNCTIONS
# ###########################################################################
# Sets are represented by sorted, deduplicated lists. To create a set
# from a list use set_create, or start with the empty_set and
# set_insert individual elements
# This is the empty set
empty_set :=
# ----------------------------------------------------------------------------
# Function: set_create
# Arguments: 1: A list of set elements
# Returns: Returns the newly created set
# ----------------------------------------------------------------------------
set_create = $(__gmsl_tr1)$(sort $1)
# ----------------------------------------------------------------------------
# Function: set_insert
# Arguments: 1: A single element to add to a set
# 2: A set
# Returns: Returns the set with the element added
# ----------------------------------------------------------------------------
set_insert = $(__gmsl_tr2)$(sort $1 $2)
# ----------------------------------------------------------------------------
# Function: set_remove
# Arguments: 1: A single element to remove from a set
# 2: A set
# Returns: Returns the set with the element removed
# ----------------------------------------------------------------------------
set_remove = $(__gmsl_tr2)$(filter-out $1,$2)
# ----------------------------------------------------------------------------
# Function: set_is_member, set_is_not_member
# Arguments: 1: A single element
# 2: A set
# Returns: (set_is_member) Returns $(true) if the element is in the set
# (set_is_not_member) Returns $(false) if the element is in the set
# ----------------------------------------------------------------------------
set_is_member = $(__gmsl_tr2)$(if $(filter $1,$2),$(true),$(false))
set_is_not_member = $(__gmsl_tr2)$(if $(filter $1,$2),$(false),$(true))
# ----------------------------------------------------------------------------
# Function: set_union
# Arguments: 1: A set
# 2: Another set
# Returns: Returns the union of the two sets
# ----------------------------------------------------------------------------
set_union = $(__gmsl_tr2)$(sort $1 $2)
# ----------------------------------------------------------------------------
# Function: set_intersection
# Arguments: 1: A set
# 2: Another set
# Returns: Returns the intersection of the two sets
# ----------------------------------------------------------------------------
set_intersection = $(__gmsl_tr2)$(filter $1,$2)
# ----------------------------------------------------------------------------
# Function: set_is_subset
# Arguments: 1: A set
# 2: Another set
# Returns: Returns $(true) if the first set is a subset of the second
# ----------------------------------------------------------------------------
set_is_subset = $(__gmsl_tr2)$(call set_equal,$(call set_intersection,$1,$2),$1)
# ----------------------------------------------------------------------------
# Function: set_equal
# Arguments: 1: A set
# 2: Another set
# Returns: Returns $(true) if the two sets are identical
# ----------------------------------------------------------------------------
set_equal = $(__gmsl_tr2)$(call seq,$1,$2)
# ###########################################################################
# ARITHMETIC LIBRARY
# ###########################################################################
# Integers a represented by lists with the equivalent number of x's.
# For example the number 4 is x x x x.
# ----------------------------------------------------------------------------
# Function: int_decode
# Arguments: 1: A number of x's representation
# Returns: Returns the integer for human consumption that is represented
# by the string of x's
# ----------------------------------------------------------------------------
int_decode = $(__gmsl_tr1)$(words $1)
# ----------------------------------------------------------------------------
# Function: int_encode
# Arguments: 1: A number in human-readable integer form
# Returns: Returns the integer encoded as a string of x's
# ----------------------------------------------------------------------------
__int_encode = $(if $1,$(if $(call seq,$(words $(wordlist 1,$1,$2)),$1),$(wordlist 1,$1,$2),$(call __int_encode,$1,$(if $2,$2 $2,x))))
__strip_leading_zero = $(if $1,$(if $(call seq,$(patsubst 0%,%,$1),$1),$1,$(call __strip_leading_zero,$(patsubst 0%,%,$1))),0)
int_encode = $(__gmsl_tr1)$(call __int_encode,$(call __strip_leading_zero,$1))
# The arithmetic library functions come in two forms: one form of each
# function takes integers as arguments and the other form takes the
# encoded form (x's created by a call to int_encode). For example,
# there are two plus functions:
#
# plus Called with integer arguments and returns an integer
# int_plus Called with encoded arguments and returns an encoded result
#
# plus will be slower than int_plus because its arguments and result
# have to be translated between the x's format and integers. If doing
# a complex calculation use the int_* forms with a single encoding of
# inputs and single decoding of the output. For simple calculations
# the direct forms can be used.
# Helper function used to wrap an int_* function into a function that
# takes a pair of integers, perhaps a function and returns an integer
# result
__gmsl_int_wrap = $(call int_decode,$(call $1,$(call int_encode,$2),$(call int_encode,$3)))
__gmsl_int_wrap1 = $(call int_decode,$(call $1,$(call int_encode,$2)))
__gmsl_int_wrap2 = $(call $1,$(call int_encode,$2),$(call int_encode,$3))
# ----------------------------------------------------------------------------
# Function: int_plus
# Arguments: 1: A number in x's representation
# 2: Another number in x's represntation
# Returns: Returns the sum of the two numbers in x's representation
# ----------------------------------------------------------------------------
int_plus = $(strip $(__gmsl_tr2)$1 $2)
# ----------------------------------------------------------------------------
# Function: plus (wrapped version of int_plus)
# Arguments: 1: An integer
# 2: Another integer
# Returns: Returns the sum of the two integers
# ----------------------------------------------------------------------------
plus = $(__gmsl_tr2)$(call __gmsl_int_wrap,int_plus,$1,$2)
# ----------------------------------------------------------------------------
# Function: int_subtract
# Arguments: 1: A number in x's representation
# 2: Another number in x's represntation
# Returns: Returns the difference of the two numbers in x's representation,
# or outputs an error on a numeric underflow
# ----------------------------------------------------------------------------
int_subtract = $(strip $(__gmsl_tr2)$(if $(call int_gte,$1,$2), \
$(filter-out xx,$(join $1,$2)), \
$(call __gmsl_warning,Subtraction underflow)))
# ----------------------------------------------------------------------------
# Function: subtract (wrapped version of int_subtract)
# Arguments: 1: An integer
# 2: Another integer
# Returns: Returns the difference of the two integers,
# or outputs an error on a numeric underflow
# ----------------------------------------------------------------------------
subtract = $(__gmsl_tr2)$(call __gmsl_int_wrap,int_subtract,$1,$2)
# ----------------------------------------------------------------------------
# Function: int_multiply
# Arguments: 1: A number in x's representation
# 2: Another number in x's represntation
# Returns: Returns the product of the two numbers in x's representation
# ----------------------------------------------------------------------------
int_multiply = $(strip $(__gmsl_tr2)$(foreach a,$1,$2))
# ----------------------------------------------------------------------------
# Function: multiply (wrapped version of int_multiply)
# Arguments: 1: An integer
# 2: Another integer
# Returns: Returns the product of the two integers
# ----------------------------------------------------------------------------
multiply = $(__gmsl_tr2)$(call __gmsl_int_wrap,int_multiply,$1,$2)
# ----------------------------------------------------------------------------
# Function: int_divide
# Arguments: 1: A number in x's representation
# 2: Another number in x's represntation
# Returns: Returns the result of integer division of argument 1 divided
# by argument 2 in x's representation
# ----------------------------------------------------------------------------
int_divide = $(__gmsl_tr2)$(strip $(if $1,$(if $2, \
$(if $(call int_gte,$1,$2), \
x $(call int_divide,$(call int_subtract,$1,$2),$2),), \
$(call __gmsl_error,Division by zero))))
# ----------------------------------------------------------------------------
# Function: divide (wrapped version of int_divide)
# Arguments: 1: An integer
# 2: Another integer
# Returns: Returns the integer division of the first argument by the second
# ----------------------------------------------------------------------------
divide = $(__gmsl_tr2)$(call __gmsl_int_wrap,int_divide,$1,$2)
# ----------------------------------------------------------------------------
# Function: int_max, int_min
# Arguments: 1: A number in x's representation
# 2: Another number in x's represntation
# Returns: Returns the maximum or minimum of its arguments in x's
# representation
# ----------------------------------------------------------------------------
int_max = $(__gmsl_tr2)$(subst xx,x,$(join $1,$2))
int_min = $(__gmsl_tr2)$(subst xx,x,$(filter xx,$(join $1,$2)))
# ----------------------------------------------------------------------------
# Function: max, min
# Arguments: 1: An integer
# 2: Another integer
# Returns: Returns the maximum or minimum of its integer arguments
# ----------------------------------------------------------------------------
max = $(__gmsl_tr2)$(call __gmsl_int_wrap,int_max,$1,$2)
min = $(__gmsl_tr2)$(call __gmsl_int_wrap,int_min,$1,$2)
# ----------------------------------------------------------------------------
# Function: int_gt, int_gte, int_lt, int_lte, int_eq, int_ne
# Arguments: Two x's representation numbers to be compared
# Returns: $(true) or $(false)
#
# int_gt First argument greater than second argument
# int_gte First argument greater than or equal to second argument
# int_lt First argument less than second argument
# int_lte First argument less than or equal to second argument
# int_eq First argument is numerically equal to the second argument
# int_ne First argument is not numerically equal to the second argument
# ----------------------------------------------------------------------------
int_gt = $(__gmsl_tr2)$(call __gmsl_make_bool, \
$(filter-out $(words $2), \
$(words $(call int_max,$1,$2))))
int_gte = $(__gmsl_tr2)$(call __gmsl_make_bool, \
$(call int_gt,$1,$2)$(call int_eq,$1,$2))
int_lt = $(__gmsl_tr2)$(call __gmsl_make_bool, \
$(filter-out $(words $1), \
$(words $(call int_max,$1,$2))))
int_lte = $(__gmsl_tr2)$(call __gmsl_make_bool, \
$(call int_lt,$1,$2)$(call int_eq,$1,$2))
int_eq = $(__gmsl_tr2)$(call __gmsl_make_bool, \
$(filter $(words $1),$(words $2)))
int_ne = $(__gmsl_tr2)$(call __gmsl_make_bool, \
$(filter-out $(words $1),$(words $2)))
# ----------------------------------------------------------------------------
# Function: gt, gte, lt, lte, eq, ne
# Arguments: Two integers to be compared
# Returns: $(true) or $(false)
#
# gt First argument greater than second argument
# gte First argument greater than or equal to second argument
# lt First argument less than second argument
# lte First argument less than or equal to second argument
# eq First argument is numerically equal to the second argument
# ne First argument is not numerically equal to the second argument
# ----------------------------------------------------------------------------
gt = $(__gmsl_tr2)$(call __gmsl_int_wrap2,int_gt,$1,$2)
gte = $(__gmsl_tr2)$(call __gmsl_int_wrap2,int_gte,$1,$2)
lt = $(__gmsl_tr2)$(call __gmsl_int_wrap2,int_lt,$1,$2)
lte = $(__gmsl_tr2)$(call __gmsl_int_wrap2,int_lte,$1,$2)
eq = $(__gmsl_tr2)$(call __gmsl_int_wrap2,int_eq,$1,$2)
ne = $(__gmsl_tr2)$(call __gmsl_int_wrap2,int_ne,$1,$2)
# increment adds 1 to its argument, decrement subtracts 1. Note that
# decrement does not range check and hence will not underflow, but
# will incorrectly say that 0 - 1 = 0
# ----------------------------------------------------------------------------
# Function: int_inc
# Arguments: 1: A number in x's representation
# Returns: The number incremented by 1 in x's representation
# ----------------------------------------------------------------------------
int_inc = $(strip $(__gmsl_tr1)$1 x)
# ----------------------------------------------------------------------------
# Function: inc
# Arguments: 1: An integer
# Returns: The argument incremented by 1
# ----------------------------------------------------------------------------
inc = $(__gmsl_tr1)$(call __gmsl_int_wrap1,int_inc,$1)
# ----------------------------------------------------------------------------
# Function: int_dec
# Arguments: 1: A number in x's representation
# Returns: The number decremented by 1 in x's representation
# ----------------------------------------------------------------------------
int_dec = $(__gmsl_tr1)$(strip \
$(if $(call sne,0,$(words $1)), \
$(wordlist 2,$(words $1),$1)))
# ----------------------------------------------------------------------------
# Function: dec
# Arguments: 1: An integer
# Returns: The argument decremented by 1
# ----------------------------------------------------------------------------
dec = $(__gmsl_tr1)$(call __gmsl_int_wrap1,int_dec,$1)
# double doubles its argument, and halve halves it
# ----------------------------------------------------------------------------
# Function: int_double
# Arguments: 1: A number in x's representation
# Returns: The number doubled (i.e. * 2) and returned in x's representation
# ----------------------------------------------------------------------------
int_double = $(strip $(__gmsl_tr1)$1 $1)
# ----------------------------------------------------------------------------
# Function: double
# Arguments: 1: An integer
# Returns: The integer times 2
# ----------------------------------------------------------------------------
double = $(__gmsl_tr1)$(call __gmsl_int_wrap1,int_double,$1)
# ----------------------------------------------------------------------------
# Function: int_halve
# Arguments: 1: A number in x's representation
# Returns: The number halved (i.e. / 2) and returned in x's representation
# ----------------------------------------------------------------------------
int_halve = $(__gmsl_tr1)$(strip $(subst xx,x,$(filter-out xy x y, \
$(join $1,$(foreach a,$1,y x)))))
# ----------------------------------------------------------------------------
# Function: halve
# Arguments: 1: An integer
# Returns: The integer divided by 2
# ----------------------------------------------------------------------------
halve = $(__gmsl_tr1)$(call __gmsl_int_wrap1,int_halve,$1)
# ----------------------------------------------------------------------------
# Function: sequence
# Arguments: 1: An integer
# 2: An integer
# Returns: The sequence [arg1, arg2] of integers if arg1 < arg2 or
# [arg2, arg1] if arg2 > arg1. If arg1 == arg1 return [arg1]
# ----------------------------------------------------------------------------
sequence = $(__gmsl_tr2)$(strip $(if $(call lte,$1,$2), \
$(call __gmsl_sequence_up,$1,$2), \
$(call __gmsl_sequence_dn,$2,$1)))
__gmsl_sequence_up = $(if $(call seq,$1,$2),$1,$1 $(call __gmsl_sequence_up,$(call inc,$1),$2))
__gmsl_sequence_dn = $(if $(call seq,$1,$2),$1,$2 $(call __gmsl_sequence_dn,$1,$(call dec,$2)))
# ----------------------------------------------------------------------------
# Function: dec2hex, dec2bin, dec2oct
# Arguments: 1: An integer
# Returns: The decimal argument converted to hexadecimal, binary or
# octal
# ----------------------------------------------------------------------------
__gmsl_digit = $(subst 15,f,$(subst 14,e,$(subst 13,d,$(subst 12,c,$(subst 11,b,$(subst 10,a,$1))))))
dec2hex = $(call __gmsl_dec2base,$(call int_encode,$1),$(call int_encode,16))
dec2bin = $(call __gmsl_dec2base,$(call int_encode,$1),$(call int_encode,2))
dec2oct = $(call __gmsl_dec2base,$(call int_encode,$1),$(call int_encode,8))
__gmsl_base_divide = $(subst $2,X ,$1)
__gmsl_q = $(strip $(filter X,$1))
__gmsl_r = $(words $(filter x,$1))
__gmsl_dec2base = $(eval __gmsl_temp := $(call __gmsl_base_divide,$1,$2))$(call __gmsl_dec2base_,$(call __gmsl_q,$(__gmsl_temp)),$(call __gmsl_r,$(__gmsl_temp)),$2)
__gmsl_dec2base_ = $(if $1,$(call __gmsl_dec2base,$(subst X,x,$1),$3))$(call __gmsl_digit,$2)
ifdef __gmsl_have_eval
# ###########################################################################
# ASSOCIATIVE ARRAYS
# ###########################################################################
# Magic string that is very unlikely to appear in a key or value
__gmsl_aa_magic := faf192c8efbc25c27992c5bc5add390393d583c6
# ----------------------------------------------------------------------------
# Function: set
# Arguments: 1: Name of associative array
# 2: The key value to associate
# 3: The value associated with the key
# Returns: Nothing
# ----------------------------------------------------------------------------
set = $(__gmsl_tr3)$(call assert_no_space,$0,$1$2)$(call assert_no_dollar,$0,$1$2$3)$(eval __gmsl_aa_$1_$(__gmsl_aa_magic)_$2_gmsl_aa_$1 := $3)
# Only used internally by memoize function
__gmsl_set = $(call set,$1,$2,$3)$3
# ----------------------------------------------------------------------------
# Function: get
# Arguments: 1: Name of associative array
# 2: The key to retrieve
# Returns: The value stored in the array for that key
# ----------------------------------------------------------------------------
get = $(strip $(__gmsl_tr2)$(call assert_no_space,$0,$1$2)$(call assert_no_dollar,$0,$1$2)$(__gmsl_aa_$1_$(__gmsl_aa_magic)_$2_gmsl_aa_$1))
# ----------------------------------------------------------------------------
# Function: keys
# Arguments: 1: Name of associative array
# Returns: Returns a list of all defined keys in the array
# ----------------------------------------------------------------------------
keys = $(__gmsl_tr1)$(call assert_no_space,$0,$1)$(call assert_no_dollar,$0,$1)$(sort $(patsubst __gmsl_aa_$1_$(__gmsl_aa_magic)_%_gmsl_aa_$1,%, \
$(filter __gmsl_aa_$1_$(__gmsl_aa_magic)_%_gmsl_aa_$1,$(.VARIABLES))))
# ----------------------------------------------------------------------------
# Function: defined
# Arguments: 1: Name of associative array
# 2: The key to test
# Returns: Returns true if the key is defined (i.e. not empty)
# ----------------------------------------------------------------------------
defined = $(__gmsl_tr2)$(call assert_no_space,$0,$1$2)$(call assert_no_dollar,$0,$1$2)$(call sne,$(call get,$1,$2),)
endif # __gmsl_have_eval
ifdef __gmsl_have_eval
# ###########################################################################
# NAMED STACKS
# ###########################################################################
# ----------------------------------------------------------------------------
# Function: push
# Arguments: 1: Name of stack
# 2: Value to push onto the top of the stack (must not contain
# a space)
# Returns: None
# ----------------------------------------------------------------------------
push = $(__gmsl_tr2)$(call assert_no_space,$0,$1$2)$(call assert_no_dollar,$0,$1$2)$(eval __gmsl_stack_$1 := $2 $(if $(filter-out undefined,\
$(origin __gmsl_stack_$1)),$(__gmsl_stack_$1)))
# ----------------------------------------------------------------------------
# Function: pop
# Arguments: 1: Name of stack
# Returns: Top element from the stack after removing it
# ----------------------------------------------------------------------------
pop = $(__gmsl_tr1)$(call assert_no_space,$0,$1)$(call assert_no_dollar,$0,$1)$(strip $(if $(filter-out undefined,$(origin __gmsl_stack_$1)), \
$(call first,$(__gmsl_stack_$1)) \
$(eval __gmsl_stack_$1 := $(call rest,$(__gmsl_stack_$1)))))
# ----------------------------------------------------------------------------
# Function: peek
# Arguments: 1: Name of stack
# Returns: Top element from the stack without removing it
# ----------------------------------------------------------------------------
peek = $(__gmsl_tr1)$(call assert_no_space,$0,$1)$(call assert_no_dollar,$0,$1)$(call first,$(__gmsl_stack_$1))
# ----------------------------------------------------------------------------
# Function: depth
# Arguments: 1: Name of stack
# Returns: Number of items on the stack
# ----------------------------------------------------------------------------
depth = $(__gmsl_tr1)$(call assert_no_space,$0,$1)$(call assert_no_dollar,$0,$1)$(words $(__gmsl_stack_$1))
endif # __gmsl_have_eval
ifdef __gmsl_have_eval
# ###########################################################################
# STRING CACHE
# ###########################################################################
# ----------------------------------------------------------------------------
# Function: memoize
# Arguments: 1. Name of the function to be called if the string
# has not been previously seen
# 2. A string
# Returns: Returns the result of a memo function (which the user must
# define) on the passed in string and remembers the result.
#
# Example: Set memo = $(shell echo "$1" | md5sum) to make a cache
# of MD5 hashes of strings. $(call memoize,memo,foo bar baz)
# ----------------------------------------------------------------------------
__gmsl_memoize = $(subst $(__gmsl_space),<2C>,$1)cc2af1bb7c4482f2ba75e338b963d3e7$(subst $(__gmsl_space),<2C>,$2)
memoize = $(__gmsl_tr2)$(strip $(if $(call defined,__gmsl_m,$(__gmsl_memoize)),\
$(call get,__gmsl_m,$(__gmsl_memoize)), \
$(call __gmsl_set,__gmsl_m,$(__gmsl_memoize),$(call $1,$2))))
endif # __gmsl_have_eval
# ###########################################################################
# DEBUGGING FACILITIES
# ###########################################################################
# ----------------------------------------------------------------------------
# Target: gmsl-print-%
# Arguments: The % should be replaced by the name of a variable that you
# wish to print out.
# Action: Echos the name of the variable that matches the % and its value.
# For example, 'make gmsl-print-SHELL' will output the value of
# the SHELL variable
# ----------------------------------------------------------------------------
gmsl-print-%: ; @echo $* = $($*)
# ----------------------------------------------------------------------------
# Function: assert
# Arguments: 1: A boolean that must be true or the assertion will fail
# 2: The message to print with the assertion
# Returns: None
# ----------------------------------------------------------------------------
assert = $(if $2,$(if $1,,$(call __gmsl_error,Assertion failure: $2)))
# ----------------------------------------------------------------------------
# Function: assert_exists
# Arguments: 1: Name of file that must exist, if it is missing an assertion
# will be generated
# Returns: None
# ----------------------------------------------------------------------------
assert_exists = $(if $0,$(call assert,$(wildcard $1),file '$1' missing))
# ----------------------------------------------------------------------------
# Function: assert_no_dollar
# Arguments: 1: Name of a function being executd
# 2: Arguments to check
# Returns: None
# ----------------------------------------------------------------------------
assert_no_dollar = $(call __gmsl_tr2)$(call assert,$(call not,$(findstring $(__gmsl_dollar),$2)),$1 called with a dollar sign in argument)
# ----------------------------------------------------------------------------
# Function: assert_no_space
# Arguments: 1: Name of a function being executd
# 2: Arguments to check
# Returns: None
# ----------------------------------------------------------------------------
ifeq ($(__gmsl_spaced_vars),$(false))
assert_no_space = $(call assert,$(call not,$(findstring $(__gmsl_aa_magic),$(subst $(__gmsl_space),$(__gmsl_aa_magic),$2))),$1 called with a space in argument)
else
assert_no_space =
endif

85
vendor/github.com/vmware/vic/infra/util/gsml/gmsl generated vendored Normal file
View File

@@ -0,0 +1,85 @@
# ----------------------------------------------------------------------------
#
# GNU Make Standard Library (GMSL)
#
# A library of functions to be used with GNU Make's $(call) that
# provides functionality not available in standard GNU Make.
#
# Copyright (c) 2005-2014 John Graham-Cumming
#
# This file is part of GMSL
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# Neither the name of the John Graham-Cumming nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# ----------------------------------------------------------------------------
# Determine if the library has already been included and if so don't
# bother including it again
ifndef __gmsl_included
# Standard definitions for true and false. true is any non-empty
# string, false is an empty string. These are intended for use with
# $(if).
true := T
false :=
# ----------------------------------------------------------------------------
# Function: not
# Arguments: 1: A boolean value
# Returns: Returns the opposite of the arg. (true -> false, false -> true)
# ----------------------------------------------------------------------------
not = $(if $1,$(false),$(true))
# Prevent reinclusion of the library
__gmsl_included := $(true)
# Try to determine where this file is located. If the caller did
# include /foo/gmsl then extract the /foo/ so that __gmsl gets
# included transparently
__gmsl_root :=
ifneq ($(MAKEFILE_LIST),)
__gmsl_root := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
# If there are any spaces in the path in __gmsl_root then give up
ifeq (1,$(words $(__gmsl_root)))
__gmsl_root := $(patsubst %gmsl,%,$(__gmsl_root))
endif
endif
include $(__gmsl_root)__gmsl
endif # __gmsl_included

30
vendor/github.com/vmware/vic/infra/util/vendor-manifest-parser/README.md generated vendored Normal file
View File

@@ -0,0 +1,30 @@
### Dependency parsing tool
This tool is here to simplify checking the vendor manifest for consistency and for which tags are being used
It queries GitHub and takes into account any specific exceptions and produces a human-readable report on what our dependencies are
#### Building
There's no point building this tool all the time. It will be used sporadically for auditing purposes.
Simply use go build <sourcefile> to create the binaries when needed
#### Running
The binaries are designed to be piped together to take each others output. This way, any intermediate json can be fed into a spreadsheet or likewise
Example usage:
```
cat ../../manifest | gettags --uid "bcorrie@vmware.com" --pwd "foobedoo" > 1.json
cat 1.json | groupbyrepo > 2.json
cat 2.json | report --exceptionFile=../../exceptions
cat ../../manifest | gettags --uid "bcorrie@vmware.com" --pwd "foobedoo" | groupbyrepo | report --exceptionFile=../../exceptions
```
#### Exceptions
It would be good practice for us to document cases in which a specific revision must be used, either because of a critical patch or equivalent. See exceptions file co-located with the manifest for examples.
Exceptions are outputted in the report and ensure that no-one attempts to modify a revision to a later tag without considering the exception

138
vendor/github.com/vmware/vic/infra/util/vendor-manifest-parser/tool/gettags.go generated vendored Normal file
View File

@@ -0,0 +1,138 @@
// Copyright 2016 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"encoding/json"
"flag"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"os"
"parser"
)
var client *http.Client
var uid = flag.String("uid", "", "valid Github userid")
var pwd = flag.String("pwd", "", "valid Github password")
func init() {
flag.Usage = func() {
fmt.Println("Usage: Queries Github for tags for each dependency sha in manifest input. Takes input via stdin. Eg: cat manifest | gettags")
fmt.Println(" Uses basic auth, specified by --uid=<uid> and --pwd=<pwd>. Failure to specify will likely hit Github API limits")
flag.PrintDefaults()
}
flag.Parse()
}
func findTagForSha(data []parser.TagData, sha string) string {
for _, tag := range data {
if tag.Commit.Sha == sha {
return tag.Name
}
}
return ""
}
func queryGit(u *url.URL) ([]byte, error) {
getPath := "https://api.github.com/repos" + u.Path + "/tags"
req, _ := http.NewRequest("GET", getPath, nil)
if *uid != "" && *pwd != "" {
req.SetBasicAuth(*uid, *pwd)
}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("Unexpected error attempting HTTP GET to %v: %v", u, err)
}
return ioutil.ReadAll(resp.Body)
}
// Currently only works for Github URLs as this is the vast majority. Rest will have HasTags == false
func findTag(entry *parser.ManifestEntry) (*parser.ManifestEntry, error) {
u, err := url.Parse(entry.Repository)
if err != nil {
return nil, fmt.Errorf("Malformed repository URL: %v", err)
}
if u.Host == "github.com" {
tagdata, err := queryGit(u)
if err != nil {
return nil, err
}
td := []parser.TagData{}
err = json.Unmarshal(tagdata, &td)
if err != nil {
return nil, fmt.Errorf("Unparsable output: %s: %v", string(tagdata), err)
}
entry.HasTags = (len(td) == 0)
if len(td) == 0 {
entry.HasTags = false
} else {
entry.HasTags = true
foundTag := findTagForSha(td, entry.Revision)
if foundTag != "" {
entry.RevisionTag = foundTag
} else {
entry.SuggestedTag = td[0].Name
entry.SuggestedRev = td[0].Commit.Sha
}
}
}
return entry, err
}
// Take the vendor manifest file and uses the Github APIs to retrieve the tag data for each Revision
// Outputted data can be consumed by other parsers, or can be input to groupbyrepo
func main() {
// Load the json from stdin (designed to support piping between binaries)
input, err := parser.ReadFromStdin()
if err != nil {
fmt.Fprint(os.Stderr, err)
flag.Usage()
os.Exit(1)
}
client = &http.Client{}
// Parse the data into useful types
m, err := parser.ParseManifest(input)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to parse input json: %v\n", err)
os.Exit(1)
}
// Find the tag for each repository. This is done serially.
var foundTag *parser.ManifestEntry
if err == nil {
for i, me := range m.Dependencies {
foundTag, err = findTag(&me)
if err != nil {
break
}
m.Dependencies[i] = *foundTag
}
}
// Output the original manifest data with new fields added (see ManifestEntry)
if err == nil {
output, _ := json.MarshalIndent(m, "", " ")
fmt.Println(string(output))
} else {
fmt.Fprintf(os.Stderr, "%v\n", err)
}
}

82
vendor/github.com/vmware/vic/infra/util/vendor-manifest-parser/tool/groupbyrepo.go generated vendored Normal file
View File

@@ -0,0 +1,82 @@
// Copyright 2016 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"encoding/json"
"flag"
"fmt"
"net/url"
"os"
"parser"
)
func init() {
flag.Usage = func() {
fmt.Println("Usage: Groups the json output of gettags by repository. Takes input via stdin. Eg: gettags | groupbyrepo")
}
flag.Parse()
}
func groupByRepo(m parser.Manifest) []*parser.ManifestByRepo {
var repoMap = make(map[string]*parser.ManifestByRepo)
for _, me := range m.Dependencies {
u, _ := url.Parse(me.Repository)
repo := u.Path
mapEntry := repoMap[repo]
if mapEntry == nil {
mapEntry = &parser.ManifestByRepo{Repository: repo}
}
mapEntry.Dependencies = append(mapEntry.Dependencies, me)
repoMap[repo] = mapEntry
}
values := make([]*parser.ManifestByRepo, len(repoMap))
i := 0
for _, k := range repoMap {
values[i] = k
i++
}
return values
}
// Takes json output from gettags and groups it by repository. Output is in json format and can be further parsed by report
func main() {
// Read json data from stdin (designed to support piping between binaries)
input, err := parser.ReadFromStdin()
if err != nil {
fmt.Fprint(os.Stderr, err)
flag.Usage()
os.Exit(1)
}
// Parse the data into useful types
m, err := parser.ParseManifest(input)
if err != nil {
fmt.Fprintf(os.Stderr, "Error parsing input json: %v", err)
flag.Usage()
os.Exit(1)
}
// Group the data into a slice of repo->dependencies
values := groupByRepo(m)
// Format the output to stdout
output, err := json.MarshalIndent(values, "", " ")
fmt.Println(string(output))
}

115
vendor/github.com/vmware/vic/infra/util/vendor-manifest-parser/tool/report.go generated vendored Normal file
View File

@@ -0,0 +1,115 @@
// Copyright 2016 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"encoding/json"
"flag"
"fmt"
"io"
"io/ioutil"
"os"
"sort"
"strings"
"parser"
)
var exceptionFile = flag.String("exceptionFile", "", "json file containing documented exceptions to using tagged releases")
func init() {
flag.Usage = func() {
fmt.Println("Usage: Formats the json output of groupbyrepo and considers json exceptions. Takes input via stdin. Eg: gettags | groupbyrepo | report")
flag.PrintDefaults()
}
flag.Parse()
}
func printOutput(out io.Writer, repos parser.SortedManifestByRepo, exceptions map[string]string) {
for _, repo := range repos {
fmt.Fprintf(out, "\n%s\n", repo.Repository)
for _, dep := range repo.Dependencies {
fmt.Fprintf(out, "\t%s", dep.Importpath)
if exceptions[dep.Revision] == "" {
if dep.HasTags {
if dep.RevisionTag != "" {
fmt.Fprintf(out, " is tagged as "+dep.RevisionTag)
} else {
fmt.Fprintf(out, " "+dep.Revision+" is an untagged version. Most recent tag is %s for sha %s", dep.SuggestedTag, dep.SuggestedRev)
}
} else {
if strings.Index(dep.Repository, "github.com") == -1 {
fmt.Fprintf(out, " "+dep.Revision+" sha is from non-Github repo")
} else {
fmt.Fprintf(out, " "+dep.Revision+" sha is from a repo with no tags")
}
}
} else {
fmt.Fprintf(out, " "+dep.Revision+" MUST BE THIS REVISION because %s", exceptions[dep.Revision])
}
fmt.Fprintf(out, "\n")
}
}
}
func loadAndParseExceptions() (map[string]string, error) {
var exceptions map[string]string
var err error
if *exceptionFile != "" {
exceptions = make(map[string]string)
data, err := ioutil.ReadFile(*exceptionFile)
if err == nil {
e := []parser.UntaggedException{}
err := json.Unmarshal(data, &e)
if err == nil {
for _, v := range e {
exceptions[v.Revision] = v.Reason
}
}
}
}
return exceptions, err
}
// Takes the output from groupbyrepo, plus exception data from a file, and produces a human-parsable report
func main() {
// Load the json from stdin (designed to support piping between binaries)
input, err := parser.ReadFromStdin()
if err != nil {
fmt.Fprint(os.Stderr, err)
flag.Usage()
os.Exit(1)
}
// Parse the data into useful types
repos, err := parser.ParseManifestByRepo(input)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to parse input json: %v\n", err)
os.Exit(1)
}
// Load the exception file into useful types
exceptions, err := loadAndParseExceptions()
if err != nil {
fmt.Fprintf(os.Stderr, "Error loading exceptions file: %v\n", err)
os.Exit(1)
}
// Write the report to stdout
sort.Sort(parser.SortedManifestByRepo(repos))
printOutput(os.Stdout, repos, exceptions)
}

105
vendor/github.com/vmware/vic/infra/util/vendor-manifest-parser/types.go generated vendored Normal file
View File

@@ -0,0 +1,105 @@
// Copyright 2016 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package parser
import (
"bufio"
"encoding/json"
"fmt"
"io"
"os"
)
// Manifest is a golang representation of the manifest json
type Manifest struct {
Version int
Dependencies []ManifestEntry
}
func ParseManifest(input []byte) (Manifest, error) {
m := Manifest{}
err := json.Unmarshal(input, &m)
return m, err
}
// ManifestByRepo is like Manifest, except that only the Dependencies for a specific repo are included
type ManifestByRepo struct {
Repository string
Dependencies []ManifestEntry
}
func ParseManifestByRepo(input []byte) ([]ManifestByRepo, error) {
m := []ManifestByRepo{}
err := json.Unmarshal(input, &m)
return m, err
}
type SortedManifestByRepo []ManifestByRepo
func (slice SortedManifestByRepo) Len() int {
return len(slice)
}
func (slice SortedManifestByRepo) Less(i, j int) bool {
return slice[i].Repository < slice[j].Repository
}
func (slice SortedManifestByRepo) Swap(i, j int) {
slice[i], slice[j] = slice[j], slice[i]
}
// ManifestEntry includes the fields from the original manifest plus some additional fields:
// HasTags - does the repository have tags
// RevisionTag - is there a tag corresponding to this revision
// SuggestedTag - if there is no tag, the most recent tag is listed
// SuggestedRev - if a tag is suggested, the revision is also suggested
type ManifestEntry struct {
Importpath string
Repository string
Vcs string
Revision string
HasTags bool
RevisionTag string
SuggestedTag string
SuggestedRev string
Branch string
}
// TagData is the golang representation of the json returned from Github
type TagData struct {
Name string
Commit TagDataCommit
}
// TagDataCommit is the golang representation of the json returned from Github
type TagDataCommit struct {
Sha string
Url string
}
// UntaggedException is the golang representation of a documented exception to revision not being tagged
type UntaggedException struct {
Revision string
Reason string
}
func ReadFromStdin() ([]byte, error) {
bio := bufio.NewReader(os.Stdin)
input, err := bio.ReadBytes(127) // Read all input to EOF
if err != io.EOF {
return nil, fmt.Errorf("Failed to read input: %v\n", err)
}
return input, nil
}