d8dfun/virtual-kubelet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Ria Bhatia
2017-12-04 13:32:57 -06:00
committed by Erik St. Martin
commit 0075e5b0f3
9056 changed files with 2523100 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
vendor/github.com/hyperhq/hypercli/docs/admin/logging/awslogs.md generated vendored Normal file
View File

@@ -0,0 +1,90 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/awslogs/"]
title = "Amazon CloudWatch Logs logging driver"
description = "Describes how to use the Amazon CloudWatch Logs logging driver."
keywords = ["AWS, Amazon, CloudWatch, logging, driver"]
[menu.main]
parent = "smn_logging"
+++
<![end-metadata]-->
# Amazon CloudWatch Logs logging driver
The `awslogs` logging driver sends container logs to
[Amazon CloudWatch Logs](https://aws.amazon.com/cloudwatch/details/#log-monitoring).
Log entries can be retrieved through the [AWS Management
Console](https://console.aws.amazon.com/cloudwatch/home#logs:) or the [AWS SDKs
and Command Line Tools](http://docs.aws.amazon.com/cli/latest/reference/logs/index.html).
## Usage
You can configure the default logging driver by passing the `--log-driver`
option to the Docker daemon:
docker daemon --log-driver=awslogs
You can set the logging driver for a specific container by using the
`--log-driver` option to `docker run`:
docker run --log-driver=awslogs ...
## Amazon CloudWatch Logs options
You can use the `--log-opt NAME=VALUE` flag to specify Amazon CloudWatch Logs logging driver options.
### awslogs-region
The `awslogs` logging driver sends your Docker logs to a specific region. Use
the `awslogs-region` log option or the `AWS_REGION` environment variable to set
the region. By default, if your Docker daemon is running on an EC2 instance
and no region is set, the driver uses the instance's region.
docker run --log-driver=awslogs --log-opt awslogs-region=us-east-1 ...
### awslogs-group
You must specify a
[log group](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/WhatIsCloudWatchLogs.html)
for the `awslogs` logging driver. You can specify the log group with the
`awslogs-group` log option:
docker run --log-driver=awslogs --log-opt awslogs-region=us-east-1 --log-opt awslogs-group=myLogGroup ...
### awslogs-stream
To configure which
[log stream](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/WhatIsCloudWatchLogs.html)
should be used, you can specify the `awslogs-stream` log option. If not
specified, the container ID is used as the log stream.
> **Note:**
> Log streams within a given log group should only be used by one container
> at a time. Using the same log stream for multiple containers concurrently
> can cause reduced logging performance.
## Credentials
You must provide AWS credentials to the Docker daemon to use the `awslogs`
logging driver. You can provide these credentials with the `AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` environment variables, the
default AWS shared credentials file (`~/.aws/credentials` of the root user), or
(if you are running the Docker daemon on an Amazon EC2 instance) the Amazon EC2
instance profile.
Credentials must have a policy applied that allows the `logs:CreateLogStream`
and `logs:PutLogEvents` actions, as shown in the following example.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": "*"
}
]
}

127
vendor/github.com/hyperhq/hypercli/docs/admin/logging/fluentd.md generated vendored Normal file
View File

@@ -0,0 +1,127 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/fluentd/"]
title = "Fluentd logging driver"
description = "Describes how to use the fluentd logging driver."
keywords = ["Fluentd, docker, logging, driver"]
[menu.main]
parent = "smn_logging"
weight=2
+++
<![end-metadata]-->
# Fluentd logging driver
The `fluentd` logging driver sends container logs to the
[Fluentd](http://www.fluentd.org/) collector as structured log data. Then, users
can use any of the [various output plugins of
Fluentd](http://www.fluentd.org/plugins) to write these logs to various
destinations.
In addition to the log message itself, the `fluentd` log
driver sends the following metadata in the structured log message:
| Field | Description |
-------------------|-------------------------------------|
| `container_id` | The full 64-character container ID. |
| `container_name` | The container name at the time it was started. If you use `docker rename` to rename a container, the new name is not reflected in the journal entries. |
| `source` | `stdout` or `stderr` |
The `docker logs` command is not available for this logging driver.
## Usage
Some options are supported by specifying `--log-opt` as many times as needed:
- `fluentd-address`: specify `host:port` to connect `localhost:24224`
- `tag`: specify tag for fluentd message, which interpret some markup, ex `{{.ID}}`, `{{.FullID}}` or `{{.Name}}` `docker.{{.ID}}`
- `fail-on-startup-error`: true/false; Should the logging driver fail container startup in case of connect error during startup. Default: true (backwards compatible)
- `buffer-limit`: Size limit (bytes) for the buffer which is used to buffer messages in case of connection outages. Default: 1M
Configure the default logging driver by passing the
`--log-driver` option to the Docker daemon:
docker daemon --log-driver=fluentd
To set the logging driver for a specific container, pass the
`--log-driver` option to `docker run`:
docker run --log-driver=fluentd ...
Before using this logging driver, launch a Fluentd daemon. The logging driver
connects to this daemon through `localhost:24224` by default. Use the
`fluentd-address` option to connect to a different address.
docker run --log-driver=fluentd --log-opt fluentd-address=myhost.local:24224
If container cannot connect to the Fluentd daemon, the container stops
immediately.
## Options
Users can use the `--log-opt NAME=VALUE` flag to specify additional Fluentd logging driver options.
### fluentd-address
By default, the logging driver connects to `localhost:24224`. Supply the
`fluentd-address` option to connect to a different address.
docker run --log-driver=fluentd --log-opt fluentd-address=myhost.local:24224
### tag
By default, Docker uses the first 12 characters of the container ID to tag log messages.
Refer to the [log tag option documentation](log_tags.md) for customizing
the log tag format.
### labels and env
The `labels` and `env` options each take a comma-separated list of keys. If there is collision between `label` and `env` keys, the value of the `env` takes precedence. Both options add additional fields to the extra attributes of a logging message.
### fail-on-startup-error
By default, if the logging driver cannot connect to the backend it will fail the entire startup of the container. If you wish to ignore potential connect error during container startup supply the `fail-on-startup-error` flag.
docker run --log-driver=fluentd --log-opt fail-on-startup-error=false
### buffer-limit
When fluent driver loses connection, or cannot connect at container startup, it will buffer the log events locally for re-transmission. Buffer limit option controls how much data will be buffered locally, **per container**. Specified in bytes.
docker run --log-driver=fluentd --log-opt buffer-limit=5242880
The above would result to use 5M buffer locally. Keep in mind that during possible connection errors all your containers will start buffering locally and thus might result in considerable memory usage.
## Fluentd daemon management with Docker
About `Fluentd` itself, see [the project webpage](http://www.fluentd.org)
and [its documents](http://docs.fluentd.org/).
To use this logging driver, start the `fluentd` daemon on a host. We recommend
that you use [the Fluentd docker
image](https://registry.hub.docker.com/u/fluent/fluentd/). This image is
especially useful if you want to aggregate multiple container logs on a each
host then, later, transfer the logs to another Fluentd node to create an
aggregate store.
### Testing container loggers
1. Write a configuration file (`test.conf`) to dump input logs:
<source>
@type forward
</source>
<match docker.**>
@type stdout
</match>
2. Launch Fluentd container with this configuration file:
$ docker run -it -p 24224:24224 -v /path/to/conf/test.conf:/fluentd/etc -e FLUENTD_CONF=test.conf fluent/fluentd:latest
3. Start one or more containers with the `fluentd` logging driver:
$ docker run --log-driver=fluentd your/application

22
vendor/github.com/hyperhq/hypercli/docs/admin/logging/index.md generated vendored Normal file
View File

@@ -0,0 +1,22 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/"]
title = "Logging"
description = "Logging and Logging Drivers"
keywords = [" docker, logging, driver"]
[menu.main]
parent = "engine_admin"
identifier = "smn_logging"
weight=8
+++
<![end-metadata]-->
# Logging Drivers
* [Configuring logging drivers](overview.md)
* [Configuring log tags](log_tags.md)
* [Fluentd logging driver](fluentd.md)
* [Journald logging driver](journald.md)
* [Amazon CloudWatch Logs logging driver](awslogs.md)
* [Splunk logging driver](splunk.md)

92
vendor/github.com/hyperhq/hypercli/docs/admin/logging/journald.md generated vendored Normal file
View File

@@ -0,0 +1,92 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/journald/"]
title = "journald logging driver"
description = "Describes how to use the fluentd logging driver."
keywords = ["Fluentd, docker, logging, driver"]
[menu.main]
parent = "smn_logging"
weight = 2
+++
<![end-metadata]-->
# Journald logging driver
The `journald` logging driver sends container logs to the [systemd
journal](http://www.freedesktop.org/software/systemd/man/systemd-journald.service.html). Log entries can be retrieved using the `journalctl`
command, through use of the journal API, or using the `docker logs` command.
In addition to the text of the log message itself, the `journald` log
driver stores the following metadata in the journal with each message:
| Field | Description |
----------------------|-------------|
| `CONTAINER_ID` | The container ID truncated to 12 characters. |
| `CONTAINER_ID_FULL` | The full 64-character container ID. |
| `CONTAINER_NAME` | The container name at the time it was started. If you use `docker rename` to rename a container, the new name is not reflected in the journal entries. |
| `CONTAINER_TAG` | The container tag ([log tag option documentation](log_tags.md)). |
## Usage
You can configure the default logging driver by passing the
`--log-driver` option to the Docker daemon:
docker daemon --log-driver=journald
You can set the logging driver for a specific container by using the
`--log-driver` option to `docker run`:
docker run --log-driver=journald ...
## Options
Users can use the `--log-opt NAME=VALUE` flag to specify additional
journald logging driver options.
### tag
Specify template to set `CONTAINER_TAG` value in journald logs. Refer to
[log tag option documentation](log_tags.md) for customizing the log tag format.
### labels and env
The `labels` and `env` options each take a comma-separated list of keys. If there is collision between `label` and `env` keys, the value of the `env` takes precedence. Both options add additional metadata in the journal with each message.
## Note regarding container names
The value logged in the `CONTAINER_NAME` field is the container name
that was set at startup. If you use `docker rename` to rename a
container, the new name will not be reflected in the journal entries.
Journal entries will continue to use the original name.
## Retrieving log messages with journalctl
You can use the `journalctl` command to retrieve log messages. You
can apply filter expressions to limit the retrieved messages to a
specific container. For example, to retrieve all log messages from a
container referenced by name:
# journalctl CONTAINER_NAME=webserver
You can make use of additional filters to further limit the messages
retrieved. For example, to see just those messages generated since
the system last booted:
# journalctl -b CONTAINER_NAME=webserver
Or to retrieve log messages in JSON format with complete metadata:
# journalctl -o json CONTAINER_NAME=webserver
## Retrieving log messages with the journal API
This example uses the `systemd` Python module to retrieve container
logs:
import systemd.journal
reader = systemd.journal.Reader()
reader.add_match('CONTAINER_NAME=web')
for msg in reader:
print '{CONTAINER_ID_FULL}: {MESSAGE}'.format(**msg)

51
vendor/github.com/hyperhq/hypercli/docs/admin/logging/log_tags.md generated vendored Normal file
View File

@@ -0,0 +1,51 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/log_tags/"]
title = "Log tags for logging driver"
description = "Describes how to format tags for."
keywords = ["docker, logging, driver, syslog, Fluentd, gelf, journald"]
[menu.main]
parent = "smn_logging"
weight = 1
+++
<![end-metadata]-->
# Log Tags
The `tag` log option specifies how to format a tag that identifies the
container's log messages. By default, the system uses the first 12 characters of
the container id. To override this behavior, specify a `tag` option:
```
docker run --log-driver=fluentd --log-opt fluentd-address=myhost.local:24224 --log-opt tag="mailer"
```
Docker supports some special template markup you can use when specifying a tag's value:
| Markup | Description |
|--------------------|------------------------------------------------------|
| `{{.ID}}` | The first 12 characters of the container id. |
| `{{.FullID}}` | The full container id. |
| `{{.Name}}` | The container name. |
| `{{.ImageID}}` | The first 12 characters of the container's image id. |
| `{{.ImageFullID}}` | The container's full image identifier. |
| `{{.ImageName}}` | The name of the image used by the container. |
For example, specifying a `--log-opt tag="{{.ImageName}}/{{.Name}}/{{.ID}}"` value yields `syslog` log lines like:
```
Aug 7 18:33:19 HOSTNAME docker/hello-world/foobar/5790672ab6a0[9103]: Hello from Docker.
```
At startup time, the system sets the `container_name` field and `{{.Name}}` in
the tags. If you use `docker rename` to rename a container, the new name is not
reflected in the log messages. Instead, these messages continue to use the
original container name.
For advanced usage, the generated tag's use [go
templates](http://golang.org/pkg/text/template/) and the container's [logging
context](https://github.com/docker/docker/blob/master/daemon/logger/context.go).
>**Note**:The driver specific log options `syslog-tag`, `fluentd-tag` and
>`gelf-tag` still work for backwards compatibility. However, going forward you
>should standardize on using the generic `tag` log option instead.

206
vendor/github.com/hyperhq/hypercli/docs/admin/logging/overview.md generated vendored Normal file
View File

@@ -0,0 +1,206 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/overview/"]
title = "Configuring Logging Drivers"
description = "Configure logging driver."
keywords = ["docker, logging, driver, Fluentd"]
[menu.main]
parent = "smn_logging"
weight=-1
+++
<![end-metadata]-->
# Configure logging drivers
The container can have a different logging driver than the Docker daemon. Use
the `--log-driver=VALUE` with the `docker run` command to configure the
container's logging driver. The following options are supported:
| `none` | Disables any logging for the container. `docker logs` won't be available with this driver. |
|-------------|-------------------------------------------------------------------------------------------------------------------------------|
| `json-file` | Default logging driver for Docker. Writes JSON messages to file. |
| `syslog` | Syslog logging driver for Docker. Writes log messages to syslog. |
| `journald` | Journald logging driver for Docker. Writes log messages to `journald`. |
| `gelf` | Graylog Extended Log Format (GELF) logging driver for Docker. Writes log messages to a GELF endpoint likeGraylog or Logstash. |
| `fluentd` | Fluentd logging driver for Docker. Writes log messages to `fluentd` (forward input). |
| `awslogs` | Amazon CloudWatch Logs logging driver for Docker. Writes log messages to Amazon CloudWatch Logs. |
| `splunk` | Splunk logging driver for Docker. Writes log messages to `splunk` using HTTP Event Collector. |
The `docker logs`command is available only for the `json-file` and `journald`
logging drivers.
The `labels` and `env` options add additional attributes for use with logging drivers that accept them. Each option takes a comma-separated list of keys. If there is collision between `label` and `env` keys, the value of the `env` takes precedence.
To use attributes, specify them when you start the Docker daemon.
```
docker daemon --log-driver=json-file --log-opt labels=foo --log-opt env=foo,fizz
```
Then, run a container and specify values for the `labels` or `env`. For example, you might use this:
```
docker run --label foo=bar -e fizz=buzz -d -P training/webapp python app.py
```
This adds additional fields to the log depending on the driver, e.g. for
`json-file` that looks like:
"attrs":{"fizz":"buzz","foo":"bar"}
## json-file options
The following logging options are supported for the `json-file` logging driver:
--log-opt max-size=[0-9+][k|m|g]
--log-opt max-file=[0-9+]
--log-opt labels=label1,label2
--log-opt env=env1,env2
Logs that reach `max-size` are rolled over. You can set the size in kilobytes(k), megabytes(m), or gigabytes(g). eg `--log-opt max-size=50m`. If `max-size` is not set, then logs are not rolled over.
`max-file` specifies the maximum number of files that a log is rolled over before being discarded. eg `--log-opt max-file=100`. If `max-size` is not set, then `max-file` is not honored.
If `max-size` and `max-file` are set, `docker logs` only returns the log lines from the newest log file.
## syslog options
The following logging options are supported for the `syslog` logging driver:
--log-opt syslog-address=[tcp|udp|tcp+tls]://host:port
--log-opt syslog-address=unix://path
--log-opt syslog-facility=daemon
--log-opt syslog-tls-ca-cert=/etc/ca-certificates/custom/ca.pem
--log-opt syslog-tls-cert=/etc/ca-certificates/custom/cert.pem
--log-opt syslog-tls-key=/etc/ca-certificates/custom/key.pem
--log-opt syslog-tls-skip-verify=true
--log-opt tag="mailer"
`syslog-address` specifies the remote syslog server address where the driver connects to.
If not specified it defaults to the local unix socket of the running system.
If transport is either `tcp` or `udp` and `port` is not specified it defaults to `514`
The following example shows how to have the `syslog` driver connect to a `syslog`
remote server at `192.168.0.42` on port `123`
$ docker run --log-driver=syslog --log-opt syslog-address=tcp://192.168.0.42:123
The `syslog-facility` option configures the syslog facility. By default, the system uses the
`daemon` value. To override this behavior, you can provide an integer of 0 to 23 or any of
the following named facilities:
* `kern`
* `user`
* `mail`
* `daemon`
* `auth`
* `syslog`
* `lpr`
* `news`
* `uucp`
* `cron`
* `authpriv`
* `ftp`
* `local0`
* `local1`
* `local2`
* `local3`
* `local4`
* `local5`
* `local6`
* `local7`
`syslog-tls-ca-cert` specifies the absolute path to the trust certificates
signed by the CA. This option is ignored if the address protocol is not `tcp+tls`.
`syslog-tls-cert` specifies the absolute path to the TLS certificate file.
This option is ignored if the address protocol is not `tcp+tls`.
`syslog-tls-key` specifies the absolute path to the TLS key file.
This option is ignored if the address protocol is not `tcp+tls`.
`syslog-tls-skip-verify` configures the TLS verification.
This verification is enabled by default, but it can be overriden by setting
this option to `true`. This option is ignored if the address protocol is not `tcp+tls`.
By default, Docker uses the first 12 characters of the container ID to tag log messages.
Refer to the [log tag option documentation](log_tags.md) for customizing
the log tag format.
## journald options
The `journald` logging driver stores the container id in the journal's `CONTAINER_ID` field. For detailed information on
working with this logging driver, see [the journald logging driver](journald.md)
reference documentation.
## gelf options
The GELF logging driver supports the following options:
--log-opt gelf-address=udp://host:port
--log-opt tag="database"
--log-opt labels=label1,label2
--log-opt env=env1,env2
The `gelf-address` option specifies the remote GELF server address that the
driver connects to. Currently, only `udp` is supported as the transport and you must
specify a `port` value. The following example shows how to connect the `gelf`
driver to a GELF remote server at `192.168.0.42` on port `12201`
$ docker run --log-driver=gelf --log-opt gelf-address=udp://192.168.0.42:12201
By default, Docker uses the first 12 characters of the container ID to tag log messages.
Refer to the [log tag option documentation](log_tags.md) for customizing
the log tag format.
The `labels` and `env` options are supported by the gelf logging
driver. It adds additional key on the `extra` fields, prefixed by an
underscore (`_`).
// […]
"_foo": "bar",
"_fizz": "buzz",
// […]
## fluentd options
You can use the `--log-opt NAME=VALUE` flag to specify these additional Fluentd logging driver options.
- `fluentd-address`: specify `host:port` to connect [localhost:24224]
- `tag`: specify tag for `fluentd` message,
- `fail-on-startup-error`: true/false; Should the logging driver fail container startup in case of connect error during startup. Default: true (backwards compatible)
- `buffer-limit`: Size limit (bytes) for the buffer which is used to buffer messages in case of connection outages. Default: 1M
For example, to specify both additional options:
`docker run --log-driver=fluentd --log-opt fluentd-address=localhost:24224 --log-opt tag=docker.{{.Name}}`
If container cannot connect to the Fluentd daemon on the specified address,
the container stops immediately. For detailed information on working with this
logging driver, see [the fluentd logging driver](fluentd.md)
## Specify Amazon CloudWatch Logs options
The Amazon CloudWatch Logs logging driver supports the following options:
--log-opt awslogs-region=<aws_region>
--log-opt awslogs-group=<log_group_name>
--log-opt awslogs-stream=<log_stream_name>
For detailed information on working with this logging driver, see [the awslogs logging driver](awslogs.md) reference documentation.
## Splunk options
The Splunk logging driver requires the following options:
--log-opt splunk-token=<splunk_http_event_collector_token>
--log-opt splunk-url=https://your_splunk_instance:8088
For detailed information about working with this logging driver, see the [Splunk logging driver](splunk.md)
reference documentation.

69
vendor/github.com/hyperhq/hypercli/docs/admin/logging/splunk.md generated vendored Normal file
View File

@@ -0,0 +1,69 @@
<!--[metadata]>
+++
aliases = ["/engine/reference/logging/splunk/"]
title = "Splunk logging driver"
description = "Describes how to use the Splunk logging driver."
keywords = ["splunk, docker, logging, driver"]
[menu.main]
parent = "smn_logging"
weight = 2
+++
<![end-metadata]-->
# Splunk logging driver
The `splunk` logging driver sends container logs to
[HTTP Event Collector](http://dev.splunk.com/view/event-collector/SP-CAAAE6M)
in Splunk Enterprise and Splunk Cloud.
## Usage
You can configure the default logging driver by passing the `--log-driver`
option to the Docker daemon:
docker daemon --log-driver=splunk
You can set the logging driver for a specific container by using the
`--log-driver` option to `docker run`:
docker run --log-driver=splunk ...
## Splunk options
You can use the `--log-opt NAME=VALUE` flag to specify these additional Splunk
logging driver options:
| Option | Required | Description |
|-----------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `splunk-token` | required | Splunk HTTP Event Collector token. |
| `splunk-url` | required | Path to your Splunk Enterprise or Splunk Cloud instance (including port and schema used by HTTP Event Collector) `https://your_splunk_instance:8088`. |
| `splunk-source` | optional | Event source. |
| `splunk-sourcetype` | optional | Event source type. |
| `splunk-index` | optional | Event index. |
| `splunk-capath` | optional | Path to root certificate. |
| `splunk-caname` | optional | Name to use for validating server certificate; by default the hostname of the `splunk-url` will be used. |
| `splunk-insecureskipverify` | optional | Ignore server certificate validation. |
| `tag` | optional | Specify tag for message, which interpret some markup. Default value is `{{.ID}}` (12 characters of the container ID). Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. |
| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. |
| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. |
If there is collision between `label` and `env` keys, the value of the `env` takes precedence.
Both options add additional fields to the attributes of a logging message.
Below is an example of the logging option specified for the Splunk Enterprise
instance. The instance is installed locally on the same machine on which the
Docker daemon is running. The path to the root certificate and Common Name is
specified using an HTTPS schema. This is used for verification.
The `SplunkServerDefaultCert` is automatically generated by Splunk certificates.
docker run --log-driver=splunk \
--log-opt splunk-token=176FCEBF-4CF5-4EDF-91BC-703796522D20 \
--log-opt splunk-url=https://splunkhost:8088 \
--log-opt splunk-capath=/path/to/cert/cacert.pem \
--log-opt splunk-caname=SplunkServerDefaultCert
--log-opt tag="{{.Name}}/{{.FullID}}"
--log-opt labels=location
--log-opt env=TEST
--env "TEST=false"
--label location=west
your/application