Add HashiCorp Nomad provider (#483)

* provider: adding Nomad provider

* updating CONTRIBUTING.md with Nomad provider

* updated README.md by adding the Nomad provider

* fix typo

* adding nomad/api and nomad/testutil deps

* adding Nomad binary dependency for provider tests

* fixed the nomad binary download command step and added tolerations to the nomad provider.

* adding nomad provider demo gif

* adding my name to authors

* adding two missing go-rootcerts files after dep ensure

* delete pod comment
This commit is contained in:
Anubhav Mishra
2019-01-08 01:18:11 +05:30
committed by Robbie Zhang
parent 5796be449b
commit a46e1dd2ce
332 changed files with 126455 additions and 2 deletions

363
vendor/github.com/hashicorp/nomad/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,363 @@
Mozilla Public License, version 2.0
1. Definitions
1.1. "Contributor"
means each individual or legal entity that creates, contributes to the
creation of, or owns Covered Software.
1.2. "Contributor Version"
means the combination of the Contributions of others (if any) used by a
Contributor and that particular Contributor's Contribution.
1.3. "Contribution"
means Covered Software of a particular Contributor.
1.4. "Covered Software"
means Source Code Form to which the initial Contributor has attached the
notice in Exhibit A, the Executable Form of such Source Code Form, and
Modifications of such Source Code Form, in each case including portions
thereof.
1.5. "Incompatible With Secondary Licenses"
means
a. that the initial Contributor has attached the notice described in
Exhibit B to the Covered Software; or
b. that the Covered Software was made available under the terms of
version 1.1 or earlier of the License, but not also under the terms of
a Secondary License.
1.6. "Executable Form"
means any form of the work other than Source Code Form.
1.7. "Larger Work"
means a work that combines Covered Software with other material, in a
separate file or files, that is not Covered Software.
1.8. "License"
means this document.
1.9. "Licensable"
means having the right to grant, to the maximum extent possible, whether
at the time of the initial grant or subsequently, any and all of the
rights conveyed by this License.
1.10. "Modifications"
means any of the following:
a. any file in Source Code Form that results from an addition to,
deletion from, or modification of the contents of Covered Software; or
b. any new file in Source Code Form that contains any Covered Software.
1.11. "Patent Claims" of a Contributor
means any patent claim(s), including without limitation, method,
process, and apparatus claims, in any patent Licensable by such
Contributor that would be infringed, but for the grant of the License,
by the making, using, selling, offering for sale, having made, import,
or transfer of either its Contributions or its Contributor Version.
1.12. "Secondary License"
means either the GNU General Public License, Version 2.0, the GNU Lesser
General Public License, Version 2.1, the GNU Affero General Public
License, Version 3.0, or any later versions of those licenses.
1.13. "Source Code Form"
means the form of the work preferred for making modifications.
1.14. "You" (or "Your")
means an individual or a legal entity exercising rights under this
License. For legal entities, "You" includes any entity that controls, is
controlled by, or is under common control with You. For purposes of this
definition, "control" means (a) the power, direct or indirect, to cause
the direction or management of such entity, whether by contract or
otherwise, or (b) ownership of more than fifty percent (50%) of the
outstanding shares or beneficial ownership of such entity.
2. License Grants and Conditions
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
a. under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or
as part of a Larger Work; and
b. under Patent Claims of such Contributor to make, use, sell, offer for
sale, have made, import, and otherwise transfer either its
Contributions or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution
become effective for each Contribution on the date the Contributor first
distributes such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under
this License. No additional rights or licenses will be implied from the
distribution or licensing of Covered Software under this License.
Notwithstanding Section 2.1(b) above, no patent license is granted by a
Contributor:
a. for any code that a Contributor has removed from Covered Software; or
b. for infringements caused by: (i) Your and any other third party's
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
c. under Patent Claims infringed by Covered Software in the absence of
its Contributions.
This License does not grant any rights in the trademarks, service marks,
or logos of any Contributor (except as may be necessary to comply with
the notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this
License (see Section 10.2) or under the terms of a Secondary License (if
permitted under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its
Contributions are its original creation(s) or it has sufficient rights to
grant the rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under
applicable copyright doctrines of fair use, fair dealing, or other
equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
Section 2.1.
3. Responsibilities
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under
the terms of this License. You must inform recipients that the Source
Code Form of the Covered Software is governed by the terms of this
License, and how they can obtain a copy of this License. You may not
attempt to alter or restrict the recipients' rights in the Source Code
Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
a. such Covered Software must also be made available in Source Code Form,
as described in Section 3.1, and You must inform recipients of the
Executable Form how they can obtain a copy of such Source Code Form by
reasonable means in a timely manner, at a charge no more than the cost
of distribution to the recipient; and
b. You may distribute such Executable Form under the terms of this
License, or sublicense it under different terms, provided that the
license for the Executable Form does not attempt to limit or alter the
recipients' rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for
the Covered Software. If the Larger Work is a combination of Covered
Software with a work governed by one or more Secondary Licenses, and the
Covered Software is not Incompatible With Secondary Licenses, this
License permits You to additionally distribute such Covered Software
under the terms of such Secondary License(s), so that the recipient of
the Larger Work may, at their option, further distribute the Covered
Software under the terms of either this License or such Secondary
License(s).
3.4. Notices
You may not remove or alter the substance of any license notices
(including copyright notices, patent notices, disclaimers of warranty, or
limitations of liability) contained within the Source Code Form of the
Covered Software, except that You may alter any license notices to the
extent required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on
behalf of any Contributor. You must make it absolutely clear that any
such warranty, support, indemnity, or liability obligation is offered by
You alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
If it is impossible for You to comply with any of the terms of this License
with respect to some or all of the Covered Software due to statute,
judicial order, or regulation then You must: (a) comply with the terms of
this License to the maximum extent possible; and (b) describe the
limitations and the code they affect. Such description must be placed in a
text file included with all distributions of the Covered Software under
this License. Except to the extent prohibited by statute or regulation,
such description must be sufficiently detailed for a recipient of ordinary
skill to be able to understand it.
5. Termination
5.1. The rights granted under this License will terminate automatically if You
fail to comply with any of its terms. However, if You become compliant,
then the rights granted under this License from a particular Contributor
are reinstated (a) provisionally, unless and until such Contributor
explicitly and finally terminates Your grants, and (b) on an ongoing
basis, if such Contributor fails to notify You of the non-compliance by
some reasonable means prior to 60 days after You have come back into
compliance. Moreover, Your grants from a particular Contributor are
reinstated on an ongoing basis if such Contributor notifies You of the
non-compliance by some reasonable means, this is the first time You have
received notice of non-compliance with this License from such
Contributor, and You become compliant prior to 30 days after Your receipt
of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions,
counter-claims, and cross-claims) alleging that a Contributor Version
directly or indirectly infringes any patent, then the rights granted to
You by any and all Contributors for the Covered Software under Section
2.1 of this License shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
license agreements (excluding distributors and resellers) which have been
validly granted by You or Your distributors under this License prior to
termination shall survive termination.
6. Disclaimer of Warranty
Covered Software is provided under this License on an "as is" basis,
without warranty of any kind, either expressed, implied, or statutory,
including, without limitation, warranties that the Covered Software is free
of defects, merchantable, fit for a particular purpose or non-infringing.
The entire risk as to the quality and performance of the Covered Software
is with You. Should any Covered Software prove defective in any respect,
You (not any Contributor) assume the cost of any necessary servicing,
repair, or correction. This disclaimer of warranty constitutes an essential
part of this License. No use of any Covered Software is authorized under
this License except under this disclaimer.
7. Limitation of Liability
Under no circumstances and under no legal theory, whether tort (including
negligence), contract, or otherwise, shall any Contributor, or anyone who
distributes Covered Software as permitted above, be liable to You for any
direct, indirect, special, incidental, or consequential damages of any
character including, without limitation, damages for lost profits, loss of
goodwill, work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses, even if such party shall have been
informed of the possibility of such damages. This limitation of liability
shall not apply to liability for death or personal injury resulting from
such party's negligence to the extent applicable law prohibits such
limitation. Some jurisdictions do not allow the exclusion or limitation of
incidental or consequential damages, so this exclusion and limitation may
not apply to You.
8. Litigation
Any litigation relating to this License may be brought only in the courts
of a jurisdiction where the defendant maintains its principal place of
business and such litigation shall be governed by laws of that
jurisdiction, without reference to its conflict-of-law provisions. Nothing
in this Section shall prevent a party's ability to bring cross-claims or
counter-claims.
9. Miscellaneous
This License represents the complete agreement concerning the subject
matter hereof. If any provision of this License is held to be
unenforceable, such provision shall be reformed only to the extent
necessary to make it enforceable. Any law or regulation which provides that
the language of a contract shall be construed against the drafter shall not
be used to construe this License against a Contributor.
10. Versions of the License
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version
of the License under which You originally received the Covered Software,
or under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a
modified version of this License if you rename the license and remove
any references to the name of the license steward (except to note that
such modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary
Licenses If You choose to distribute Source Code Form that is
Incompatible With Secondary Licenses under the terms of this version of
the License, the notice described in Exhibit B of this License must be
attached.
Exhibit A - Source Code Form License Notice
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular file,
then You may include the notice in a location (such as a LICENSE file in a
relevant directory) where a recipient would be likely to look for such a
notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - "Incompatible With Secondary Licenses" Notice
This Source Code Form is "Incompatible
With Secondary Licenses", as defined by
the Mozilla Public License, v. 2.0.

281
vendor/github.com/hashicorp/nomad/acl/acl.go generated vendored Normal file
View File

@@ -0,0 +1,281 @@
package acl
import (
"fmt"
iradix "github.com/hashicorp/go-immutable-radix"
)
// ManagementACL is a singleton used for management tokens
var ManagementACL *ACL
func init() {
var err error
ManagementACL, err = NewACL(true, nil)
if err != nil {
panic(fmt.Errorf("failed to setup management ACL: %v", err))
}
}
// capabilitySet is a type wrapper to help managing a set of capabilities
type capabilitySet map[string]struct{}
func (c capabilitySet) Check(k string) bool {
_, ok := c[k]
return ok
}
func (c capabilitySet) Set(k string) {
c[k] = struct{}{}
}
func (c capabilitySet) Clear() {
for cap := range c {
delete(c, cap)
}
}
// ACL object is used to convert a set of policies into a structure that
// can be efficiently evaluated to determine if an action is allowed.
type ACL struct {
// management tokens are allowed to do anything
management bool
// namespaces maps a namespace to a capabilitySet
namespaces *iradix.Tree
agent string
node string
operator string
quota string
}
// maxPrivilege returns the policy which grants the most privilege
// This handles the case of Deny always taking maximum precedence.
func maxPrivilege(a, b string) string {
switch {
case a == PolicyDeny || b == PolicyDeny:
return PolicyDeny
case a == PolicyWrite || b == PolicyWrite:
return PolicyWrite
case a == PolicyRead || b == PolicyRead:
return PolicyRead
default:
return ""
}
}
// NewACL compiles a set of policies into an ACL object
func NewACL(management bool, policies []*Policy) (*ACL, error) {
// Hot-path management tokens
if management {
return &ACL{management: true}, nil
}
// Create the ACL object
acl := &ACL{}
nsTxn := iradix.New().Txn()
for _, policy := range policies {
NAMESPACES:
for _, ns := range policy.Namespaces {
// Check for existing capabilities
var capabilities capabilitySet
raw, ok := nsTxn.Get([]byte(ns.Name))
if ok {
capabilities = raw.(capabilitySet)
} else {
capabilities = make(capabilitySet)
nsTxn.Insert([]byte(ns.Name), capabilities)
}
// Deny always takes precedence
if capabilities.Check(NamespaceCapabilityDeny) {
continue NAMESPACES
}
// Add in all the capabilities
for _, cap := range ns.Capabilities {
if cap == NamespaceCapabilityDeny {
// Overwrite any existing capabilities
capabilities.Clear()
capabilities.Set(NamespaceCapabilityDeny)
continue NAMESPACES
}
capabilities.Set(cap)
}
}
// Take the maximum privilege for agent, node, and operator
if policy.Agent != nil {
acl.agent = maxPrivilege(acl.agent, policy.Agent.Policy)
}
if policy.Node != nil {
acl.node = maxPrivilege(acl.node, policy.Node.Policy)
}
if policy.Operator != nil {
acl.operator = maxPrivilege(acl.operator, policy.Operator.Policy)
}
if policy.Quota != nil {
acl.quota = maxPrivilege(acl.quota, policy.Quota.Policy)
}
}
// Finalize the namespaces
acl.namespaces = nsTxn.Commit()
return acl, nil
}
// AllowNsOp is shorthand for AllowNamespaceOperation
func (a *ACL) AllowNsOp(ns string, op string) bool {
return a.AllowNamespaceOperation(ns, op)
}
// AllowNamespaceOperation checks if a given operation is allowed for a namespace
func (a *ACL) AllowNamespaceOperation(ns string, op string) bool {
// Hot path management tokens
if a.management {
return true
}
// Check for a matching capability set
raw, ok := a.namespaces.Get([]byte(ns))
if !ok {
return false
}
// Check if the capability has been granted
capabilities := raw.(capabilitySet)
return capabilities.Check(op)
}
// AllowNamespace checks if any operations are allowed for a namespace
func (a *ACL) AllowNamespace(ns string) bool {
// Hot path management tokens
if a.management {
return true
}
// Check for a matching capability set
raw, ok := a.namespaces.Get([]byte(ns))
if !ok {
return false
}
// Check if the capability has been granted
capabilities := raw.(capabilitySet)
if len(capabilities) == 0 {
return false
}
return !capabilities.Check(PolicyDeny)
}
// AllowAgentRead checks if read operations are allowed for an agent
func (a *ACL) AllowAgentRead() bool {
switch {
case a.management:
return true
case a.agent == PolicyWrite:
return true
case a.agent == PolicyRead:
return true
default:
return false
}
}
// AllowAgentWrite checks if write operations are allowed for an agent
func (a *ACL) AllowAgentWrite() bool {
switch {
case a.management:
return true
case a.agent == PolicyWrite:
return true
default:
return false
}
}
// AllowNodeRead checks if read operations are allowed for a node
func (a *ACL) AllowNodeRead() bool {
switch {
case a.management:
return true
case a.node == PolicyWrite:
return true
case a.node == PolicyRead:
return true
default:
return false
}
}
// AllowNodeWrite checks if write operations are allowed for a node
func (a *ACL) AllowNodeWrite() bool {
switch {
case a.management:
return true
case a.node == PolicyWrite:
return true
default:
return false
}
}
// AllowOperatorRead checks if read operations are allowed for a operator
func (a *ACL) AllowOperatorRead() bool {
switch {
case a.management:
return true
case a.operator == PolicyWrite:
return true
case a.operator == PolicyRead:
return true
default:
return false
}
}
// AllowOperatorWrite checks if write operations are allowed for a operator
func (a *ACL) AllowOperatorWrite() bool {
switch {
case a.management:
return true
case a.operator == PolicyWrite:
return true
default:
return false
}
}
// AllowQuotaRead checks if read operations are allowed for all quotas
func (a *ACL) AllowQuotaRead() bool {
switch {
case a.management:
return true
case a.quota == PolicyWrite:
return true
case a.quota == PolicyRead:
return true
default:
return false
}
}
// AllowQuotaWrite checks if write operations are allowed for quotas
func (a *ACL) AllowQuotaWrite() bool {
switch {
case a.management:
return true
case a.quota == PolicyWrite:
return true
default:
return false
}
}
// IsManagement checks if this represents a management token
func (a *ACL) IsManagement() bool {
return a.management
}

191
vendor/github.com/hashicorp/nomad/acl/policy.go generated vendored Normal file
View File

@@ -0,0 +1,191 @@
package acl
import (
"fmt"
"regexp"
"github.com/hashicorp/hcl"
)
const (
// The following levels are the only valid values for the `policy = "read"` stanza.
// When policies are merged together, the most privilege is granted, except for deny
// which always takes precedence and supercedes.
PolicyDeny = "deny"
PolicyRead = "read"
PolicyWrite = "write"
)
const (
// The following are the fine-grained capabilities that can be granted within a namespace.
// The Policy stanza is a short hand for granting several of these. When capabilities are
// combined we take the union of all capabilities. If the deny capability is present, it
// takes precedence and overwrites all other capabilities.
NamespaceCapabilityDeny = "deny"
NamespaceCapabilityListJobs = "list-jobs"
NamespaceCapabilityReadJob = "read-job"
NamespaceCapabilitySubmitJob = "submit-job"
NamespaceCapabilityDispatchJob = "dispatch-job"
NamespaceCapabilityReadLogs = "read-logs"
NamespaceCapabilityReadFS = "read-fs"
NamespaceCapabilitySentinelOverride = "sentinel-override"
)
var (
validNamespace = regexp.MustCompile("^[a-zA-Z0-9-]{1,128}$")
)
// Policy represents a parsed HCL or JSON policy.
type Policy struct {
Namespaces []*NamespacePolicy `hcl:"namespace,expand"`
Agent *AgentPolicy `hcl:"agent"`
Node *NodePolicy `hcl:"node"`
Operator *OperatorPolicy `hcl:"operator"`
Quota *QuotaPolicy `hcl:"quota"`
Raw string `hcl:"-"`
}
// IsEmpty checks to make sure that at least one policy has been set and is not
// comprised of only a raw policy.
func (p *Policy) IsEmpty() bool {
return len(p.Namespaces) == 0 &&
p.Agent == nil &&
p.Node == nil &&
p.Operator == nil &&
p.Quota == nil
}
// NamespacePolicy is the policy for a specific namespace
type NamespacePolicy struct {
Name string `hcl:",key"`
Policy string
Capabilities []string
}
type AgentPolicy struct {
Policy string
}
type NodePolicy struct {
Policy string
}
type OperatorPolicy struct {
Policy string
}
type QuotaPolicy struct {
Policy string
}
// isPolicyValid makes sure the given string matches one of the valid policies.
func isPolicyValid(policy string) bool {
switch policy {
case PolicyDeny, PolicyRead, PolicyWrite:
return true
default:
return false
}
}
// isNamespaceCapabilityValid ensures the given capability is valid for a namespace policy
func isNamespaceCapabilityValid(cap string) bool {
switch cap {
case NamespaceCapabilityDeny, NamespaceCapabilityListJobs, NamespaceCapabilityReadJob,
NamespaceCapabilitySubmitJob, NamespaceCapabilityDispatchJob, NamespaceCapabilityReadLogs,
NamespaceCapabilityReadFS:
return true
// Separate the enterprise-only capabilities
case NamespaceCapabilitySentinelOverride:
return true
default:
return false
}
}
// expandNamespacePolicy provides the equivalent set of capabilities for
// a namespace policy
func expandNamespacePolicy(policy string) []string {
switch policy {
case PolicyDeny:
return []string{NamespaceCapabilityDeny}
case PolicyRead:
return []string{
NamespaceCapabilityListJobs,
NamespaceCapabilityReadJob,
}
case PolicyWrite:
return []string{
NamespaceCapabilityListJobs,
NamespaceCapabilityReadJob,
NamespaceCapabilitySubmitJob,
NamespaceCapabilityDispatchJob,
NamespaceCapabilityReadLogs,
NamespaceCapabilityReadFS,
}
default:
return nil
}
}
// Parse is used to parse the specified ACL rules into an
// intermediary set of policies, before being compiled into
// the ACL
func Parse(rules string) (*Policy, error) {
// Decode the rules
p := &Policy{Raw: rules}
if rules == "" {
// Hot path for empty rules
return p, nil
}
// Attempt to parse
if err := hcl.Decode(p, rules); err != nil {
return nil, fmt.Errorf("Failed to parse ACL Policy: %v", err)
}
// At least one valid policy must be specified, we don't want to store only
// raw data
if p.IsEmpty() {
return nil, fmt.Errorf("Invalid policy: %s", p.Raw)
}
// Validate the policy
for _, ns := range p.Namespaces {
if !validNamespace.MatchString(ns.Name) {
return nil, fmt.Errorf("Invalid namespace name: %#v", ns)
}
if ns.Policy != "" && !isPolicyValid(ns.Policy) {
return nil, fmt.Errorf("Invalid namespace policy: %#v", ns)
}
for _, cap := range ns.Capabilities {
if !isNamespaceCapabilityValid(cap) {
return nil, fmt.Errorf("Invalid namespace capability '%s': %#v", cap, ns)
}
}
// Expand the short hand policy to the capabilities and
// add to any existing capabilities
if ns.Policy != "" {
extraCap := expandNamespacePolicy(ns.Policy)
ns.Capabilities = append(ns.Capabilities, extraCap...)
}
}
if p.Agent != nil && !isPolicyValid(p.Agent.Policy) {
return nil, fmt.Errorf("Invalid agent policy: %#v", p.Agent)
}
if p.Node != nil && !isPolicyValid(p.Node.Policy) {
return nil, fmt.Errorf("Invalid node policy: %#v", p.Node)
}
if p.Operator != nil && !isPolicyValid(p.Operator.Policy) {
return nil, fmt.Errorf("Invalid operator policy: %#v", p.Operator)
}
if p.Quota != nil && !isPolicyValid(p.Quota.Policy) {
return nil, fmt.Errorf("Invalid quota policy: %#v", p.Quota)
}
return p, nil
}

196
vendor/github.com/hashicorp/nomad/api/acl.go generated vendored Normal file
View File

@@ -0,0 +1,196 @@
package api
import (
"fmt"
"time"
)
// ACLPolicies is used to query the ACL Policy endpoints.
type ACLPolicies struct {
client *Client
}
// ACLPolicies returns a new handle on the ACL policies.
func (c *Client) ACLPolicies() *ACLPolicies {
return &ACLPolicies{client: c}
}
// List is used to dump all of the policies.
func (a *ACLPolicies) List(q *QueryOptions) ([]*ACLPolicyListStub, *QueryMeta, error) {
var resp []*ACLPolicyListStub
qm, err := a.client.query("/v1/acl/policies", &resp, q)
if err != nil {
return nil, nil, err
}
return resp, qm, nil
}
// Upsert is used to create or update a policy
func (a *ACLPolicies) Upsert(policy *ACLPolicy, q *WriteOptions) (*WriteMeta, error) {
if policy == nil || policy.Name == "" {
return nil, fmt.Errorf("missing policy name")
}
wm, err := a.client.write("/v1/acl/policy/"+policy.Name, policy, nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Delete is used to delete a policy
func (a *ACLPolicies) Delete(policyName string, q *WriteOptions) (*WriteMeta, error) {
if policyName == "" {
return nil, fmt.Errorf("missing policy name")
}
wm, err := a.client.delete("/v1/acl/policy/"+policyName, nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Info is used to query a specific policy
func (a *ACLPolicies) Info(policyName string, q *QueryOptions) (*ACLPolicy, *QueryMeta, error) {
if policyName == "" {
return nil, nil, fmt.Errorf("missing policy name")
}
var resp ACLPolicy
wm, err := a.client.query("/v1/acl/policy/"+policyName, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// ACLTokens is used to query the ACL token endpoints.
type ACLTokens struct {
client *Client
}
// ACLTokens returns a new handle on the ACL tokens.
func (c *Client) ACLTokens() *ACLTokens {
return &ACLTokens{client: c}
}
// Bootstrap is used to get the initial bootstrap token
func (a *ACLTokens) Bootstrap(q *WriteOptions) (*ACLToken, *WriteMeta, error) {
var resp ACLToken
wm, err := a.client.write("/v1/acl/bootstrap", nil, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// List is used to dump all of the tokens.
func (a *ACLTokens) List(q *QueryOptions) ([]*ACLTokenListStub, *QueryMeta, error) {
var resp []*ACLTokenListStub
qm, err := a.client.query("/v1/acl/tokens", &resp, q)
if err != nil {
return nil, nil, err
}
return resp, qm, nil
}
// Create is used to create a token
func (a *ACLTokens) Create(token *ACLToken, q *WriteOptions) (*ACLToken, *WriteMeta, error) {
if token.AccessorID != "" {
return nil, nil, fmt.Errorf("cannot specify Accessor ID")
}
var resp ACLToken
wm, err := a.client.write("/v1/acl/token", token, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// Update is used to update an existing token
func (a *ACLTokens) Update(token *ACLToken, q *WriteOptions) (*ACLToken, *WriteMeta, error) {
if token.AccessorID == "" {
return nil, nil, fmt.Errorf("missing accessor ID")
}
var resp ACLToken
wm, err := a.client.write("/v1/acl/token/"+token.AccessorID,
token, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// Delete is used to delete a token
func (a *ACLTokens) Delete(accessorID string, q *WriteOptions) (*WriteMeta, error) {
if accessorID == "" {
return nil, fmt.Errorf("missing accessor ID")
}
wm, err := a.client.delete("/v1/acl/token/"+accessorID, nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Info is used to query a token
func (a *ACLTokens) Info(accessorID string, q *QueryOptions) (*ACLToken, *QueryMeta, error) {
if accessorID == "" {
return nil, nil, fmt.Errorf("missing accessor ID")
}
var resp ACLToken
wm, err := a.client.query("/v1/acl/token/"+accessorID, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// Self is used to query our own token
func (a *ACLTokens) Self(q *QueryOptions) (*ACLToken, *QueryMeta, error) {
var resp ACLToken
wm, err := a.client.query("/v1/acl/token/self", &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// ACLPolicyListStub is used to for listing ACL policies
type ACLPolicyListStub struct {
Name string
Description string
CreateIndex uint64
ModifyIndex uint64
}
// ACLPolicy is used to represent an ACL policy
type ACLPolicy struct {
Name string
Description string
Rules string
CreateIndex uint64
ModifyIndex uint64
}
// ACLToken represents a client token which is used to Authenticate
type ACLToken struct {
AccessorID string
SecretID string
Name string
Type string
Policies []string
Global bool
CreateTime time.Time
CreateIndex uint64
ModifyIndex uint64
}
type ACLTokenListStub struct {
AccessorID string
Name string
Type string
Policies []string
Global bool
CreateTime time.Time
CreateIndex uint64
ModifyIndex uint64
}

308
vendor/github.com/hashicorp/nomad/api/agent.go generated vendored Normal file
View File

@@ -0,0 +1,308 @@
package api
import (
"encoding/json"
"fmt"
"net/url"
)
// Agent encapsulates an API client which talks to Nomad's
// agent endpoints for a specific node.
type Agent struct {
client *Client
// Cache static agent info
nodeName string
datacenter string
region string
}
// KeyringResponse is a unified key response and can be used for install,
// remove, use, as well as listing key queries.
type KeyringResponse struct {
Messages map[string]string
Keys map[string]int
NumNodes int
}
// KeyringRequest is request objects for serf key operations.
type KeyringRequest struct {
Key string
}
// Agent returns a new agent which can be used to query
// the agent-specific endpoints.
func (c *Client) Agent() *Agent {
return &Agent{client: c}
}
// Self is used to query the /v1/agent/self endpoint and
// returns information specific to the running agent.
func (a *Agent) Self() (*AgentSelf, error) {
var out *AgentSelf
// Query the self endpoint on the agent
_, err := a.client.query("/v1/agent/self", &out, nil)
if err != nil {
return nil, fmt.Errorf("failed querying self endpoint: %s", err)
}
// Populate the cache for faster queries
a.populateCache(out)
return out, nil
}
// populateCache is used to insert various pieces of static
// data into the agent handle. This is used during subsequent
// lookups for the same data later on to save the round trip.
func (a *Agent) populateCache(self *AgentSelf) {
if a.nodeName == "" {
a.nodeName = self.Member.Name
}
if a.datacenter == "" {
if val, ok := self.Config["Datacenter"]; ok {
a.datacenter, _ = val.(string)
}
}
if a.region == "" {
if val, ok := self.Config["Region"]; ok {
a.region, _ = val.(string)
}
}
}
// NodeName is used to query the Nomad agent for its node name.
func (a *Agent) NodeName() (string, error) {
// Return from cache if we have it
if a.nodeName != "" {
return a.nodeName, nil
}
// Query the node name
_, err := a.Self()
return a.nodeName, err
}
// Datacenter is used to return the name of the datacenter which
// the agent is a member of.
func (a *Agent) Datacenter() (string, error) {
// Return from cache if we have it
if a.datacenter != "" {
return a.datacenter, nil
}
// Query the agent for the DC
_, err := a.Self()
return a.datacenter, err
}
// Region is used to look up the region the agent is in.
func (a *Agent) Region() (string, error) {
// Return from cache if we have it
if a.region != "" {
return a.region, nil
}
// Query the agent for the region
_, err := a.Self()
return a.region, err
}
// Join is used to instruct a server node to join another server
// via the gossip protocol. Multiple addresses may be specified.
// We attempt to join all of the hosts in the list. Returns the
// number of nodes successfully joined and any error. If one or
// more nodes have a successful result, no error is returned.
func (a *Agent) Join(addrs ...string) (int, error) {
// Accumulate the addresses
v := url.Values{}
for _, addr := range addrs {
v.Add("address", addr)
}
// Send the join request
var resp joinResponse
_, err := a.client.write("/v1/agent/join?"+v.Encode(), nil, &resp, nil)
if err != nil {
return 0, fmt.Errorf("failed joining: %s", err)
}
if resp.Error != "" {
return 0, fmt.Errorf("failed joining: %s", resp.Error)
}
return resp.NumJoined, nil
}
// Members is used to query all of the known server members
func (a *Agent) Members() (*ServerMembers, error) {
var resp *ServerMembers
// Query the known members
_, err := a.client.query("/v1/agent/members", &resp, nil)
if err != nil {
return nil, err
}
return resp, nil
}
// ForceLeave is used to eject an existing node from the cluster.
func (a *Agent) ForceLeave(node string) error {
_, err := a.client.write("/v1/agent/force-leave?node="+node, nil, nil, nil)
return err
}
// Servers is used to query the list of servers on a client node.
func (a *Agent) Servers() ([]string, error) {
var resp []string
_, err := a.client.query("/v1/agent/servers", &resp, nil)
if err != nil {
return nil, err
}
return resp, nil
}
// SetServers is used to update the list of servers on a client node.
func (a *Agent) SetServers(addrs []string) error {
// Accumulate the addresses
v := url.Values{}
for _, addr := range addrs {
v.Add("address", addr)
}
_, err := a.client.write("/v1/agent/servers?"+v.Encode(), nil, nil, nil)
return err
}
// ListKeys returns the list of installed keys
func (a *Agent) ListKeys() (*KeyringResponse, error) {
var resp KeyringResponse
_, err := a.client.query("/v1/agent/keyring/list", &resp, nil)
if err != nil {
return nil, err
}
return &resp, nil
}
// InstallKey installs a key in the keyrings of all the serf members
func (a *Agent) InstallKey(key string) (*KeyringResponse, error) {
args := KeyringRequest{
Key: key,
}
var resp KeyringResponse
_, err := a.client.write("/v1/agent/keyring/install", &args, &resp, nil)
return &resp, err
}
// UseKey uses a key from the keyring of serf members
func (a *Agent) UseKey(key string) (*KeyringResponse, error) {
args := KeyringRequest{
Key: key,
}
var resp KeyringResponse
_, err := a.client.write("/v1/agent/keyring/use", &args, &resp, nil)
return &resp, err
}
// RemoveKey removes a particular key from keyrings of serf members
func (a *Agent) RemoveKey(key string) (*KeyringResponse, error) {
args := KeyringRequest{
Key: key,
}
var resp KeyringResponse
_, err := a.client.write("/v1/agent/keyring/remove", &args, &resp, nil)
return &resp, err
}
// Health queries the agent's health
func (a *Agent) Health() (*AgentHealthResponse, error) {
req, err := a.client.newRequest("GET", "/v1/agent/health")
if err != nil {
return nil, err
}
var health AgentHealthResponse
_, resp, err := a.client.doRequest(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
// Always try to decode the response as JSON
err = json.NewDecoder(resp.Body).Decode(&health)
if err == nil {
return &health, nil
}
// Return custom error when response is not expected JSON format
return nil, fmt.Errorf("unable to unmarshal response with status %d: %v", resp.StatusCode, err)
}
// joinResponse is used to decode the response we get while
// sending a member join request.
type joinResponse struct {
NumJoined int `json:"num_joined"`
Error string `json:"error"`
}
type ServerMembers struct {
ServerName string
ServerRegion string
ServerDC string
Members []*AgentMember
}
type AgentSelf struct {
Config map[string]interface{} `json:"config"`
Member AgentMember `json:"member"`
Stats map[string]map[string]string `json:"stats"`
}
// AgentMember represents a cluster member known to the agent
type AgentMember struct {
Name string
Addr string
Port uint16
Tags map[string]string
Status string
ProtocolMin uint8
ProtocolMax uint8
ProtocolCur uint8
DelegateMin uint8
DelegateMax uint8
DelegateCur uint8
}
// AgentMembersNameSort implements sort.Interface for []*AgentMembersNameSort
// based on the Name, DC and Region
type AgentMembersNameSort []*AgentMember
func (a AgentMembersNameSort) Len() int { return len(a) }
func (a AgentMembersNameSort) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a AgentMembersNameSort) Less(i, j int) bool {
if a[i].Tags["region"] != a[j].Tags["region"] {
return a[i].Tags["region"] < a[j].Tags["region"]
}
if a[i].Tags["dc"] != a[j].Tags["dc"] {
return a[i].Tags["dc"] < a[j].Tags["dc"]
}
return a[i].Name < a[j].Name
}
// AgentHealthResponse is the response from the Health endpoint describing an
// agent's health.
type AgentHealthResponse struct {
Client *AgentHealth `json:"client,omitempty"`
Server *AgentHealth `json:"server,omitempty"`
}
// AgentHealth describes the Client or Server's health in a Health request.
type AgentHealth struct {
// Ok is false if the agent is unhealthy
Ok bool `json:"ok"`
// Message describes why the agent is unhealthy
Message string `json:"message"`
}

228
vendor/github.com/hashicorp/nomad/api/allocations.go generated vendored Normal file
View File

@@ -0,0 +1,228 @@
package api
import (
"fmt"
"sort"
"time"
)
var (
// NodeDownErr marks an operation as not able to complete since the node is
// down.
NodeDownErr = fmt.Errorf("node down")
)
// Allocations is used to query the alloc-related endpoints.
type Allocations struct {
client *Client
}
// Allocations returns a handle on the allocs endpoints.
func (c *Client) Allocations() *Allocations {
return &Allocations{client: c}
}
// List returns a list of all of the allocations.
func (a *Allocations) List(q *QueryOptions) ([]*AllocationListStub, *QueryMeta, error) {
var resp []*AllocationListStub
qm, err := a.client.query("/v1/allocations", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(AllocIndexSort(resp))
return resp, qm, nil
}
func (a *Allocations) PrefixList(prefix string) ([]*AllocationListStub, *QueryMeta, error) {
return a.List(&QueryOptions{Prefix: prefix})
}
// Info is used to retrieve a single allocation.
func (a *Allocations) Info(allocID string, q *QueryOptions) (*Allocation, *QueryMeta, error) {
var resp Allocation
qm, err := a.client.query("/v1/allocation/"+allocID, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
func (a *Allocations) Stats(alloc *Allocation, q *QueryOptions) (*AllocResourceUsage, error) {
var resp AllocResourceUsage
path := fmt.Sprintf("/v1/client/allocation/%s/stats", alloc.ID)
_, err := a.client.query(path, &resp, q)
return &resp, err
}
func (a *Allocations) GC(alloc *Allocation, q *QueryOptions) error {
nodeClient, err := a.client.GetNodeClient(alloc.NodeID, q)
if err != nil {
return err
}
var resp struct{}
_, err = nodeClient.query("/v1/client/allocation/"+alloc.ID+"/gc", &resp, nil)
return err
}
// Allocation is used for serialization of allocations.
type Allocation struct {
ID string
Namespace string
EvalID string
Name string
NodeID string
JobID string
Job *Job
TaskGroup string
Resources *Resources
TaskResources map[string]*Resources
Services map[string]string
Metrics *AllocationMetric
DesiredStatus string
DesiredDescription string
DesiredTransition DesiredTransition
ClientStatus string
ClientDescription string
TaskStates map[string]*TaskState
DeploymentID string
DeploymentStatus *AllocDeploymentStatus
FollowupEvalID string
PreviousAllocation string
NextAllocation string
RescheduleTracker *RescheduleTracker
CreateIndex uint64
ModifyIndex uint64
AllocModifyIndex uint64
CreateTime int64
ModifyTime int64
}
// AllocationMetric is used to deserialize allocation metrics.
type AllocationMetric struct {
NodesEvaluated int
NodesFiltered int
NodesAvailable map[string]int
ClassFiltered map[string]int
ConstraintFiltered map[string]int
NodesExhausted int
ClassExhausted map[string]int
DimensionExhausted map[string]int
QuotaExhausted []string
Scores map[string]float64
AllocationTime time.Duration
CoalescedFailures int
}
// AllocationListStub is used to return a subset of an allocation
// during list operations.
type AllocationListStub struct {
ID string
EvalID string
Name string
NodeID string
JobID string
JobVersion uint64
TaskGroup string
DesiredStatus string
DesiredDescription string
ClientStatus string
ClientDescription string
TaskStates map[string]*TaskState
DeploymentStatus *AllocDeploymentStatus
FollowupEvalID string
RescheduleTracker *RescheduleTracker
CreateIndex uint64
ModifyIndex uint64
CreateTime int64
ModifyTime int64
}
// AllocDeploymentStatus captures the status of the allocation as part of the
// deployment. This can include things like if the allocation has been marked as
// healthy.
type AllocDeploymentStatus struct {
Healthy *bool
Timestamp time.Time
Canary bool
ModifyIndex uint64
}
// AllocIndexSort reverse sorts allocs by CreateIndex.
type AllocIndexSort []*AllocationListStub
func (a AllocIndexSort) Len() int {
return len(a)
}
func (a AllocIndexSort) Less(i, j int) bool {
return a[i].CreateIndex > a[j].CreateIndex
}
func (a AllocIndexSort) Swap(i, j int) {
a[i], a[j] = a[j], a[i]
}
// RescheduleInfo is used to calculate remaining reschedule attempts
// according to the given time and the task groups reschedule policy
func (a Allocation) RescheduleInfo(t time.Time) (int, int) {
var reschedulePolicy *ReschedulePolicy
for _, tg := range a.Job.TaskGroups {
if *tg.Name == a.TaskGroup {
reschedulePolicy = tg.ReschedulePolicy
}
}
if reschedulePolicy == nil {
return 0, 0
}
availableAttempts := *reschedulePolicy.Attempts
interval := *reschedulePolicy.Interval
attempted := 0
// Loop over reschedule tracker to find attempts within the restart policy's interval
if a.RescheduleTracker != nil && availableAttempts > 0 && interval > 0 {
for j := len(a.RescheduleTracker.Events) - 1; j >= 0; j-- {
lastAttempt := a.RescheduleTracker.Events[j].RescheduleTime
timeDiff := t.UTC().UnixNano() - lastAttempt
if timeDiff < interval.Nanoseconds() {
attempted += 1
}
}
}
return attempted, availableAttempts
}
// RescheduleTracker encapsulates previous reschedule events
type RescheduleTracker struct {
Events []*RescheduleEvent
}
// RescheduleEvent is used to keep track of previous attempts at rescheduling an allocation
type RescheduleEvent struct {
// RescheduleTime is the timestamp of a reschedule attempt
RescheduleTime int64
// PrevAllocID is the ID of the previous allocation being restarted
PrevAllocID string
// PrevNodeID is the node ID of the previous allocation
PrevNodeID string
}
// DesiredTransition is used to mark an allocation as having a desired state
// transition. This information can be used by the scheduler to make the
// correct decision.
type DesiredTransition struct {
// Migrate is used to indicate that this allocation should be stopped and
// migrated to another node.
Migrate *bool
// Reschedule is used to indicate that this allocation is eligible to be
// rescheduled.
Reschedule *bool
}
// ShouldMigrate returns whether the transition object dictates a migration.
func (d DesiredTransition) ShouldMigrate() bool {
return d.Migrate != nil && *d.Migrate
}

832
vendor/github.com/hashicorp/nomad/api/api.go generated vendored Normal file
View File

@@ -0,0 +1,832 @@
package api
import (
"bytes"
"compress/gzip"
"crypto/tls"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
"github.com/hashicorp/go-cleanhttp"
rootcerts "github.com/hashicorp/go-rootcerts"
)
var (
// ClientConnTimeout is the timeout applied when attempting to contact a
// client directly before switching to a connection through the Nomad
// server.
ClientConnTimeout = 1 * time.Second
)
// QueryOptions are used to parameterize a query
type QueryOptions struct {
// Providing a datacenter overwrites the region provided
// by the Config
Region string
// Namespace is the target namespace for the query.
Namespace string
// AllowStale allows any Nomad server (non-leader) to service
// a read. This allows for lower latency and higher throughput
AllowStale bool
// WaitIndex is used to enable a blocking query. Waits
// until the timeout or the next index is reached
WaitIndex uint64
// WaitTime is used to bound the duration of a wait.
// Defaults to that of the Config, but can be overridden.
WaitTime time.Duration
// If set, used as prefix for resource list searches
Prefix string
// Set HTTP parameters on the query.
Params map[string]string
// AuthToken is the secret ID of an ACL token
AuthToken string
}
// WriteOptions are used to parameterize a write
type WriteOptions struct {
// Providing a datacenter overwrites the region provided
// by the Config
Region string
// Namespace is the target namespace for the write.
Namespace string
// AuthToken is the secret ID of an ACL token
AuthToken string
}
// QueryMeta is used to return meta data about a query
type QueryMeta struct {
// LastIndex. This can be used as a WaitIndex to perform
// a blocking query
LastIndex uint64
// Time of last contact from the leader for the
// server servicing the request
LastContact time.Duration
// Is there a known leader
KnownLeader bool
// How long did the request take
RequestTime time.Duration
}
// WriteMeta is used to return meta data about a write
type WriteMeta struct {
// LastIndex. This can be used as a WaitIndex to perform
// a blocking query
LastIndex uint64
// How long did the request take
RequestTime time.Duration
}
// HttpBasicAuth is used to authenticate http client with HTTP Basic Authentication
type HttpBasicAuth struct {
// Username to use for HTTP Basic Authentication
Username string
// Password to use for HTTP Basic Authentication
Password string
}
// Config is used to configure the creation of a client
type Config struct {
// Address is the address of the Nomad agent
Address string
// Region to use. If not provided, the default agent region is used.
Region string
// SecretID to use. This can be overwritten per request.
SecretID string
// Namespace to use. If not provided the default namespace is used.
Namespace string
// httpClient is the client to use. Default will be used if not provided.
httpClient *http.Client
// HttpAuth is the auth info to use for http access.
HttpAuth *HttpBasicAuth
// WaitTime limits how long a Watch will block. If not provided,
// the agent default values will be used.
WaitTime time.Duration
// TLSConfig provides the various TLS related configurations for the http
// client
TLSConfig *TLSConfig
}
// ClientConfig copies the configuration with a new client address, region, and
// whether the client has TLS enabled.
func (c *Config) ClientConfig(region, address string, tlsEnabled bool) *Config {
scheme := "http"
if tlsEnabled {
scheme = "https"
}
defaultConfig := DefaultConfig()
config := &Config{
Address: fmt.Sprintf("%s://%s", scheme, address),
Region: region,
Namespace: c.Namespace,
httpClient: defaultConfig.httpClient,
SecretID: c.SecretID,
HttpAuth: c.HttpAuth,
WaitTime: c.WaitTime,
TLSConfig: c.TLSConfig.Copy(),
}
// Update the tls server name for connecting to a client
if tlsEnabled && config.TLSConfig != nil {
config.TLSConfig.TLSServerName = fmt.Sprintf("client.%s.nomad", region)
}
return config
}
// TLSConfig contains the parameters needed to configure TLS on the HTTP client
// used to communicate with Nomad.
type TLSConfig struct {
// CACert is the path to a PEM-encoded CA cert file to use to verify the
// Nomad server SSL certificate.
CACert string
// CAPath is the path to a directory of PEM-encoded CA cert files to verify
// the Nomad server SSL certificate.
CAPath string
// ClientCert is the path to the certificate for Nomad communication
ClientCert string
// ClientKey is the path to the private key for Nomad communication
ClientKey string
// TLSServerName, if set, is used to set the SNI host when connecting via
// TLS.
TLSServerName string
// Insecure enables or disables SSL verification
Insecure bool
}
func (t *TLSConfig) Copy() *TLSConfig {
if t == nil {
return nil
}
nt := new(TLSConfig)
*nt = *t
return nt
}
// DefaultConfig returns a default configuration for the client
func DefaultConfig() *Config {
config := &Config{
Address: "http://127.0.0.1:4646",
httpClient: cleanhttp.DefaultClient(),
TLSConfig: &TLSConfig{},
}
transport := config.httpClient.Transport.(*http.Transport)
transport.TLSHandshakeTimeout = 10 * time.Second
transport.TLSClientConfig = &tls.Config{
MinVersion: tls.VersionTLS12,
}
if addr := os.Getenv("NOMAD_ADDR"); addr != "" {
config.Address = addr
}
if v := os.Getenv("NOMAD_REGION"); v != "" {
config.Region = v
}
if v := os.Getenv("NOMAD_NAMESPACE"); v != "" {
config.Namespace = v
}
if auth := os.Getenv("NOMAD_HTTP_AUTH"); auth != "" {
var username, password string
if strings.Contains(auth, ":") {
split := strings.SplitN(auth, ":", 2)
username = split[0]
password = split[1]
} else {
username = auth
}
config.HttpAuth = &HttpBasicAuth{
Username: username,
Password: password,
}
}
// Read TLS specific env vars
if v := os.Getenv("NOMAD_CACERT"); v != "" {
config.TLSConfig.CACert = v
}
if v := os.Getenv("NOMAD_CAPATH"); v != "" {
config.TLSConfig.CAPath = v
}
if v := os.Getenv("NOMAD_CLIENT_CERT"); v != "" {
config.TLSConfig.ClientCert = v
}
if v := os.Getenv("NOMAD_CLIENT_KEY"); v != "" {
config.TLSConfig.ClientKey = v
}
if v := os.Getenv("NOMAD_SKIP_VERIFY"); v != "" {
if insecure, err := strconv.ParseBool(v); err == nil {
config.TLSConfig.Insecure = insecure
}
}
if v := os.Getenv("NOMAD_TOKEN"); v != "" {
config.SecretID = v
}
return config
}
// SetTimeout is used to place a timeout for connecting to Nomad. A negative
// duration is ignored, a duration of zero means no timeout, and any other value
// will add a timeout.
func (c *Config) SetTimeout(t time.Duration) error {
if c == nil {
return fmt.Errorf("nil config")
} else if c.httpClient == nil {
return fmt.Errorf("nil HTTP client")
} else if c.httpClient.Transport == nil {
return fmt.Errorf("nil HTTP client transport")
}
// Apply a timeout.
if t.Nanoseconds() >= 0 {
transport, ok := c.httpClient.Transport.(*http.Transport)
if !ok {
return fmt.Errorf("unexpected HTTP transport: %T", c.httpClient.Transport)
}
transport.DialContext = (&net.Dialer{
Timeout: t,
KeepAlive: 30 * time.Second,
}).DialContext
}
return nil
}
// ConfigureTLS applies a set of TLS configurations to the the HTTP client.
func (c *Config) ConfigureTLS() error {
if c.TLSConfig == nil {
return nil
}
if c.httpClient == nil {
return fmt.Errorf("config HTTP Client must be set")
}
var clientCert tls.Certificate
foundClientCert := false
if c.TLSConfig.ClientCert != "" || c.TLSConfig.ClientKey != "" {
if c.TLSConfig.ClientCert != "" && c.TLSConfig.ClientKey != "" {
var err error
clientCert, err = tls.LoadX509KeyPair(c.TLSConfig.ClientCert, c.TLSConfig.ClientKey)
if err != nil {
return err
}
foundClientCert = true
} else {
return fmt.Errorf("Both client cert and client key must be provided")
}
}
clientTLSConfig := c.httpClient.Transport.(*http.Transport).TLSClientConfig
rootConfig := &rootcerts.Config{
CAFile: c.TLSConfig.CACert,
CAPath: c.TLSConfig.CAPath,
}
if err := rootcerts.ConfigureTLS(clientTLSConfig, rootConfig); err != nil {
return err
}
clientTLSConfig.InsecureSkipVerify = c.TLSConfig.Insecure
if foundClientCert {
clientTLSConfig.Certificates = []tls.Certificate{clientCert}
}
if c.TLSConfig.TLSServerName != "" {
clientTLSConfig.ServerName = c.TLSConfig.TLSServerName
}
return nil
}
// Client provides a client to the Nomad API
type Client struct {
config Config
}
// NewClient returns a new client
func NewClient(config *Config) (*Client, error) {
// bootstrap the config
defConfig := DefaultConfig()
if config.Address == "" {
config.Address = defConfig.Address
} else if _, err := url.Parse(config.Address); err != nil {
return nil, fmt.Errorf("invalid address '%s': %v", config.Address, err)
}
if config.httpClient == nil {
config.httpClient = defConfig.httpClient
}
// Configure the TLS configurations
if err := config.ConfigureTLS(); err != nil {
return nil, err
}
client := &Client{
config: *config,
}
return client, nil
}
// Address return the address of the Nomad agent
func (c *Client) Address() string {
return c.config.Address
}
// SetRegion sets the region to forward API requests to.
func (c *Client) SetRegion(region string) {
c.config.Region = region
}
// SetNamespace sets the namespace to forward API requests to.
func (c *Client) SetNamespace(namespace string) {
c.config.Namespace = namespace
}
// GetNodeClient returns a new Client that will dial the specified node. If the
// QueryOptions is set, its region will be used.
func (c *Client) GetNodeClient(nodeID string, q *QueryOptions) (*Client, error) {
return c.getNodeClientImpl(nodeID, -1, q, c.Nodes().Info)
}
// GetNodeClientWithTimeout returns a new Client that will dial the specified
// node using the specified timeout. If the QueryOptions is set, its region will
// be used.
func (c *Client) GetNodeClientWithTimeout(
nodeID string, timeout time.Duration, q *QueryOptions) (*Client, error) {
return c.getNodeClientImpl(nodeID, timeout, q, c.Nodes().Info)
}
// nodeLookup is the definition of a function used to lookup a node. This is
// largely used to mock the lookup in tests.
type nodeLookup func(nodeID string, q *QueryOptions) (*Node, *QueryMeta, error)
// getNodeClientImpl is the implementation of creating a API client for
// contacting a node. It takes a function to lookup the node such that it can be
// mocked during tests.
func (c *Client) getNodeClientImpl(nodeID string, timeout time.Duration, q *QueryOptions, lookup nodeLookup) (*Client, error) {
node, _, err := lookup(nodeID, q)
if err != nil {
return nil, err
}
if node.Status == "down" {
return nil, NodeDownErr
}
if node.HTTPAddr == "" {
return nil, fmt.Errorf("http addr of node %q (%s) is not advertised", node.Name, nodeID)
}
var region string
switch {
case q != nil && q.Region != "":
// Prefer the region set in the query parameter
region = q.Region
case c.config.Region != "":
// If the client is configured for a particular region use that
region = c.config.Region
default:
// No region information is given so use the default.
region = "global"
}
// Get an API client for the node
conf := c.config.ClientConfig(region, node.HTTPAddr, node.TLSEnabled)
// Set the timeout
conf.SetTimeout(timeout)
return NewClient(conf)
}
// SetSecretID sets the ACL token secret for API requests.
func (c *Client) SetSecretID(secretID string) {
c.config.SecretID = secretID
}
// request is used to help build up a request
type request struct {
config *Config
method string
url *url.URL
params url.Values
token string
body io.Reader
obj interface{}
}
// setQueryOptions is used to annotate the request with
// additional query options
func (r *request) setQueryOptions(q *QueryOptions) {
if q == nil {
return
}
if q.Region != "" {
r.params.Set("region", q.Region)
}
if q.Namespace != "" {
r.params.Set("namespace", q.Namespace)
}
if q.AuthToken != "" {
r.token = q.AuthToken
}
if q.AllowStale {
r.params.Set("stale", "")
}
if q.WaitIndex != 0 {
r.params.Set("index", strconv.FormatUint(q.WaitIndex, 10))
}
if q.WaitTime != 0 {
r.params.Set("wait", durToMsec(q.WaitTime))
}
if q.Prefix != "" {
r.params.Set("prefix", q.Prefix)
}
for k, v := range q.Params {
r.params.Set(k, v)
}
}
// durToMsec converts a duration to a millisecond specified string
func durToMsec(dur time.Duration) string {
return fmt.Sprintf("%dms", dur/time.Millisecond)
}
// setWriteOptions is used to annotate the request with
// additional write options
func (r *request) setWriteOptions(q *WriteOptions) {
if q == nil {
return
}
if q.Region != "" {
r.params.Set("region", q.Region)
}
if q.Namespace != "" {
r.params.Set("namespace", q.Namespace)
}
if q.AuthToken != "" {
r.token = q.AuthToken
}
}
// toHTTP converts the request to an HTTP request
func (r *request) toHTTP() (*http.Request, error) {
// Encode the query parameters
r.url.RawQuery = r.params.Encode()
// Check if we should encode the body
if r.body == nil && r.obj != nil {
if b, err := encodeBody(r.obj); err != nil {
return nil, err
} else {
r.body = b
}
}
// Create the HTTP request
req, err := http.NewRequest(r.method, r.url.RequestURI(), r.body)
if err != nil {
return nil, err
}
// Optionally configure HTTP basic authentication
if r.url.User != nil {
username := r.url.User.Username()
password, _ := r.url.User.Password()
req.SetBasicAuth(username, password)
} else if r.config.HttpAuth != nil {
req.SetBasicAuth(r.config.HttpAuth.Username, r.config.HttpAuth.Password)
}
req.Header.Add("Accept-Encoding", "gzip")
if r.token != "" {
req.Header.Set("X-Nomad-Token", r.token)
}
req.URL.Host = r.url.Host
req.URL.Scheme = r.url.Scheme
req.Host = r.url.Host
return req, nil
}
// newRequest is used to create a new request
func (c *Client) newRequest(method, path string) (*request, error) {
base, _ := url.Parse(c.config.Address)
u, err := url.Parse(path)
if err != nil {
return nil, err
}
r := &request{
config: &c.config,
method: method,
url: &url.URL{
Scheme: base.Scheme,
User: base.User,
Host: base.Host,
Path: u.Path,
},
params: make(map[string][]string),
}
if c.config.Region != "" {
r.params.Set("region", c.config.Region)
}
if c.config.Namespace != "" {
r.params.Set("namespace", c.config.Namespace)
}
if c.config.WaitTime != 0 {
r.params.Set("wait", durToMsec(r.config.WaitTime))
}
if c.config.SecretID != "" {
r.token = r.config.SecretID
}
// Add in the query parameters, if any
for key, values := range u.Query() {
for _, value := range values {
r.params.Add(key, value)
}
}
return r, nil
}
// multiCloser is to wrap a ReadCloser such that when close is called, multiple
// Closes occur.
type multiCloser struct {
reader io.Reader
inorderClose []io.Closer
}
func (m *multiCloser) Close() error {
for _, c := range m.inorderClose {
if err := c.Close(); err != nil {
return err
}
}
return nil
}
func (m *multiCloser) Read(p []byte) (int, error) {
return m.reader.Read(p)
}
// doRequest runs a request with our client
func (c *Client) doRequest(r *request) (time.Duration, *http.Response, error) {
req, err := r.toHTTP()
if err != nil {
return 0, nil, err
}
start := time.Now()
resp, err := c.config.httpClient.Do(req)
diff := time.Now().Sub(start)
// If the response is compressed, we swap the body's reader.
if resp != nil && resp.Header != nil {
var reader io.ReadCloser
switch resp.Header.Get("Content-Encoding") {
case "gzip":
greader, err := gzip.NewReader(resp.Body)
if err != nil {
return 0, nil, err
}
// The gzip reader doesn't close the wrapped reader so we use
// multiCloser.
reader = &multiCloser{
reader: greader,
inorderClose: []io.Closer{greader, resp.Body},
}
default:
reader = resp.Body
}
resp.Body = reader
}
return diff, resp, err
}
// rawQuery makes a GET request to the specified endpoint but returns just the
// response body.
func (c *Client) rawQuery(endpoint string, q *QueryOptions) (io.ReadCloser, error) {
r, err := c.newRequest("GET", endpoint)
if err != nil {
return nil, err
}
r.setQueryOptions(q)
_, resp, err := requireOK(c.doRequest(r))
if err != nil {
return nil, err
}
return resp.Body, nil
}
// query is used to do a GET request against an endpoint
// and deserialize the response into an interface using
// standard Nomad conventions.
func (c *Client) query(endpoint string, out interface{}, q *QueryOptions) (*QueryMeta, error) {
r, err := c.newRequest("GET", endpoint)
if err != nil {
return nil, err
}
r.setQueryOptions(q)
rtt, resp, err := requireOK(c.doRequest(r))
if err != nil {
return nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
if err := decodeBody(resp, out); err != nil {
return nil, err
}
return qm, nil
}
// putQuery is used to do a PUT request when doing a read against an endpoint
// and deserialize the response into an interface using standard Nomad
// conventions.
func (c *Client) putQuery(endpoint string, in, out interface{}, q *QueryOptions) (*QueryMeta, error) {
r, err := c.newRequest("PUT", endpoint)
if err != nil {
return nil, err
}
r.setQueryOptions(q)
r.obj = in
rtt, resp, err := requireOK(c.doRequest(r))
if err != nil {
return nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
if err := decodeBody(resp, out); err != nil {
return nil, err
}
return qm, nil
}
// write is used to do a PUT request against an endpoint
// and serialize/deserialized using the standard Nomad conventions.
func (c *Client) write(endpoint string, in, out interface{}, q *WriteOptions) (*WriteMeta, error) {
r, err := c.newRequest("PUT", endpoint)
if err != nil {
return nil, err
}
r.setWriteOptions(q)
r.obj = in
rtt, resp, err := requireOK(c.doRequest(r))
if err != nil {
return nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
parseWriteMeta(resp, wm)
if out != nil {
if err := decodeBody(resp, &out); err != nil {
return nil, err
}
}
return wm, nil
}
// delete is used to do a DELETE request against an endpoint
// and serialize/deserialized using the standard Nomad conventions.
func (c *Client) delete(endpoint string, out interface{}, q *WriteOptions) (*WriteMeta, error) {
r, err := c.newRequest("DELETE", endpoint)
if err != nil {
return nil, err
}
r.setWriteOptions(q)
rtt, resp, err := requireOK(c.doRequest(r))
if err != nil {
return nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
parseWriteMeta(resp, wm)
if out != nil {
if err := decodeBody(resp, &out); err != nil {
return nil, err
}
}
return wm, nil
}
// parseQueryMeta is used to help parse query meta-data
func parseQueryMeta(resp *http.Response, q *QueryMeta) error {
header := resp.Header
// Parse the X-Nomad-Index
index, err := strconv.ParseUint(header.Get("X-Nomad-Index"), 10, 64)
if err != nil {
return fmt.Errorf("Failed to parse X-Nomad-Index: %v", err)
}
q.LastIndex = index
// Parse the X-Nomad-LastContact
last, err := strconv.ParseUint(header.Get("X-Nomad-LastContact"), 10, 64)
if err != nil {
return fmt.Errorf("Failed to parse X-Nomad-LastContact: %v", err)
}
q.LastContact = time.Duration(last) * time.Millisecond
// Parse the X-Nomad-KnownLeader
switch header.Get("X-Nomad-KnownLeader") {
case "true":
q.KnownLeader = true
default:
q.KnownLeader = false
}
return nil
}
// parseWriteMeta is used to help parse write meta-data
func parseWriteMeta(resp *http.Response, q *WriteMeta) error {
header := resp.Header
// Parse the X-Nomad-Index
index, err := strconv.ParseUint(header.Get("X-Nomad-Index"), 10, 64)
if err != nil {
return fmt.Errorf("Failed to parse X-Nomad-Index: %v", err)
}
q.LastIndex = index
return nil
}
// decodeBody is used to JSON decode a body
func decodeBody(resp *http.Response, out interface{}) error {
dec := json.NewDecoder(resp.Body)
return dec.Decode(out)
}
// encodeBody is used to encode a request body
func encodeBody(obj interface{}) (io.Reader, error) {
buf := bytes.NewBuffer(nil)
enc := json.NewEncoder(buf)
if err := enc.Encode(obj); err != nil {
return nil, err
}
return buf, nil
}
// requireOK is used to wrap doRequest and check for a 200
func requireOK(d time.Duration, resp *http.Response, e error) (time.Duration, *http.Response, error) {
if e != nil {
if resp != nil {
resp.Body.Close()
}
return d, nil, e
}
if resp.StatusCode != 200 {
var buf bytes.Buffer
io.Copy(&buf, resp.Body)
resp.Body.Close()
return d, nil, fmt.Errorf("Unexpected response code: %d (%s)", resp.StatusCode, buf.Bytes())
}
return d, resp, nil
}

17
vendor/github.com/hashicorp/nomad/api/constraint.go generated vendored Normal file
View File

@@ -0,0 +1,17 @@
package api
// Constraint is used to serialize a job placement constraint.
type Constraint struct {
LTarget string
RTarget string
Operand string
}
// NewConstraint generates a new job placement constraint.
func NewConstraint(left, operand, right string) *Constraint {
return &Constraint{
LTarget: left,
RTarget: right,
Operand: operand,
}
}

View File

@@ -0,0 +1,15 @@
package contexts
// Context defines the scope in which a search for Nomad object operates
type Context string
const (
Allocs Context = "allocs"
Deployments Context = "deployment"
Evals Context = "evals"
Jobs Context = "jobs"
Nodes Context = "nodes"
Namespaces Context = "namespaces"
Quotas Context = "quotas"
All Context = "all"
)

264
vendor/github.com/hashicorp/nomad/api/deployments.go generated vendored Normal file
View File

@@ -0,0 +1,264 @@
package api
import (
"sort"
"time"
)
// Deployments is used to query the deployments endpoints.
type Deployments struct {
client *Client
}
// Deployments returns a new handle on the deployments.
func (c *Client) Deployments() *Deployments {
return &Deployments{client: c}
}
// List is used to dump all of the deployments.
func (d *Deployments) List(q *QueryOptions) ([]*Deployment, *QueryMeta, error) {
var resp []*Deployment
qm, err := d.client.query("/v1/deployments", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(DeploymentIndexSort(resp))
return resp, qm, nil
}
func (d *Deployments) PrefixList(prefix string) ([]*Deployment, *QueryMeta, error) {
return d.List(&QueryOptions{Prefix: prefix})
}
// Info is used to query a single deployment by its ID.
func (d *Deployments) Info(deploymentID string, q *QueryOptions) (*Deployment, *QueryMeta, error) {
var resp Deployment
qm, err := d.client.query("/v1/deployment/"+deploymentID, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// Allocations is used to retrieve a set of allocations that are part of the
// deployment
func (d *Deployments) Allocations(deploymentID string, q *QueryOptions) ([]*AllocationListStub, *QueryMeta, error) {
var resp []*AllocationListStub
qm, err := d.client.query("/v1/deployment/allocations/"+deploymentID, &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(AllocIndexSort(resp))
return resp, qm, nil
}
// Fail is used to fail the given deployment.
func (d *Deployments) Fail(deploymentID string, q *WriteOptions) (*DeploymentUpdateResponse, *WriteMeta, error) {
var resp DeploymentUpdateResponse
req := &DeploymentFailRequest{
DeploymentID: deploymentID,
}
wm, err := d.client.write("/v1/deployment/fail/"+deploymentID, req, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// Pause is used to pause or unpause the given deployment.
func (d *Deployments) Pause(deploymentID string, pause bool, q *WriteOptions) (*DeploymentUpdateResponse, *WriteMeta, error) {
var resp DeploymentUpdateResponse
req := &DeploymentPauseRequest{
DeploymentID: deploymentID,
Pause: pause,
}
wm, err := d.client.write("/v1/deployment/pause/"+deploymentID, req, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// PromoteAll is used to promote all canaries in the given deployment
func (d *Deployments) PromoteAll(deploymentID string, q *WriteOptions) (*DeploymentUpdateResponse, *WriteMeta, error) {
var resp DeploymentUpdateResponse
req := &DeploymentPromoteRequest{
DeploymentID: deploymentID,
All: true,
}
wm, err := d.client.write("/v1/deployment/promote/"+deploymentID, req, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// PromoteGroups is used to promote canaries in the passed groups in the given deployment
func (d *Deployments) PromoteGroups(deploymentID string, groups []string, q *WriteOptions) (*DeploymentUpdateResponse, *WriteMeta, error) {
var resp DeploymentUpdateResponse
req := &DeploymentPromoteRequest{
DeploymentID: deploymentID,
Groups: groups,
}
wm, err := d.client.write("/v1/deployment/promote/"+deploymentID, req, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// SetAllocHealth is used to set allocation health for allocs that are part of
// the given deployment
func (d *Deployments) SetAllocHealth(deploymentID string, healthy, unhealthy []string, q *WriteOptions) (*DeploymentUpdateResponse, *WriteMeta, error) {
var resp DeploymentUpdateResponse
req := &DeploymentAllocHealthRequest{
DeploymentID: deploymentID,
HealthyAllocationIDs: healthy,
UnhealthyAllocationIDs: unhealthy,
}
wm, err := d.client.write("/v1/deployment/allocation-health/"+deploymentID, req, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
// Deployment is used to serialize an deployment.
type Deployment struct {
// ID is a generated UUID for the deployment
ID string
// Namespace is the namespace the deployment is created in
Namespace string
// JobID is the job the deployment is created for
JobID string
// JobVersion is the version of the job at which the deployment is tracking
JobVersion uint64
// JobModifyIndex is the ModifyIndex of the job which the deployment is
// tracking.
JobModifyIndex uint64
// JobSpecModifyIndex is the JobModifyIndex of the job which the
// deployment is tracking.
JobSpecModifyIndex uint64
// JobCreateIndex is the create index of the job which the deployment is
// tracking. It is needed so that if the job gets stopped and reran we can
// present the correct list of deployments for the job and not old ones.
JobCreateIndex uint64
// TaskGroups is the set of task groups effected by the deployment and their
// current deployment status.
TaskGroups map[string]*DeploymentState
// The status of the deployment
Status string
// StatusDescription allows a human readable description of the deployment
// status.
StatusDescription string
CreateIndex uint64
ModifyIndex uint64
}
// DeploymentState tracks the state of a deployment for a given task group.
type DeploymentState struct {
PlacedCanaries []string
AutoRevert bool
ProgressDeadline time.Duration
RequireProgressBy time.Time
Promoted bool
DesiredCanaries int
DesiredTotal int
PlacedAllocs int
HealthyAllocs int
UnhealthyAllocs int
}
// DeploymentIndexSort is a wrapper to sort deployments by CreateIndex. We
// reverse the test so that we get the highest index first.
type DeploymentIndexSort []*Deployment
func (d DeploymentIndexSort) Len() int {
return len(d)
}
func (d DeploymentIndexSort) Less(i, j int) bool {
return d[i].CreateIndex > d[j].CreateIndex
}
func (d DeploymentIndexSort) Swap(i, j int) {
d[i], d[j] = d[j], d[i]
}
// DeploymentUpdateResponse is used to respond to a deployment change. The
// response will include the modify index of the deployment as well as details
// of any triggered evaluation.
type DeploymentUpdateResponse struct {
EvalID string
EvalCreateIndex uint64
DeploymentModifyIndex uint64
RevertedJobVersion *uint64
WriteMeta
}
// DeploymentAllocHealthRequest is used to set the health of a set of
// allocations as part of a deployment.
type DeploymentAllocHealthRequest struct {
DeploymentID string
// Marks these allocations as healthy, allow further allocations
// to be rolled.
HealthyAllocationIDs []string
// Any unhealthy allocations fail the deployment
UnhealthyAllocationIDs []string
WriteRequest
}
// DeploymentPromoteRequest is used to promote task groups in a deployment
type DeploymentPromoteRequest struct {
DeploymentID string
// All is to promote all task groups
All bool
// Groups is used to set the promotion status per task group
Groups []string
WriteRequest
}
// DeploymentPauseRequest is used to pause a deployment
type DeploymentPauseRequest struct {
DeploymentID string
// Pause sets the pause status
Pause bool
WriteRequest
}
// DeploymentSpecificRequest is used to make a request specific to a particular
// deployment
type DeploymentSpecificRequest struct {
DeploymentID string
QueryOptions
}
// DeploymentFailRequest is used to fail a particular deployment
type DeploymentFailRequest struct {
DeploymentID string
WriteRequest
}
// SingleDeploymentResponse is used to respond with a single deployment
type SingleDeploymentResponse struct {
Deployment *Deployment
QueryMeta
}

99
vendor/github.com/hashicorp/nomad/api/evaluations.go generated vendored Normal file
View File

@@ -0,0 +1,99 @@
package api
import (
"sort"
"time"
)
// Evaluations is used to query the evaluation endpoints.
type Evaluations struct {
client *Client
}
// Evaluations returns a new handle on the evaluations.
func (c *Client) Evaluations() *Evaluations {
return &Evaluations{client: c}
}
// List is used to dump all of the evaluations.
func (e *Evaluations) List(q *QueryOptions) ([]*Evaluation, *QueryMeta, error) {
var resp []*Evaluation
qm, err := e.client.query("/v1/evaluations", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(EvalIndexSort(resp))
return resp, qm, nil
}
func (e *Evaluations) PrefixList(prefix string) ([]*Evaluation, *QueryMeta, error) {
return e.List(&QueryOptions{Prefix: prefix})
}
// Info is used to query a single evaluation by its ID.
func (e *Evaluations) Info(evalID string, q *QueryOptions) (*Evaluation, *QueryMeta, error) {
var resp Evaluation
qm, err := e.client.query("/v1/evaluation/"+evalID, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// Allocations is used to retrieve a set of allocations given
// an evaluation ID.
func (e *Evaluations) Allocations(evalID string, q *QueryOptions) ([]*AllocationListStub, *QueryMeta, error) {
var resp []*AllocationListStub
qm, err := e.client.query("/v1/evaluation/"+evalID+"/allocations", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(AllocIndexSort(resp))
return resp, qm, nil
}
// Evaluation is used to serialize an evaluation.
type Evaluation struct {
ID string
Priority int
Type string
TriggeredBy string
Namespace string
JobID string
JobModifyIndex uint64
NodeID string
NodeModifyIndex uint64
DeploymentID string
Status string
StatusDescription string
Wait time.Duration
WaitUntil time.Time
NextEval string
PreviousEval string
BlockedEval string
FailedTGAllocs map[string]*AllocationMetric
ClassEligibility map[string]bool
EscapedComputedClass bool
QuotaLimitReached string
AnnotatePlan bool
QueuedAllocations map[string]int
SnapshotIndex uint64
CreateIndex uint64
ModifyIndex uint64
}
// EvalIndexSort is a wrapper to sort evaluations by CreateIndex.
// We reverse the test so that we get the highest index first.
type EvalIndexSort []*Evaluation
func (e EvalIndexSort) Len() int {
return len(e)
}
func (e EvalIndexSort) Less(i, j int) bool {
return e[i].CreateIndex > e[j].CreateIndex
}
func (e EvalIndexSort) Swap(i, j int) {
e[i], e[j] = e[j], e[i]
}

439
vendor/github.com/hashicorp/nomad/api/fs.go generated vendored Normal file
View File

@@ -0,0 +1,439 @@
package api
import (
"encoding/json"
"fmt"
"io"
"net"
"strconv"
"sync"
"time"
)
const (
// OriginStart and OriginEnd are the available parameters for the origin
// argument when streaming a file. They respectively offset from the start
// and end of a file.
OriginStart = "start"
OriginEnd = "end"
)
// AllocFileInfo holds information about a file inside the AllocDir
type AllocFileInfo struct {
Name string
IsDir bool
Size int64
FileMode string
ModTime time.Time
}
// StreamFrame is used to frame data of a file when streaming
type StreamFrame struct {
Offset int64 `json:",omitempty"`
Data []byte `json:",omitempty"`
File string `json:",omitempty"`
FileEvent string `json:",omitempty"`
}
// IsHeartbeat returns if the frame is a heartbeat frame
func (s *StreamFrame) IsHeartbeat() bool {
return len(s.Data) == 0 && s.FileEvent == "" && s.File == "" && s.Offset == 0
}
// AllocFS is used to introspect an allocation directory on a Nomad client
type AllocFS struct {
client *Client
}
// AllocFS returns an handle to the AllocFS endpoints
func (c *Client) AllocFS() *AllocFS {
return &AllocFS{client: c}
}
// List is used to list the files at a given path of an allocation directory
func (a *AllocFS) List(alloc *Allocation, path string, q *QueryOptions) ([]*AllocFileInfo, *QueryMeta, error) {
if q == nil {
q = &QueryOptions{}
}
if q.Params == nil {
q.Params = make(map[string]string)
}
q.Params["path"] = path
var resp []*AllocFileInfo
qm, err := a.client.query(fmt.Sprintf("/v1/client/fs/ls/%s", alloc.ID), &resp, q)
if err != nil {
return nil, nil, err
}
return resp, qm, nil
}
// Stat is used to stat a file at a given path of an allocation directory
func (a *AllocFS) Stat(alloc *Allocation, path string, q *QueryOptions) (*AllocFileInfo, *QueryMeta, error) {
if q == nil {
q = &QueryOptions{}
}
if q.Params == nil {
q.Params = make(map[string]string)
}
q.Params["path"] = path
var resp AllocFileInfo
qm, err := a.client.query(fmt.Sprintf("/v1/client/fs/stat/%s", alloc.ID), &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// ReadAt is used to read bytes at a given offset until limit at the given path
// in an allocation directory. If limit is <= 0, there is no limit.
func (a *AllocFS) ReadAt(alloc *Allocation, path string, offset int64, limit int64, q *QueryOptions) (io.ReadCloser, error) {
nodeClient, err := a.client.GetNodeClientWithTimeout(alloc.NodeID, ClientConnTimeout, q)
if err != nil {
return nil, err
}
if q == nil {
q = &QueryOptions{}
}
if q.Params == nil {
q.Params = make(map[string]string)
}
q.Params["path"] = path
q.Params["offset"] = strconv.FormatInt(offset, 10)
q.Params["limit"] = strconv.FormatInt(limit, 10)
reqPath := fmt.Sprintf("/v1/client/fs/readat/%s", alloc.ID)
r, err := nodeClient.rawQuery(reqPath, q)
if err != nil {
// There was a networking error when talking directly to the client.
if _, ok := err.(net.Error); !ok {
return nil, err
}
// Try via the server
r, err = a.client.rawQuery(reqPath, q)
if err != nil {
return nil, err
}
}
return r, nil
}
// Cat is used to read contents of a file at the given path in an allocation
// directory
func (a *AllocFS) Cat(alloc *Allocation, path string, q *QueryOptions) (io.ReadCloser, error) {
nodeClient, err := a.client.GetNodeClientWithTimeout(alloc.NodeID, ClientConnTimeout, q)
if err != nil {
return nil, err
}
if q == nil {
q = &QueryOptions{}
}
if q.Params == nil {
q.Params = make(map[string]string)
}
q.Params["path"] = path
reqPath := fmt.Sprintf("/v1/client/fs/cat/%s", alloc.ID)
r, err := nodeClient.rawQuery(reqPath, q)
if err != nil {
// There was a networking error when talking directly to the client.
if _, ok := err.(net.Error); !ok {
return nil, err
}
// Try via the server
r, err = a.client.rawQuery(reqPath, q)
if err != nil {
return nil, err
}
}
return r, nil
}
// Stream streams the content of a file blocking on EOF.
// The parameters are:
// * path: path to file to stream.
// * offset: The offset to start streaming data at.
// * origin: Either "start" or "end" and defines from where the offset is applied.
// * cancel: A channel that when closed, streaming will end.
//
// The return value is a channel that will emit StreamFrames as they are read.
func (a *AllocFS) Stream(alloc *Allocation, path, origin string, offset int64,
cancel <-chan struct{}, q *QueryOptions) (<-chan *StreamFrame, <-chan error) {
errCh := make(chan error, 1)
nodeClient, err := a.client.GetNodeClientWithTimeout(alloc.NodeID, ClientConnTimeout, q)
if err != nil {
errCh <- err
return nil, errCh
}
if q == nil {
q = &QueryOptions{}
}
if q.Params == nil {
q.Params = make(map[string]string)
}
q.Params["path"] = path
q.Params["offset"] = strconv.FormatInt(offset, 10)
q.Params["origin"] = origin
reqPath := fmt.Sprintf("/v1/client/fs/stream/%s", alloc.ID)
r, err := nodeClient.rawQuery(reqPath, q)
if err != nil {
// There was a networking error when talking directly to the client.
if _, ok := err.(net.Error); !ok {
errCh <- err
return nil, errCh
}
// Try via the server
r, err = a.client.rawQuery(reqPath, q)
if err != nil {
errCh <- err
return nil, errCh
}
}
// Create the output channel
frames := make(chan *StreamFrame, 10)
go func() {
// Close the body
defer r.Close()
// Create a decoder
dec := json.NewDecoder(r)
for {
// Check if we have been cancelled
select {
case <-cancel:
return
default:
}
// Decode the next frame
var frame StreamFrame
if err := dec.Decode(&frame); err != nil {
errCh <- err
close(frames)
return
}
// Discard heartbeat frames
if frame.IsHeartbeat() {
continue
}
frames <- &frame
}
}()
return frames, errCh
}
// Logs streams the content of a tasks logs blocking on EOF.
// The parameters are:
// * allocation: the allocation to stream from.
// * follow: Whether the logs should be followed.
// * task: the tasks name to stream logs for.
// * logType: Either "stdout" or "stderr"
// * origin: Either "start" or "end" and defines from where the offset is applied.
// * offset: The offset to start streaming data at.
// * cancel: A channel that when closed, streaming will end.
//
// The return value is a channel that will emit StreamFrames as they are read.
// The chan will be closed when follow=false and the end of the file is
// reached.
//
// Unexpected (non-EOF) errors will be sent on the error chan.
func (a *AllocFS) Logs(alloc *Allocation, follow bool, task, logType, origin string,
offset int64, cancel <-chan struct{}, q *QueryOptions) (<-chan *StreamFrame, <-chan error) {
errCh := make(chan error, 1)
nodeClient, err := a.client.GetNodeClientWithTimeout(alloc.NodeID, ClientConnTimeout, q)
if err != nil {
errCh <- err
return nil, errCh
}
if q == nil {
q = &QueryOptions{}
}
if q.Params == nil {
q.Params = make(map[string]string)
}
q.Params["follow"] = strconv.FormatBool(follow)
q.Params["task"] = task
q.Params["type"] = logType
q.Params["origin"] = origin
q.Params["offset"] = strconv.FormatInt(offset, 10)
reqPath := fmt.Sprintf("/v1/client/fs/logs/%s", alloc.ID)
r, err := nodeClient.rawQuery(reqPath, q)
if err != nil {
// There was a networking error when talking directly to the client.
if _, ok := err.(net.Error); !ok {
errCh <- err
return nil, errCh
}
// Try via the server
r, err = a.client.rawQuery(reqPath, q)
if err != nil {
errCh <- err
return nil, errCh
}
}
// Create the output channel
frames := make(chan *StreamFrame, 10)
go func() {
// Close the body
defer r.Close()
// Create a decoder
dec := json.NewDecoder(r)
for {
// Check if we have been cancelled
select {
case <-cancel:
return
default:
}
// Decode the next frame
var frame StreamFrame
if err := dec.Decode(&frame); err != nil {
if err == io.EOF || err == io.ErrClosedPipe {
close(frames)
} else {
errCh <- err
}
return
}
// Discard heartbeat frames
if frame.IsHeartbeat() {
continue
}
frames <- &frame
}
}()
return frames, errCh
}
// FrameReader is used to convert a stream of frames into a read closer.
type FrameReader struct {
frames <-chan *StreamFrame
errCh <-chan error
cancelCh chan struct{}
closedLock sync.Mutex
closed bool
unblockTime time.Duration
frame *StreamFrame
frameOffset int
byteOffset int
}
// NewFrameReader takes a channel of frames and returns a FrameReader which
// implements io.ReadCloser
func NewFrameReader(frames <-chan *StreamFrame, errCh <-chan error, cancelCh chan struct{}) *FrameReader {
return &FrameReader{
frames: frames,
errCh: errCh,
cancelCh: cancelCh,
}
}
// SetUnblockTime sets the time to unblock and return zero bytes read. If the
// duration is unset or is zero or less, the read will block until data is read.
func (f *FrameReader) SetUnblockTime(d time.Duration) {
f.unblockTime = d
}
// Offset returns the offset into the stream.
func (f *FrameReader) Offset() int {
return f.byteOffset
}
// Read reads the data of the incoming frames into the bytes buffer. Returns EOF
// when there are no more frames.
func (f *FrameReader) Read(p []byte) (n int, err error) {
f.closedLock.Lock()
closed := f.closed
f.closedLock.Unlock()
if closed {
return 0, io.EOF
}
if f.frame == nil {
var unblock <-chan time.Time
if f.unblockTime.Nanoseconds() > 0 {
unblock = time.After(f.unblockTime)
}
select {
case frame, ok := <-f.frames:
if !ok {
return 0, io.EOF
}
f.frame = frame
// Store the total offset into the file
f.byteOffset = int(f.frame.Offset)
case <-unblock:
return 0, nil
case err := <-f.errCh:
return 0, err
case <-f.cancelCh:
return 0, io.EOF
}
}
// Copy the data out of the frame and update our offset
n = copy(p, f.frame.Data[f.frameOffset:])
f.frameOffset += n
// Clear the frame and its offset once we have read everything
if len(f.frame.Data) == f.frameOffset {
f.frame = nil
f.frameOffset = 0
}
return n, nil
}
// Close cancels the stream of frames
func (f *FrameReader) Close() error {
f.closedLock.Lock()
defer f.closedLock.Unlock()
if f.closed {
return nil
}
close(f.cancelCh)
f.closed = true
return nil
}

1063
vendor/github.com/hashicorp/nomad/api/jobs.go generated vendored Normal file

File diff suppressed because it is too large Load Diff

110
vendor/github.com/hashicorp/nomad/api/jobs_testing.go generated vendored Normal file
View File

@@ -0,0 +1,110 @@
package api
import (
"time"
"github.com/hashicorp/nomad/helper"
"github.com/hashicorp/nomad/helper/uuid"
)
func MockJob() *Job {
job := &Job{
Region: helper.StringToPtr("global"),
ID: helper.StringToPtr(uuid.Generate()),
Name: helper.StringToPtr("my-job"),
Type: helper.StringToPtr("service"),
Priority: helper.IntToPtr(50),
AllAtOnce: helper.BoolToPtr(false),
Datacenters: []string{"dc1"},
Constraints: []*Constraint{
{
LTarget: "${attr.kernel.name}",
RTarget: "linux",
Operand: "=",
},
},
TaskGroups: []*TaskGroup{
{
Name: helper.StringToPtr("web"),
Count: helper.IntToPtr(10),
EphemeralDisk: &EphemeralDisk{
SizeMB: helper.IntToPtr(150),
},
RestartPolicy: &RestartPolicy{
Attempts: helper.IntToPtr(3),
Interval: helper.TimeToPtr(10 * time.Minute),
Delay: helper.TimeToPtr(1 * time.Minute),
Mode: helper.StringToPtr("delay"),
},
Tasks: []*Task{
{
Name: "web",
Driver: "exec",
Config: map[string]interface{}{
"command": "/bin/date",
},
Env: map[string]string{
"FOO": "bar",
},
Services: []*Service{
{
Name: "${TASK}-frontend",
PortLabel: "http",
Tags: []string{"pci:${meta.pci-dss}", "datacenter:${node.datacenter}"},
Checks: []ServiceCheck{
{
Name: "check-table",
Type: "script",
Command: "/usr/local/check-table-${meta.database}",
Args: []string{"${meta.version}"},
Interval: 30 * time.Second,
Timeout: 5 * time.Second,
},
},
},
{
Name: "${TASK}-admin",
PortLabel: "admin",
},
},
LogConfig: DefaultLogConfig(),
Resources: &Resources{
CPU: helper.IntToPtr(500),
MemoryMB: helper.IntToPtr(256),
Networks: []*NetworkResource{
{
MBits: helper.IntToPtr(50),
DynamicPorts: []Port{{Label: "http"}, {Label: "admin"}},
},
},
},
Meta: map[string]string{
"foo": "bar",
},
},
},
Meta: map[string]string{
"elb_check_type": "http",
"elb_check_interval": "30s",
"elb_check_min": "3",
},
},
},
Meta: map[string]string{
"owner": "armon",
},
}
job.Canonicalize()
return job
}
func MockPeriodicJob() *Job {
j := MockJob()
j.Type = helper.StringToPtr("batch")
j.Periodic = &PeriodicConfig{
Enabled: helper.BoolToPtr(true),
SpecType: helper.StringToPtr("cron"),
Spec: helper.StringToPtr("*/30 * * * *"),
}
return j
}

91
vendor/github.com/hashicorp/nomad/api/namespace.go generated vendored Normal file
View File

@@ -0,0 +1,91 @@
package api
import (
"fmt"
"sort"
)
// Namespaces is used to query the namespace endpoints.
type Namespaces struct {
client *Client
}
// Namespaces returns a new handle on the namespaces.
func (c *Client) Namespaces() *Namespaces {
return &Namespaces{client: c}
}
// List is used to dump all of the namespaces.
func (n *Namespaces) List(q *QueryOptions) ([]*Namespace, *QueryMeta, error) {
var resp []*Namespace
qm, err := n.client.query("/v1/namespaces", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(NamespaceIndexSort(resp))
return resp, qm, nil
}
// PrefixList is used to do a PrefixList search over namespaces
func (n *Namespaces) PrefixList(prefix string, q *QueryOptions) ([]*Namespace, *QueryMeta, error) {
if q == nil {
q = &QueryOptions{Prefix: prefix}
} else {
q.Prefix = prefix
}
return n.List(q)
}
// Info is used to query a single namespace by its name.
func (n *Namespaces) Info(name string, q *QueryOptions) (*Namespace, *QueryMeta, error) {
var resp Namespace
qm, err := n.client.query("/v1/namespace/"+name, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// Register is used to register a namespace.
func (n *Namespaces) Register(namespace *Namespace, q *WriteOptions) (*WriteMeta, error) {
wm, err := n.client.write("/v1/namespace", namespace, nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Delete is used to delete a namespace
func (n *Namespaces) Delete(namespace string, q *WriteOptions) (*WriteMeta, error) {
wm, err := n.client.delete(fmt.Sprintf("/v1/namespace/%s", namespace), nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Namespace is used to serialize a namespace.
type Namespace struct {
Name string
Description string
Quota string
CreateIndex uint64
ModifyIndex uint64
}
// NamespaceIndexSort is a wrapper to sort Namespaces by CreateIndex. We
// reverse the test so that we get the highest index first.
type NamespaceIndexSort []*Namespace
func (n NamespaceIndexSort) Len() int {
return len(n)
}
func (n NamespaceIndexSort) Less(i, j int) bool {
return n[i].CreateIndex > n[j].CreateIndex
}
func (n NamespaceIndexSort) Swap(i, j int) {
n[i], n[j] = n[j], n[i]
}

611
vendor/github.com/hashicorp/nomad/api/nodes.go generated vendored Normal file
View File

@@ -0,0 +1,611 @@
package api
import (
"context"
"fmt"
"sort"
"time"
"github.com/hashicorp/nomad/nomad/structs"
)
// Nodes is used to query node-related API endpoints
type Nodes struct {
client *Client
}
// Nodes returns a handle on the node endpoints.
func (c *Client) Nodes() *Nodes {
return &Nodes{client: c}
}
// List is used to list out all of the nodes
func (n *Nodes) List(q *QueryOptions) ([]*NodeListStub, *QueryMeta, error) {
var resp NodeIndexSort
qm, err := n.client.query("/v1/nodes", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(resp)
return resp, qm, nil
}
func (n *Nodes) PrefixList(prefix string) ([]*NodeListStub, *QueryMeta, error) {
return n.List(&QueryOptions{Prefix: prefix})
}
// Info is used to query a specific node by its ID.
func (n *Nodes) Info(nodeID string, q *QueryOptions) (*Node, *QueryMeta, error) {
var resp Node
qm, err := n.client.query("/v1/node/"+nodeID, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// NodeUpdateDrainRequest is used to update the drain specification for a node.
type NodeUpdateDrainRequest struct {
// NodeID is the node to update the drain specification for.
NodeID string
// DrainSpec is the drain specification to set for the node. A nil DrainSpec
// will disable draining.
DrainSpec *DrainSpec
// MarkEligible marks the node as eligible for scheduling if removing
// the drain strategy.
MarkEligible bool
}
// NodeDrainUpdateResponse is used to respond to a node drain update
type NodeDrainUpdateResponse struct {
NodeModifyIndex uint64
EvalIDs []string
EvalCreateIndex uint64
WriteMeta
}
// UpdateDrain is used to update the drain strategy for a given node. If
// markEligible is true and the drain is being removed, the node will be marked
// as having its scheduling being eligible
func (n *Nodes) UpdateDrain(nodeID string, spec *DrainSpec, markEligible bool, q *WriteOptions) (*NodeDrainUpdateResponse, error) {
req := &NodeUpdateDrainRequest{
NodeID: nodeID,
DrainSpec: spec,
MarkEligible: markEligible,
}
var resp NodeDrainUpdateResponse
wm, err := n.client.write("/v1/node/"+nodeID+"/drain", req, &resp, q)
if err != nil {
return nil, err
}
resp.WriteMeta = *wm
return &resp, nil
}
// MonitorMsgLevels represents the severity log level of a MonitorMessage.
type MonitorMsgLevel int
const (
MonitorMsgLevelNormal MonitorMsgLevel = 0
MonitorMsgLevelInfo MonitorMsgLevel = 1
MonitorMsgLevelWarn MonitorMsgLevel = 2
MonitorMsgLevelError MonitorMsgLevel = 3
)
// MonitorMessage contains a message and log level.
type MonitorMessage struct {
Level MonitorMsgLevel
Message string
}
// Messagef formats a new MonitorMessage.
func Messagef(lvl MonitorMsgLevel, msg string, args ...interface{}) *MonitorMessage {
return &MonitorMessage{
Level: lvl,
Message: fmt.Sprintf(msg, args...),
}
}
func (m *MonitorMessage) String() string {
return m.Message
}
// MonitorDrain emits drain related events on the returned string channel. The
// channel will be closed when all allocations on the draining node have
// stopped or the context is canceled.
func (n *Nodes) MonitorDrain(ctx context.Context, nodeID string, index uint64, ignoreSys bool) <-chan *MonitorMessage {
outCh := make(chan *MonitorMessage, 8)
nodeCh := make(chan *MonitorMessage, 1)
allocCh := make(chan *MonitorMessage, 8)
// Multiplex node and alloc chans onto outCh. This goroutine closes
// outCh when other chans have been closed or context canceled.
multiplexCtx, cancel := context.WithCancel(ctx)
go n.monitorDrainMultiplex(multiplexCtx, cancel, outCh, nodeCh, allocCh)
// Monitor node for updates
go n.monitorDrainNode(multiplexCtx, cancel, nodeID, index, nodeCh)
// Monitor allocs on node for updates
go n.monitorDrainAllocs(multiplexCtx, nodeID, ignoreSys, allocCh)
return outCh
}
// monitorDrainMultiplex multiplexes node and alloc updates onto the out chan.
// Closes out chan when either the context is canceled, both update chans are
// closed, or an error occurs.
func (n *Nodes) monitorDrainMultiplex(ctx context.Context, cancel func(),
outCh chan<- *MonitorMessage, nodeCh, allocCh <-chan *MonitorMessage) {
defer cancel()
defer close(outCh)
nodeOk := true
allocOk := true
var msg *MonitorMessage
for {
// If both chans have been closed, close the output chan
if !nodeOk && !allocOk {
return
}
select {
case msg, nodeOk = <-nodeCh:
if !nodeOk {
// nil chan to prevent further recvs
nodeCh = nil
}
case msg, allocOk = <-allocCh:
if !allocOk {
// nil chan to prevent further recvs
allocCh = nil
}
case <-ctx.Done():
return
}
if msg == nil {
continue
}
select {
case outCh <- msg:
case <-ctx.Done():
// If we are exiting but we have a message, attempt to send it
// so we don't lose a message but do not block.
select {
case outCh <- msg:
default:
}
return
}
// Abort on error messages
if msg.Level == MonitorMsgLevelError {
return
}
}
}
// monitorDrainNode emits node updates on nodeCh and closes the channel when
// the node has finished draining.
func (n *Nodes) monitorDrainNode(ctx context.Context, cancel func(),
nodeID string, index uint64, nodeCh chan<- *MonitorMessage) {
defer close(nodeCh)
var lastStrategy *DrainStrategy
var strategyChanged bool
q := QueryOptions{
AllowStale: true,
WaitIndex: index,
}
for {
node, meta, err := n.Info(nodeID, &q)
if err != nil {
msg := Messagef(MonitorMsgLevelError, "Error monitoring node: %v", err)
select {
case nodeCh <- msg:
case <-ctx.Done():
}
return
}
if node.DrainStrategy == nil {
var msg *MonitorMessage
if strategyChanged {
msg = Messagef(MonitorMsgLevelInfo, "Node %q has marked all allocations for migration", nodeID)
} else {
msg = Messagef(MonitorMsgLevelInfo, "No drain strategy set for node %s", nodeID)
defer cancel()
}
select {
case nodeCh <- msg:
case <-ctx.Done():
}
return
}
if node.Status == structs.NodeStatusDown {
msg := Messagef(MonitorMsgLevelWarn, "Node %q down", nodeID)
select {
case nodeCh <- msg:
case <-ctx.Done():
}
}
// DrainStrategy changed
if lastStrategy != nil && !node.DrainStrategy.Equal(lastStrategy) {
msg := Messagef(MonitorMsgLevelInfo, "Node %q drain updated: %s", nodeID, node.DrainStrategy)
select {
case nodeCh <- msg:
case <-ctx.Done():
return
}
}
lastStrategy = node.DrainStrategy
strategyChanged = true
// Drain still ongoing, update index and block for updates
q.WaitIndex = meta.LastIndex
}
}
// monitorDrainAllocs emits alloc updates on allocCh and closes the channel
// when the node has finished draining.
func (n *Nodes) monitorDrainAllocs(ctx context.Context, nodeID string, ignoreSys bool, allocCh chan<- *MonitorMessage) {
defer close(allocCh)
q := QueryOptions{AllowStale: true}
initial := make(map[string]*Allocation, 4)
for {
allocs, meta, err := n.Allocations(nodeID, &q)
if err != nil {
msg := Messagef(MonitorMsgLevelError, "Error monitoring allocations: %v", err)
select {
case allocCh <- msg:
case <-ctx.Done():
}
return
}
q.WaitIndex = meta.LastIndex
runningAllocs := 0
for _, a := range allocs {
// Get previous version of alloc
orig, existing := initial[a.ID]
// Update local alloc state
initial[a.ID] = a
migrating := a.DesiredTransition.ShouldMigrate()
var msg string
switch {
case !existing:
// Should only be possible if response
// from initial Allocations call was
// stale. No need to output
case orig.ClientStatus != a.ClientStatus:
// Alloc status has changed; output
msg = fmt.Sprintf("status %s -> %s", orig.ClientStatus, a.ClientStatus)
case migrating && !orig.DesiredTransition.ShouldMigrate():
// Alloc was marked for migration
msg = "marked for migration"
case migrating && (orig.DesiredStatus != a.DesiredStatus) && a.DesiredStatus == structs.AllocDesiredStatusStop:
// Alloc has already been marked for migration and is now being stopped
msg = "draining"
}
if msg != "" {
select {
case allocCh <- Messagef(MonitorMsgLevelNormal, "Alloc %q %s", a.ID, msg):
case <-ctx.Done():
return
}
}
// Ignore malformed allocs
if a.Job == nil || a.Job.Type == nil {
continue
}
// Track how many allocs are still running
if ignoreSys && a.Job.Type != nil && *a.Job.Type == structs.JobTypeSystem {
continue
}
switch a.ClientStatus {
case structs.AllocClientStatusPending, structs.AllocClientStatusRunning:
runningAllocs++
}
}
// Exit if all allocs are terminal
if runningAllocs == 0 {
msg := Messagef(MonitorMsgLevelInfo, "All allocations on node %q have stopped.", nodeID)
select {
case allocCh <- msg:
case <-ctx.Done():
}
return
}
}
}
// NodeUpdateEligibilityRequest is used to update the drain specification for a node.
type NodeUpdateEligibilityRequest struct {
// NodeID is the node to update the drain specification for.
NodeID string
Eligibility string
}
// NodeEligibilityUpdateResponse is used to respond to a node eligibility update
type NodeEligibilityUpdateResponse struct {
NodeModifyIndex uint64
EvalIDs []string
EvalCreateIndex uint64
WriteMeta
}
// ToggleEligibility is used to update the scheduling eligibility of the node
func (n *Nodes) ToggleEligibility(nodeID string, eligible bool, q *WriteOptions) (*NodeEligibilityUpdateResponse, error) {
e := structs.NodeSchedulingEligible
if !eligible {
e = structs.NodeSchedulingIneligible
}
req := &NodeUpdateEligibilityRequest{
NodeID: nodeID,
Eligibility: e,
}
var resp NodeEligibilityUpdateResponse
wm, err := n.client.write("/v1/node/"+nodeID+"/eligibility", req, &resp, q)
if err != nil {
return nil, err
}
resp.WriteMeta = *wm
return &resp, nil
}
// Allocations is used to return the allocations associated with a node.
func (n *Nodes) Allocations(nodeID string, q *QueryOptions) ([]*Allocation, *QueryMeta, error) {
var resp []*Allocation
qm, err := n.client.query("/v1/node/"+nodeID+"/allocations", &resp, q)
if err != nil {
return nil, nil, err
}
sort.Sort(AllocationSort(resp))
return resp, qm, nil
}
// ForceEvaluate is used to force-evaluate an existing node.
func (n *Nodes) ForceEvaluate(nodeID string, q *WriteOptions) (string, *WriteMeta, error) {
var resp nodeEvalResponse
wm, err := n.client.write("/v1/node/"+nodeID+"/evaluate", nil, &resp, q)
if err != nil {
return "", nil, err
}
return resp.EvalID, wm, nil
}
func (n *Nodes) Stats(nodeID string, q *QueryOptions) (*HostStats, error) {
var resp HostStats
path := fmt.Sprintf("/v1/client/stats?node_id=%s", nodeID)
if _, err := n.client.query(path, &resp, q); err != nil {
return nil, err
}
return &resp, nil
}
func (n *Nodes) GC(nodeID string, q *QueryOptions) error {
var resp struct{}
path := fmt.Sprintf("/v1/client/gc?node_id=%s", nodeID)
_, err := n.client.query(path, &resp, q)
return err
}
// TODO Add tests
func (n *Nodes) GcAlloc(allocID string, q *QueryOptions) error {
var resp struct{}
path := fmt.Sprintf("/v1/client/allocation/%s/gc", allocID)
_, err := n.client.query(path, &resp, q)
return err
}
// DriverInfo is used to deserialize a DriverInfo entry
type DriverInfo struct {
Attributes map[string]string
Detected bool
Healthy bool
HealthDescription string
UpdateTime time.Time
}
// Node is used to deserialize a node entry.
type Node struct {
ID string
Datacenter string
Name string
HTTPAddr string
TLSEnabled bool
Attributes map[string]string
Resources *Resources
Reserved *Resources
Links map[string]string
Meta map[string]string
NodeClass string
Drain bool
DrainStrategy *DrainStrategy
SchedulingEligibility string
Status string
StatusDescription string
StatusUpdatedAt int64
Events []*NodeEvent
Drivers map[string]*DriverInfo
CreateIndex uint64
ModifyIndex uint64
}
// DrainStrategy describes a Node's drain behavior.
type DrainStrategy struct {
// DrainSpec is the user declared drain specification
DrainSpec
// ForceDeadline is the deadline time for the drain after which drains will
// be forced
ForceDeadline time.Time
}
// DrainSpec describes a Node's drain behavior.
type DrainSpec struct {
// Deadline is the duration after StartTime when the remaining
// allocations on a draining Node should be told to stop.
Deadline time.Duration
// IgnoreSystemJobs allows systems jobs to remain on the node even though it
// has been marked for draining.
IgnoreSystemJobs bool
}
func (d *DrainStrategy) Equal(o *DrainStrategy) bool {
if d == nil || o == nil {
return d == o
}
if d.ForceDeadline != o.ForceDeadline {
return false
}
if d.Deadline != o.Deadline {
return false
}
if d.IgnoreSystemJobs != o.IgnoreSystemJobs {
return false
}
return true
}
// String returns a human readable version of the drain strategy.
func (d *DrainStrategy) String() string {
if d.IgnoreSystemJobs {
return fmt.Sprintf("drain ignoring system jobs and deadline at %s", d.ForceDeadline)
}
return fmt.Sprintf("drain with deadline at %s", d.ForceDeadline)
}
const (
NodeEventSubsystemDrain = "Drain"
NodeEventSubsystemDriver = "Driver"
NodeEventSubsystemHeartbeat = "Heartbeat"
NodeEventSubsystemCluster = "Cluster"
)
// NodeEvent is a single unit representing a nodes state change
type NodeEvent struct {
Message string
Subsystem string
Details map[string]string
Timestamp time.Time
CreateIndex uint64
}
// HostStats represents resource usage stats of the host running a Nomad client
type HostStats struct {
Memory *HostMemoryStats
CPU []*HostCPUStats
DiskStats []*HostDiskStats
Uptime uint64
CPUTicksConsumed float64
}
type HostMemoryStats struct {
Total uint64
Available uint64
Used uint64
Free uint64
}
type HostCPUStats struct {
CPU string
User float64
System float64
Idle float64
}
type HostDiskStats struct {
Device string
Mountpoint string
Size uint64
Used uint64
Available uint64
UsedPercent float64
InodesUsedPercent float64
}
// NodeListStub is a subset of information returned during
// node list operations.
type NodeListStub struct {
Address string
ID string
Datacenter string
Name string
NodeClass string
Version string
Drain bool
SchedulingEligibility string
Status string
StatusDescription string
Drivers map[string]*DriverInfo
CreateIndex uint64
ModifyIndex uint64
}
// NodeIndexSort reverse sorts nodes by CreateIndex
type NodeIndexSort []*NodeListStub
func (n NodeIndexSort) Len() int {
return len(n)
}
func (n NodeIndexSort) Less(i, j int) bool {
return n[i].CreateIndex > n[j].CreateIndex
}
func (n NodeIndexSort) Swap(i, j int) {
n[i], n[j] = n[j], n[i]
}
// nodeEvalResponse is used to decode a force-eval.
type nodeEvalResponse struct {
EvalID string
}
// AllocationSort reverse sorts allocs by CreateIndex.
type AllocationSort []*Allocation
func (a AllocationSort) Len() int {
return len(a)
}
func (a AllocationSort) Less(i, j int) bool {
return a[i].CreateIndex > a[j].CreateIndex
}
func (a AllocationSort) Swap(i, j int) {
a[i], a[j] = a[j], a[i]
}

108
vendor/github.com/hashicorp/nomad/api/operator.go generated vendored Normal file
View File

@@ -0,0 +1,108 @@
package api
// Operator can be used to perform low-level operator tasks for Nomad.
type Operator struct {
c *Client
}
// Operator returns a handle to the operator endpoints.
func (c *Client) Operator() *Operator {
return &Operator{c}
}
// RaftServer has information about a server in the Raft configuration.
type RaftServer struct {
// ID is the unique ID for the server. These are currently the same
// as the address, but they will be changed to a real GUID in a future
// release of Nomad.
ID string
// Node is the node name of the server, as known by Nomad, or this
// will be set to "(unknown)" otherwise.
Node string
// Address is the IP:port of the server, used for Raft communications.
Address string
// Leader is true if this server is the current cluster leader.
Leader bool
// Voter is true if this server has a vote in the cluster. This might
// be false if the server is staging and still coming online, or if
// it's a non-voting server, which will be added in a future release of
// Nomad.
Voter bool
// RaftProtocol is the version of the Raft protocol spoken by this server.
RaftProtocol string
}
// RaftConfiguration is returned when querying for the current Raft configuration.
type RaftConfiguration struct {
// Servers has the list of servers in the Raft configuration.
Servers []*RaftServer
// Index has the Raft index of this configuration.
Index uint64
}
// RaftGetConfiguration is used to query the current Raft peer set.
func (op *Operator) RaftGetConfiguration(q *QueryOptions) (*RaftConfiguration, error) {
r, err := op.c.newRequest("GET", "/v1/operator/raft/configuration")
if err != nil {
return nil, err
}
r.setQueryOptions(q)
_, resp, err := requireOK(op.c.doRequest(r))
if err != nil {
return nil, err
}
defer resp.Body.Close()
var out RaftConfiguration
if err := decodeBody(resp, &out); err != nil {
return nil, err
}
return &out, nil
}
// RaftRemovePeerByAddress is used to kick a stale peer (one that it in the Raft
// quorum but no longer known to Serf or the catalog) by address in the form of
// "IP:port".
func (op *Operator) RaftRemovePeerByAddress(address string, q *WriteOptions) error {
r, err := op.c.newRequest("DELETE", "/v1/operator/raft/peer")
if err != nil {
return err
}
r.setWriteOptions(q)
r.params.Set("address", address)
_, resp, err := requireOK(op.c.doRequest(r))
if err != nil {
return err
}
resp.Body.Close()
return nil
}
// RaftRemovePeerByID is used to kick a stale peer (one that is in the Raft
// quorum but no longer known to Serf or the catalog) by ID.
func (op *Operator) RaftRemovePeerByID(id string, q *WriteOptions) error {
r, err := op.c.newRequest("DELETE", "/v1/operator/raft/peer")
if err != nil {
return err
}
r.setWriteOptions(q)
r.params.Set("id", id)
_, resp, err := requireOK(op.c.doRequest(r))
if err != nil {
return err
}
resp.Body.Close()
return nil
}

View File

@@ -0,0 +1,218 @@
package api
import (
"encoding/json"
"strconv"
"time"
)
// AutopilotConfiguration is used for querying/setting the Autopilot configuration.
// Autopilot helps manage operator tasks related to Nomad servers like removing
// failed servers from the Raft quorum.
type AutopilotConfiguration struct {
// CleanupDeadServers controls whether to remove dead servers from the Raft
// peer list when a new server joins
CleanupDeadServers bool
// LastContactThreshold is the limit on the amount of time a server can go
// without leader contact before being considered unhealthy.
LastContactThreshold time.Duration
// MaxTrailingLogs is the amount of entries in the Raft Log that a server can
// be behind before being considered unhealthy.
MaxTrailingLogs uint64
// ServerStabilizationTime is the minimum amount of time a server must be
// in a stable, healthy state before it can be added to the cluster. Only
// applicable with Raft protocol version 3 or higher.
ServerStabilizationTime time.Duration
// (Enterprise-only) EnableRedundancyZones specifies whether to enable redundancy zones.
EnableRedundancyZones bool
// (Enterprise-only) DisableUpgradeMigration will disable Autopilot's upgrade migration
// strategy of waiting until enough newer-versioned servers have been added to the
// cluster before promoting them to voters.
DisableUpgradeMigration bool
// (Enterprise-only) EnableCustomUpgrades specifies whether to enable using custom
// upgrade versions when performing migrations.
EnableCustomUpgrades bool
// CreateIndex holds the index corresponding the creation of this configuration.
// This is a read-only field.
CreateIndex uint64
// ModifyIndex will be set to the index of the last update when retrieving the
// Autopilot configuration. Resubmitting a configuration with
// AutopilotCASConfiguration will perform a check-and-set operation which ensures
// there hasn't been a subsequent update since the configuration was retrieved.
ModifyIndex uint64
}
func (u *AutopilotConfiguration) MarshalJSON() ([]byte, error) {
type Alias AutopilotConfiguration
return json.Marshal(&struct {
LastContactThreshold string
ServerStabilizationTime string
*Alias
}{
LastContactThreshold: u.LastContactThreshold.String(),
ServerStabilizationTime: u.ServerStabilizationTime.String(),
Alias: (*Alias)(u),
})
}
func (u *AutopilotConfiguration) UnmarshalJSON(data []byte) error {
type Alias AutopilotConfiguration
aux := &struct {
LastContactThreshold string
ServerStabilizationTime string
*Alias
}{
Alias: (*Alias)(u),
}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
var err error
if aux.LastContactThreshold != "" {
if u.LastContactThreshold, err = time.ParseDuration(aux.LastContactThreshold); err != nil {
return err
}
}
if aux.ServerStabilizationTime != "" {
if u.ServerStabilizationTime, err = time.ParseDuration(aux.ServerStabilizationTime); err != nil {
return err
}
}
return nil
}
// ServerHealth is the health (from the leader's point of view) of a server.
type ServerHealth struct {
// ID is the raft ID of the server.
ID string
// Name is the node name of the server.
Name string
// Address is the address of the server.
Address string
// The status of the SerfHealth check for the server.
SerfStatus string
// Version is the Nomad version of the server.
Version string
// Leader is whether this server is currently the leader.
Leader bool
// LastContact is the time since this node's last contact with the leader.
LastContact time.Duration
// LastTerm is the highest leader term this server has a record of in its Raft log.
LastTerm uint64
// LastIndex is the last log index this server has a record of in its Raft log.
LastIndex uint64
// Healthy is whether or not the server is healthy according to the current
// Autopilot config.
Healthy bool
// Voter is whether this is a voting server.
Voter bool
// StableSince is the last time this server's Healthy value changed.
StableSince time.Time
}
func (u *ServerHealth) MarshalJSON() ([]byte, error) {
type Alias ServerHealth
return json.Marshal(&struct {
LastContact string
*Alias
}{
LastContact: u.LastContact.String(),
Alias: (*Alias)(u),
})
}
func (u *ServerHealth) UnmarshalJSON(data []byte) error {
type Alias ServerHealth
aux := &struct {
LastContact string
*Alias
}{
Alias: (*Alias)(u),
}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
var err error
if aux.LastContact != "" {
if u.LastContact, err = time.ParseDuration(aux.LastContact); err != nil {
return err
}
}
return nil
}
// OperatorHealthReply is a representation of the overall health of the cluster
type OperatorHealthReply struct {
// Healthy is true if all the servers in the cluster are healthy.
Healthy bool
// FailureTolerance is the number of healthy servers that could be lost without
// an outage occurring.
FailureTolerance int
// Servers holds the health of each server.
Servers []ServerHealth
}
// AutopilotGetConfiguration is used to query the current Autopilot configuration.
func (op *Operator) AutopilotGetConfiguration(q *QueryOptions) (*AutopilotConfiguration, *QueryMeta, error) {
var resp AutopilotConfiguration
qm, err := op.c.query("/v1/operator/autopilot/configuration", &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// AutopilotSetConfiguration is used to set the current Autopilot configuration.
func (op *Operator) AutopilotSetConfiguration(conf *AutopilotConfiguration, q *WriteOptions) (*WriteMeta, error) {
var out bool
wm, err := op.c.write("/v1/operator/autopilot/configuration", conf, &out, q)
if err != nil {
return nil, err
}
return wm, nil
}
// AutopilotCASConfiguration is used to perform a Check-And-Set update on the
// Autopilot configuration. The ModifyIndex value will be respected. Returns
// true on success or false on failures.
func (op *Operator) AutopilotCASConfiguration(conf *AutopilotConfiguration, q *WriteOptions) (bool, *WriteMeta, error) {
var out bool
wm, err := op.c.write("/v1/operator/autopilot/configuration?cas="+strconv.FormatUint(conf.ModifyIndex, 10), conf, &out, q)
if err != nil {
return false, nil, err
}
return out, wm, nil
}
// AutopilotServerHealth is used to query Autopilot's top-level view of the health
// of each Nomad server.
func (op *Operator) AutopilotServerHealth(q *QueryOptions) (*OperatorHealthReply, *QueryMeta, error) {
var out OperatorHealthReply
qm, err := op.c.query("/v1/operator/autopilot/health", &out, q)
if err != nil {
return nil, nil, err
}
return &out, qm, nil
}

186
vendor/github.com/hashicorp/nomad/api/quota.go generated vendored Normal file
View File

@@ -0,0 +1,186 @@
package api
import (
"fmt"
"sort"
)
// Quotas is used to query the quotas endpoints.
type Quotas struct {
client *Client
}
// Quotas returns a new handle on the quotas.
func (c *Client) Quotas() *Quotas {
return &Quotas{client: c}
}
// List is used to dump all of the quota specs
func (q *Quotas) List(qo *QueryOptions) ([]*QuotaSpec, *QueryMeta, error) {
var resp []*QuotaSpec
qm, err := q.client.query("/v1/quotas", &resp, qo)
if err != nil {
return nil, nil, err
}
sort.Sort(QuotaSpecIndexSort(resp))
return resp, qm, nil
}
// PrefixList is used to do a PrefixList search over quota specs
func (q *Quotas) PrefixList(prefix string, qo *QueryOptions) ([]*QuotaSpec, *QueryMeta, error) {
if qo == nil {
qo = &QueryOptions{Prefix: prefix}
} else {
qo.Prefix = prefix
}
return q.List(qo)
}
// ListUsage is used to dump all of the quota usages
func (q *Quotas) ListUsage(qo *QueryOptions) ([]*QuotaUsage, *QueryMeta, error) {
var resp []*QuotaUsage
qm, err := q.client.query("/v1/quota-usages", &resp, qo)
if err != nil {
return nil, nil, err
}
sort.Sort(QuotaUsageIndexSort(resp))
return resp, qm, nil
}
// PrefixList is used to do a PrefixList search over quota usages
func (q *Quotas) PrefixListUsage(prefix string, qo *QueryOptions) ([]*QuotaUsage, *QueryMeta, error) {
if qo == nil {
qo = &QueryOptions{Prefix: prefix}
} else {
qo.Prefix = prefix
}
return q.ListUsage(qo)
}
// Info is used to query a single quota spec by its name.
func (q *Quotas) Info(name string, qo *QueryOptions) (*QuotaSpec, *QueryMeta, error) {
var resp QuotaSpec
qm, err := q.client.query("/v1/quota/"+name, &resp, qo)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// Usage is used to query a single quota usage by its name.
func (q *Quotas) Usage(name string, qo *QueryOptions) (*QuotaUsage, *QueryMeta, error) {
var resp QuotaUsage
qm, err := q.client.query("/v1/quota/usage/"+name, &resp, qo)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
// Register is used to register a quota spec.
func (q *Quotas) Register(spec *QuotaSpec, qo *WriteOptions) (*WriteMeta, error) {
wm, err := q.client.write("/v1/quota", spec, nil, qo)
if err != nil {
return nil, err
}
return wm, nil
}
// Delete is used to delete a quota spec
func (q *Quotas) Delete(quota string, qo *WriteOptions) (*WriteMeta, error) {
wm, err := q.client.delete(fmt.Sprintf("/v1/quota/%s", quota), nil, qo)
if err != nil {
return nil, err
}
return wm, nil
}
// QuotaSpec specifies the allowed resource usage across regions.
type QuotaSpec struct {
// Name is the name for the quota object
Name string
// Description is an optional description for the quota object
Description string
// Limits is the set of quota limits encapsulated by this quota object. Each
// limit applies quota in a particular region and in the future over a
// particular priority range and datacenter set.
Limits []*QuotaLimit
// Raft indexes to track creation and modification
CreateIndex uint64
ModifyIndex uint64
}
// QuotaLimit describes the resource limit in a particular region.
type QuotaLimit struct {
// Region is the region in which this limit has affect
Region string
// RegionLimit is the quota limit that applies to any allocation within a
// referencing namespace in the region. A value of zero is treated as
// unlimited and a negative value is treated as fully disallowed. This is
// useful for once we support GPUs
RegionLimit *Resources
// Hash is the hash of the object and is used to make replication efficient.
Hash []byte
}
// QuotaUsage is the resource usage of a Quota
type QuotaUsage struct {
Name string
Used map[string]*QuotaLimit
CreateIndex uint64
ModifyIndex uint64
}
// QuotaSpecIndexSort is a wrapper to sort QuotaSpecs by CreateIndex. We
// reverse the test so that we get the highest index first.
type QuotaSpecIndexSort []*QuotaSpec
func (q QuotaSpecIndexSort) Len() int {
return len(q)
}
func (q QuotaSpecIndexSort) Less(i, j int) bool {
return q[i].CreateIndex > q[j].CreateIndex
}
func (q QuotaSpecIndexSort) Swap(i, j int) {
q[i], q[j] = q[j], q[i]
}
// QuotaUsageIndexSort is a wrapper to sort QuotaUsages by CreateIndex. We
// reverse the test so that we get the highest index first.
type QuotaUsageIndexSort []*QuotaUsage
func (q QuotaUsageIndexSort) Len() int {
return len(q)
}
func (q QuotaUsageIndexSort) Less(i, j int) bool {
return q[i].CreateIndex > q[j].CreateIndex
}
func (q QuotaUsageIndexSort) Swap(i, j int) {
q[i], q[j] = q[j], q[i]
}
// QuotaLimitSort is a wrapper to sort QuotaLimits
type QuotaLimitSort []*QuotaLimit
func (q QuotaLimitSort) Len() int {
return len(q)
}
func (q QuotaLimitSort) Less(i, j int) bool {
return q[i].Region < q[j].Region
}
func (q QuotaLimitSort) Swap(i, j int) {
q[i], q[j] = q[j], q[i]
}

38
vendor/github.com/hashicorp/nomad/api/raw.go generated vendored Normal file
View File

@@ -0,0 +1,38 @@
package api
import "io"
// Raw can be used to do raw queries against custom endpoints
type Raw struct {
c *Client
}
// Raw returns a handle to query endpoints
func (c *Client) Raw() *Raw {
return &Raw{c}
}
// Query is used to do a GET request against an endpoint
// and deserialize the response into an interface using
// standard Nomad conventions.
func (raw *Raw) Query(endpoint string, out interface{}, q *QueryOptions) (*QueryMeta, error) {
return raw.c.query(endpoint, out, q)
}
// Response is used to make a GET request against an endpoint and returns the
// response body
func (raw *Raw) Response(endpoint string, q *QueryOptions) (io.ReadCloser, error) {
return raw.c.rawQuery(endpoint, q)
}
// Write is used to do a PUT request against an endpoint
// and serialize/deserialized using the standard Nomad conventions.
func (raw *Raw) Write(endpoint string, in, out interface{}, q *WriteOptions) (*WriteMeta, error) {
return raw.c.write(endpoint, in, out, q)
}
// Delete is used to do a DELETE request against an endpoint
// and serialize/deserialized using the standard Nomad conventions.
func (raw *Raw) Delete(endpoint string, out interface{}, q *WriteOptions) (*WriteMeta, error) {
return raw.c.delete(endpoint, out, q)
}

23
vendor/github.com/hashicorp/nomad/api/regions.go generated vendored Normal file
View File

@@ -0,0 +1,23 @@
package api
import "sort"
// Regions is used to query the regions in the cluster.
type Regions struct {
client *Client
}
// Regions returns a handle on the regions endpoints.
func (c *Client) Regions() *Regions {
return &Regions{client: c}
}
// List returns a list of all of the regions.
func (r *Regions) List() ([]string, error) {
var resp []string
if _, err := r.client.query("/v1/regions", &resp, nil); err != nil {
return nil, err
}
sort.Strings(resp)
return resp, nil
}

100
vendor/github.com/hashicorp/nomad/api/resources.go generated vendored Normal file
View File

@@ -0,0 +1,100 @@
package api
import "github.com/hashicorp/nomad/helper"
// Resources encapsulates the required resources of
// a given task or task group.
type Resources struct {
CPU *int
MemoryMB *int `mapstructure:"memory"`
DiskMB *int `mapstructure:"disk"`
IOPS *int
Networks []*NetworkResource
}
// Canonicalize will supply missing values in the cases
// where they are not provided.
func (r *Resources) Canonicalize() {
defaultResources := DefaultResources()
if r.CPU == nil {
r.CPU = defaultResources.CPU
}
if r.MemoryMB == nil {
r.MemoryMB = defaultResources.MemoryMB
}
if r.IOPS == nil {
r.IOPS = defaultResources.IOPS
}
for _, n := range r.Networks {
n.Canonicalize()
}
}
// DefaultResources is a small resources object that contains the
// default resources requests that we will provide to an object.
// --- THIS FUNCTION IS REPLICATED IN nomad/structs/structs.go
// and should be kept in sync.
func DefaultResources() *Resources {
return &Resources{
CPU: helper.IntToPtr(100),
MemoryMB: helper.IntToPtr(300),
IOPS: helper.IntToPtr(0),
}
}
// MinResources is a small resources object that contains the
// absolute minimum resources that we will provide to an object.
// This should not be confused with the defaults which are
// provided in DefaultResources() --- THIS LOGIC IS REPLICATED
// IN nomad/structs/structs.go and should be kept in sync.
func MinResources() *Resources {
return &Resources{
CPU: helper.IntToPtr(20),
MemoryMB: helper.IntToPtr(10),
IOPS: helper.IntToPtr(0),
}
}
// Merge merges this resource with another resource.
func (r *Resources) Merge(other *Resources) {
if other == nil {
return
}
if other.CPU != nil {
r.CPU = other.CPU
}
if other.MemoryMB != nil {
r.MemoryMB = other.MemoryMB
}
if other.DiskMB != nil {
r.DiskMB = other.DiskMB
}
if other.IOPS != nil {
r.IOPS = other.IOPS
}
if len(other.Networks) != 0 {
r.Networks = other.Networks
}
}
type Port struct {
Label string
Value int `mapstructure:"static"`
}
// NetworkResource is used to describe required network
// resources of a given task.
type NetworkResource struct {
Device string
CIDR string
IP string
MBits *int
ReservedPorts []Port
DynamicPorts []Port
}
func (n *NetworkResource) Canonicalize() {
if n.MBits == nil {
n.MBits = helper.IntToPtr(10)
}
}

39
vendor/github.com/hashicorp/nomad/api/search.go generated vendored Normal file
View File

@@ -0,0 +1,39 @@
package api
import (
"github.com/hashicorp/nomad/api/contexts"
)
type Search struct {
client *Client
}
// Search returns a handle on the Search endpoints
func (c *Client) Search() *Search {
return &Search{client: c}
}
// PrefixSearch returns a list of matches for a particular context and prefix.
func (s *Search) PrefixSearch(prefix string, context contexts.Context, q *QueryOptions) (*SearchResponse, *QueryMeta, error) {
var resp SearchResponse
req := &SearchRequest{Prefix: prefix, Context: context}
qm, err := s.client.putQuery("/v1/search", req, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, qm, nil
}
type SearchRequest struct {
Prefix string
Context contexts.Context
QueryOptions
}
type SearchResponse struct {
Matches map[contexts.Context][]string
Truncations map[contexts.Context]bool
QueryMeta
}

79
vendor/github.com/hashicorp/nomad/api/sentinel.go generated vendored Normal file
View File

@@ -0,0 +1,79 @@
package api
import "fmt"
// SentinelPolicies is used to query the Sentinel Policy endpoints.
type SentinelPolicies struct {
client *Client
}
// SentinelPolicies returns a new handle on the Sentinel policies.
func (c *Client) SentinelPolicies() *SentinelPolicies {
return &SentinelPolicies{client: c}
}
// List is used to dump all of the policies.
func (a *SentinelPolicies) List(q *QueryOptions) ([]*SentinelPolicyListStub, *QueryMeta, error) {
var resp []*SentinelPolicyListStub
qm, err := a.client.query("/v1/sentinel/policies", &resp, q)
if err != nil {
return nil, nil, err
}
return resp, qm, nil
}
// Upsert is used to create or update a policy
func (a *SentinelPolicies) Upsert(policy *SentinelPolicy, q *WriteOptions) (*WriteMeta, error) {
if policy == nil || policy.Name == "" {
return nil, fmt.Errorf("missing policy name")
}
wm, err := a.client.write("/v1/sentinel/policy/"+policy.Name, policy, nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Delete is used to delete a policy
func (a *SentinelPolicies) Delete(policyName string, q *WriteOptions) (*WriteMeta, error) {
if policyName == "" {
return nil, fmt.Errorf("missing policy name")
}
wm, err := a.client.delete("/v1/sentinel/policy/"+policyName, nil, q)
if err != nil {
return nil, err
}
return wm, nil
}
// Info is used to query a specific policy
func (a *SentinelPolicies) Info(policyName string, q *QueryOptions) (*SentinelPolicy, *QueryMeta, error) {
if policyName == "" {
return nil, nil, fmt.Errorf("missing policy name")
}
var resp SentinelPolicy
wm, err := a.client.query("/v1/sentinel/policy/"+policyName, &resp, q)
if err != nil {
return nil, nil, err
}
return &resp, wm, nil
}
type SentinelPolicy struct {
Name string
Description string
Scope string
EnforcementLevel string
Policy string
CreateIndex uint64
ModifyIndex uint64
}
type SentinelPolicyListStub struct {
Name string
Description string
Scope string
EnforcementLevel string
CreateIndex uint64
ModifyIndex uint64
}

43
vendor/github.com/hashicorp/nomad/api/status.go generated vendored Normal file
View File

@@ -0,0 +1,43 @@
package api
// Status is used to query the status-related endpoints.
type Status struct {
client *Client
}
// Status returns a handle on the status endpoints.
func (c *Client) Status() *Status {
return &Status{client: c}
}
// Leader is used to query for the current cluster leader.
func (s *Status) Leader() (string, error) {
var resp string
_, err := s.client.query("/v1/status/leader", &resp, nil)
if err != nil {
return "", err
}
return resp, nil
}
// RegionLeader is used to query for the leader in the passed region.
func (s *Status) RegionLeader(region string) (string, error) {
var resp string
q := QueryOptions{Region: region}
_, err := s.client.query("/v1/status/leader", &resp, &q)
if err != nil {
return "", err
}
return resp, nil
}
// Peers is used to query the addresses of the server peers
// in the cluster.
func (s *Status) Peers() ([]string, error) {
var resp []string
_, err := s.client.query("/v1/status/peers", &resp, nil)
if err != nil {
return nil, err
}
return resp, nil
}

23
vendor/github.com/hashicorp/nomad/api/system.go generated vendored Normal file
View File

@@ -0,0 +1,23 @@
package api
// Status is used to query the status-related endpoints.
type System struct {
client *Client
}
// System returns a handle on the system endpoints.
func (c *Client) System() *System {
return &System{client: c}
}
func (s *System) GarbageCollect() error {
var req struct{}
_, err := s.client.write("/v1/system/gc", &req, nil, nil)
return err
}
func (s *System) ReconcileSummaries() error {
var req struct{}
_, err := s.client.write("/v1/system/reconcile/summaries", &req, nil, nil)
return err
}

843
vendor/github.com/hashicorp/nomad/api/tasks.go generated vendored Normal file
View File

@@ -0,0 +1,843 @@
package api
import (
"fmt"
"path"
"path/filepath"
"strings"
"time"
"github.com/hashicorp/nomad/helper"
"github.com/hashicorp/nomad/nomad/structs"
)
// MemoryStats holds memory usage related stats
type MemoryStats struct {
RSS uint64
Cache uint64
Swap uint64
MaxUsage uint64
KernelUsage uint64
KernelMaxUsage uint64
Measured []string
}
// CpuStats holds cpu usage related stats
type CpuStats struct {
SystemMode float64
UserMode float64
TotalTicks float64
ThrottledPeriods uint64
ThrottledTime uint64
Percent float64
Measured []string
}
// ResourceUsage holds information related to cpu and memory stats
type ResourceUsage struct {
MemoryStats *MemoryStats
CpuStats *CpuStats
}
// TaskResourceUsage holds aggregated resource usage of all processes in a Task
// and the resource usage of the individual pids
type TaskResourceUsage struct {
ResourceUsage *ResourceUsage
Timestamp int64
Pids map[string]*ResourceUsage
}
// AllocResourceUsage holds the aggregated task resource usage of the
// allocation.
type AllocResourceUsage struct {
ResourceUsage *ResourceUsage
Tasks map[string]*TaskResourceUsage
Timestamp int64
}
// RestartPolicy defines how the Nomad client restarts
// tasks in a taskgroup when they fail
type RestartPolicy struct {
Interval *time.Duration
Attempts *int
Delay *time.Duration
Mode *string
}
func (r *RestartPolicy) Merge(rp *RestartPolicy) {
if rp.Interval != nil {
r.Interval = rp.Interval
}
if rp.Attempts != nil {
r.Attempts = rp.Attempts
}
if rp.Delay != nil {
r.Delay = rp.Delay
}
if rp.Mode != nil {
r.Mode = rp.Mode
}
}
// Reschedule configures how Tasks are rescheduled when they crash or fail.
type ReschedulePolicy struct {
// Attempts limits the number of rescheduling attempts that can occur in an interval.
Attempts *int `mapstructure:"attempts"`
// Interval is a duration in which we can limit the number of reschedule attempts.
Interval *time.Duration `mapstructure:"interval"`
// Delay is a minimum duration to wait between reschedule attempts.
// The delay function determines how much subsequent reschedule attempts are delayed by.
Delay *time.Duration `mapstructure:"delay"`
// DelayFunction determines how the delay progressively changes on subsequent reschedule
// attempts. Valid values are "exponential", "constant", and "fibonacci".
DelayFunction *string `mapstructure:"delay_function"`
// MaxDelay is an upper bound on the delay.
MaxDelay *time.Duration `mapstructure:"max_delay"`
// Unlimited allows rescheduling attempts until they succeed
Unlimited *bool `mapstructure:"unlimited"`
}
func (r *ReschedulePolicy) Merge(rp *ReschedulePolicy) {
if rp == nil {
return
}
if rp.Interval != nil {
r.Interval = rp.Interval
}
if rp.Attempts != nil {
r.Attempts = rp.Attempts
}
if rp.Delay != nil {
r.Delay = rp.Delay
}
if rp.DelayFunction != nil {
r.DelayFunction = rp.DelayFunction
}
if rp.MaxDelay != nil {
r.MaxDelay = rp.MaxDelay
}
if rp.Unlimited != nil {
r.Unlimited = rp.Unlimited
}
}
func (r *ReschedulePolicy) Canonicalize(jobType string) {
dp := NewDefaultReschedulePolicy(jobType)
if r.Interval == nil {
r.Interval = dp.Interval
}
if r.Attempts == nil {
r.Attempts = dp.Attempts
}
if r.Delay == nil {
r.Delay = dp.Delay
}
if r.DelayFunction == nil {
r.DelayFunction = dp.DelayFunction
}
if r.MaxDelay == nil {
r.MaxDelay = dp.MaxDelay
}
if r.Unlimited == nil {
r.Unlimited = dp.Unlimited
}
}
func NewDefaultReschedulePolicy(jobType string) *ReschedulePolicy {
var dp *ReschedulePolicy
switch jobType {
case "service":
dp = &ReschedulePolicy{
Attempts: helper.IntToPtr(structs.DefaultServiceJobReschedulePolicy.Attempts),
Interval: helper.TimeToPtr(structs.DefaultServiceJobReschedulePolicy.Interval),
Delay: helper.TimeToPtr(structs.DefaultServiceJobReschedulePolicy.Delay),
DelayFunction: helper.StringToPtr(structs.DefaultServiceJobReschedulePolicy.DelayFunction),
MaxDelay: helper.TimeToPtr(structs.DefaultServiceJobReschedulePolicy.MaxDelay),
Unlimited: helper.BoolToPtr(structs.DefaultServiceJobReschedulePolicy.Unlimited),
}
case "batch":
dp = &ReschedulePolicy{
Attempts: helper.IntToPtr(structs.DefaultBatchJobReschedulePolicy.Attempts),
Interval: helper.TimeToPtr(structs.DefaultBatchJobReschedulePolicy.Interval),
Delay: helper.TimeToPtr(structs.DefaultBatchJobReschedulePolicy.Delay),
DelayFunction: helper.StringToPtr(structs.DefaultBatchJobReschedulePolicy.DelayFunction),
MaxDelay: helper.TimeToPtr(structs.DefaultBatchJobReschedulePolicy.MaxDelay),
Unlimited: helper.BoolToPtr(structs.DefaultBatchJobReschedulePolicy.Unlimited),
}
case "system":
dp = &ReschedulePolicy{
Attempts: helper.IntToPtr(0),
Interval: helper.TimeToPtr(0),
Delay: helper.TimeToPtr(0),
DelayFunction: helper.StringToPtr(""),
MaxDelay: helper.TimeToPtr(0),
Unlimited: helper.BoolToPtr(false),
}
}
return dp
}
func (r *ReschedulePolicy) Copy() *ReschedulePolicy {
if r == nil {
return nil
}
nrp := new(ReschedulePolicy)
*nrp = *r
return nrp
}
func (p *ReschedulePolicy) String() string {
if p == nil {
return ""
}
if *p.Unlimited {
return fmt.Sprintf("unlimited with %v delay, max_delay = %v", *p.DelayFunction, *p.MaxDelay)
}
return fmt.Sprintf("%v in %v with %v delay, max_delay = %v", *p.Attempts, *p.Interval, *p.DelayFunction, *p.MaxDelay)
}
// CheckRestart describes if and when a task should be restarted based on
// failing health checks.
type CheckRestart struct {
Limit int `mapstructure:"limit"`
Grace *time.Duration `mapstructure:"grace"`
IgnoreWarnings bool `mapstructure:"ignore_warnings"`
}
// Canonicalize CheckRestart fields if not nil.
func (c *CheckRestart) Canonicalize() {
if c == nil {
return
}
if c.Grace == nil {
c.Grace = helper.TimeToPtr(1 * time.Second)
}
}
// Copy returns a copy of CheckRestart or nil if unset.
func (c *CheckRestart) Copy() *CheckRestart {
if c == nil {
return nil
}
nc := new(CheckRestart)
nc.Limit = c.Limit
if c.Grace != nil {
g := *c.Grace
nc.Grace = &g
}
nc.IgnoreWarnings = c.IgnoreWarnings
return nc
}
// Merge values from other CheckRestart over default values on this
// CheckRestart and return merged copy.
func (c *CheckRestart) Merge(o *CheckRestart) *CheckRestart {
if c == nil {
// Just return other
return o
}
nc := c.Copy()
if o == nil {
// Nothing to merge
return nc
}
if o.Limit > 0 {
nc.Limit = o.Limit
}
if o.Grace != nil {
nc.Grace = o.Grace
}
if o.IgnoreWarnings {
nc.IgnoreWarnings = o.IgnoreWarnings
}
return nc
}
// The ServiceCheck data model represents the consul health check that
// Nomad registers for a Task
type ServiceCheck struct {
Id string
Name string
Type string
Command string
Args []string
Path string
Protocol string
PortLabel string `mapstructure:"port"`
AddressMode string `mapstructure:"address_mode"`
Interval time.Duration
Timeout time.Duration
InitialStatus string `mapstructure:"initial_status"`
TLSSkipVerify bool `mapstructure:"tls_skip_verify"`
Header map[string][]string
Method string
CheckRestart *CheckRestart `mapstructure:"check_restart"`
GRPCService string `mapstructure:"grpc_service"`
GRPCUseTLS bool `mapstructure:"grpc_use_tls"`
}
// The Service model represents a Consul service definition
type Service struct {
Id string
Name string
Tags []string
CanaryTags []string `mapstructure:"canary_tags"`
PortLabel string `mapstructure:"port"`
AddressMode string `mapstructure:"address_mode"`
Checks []ServiceCheck
CheckRestart *CheckRestart `mapstructure:"check_restart"`
}
func (s *Service) Canonicalize(t *Task, tg *TaskGroup, job *Job) {
if s.Name == "" {
s.Name = fmt.Sprintf("%s-%s-%s", *job.Name, *tg.Name, t.Name)
}
// Default to AddressModeAuto
if s.AddressMode == "" {
s.AddressMode = "auto"
}
// Canonicalize CheckRestart on Checks and merge Service.CheckRestart
// into each check.
for i, check := range s.Checks {
s.Checks[i].CheckRestart = s.CheckRestart.Merge(check.CheckRestart)
s.Checks[i].CheckRestart.Canonicalize()
}
}
// EphemeralDisk is an ephemeral disk object
type EphemeralDisk struct {
Sticky *bool
Migrate *bool
SizeMB *int `mapstructure:"size"`
}
func DefaultEphemeralDisk() *EphemeralDisk {
return &EphemeralDisk{
Sticky: helper.BoolToPtr(false),
Migrate: helper.BoolToPtr(false),
SizeMB: helper.IntToPtr(300),
}
}
func (e *EphemeralDisk) Canonicalize() {
if e.Sticky == nil {
e.Sticky = helper.BoolToPtr(false)
}
if e.Migrate == nil {
e.Migrate = helper.BoolToPtr(false)
}
if e.SizeMB == nil {
e.SizeMB = helper.IntToPtr(300)
}
}
// MigrateStrategy describes how allocations for a task group should be
// migrated between nodes (eg when draining).
type MigrateStrategy struct {
MaxParallel *int `mapstructure:"max_parallel"`
HealthCheck *string `mapstructure:"health_check"`
MinHealthyTime *time.Duration `mapstructure:"min_healthy_time"`
HealthyDeadline *time.Duration `mapstructure:"healthy_deadline"`
}
func DefaultMigrateStrategy() *MigrateStrategy {
return &MigrateStrategy{
MaxParallel: helper.IntToPtr(1),
HealthCheck: helper.StringToPtr("checks"),
MinHealthyTime: helper.TimeToPtr(10 * time.Second),
HealthyDeadline: helper.TimeToPtr(5 * time.Minute),
}
}
func (m *MigrateStrategy) Canonicalize() {
if m == nil {
return
}
defaults := DefaultMigrateStrategy()
if m.MaxParallel == nil {
m.MaxParallel = defaults.MaxParallel
}
if m.HealthCheck == nil {
m.HealthCheck = defaults.HealthCheck
}
if m.MinHealthyTime == nil {
m.MinHealthyTime = defaults.MinHealthyTime
}
if m.HealthyDeadline == nil {
m.HealthyDeadline = defaults.HealthyDeadline
}
}
func (m *MigrateStrategy) Merge(o *MigrateStrategy) {
if o.MaxParallel != nil {
m.MaxParallel = o.MaxParallel
}
if o.HealthCheck != nil {
m.HealthCheck = o.HealthCheck
}
if o.MinHealthyTime != nil {
m.MinHealthyTime = o.MinHealthyTime
}
if o.HealthyDeadline != nil {
m.HealthyDeadline = o.HealthyDeadline
}
}
func (m *MigrateStrategy) Copy() *MigrateStrategy {
if m == nil {
return nil
}
nm := new(MigrateStrategy)
*nm = *m
return nm
}
// TaskGroup is the unit of scheduling.
type TaskGroup struct {
Name *string
Count *int
Constraints []*Constraint
Tasks []*Task
RestartPolicy *RestartPolicy
ReschedulePolicy *ReschedulePolicy
EphemeralDisk *EphemeralDisk
Update *UpdateStrategy
Migrate *MigrateStrategy
Meta map[string]string
}
// NewTaskGroup creates a new TaskGroup.
func NewTaskGroup(name string, count int) *TaskGroup {
return &TaskGroup{
Name: helper.StringToPtr(name),
Count: helper.IntToPtr(count),
}
}
func (g *TaskGroup) Canonicalize(job *Job) {
if g.Name == nil {
g.Name = helper.StringToPtr("")
}
if g.Count == nil {
g.Count = helper.IntToPtr(1)
}
for _, t := range g.Tasks {
t.Canonicalize(g, job)
}
if g.EphemeralDisk == nil {
g.EphemeralDisk = DefaultEphemeralDisk()
} else {
g.EphemeralDisk.Canonicalize()
}
// Merge the update policy from the job
if ju, tu := job.Update != nil, g.Update != nil; ju && tu {
// Merge the jobs and task groups definition of the update strategy
jc := job.Update.Copy()
jc.Merge(g.Update)
g.Update = jc
} else if ju && !job.Update.Empty() {
// Inherit the jobs as long as it is non-empty.
jc := job.Update.Copy()
g.Update = jc
}
if g.Update != nil {
g.Update.Canonicalize()
}
// Merge the reschedule policy from the job
if jr, tr := job.Reschedule != nil, g.ReschedulePolicy != nil; jr && tr {
jobReschedule := job.Reschedule.Copy()
jobReschedule.Merge(g.ReschedulePolicy)
g.ReschedulePolicy = jobReschedule
} else if jr {
jobReschedule := job.Reschedule.Copy()
g.ReschedulePolicy = jobReschedule
}
// Only use default reschedule policy for non system jobs
if g.ReschedulePolicy == nil && *job.Type != "system" {
g.ReschedulePolicy = NewDefaultReschedulePolicy(*job.Type)
}
if g.ReschedulePolicy != nil {
g.ReschedulePolicy.Canonicalize(*job.Type)
}
// Merge the migrate strategy from the job
if jm, tm := job.Migrate != nil, g.Migrate != nil; jm && tm {
jobMigrate := job.Migrate.Copy()
jobMigrate.Merge(g.Migrate)
g.Migrate = jobMigrate
} else if jm {
jobMigrate := job.Migrate.Copy()
g.Migrate = jobMigrate
}
// Merge with default reschedule policy
if *job.Type == "service" {
defaultMigrateStrategy := &MigrateStrategy{}
defaultMigrateStrategy.Canonicalize()
if g.Migrate != nil {
defaultMigrateStrategy.Merge(g.Migrate)
}
g.Migrate = defaultMigrateStrategy
}
var defaultRestartPolicy *RestartPolicy
switch *job.Type {
case "service", "system":
defaultRestartPolicy = &RestartPolicy{
Delay: helper.TimeToPtr(structs.DefaultServiceJobRestartPolicy.Delay),
Attempts: helper.IntToPtr(structs.DefaultServiceJobRestartPolicy.Attempts),
Interval: helper.TimeToPtr(structs.DefaultServiceJobRestartPolicy.Interval),
Mode: helper.StringToPtr(structs.DefaultServiceJobRestartPolicy.Mode),
}
default:
defaultRestartPolicy = &RestartPolicy{
Delay: helper.TimeToPtr(structs.DefaultBatchJobRestartPolicy.Delay),
Attempts: helper.IntToPtr(structs.DefaultBatchJobRestartPolicy.Attempts),
Interval: helper.TimeToPtr(structs.DefaultBatchJobRestartPolicy.Interval),
Mode: helper.StringToPtr(structs.DefaultBatchJobRestartPolicy.Mode),
}
}
if g.RestartPolicy != nil {
defaultRestartPolicy.Merge(g.RestartPolicy)
}
g.RestartPolicy = defaultRestartPolicy
}
// Constrain is used to add a constraint to a task group.
func (g *TaskGroup) Constrain(c *Constraint) *TaskGroup {
g.Constraints = append(g.Constraints, c)
return g
}
// AddMeta is used to add a meta k/v pair to a task group
func (g *TaskGroup) SetMeta(key, val string) *TaskGroup {
if g.Meta == nil {
g.Meta = make(map[string]string)
}
g.Meta[key] = val
return g
}
// AddTask is used to add a new task to a task group.
func (g *TaskGroup) AddTask(t *Task) *TaskGroup {
g.Tasks = append(g.Tasks, t)
return g
}
// RequireDisk adds a ephemeral disk to the task group
func (g *TaskGroup) RequireDisk(disk *EphemeralDisk) *TaskGroup {
g.EphemeralDisk = disk
return g
}
// LogConfig provides configuration for log rotation
type LogConfig struct {
MaxFiles *int `mapstructure:"max_files"`
MaxFileSizeMB *int `mapstructure:"max_file_size"`
}
func DefaultLogConfig() *LogConfig {
return &LogConfig{
MaxFiles: helper.IntToPtr(10),
MaxFileSizeMB: helper.IntToPtr(10),
}
}
func (l *LogConfig) Canonicalize() {
if l.MaxFiles == nil {
l.MaxFiles = helper.IntToPtr(10)
}
if l.MaxFileSizeMB == nil {
l.MaxFileSizeMB = helper.IntToPtr(10)
}
}
// DispatchPayloadConfig configures how a task gets its input from a job dispatch
type DispatchPayloadConfig struct {
File string
}
// Task is a single process in a task group.
type Task struct {
Name string
Driver string
User string
Config map[string]interface{}
Constraints []*Constraint
Env map[string]string
Services []*Service
Resources *Resources
Meta map[string]string
KillTimeout *time.Duration `mapstructure:"kill_timeout"`
LogConfig *LogConfig `mapstructure:"logs"`
Artifacts []*TaskArtifact
Vault *Vault
Templates []*Template
DispatchPayload *DispatchPayloadConfig
Leader bool
ShutdownDelay time.Duration `mapstructure:"shutdown_delay"`
KillSignal string `mapstructure:"kill_signal"`
}
func (t *Task) Canonicalize(tg *TaskGroup, job *Job) {
if t.Resources == nil {
t.Resources = &Resources{}
}
t.Resources.Canonicalize()
if t.KillTimeout == nil {
t.KillTimeout = helper.TimeToPtr(5 * time.Second)
}
if t.LogConfig == nil {
t.LogConfig = DefaultLogConfig()
} else {
t.LogConfig.Canonicalize()
}
for _, artifact := range t.Artifacts {
artifact.Canonicalize()
}
if t.Vault != nil {
t.Vault.Canonicalize()
}
for _, tmpl := range t.Templates {
tmpl.Canonicalize()
}
for _, s := range t.Services {
s.Canonicalize(t, tg, job)
}
}
// TaskArtifact is used to download artifacts before running a task.
type TaskArtifact struct {
GetterSource *string `mapstructure:"source"`
GetterOptions map[string]string `mapstructure:"options"`
GetterMode *string `mapstructure:"mode"`
RelativeDest *string `mapstructure:"destination"`
}
func (a *TaskArtifact) Canonicalize() {
if a.GetterMode == nil {
a.GetterMode = helper.StringToPtr("any")
}
if a.GetterSource == nil {
// Shouldn't be possible, but we don't want to panic
a.GetterSource = helper.StringToPtr("")
}
if a.RelativeDest == nil {
switch *a.GetterMode {
case "file":
// File mode should default to local/filename
dest := *a.GetterSource
dest = path.Base(dest)
dest = filepath.Join("local", dest)
a.RelativeDest = &dest
default:
// Default to a directory
a.RelativeDest = helper.StringToPtr("local/")
}
}
}
type Template struct {
SourcePath *string `mapstructure:"source"`
DestPath *string `mapstructure:"destination"`
EmbeddedTmpl *string `mapstructure:"data"`
ChangeMode *string `mapstructure:"change_mode"`
ChangeSignal *string `mapstructure:"change_signal"`
Splay *time.Duration `mapstructure:"splay"`
Perms *string `mapstructure:"perms"`
LeftDelim *string `mapstructure:"left_delimiter"`
RightDelim *string `mapstructure:"right_delimiter"`
Envvars *bool `mapstructure:"env"`
VaultGrace *time.Duration `mapstructure:"vault_grace"`
}
func (tmpl *Template) Canonicalize() {
if tmpl.SourcePath == nil {
tmpl.SourcePath = helper.StringToPtr("")
}
if tmpl.DestPath == nil {
tmpl.DestPath = helper.StringToPtr("")
}
if tmpl.EmbeddedTmpl == nil {
tmpl.EmbeddedTmpl = helper.StringToPtr("")
}
if tmpl.ChangeMode == nil {
tmpl.ChangeMode = helper.StringToPtr("restart")
}
if tmpl.ChangeSignal == nil {
if *tmpl.ChangeMode == "signal" {
tmpl.ChangeSignal = helper.StringToPtr("SIGHUP")
} else {
tmpl.ChangeSignal = helper.StringToPtr("")
}
} else {
sig := *tmpl.ChangeSignal
tmpl.ChangeSignal = helper.StringToPtr(strings.ToUpper(sig))
}
if tmpl.Splay == nil {
tmpl.Splay = helper.TimeToPtr(5 * time.Second)
}
if tmpl.Perms == nil {
tmpl.Perms = helper.StringToPtr("0644")
}
if tmpl.LeftDelim == nil {
tmpl.LeftDelim = helper.StringToPtr("{{")
}
if tmpl.RightDelim == nil {
tmpl.RightDelim = helper.StringToPtr("}}")
}
if tmpl.Envvars == nil {
tmpl.Envvars = helper.BoolToPtr(false)
}
if tmpl.VaultGrace == nil {
tmpl.VaultGrace = helper.TimeToPtr(15 * time.Second)
}
}
type Vault struct {
Policies []string
Env *bool
ChangeMode *string `mapstructure:"change_mode"`
ChangeSignal *string `mapstructure:"change_signal"`
}
func (v *Vault) Canonicalize() {
if v.Env == nil {
v.Env = helper.BoolToPtr(true)
}
if v.ChangeMode == nil {
v.ChangeMode = helper.StringToPtr("restart")
}
if v.ChangeSignal == nil {
v.ChangeSignal = helper.StringToPtr("SIGHUP")
}
}
// NewTask creates and initializes a new Task.
func NewTask(name, driver string) *Task {
return &Task{
Name: name,
Driver: driver,
}
}
// Configure is used to configure a single k/v pair on
// the task.
func (t *Task) SetConfig(key string, val interface{}) *Task {
if t.Config == nil {
t.Config = make(map[string]interface{})
}
t.Config[key] = val
return t
}
// SetMeta is used to add metadata k/v pairs to the task.
func (t *Task) SetMeta(key, val string) *Task {
if t.Meta == nil {
t.Meta = make(map[string]string)
}
t.Meta[key] = val
return t
}
// Require is used to add resource requirements to a task.
func (t *Task) Require(r *Resources) *Task {
t.Resources = r
return t
}
// Constraint adds a new constraints to a single task.
func (t *Task) Constrain(c *Constraint) *Task {
t.Constraints = append(t.Constraints, c)
return t
}
// SetLogConfig sets a log config to a task
func (t *Task) SetLogConfig(l *LogConfig) *Task {
t.LogConfig = l
return t
}
// TaskState tracks the current state of a task and events that caused state
// transitions.
type TaskState struct {
State string
Failed bool
Restarts uint64
LastRestart time.Time
StartedAt time.Time
FinishedAt time.Time
Events []*TaskEvent
}
const (
TaskSetup = "Task Setup"
TaskSetupFailure = "Setup Failure"
TaskDriverFailure = "Driver Failure"
TaskDriverMessage = "Driver"
TaskReceived = "Received"
TaskFailedValidation = "Failed Validation"
TaskStarted = "Started"
TaskTerminated = "Terminated"
TaskKilling = "Killing"
TaskKilled = "Killed"
TaskRestarting = "Restarting"
TaskNotRestarting = "Not Restarting"
TaskDownloadingArtifacts = "Downloading Artifacts"
TaskArtifactDownloadFailed = "Failed Artifact Download"
TaskSiblingFailed = "Sibling Task Failed"
TaskSignaling = "Signaling"
TaskRestartSignal = "Restart Signaled"
TaskLeaderDead = "Leader Task Dead"
TaskBuildingTaskDir = "Building Task Directory"
)
// TaskEvent is an event that effects the state of a task and contains meta-data
// appropriate to the events type.
type TaskEvent struct {
Type string
Time int64
DisplayMessage string
Details map[string]string
// DEPRECATION NOTICE: The following fields are all deprecated. see TaskEvent struct in structs.go for details.
FailsTask bool
RestartReason string
SetupError string
DriverError string
DriverMessage string
ExitCode int
Signal int
Message string
KillReason string
KillTimeout time.Duration
KillError string
StartDelay int64
DownloadError string
ValidationError string
DiskLimit int64
DiskSize int64
FailedSibling string
VaultError string
TaskSignalReason string
TaskSignal string
GenericSource string
}

28
vendor/github.com/hashicorp/nomad/helper/args/args.go generated vendored Normal file
View File

@@ -0,0 +1,28 @@
package args
import "regexp"
var (
envRe = regexp.MustCompile(`\${[a-zA-Z0-9_\-\.]+}`)
)
// ReplaceEnv takes an arg and replaces all occurrences of environment variables.
// If the variable is found in the passed map it is replaced, otherwise the
// original string is returned.
func ReplaceEnv(arg string, environments ...map[string]string) string {
return envRe.ReplaceAllStringFunc(arg, func(arg string) string {
stripped := arg[2 : len(arg)-1]
for _, env := range environments {
if value, ok := env[stripped]; ok {
return value
}
}
return arg
})
}
// ReplaceEnvWithPlaceHolder replaces all occurrences of environment variables with the placeholder string.
func ReplaceEnvWithPlaceHolder(arg string, placeholder string) string {
return envRe.ReplaceAllString(arg, placeholder)
}

View File

@@ -0,0 +1,66 @@
package discover
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
)
// Checks the current executable, then $GOPATH/bin, and finally the CWD, in that
// order. If it can't be found, an error is returned.
func NomadExecutable() (string, error) {
nomadExe := "nomad"
if runtime.GOOS == "windows" {
nomadExe = "nomad.exe"
}
// Check the current executable.
bin, err := os.Executable()
if err != nil {
return "", fmt.Errorf("Failed to determine the nomad executable: %v", err)
}
if _, err := os.Stat(bin); err == nil && isNomad(bin, nomadExe) {
return bin, nil
}
// Check the $PATH
if bin, err := exec.LookPath(nomadExe); err == nil {
return bin, nil
}
// Check the $GOPATH.
bin = filepath.Join(os.Getenv("GOPATH"), "bin", nomadExe)
if _, err := os.Stat(bin); err == nil {
return bin, nil
}
// Check the CWD.
pwd, err := os.Getwd()
if err != nil {
return "", fmt.Errorf("Could not find Nomad executable (%v): %v", nomadExe, err)
}
bin = filepath.Join(pwd, nomadExe)
if _, err := os.Stat(bin); err == nil {
return bin, nil
}
// Check CWD/bin
bin = filepath.Join(pwd, "bin", nomadExe)
if _, err := os.Stat(bin); err == nil {
return bin, nil
}
return "", fmt.Errorf("Could not find Nomad executable (%v)", nomadExe)
}
func isNomad(path, nomadExe string) bool {
if strings.HasSuffix(path, ".test") || strings.HasSuffix(path, ".test.exe") {
return false
}
return true
}

View File

@@ -0,0 +1,129 @@
package flatmap
import (
"fmt"
"reflect"
)
// Flatten takes an object and returns a flat map of the object. The keys of the
// map is the path of the field names until a primitive field is reached and the
// value is a string representation of the terminal field.
func Flatten(obj interface{}, filter []string, primitiveOnly bool) map[string]string {
flat := make(map[string]string)
v := reflect.ValueOf(obj)
if !v.IsValid() {
return nil
}
flatten("", v, primitiveOnly, false, flat)
for _, f := range filter {
if _, ok := flat[f]; ok {
delete(flat, f)
}
}
return flat
}
// flatten recursively calls itself to create a flatmap representation of the
// passed value. The results are stored into the output map and the keys are
// the fields prepended with the passed prefix.
// XXX: A current restriction is that maps only support string keys.
func flatten(prefix string, v reflect.Value, primitiveOnly, enteredStruct bool, output map[string]string) {
switch v.Kind() {
case reflect.Bool:
output[prefix] = fmt.Sprintf("%v", v.Bool())
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
output[prefix] = fmt.Sprintf("%v", v.Int())
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
output[prefix] = fmt.Sprintf("%v", v.Uint())
case reflect.Float32, reflect.Float64:
output[prefix] = fmt.Sprintf("%v", v.Float())
case reflect.Complex64, reflect.Complex128:
output[prefix] = fmt.Sprintf("%v", v.Complex())
case reflect.String:
output[prefix] = fmt.Sprintf("%v", v.String())
case reflect.Invalid:
output[prefix] = "nil"
case reflect.Ptr:
if primitiveOnly && enteredStruct {
return
}
e := v.Elem()
if !e.IsValid() {
output[prefix] = "nil"
}
flatten(prefix, e, primitiveOnly, enteredStruct, output)
case reflect.Map:
for _, k := range v.MapKeys() {
if k.Kind() == reflect.Interface {
k = k.Elem()
}
if k.Kind() != reflect.String {
panic(fmt.Sprintf("%q: map key is not string: %s", prefix, k))
}
flatten(getSubKeyPrefix(prefix, k.String()), v.MapIndex(k), primitiveOnly, enteredStruct, output)
}
case reflect.Struct:
if primitiveOnly && enteredStruct {
return
}
enteredStruct = true
t := v.Type()
for i := 0; i < v.NumField(); i++ {
name := t.Field(i).Name
val := v.Field(i)
if val.Kind() == reflect.Interface && !val.IsNil() {
val = val.Elem()
}
flatten(getSubPrefix(prefix, name), val, primitiveOnly, enteredStruct, output)
}
case reflect.Interface:
if primitiveOnly {
return
}
e := v.Elem()
if !e.IsValid() {
output[prefix] = "nil"
return
}
flatten(prefix, e, primitiveOnly, enteredStruct, output)
case reflect.Array, reflect.Slice:
if primitiveOnly {
return
}
if v.Kind() == reflect.Slice && v.IsNil() {
output[prefix] = "nil"
return
}
for i := 0; i < v.Len(); i++ {
flatten(fmt.Sprintf("%s[%d]", prefix, i), v.Index(i), primitiveOnly, enteredStruct, output)
}
default:
panic(fmt.Sprintf("prefix %q; unsupported type %v", prefix, v.Kind()))
}
}
// getSubPrefix takes the current prefix and the next subfield and returns an
// appropriate prefix.
func getSubPrefix(curPrefix, subField string) string {
if curPrefix != "" {
return fmt.Sprintf("%s.%s", curPrefix, subField)
}
return fmt.Sprintf("%s", subField)
}
// getSubKeyPrefix takes the current prefix and the next subfield and returns an
// appropriate prefix for a map field.
func getSubKeyPrefix(curPrefix, subField string) string {
if curPrefix != "" {
return fmt.Sprintf("%s[%s]", curPrefix, subField)
}
return fmt.Sprintf("%s", subField)
}

304
vendor/github.com/hashicorp/nomad/helper/funcs.go generated vendored Normal file
View File

@@ -0,0 +1,304 @@
package helper
import (
"crypto/sha512"
"fmt"
"regexp"
"time"
multierror "github.com/hashicorp/go-multierror"
"github.com/hashicorp/hcl/hcl/ast"
)
// validUUID is used to check if a given string looks like a UUID
var validUUID = regexp.MustCompile(`(?i)^[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12}$`)
// IsUUID returns true if the given string is a valid UUID.
func IsUUID(str string) bool {
const uuidLen = 36
if len(str) != uuidLen {
return false
}
return validUUID.MatchString(str)
}
// HashUUID takes an input UUID and returns a hashed version of the UUID to
// ensure it is well distributed.
func HashUUID(input string) (output string, hashed bool) {
if !IsUUID(input) {
return "", false
}
// Hash the input
buf := sha512.Sum512([]byte(input))
output = fmt.Sprintf("%08x-%04x-%04x-%04x-%12x",
buf[0:4],
buf[4:6],
buf[6:8],
buf[8:10],
buf[10:16])
return output, true
}
// boolToPtr returns the pointer to a boolean
func BoolToPtr(b bool) *bool {
return &b
}
// IntToPtr returns the pointer to an int
func IntToPtr(i int) *int {
return &i
}
// Int64ToPtr returns the pointer to an int
func Int64ToPtr(i int64) *int64 {
return &i
}
// UintToPtr returns the pointer to an uint
func Uint64ToPtr(u uint64) *uint64 {
return &u
}
// StringToPtr returns the pointer to a string
func StringToPtr(str string) *string {
return &str
}
// TimeToPtr returns the pointer to a time stamp
func TimeToPtr(t time.Duration) *time.Duration {
return &t
}
func IntMin(a, b int) int {
if a < b {
return a
}
return b
}
func IntMax(a, b int) int {
if a > b {
return a
}
return b
}
func Uint64Max(a, b uint64) uint64 {
if a > b {
return a
}
return b
}
// MapStringStringSliceValueSet returns the set of values in a map[string][]string
func MapStringStringSliceValueSet(m map[string][]string) []string {
set := make(map[string]struct{})
for _, slice := range m {
for _, v := range slice {
set[v] = struct{}{}
}
}
flat := make([]string, 0, len(set))
for k := range set {
flat = append(flat, k)
}
return flat
}
func SliceStringToSet(s []string) map[string]struct{} {
m := make(map[string]struct{}, (len(s)+1)/2)
for _, k := range s {
m[k] = struct{}{}
}
return m
}
// SliceStringIsSubset returns whether the smaller set of strings is a subset of
// the larger. If the smaller slice is not a subset, the offending elements are
// returned.
func SliceStringIsSubset(larger, smaller []string) (bool, []string) {
largerSet := make(map[string]struct{}, len(larger))
for _, l := range larger {
largerSet[l] = struct{}{}
}
subset := true
var offending []string
for _, s := range smaller {
if _, ok := largerSet[s]; !ok {
subset = false
offending = append(offending, s)
}
}
return subset, offending
}
func SliceSetDisjoint(first, second []string) (bool, []string) {
contained := make(map[string]struct{}, len(first))
for _, k := range first {
contained[k] = struct{}{}
}
offending := make(map[string]struct{})
for _, k := range second {
if _, ok := contained[k]; ok {
offending[k] = struct{}{}
}
}
if len(offending) == 0 {
return true, nil
}
flattened := make([]string, 0, len(offending))
for k := range offending {
flattened = append(flattened, k)
}
return false, flattened
}
// Helpers for copying generic structures.
func CopyMapStringString(m map[string]string) map[string]string {
l := len(m)
if l == 0 {
return nil
}
c := make(map[string]string, l)
for k, v := range m {
c[k] = v
}
return c
}
func CopyMapStringStruct(m map[string]struct{}) map[string]struct{} {
l := len(m)
if l == 0 {
return nil
}
c := make(map[string]struct{}, l)
for k := range m {
c[k] = struct{}{}
}
return c
}
func CopyMapStringInt(m map[string]int) map[string]int {
l := len(m)
if l == 0 {
return nil
}
c := make(map[string]int, l)
for k, v := range m {
c[k] = v
}
return c
}
func CopyMapStringFloat64(m map[string]float64) map[string]float64 {
l := len(m)
if l == 0 {
return nil
}
c := make(map[string]float64, l)
for k, v := range m {
c[k] = v
}
return c
}
// CopyMapStringSliceString copies a map of strings to string slices such as
// http.Header
func CopyMapStringSliceString(m map[string][]string) map[string][]string {
l := len(m)
if l == 0 {
return nil
}
c := make(map[string][]string, l)
for k, v := range m {
c[k] = CopySliceString(v)
}
return c
}
func CopySliceString(s []string) []string {
l := len(s)
if l == 0 {
return nil
}
c := make([]string, l)
for i, v := range s {
c[i] = v
}
return c
}
func CopySliceInt(s []int) []int {
l := len(s)
if l == 0 {
return nil
}
c := make([]int, l)
for i, v := range s {
c[i] = v
}
return c
}
// CleanEnvVar replaces all occurrences of illegal characters in an environment
// variable with the specified byte.
func CleanEnvVar(s string, r byte) string {
b := []byte(s)
for i, c := range b {
switch {
case c == '_':
case c == '.':
case c >= 'a' && c <= 'z':
case c >= 'A' && c <= 'Z':
case i > 0 && c >= '0' && c <= '9':
default:
// Replace!
b[i] = r
}
}
return string(b)
}
func CheckHCLKeys(node ast.Node, valid []string) error {
var list *ast.ObjectList
switch n := node.(type) {
case *ast.ObjectList:
list = n
case *ast.ObjectType:
list = n.List
default:
return fmt.Errorf("cannot check HCL keys of type %T", n)
}
validMap := make(map[string]struct{}, len(valid))
for _, v := range valid {
validMap[v] = struct{}{}
}
var result error
for _, item := range list.Items {
key := item.Keys[0].Token.Value().(string)
if _, ok := validMap[key]; !ok {
result = multierror.Append(result, fmt.Errorf(
"invalid key: %s", key))
}
}
return result
}

View File

@@ -0,0 +1,58 @@
// Package testlog creates a *log.Logger backed by *testing.T to ease logging
// in tests. This allows logs from components being tested to only be printed
// if the test fails (or the verbose flag is specified).
package testlog
import (
"io"
"log"
"os"
)
// UseStdout returns true if NOMAD_TEST_STDOUT=1 and sends logs to stdout.
func UseStdout() bool {
return os.Getenv("NOMAD_TEST_STDOUT") == "1"
}
// LogPrinter is the methods of testing.T (or testing.B) needed by the test
// logger.
type LogPrinter interface {
Logf(format string, args ...interface{})
}
// writer implements io.Writer on top of a Logger.
type writer struct {
t LogPrinter
}
// Write to an underlying Logger. Never returns an error.
func (w *writer) Write(p []byte) (n int, err error) {
w.t.Logf(string(p))
return len(p), nil
}
// NewWriter creates a new io.Writer backed by a Logger.
func NewWriter(t LogPrinter) io.Writer {
if UseStdout() {
return os.Stdout
}
return &writer{t}
}
// New returns a new test logger. See https://golang.org/pkg/log/#New
func New(t LogPrinter, prefix string, flag int) *log.Logger {
if UseStdout() {
return log.New(os.Stdout, prefix, flag)
}
return log.New(&writer{t}, prefix, flag)
}
// WithPrefix returns a new test logger with the Lmicroseconds flag set.
func WithPrefix(t LogPrinter, prefix string) *log.Logger {
return New(t, prefix, log.Lmicroseconds)
}
// NewLog logger with "TEST" prefix and the Lmicroseconds flag.
func Logger(t LogPrinter) *log.Logger {
return WithPrefix(t, "")
}

21
vendor/github.com/hashicorp/nomad/helper/uuid/uuid.go generated vendored Normal file
View File

@@ -0,0 +1,21 @@
package uuid
import (
crand "crypto/rand"
"fmt"
)
// Generate is used to generate a random UUID
func Generate() string {
buf := make([]byte, 16)
if _, err := crand.Read(buf); err != nil {
panic(fmt.Errorf("failed to read random bytes: %v", err))
}
return fmt.Sprintf("%08x-%04x-%04x-%04x-%12x",
buf[0:4],
buf[4:6],
buf[6:8],
buf[8:10],
buf[10:16])
}

View File

@@ -0,0 +1,43 @@
package structs
// BatchFuture is used to wait on a batch update to complete
type BatchFuture struct {
doneCh chan struct{}
err error
index uint64
}
// NewBatchFuture creates a new batch future
func NewBatchFuture() *BatchFuture {
return &BatchFuture{
doneCh: make(chan struct{}),
}
}
// Wait is used to block for the future to complete and returns the error
func (b *BatchFuture) Wait() error {
<-b.doneCh
return b.err
}
// WaitCh is used to block for the future to complete
func (b *BatchFuture) WaitCh() <-chan struct{} {
return b.doneCh
}
// Error is used to return the error of the batch, only after Wait()
func (b *BatchFuture) Error() error {
return b.err
}
// Index is used to return the index of the batch, only after Wait()
func (b *BatchFuture) Index() uint64 {
return b.index
}
// Respond is used to unblock the future
func (b *BatchFuture) Respond(index uint64, err error) {
b.index = index
b.err = err
close(b.doneCh)
}

View File

@@ -0,0 +1,78 @@
package structs
import "fmt"
// Bitmap is a simple uncompressed bitmap
type Bitmap []byte
// NewBitmap returns a bitmap with up to size indexes
func NewBitmap(size uint) (Bitmap, error) {
if size == 0 {
return nil, fmt.Errorf("bitmap must be positive size")
}
if size&7 != 0 {
return nil, fmt.Errorf("bitmap must be byte aligned")
}
b := make([]byte, size>>3)
return Bitmap(b), nil
}
// Copy returns a copy of the Bitmap
func (b Bitmap) Copy() (Bitmap, error) {
if b == nil {
return nil, fmt.Errorf("can't copy nil Bitmap")
}
raw := make([]byte, len(b))
copy(raw, b)
return Bitmap(raw), nil
}
// Size returns the size of the bitmap
func (b Bitmap) Size() uint {
return uint(len(b) << 3)
}
// Set is used to set the given index of the bitmap
func (b Bitmap) Set(idx uint) {
bucket := idx >> 3
mask := byte(1 << (idx & 7))
b[bucket] |= mask
}
// Unset is used to unset the given index of the bitmap
func (b Bitmap) Unset(idx uint) {
bucket := idx >> 3
// Mask should be all ones minus the idx position
offset := 1 << (idx & 7)
mask := byte(offset ^ 0xff)
b[bucket] &= mask
}
// Check is used to check the given index of the bitmap
func (b Bitmap) Check(idx uint) bool {
bucket := idx >> 3
mask := byte(1 << (idx & 7))
return (b[bucket] & mask) != 0
}
// Clear is used to efficiently clear the bitmap
func (b Bitmap) Clear() {
for i := range b {
b[i] = 0
}
}
// IndexesInRange returns the indexes in which the values are either set or unset based
// on the passed parameter in the passed range
func (b Bitmap) IndexesInRange(set bool, from, to uint) []int {
var indexes []int
for i := from; i <= to && i < b.Size(); i++ {
c := b.Check(i)
if c && set || !c && !set {
indexes = append(indexes, int(i))
}
}
return indexes
}

View File

@@ -0,0 +1,102 @@
package config
import (
"time"
"github.com/hashicorp/nomad/helper"
)
type AutopilotConfig struct {
// CleanupDeadServers controls whether to remove dead servers when a new
// server is added to the Raft peers.
CleanupDeadServers *bool `mapstructure:"cleanup_dead_servers"`
// ServerStabilizationTime is the minimum amount of time a server must be
// in a stable, healthy state before it can be added to the cluster. Only
// applicable with Raft protocol version 3 or higher.
ServerStabilizationTime time.Duration `mapstructure:"server_stabilization_time"`
// LastContactThreshold is the limit on the amount of time a server can go
// without leader contact before being considered unhealthy.
LastContactThreshold time.Duration `mapstructure:"last_contact_threshold"`
// MaxTrailingLogs is the amount of entries in the Raft Log that a server can
// be behind before being considered unhealthy.
MaxTrailingLogs int `mapstructure:"max_trailing_logs"`
// (Enterprise-only) EnableRedundancyZones specifies whether to enable redundancy zones.
EnableRedundancyZones *bool `mapstructure:"enable_redundancy_zones"`
// (Enterprise-only) DisableUpgradeMigration will disable Autopilot's upgrade migration
// strategy of waiting until enough newer-versioned servers have been added to the
// cluster before promoting them to voters.
DisableUpgradeMigration *bool `mapstructure:"disable_upgrade_migration"`
// (Enterprise-only) EnableCustomUpgrades specifies whether to enable using custom
// upgrade versions when performing migrations.
EnableCustomUpgrades *bool `mapstructure:"enable_custom_upgrades"`
}
// DefaultAutopilotConfig() returns the canonical defaults for the Nomad
// `autopilot` configuration.
func DefaultAutopilotConfig() *AutopilotConfig {
return &AutopilotConfig{
LastContactThreshold: 200 * time.Millisecond,
MaxTrailingLogs: 250,
ServerStabilizationTime: 10 * time.Second,
}
}
func (a *AutopilotConfig) Merge(b *AutopilotConfig) *AutopilotConfig {
result := a.Copy()
if b.CleanupDeadServers != nil {
result.CleanupDeadServers = helper.BoolToPtr(*b.CleanupDeadServers)
}
if b.ServerStabilizationTime != 0 {
result.ServerStabilizationTime = b.ServerStabilizationTime
}
if b.LastContactThreshold != 0 {
result.LastContactThreshold = b.LastContactThreshold
}
if b.MaxTrailingLogs != 0 {
result.MaxTrailingLogs = b.MaxTrailingLogs
}
if b.EnableRedundancyZones != nil {
result.EnableRedundancyZones = b.EnableRedundancyZones
}
if b.DisableUpgradeMigration != nil {
result.DisableUpgradeMigration = helper.BoolToPtr(*b.DisableUpgradeMigration)
}
if b.EnableCustomUpgrades != nil {
result.EnableCustomUpgrades = b.EnableCustomUpgrades
}
return result
}
// Copy returns a copy of this Autopilot config.
func (a *AutopilotConfig) Copy() *AutopilotConfig {
if a == nil {
return nil
}
nc := new(AutopilotConfig)
*nc = *a
// Copy the bools
if a.CleanupDeadServers != nil {
nc.CleanupDeadServers = helper.BoolToPtr(*a.CleanupDeadServers)
}
if a.EnableRedundancyZones != nil {
nc.EnableRedundancyZones = helper.BoolToPtr(*a.EnableRedundancyZones)
}
if a.DisableUpgradeMigration != nil {
nc.DisableUpgradeMigration = helper.BoolToPtr(*a.DisableUpgradeMigration)
}
if a.EnableCustomUpgrades != nil {
nc.EnableCustomUpgrades = helper.BoolToPtr(*a.EnableCustomUpgrades)
}
return nc
}

View File

@@ -0,0 +1,263 @@
package config
import (
"net/http"
"strings"
"time"
consul "github.com/hashicorp/consul/api"
"github.com/hashicorp/nomad/helper"
)
// ConsulConfig contains the configuration information necessary to
// communicate with a Consul Agent in order to:
//
// - Register services and their checks with Consul
//
// - Bootstrap this Nomad Client with the list of Nomad Servers registered
// with Consul
//
// Both the Agent and the executor need to be able to import ConsulConfig.
type ConsulConfig struct {
// ServerServiceName is the name of the service that Nomad uses to register
// servers with Consul
ServerServiceName string `mapstructure:"server_service_name"`
// ServerHTTPCheckName is the name of the health check that Nomad uses
// to register the server HTTP health check with Consul
ServerHTTPCheckName string `mapstructure:"server_http_check_name"`
// ServerSerfCheckName is the name of the health check that Nomad uses
// to register the server Serf health check with Consul
ServerSerfCheckName string `mapstructure:"server_serf_check_name"`
// ServerRPCCheckName is the name of the health check that Nomad uses
// to register the server RPC health check with Consul
ServerRPCCheckName string `mapstructure:"server_rpc_check_name"`
// ClientServiceName is the name of the service that Nomad uses to register
// clients with Consul
ClientServiceName string `mapstructure:"client_service_name"`
// ClientHTTPCheckName is the name of the health check that Nomad uses
// to register the client HTTP health check with Consul
ClientHTTPCheckName string `mapstructure:"client_http_check_name"`
// AutoAdvertise determines if this Nomad Agent will advertise its
// services via Consul. When true, Nomad Agent will register
// services with Consul.
AutoAdvertise *bool `mapstructure:"auto_advertise"`
// ChecksUseAdvertise specifies that Consul checks should use advertise
// address instead of bind address
ChecksUseAdvertise *bool `mapstructure:"checks_use_advertise"`
// Addr is the address of the local Consul agent
Addr string `mapstructure:"address"`
// Timeout is used by Consul HTTP Client
Timeout time.Duration `mapstructure:"timeout"`
// Token is used to provide a per-request ACL token. This options overrides
// the agent's default token
Token string `mapstructure:"token"`
// Auth is the information to use for http access to Consul agent
Auth string `mapstructure:"auth"`
// EnableSSL sets the transport scheme to talk to the Consul agent as https
EnableSSL *bool `mapstructure:"ssl"`
// VerifySSL enables or disables SSL verification when the transport scheme
// for the consul api client is https
VerifySSL *bool `mapstructure:"verify_ssl"`
// CAFile is the path to the ca certificate used for Consul communication
CAFile string `mapstructure:"ca_file"`
// CertFile is the path to the certificate for Consul communication
CertFile string `mapstructure:"cert_file"`
// KeyFile is the path to the private key for Consul communication
KeyFile string `mapstructure:"key_file"`
// ServerAutoJoin enables Nomad servers to find peers by querying Consul and
// joining them
ServerAutoJoin *bool `mapstructure:"server_auto_join"`
// ClientAutoJoin enables Nomad servers to find addresses of Nomad servers
// and register with them
ClientAutoJoin *bool `mapstructure:"client_auto_join"`
}
// DefaultConsulConfig() returns the canonical defaults for the Nomad
// `consul` configuration.
func DefaultConsulConfig() *ConsulConfig {
return &ConsulConfig{
ServerServiceName: "nomad",
ServerHTTPCheckName: "Nomad Server HTTP Check",
ServerSerfCheckName: "Nomad Server Serf Check",
ServerRPCCheckName: "Nomad Server RPC Check",
ClientServiceName: "nomad-client",
ClientHTTPCheckName: "Nomad Client HTTP Check",
AutoAdvertise: helper.BoolToPtr(true),
ChecksUseAdvertise: helper.BoolToPtr(false),
EnableSSL: helper.BoolToPtr(false),
VerifySSL: helper.BoolToPtr(true),
ServerAutoJoin: helper.BoolToPtr(true),
ClientAutoJoin: helper.BoolToPtr(true),
Timeout: 5 * time.Second,
}
}
// Merge merges two Consul Configurations together.
func (a *ConsulConfig) Merge(b *ConsulConfig) *ConsulConfig {
result := a.Copy()
if b.ServerServiceName != "" {
result.ServerServiceName = b.ServerServiceName
}
if b.ServerHTTPCheckName != "" {
result.ServerHTTPCheckName = b.ServerHTTPCheckName
}
if b.ServerSerfCheckName != "" {
result.ServerSerfCheckName = b.ServerSerfCheckName
}
if b.ServerRPCCheckName != "" {
result.ServerRPCCheckName = b.ServerRPCCheckName
}
if b.ClientServiceName != "" {
result.ClientServiceName = b.ClientServiceName
}
if b.ClientHTTPCheckName != "" {
result.ClientHTTPCheckName = b.ClientHTTPCheckName
}
if b.AutoAdvertise != nil {
result.AutoAdvertise = helper.BoolToPtr(*b.AutoAdvertise)
}
if b.Addr != "" {
result.Addr = b.Addr
}
if b.Timeout != 0 {
result.Timeout = b.Timeout
}
if b.Token != "" {
result.Token = b.Token
}
if b.Auth != "" {
result.Auth = b.Auth
}
if b.EnableSSL != nil {
result.EnableSSL = helper.BoolToPtr(*b.EnableSSL)
}
if b.VerifySSL != nil {
result.VerifySSL = helper.BoolToPtr(*b.VerifySSL)
}
if b.CAFile != "" {
result.CAFile = b.CAFile
}
if b.CertFile != "" {
result.CertFile = b.CertFile
}
if b.KeyFile != "" {
result.KeyFile = b.KeyFile
}
if b.ServerAutoJoin != nil {
result.ServerAutoJoin = helper.BoolToPtr(*b.ServerAutoJoin)
}
if b.ClientAutoJoin != nil {
result.ClientAutoJoin = helper.BoolToPtr(*b.ClientAutoJoin)
}
if b.ChecksUseAdvertise != nil {
result.ChecksUseAdvertise = helper.BoolToPtr(*b.ChecksUseAdvertise)
}
return result
}
// ApiConfig returns a usable Consul config that can be passed directly to
// hashicorp/consul/api. NOTE: datacenter is not set
func (c *ConsulConfig) ApiConfig() (*consul.Config, error) {
// Get the default config from consul to reuse things like the default
// http.Transport.
config := consul.DefaultConfig()
if c.Addr != "" {
config.Address = c.Addr
}
if c.Token != "" {
config.Token = c.Token
}
if c.Timeout != 0 {
// Create a custom Client to set the timeout
if config.HttpClient == nil {
config.HttpClient = &http.Client{}
}
config.HttpClient.Timeout = c.Timeout
config.HttpClient.Transport = config.Transport
}
if c.Auth != "" {
var username, password string
if strings.Contains(c.Auth, ":") {
split := strings.SplitN(c.Auth, ":", 2)
username = split[0]
password = split[1]
} else {
username = c.Auth
}
config.HttpAuth = &consul.HttpBasicAuth{
Username: username,
Password: password,
}
}
if c.EnableSSL != nil && *c.EnableSSL {
config.Scheme = "https"
config.TLSConfig = consul.TLSConfig{
Address: config.Address,
CAFile: c.CAFile,
CertFile: c.CertFile,
KeyFile: c.KeyFile,
}
if c.VerifySSL != nil {
config.TLSConfig.InsecureSkipVerify = !*c.VerifySSL
}
tlsConfig, err := consul.SetupTLSConfig(&config.TLSConfig)
if err != nil {
return nil, err
}
config.Transport.TLSClientConfig = tlsConfig
}
return config, nil
}
// Copy returns a copy of this Consul config.
func (c *ConsulConfig) Copy() *ConsulConfig {
if c == nil {
return nil
}
nc := new(ConsulConfig)
*nc = *c
// Copy the bools
if nc.AutoAdvertise != nil {
nc.AutoAdvertise = helper.BoolToPtr(*nc.AutoAdvertise)
}
if nc.ChecksUseAdvertise != nil {
nc.ChecksUseAdvertise = helper.BoolToPtr(*nc.ChecksUseAdvertise)
}
if nc.EnableSSL != nil {
nc.EnableSSL = helper.BoolToPtr(*nc.EnableSSL)
}
if nc.VerifySSL != nil {
nc.VerifySSL = helper.BoolToPtr(*nc.VerifySSL)
}
if nc.ServerAutoJoin != nil {
nc.ServerAutoJoin = helper.BoolToPtr(*nc.ServerAutoJoin)
}
if nc.ClientAutoJoin != nil {
nc.ClientAutoJoin = helper.BoolToPtr(*nc.ClientAutoJoin)
}
return nc
}

View File

@@ -0,0 +1,23 @@
package config
// SentinelConfig is configuration specific to Sentinel
type SentinelConfig struct {
// Imports are the configured imports
Imports []*SentinelImport `hcl:"import,expand"`
}
// SentinelImport is used per configured import
type SentinelImport struct {
Name string `hcl:",key"`
Path string `hcl:"path"`
Args []string `hcl:"args"`
}
// Merge is used to merge two Sentinel configs together. The settings from the input always take precedence.
func (a *SentinelConfig) Merge(b *SentinelConfig) *SentinelConfig {
result := *a
if len(b.Imports) > 0 {
result.Imports = append(result.Imports, b.Imports...)
}
return &result
}

View File

@@ -0,0 +1,301 @@
package config
import (
"crypto/md5"
"crypto/tls"
"encoding/hex"
"fmt"
"io"
"os"
"sync"
)
// TLSConfig provides TLS related configuration
type TLSConfig struct {
// EnableHTTP enabled TLS for http traffic to the Nomad server and clients
EnableHTTP bool `mapstructure:"http"`
// EnableRPC enables TLS for RPC and Raft traffic to the Nomad servers
EnableRPC bool `mapstructure:"rpc"`
// VerifyServerHostname is used to enable hostname verification of servers. This
// ensures that the certificate presented is valid for server.<region>.nomad
// This prevents a compromised client from being restarted as a server, and then
// intercepting request traffic as well as being added as a raft peer. This should be
// enabled by default with VerifyOutgoing, but for legacy reasons we cannot break
// existing clients.
VerifyServerHostname bool `mapstructure:"verify_server_hostname"`
// CAFile is a path to a certificate authority file. This is used with VerifyIncoming
// or VerifyOutgoing to verify the TLS connection.
CAFile string `mapstructure:"ca_file"`
// CertFile is used to provide a TLS certificate that is used for serving TLS connections.
// Must be provided to serve TLS connections.
CertFile string `mapstructure:"cert_file"`
// KeyLoader is a helper to dynamically reload TLS configuration
KeyLoader *KeyLoader
keyloaderLock sync.Mutex
// KeyFile is used to provide a TLS key that is used for serving TLS connections.
// Must be provided to serve TLS connections.
KeyFile string `mapstructure:"key_file"`
// RPCUpgradeMode should be enabled when a cluster is being upgraded
// to TLS. Allows servers to accept both plaintext and TLS connections and
// should only be a temporary state.
RPCUpgradeMode bool `mapstructure:"rpc_upgrade_mode"`
// Verify connections to the HTTPS API
VerifyHTTPSClient bool `mapstructure:"verify_https_client"`
// Checksum is a MD5 hash of the certificate CA File, Certificate file, and
// key file.
Checksum string
// TLSCipherSuites are operator-defined ciphers to be used in Nomad TLS
// connections
TLSCipherSuites string `mapstructure:"tls_cipher_suites"`
// TLSMinVersion is used to set the minimum TLS version used for TLS
// connections. Should be either "tls10", "tls11", or "tls12".
TLSMinVersion string `mapstructure:"tls_min_version"`
// TLSPreferServerCipherSuites controls whether the server selects the
// client's most preferred ciphersuite, or the server's most preferred
// ciphersuite. If true then the server's preference, as expressed in
// the order of elements in CipherSuites, is used.
TLSPreferServerCipherSuites bool `mapstructure:"tls_prefer_server_cipher_suites"`
}
type KeyLoader struct {
cacheLock sync.Mutex
certificate *tls.Certificate
}
// LoadKeyPair reloads the TLS certificate based on the specified certificate
// and key file. If successful, stores the certificate for further use.
func (k *KeyLoader) LoadKeyPair(certFile, keyFile string) (*tls.Certificate, error) {
k.cacheLock.Lock()
defer k.cacheLock.Unlock()
// Allow downgrading
if certFile == "" && keyFile == "" {
k.certificate = nil
return nil, nil
}
cert, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
return nil, fmt.Errorf("Failed to load cert/key pair: %v", err)
}
k.certificate = &cert
return k.certificate, nil
}
// GetOutgoingCertificate fetches the currently-loaded certificate when
// accepting a TLS connection. This currently does not consider information in
// the ClientHello and only returns the certificate that was last loaded.
func (k *KeyLoader) GetOutgoingCertificate(*tls.ClientHelloInfo) (*tls.Certificate, error) {
k.cacheLock.Lock()
defer k.cacheLock.Unlock()
return k.certificate, nil
}
// GetClientCertificate fetches the currently-loaded certificate when the Server
// requests a certificate from the caller. This currently does not consider
// information in the ClientHello and only returns the certificate that was last
// loaded.
func (k *KeyLoader) GetClientCertificate(*tls.CertificateRequestInfo) (*tls.Certificate, error) {
k.cacheLock.Lock()
defer k.cacheLock.Unlock()
return k.certificate, nil
}
func (k *KeyLoader) Copy() *KeyLoader {
if k == nil {
return nil
}
new := KeyLoader{}
new.certificate = k.certificate
return &new
}
// GetKeyLoader returns the keyloader for a TLSConfig object. If the keyloader
// has not been initialized, it will first do so.
func (t *TLSConfig) GetKeyLoader() *KeyLoader {
t.keyloaderLock.Lock()
defer t.keyloaderLock.Unlock()
// If the keyloader has not yet been initialized, do it here
if t.KeyLoader == nil {
t.KeyLoader = &KeyLoader{}
}
return t.KeyLoader
}
// Copy copies the fields of TLSConfig to another TLSConfig object. Required as
// to not copy mutexes between objects.
func (t *TLSConfig) Copy() *TLSConfig {
if t == nil {
return t
}
new := &TLSConfig{}
new.EnableHTTP = t.EnableHTTP
new.EnableRPC = t.EnableRPC
new.VerifyServerHostname = t.VerifyServerHostname
new.CAFile = t.CAFile
new.CertFile = t.CertFile
t.keyloaderLock.Lock()
new.KeyLoader = t.KeyLoader.Copy()
t.keyloaderLock.Unlock()
new.KeyFile = t.KeyFile
new.RPCUpgradeMode = t.RPCUpgradeMode
new.VerifyHTTPSClient = t.VerifyHTTPSClient
new.TLSCipherSuites = t.TLSCipherSuites
new.TLSMinVersion = t.TLSMinVersion
new.TLSPreferServerCipherSuites = t.TLSPreferServerCipherSuites
new.SetChecksum()
return new
}
func (t *TLSConfig) IsEmpty() bool {
if t == nil {
return true
}
return t.EnableHTTP == false &&
t.EnableRPC == false &&
t.VerifyServerHostname == false &&
t.CAFile == "" &&
t.CertFile == "" &&
t.KeyFile == "" &&
t.VerifyHTTPSClient == false
}
// Merge is used to merge two TLS configs together
func (t *TLSConfig) Merge(b *TLSConfig) *TLSConfig {
result := t.Copy()
if b.EnableHTTP {
result.EnableHTTP = true
}
if b.EnableRPC {
result.EnableRPC = true
}
if b.VerifyServerHostname {
result.VerifyServerHostname = true
}
if b.CAFile != "" {
result.CAFile = b.CAFile
}
if b.CertFile != "" {
result.CertFile = b.CertFile
}
if b.KeyFile != "" {
result.KeyFile = b.KeyFile
}
if b.VerifyHTTPSClient {
result.VerifyHTTPSClient = true
}
if b.RPCUpgradeMode {
result.RPCUpgradeMode = true
}
if b.TLSCipherSuites != "" {
result.TLSCipherSuites = b.TLSCipherSuites
}
if b.TLSMinVersion != "" {
result.TLSMinVersion = b.TLSMinVersion
}
if b.TLSPreferServerCipherSuites {
result.TLSPreferServerCipherSuites = true
}
return result
}
// CertificateInfoIsEqual compares the fields of two TLS configuration objects
// for the fields that are specific to configuring a TLS connection
// It is possible for either the calling TLSConfig to be nil, or the TLSConfig
// that it is being compared against, so we need to handle both places. See
// server.go Reload for example.
func (t *TLSConfig) CertificateInfoIsEqual(newConfig *TLSConfig) (bool, error) {
if t == nil || newConfig == nil {
return t == newConfig, nil
}
if t.IsEmpty() && newConfig.IsEmpty() {
return true, nil
} else if t.IsEmpty() || newConfig.IsEmpty() {
return false, nil
}
// Set the checksum if it hasn't yet been set (this should happen when the
// config is parsed but this provides safety in depth)
if newConfig.Checksum == "" {
err := newConfig.SetChecksum()
if err != nil {
return false, err
}
}
if t.Checksum == "" {
err := t.SetChecksum()
if err != nil {
return false, err
}
}
return t.Checksum == newConfig.Checksum, nil
}
// SetChecksum generates and sets the checksum for a TLS configuration
func (t *TLSConfig) SetChecksum() error {
newCertChecksum, err := createChecksumOfFiles(t.CAFile, t.CertFile, t.KeyFile)
if err != nil {
return err
}
t.Checksum = newCertChecksum
return nil
}
func getFileChecksum(filepath string) (string, error) {
f, err := os.Open(filepath)
if err != nil {
return "", err
}
defer f.Close()
h := md5.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
func createChecksumOfFiles(inputs ...string) (string, error) {
h := md5.New()
for _, input := range inputs {
checksum, err := getFileChecksum(input)
if err != nil {
return "", err
}
io.WriteString(h, checksum)
}
return hex.EncodeToString(h.Sum(nil)), nil
}

View File

@@ -0,0 +1,235 @@
package config
import (
"time"
vault "github.com/hashicorp/vault/api"
)
const (
// DefaultVaultConnectRetryIntv is the retry interval between trying to
// connect to Vault
DefaultVaultConnectRetryIntv = 30 * time.Second
)
// VaultConfig contains the configuration information necessary to
// communicate with Vault in order to:
//
// - Renew Vault tokens/leases.
//
// - Pass a token for the Nomad Server to derive sub-tokens.
//
// - Create child tokens with policy subsets of the Server's token.
type VaultConfig struct {
// Enabled enables or disables Vault support.
Enabled *bool `mapstructure:"enabled"`
// Token is the Vault token given to Nomad such that it can
// derive child tokens. Nomad will renew this token at half its lease
// lifetime.
Token string `mapstructure:"token"`
// Role sets the role in which to create tokens from. The Token given to
// Nomad does not have to be created from this role but must have "update"
// capability on "auth/token/create/<create_from_role>". If this value is
// unset and the token is created from a role, the value is defaulted to the
// role the token is from.
Role string `mapstructure:"create_from_role"`
// AllowUnauthenticated allows users to submit jobs requiring Vault tokens
// without providing a Vault token proving they have access to these
// policies.
AllowUnauthenticated *bool `mapstructure:"allow_unauthenticated"`
// TaskTokenTTL is the TTL of the tokens created by Nomad Servers and used
// by the client. There should be a minimum time value such that the client
// does not have to renew with Vault at a very high frequency
TaskTokenTTL string `mapstructure:"task_token_ttl"`
// Addr is the address of the local Vault agent. This should be a complete
// URL such as "http://vault.example.com"
Addr string `mapstructure:"address"`
// ConnectionRetryIntv is the interval to wait before re-attempting to
// connect to Vault.
ConnectionRetryIntv time.Duration
// TLSCaFile is the path to a PEM-encoded CA cert file to use to verify the
// Vault server SSL certificate.
TLSCaFile string `mapstructure:"ca_file"`
// TLSCaFile is the path to a directory of PEM-encoded CA cert files to
// verify the Vault server SSL certificate.
TLSCaPath string `mapstructure:"ca_path"`
// TLSCertFile is the path to the certificate for Vault communication
TLSCertFile string `mapstructure:"cert_file"`
// TLSKeyFile is the path to the private key for Vault communication
TLSKeyFile string `mapstructure:"key_file"`
// TLSSkipVerify enables or disables SSL verification
TLSSkipVerify *bool `mapstructure:"tls_skip_verify"`
// TLSServerName, if set, is used to set the SNI host when connecting via TLS.
TLSServerName string `mapstructure:"tls_server_name"`
}
// DefaultVaultConfig() returns the canonical defaults for the Nomad
// `vault` configuration.
func DefaultVaultConfig() *VaultConfig {
return &VaultConfig{
Addr: "https://vault.service.consul:8200",
ConnectionRetryIntv: DefaultVaultConnectRetryIntv,
AllowUnauthenticated: func(b bool) *bool {
return &b
}(true),
}
}
// IsEnabled returns whether the config enables Vault integration
func (a *VaultConfig) IsEnabled() bool {
return a.Enabled != nil && *a.Enabled
}
// AllowsUnauthenticated returns whether the config allows unauthenticated
// access to Vault
func (a *VaultConfig) AllowsUnauthenticated() bool {
return a.AllowUnauthenticated != nil && *a.AllowUnauthenticated
}
// Merge merges two Vault configurations together.
func (a *VaultConfig) Merge(b *VaultConfig) *VaultConfig {
result := *a
if b.Token != "" {
result.Token = b.Token
}
if b.Role != "" {
result.Role = b.Role
}
if b.TaskTokenTTL != "" {
result.TaskTokenTTL = b.TaskTokenTTL
}
if b.Addr != "" {
result.Addr = b.Addr
}
if b.ConnectionRetryIntv.Nanoseconds() != 0 {
result.ConnectionRetryIntv = b.ConnectionRetryIntv
}
if b.TLSCaFile != "" {
result.TLSCaFile = b.TLSCaFile
}
if b.TLSCaPath != "" {
result.TLSCaPath = b.TLSCaPath
}
if b.TLSCertFile != "" {
result.TLSCertFile = b.TLSCertFile
}
if b.TLSKeyFile != "" {
result.TLSKeyFile = b.TLSKeyFile
}
if b.TLSServerName != "" {
result.TLSServerName = b.TLSServerName
}
if b.AllowUnauthenticated != nil {
result.AllowUnauthenticated = b.AllowUnauthenticated
}
if b.TLSSkipVerify != nil {
result.TLSSkipVerify = b.TLSSkipVerify
}
if b.Enabled != nil {
result.Enabled = b.Enabled
}
return &result
}
// ApiConfig() returns a usable Vault config that can be passed directly to
// hashicorp/vault/api.
func (c *VaultConfig) ApiConfig() (*vault.Config, error) {
conf := vault.DefaultConfig()
tlsConf := &vault.TLSConfig{
CACert: c.TLSCaFile,
CAPath: c.TLSCaPath,
ClientCert: c.TLSCertFile,
ClientKey: c.TLSKeyFile,
TLSServerName: c.TLSServerName,
}
if c.TLSSkipVerify != nil {
tlsConf.Insecure = *c.TLSSkipVerify
} else {
tlsConf.Insecure = false
}
if err := conf.ConfigureTLS(tlsConf); err != nil {
return nil, err
}
conf.Address = c.Addr
return conf, nil
}
// Copy returns a copy of this Vault config.
func (c *VaultConfig) Copy() *VaultConfig {
if c == nil {
return nil
}
nc := new(VaultConfig)
*nc = *c
return nc
}
// IsEqual compares two Vault configurations and returns a boolean indicating
// if they are equal.
func (a *VaultConfig) IsEqual(b *VaultConfig) bool {
if a == nil && b != nil {
return false
}
if a != nil && b == nil {
return false
}
if a.Token != b.Token {
return false
}
if a.Role != b.Role {
return false
}
if a.TaskTokenTTL != b.TaskTokenTTL {
return false
}
if a.Addr != b.Addr {
return false
}
if a.ConnectionRetryIntv.Nanoseconds() != b.ConnectionRetryIntv.Nanoseconds() {
return false
}
if a.TLSCaFile != b.TLSCaFile {
return false
}
if a.TLSCaPath != b.TLSCaPath {
return false
}
if a.TLSCertFile != b.TLSCertFile {
return false
}
if a.TLSKeyFile != b.TLSKeyFile {
return false
}
if a.TLSServerName != b.TLSServerName {
return false
}
if a.AllowUnauthenticated != b.AllowUnauthenticated {
return false
}
if a.TLSSkipVerify != b.TLSSkipVerify {
return false
}
if a.Enabled != b.Enabled {
return false
}
return true
}

1282
vendor/github.com/hashicorp/nomad/nomad/structs/diff.go generated vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,142 @@
package structs
import (
"errors"
"fmt"
"strings"
)
const (
errNoLeader = "No cluster leader"
errNoRegionPath = "No path to region"
errTokenNotFound = "ACL token not found"
errPermissionDenied = "Permission denied"
errNoNodeConn = "No path to node"
errUnknownMethod = "Unknown rpc method"
errUnknownNomadVersion = "Unable to determine Nomad version"
errNodeLacksRpc = "Node does not support RPC; requires 0.8 or later"
// Prefix based errors that are used to check if the error is of a given
// type. These errors should be created with the associated constructor.
ErrUnknownAllocationPrefix = "Unknown allocation"
ErrUnknownNodePrefix = "Unknown node"
ErrUnknownJobPrefix = "Unknown job"
ErrUnknownEvaluationPrefix = "Unknown evaluation"
ErrUnknownDeploymentPrefix = "Unknown deployment"
)
var (
ErrNoLeader = errors.New(errNoLeader)
ErrNoRegionPath = errors.New(errNoRegionPath)
ErrTokenNotFound = errors.New(errTokenNotFound)
ErrPermissionDenied = errors.New(errPermissionDenied)
ErrNoNodeConn = errors.New(errNoNodeConn)
ErrUnknownMethod = errors.New(errUnknownMethod)
ErrUnknownNomadVersion = errors.New(errUnknownNomadVersion)
ErrNodeLacksRpc = errors.New(errNodeLacksRpc)
)
// IsErrNoLeader returns whether the error is due to there being no leader.
func IsErrNoLeader(err error) bool {
return err != nil && strings.Contains(err.Error(), errNoLeader)
}
// IsErrNoRegionPath returns whether the error is due to there being no path to
// the given region.
func IsErrNoRegionPath(err error) bool {
return err != nil && strings.Contains(err.Error(), errNoRegionPath)
}
// IsErrTokenNotFound returns whether the error is due to the passed token not
// being resolvable.
func IsErrTokenNotFound(err error) bool {
return err != nil && strings.Contains(err.Error(), errTokenNotFound)
}
// IsErrPermissionDenied returns whether the error is due to the operation not
// being allowed due to lack of permissions.
func IsErrPermissionDenied(err error) bool {
return err != nil && strings.Contains(err.Error(), errPermissionDenied)
}
// IsErrNoNodeConn returns whether the error is due to there being no path to
// the given node.
func IsErrNoNodeConn(err error) bool {
return err != nil && strings.Contains(err.Error(), errNoNodeConn)
}
// IsErrUnknownMethod returns whether the error is due to the operation not
// being allowed due to lack of permissions.
func IsErrUnknownMethod(err error) bool {
return err != nil && strings.Contains(err.Error(), errUnknownMethod)
}
// NewErrUnknownAllocation returns a new error caused by the allocation being
// unknown.
func NewErrUnknownAllocation(allocID string) error {
return fmt.Errorf("%s %q", ErrUnknownAllocationPrefix, allocID)
}
// NewErrUnknownNode returns a new error caused by the node being unknown.
func NewErrUnknownNode(nodeID string) error {
return fmt.Errorf("%s %q", ErrUnknownNodePrefix, nodeID)
}
// NewErrUnknownJob returns a new error caused by the job being unknown.
func NewErrUnknownJob(jobID string) error {
return fmt.Errorf("%s %q", ErrUnknownJobPrefix, jobID)
}
// NewErrUnknownEvaluation returns a new error caused by the evaluation being
// unknown.
func NewErrUnknownEvaluation(evaluationID string) error {
return fmt.Errorf("%s %q", ErrUnknownEvaluationPrefix, evaluationID)
}
// NewErrUnknownDeployment returns a new error caused by the deployment being
// unknown.
func NewErrUnknownDeployment(deploymentID string) error {
return fmt.Errorf("%s %q", ErrUnknownDeploymentPrefix, deploymentID)
}
// IsErrUnknownAllocation returns whether the error is due to an unknown
// allocation.
func IsErrUnknownAllocation(err error) bool {
return err != nil && strings.Contains(err.Error(), ErrUnknownAllocationPrefix)
}
// IsErrUnknownNode returns whether the error is due to an unknown
// node.
func IsErrUnknownNode(err error) bool {
return err != nil && strings.Contains(err.Error(), ErrUnknownNodePrefix)
}
// IsErrUnknownJob returns whether the error is due to an unknown
// job.
func IsErrUnknownJob(err error) bool {
return err != nil && strings.Contains(err.Error(), ErrUnknownJobPrefix)
}
// IsErrUnknownEvaluation returns whether the error is due to an unknown
// evaluation.
func IsErrUnknownEvaluation(err error) bool {
return err != nil && strings.Contains(err.Error(), ErrUnknownEvaluationPrefix)
}
// IsErrUnknownDeployment returns whether the error is due to an unknown
// deployment.
func IsErrUnknownDeployment(err error) bool {
return err != nil && strings.Contains(err.Error(), ErrUnknownDeploymentPrefix)
}
// IsErrUnknownNomadVersion returns whether the error is due to Nomad being
// unable to determine the version of a node.
func IsErrUnknownNomadVersion(err error) bool {
return err != nil && strings.Contains(err.Error(), errUnknownNomadVersion)
}
// IsErrNodeLacksRpc returns whether error is due to a Nomad server being
// unable to connect to a client node because the client is too old (pre-v0.8).
func IsErrNodeLacksRpc(err error) bool {
return err != nil && strings.Contains(err.Error(), errNodeLacksRpc)
}

View File

@@ -0,0 +1,328 @@
package structs
import (
"crypto/subtle"
"encoding/base64"
"encoding/binary"
"fmt"
"math"
"sort"
"strings"
"golang.org/x/crypto/blake2b"
multierror "github.com/hashicorp/go-multierror"
lru "github.com/hashicorp/golang-lru"
"github.com/hashicorp/nomad/acl"
)
// MergeMultierrorWarnings takes job warnings and canonicalize warnings and
// merges them into a returnable string. Both the errors may be nil.
func MergeMultierrorWarnings(warnings ...error) string {
var warningMsg multierror.Error
for _, warn := range warnings {
if warn != nil {
multierror.Append(&warningMsg, warn)
}
}
if len(warningMsg.Errors) == 0 {
return ""
}
// Set the formatter
warningMsg.ErrorFormat = warningsFormatter
return warningMsg.Error()
}
// warningsFormatter is used to format job warnings
func warningsFormatter(es []error) string {
points := make([]string, len(es))
for i, err := range es {
points[i] = fmt.Sprintf("* %s", err)
}
return fmt.Sprintf(
"%d warning(s):\n\n%s",
len(es), strings.Join(points, "\n"))
}
// RemoveAllocs is used to remove any allocs with the given IDs
// from the list of allocations
func RemoveAllocs(alloc []*Allocation, remove []*Allocation) []*Allocation {
// Convert remove into a set
removeSet := make(map[string]struct{})
for _, remove := range remove {
removeSet[remove.ID] = struct{}{}
}
n := len(alloc)
for i := 0; i < n; i++ {
if _, ok := removeSet[alloc[i].ID]; ok {
alloc[i], alloc[n-1] = alloc[n-1], nil
i--
n--
}
}
alloc = alloc[:n]
return alloc
}
// FilterTerminalAllocs filters out all allocations in a terminal state and
// returns the latest terminal allocations
func FilterTerminalAllocs(allocs []*Allocation) ([]*Allocation, map[string]*Allocation) {
terminalAllocsByName := make(map[string]*Allocation)
n := len(allocs)
for i := 0; i < n; i++ {
if allocs[i].TerminalStatus() {
// Add the allocation to the terminal allocs map if it's not already
// added or has a higher create index than the one which is
// currently present.
alloc, ok := terminalAllocsByName[allocs[i].Name]
if !ok || alloc.CreateIndex < allocs[i].CreateIndex {
terminalAllocsByName[allocs[i].Name] = allocs[i]
}
// Remove the allocation
allocs[i], allocs[n-1] = allocs[n-1], nil
i--
n--
}
}
return allocs[:n], terminalAllocsByName
}
// AllocsFit checks if a given set of allocations will fit on a node.
// The netIdx can optionally be provided if its already been computed.
// If the netIdx is provided, it is assumed that the client has already
// ensured there are no collisions.
func AllocsFit(node *Node, allocs []*Allocation, netIdx *NetworkIndex) (bool, string, *Resources, error) {
// Compute the utilization from zero
used := new(Resources)
// Add the reserved resources of the node
if node.Reserved != nil {
if err := used.Add(node.Reserved); err != nil {
return false, "", nil, err
}
}
// For each alloc, add the resources
for _, alloc := range allocs {
// Do not consider the resource impact of terminal allocations
if alloc.TerminalStatus() {
continue
}
if alloc.Resources != nil {
if err := used.Add(alloc.Resources); err != nil {
return false, "", nil, err
}
} else if alloc.TaskResources != nil {
// Adding the shared resource asks for the allocation to the used
// resources
if err := used.Add(alloc.SharedResources); err != nil {
return false, "", nil, err
}
// Allocations within the plan have the combined resources stripped
// to save space, so sum up the individual task resources.
for _, taskResource := range alloc.TaskResources {
if err := used.Add(taskResource); err != nil {
return false, "", nil, err
}
}
} else {
return false, "", nil, fmt.Errorf("allocation %q has no resources set", alloc.ID)
}
}
// Check that the node resources are a super set of those
// that are being allocated
if superset, dimension := node.Resources.Superset(used); !superset {
return false, dimension, used, nil
}
// Create the network index if missing
if netIdx == nil {
netIdx = NewNetworkIndex()
defer netIdx.Release()
if netIdx.SetNode(node) || netIdx.AddAllocs(allocs) {
return false, "reserved port collision", used, nil
}
}
// Check if the network is overcommitted
if netIdx.Overcommitted() {
return false, "bandwidth exceeded", used, nil
}
// Allocations fit!
return true, "", used, nil
}
// ScoreFit is used to score the fit based on the Google work published here:
// http://www.columbia.edu/~cs2035/courses/ieor4405.S13/datacenter_scheduling.ppt
// This is equivalent to their BestFit v3
func ScoreFit(node *Node, util *Resources) float64 {
// Determine the node availability
nodeCpu := float64(node.Resources.CPU)
if node.Reserved != nil {
nodeCpu -= float64(node.Reserved.CPU)
}
nodeMem := float64(node.Resources.MemoryMB)
if node.Reserved != nil {
nodeMem -= float64(node.Reserved.MemoryMB)
}
// Compute the free percentage
freePctCpu := 1 - (float64(util.CPU) / nodeCpu)
freePctRam := 1 - (float64(util.MemoryMB) / nodeMem)
// Total will be "maximized" the smaller the value is.
// At 100% utilization, the total is 2, while at 0% util it is 20.
total := math.Pow(10, freePctCpu) + math.Pow(10, freePctRam)
// Invert so that the "maximized" total represents a high-value
// score. Because the floor is 20, we simply use that as an anchor.
// This means at a perfect fit, we return 18 as the score.
score := 20.0 - total
// Bound the score, just in case
// If the score is over 18, that means we've overfit the node.
if score > 18.0 {
score = 18.0
} else if score < 0 {
score = 0
}
return score
}
func CopySliceConstraints(s []*Constraint) []*Constraint {
l := len(s)
if l == 0 {
return nil
}
c := make([]*Constraint, l)
for i, v := range s {
c[i] = v.Copy()
}
return c
}
// VaultPoliciesSet takes the structure returned by VaultPolicies and returns
// the set of required policies
func VaultPoliciesSet(policies map[string]map[string]*Vault) []string {
set := make(map[string]struct{})
for _, tgp := range policies {
for _, tp := range tgp {
for _, p := range tp.Policies {
set[p] = struct{}{}
}
}
}
flattened := make([]string, 0, len(set))
for p := range set {
flattened = append(flattened, p)
}
return flattened
}
// DenormalizeAllocationJobs is used to attach a job to all allocations that are
// non-terminal and do not have a job already. This is useful in cases where the
// job is normalized.
func DenormalizeAllocationJobs(job *Job, allocs []*Allocation) {
if job != nil {
for _, alloc := range allocs {
if alloc.Job == nil && !alloc.TerminalStatus() {
alloc.Job = job
}
}
}
}
// AllocName returns the name of the allocation given the input.
func AllocName(job, group string, idx uint) string {
return fmt.Sprintf("%s.%s[%d]", job, group, idx)
}
// ACLPolicyListHash returns a consistent hash for a set of policies.
func ACLPolicyListHash(policies []*ACLPolicy) string {
cacheKeyHash, err := blake2b.New256(nil)
if err != nil {
panic(err)
}
for _, policy := range policies {
cacheKeyHash.Write([]byte(policy.Name))
binary.Write(cacheKeyHash, binary.BigEndian, policy.ModifyIndex)
}
cacheKey := string(cacheKeyHash.Sum(nil))
return cacheKey
}
// CompileACLObject compiles a set of ACL policies into an ACL object with a cache
func CompileACLObject(cache *lru.TwoQueueCache, policies []*ACLPolicy) (*acl.ACL, error) {
// Sort the policies to ensure consistent ordering
sort.Slice(policies, func(i, j int) bool {
return policies[i].Name < policies[j].Name
})
// Determine the cache key
cacheKey := ACLPolicyListHash(policies)
aclRaw, ok := cache.Get(cacheKey)
if ok {
return aclRaw.(*acl.ACL), nil
}
// Parse the policies
parsed := make([]*acl.Policy, 0, len(policies))
for _, policy := range policies {
p, err := acl.Parse(policy.Rules)
if err != nil {
return nil, fmt.Errorf("failed to parse %q: %v", policy.Name, err)
}
parsed = append(parsed, p)
}
// Create the ACL object
aclObj, err := acl.NewACL(false, parsed)
if err != nil {
return nil, fmt.Errorf("failed to construct ACL: %v", err)
}
// Update the cache
cache.Add(cacheKey, aclObj)
return aclObj, nil
}
// GenerateMigrateToken will create a token for a client to access an
// authenticated volume of another client to migrate data for sticky volumes.
func GenerateMigrateToken(allocID, nodeSecretID string) (string, error) {
h, err := blake2b.New512([]byte(nodeSecretID))
if err != nil {
return "", err
}
h.Write([]byte(allocID))
return base64.URLEncoding.EncodeToString(h.Sum(nil)), nil
}
// CompareMigrateToken returns true if two migration tokens can be computed and
// are equal.
func CompareMigrateToken(allocID, nodeSecretID, otherMigrateToken string) bool {
h, err := blake2b.New512([]byte(nodeSecretID))
if err != nil {
return false
}
h.Write([]byte(allocID))
otherBytes, err := base64.URLEncoding.DecodeString(otherMigrateToken)
if err != nil {
return false
}
return subtle.ConstantTimeCompare(h.Sum(nil), otherBytes) == 1
}

View File

@@ -0,0 +1,326 @@
package structs
import (
"fmt"
"math/rand"
"net"
"sync"
)
const (
// MinDynamicPort is the smallest dynamic port generated
MinDynamicPort = 20000
// MaxDynamicPort is the largest dynamic port generated
MaxDynamicPort = 32000
// maxRandPortAttempts is the maximum number of attempt
// to assign a random port
maxRandPortAttempts = 20
// maxValidPort is the max valid port number
maxValidPort = 65536
)
var (
// bitmapPool is used to pool the bitmaps used for port collision
// checking. They are fairly large (8K) so we can re-use them to
// avoid GC pressure. Care should be taken to call Clear() on any
// bitmap coming from the pool.
bitmapPool = new(sync.Pool)
)
// NetworkIndex is used to index the available network resources
// and the used network resources on a machine given allocations
type NetworkIndex struct {
AvailNetworks []*NetworkResource // List of available networks
AvailBandwidth map[string]int // Bandwidth by device
UsedPorts map[string]Bitmap // Ports by IP
UsedBandwidth map[string]int // Bandwidth by device
}
// NewNetworkIndex is used to construct a new network index
func NewNetworkIndex() *NetworkIndex {
return &NetworkIndex{
AvailBandwidth: make(map[string]int),
UsedPorts: make(map[string]Bitmap),
UsedBandwidth: make(map[string]int),
}
}
// Release is called when the network index is no longer needed
// to attempt to re-use some of the memory it has allocated
func (idx *NetworkIndex) Release() {
for _, b := range idx.UsedPorts {
bitmapPool.Put(b)
}
}
// Overcommitted checks if the network is overcommitted
func (idx *NetworkIndex) Overcommitted() bool {
for device, used := range idx.UsedBandwidth {
avail := idx.AvailBandwidth[device]
if used > avail {
return true
}
}
return false
}
// SetNode is used to setup the available network resources. Returns
// true if there is a collision
func (idx *NetworkIndex) SetNode(node *Node) (collide bool) {
// Add the available CIDR blocks
for _, n := range node.Resources.Networks {
if n.Device != "" {
idx.AvailNetworks = append(idx.AvailNetworks, n)
idx.AvailBandwidth[n.Device] = n.MBits
}
}
// Add the reserved resources
if r := node.Reserved; r != nil {
for _, n := range r.Networks {
if idx.AddReserved(n) {
collide = true
}
}
}
return
}
// AddAllocs is used to add the used network resources. Returns
// true if there is a collision
func (idx *NetworkIndex) AddAllocs(allocs []*Allocation) (collide bool) {
for _, alloc := range allocs {
for _, task := range alloc.TaskResources {
if len(task.Networks) == 0 {
continue
}
n := task.Networks[0]
if idx.AddReserved(n) {
collide = true
}
}
}
return
}
// AddReserved is used to add a reserved network usage, returns true
// if there is a port collision
func (idx *NetworkIndex) AddReserved(n *NetworkResource) (collide bool) {
// Add the port usage
used := idx.UsedPorts[n.IP]
if used == nil {
// Try to get a bitmap from the pool, else create
raw := bitmapPool.Get()
if raw != nil {
used = raw.(Bitmap)
used.Clear()
} else {
used, _ = NewBitmap(maxValidPort)
}
idx.UsedPorts[n.IP] = used
}
for _, ports := range [][]Port{n.ReservedPorts, n.DynamicPorts} {
for _, port := range ports {
// Guard against invalid port
if port.Value < 0 || port.Value >= maxValidPort {
return true
}
if used.Check(uint(port.Value)) {
collide = true
} else {
used.Set(uint(port.Value))
}
}
}
// Add the bandwidth
idx.UsedBandwidth[n.Device] += n.MBits
return
}
// yieldIP is used to iteratively invoke the callback with
// an available IP
func (idx *NetworkIndex) yieldIP(cb func(net *NetworkResource, ip net.IP) bool) {
inc := func(ip net.IP) {
for j := len(ip) - 1; j >= 0; j-- {
ip[j]++
if ip[j] > 0 {
break
}
}
}
for _, n := range idx.AvailNetworks {
ip, ipnet, err := net.ParseCIDR(n.CIDR)
if err != nil {
continue
}
for ip := ip.Mask(ipnet.Mask); ipnet.Contains(ip); inc(ip) {
if cb(n, ip) {
return
}
}
}
}
// AssignNetwork is used to assign network resources given an ask.
// If the ask cannot be satisfied, returns nil
func (idx *NetworkIndex) AssignNetwork(ask *NetworkResource) (out *NetworkResource, err error) {
err = fmt.Errorf("no networks available")
idx.yieldIP(func(n *NetworkResource, ip net.IP) (stop bool) {
// Convert the IP to a string
ipStr := ip.String()
// Check if we would exceed the bandwidth cap
availBandwidth := idx.AvailBandwidth[n.Device]
usedBandwidth := idx.UsedBandwidth[n.Device]
if usedBandwidth+ask.MBits > availBandwidth {
err = fmt.Errorf("bandwidth exceeded")
return
}
used := idx.UsedPorts[ipStr]
// Check if any of the reserved ports are in use
for _, port := range ask.ReservedPorts {
// Guard against invalid port
if port.Value < 0 || port.Value >= maxValidPort {
err = fmt.Errorf("invalid port %d (out of range)", port.Value)
return
}
// Check if in use
if used != nil && used.Check(uint(port.Value)) {
err = fmt.Errorf("reserved port collision")
return
}
}
// Create the offer
offer := &NetworkResource{
Device: n.Device,
IP: ipStr,
MBits: ask.MBits,
ReservedPorts: ask.ReservedPorts,
DynamicPorts: ask.DynamicPorts,
}
// Try to stochastically pick the dynamic ports as it is faster and
// lower memory usage.
var dynPorts []int
var dynErr error
dynPorts, dynErr = getDynamicPortsStochastic(used, ask)
if dynErr == nil {
goto BUILD_OFFER
}
// Fall back to the precise method if the random sampling failed.
dynPorts, dynErr = getDynamicPortsPrecise(used, ask)
if dynErr != nil {
err = dynErr
return
}
BUILD_OFFER:
for i, port := range dynPorts {
offer.DynamicPorts[i].Value = port
}
// Stop, we have an offer!
out = offer
err = nil
return true
})
return
}
// getDynamicPortsPrecise takes the nodes used port bitmap which may be nil if
// no ports have been allocated yet, the network ask and returns a set of unused
// ports to fullfil the ask's DynamicPorts or an error if it failed. An error
// means the ask can not be satisfied as the method does a precise search.
func getDynamicPortsPrecise(nodeUsed Bitmap, ask *NetworkResource) ([]int, error) {
// Create a copy of the used ports and apply the new reserves
var usedSet Bitmap
var err error
if nodeUsed != nil {
usedSet, err = nodeUsed.Copy()
if err != nil {
return nil, err
}
} else {
usedSet, err = NewBitmap(maxValidPort)
if err != nil {
return nil, err
}
}
for _, port := range ask.ReservedPorts {
usedSet.Set(uint(port.Value))
}
// Get the indexes of the unset
availablePorts := usedSet.IndexesInRange(false, MinDynamicPort, MaxDynamicPort)
// Randomize the amount we need
numDyn := len(ask.DynamicPorts)
if len(availablePorts) < numDyn {
return nil, fmt.Errorf("dynamic port selection failed")
}
numAvailable := len(availablePorts)
for i := 0; i < numDyn; i++ {
j := rand.Intn(numAvailable)
availablePorts[i], availablePorts[j] = availablePorts[j], availablePorts[i]
}
return availablePorts[:numDyn], nil
}
// getDynamicPortsStochastic takes the nodes used port bitmap which may be nil if
// no ports have been allocated yet, the network ask and returns a set of unused
// ports to fullfil the ask's DynamicPorts or an error if it failed. An error
// does not mean the ask can not be satisfied as the method has a fixed amount
// of random probes and if these fail, the search is aborted.
func getDynamicPortsStochastic(nodeUsed Bitmap, ask *NetworkResource) ([]int, error) {
var reserved, dynamic []int
for _, port := range ask.ReservedPorts {
reserved = append(reserved, port.Value)
}
for i := 0; i < len(ask.DynamicPorts); i++ {
attempts := 0
PICK:
attempts++
if attempts > maxRandPortAttempts {
return nil, fmt.Errorf("stochastic dynamic port selection failed")
}
randPort := MinDynamicPort + rand.Intn(MaxDynamicPort-MinDynamicPort)
if nodeUsed != nil && nodeUsed.Check(uint(randPort)) {
goto PICK
}
for _, ports := range [][]int{reserved, dynamic} {
if isPortReserved(ports, randPort) {
goto PICK
}
}
dynamic = append(dynamic, randPort)
}
return dynamic, nil
}
// IntContains scans an integer slice for a value
func isPortReserved(haystack []int, needle int) bool {
for _, item := range haystack {
if item == needle {
return true
}
}
return false
}

View File

@@ -0,0 +1,62 @@
package structs
import (
"time"
"github.com/hashicorp/nomad/helper"
)
// DriverInfo is the current state of a single driver. This is updated
// regularly as driver health changes on the node.
type DriverInfo struct {
Attributes map[string]string
Detected bool
Healthy bool
HealthDescription string
UpdateTime time.Time
}
func (di *DriverInfo) Copy() *DriverInfo {
if di == nil {
return nil
}
cdi := new(DriverInfo)
*cdi = *di
cdi.Attributes = helper.CopyMapStringString(di.Attributes)
return cdi
}
// MergeHealthCheck merges information from a health check for a drier into a
// node's driver info
func (di *DriverInfo) MergeHealthCheck(other *DriverInfo) {
di.Healthy = other.Healthy
di.HealthDescription = other.HealthDescription
di.UpdateTime = other.UpdateTime
}
// MergeFingerprint merges information from fingerprinting a node for a driver
// into a node's driver info for that driver.
func (di *DriverInfo) MergeFingerprintInfo(other *DriverInfo) {
di.Detected = other.Detected
di.Attributes = other.Attributes
}
// DriverInfo determines if two driver info objects are equal..As this is used
// in the process of health checking, we only check the fields that are
// computed by the health checker. In the future, this will be merged.
func (di *DriverInfo) HealthCheckEquals(other *DriverInfo) bool {
if di == nil && other == nil {
return true
}
if di.Healthy != other.Healthy {
return false
}
if di.HealthDescription != other.HealthDescription {
return false
}
return true
}

View File

@@ -0,0 +1,94 @@
package structs
import (
"fmt"
"strings"
"github.com/mitchellh/hashstructure"
)
const (
// NodeUniqueNamespace is a prefix that can be appended to node meta or
// attribute keys to mark them for exclusion in computed node class.
NodeUniqueNamespace = "unique."
)
// UniqueNamespace takes a key and returns the key marked under the unique
// namespace.
func UniqueNamespace(key string) string {
return fmt.Sprintf("%s%s", NodeUniqueNamespace, key)
}
// IsUniqueNamespace returns whether the key is under the unique namespace.
func IsUniqueNamespace(key string) bool {
return strings.HasPrefix(key, NodeUniqueNamespace)
}
// ComputeClass computes a derived class for the node based on its attributes.
// ComputedClass is a unique id that identifies nodes with a common set of
// attributes and capabilities. Thus, when calculating a node's computed class
// we avoid including any uniquely identifying fields.
func (n *Node) ComputeClass() error {
hash, err := hashstructure.Hash(n, nil)
if err != nil {
return err
}
n.ComputedClass = fmt.Sprintf("v1:%d", hash)
return nil
}
// HashInclude is used to blacklist uniquely identifying node fields from being
// included in the computed node class.
func (n Node) HashInclude(field string, v interface{}) (bool, error) {
switch field {
case "Datacenter", "Attributes", "Meta", "NodeClass":
return true, nil
default:
return false, nil
}
}
// HashIncludeMap is used to blacklist uniquely identifying node map keys from being
// included in the computed node class.
func (n Node) HashIncludeMap(field string, k, v interface{}) (bool, error) {
key, ok := k.(string)
if !ok {
return false, fmt.Errorf("map key %v not a string", k)
}
switch field {
case "Meta", "Attributes":
return !IsUniqueNamespace(key), nil
default:
return false, fmt.Errorf("unexpected map field: %v", field)
}
}
// EscapedConstraints takes a set of constraints and returns the set that
// escapes computed node classes.
func EscapedConstraints(constraints []*Constraint) []*Constraint {
var escaped []*Constraint
for _, c := range constraints {
if constraintTargetEscapes(c.LTarget) || constraintTargetEscapes(c.RTarget) {
escaped = append(escaped, c)
}
}
return escaped
}
// constraintTargetEscapes returns whether the target of a constraint escapes
// computed node class optimization.
func constraintTargetEscapes(target string) bool {
switch {
case strings.HasPrefix(target, "${node.unique."):
return true
case strings.HasPrefix(target, "${attr.unique."):
return true
case strings.HasPrefix(target, "${meta.unique."):
return true
default:
return false
}
}

View File

@@ -0,0 +1,121 @@
package structs
import (
"time"
"github.com/hashicorp/raft"
)
// RaftServer has information about a server in the Raft configuration.
type RaftServer struct {
// ID is the unique ID for the server. These are currently the same
// as the address, but they will be changed to a real GUID in a future
// release of Nomad.
ID raft.ServerID
// Node is the node name of the server, as known by Nomad, or this
// will be set to "(unknown)" otherwise.
Node string
// Address is the IP:port of the server, used for Raft communications.
Address raft.ServerAddress
// Leader is true if this server is the current cluster leader.
Leader bool
// Voter is true if this server has a vote in the cluster. This might
// be false if the server is staging and still coming online, or if
// it's a non-voting server, which will be added in a future release of
// Nomad.
Voter bool
// RaftProtocol is the version of the Raft protocol spoken by this server.
RaftProtocol string
}
// RaftConfigurationResponse is returned when querying for the current Raft
// configuration.
type RaftConfigurationResponse struct {
// Servers has the list of servers in the Raft configuration.
Servers []*RaftServer
// Index has the Raft index of this configuration.
Index uint64
}
// RaftPeerByAddressRequest is used by the Operator endpoint to apply a Raft
// operation on a specific Raft peer by address in the form of "IP:port".
type RaftPeerByAddressRequest struct {
// Address is the peer to remove, in the form "IP:port".
Address raft.ServerAddress
// WriteRequest holds the Region for this request.
WriteRequest
}
// RaftPeerByIDRequest is used by the Operator endpoint to apply a Raft
// operation on a specific Raft peer by ID.
type RaftPeerByIDRequest struct {
// ID is the peer ID to remove.
ID raft.ServerID
// WriteRequest holds the Region for this request.
WriteRequest
}
// AutopilotSetConfigRequest is used by the Operator endpoint to update the
// current Autopilot configuration of the cluster.
type AutopilotSetConfigRequest struct {
// Datacenter is the target this request is intended for.
Datacenter string
// Config is the new Autopilot configuration to use.
Config AutopilotConfig
// CAS controls whether to use check-and-set semantics for this request.
CAS bool
// WriteRequest holds the ACL token to go along with this request.
WriteRequest
}
// RequestDatacenter returns the datacenter for a given request.
func (op *AutopilotSetConfigRequest) RequestDatacenter() string {
return op.Datacenter
}
// AutopilotConfig is the internal config for the Autopilot mechanism.
type AutopilotConfig struct {
// CleanupDeadServers controls whether to remove dead servers when a new
// server is added to the Raft peers.
CleanupDeadServers bool
// ServerStabilizationTime is the minimum amount of time a server must be
// in a stable, healthy state before it can be added to the cluster. Only
// applicable with Raft protocol version 3 or higher.
ServerStabilizationTime time.Duration
// LastContactThreshold is the limit on the amount of time a server can go
// without leader contact before being considered unhealthy.
LastContactThreshold time.Duration
// MaxTrailingLogs is the amount of entries in the Raft Log that a server can
// be behind before being considered unhealthy.
MaxTrailingLogs uint64
// (Enterprise-only) EnableRedundancyZones specifies whether to enable redundancy zones.
EnableRedundancyZones bool
// (Enterprise-only) DisableUpgradeMigration will disable Autopilot's upgrade migration
// strategy of waiting until enough newer-versioned servers have been added to the
// cluster before promoting them to voters.
DisableUpgradeMigration bool
// (Enterprise-only) EnableCustomUpgrades specifies whether to enable using custom
// upgrade versions when performing migrations.
EnableCustomUpgrades bool
// CreateIndex/ModifyIndex store the create/modify indexes of this configuration.
CreateIndex uint64
ModifyIndex uint64
}

View File

@@ -0,0 +1,72 @@
package structs
import (
"fmt"
"io"
"sync"
)
// StreamingRpcHeader is the first struct serialized after entering the
// streaming RPC mode. The header is used to dispatch to the correct method.
type StreamingRpcHeader struct {
// Method is the name of the method to invoke.
Method string
}
// StreamingRpcAck is used to acknowledge receiving the StreamingRpcHeader and
// routing to the requested handler.
type StreamingRpcAck struct {
// Error is used to return whether an error occurred establishing the
// streaming RPC. This error occurs before entering the RPC handler.
Error string
}
// StreamingRpcHandler defines the handler for a streaming RPC.
type StreamingRpcHandler func(conn io.ReadWriteCloser)
// StreamingRpcRegistry is used to add and retrieve handlers
type StreamingRpcRegistry struct {
registry map[string]StreamingRpcHandler
}
// NewStreamingRpcRegistry creates a new registry. All registrations of
// handlers should be done before retrieving handlers.
func NewStreamingRpcRegistry() *StreamingRpcRegistry {
return &StreamingRpcRegistry{
registry: make(map[string]StreamingRpcHandler),
}
}
// Register registers a new handler for the given method name
func (s *StreamingRpcRegistry) Register(method string, handler StreamingRpcHandler) {
s.registry[method] = handler
}
// GetHandler returns a handler for the given method or an error if it doesn't exist.
func (s *StreamingRpcRegistry) GetHandler(method string) (StreamingRpcHandler, error) {
h, ok := s.registry[method]
if !ok {
return nil, fmt.Errorf("%s: %q", ErrUnknownMethod, method)
}
return h, nil
}
// Bridge is used to just link two connections together and copy traffic
func Bridge(a, b io.ReadWriteCloser) {
wg := sync.WaitGroup{}
wg.Add(2)
go func() {
defer wg.Done()
io.Copy(a, b)
a.Close()
b.Close()
}()
go func() {
defer wg.Done()
io.Copy(b, a)
a.Close()
b.Close()
}()
wg.Wait()
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,3 @@
package structs
//go:generate ./generate.sh

378
vendor/github.com/hashicorp/nomad/testutil/server.go generated vendored Normal file
View File

@@ -0,0 +1,378 @@
package testutil
// TestServer is a test helper. It uses a fork/exec model to create
// a test Nomad server instance in the background and initialize it
// with some data and/or services. The test server can then be used
// to run a unit test, and offers an easy API to tear itself down
// when the test has completed. The only prerequisite is to have a nomad
// binary available on the $PATH.
//
// This package does not use Nomad's official API client. This is
// because we use TestServer to test the API client, which would
// otherwise cause an import cycle.
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"os/exec"
"github.com/hashicorp/consul/lib/freeport"
cleanhttp "github.com/hashicorp/go-cleanhttp"
"github.com/hashicorp/nomad/helper/discover"
testing "github.com/mitchellh/go-testing-interface"
)
// TestServerConfig is the main server configuration struct.
type TestServerConfig struct {
NodeName string `json:"name,omitempty"`
DataDir string `json:"data_dir,omitempty"`
Region string `json:"region,omitempty"`
DisableCheckpoint bool `json:"disable_update_check"`
LogLevel string `json:"log_level,omitempty"`
Consul *Consul `json:"consul,omitempty"`
AdvertiseAddrs *Advertise `json:"advertise,omitempty"`
Ports *PortsConfig `json:"ports,omitempty"`
Server *ServerConfig `json:"server,omitempty"`
Client *ClientConfig `json:"client,omitempty"`
Vault *VaultConfig `json:"vault,omitempty"`
ACL *ACLConfig `json:"acl,omitempty"`
DevMode bool `json:"-"`
Stdout, Stderr io.Writer `json:"-"`
}
// Consul is used to configure the communication with Consul
type Consul struct {
Address string `json:"address,omitempty"`
Auth string `json:"auth,omitempty"`
Token string `json:"token,omitempty"`
}
// Advertise is used to configure the addresses to advertise
type Advertise struct {
HTTP string `json:"http,omitempty"`
RPC string `json:"rpc,omitempty"`
Serf string `json:"serf,omitempty"`
}
// PortsConfig is used to configure the network ports we use.
type PortsConfig struct {
HTTP int `json:"http,omitempty"`
RPC int `json:"rpc,omitempty"`
Serf int `json:"serf,omitempty"`
}
// ServerConfig is used to configure the nomad server.
type ServerConfig struct {
Enabled bool `json:"enabled"`
BootstrapExpect int `json:"bootstrap_expect"`
RaftProtocol int `json:"raft_protocol,omitempty"`
}
// ClientConfig is used to configure the client
type ClientConfig struct {
Enabled bool `json:"enabled"`
}
// VaultConfig is used to configure Vault
type VaultConfig struct {
Enabled bool `json:"enabled"`
}
// ACLConfig is used to configure ACLs
type ACLConfig struct {
Enabled bool `json:"enabled"`
}
// ServerConfigCallback is a function interface which can be
// passed to NewTestServerConfig to modify the server config.
type ServerConfigCallback func(c *TestServerConfig)
// defaultServerConfig returns a new TestServerConfig struct
// with all of the listen ports incremented by one.
func defaultServerConfig(t testing.T) *TestServerConfig {
ports := freeport.GetT(t, 3)
return &TestServerConfig{
NodeName: fmt.Sprintf("node-%d", ports[0]),
DisableCheckpoint: true,
LogLevel: "DEBUG",
Ports: &PortsConfig{
HTTP: ports[0],
RPC: ports[1],
Serf: ports[2],
},
Server: &ServerConfig{
Enabled: true,
BootstrapExpect: 1,
},
Client: &ClientConfig{
Enabled: false,
},
Vault: &VaultConfig{
Enabled: false,
},
ACL: &ACLConfig{
Enabled: false,
},
}
}
// TestServer is the main server wrapper struct.
type TestServer struct {
cmd *exec.Cmd
Config *TestServerConfig
t testing.T
HTTPAddr string
SerfAddr string
HTTPClient *http.Client
}
// NewTestServer creates a new TestServer, and makes a call to
// an optional callback function to modify the configuration.
func NewTestServer(t testing.T, cb ServerConfigCallback) *TestServer {
path, err := discover.NomadExecutable()
if err != nil {
t.Skipf("nomad not found, skipping: %v", err)
}
// Do a sanity check that we are actually running nomad
vcmd := exec.Command(path, "-version")
vcmd.Stdout = nil
vcmd.Stderr = nil
if err := vcmd.Run(); err != nil {
t.Skipf("nomad version failed: %v", err)
}
dataDir, err := ioutil.TempDir("", "nomad")
if err != nil {
t.Fatalf("err: %s", err)
}
configFile, err := ioutil.TempFile(dataDir, "nomad")
if err != nil {
defer os.RemoveAll(dataDir)
t.Fatalf("err: %s", err)
}
defer configFile.Close()
nomadConfig := defaultServerConfig(t)
nomadConfig.DataDir = dataDir
if cb != nil {
cb(nomadConfig)
}
configContent, err := json.Marshal(nomadConfig)
if err != nil {
t.Fatalf("err: %s", err)
}
if _, err := configFile.Write(configContent); err != nil {
t.Fatalf("err: %s", err)
}
configFile.Close()
stdout := io.Writer(os.Stdout)
if nomadConfig.Stdout != nil {
stdout = nomadConfig.Stdout
}
stderr := io.Writer(os.Stderr)
if nomadConfig.Stderr != nil {
stderr = nomadConfig.Stderr
}
args := []string{"agent", "-config", configFile.Name()}
if nomadConfig.DevMode {
args = append(args, "-dev")
}
// Start the server
cmd := exec.Command(path, args...)
cmd.Stdout = stdout
cmd.Stderr = stderr
if err := cmd.Start(); err != nil {
t.Fatalf("err: %s", err)
}
client := cleanhttp.DefaultClient()
server := &TestServer{
Config: nomadConfig,
cmd: cmd,
t: t,
HTTPAddr: fmt.Sprintf("127.0.0.1:%d", nomadConfig.Ports.HTTP),
SerfAddr: fmt.Sprintf("127.0.0.1:%d", nomadConfig.Ports.Serf),
HTTPClient: client,
}
// Wait for the server to be ready
if nomadConfig.Server.Enabled && nomadConfig.Server.BootstrapExpect != 0 {
server.waitForLeader()
} else {
server.waitForAPI()
}
// Wait for the client to be ready
if nomadConfig.DevMode {
server.waitForClient()
}
return server
}
// Stop stops the test Nomad server, and removes the Nomad data
// directory once we are done.
func (s *TestServer) Stop() {
defer os.RemoveAll(s.Config.DataDir)
if err := s.cmd.Process.Kill(); err != nil {
s.t.Errorf("err: %s", err)
}
// wait for the process to exit to be sure that the data dir can be
// deleted on all platforms.
s.cmd.Wait()
}
// waitForAPI waits for only the agent HTTP endpoint to start
// responding. This is an indication that the agent has started,
// but will likely return before a leader is elected.
func (s *TestServer) waitForAPI() {
WaitForResult(func() (bool, error) {
// Using this endpoint as it is does not have restricted access
resp, err := s.HTTPClient.Get(s.url("/v1/metrics"))
if err != nil {
return false, err
}
defer resp.Body.Close()
if err := s.requireOK(resp); err != nil {
return false, err
}
return true, nil
}, func(err error) {
defer s.Stop()
s.t.Fatalf("err: %s", err)
})
}
// waitForLeader waits for the Nomad server's HTTP API to become
// available, and then waits for a known leader and an index of
// 1 or more to be observed to confirm leader election is done.
func (s *TestServer) waitForLeader() {
WaitForResult(func() (bool, error) {
// Query the API and check the status code
// Using this endpoint as it is does not have restricted access
resp, err := s.HTTPClient.Get(s.url("/v1/status/leader"))
if err != nil {
return false, err
}
defer resp.Body.Close()
if err := s.requireOK(resp); err != nil {
return false, err
}
return true, nil
}, func(err error) {
defer s.Stop()
s.t.Fatalf("err: %s", err)
})
}
// waitForClient waits for the Nomad client to be ready. The function returns
// immediately if the server is not in dev mode.
func (s *TestServer) waitForClient() {
if !s.Config.DevMode {
return
}
WaitForResult(func() (bool, error) {
resp, err := s.HTTPClient.Get(s.url("/v1/nodes"))
if err != nil {
return false, err
}
defer resp.Body.Close()
if err := s.requireOK(resp); err != nil {
return false, err
}
var decoded []struct {
ID string
Status string
}
dec := json.NewDecoder(resp.Body)
if err := dec.Decode(&decoded); err != nil {
return false, err
}
if len(decoded) != 1 || decoded[0].Status != "ready" {
return false, fmt.Errorf("Node not ready: %v", decoded)
}
return true, nil
}, func(err error) {
defer s.Stop()
s.t.Fatalf("err: %s", err)
})
}
// url is a helper function which takes a relative URL and
// makes it into a proper URL against the local Nomad server.
func (s *TestServer) url(path string) string {
return fmt.Sprintf("http://%s%s", s.HTTPAddr, path)
}
// requireOK checks the HTTP response code and ensures it is acceptable.
func (s *TestServer) requireOK(resp *http.Response) error {
if resp.StatusCode != 200 {
return fmt.Errorf("Bad status code: %d", resp.StatusCode)
}
return nil
}
// put performs a new HTTP PUT request.
func (s *TestServer) put(path string, body io.Reader) *http.Response {
req, err := http.NewRequest("PUT", s.url(path), body)
if err != nil {
s.t.Fatalf("err: %s", err)
}
resp, err := s.HTTPClient.Do(req)
if err != nil {
s.t.Fatalf("err: %s", err)
}
if err := s.requireOK(resp); err != nil {
defer resp.Body.Close()
s.t.Fatal(err)
}
return resp
}
// get performs a new HTTP GET request.
func (s *TestServer) get(path string) *http.Response {
resp, err := s.HTTPClient.Get(s.url(path))
if err != nil {
s.t.Fatalf("err: %s", err)
}
if err := s.requireOK(resp); err != nil {
defer resp.Body.Close()
s.t.Fatal(err)
}
return resp
}
// encodePayload returns a new io.Reader wrapping the encoded contents
// of the payload, suitable for passing directly to a new request.
func (s *TestServer) encodePayload(payload interface{}) io.Reader {
var encoded bytes.Buffer
enc := json.NewEncoder(&encoded)
if err := enc.Encode(payload); err != nil {
s.t.Fatalf("err: %s", err)
}
return &encoded
}

15
vendor/github.com/hashicorp/nomad/testutil/slow.go generated vendored Normal file
View File

@@ -0,0 +1,15 @@
package testutil
import (
"os"
testing "github.com/mitchellh/go-testing-interface"
)
// SkipSlow skips a slow test unless the NOMAD_SLOW_TEST environment variable
// is set.
func SkipSlow(t testing.T) {
if os.Getenv("NOMAD_SLOW_TEST") == "" {
t.Skip("Skipping slow test. Set NOMAD_SLOW_TEST=1 to run.")
}
}

227
vendor/github.com/hashicorp/nomad/testutil/vault.go generated vendored Normal file
View File

@@ -0,0 +1,227 @@
package testutil
import (
"fmt"
"math/rand"
"os"
"os/exec"
"time"
"github.com/hashicorp/consul/lib/freeport"
"github.com/hashicorp/nomad/helper/testlog"
"github.com/hashicorp/nomad/helper/uuid"
"github.com/hashicorp/nomad/nomad/structs/config"
vapi "github.com/hashicorp/vault/api"
"github.com/mitchellh/go-testing-interface"
)
// TestVault is a test helper. It uses a fork/exec model to create a test Vault
// server instance in the background and can be initialized with policies, roles
// and backends mounted. The test Vault instances can be used to run a unit test
// and offers and easy API to tear itself down on test end. The only
// prerequisite is that the Vault binary is on the $PATH.
// TestVault wraps a test Vault server launched in dev mode, suitable for
// testing.
type TestVault struct {
cmd *exec.Cmd
t testing.T
waitCh chan error
Addr string
HTTPAddr string
RootToken string
Config *config.VaultConfig
Client *vapi.Client
}
func NewTestVaultFromPath(t testing.T, binary string) *TestVault {
for i := 10; i >= 0; i-- {
port := freeport.GetT(t, 1)[0]
token := uuid.Generate()
bind := fmt.Sprintf("-dev-listen-address=127.0.0.1:%d", port)
http := fmt.Sprintf("http://127.0.0.1:%d", port)
root := fmt.Sprintf("-dev-root-token-id=%s", token)
cmd := exec.Command(binary, "server", "-dev", bind, root)
cmd.Stdout = testlog.NewWriter(t)
cmd.Stderr = testlog.NewWriter(t)
// Build the config
conf := vapi.DefaultConfig()
conf.Address = http
// Make the client and set the token to the root token
client, err := vapi.NewClient(conf)
if err != nil {
t.Fatalf("failed to build Vault API client: %v", err)
}
client.SetToken(token)
enable := true
tv := &TestVault{
cmd: cmd,
t: t,
Addr: bind,
HTTPAddr: http,
RootToken: token,
Client: client,
Config: &config.VaultConfig{
Enabled: &enable,
Token: token,
Addr: http,
},
}
if err := tv.cmd.Start(); err != nil {
tv.t.Fatalf("failed to start vault: %v", err)
}
// Start the waiter
tv.waitCh = make(chan error, 1)
go func() {
err := tv.cmd.Wait()
tv.waitCh <- err
}()
// Ensure Vault started
var startErr error
select {
case startErr = <-tv.waitCh:
case <-time.After(time.Duration(500*TestMultiplier()) * time.Millisecond):
}
if startErr != nil && i == 0 {
t.Fatalf("failed to start vault: %v", startErr)
} else if startErr != nil {
wait := time.Duration(rand.Int31n(2000)) * time.Millisecond
time.Sleep(wait)
continue
}
waitErr := tv.waitForAPI()
if waitErr != nil && i == 0 {
t.Fatalf("failed to start vault: %v", waitErr)
} else if waitErr != nil {
wait := time.Duration(rand.Int31n(2000)) * time.Millisecond
time.Sleep(wait)
continue
}
return tv
}
return nil
}
// NewTestVault returns a new TestVault instance that has yet to be started
func NewTestVault(t testing.T) *TestVault {
// Lookup vault from the path
return NewTestVaultFromPath(t, "vault")
}
// NewTestVaultDelayed returns a test Vault server that has not been started.
// Start must be called and it is the callers responsibility to deal with any
// port conflicts that may occur and retry accordingly.
func NewTestVaultDelayed(t testing.T) *TestVault {
port := freeport.GetT(t, 1)[0]
token := uuid.Generate()
bind := fmt.Sprintf("-dev-listen-address=127.0.0.1:%d", port)
http := fmt.Sprintf("http://127.0.0.1:%d", port)
root := fmt.Sprintf("-dev-root-token-id=%s", token)
cmd := exec.Command("vault", "server", "-dev", bind, root)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
// Build the config
conf := vapi.DefaultConfig()
conf.Address = http
// Make the client and set the token to the root token
client, err := vapi.NewClient(conf)
if err != nil {
t.Fatalf("failed to build Vault API client: %v", err)
}
client.SetToken(token)
enable := true
tv := &TestVault{
cmd: cmd,
t: t,
Addr: bind,
HTTPAddr: http,
RootToken: token,
Client: client,
Config: &config.VaultConfig{
Enabled: &enable,
Token: token,
Addr: http,
},
}
return tv
}
// Start starts the test Vault server and waits for it to respond to its HTTP
// API
func (tv *TestVault) Start() error {
if err := tv.cmd.Start(); err != nil {
tv.t.Fatalf("failed to start vault: %v", err)
}
// Start the waiter
tv.waitCh = make(chan error, 1)
go func() {
err := tv.cmd.Wait()
tv.waitCh <- err
}()
// Ensure Vault started
select {
case err := <-tv.waitCh:
return err
case <-time.After(time.Duration(500*TestMultiplier()) * time.Millisecond):
}
return tv.waitForAPI()
}
// Stop stops the test Vault server
func (tv *TestVault) Stop() {
if tv.cmd.Process == nil {
return
}
if err := tv.cmd.Process.Kill(); err != nil {
tv.t.Errorf("err: %s", err)
}
if tv.waitCh != nil {
<-tv.waitCh
}
}
// waitForAPI waits for the Vault HTTP endpoint to start
// responding. This is an indication that the agent has started.
func (tv *TestVault) waitForAPI() error {
var waitErr error
WaitForResult(func() (bool, error) {
inited, err := tv.Client.Sys().InitStatus()
if err != nil {
return false, err
}
return inited, nil
}, func(err error) {
waitErr = err
})
return waitErr
}
// VaultVersion returns the Vault version as a string or an error if it couldn't
// be determined
func VaultVersion() (string, error) {
cmd := exec.Command("vault", "version")
out, err := cmd.Output()
return string(out), err
}

133
vendor/github.com/hashicorp/nomad/testutil/wait.go generated vendored Normal file
View File

@@ -0,0 +1,133 @@
package testutil
import (
"fmt"
"os"
"time"
"github.com/hashicorp/nomad/nomad/structs"
"github.com/mitchellh/go-testing-interface"
)
const (
// TravisRunEnv is an environment variable that is set if being run by
// Travis.
TravisRunEnv = "CI"
)
type testFn func() (bool, error)
type errorFn func(error)
func WaitForResult(test testFn, error errorFn) {
WaitForResultRetries(500*TestMultiplier(), test, error)
}
func WaitForResultRetries(retries int64, test testFn, error errorFn) {
for retries > 0 {
time.Sleep(10 * time.Millisecond)
retries--
success, err := test()
if success {
return
}
if retries == 0 {
error(err)
}
}
}
// AssertUntil asserts the test function passes throughout the given duration.
// Otherwise error is called on failure.
func AssertUntil(until time.Duration, test testFn, error errorFn) {
deadline := time.Now().Add(until)
for time.Now().Before(deadline) {
success, err := test()
if !success {
error(err)
return
}
// Sleep some arbitrary fraction of the deadline
time.Sleep(until / 30)
}
}
// TestMultiplier returns a multiplier for retries and waits given environment
// the tests are being run under.
func TestMultiplier() int64 {
if IsTravis() {
return 4
}
return 1
}
// Timeout takes the desired timeout and increases it if running in Travis
func Timeout(original time.Duration) time.Duration {
return original * time.Duration(TestMultiplier())
}
func IsTravis() bool {
_, ok := os.LookupEnv(TravisRunEnv)
return ok
}
type rpcFn func(string, interface{}, interface{}) error
// WaitForLeader blocks until a leader is elected.
func WaitForLeader(t testing.T, rpc rpcFn) {
WaitForResult(func() (bool, error) {
args := &structs.GenericRequest{}
var leader string
err := rpc("Status.Leader", args, &leader)
return leader != "", err
}, func(err error) {
t.Fatalf("failed to find leader: %v", err)
})
}
// WaitForRunning runs a job and blocks until it is running.
func WaitForRunning(t testing.T, rpc rpcFn, job *structs.Job) {
registered := false
WaitForResult(func() (bool, error) {
if !registered {
args := &structs.JobRegisterRequest{}
args.Job = job
args.WriteRequest.Region = "global"
var jobResp structs.JobRegisterResponse
err := rpc("Job.Register", args, &jobResp)
if err != nil {
return false, fmt.Errorf("Job.Register error: %v", err)
}
// Only register once
registered = true
}
args := &structs.JobSummaryRequest{}
args.JobID = job.ID
args.QueryOptions.Region = "global"
var resp structs.JobSummaryResponse
err := rpc("Job.Summary", args, &resp)
if err != nil {
return false, fmt.Errorf("Job.Summary error: %v", err)
}
tgs := len(job.TaskGroups)
summaries := len(resp.JobSummary.Summary)
if tgs != summaries {
return false, fmt.Errorf("task_groups=%d summaries=%d", tgs, summaries)
}
for tg, summary := range resp.JobSummary.Summary {
if summary.Running == 0 {
return false, fmt.Errorf("task_group=%s %#v", tg, resp.JobSummary.Summary)
}
}
return true, nil
}, func(err error) {
t.Fatalf("job not running: %v", err)
})
}

10
vendor/github.com/hashicorp/nomad/website/LICENSE.md generated vendored Normal file
View File

@@ -0,0 +1,10 @@
# Proprietary License
This license is temporary while a more official one is drafted. However,
this should make it clear:
The text contents of this website are MPL 2.0 licensed.
The design contents of this website are proprietary and may not be reproduced
or reused in any way other than to run the website locally. The license for
the design is owned solely by HashiCorp, Inc.