Fix the dependency issue (#231)

This commit is contained in:
Robbie Zhang
2018-06-21 12:09:42 -07:00
committed by GitHub
parent 027b76651d
commit 6ec1098bb8
16629 changed files with 74837 additions and 4975021 deletions

View File

@@ -1,21 +0,0 @@
### HCL Template
```hcl
# Place your HCL configuration file here
```
### Expected behavior
What should have happened?
### Actual behavior
What actually happened?
### Steps to reproduce
1.
2.
3.
### References
Are there any other GitHub issues (open or closed) that should
be linked here? For example:
- GH-1234
- ...

View File

@@ -1,9 +0,0 @@
y.output
# ignore intellij files
.idea
*.iml
*.ipr
*.iws
*.test

View File

@@ -1,13 +0,0 @@
sudo: false
language: go
go:
- 1.x
- tip
branches:
only:
- master
script: make test

View File

@@ -1,18 +0,0 @@
TEST?=./...
default: test
fmt: generate
go fmt ./...
test: generate
go get -t ./...
go test $(TEST) $(TESTARGS)
generate:
go generate ./...
updatedeps:
go get -u golang.org/x/tools/cmd/stringer
.PHONY: default generate test updatedeps

View File

@@ -1,125 +0,0 @@
# HCL
[![GoDoc](https://godoc.org/github.com/hashicorp/hcl?status.png)](https://godoc.org/github.com/hashicorp/hcl) [![Build Status](https://travis-ci.org/hashicorp/hcl.svg?branch=master)](https://travis-ci.org/hashicorp/hcl)
HCL (HashiCorp Configuration Language) is a configuration language built
by HashiCorp. The goal of HCL is to build a structured configuration language
that is both human and machine friendly for use with command-line tools, but
specifically targeted towards DevOps tools, servers, etc.
HCL is also fully JSON compatible. That is, JSON can be used as completely
valid input to a system expecting HCL. This helps makes systems
interoperable with other systems.
HCL is heavily inspired by
[libucl](https://github.com/vstakhov/libucl),
nginx configuration, and others similar.
## Why?
A common question when viewing HCL is to ask the question: why not
JSON, YAML, etc.?
Prior to HCL, the tools we built at [HashiCorp](http://www.hashicorp.com)
used a variety of configuration languages from full programming languages
such as Ruby to complete data structure languages such as JSON. What we
learned is that some people wanted human-friendly configuration languages
and some people wanted machine-friendly languages.
JSON fits a nice balance in this, but is fairly verbose and most
importantly doesn't support comments. With YAML, we found that beginners
had a really hard time determining what the actual structure was, and
ended up guessing more often than not whether to use a hyphen, colon, etc.
in order to represent some configuration key.
Full programming languages such as Ruby enable complex behavior
a configuration language shouldn't usually allow, and also forces
people to learn some set of Ruby.
Because of this, we decided to create our own configuration language
that is JSON-compatible. Our configuration language (HCL) is designed
to be written and modified by humans. The API for HCL allows JSON
as an input so that it is also machine-friendly (machines can generate
JSON instead of trying to generate HCL).
Our goal with HCL is not to alienate other configuration languages.
It is instead to provide HCL as a specialized language for our tools,
and JSON as the interoperability layer.
## Syntax
For a complete grammar, please see the parser itself. A high-level overview
of the syntax and grammar is listed here.
* Single line comments start with `#` or `//`
* Multi-line comments are wrapped in `/*` and `*/`. Nested block comments
are not allowed. A multi-line comment (also known as a block comment)
terminates at the first `*/` found.
* Values are assigned with the syntax `key = value` (whitespace doesn't
matter). The value can be any primitive: a string, number, boolean,
object, or list.
* Strings are double-quoted and can contain any UTF-8 characters.
Example: `"Hello, World"`
* Multi-line strings start with `<<EOF` at the end of a line, and end
with `EOF` on its own line ([here documents](https://en.wikipedia.org/wiki/Here_document)).
Any text may be used in place of `EOF`. Example:
```
<<FOO
hello
world
FOO
```
* Numbers are assumed to be base 10. If you prefix a number with 0x,
it is treated as a hexadecimal. If it is prefixed with 0, it is
treated as an octal. Numbers can be in scientific notation: "1e10".
* Boolean values: `true`, `false`
* Arrays can be made by wrapping it in `[]`. Example:
`["foo", "bar", 42]`. Arrays can contain primitives,
other arrays, and objects. As an alternative, lists
of objects can be created with repeated blocks, using
this structure:
```hcl
service {
key = "value"
}
service {
key = "value"
}
```
Objects and nested objects are created using the structure shown below:
```
variable "ami" {
description = "the AMI to use"
}
```
This would be equivalent to the following json:
``` json
{
"variable": {
"ami": {
"description": "the AMI to use"
}
}
}
```
## Thanks
Thanks to:
* [@vstakhov](https://github.com/vstakhov) - The original libucl parser
and syntax that HCL was based off of.
* [@fatih](https://github.com/fatih) - The rewritten HCL parser
in pure Go (no goyacc) and support for a printer.

View File

@@ -1,19 +0,0 @@
version: "build-{branch}-{build}"
image: Visual Studio 2015
clone_folder: c:\gopath\src\github.com\hashicorp\hcl
environment:
GOPATH: c:\gopath
init:
- git config --global core.autocrlf false
install:
- cmd: >-
echo %Path%
go version
go env
go get -t ./...
build_script:
- cmd: go test -v ./...

File diff suppressed because it is too large Load Diff

View File

@@ -1,200 +0,0 @@
package ast
import (
"reflect"
"strings"
"testing"
"github.com/hashicorp/hcl/hcl/token"
)
func TestObjectListFilter(t *testing.T) {
var cases = []struct {
Filter []string
Input []*ObjectItem
Output []*ObjectItem
}{
{
[]string{"foo"},
[]*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{
Token: token.Token{Type: token.STRING, Text: `"foo"`},
},
},
},
},
[]*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{},
},
},
},
{
[]string{"foo"},
[]*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"foo"`}},
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"bar"`}},
},
},
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"baz"`}},
},
},
},
[]*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"bar"`}},
},
},
},
},
}
for _, tc := range cases {
input := &ObjectList{Items: tc.Input}
expected := &ObjectList{Items: tc.Output}
if actual := input.Filter(tc.Filter...); !reflect.DeepEqual(actual, expected) {
t.Fatalf("in order: input, expected, actual\n\n%#v\n\n%#v\n\n%#v", input, expected, actual)
}
}
}
func TestWalk(t *testing.T) {
items := []*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"foo"`}},
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"bar"`}},
},
Val: &LiteralType{Token: token.Token{Type: token.STRING, Text: `"example"`}},
},
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"baz"`}},
},
},
}
node := &ObjectList{Items: items}
order := []string{
"*ast.ObjectList",
"*ast.ObjectItem",
"*ast.ObjectKey",
"*ast.ObjectKey",
"*ast.LiteralType",
"*ast.ObjectItem",
"*ast.ObjectKey",
}
count := 0
Walk(node, func(n Node) (Node, bool) {
if n == nil {
return n, false
}
typeName := reflect.TypeOf(n).String()
if order[count] != typeName {
t.Errorf("expected '%s' got: '%s'", order[count], typeName)
}
count++
return n, true
})
}
func TestWalkEquality(t *testing.T) {
items := []*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"foo"`}},
},
},
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"bar"`}},
},
},
}
node := &ObjectList{Items: items}
rewritten := Walk(node, func(n Node) (Node, bool) { return n, true })
newNode, ok := rewritten.(*ObjectList)
if !ok {
t.Fatalf("expected Objectlist, got %T", rewritten)
}
if !reflect.DeepEqual(node, newNode) {
t.Fatal("rewritten node is not equal to the given node")
}
if len(newNode.Items) != 2 {
t.Error("expected newNode length 2, got: %d", len(newNode.Items))
}
expected := []string{
`"foo"`,
`"bar"`,
}
for i, item := range newNode.Items {
if len(item.Keys) != 1 {
t.Error("expected keys newNode length 1, got: %d", len(item.Keys))
}
if item.Keys[0].Token.Text != expected[i] {
t.Errorf("expected key %s, got %s", expected[i], item.Keys[0].Token.Text)
}
if item.Val != nil {
t.Errorf("expected item value should be nil")
}
}
}
func TestWalkRewrite(t *testing.T) {
items := []*ObjectItem{
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"foo"`}},
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"bar"`}},
},
},
&ObjectItem{
Keys: []*ObjectKey{
&ObjectKey{Token: token.Token{Type: token.STRING, Text: `"baz"`}},
},
},
}
node := &ObjectList{Items: items}
suffix := "_example"
node = Walk(node, func(n Node) (Node, bool) {
switch i := n.(type) {
case *ObjectKey:
i.Token.Text = i.Token.Text + suffix
n = i
}
return n, true
}).(*ObjectList)
Walk(node, func(n Node) (Node, bool) {
switch i := n.(type) {
case *ObjectKey:
if !strings.HasSuffix(i.Token.Text, suffix) {
t.Errorf("Token '%s' should have suffix: %s", i.Token.Text, suffix)
}
}
return n, true
})
}

View File

@@ -1,162 +0,0 @@
// Derivative work from:
// - https://golang.org/src/cmd/gofmt/gofmt.go
// - https://github.com/fatih/hclfmt
package fmtcmd
import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/hashicorp/hcl/hcl/printer"
)
var (
ErrWriteStdin = errors.New("cannot use write option with standard input")
)
type Options struct {
List bool // list files whose formatting differs
Write bool // write result to (source) file instead of stdout
Diff bool // display diffs of formatting changes
}
func isValidFile(f os.FileInfo, extensions []string) bool {
if !f.IsDir() && !strings.HasPrefix(f.Name(), ".") {
for _, ext := range extensions {
if strings.HasSuffix(f.Name(), "."+ext) {
return true
}
}
}
return false
}
// If in == nil, the source is the contents of the file with the given filename.
func processFile(filename string, in io.Reader, out io.Writer, stdin bool, opts Options) error {
if in == nil {
f, err := os.Open(filename)
if err != nil {
return err
}
defer f.Close()
in = f
}
src, err := ioutil.ReadAll(in)
if err != nil {
return err
}
res, err := printer.Format(src)
if err != nil {
return fmt.Errorf("In %s: %s", filename, err)
}
if !bytes.Equal(src, res) {
// formatting has changed
if opts.List {
fmt.Fprintln(out, filename)
}
if opts.Write {
err = ioutil.WriteFile(filename, res, 0644)
if err != nil {
return err
}
}
if opts.Diff {
data, err := diff(src, res)
if err != nil {
return fmt.Errorf("computing diff: %s", err)
}
fmt.Fprintf(out, "diff a/%s b/%s\n", filename, filename)
out.Write(data)
}
}
if !opts.List && !opts.Write && !opts.Diff {
_, err = out.Write(res)
}
return err
}
func walkDir(path string, extensions []string, stdout io.Writer, opts Options) error {
visitFile := func(path string, f os.FileInfo, err error) error {
if err == nil && isValidFile(f, extensions) {
err = processFile(path, nil, stdout, false, opts)
}
return err
}
return filepath.Walk(path, visitFile)
}
func Run(
paths, extensions []string,
stdin io.Reader,
stdout io.Writer,
opts Options,
) error {
if len(paths) == 0 {
if opts.Write {
return ErrWriteStdin
}
if err := processFile("<standard input>", stdin, stdout, true, opts); err != nil {
return err
}
return nil
}
for _, path := range paths {
switch dir, err := os.Stat(path); {
case err != nil:
return err
case dir.IsDir():
if err := walkDir(path, extensions, stdout, opts); err != nil {
return err
}
default:
if err := processFile(path, nil, stdout, false, opts); err != nil {
return err
}
}
}
return nil
}
func diff(b1, b2 []byte) (data []byte, err error) {
f1, err := ioutil.TempFile("", "")
if err != nil {
return
}
defer os.Remove(f1.Name())
defer f1.Close()
f2, err := ioutil.TempFile("", "")
if err != nil {
return
}
defer os.Remove(f2.Name())
defer f2.Close()
f1.Write(b1)
f2.Write(b2)
data, err = exec.Command("diff", "-u", f1.Name(), f2.Name()).CombinedOutput()
if len(data) > 0 {
// diff exits with a non-zero status when the files don't match.
// Ignore that failure as long as we get output.
err = nil
}
return
}

View File

@@ -1,440 +0,0 @@
// +build !windows
// TODO(jen20): These need fixing on Windows but fmt is not used right now
// and red CI is making it harder to process other bugs, so ignore until
// we get around to fixing them.
package fmtcmd
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"reflect"
"regexp"
"sort"
"syscall"
"testing"
"github.com/hashicorp/hcl/testhelper"
)
var fixtureExtensions = []string{"hcl"}
func init() {
sort.Sort(ByFilename(fixtures))
}
func TestIsValidFile(t *testing.T) {
const fixtureDir = "./test-fixtures"
cases := []struct {
Path string
Expected bool
}{
{"good.hcl", true},
{".hidden.ignore", false},
{"file.ignore", false},
{"dir.ignore", false},
}
for _, tc := range cases {
file, err := os.Stat(filepath.Join(fixtureDir, tc.Path))
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if res := isValidFile(file, fixtureExtensions); res != tc.Expected {
t.Errorf("want: %b, got: %b", tc.Expected, res)
}
}
}
func TestRunMultiplePaths(t *testing.T) {
path1, err := renderFixtures("")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path1)
path2, err := renderFixtures("")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path2)
var expectedOut bytes.Buffer
for _, path := range []string{path1, path2} {
for _, fixture := range fixtures {
if !bytes.Equal(fixture.golden, fixture.input) {
expectedOut.WriteString(filepath.Join(path, fixture.filename) + "\n")
}
}
}
_, stdout := mockIO()
err = Run(
[]string{path1, path2},
fixtureExtensions,
nil, stdout,
Options{
List: true,
},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if stdout.String() != expectedOut.String() {
t.Errorf("stdout want:\n%s\ngot:\n%s", expectedOut, stdout)
}
}
func TestRunSubDirectories(t *testing.T) {
pathParent, err := ioutil.TempDir("", "")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(pathParent)
path1, err := renderFixtures(pathParent)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
path2, err := renderFixtures(pathParent)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
paths := []string{path1, path2}
sort.Strings(paths)
var expectedOut bytes.Buffer
for _, path := range paths {
for _, fixture := range fixtures {
if !bytes.Equal(fixture.golden, fixture.input) {
expectedOut.WriteString(filepath.Join(path, fixture.filename) + "\n")
}
}
}
_, stdout := mockIO()
err = Run(
[]string{pathParent},
fixtureExtensions,
nil, stdout,
Options{
List: true,
},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if stdout.String() != expectedOut.String() {
t.Errorf("stdout want:\n%s\ngot:\n%s", expectedOut, stdout)
}
}
func TestRunStdin(t *testing.T) {
var expectedOut bytes.Buffer
for i, fixture := range fixtures {
if i != 0 {
expectedOut.WriteString("\n")
}
expectedOut.Write(fixture.golden)
}
stdin, stdout := mockIO()
for _, fixture := range fixtures {
stdin.Write(fixture.input)
}
err := Run(
[]string{},
fixtureExtensions,
stdin, stdout,
Options{},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if !bytes.Equal(stdout.Bytes(), expectedOut.Bytes()) {
t.Errorf("stdout want:\n%s\ngot:\n%s", expectedOut, stdout)
}
}
func TestRunStdinAndWrite(t *testing.T) {
var expectedOut = []byte{}
stdin, stdout := mockIO()
stdin.WriteString("")
err := Run(
[]string{}, []string{},
stdin, stdout,
Options{
Write: true,
},
)
if err != ErrWriteStdin {
t.Errorf("error want:\n%s\ngot:\n%s", ErrWriteStdin, err)
}
if !bytes.Equal(stdout.Bytes(), expectedOut) {
t.Errorf("stdout want:\n%s\ngot:\n%s", expectedOut, stdout)
}
}
func TestRunFileError(t *testing.T) {
path, err := ioutil.TempDir("", "")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path)
filename := filepath.Join(path, "unreadable.hcl")
var expectedError = &os.PathError{
Op: "open",
Path: filename,
Err: syscall.EACCES,
}
err = ioutil.WriteFile(filename, []byte{}, 0000)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
_, stdout := mockIO()
err = Run(
[]string{path},
fixtureExtensions,
nil, stdout,
Options{},
)
if !reflect.DeepEqual(err, expectedError) {
t.Errorf("error want: %#v, got: %#v", expectedError, err)
}
}
func TestRunNoOptions(t *testing.T) {
path, err := renderFixtures("")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path)
var expectedOut bytes.Buffer
for _, fixture := range fixtures {
expectedOut.Write(fixture.golden)
}
_, stdout := mockIO()
err = Run(
[]string{path},
fixtureExtensions,
nil, stdout,
Options{},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if stdout.String() != expectedOut.String() {
t.Errorf("stdout want:\n%s\ngot:\n%s", expectedOut, stdout)
}
}
func TestRunList(t *testing.T) {
path, err := renderFixtures("")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path)
var expectedOut bytes.Buffer
for _, fixture := range fixtures {
if !bytes.Equal(fixture.golden, fixture.input) {
expectedOut.WriteString(fmt.Sprintln(filepath.Join(path, fixture.filename)))
}
}
_, stdout := mockIO()
err = Run(
[]string{path},
fixtureExtensions,
nil, stdout,
Options{
List: true,
},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if stdout.String() != expectedOut.String() {
t.Errorf("stdout want:\n%s\ngot:\n%s", expectedOut, stdout)
}
}
func TestRunWrite(t *testing.T) {
path, err := renderFixtures("")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path)
_, stdout := mockIO()
err = Run(
[]string{path},
fixtureExtensions,
nil, stdout,
Options{
Write: true,
},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
for _, fixture := range fixtures {
res, err := ioutil.ReadFile(filepath.Join(path, fixture.filename))
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if !bytes.Equal(res, fixture.golden) {
t.Errorf("file %q contents want:\n%s\ngot:\n%s", fixture.filename, fixture.golden, res)
}
}
}
func TestRunDiff(t *testing.T) {
path, err := renderFixtures("")
if err != nil {
t.Errorf("unexpected error: %s", err)
}
defer os.RemoveAll(path)
var expectedOut bytes.Buffer
for _, fixture := range fixtures {
if len(fixture.diff) > 0 {
expectedOut.WriteString(
regexp.QuoteMeta(
fmt.Sprintf("diff a/%s/%s b/%s/%s\n", path, fixture.filename, path, fixture.filename),
),
)
// Need to use regex to ignore datetimes in diff.
expectedOut.WriteString(`--- .+?\n`)
expectedOut.WriteString(`\+\+\+ .+?\n`)
expectedOut.WriteString(regexp.QuoteMeta(string(fixture.diff)))
}
}
expectedOutString := testhelper.Unix2dos(expectedOut.String())
_, stdout := mockIO()
err = Run(
[]string{path},
fixtureExtensions,
nil, stdout,
Options{
Diff: true,
},
)
if err != nil {
t.Errorf("unexpected error: %s", err)
}
if !regexp.MustCompile(expectedOutString).Match(stdout.Bytes()) {
t.Errorf("stdout want match:\n%s\ngot:\n%q", expectedOutString, stdout)
}
}
func mockIO() (stdin, stdout *bytes.Buffer) {
return new(bytes.Buffer), new(bytes.Buffer)
}
type fixture struct {
filename string
input, golden, diff []byte
}
type ByFilename []fixture
func (s ByFilename) Len() int { return len(s) }
func (s ByFilename) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
func (s ByFilename) Less(i, j int) bool { return len(s[i].filename) > len(s[j].filename) }
var fixtures = []fixture{
{
"noop.hcl",
[]byte(`resource "aws_security_group" "firewall" {
count = 5
}
`),
[]byte(`resource "aws_security_group" "firewall" {
count = 5
}
`),
[]byte(``),
}, {
"align_equals.hcl",
[]byte(`variable "foo" {
default = "bar"
description = "bar"
}
`),
[]byte(`variable "foo" {
default = "bar"
description = "bar"
}
`),
[]byte(`@@ -1,4 +1,4 @@
variable "foo" {
- default = "bar"
+ default = "bar"
description = "bar"
}
`),
}, {
"indentation.hcl",
[]byte(`provider "aws" {
access_key = "foo"
secret_key = "bar"
}
`),
[]byte(`provider "aws" {
access_key = "foo"
secret_key = "bar"
}
`),
[]byte(`@@ -1,4 +1,4 @@
provider "aws" {
- access_key = "foo"
- secret_key = "bar"
+ access_key = "foo"
+ secret_key = "bar"
}
`),
},
}
// parent can be an empty string, in which case the system's default
// temporary directory will be used.
func renderFixtures(parent string) (path string, err error) {
path, err = ioutil.TempDir(parent, "")
if err != nil {
return "", err
}
for _, fixture := range fixtures {
err = ioutil.WriteFile(filepath.Join(path, fixture.filename), []byte(fixture.input), 0644)
if err != nil {
os.RemoveAll(path)
return "", err
}
}
return path, nil
}

View File

@@ -1 +0,0 @@
invalid

View File

@@ -1 +0,0 @@
invalid

View File

@@ -1,9 +0,0 @@
package parser
import (
"testing"
)
func TestPosError_impl(t *testing.T) {
var _ error = new(PosError)
}

View File

@@ -205,6 +205,12 @@ func (p *Parser) objectItem() (*ast.ObjectItem, error) {
}
}
// key=#comment
// val
if p.lineComment != nil {
o.LineComment, p.lineComment = p.lineComment, nil
}
// do a look-ahead for line comment
p.scan()
if len(keys) > 0 && o.Val.Pos().Line == keys[0].Pos().Line && p.lineComment != nil {

View File

@@ -1,575 +0,0 @@
package parser
import (
"fmt"
"io/ioutil"
"path/filepath"
"reflect"
"runtime"
"strings"
"testing"
"github.com/hashicorp/hcl/hcl/ast"
"github.com/hashicorp/hcl/hcl/token"
)
func TestType(t *testing.T) {
var literals = []struct {
typ token.Type
src string
}{
{token.STRING, `foo = "foo"`},
{token.NUMBER, `foo = 123`},
{token.NUMBER, `foo = -29`},
{token.FLOAT, `foo = 123.12`},
{token.FLOAT, `foo = -123.12`},
{token.BOOL, `foo = true`},
{token.HEREDOC, "foo = <<EOF\nHello\nWorld\nEOF"},
}
for _, l := range literals {
p := newParser([]byte(l.src))
item, err := p.objectItem()
if err != nil {
t.Error(err)
}
lit, ok := item.Val.(*ast.LiteralType)
if !ok {
t.Errorf("node should be of type LiteralType, got: %T", item.Val)
}
if lit.Token.Type != l.typ {
t.Errorf("want: %s, got: %s", l.typ, lit.Token.Type)
}
}
}
func TestListType(t *testing.T) {
var literals = []struct {
src string
tokens []token.Type
}{
{
`foo = ["123", 123]`,
[]token.Type{token.STRING, token.NUMBER},
},
{
`foo = [123, "123",]`,
[]token.Type{token.NUMBER, token.STRING},
},
{
`foo = [false]`,
[]token.Type{token.BOOL},
},
{
`foo = []`,
[]token.Type{},
},
{
`foo = [1,
"string",
<<EOF
heredoc contents
EOF
]`,
[]token.Type{token.NUMBER, token.STRING, token.HEREDOC},
},
}
for _, l := range literals {
p := newParser([]byte(l.src))
item, err := p.objectItem()
if err != nil {
t.Error(err)
}
list, ok := item.Val.(*ast.ListType)
if !ok {
t.Errorf("node should be of type LiteralType, got: %T", item.Val)
}
tokens := []token.Type{}
for _, li := range list.List {
if tp, ok := li.(*ast.LiteralType); ok {
tokens = append(tokens, tp.Token.Type)
}
}
equals(t, l.tokens, tokens)
}
}
func TestListOfMaps(t *testing.T) {
src := `foo = [
{key = "bar"},
{key = "baz", key2 = "qux"},
]`
p := newParser([]byte(src))
file, err := p.Parse()
if err != nil {
t.Fatalf("err: %s", err)
}
// Here we make all sorts of assumptions about the input structure w/ type
// assertions. The intent is only for this to be a "smoke test" ensuring
// parsing actually performed its duty - giving this test something a bit
// more robust than _just_ "no error occurred".
expected := []string{`"bar"`, `"baz"`, `"qux"`}
actual := make([]string, 0, 3)
ol := file.Node.(*ast.ObjectList)
objItem := ol.Items[0]
list := objItem.Val.(*ast.ListType)
for _, node := range list.List {
obj := node.(*ast.ObjectType)
for _, item := range obj.List.Items {
val := item.Val.(*ast.LiteralType)
actual = append(actual, val.Token.Text)
}
}
if !reflect.DeepEqual(expected, actual) {
t.Fatalf("Expected: %#v, got %#v", expected, actual)
}
}
func TestListOfMaps_requiresComma(t *testing.T) {
src := `foo = [
{key = "bar"}
{key = "baz"}
]`
p := newParser([]byte(src))
_, err := p.Parse()
if err == nil {
t.Fatalf("Expected error, got none!")
}
expected := "error parsing list, expected comma or list end"
if !strings.Contains(err.Error(), expected) {
t.Fatalf("Expected err:\n %s\nTo contain:\n %s\n", err, expected)
}
}
func TestListType_leadComment(t *testing.T) {
var literals = []struct {
src string
comment []string
}{
{
`foo = [
1,
# bar
2,
3,
]`,
[]string{"", "# bar", ""},
},
}
for _, l := range literals {
p := newParser([]byte(l.src))
item, err := p.objectItem()
if err != nil {
t.Fatal(err)
}
list, ok := item.Val.(*ast.ListType)
if !ok {
t.Fatalf("node should be of type LiteralType, got: %T", item.Val)
}
if len(list.List) != len(l.comment) {
t.Fatalf("bad: %d", len(list.List))
}
for i, li := range list.List {
lt := li.(*ast.LiteralType)
comment := l.comment[i]
if (lt.LeadComment == nil) != (comment == "") {
t.Fatalf("bad: %#v", lt)
}
if comment == "" {
continue
}
actual := lt.LeadComment.List[0].Text
if actual != comment {
t.Fatalf("bad: %q %q", actual, comment)
}
}
}
}
func TestListType_lineComment(t *testing.T) {
var literals = []struct {
src string
comment []string
}{
{
`foo = [
1,
2, # bar
3,
]`,
[]string{"", "# bar", ""},
},
}
for _, l := range literals {
p := newParser([]byte(l.src))
item, err := p.objectItem()
if err != nil {
t.Fatal(err)
}
list, ok := item.Val.(*ast.ListType)
if !ok {
t.Fatalf("node should be of type LiteralType, got: %T", item.Val)
}
if len(list.List) != len(l.comment) {
t.Fatalf("bad: %d", len(list.List))
}
for i, li := range list.List {
lt := li.(*ast.LiteralType)
comment := l.comment[i]
if (lt.LineComment == nil) != (comment == "") {
t.Fatalf("bad: %s", lt)
}
if comment == "" {
continue
}
actual := lt.LineComment.List[0].Text
if actual != comment {
t.Fatalf("bad: %q %q", actual, comment)
}
}
}
}
func TestObjectType(t *testing.T) {
var literals = []struct {
src string
nodeType []ast.Node
itemLen int
}{
{
`foo = {}`,
nil,
0,
},
{
`foo = {
bar = "fatih"
}`,
[]ast.Node{&ast.LiteralType{}},
1,
},
{
`foo = {
bar = "fatih"
baz = ["arslan"]
}`,
[]ast.Node{
&ast.LiteralType{},
&ast.ListType{},
},
2,
},
{
`foo = {
bar {}
}`,
[]ast.Node{
&ast.ObjectType{},
},
1,
},
{
`foo {
bar {}
foo = true
}`,
[]ast.Node{
&ast.ObjectType{},
&ast.LiteralType{},
},
2,
},
}
for _, l := range literals {
t.Logf("Source: %s", l.src)
p := newParser([]byte(l.src))
// p.enableTrace = true
item, err := p.objectItem()
if err != nil {
t.Error(err)
continue
}
// we know that the ObjectKey name is foo for all cases, what matters
// is the object
obj, ok := item.Val.(*ast.ObjectType)
if !ok {
t.Errorf("node should be of type LiteralType, got: %T", item.Val)
continue
}
// check if the total length of items are correct
equals(t, l.itemLen, len(obj.List.Items))
// check if the types are correct
for i, item := range obj.List.Items {
equals(t, reflect.TypeOf(l.nodeType[i]), reflect.TypeOf(item.Val))
}
}
}
func TestObjectKey(t *testing.T) {
keys := []struct {
exp []token.Type
src string
}{
{[]token.Type{token.IDENT}, `foo {}`},
{[]token.Type{token.IDENT}, `foo = {}`},
{[]token.Type{token.IDENT}, `foo = bar`},
{[]token.Type{token.IDENT}, `foo = 123`},
{[]token.Type{token.IDENT}, `foo = "${var.bar}`},
{[]token.Type{token.STRING}, `"foo" {}`},
{[]token.Type{token.STRING}, `"foo" = {}`},
{[]token.Type{token.STRING}, `"foo" = "${var.bar}`},
{[]token.Type{token.IDENT, token.IDENT}, `foo bar {}`},
{[]token.Type{token.IDENT, token.STRING}, `foo "bar" {}`},
{[]token.Type{token.STRING, token.IDENT}, `"foo" bar {}`},
{[]token.Type{token.IDENT, token.IDENT, token.IDENT}, `foo bar baz {}`},
}
for _, k := range keys {
p := newParser([]byte(k.src))
keys, err := p.objectKey()
if err != nil {
t.Fatal(err)
}
tokens := []token.Type{}
for _, o := range keys {
tokens = append(tokens, o.Token.Type)
}
equals(t, k.exp, tokens)
}
errKeys := []struct {
src string
}{
{`foo 12 {}`},
{`foo bar = {}`},
{`foo []`},
{`12 {}`},
}
for _, k := range errKeys {
p := newParser([]byte(k.src))
_, err := p.objectKey()
if err == nil {
t.Errorf("case '%s' should give an error", k.src)
}
}
}
func TestCommentGroup(t *testing.T) {
var cases = []struct {
src string
groups int
}{
{"# Hello\n# World", 1},
{"# Hello\r\n# Windows", 1},
}
for _, tc := range cases {
t.Run(tc.src, func(t *testing.T) {
p := newParser([]byte(tc.src))
file, err := p.Parse()
if err != nil {
t.Fatalf("parse error: %s", err)
}
if len(file.Comments) != tc.groups {
t.Fatalf("bad: %#v", file.Comments)
}
})
}
}
// Official HCL tests
func TestParse(t *testing.T) {
cases := []struct {
Name string
Err bool
}{
{
"assign_colon.hcl",
true,
},
{
"comment.hcl",
false,
},
{
"comment_crlf.hcl",
false,
},
{
"comment_lastline.hcl",
false,
},
{
"comment_single.hcl",
false,
},
{
"empty.hcl",
false,
},
{
"list_comma.hcl",
false,
},
{
"multiple.hcl",
false,
},
{
"object_list_comma.hcl",
false,
},
{
"structure.hcl",
false,
},
{
"structure_basic.hcl",
false,
},
{
"structure_empty.hcl",
false,
},
{
"complex.hcl",
false,
},
{
"complex_crlf.hcl",
false,
},
{
"types.hcl",
false,
},
{
"array_comment.hcl",
false,
},
{
"array_comment_2.hcl",
true,
},
{
"missing_braces.hcl",
true,
},
{
"unterminated_object.hcl",
true,
},
{
"unterminated_object_2.hcl",
true,
},
{
"key_without_value.hcl",
true,
},
{
"object_key_without_value.hcl",
true,
},
{
"object_key_assign_without_value.hcl",
true,
},
{
"object_key_assign_without_value2.hcl",
true,
},
{
"object_key_assign_without_value3.hcl",
true,
},
{
"git_crypt.hcl",
true,
},
}
const fixtureDir = "./test-fixtures"
for _, tc := range cases {
t.Run(tc.Name, func(t *testing.T) {
d, err := ioutil.ReadFile(filepath.Join(fixtureDir, tc.Name))
if err != nil {
t.Fatalf("err: %s", err)
}
v, err := Parse(d)
if (err != nil) != tc.Err {
t.Fatalf("Input: %s\n\nError: %s\n\nAST: %#v", tc.Name, err, v)
}
})
}
}
func TestParse_inline(t *testing.T) {
cases := []struct {
Value string
Err bool
}{
{"t t e{{}}", true},
{"o{{}}", true},
{"t t e d N{{}}", true},
{"t t e d{{}}", true},
{"N{}N{{}}", true},
{"v\nN{{}}", true},
{"v=/\n[,", true},
{"v=10kb", true},
{"v=/foo", true},
}
for _, tc := range cases {
t.Logf("Testing: %q", tc.Value)
ast, err := Parse([]byte(tc.Value))
if (err != nil) != tc.Err {
t.Fatalf("Input: %q\n\nError: %s\n\nAST: %#v", tc.Value, err, ast)
}
}
}
// equals fails the test if exp is not equal to act.
func equals(tb testing.TB, exp, act interface{}) {
if !reflect.DeepEqual(exp, act) {
_, file, line, _ := runtime.Caller(1)
fmt.Printf("\033[31m%s:%d:\n\n\texp: %#v\n\n\tgot: %#v\033[39m\n\n", filepath.Base(file), line, exp, act)
tb.FailNow()
}
}

View File

@@ -1,4 +0,0 @@
foo = [
"1",
"2", # comment
]

View File

@@ -1,6 +0,0 @@
provisioner "remote-exec" {
scripts = [
"${path.module}/scripts/install-consul.sh" // missing comma
"${path.module}/scripts/install-haproxy.sh"
]
}

View File

@@ -1,6 +0,0 @@
resource = [{
"foo": {
"bar": {},
"baz": [1, 2, "foo"],
}
}]

View File

@@ -1,5 +0,0 @@
resource = [{
foo = [{
bar = {}
}]
}]

View File

@@ -1,15 +0,0 @@
// Foo
/* Bar */
/*
/*
Baz
*/
# Another
# Multiple
# Lines
foo = "bar"

View File

@@ -1,15 +0,0 @@
// Foo
/* Bar */
/*
/*
Baz
*/
# Another
# Multiple
# Lines
foo = "bar"

View File

@@ -1 +0,0 @@
#foo

View File

@@ -1 +0,0 @@
# Hello

View File

@@ -1,42 +0,0 @@
variable "foo" {
default = "bar"
description = "bar"
}
variable "groups" { }
provider "aws" {
access_key = "foo"
secret_key = "bar"
}
provider "do" {
api_key = "${var.foo}"
}
resource "aws_security_group" "firewall" {
count = 5
}
resource aws_instance "web" {
ami = "${var.foo}"
security_groups = [
"foo",
"${aws_security_group.firewall.foo}",
"${element(split(\",\", var.groups)}",
]
network_interface = {
device_index = 0
description = "Main network interface"
}
}
resource "aws_instance" "db" {
security_groups = "${aws_security_group.firewall.*.id}"
VPC = "foo"
depends_on = ["aws_instance.web"]
}
output "web_ip" {
value = "${aws_instance.web.private_ip}"
}

View File

@@ -1,42 +0,0 @@
variable "foo" {
default = "bar"
description = "bar"
}
variable "groups" { }
provider "aws" {
access_key = "foo"
secret_key = "bar"
}
provider "do" {
api_key = "${var.foo}"
}
resource "aws_security_group" "firewall" {
count = 5
}
resource aws_instance "web" {
ami = "${var.foo}"
security_groups = [
"foo",
"${aws_security_group.firewall.foo}",
"${element(split(\",\", var.groups)}",
]
network_interface = {
device_index = 0
description = "Main network interface"
}
}
resource "aws_instance" "db" {
security_groups = "${aws_security_group.firewall.*.id}"
VPC = "foo"
depends_on = ["aws_instance.web"]
}
output "web_ip" {
value = "${aws_instance.web.private_ip}"
}

View File

@@ -1 +0,0 @@
foo.bar = "baz"

View File

@@ -1 +0,0 @@
foo

View File

@@ -1 +0,0 @@
foo = [1, 2, "foo"]

View File

@@ -1 +0,0 @@
foo = [1, 2, "foo",]

View File

@@ -1,4 +0,0 @@
# should error, but not crash
resource "template_file" "cloud_config" {
template = "$file("${path.module}/some/path")"
}

View File

@@ -1,2 +0,0 @@
foo = "bar"
key = 7

View File

@@ -1,3 +0,0 @@
foo {
bar =
}

View File

@@ -1,4 +0,0 @@
foo {
baz = 7
bar =
}

View File

@@ -1,4 +0,0 @@
foo {
bar =
baz = 7
}

View File

@@ -1,3 +0,0 @@
foo {
bar
}

View File

@@ -1 +0,0 @@
foo = {one = 1, two = 2}

View File

@@ -1,3 +0,0 @@
default = {
"eu-west-1": "ami-b1cf19c6",
}

View File

@@ -1,5 +0,0 @@
// This is a test structure for the lexer
foo bar "baz" {
key = 7
foo = "bar"
}

View File

@@ -1,5 +0,0 @@
foo {
value = 7
"value" = 8
"complex::value" = 9
}

View File

@@ -1 +0,0 @@
resource "foo" "bar" {}

View File

@@ -1,7 +0,0 @@
foo = "bar"
bar = 7
baz = [1,2,3]
foo = -12
bar = 3.14159
foo = true
bar = false

View File

@@ -1,2 +0,0 @@
foo "baz" {
bar = "baz"

View File

@@ -1,6 +0,0 @@
resource "aws_eip" "EIP1" { a { a { a { a { a {
count = "1"
resource "aws_eip" "EIP2" {
count = "1"
}

View File

@@ -252,6 +252,14 @@ func (p *printer) objectItem(o *ast.ObjectItem) []byte {
}
}
// If key and val are on different lines, treat line comments like lead comments.
if o.LineComment != nil && o.Val.Pos().Line != o.Keys[0].Pos().Line {
for _, comment := range o.LineComment.List {
buf.WriteString(comment.Text)
buf.WriteByte(newline)
}
}
for i, k := range o.Keys {
buf.WriteString(k.Token.Text)
buf.WriteByte(blank)
@@ -265,7 +273,7 @@ func (p *printer) objectItem(o *ast.ObjectItem) []byte {
buf.Write(p.output(o.Val))
if o.Val.Pos().Line == o.Keys[0].Pos().Line && o.LineComment != nil {
if o.LineComment != nil && o.Val.Pos().Line == o.Keys[0].Pos().Line {
buf.WriteByte(blank)
for _, comment := range o.LineComment.List {
buf.WriteString(comment.Text)
@@ -509,8 +517,13 @@ func (p *printer) alignedItems(items []*ast.ObjectItem) []byte {
// list returns the printable HCL form of an list type.
func (p *printer) list(l *ast.ListType) []byte {
if p.isSingleLineList(l) {
return p.singleLineList(l)
}
var buf bytes.Buffer
buf.WriteString("[")
buf.WriteByte(newline)
var longestLine int
for _, item := range l.List {
@@ -523,115 +536,112 @@ func (p *printer) list(l *ast.ListType) []byte {
}
}
insertSpaceBeforeItem := false
lastHadLeadComment := false
haveEmptyLine := false
for i, item := range l.List {
// Keep track of whether this item is a heredoc since that has
// unique behavior.
heredoc := false
// If we have a lead comment, then we want to write that first
leadComment := false
if lit, ok := item.(*ast.LiteralType); ok && lit.LeadComment != nil {
leadComment = true
// Ensure an empty line before every element with a
// lead comment (except the first item in a list).
if !haveEmptyLine && i != 0 {
buf.WriteByte(newline)
}
for _, comment := range lit.LeadComment.List {
buf.Write(p.indent([]byte(comment.Text)))
buf.WriteByte(newline)
}
}
// also indent each line
val := p.output(item)
curLen := len(val)
buf.Write(p.indent(val))
// if this item is a heredoc, then we output the comma on
// the next line. This is the only case this happens.
comma := []byte{','}
if lit, ok := item.(*ast.LiteralType); ok && lit.Token.Type == token.HEREDOC {
heredoc = true
}
if item.Pos().Line != l.Lbrack.Line {
// multiline list, add newline before we add each item
buf.WriteByte(newline)
insertSpaceBeforeItem = false
comma = p.indent(comma)
}
// If we have a lead comment, then we want to write that first
leadComment := false
if lit, ok := item.(*ast.LiteralType); ok && lit.LeadComment != nil {
leadComment = true
buf.Write(comma)
// If this isn't the first item and the previous element
// didn't have a lead comment, then we need to add an extra
// newline to properly space things out. If it did have a
// lead comment previously then this would be done
// automatically.
if i > 0 && !lastHadLeadComment {
buf.WriteByte(newline)
}
for _, comment := range lit.LeadComment.List {
buf.Write(p.indent([]byte(comment.Text)))
buf.WriteByte(newline)
}
}
// also indent each line
val := p.output(item)
curLen := len(val)
buf.Write(p.indent(val))
// if this item is a heredoc, then we output the comma on
// the next line. This is the only case this happens.
comma := []byte{','}
if heredoc {
buf.WriteByte(newline)
comma = p.indent(comma)
}
buf.Write(comma)
if lit, ok := item.(*ast.LiteralType); ok && lit.LineComment != nil {
// if the next item doesn't have any comments, do not align
buf.WriteByte(blank) // align one space
for i := 0; i < longestLine-curLen; i++ {
buf.WriteByte(blank)
}
for _, comment := range lit.LineComment.List {
buf.WriteString(comment.Text)
}
}
lastItem := i == len(l.List)-1
if lastItem {
buf.WriteByte(newline)
}
if leadComment && !lastItem {
buf.WriteByte(newline)
}
lastHadLeadComment = leadComment
} else {
if insertSpaceBeforeItem {
if lit, ok := item.(*ast.LiteralType); ok && lit.LineComment != nil {
// if the next item doesn't have any comments, do not align
buf.WriteByte(blank) // align one space
for i := 0; i < longestLine-curLen; i++ {
buf.WriteByte(blank)
insertSpaceBeforeItem = false
}
// Output the item itself
// also indent each line
val := p.output(item)
curLen := len(val)
buf.Write(val)
// If this is a heredoc item we always have to output a newline
// so that it parses properly.
if heredoc {
buf.WriteByte(newline)
}
// If this isn't the last element, write a comma.
if i != len(l.List)-1 {
buf.WriteString(",")
insertSpaceBeforeItem = true
}
if lit, ok := item.(*ast.LiteralType); ok && lit.LineComment != nil {
// if the next item doesn't have any comments, do not align
buf.WriteByte(blank) // align one space
for i := 0; i < longestLine-curLen; i++ {
buf.WriteByte(blank)
}
for _, comment := range lit.LineComment.List {
buf.WriteString(comment.Text)
}
for _, comment := range lit.LineComment.List {
buf.WriteString(comment.Text)
}
}
buf.WriteByte(newline)
// Ensure an empty line after every element with a
// lead comment (except the first item in a list).
haveEmptyLine = leadComment && i != len(l.List)-1
if haveEmptyLine {
buf.WriteByte(newline)
}
}
buf.WriteString("]")
return buf.Bytes()
}
// isSingleLineList returns true if:
// * they were previously formatted entirely on one line
// * they consist entirely of literals
// * there are either no heredoc strings or the list has exactly one element
// * there are no line comments
func (printer) isSingleLineList(l *ast.ListType) bool {
for _, item := range l.List {
if item.Pos().Line != l.Lbrack.Line {
return false
}
lit, ok := item.(*ast.LiteralType)
if !ok {
return false
}
if lit.Token.Type == token.HEREDOC && len(l.List) != 1 {
return false
}
if lit.LineComment != nil {
return false
}
}
return true
}
// singleLineList prints a simple single line list.
// For a definition of "simple", see isSingleLineList above.
func (p *printer) singleLineList(l *ast.ListType) []byte {
buf := &bytes.Buffer{}
buf.WriteString("[")
for i, item := range l.List {
if i != 0 {
buf.WriteString(", ")
}
// Output the item itself
buf.Write(p.output(item))
// The heredoc marker needs to be at the end of line.
if lit, ok := item.(*ast.LiteralType); ok && lit.Token.Type == token.HEREDOC {
buf.WriteByte(newline)
}
}
buf.WriteString("]")

View File

@@ -1,149 +0,0 @@
package printer
import (
"bytes"
"errors"
"flag"
"fmt"
"io/ioutil"
"path/filepath"
"testing"
"github.com/hashicorp/hcl/hcl/parser"
)
var update = flag.Bool("update", false, "update golden files")
const (
dataDir = "testdata"
)
type entry struct {
source, golden string
}
// Use go test -update to create/update the respective golden files.
var data = []entry{
{"complexhcl.input", "complexhcl.golden"},
{"list.input", "list.golden"},
{"list_comment.input", "list_comment.golden"},
{"comment.input", "comment.golden"},
{"comment_crlf.input", "comment.golden"},
{"comment_aligned.input", "comment_aligned.golden"},
{"comment_array.input", "comment_array.golden"},
{"comment_end_file.input", "comment_end_file.golden"},
{"comment_multiline_indent.input", "comment_multiline_indent.golden"},
{"comment_multiline_no_stanza.input", "comment_multiline_no_stanza.golden"},
{"comment_multiline_stanza.input", "comment_multiline_stanza.golden"},
{"comment_newline.input", "comment_newline.golden"},
{"comment_object_multi.input", "comment_object_multi.golden"},
{"comment_standalone.input", "comment_standalone.golden"},
{"empty_block.input", "empty_block.golden"},
{"list_of_objects.input", "list_of_objects.golden"},
{"multiline_string.input", "multiline_string.golden"},
{"object_singleline.input", "object_singleline.golden"},
{"object_with_heredoc.input", "object_with_heredoc.golden"},
}
func TestFiles(t *testing.T) {
for _, e := range data {
source := filepath.Join(dataDir, e.source)
golden := filepath.Join(dataDir, e.golden)
t.Run(e.source, func(t *testing.T) {
check(t, source, golden)
})
}
}
func check(t *testing.T, source, golden string) {
src, err := ioutil.ReadFile(source)
if err != nil {
t.Error(err)
return
}
res, err := format(src)
if err != nil {
t.Error(err)
return
}
// update golden files if necessary
if *update {
if err := ioutil.WriteFile(golden, res, 0644); err != nil {
t.Error(err)
}
return
}
// get golden
gld, err := ioutil.ReadFile(golden)
if err != nil {
t.Error(err)
return
}
// formatted source and golden must be the same
if err := diff(source, golden, res, gld); err != nil {
t.Error(err)
return
}
}
// diff compares a and b.
func diff(aname, bname string, a, b []byte) error {
var buf bytes.Buffer // holding long error message
// compare lengths
if len(a) != len(b) {
fmt.Fprintf(&buf, "\nlength changed: len(%s) = %d, len(%s) = %d", aname, len(a), bname, len(b))
}
// compare contents
line := 1
offs := 1
for i := 0; i < len(a) && i < len(b); i++ {
ch := a[i]
if ch != b[i] {
fmt.Fprintf(&buf, "\n%s:%d:%d: %q", aname, line, i-offs+1, lineAt(a, offs))
fmt.Fprintf(&buf, "\n%s:%d:%d: %q", bname, line, i-offs+1, lineAt(b, offs))
fmt.Fprintf(&buf, "\n\n")
break
}
if ch == '\n' {
line++
offs = i + 1
}
}
if buf.Len() > 0 {
return errors.New(buf.String())
}
return nil
}
// format parses src, prints the corresponding AST, verifies the resulting
// src is syntactically correct, and returns the resulting src or an error
// if any.
func format(src []byte) ([]byte, error) {
formatted, err := Format(src)
if err != nil {
return nil, err
}
// make sure formatted output is syntactically correct
if _, err := parser.Parse(formatted); err != nil {
return nil, fmt.Errorf("parse: %s\n%s", err, formatted)
}
return formatted, nil
}
// lineAt returns the line in text starting at offset offs.
func lineAt(text []byte, offs int) []byte {
i := offs
for i < len(text) && text[i] != '\n' {
i++
}
return text[offs:i]
}

View File

@@ -1,36 +0,0 @@
// A standalone comment is a comment which is not attached to any kind of node
// This comes from Terraform, as a test
variable "foo" {
# Standalone comment should be still here
default = "bar"
description = "bar" # yooo
}
/* This is a multi line standalone
comment*/
// fatih arslan
/* This is a developer test
account and a multine comment */
developer = ["fatih", "arslan"] // fatih arslan
# One line here
numbers = [1, 2] // another line here
# Another comment
variable = {
description = "bar" # another yooo
foo {
# Nested standalone
bar = "fatih"
}
}
// lead comment
foo {
bar = "fatih" // line comment 2
} // line comment 3

View File

@@ -1,37 +0,0 @@
// A standalone comment is a comment which is not attached to any kind of node
// This comes from Terraform, as a test
variable "foo" {
# Standalone comment should be still here
default = "bar"
description = "bar" # yooo
}
/* This is a multi line standalone
comment*/
// fatih arslan
/* This is a developer test
account and a multine comment */
developer = [ "fatih", "arslan"] // fatih arslan
# One line here
numbers = [1,2] // another line here
# Another comment
variable = {
description = "bar" # another yooo
foo {
# Nested standalone
bar = "fatih"
}
}
// lead comment
foo {
bar = "fatih" // line comment 2
} // line comment 3

View File

@@ -1,32 +0,0 @@
aligned {
# We have some aligned items below
foo = "fatih" # yoo1
default = "bar" # yoo2
bar = "bar and foo" # yoo3
default = {
bar = "example"
}
#deneme arslan
fatih = ["fatih"] # yoo4
#fatih arslan
fatiharslan = ["arslan"] // yoo5
default = {
bar = "example"
}
security_groups = [
"foo", # kenya 1
"${aws_security_group.firewall.foo}", # kenya 2
]
security_groups2 = [
"foo", # kenya 1
"bar", # kenya 1.5
"${aws_security_group.firewall.foo}", # kenya 2
"foobar", # kenya 3
]
}

View File

@@ -1,28 +0,0 @@
aligned {
# We have some aligned items below
foo = "fatih" # yoo1
default = "bar" # yoo2
bar = "bar and foo" # yoo3
default = {
bar = "example"
}
#deneme arslan
fatih = ["fatih"] # yoo4
#fatih arslan
fatiharslan = ["arslan"] // yoo5
default = {
bar = "example"
}
security_groups = [
"foo", # kenya 1
"${aws_security_group.firewall.foo}", # kenya 2
]
security_groups2 = [
"foo", # kenya 1
"bar", # kenya 1.5
"${aws_security_group.firewall.foo}", # kenya 2
"foobar", # kenya 3
]
}

View File

@@ -1,13 +0,0 @@
banana = [
# I really want to comment this item in the array.
"a",
# This as well
"b",
"c", # And C
"d",
# And another
"e",
]

View File

@@ -1,13 +0,0 @@
banana = [
# I really want to comment this item in the array.
"a",
# This as well
"b",
"c", # And C
"d",
# And another
"e",
]

View File

@@ -1,37 +0,0 @@
// A standalone comment is a comment which is not attached to any kind of node
// This comes from Terraform, as a test
variable "foo" {
# Standalone comment should be still here
default = "bar"
description = "bar" # yooo
}
/* This is a multi line standalone
comment*/
// fatih arslan
/* This is a developer test
account and a multine comment */
developer = [ "fatih", "arslan"] // fatih arslan
# One line here
numbers = [1,2] // another line here
# Another comment
variable = {
description = "bar" # another yooo
foo {
# Nested standalone
bar = "fatih"
}
}
// lead comment
foo {
bar = "fatih" // line comment 2
} // line comment 3

View File

@@ -1,6 +0,0 @@
resource "blah" "blah" {}
//
//
//

View File

@@ -1,5 +0,0 @@
resource "blah" "blah" {}
//
//
//

View File

@@ -1,12 +0,0 @@
resource "provider" "resource" {
/*
SPACE_SENSITIVE_CODE = <<EOF
yaml code:
foo: ""
bar: ""
EOF
*/
/*
OTHER
*/
}

View File

@@ -1,13 +0,0 @@
resource "provider" "resource" {
/*
SPACE_SENSITIVE_CODE = <<EOF
yaml code:
foo: ""
bar: ""
EOF
*/
/*
OTHER
*/
}

View File

@@ -1,7 +0,0 @@
# This is a multiline comment
# That has values like this:
#
# ami-abcd1234
#
# Do not delete this comment

View File

@@ -1,6 +0,0 @@
# This is a multiline comment
# That has values like this:
#
# ami-abcd1234
#
# Do not delete this comment

View File

@@ -1,10 +0,0 @@
# This is a multiline comment
# That has values like this:
#
# ami-abcd1234
#
# Do not delete this comment
resource "aws_instance" "web" {
ami_id = "ami-abcd1234"
}

View File

@@ -1,10 +0,0 @@
# This is a multiline comment
# That has values like this:
#
# ami-abcd1234
#
# Do not delete this comment
resource "aws_instance" "web" {
ami_id = "ami-abcd1234"
}

View File

@@ -1,3 +0,0 @@
# Hello
# World

View File

@@ -1,2 +0,0 @@
# Hello
# World

View File

@@ -1,9 +0,0 @@
variable "environment" {
default = {}
# default {
# "region" = "us-west-2"
# "sg" = "playground"
# "env" = "prod"
# }
}

View File

@@ -1,9 +0,0 @@
variable "environment" {
default = {}
# default {
# "region" = "us-west-2"
# "sg" = "playground"
# "env" = "prod"
# }
}

View File

@@ -1,17 +0,0 @@
// A standalone comment
aligned {
# Standalone 1
a = "bar" # yoo1
default = "bar" # yoo2
# Standalone 2
}
# Standalone 3
numbers = [1, 2] // another line here
# Standalone 4

View File

@@ -1,16 +0,0 @@
// A standalone comment
aligned {
# Standalone 1
a = "bar" # yoo1
default = "bar" # yoo2
# Standalone 2
}
# Standalone 3
numbers = [1,2] // another line here
# Standalone 4

View File

@@ -1,54 +0,0 @@
variable "foo" {
default = "bar"
description = "bar"
}
developer = ["fatih", "arslan"]
provider "aws" {
access_key = "foo"
secret_key = "bar"
}
provider "do" {
api_key = "${var.foo}"
}
resource "aws_security_group" "firewall" {
count = 5
}
resource aws_instance "web" {
ami = "${var.foo}"
security_groups = [
"foo",
"${aws_security_group.firewall.foo}",
]
network_interface {
device_index = 0
description = "Main network interface"
}
network_interface = {
device_index = 1
description = <<EOF
ANOTHER NETWORK INTERFACE
EOF
}
}
resource "aws_instance" "db" {
security_groups = "${aws_security_group.firewall.*.id}"
VPC = "foo"
depends_on = ["aws_instance.web"]
}
output "web_ip" {
value = <<EOF
TUBES
EOF
}

View File

@@ -1,53 +0,0 @@
variable "foo" {
default = "bar"
description = "bar"
}
developer = [ "fatih", "arslan"]
provider "aws" {
access_key ="foo"
secret_key = "bar"
}
provider "do" {
api_key = "${var.foo}"
}
resource "aws_security_group" "firewall" {
count = 5
}
resource aws_instance "web" {
ami = "${var.foo}"
security_groups = [
"foo",
"${aws_security_group.firewall.foo}"
]
network_interface {
device_index = 0
description = "Main network interface"
}
network_interface = {
device_index = 1
description = <<EOF
ANOTHER NETWORK INTERFACE
EOF
}
}
resource "aws_instance" "db" {
security_groups = "${aws_security_group.firewall.*.id}"
VPC = "foo"
depends_on = ["aws_instance.web"]
}
output "web_ip" {
value=<<EOF
TUBES
EOF
}

View File

@@ -1,12 +0,0 @@
variable "foo" {}
variable "foo" {}
variable "foo" {
# Standalone comment should be still here
}
foo {}
foo {
bar = "mssola"
}

View File

@@ -1,14 +0,0 @@
variable "foo" {}
variable "foo" {
}
variable "foo" {
# Standalone comment should be still here
}
foo {
}
foo {
bar = "mssola"
}

View File

@@ -1,43 +0,0 @@
foo = ["fatih", "arslan"]
foo = ["bar", "qaz"]
foo = ["zeynep",
"arslan",
]
foo = ["fatih", "zeynep",
"arslan",
]
foo = [
"vim-go",
"golang",
"hcl",
]
foo = []
foo = [1, 2, 3, 4]
foo = [
"kenya",
"ethiopia",
"columbia",
]
foo = [
<<EOS
one
EOS
,
<<EOS
two
EOS
,
]
foo = [<<EOS
one
EOS
]

View File

@@ -1,37 +0,0 @@
foo = ["fatih", "arslan" ]
foo = [ "bar", "qaz", ]
foo = [ "zeynep",
"arslan", ]
foo = ["fatih", "zeynep",
"arslan", ]
foo = [
"vim-go",
"golang", "hcl"]
foo = []
foo = [1, 2,3, 4]
foo = [
"kenya", "ethiopia",
"columbia"]
foo = [
<<EOS
one
EOS
,
<<EOS
two
EOS
,
]
foo = [<<EOS
one
EOS
]

View File

@@ -1,7 +0,0 @@
foo = [1, # Hello
2,
]
foo = [1, # Hello
2, # World
]

View File

@@ -1,6 +0,0 @@
foo = [1, # Hello
2]
foo = [1, # Hello
2, # World
]

View File

@@ -1,10 +0,0 @@
list_of_objects = [
{
key1 = "value1"
key2 = "value2"
},
{
key3 = "value3"
key4 = "value4"
},
]

View File

@@ -1,10 +0,0 @@
list_of_objects = [
{
key1 = "value1"
key2 = "value2"
},
{
key3 = "value3"
key4 = "value4"
}
]

View File

@@ -1,7 +0,0 @@
resource "null_resource" "some_command" {
provisioner "local-exec" {
command = "${echo '
some newlines
and additonal output'}"
}
}

View File

@@ -1,7 +0,0 @@
resource "null_resource" "some_command" {
provisioner "local-exec" {
command = "${echo '
some newlines
and additonal output'}"
}
}

View File

@@ -1,26 +0,0 @@
variable "foo" {}
variable "bar" {}
variable "baz" {}
variable "qux" {}
variable "foo" {
foo = "bar"
}
variable "foo" {}
# lead comment
variable "bar" {}
variable "foo" {
default = "bar"
}
variable "bar" {}
# Purposeful newline check below:
variable "foo" {}
variable "purposeful-newline" {}

View File

@@ -1,19 +0,0 @@
variable "foo" {}
variable "bar" {}
variable "baz" {}
variable "qux" {}
variable "foo" { foo = "bar" }
variable "foo" {}
# lead comment
variable "bar" {}
variable "foo" { default = "bar" }
variable "bar" {}
# Purposeful newline check below:
variable "foo" {}
variable "purposeful-newline" {}

View File

@@ -1,6 +0,0 @@
obj {
foo = [<<EOF
TEXT!
EOF
]
}

View File

@@ -1,6 +0,0 @@
obj {
foo = [<<EOF
TEXT!
EOF
]
}

View File

@@ -74,14 +74,6 @@ func (s *Scanner) next() rune {
return eof
}
if ch == utf8.RuneError && size == 1 {
s.srcPos.Column++
s.srcPos.Offset += size
s.lastCharLen = size
s.err("illegal UTF-8 encoding")
return ch
}
// remember last position
s.prevPos = s.srcPos
@@ -89,18 +81,27 @@ func (s *Scanner) next() rune {
s.lastCharLen = size
s.srcPos.Offset += size
if ch == utf8.RuneError && size == 1 {
s.err("illegal UTF-8 encoding")
return ch
}
if ch == '\n' {
s.srcPos.Line++
s.lastLineLen = s.srcPos.Column
s.srcPos.Column = 0
}
// If we see a null character with data left, then that is an error
if ch == '\x00' && s.buf.Len() > 0 {
if ch == '\x00' {
s.err("unexpected null character (0x00)")
return eof
}
if ch == '\uE123' {
s.err("unicode code point U+E123 reserved for internal use")
return utf8.RuneError
}
// debug
// fmt.Printf("ch: %q, offset:column: %d:%d\n", ch, s.srcPos.Offset, s.srcPos.Column)
return ch
@@ -432,16 +433,16 @@ func (s *Scanner) scanHeredoc() {
// Read the identifier
identBytes := s.src[offs : s.srcPos.Offset-s.lastCharLen]
if len(identBytes) == 0 {
if len(identBytes) == 0 || (len(identBytes) == 1 && identBytes[0] == '-') {
s.err("zero-length heredoc anchor")
return
}
var identRegexp *regexp.Regexp
if identBytes[0] == '-' {
identRegexp = regexp.MustCompile(fmt.Sprintf(`[[:space:]]*%s\z`, identBytes[1:]))
identRegexp = regexp.MustCompile(fmt.Sprintf(`^[[:space:]]*%s\r*\z`, identBytes[1:]))
} else {
identRegexp = regexp.MustCompile(fmt.Sprintf(`[[:space:]]*%s\z`, identBytes))
identRegexp = regexp.MustCompile(fmt.Sprintf(`^[[:space:]]*%s\r*\z`, identBytes))
}
// Read the actual string value
@@ -551,7 +552,7 @@ func (s *Scanner) scanDigits(ch rune, base, n int) rune {
s.err("illegal char escape")
}
if n != start {
if n != start && ch != eof {
// we scanned all digits, put the last non digit char back,
// only if we read anything at all
s.unread()

View File

@@ -1,591 +0,0 @@
package scanner
import (
"bytes"
"fmt"
"testing"
"strings"
"github.com/hashicorp/hcl/hcl/token"
)
var f100 = strings.Repeat("f", 100)
type tokenPair struct {
tok token.Type
text string
}
var tokenLists = map[string][]tokenPair{
"comment": []tokenPair{
{token.COMMENT, "//"},
{token.COMMENT, "////"},
{token.COMMENT, "// comment"},
{token.COMMENT, "// /* comment */"},
{token.COMMENT, "// // comment //"},
{token.COMMENT, "//" + f100},
{token.COMMENT, "#"},
{token.COMMENT, "##"},
{token.COMMENT, "# comment"},
{token.COMMENT, "# /* comment */"},
{token.COMMENT, "# # comment #"},
{token.COMMENT, "#" + f100},
{token.COMMENT, "/**/"},
{token.COMMENT, "/***/"},
{token.COMMENT, "/* comment */"},
{token.COMMENT, "/* // comment */"},
{token.COMMENT, "/* /* comment */"},
{token.COMMENT, "/*\n comment\n*/"},
{token.COMMENT, "/*" + f100 + "*/"},
},
"operator": []tokenPair{
{token.LBRACK, "["},
{token.LBRACE, "{"},
{token.COMMA, ","},
{token.PERIOD, "."},
{token.RBRACK, "]"},
{token.RBRACE, "}"},
{token.ASSIGN, "="},
{token.ADD, "+"},
{token.SUB, "-"},
},
"bool": []tokenPair{
{token.BOOL, "true"},
{token.BOOL, "false"},
},
"ident": []tokenPair{
{token.IDENT, "a"},
{token.IDENT, "a0"},
{token.IDENT, "foobar"},
{token.IDENT, "foo-bar"},
{token.IDENT, "abc123"},
{token.IDENT, "LGTM"},
{token.IDENT, "_"},
{token.IDENT, "_abc123"},
{token.IDENT, "abc123_"},
{token.IDENT, "_abc_123_"},
{token.IDENT, "_äöü"},
{token.IDENT, "_本"},
{token.IDENT, "äöü"},
{token.IDENT, "本"},
{token.IDENT, "a۰۱۸"},
{token.IDENT, "foo६४"},
{token.IDENT, "bar"},
},
"heredoc": []tokenPair{
{token.HEREDOC, "<<EOF\nhello\nworld\nEOF"},
{token.HEREDOC, "<<EOF123\nhello\nworld\nEOF123"},
},
"string": []tokenPair{
{token.STRING, `" "`},
{token.STRING, `"a"`},
{token.STRING, `"本"`},
{token.STRING, `"${file("foo")}"`},
{token.STRING, `"${file(\"foo\")}"`},
{token.STRING, `"\a"`},
{token.STRING, `"\b"`},
{token.STRING, `"\f"`},
{token.STRING, `"\n"`},
{token.STRING, `"\r"`},
{token.STRING, `"\t"`},
{token.STRING, `"\v"`},
{token.STRING, `"\""`},
{token.STRING, `"\000"`},
{token.STRING, `"\777"`},
{token.STRING, `"\x00"`},
{token.STRING, `"\xff"`},
{token.STRING, `"\u0000"`},
{token.STRING, `"\ufA16"`},
{token.STRING, `"\U00000000"`},
{token.STRING, `"\U0000ffAB"`},
{token.STRING, `"` + f100 + `"`},
},
"number": []tokenPair{
{token.NUMBER, "0"},
{token.NUMBER, "1"},
{token.NUMBER, "9"},
{token.NUMBER, "42"},
{token.NUMBER, "1234567890"},
{token.NUMBER, "00"},
{token.NUMBER, "01"},
{token.NUMBER, "07"},
{token.NUMBER, "042"},
{token.NUMBER, "01234567"},
{token.NUMBER, "0x0"},
{token.NUMBER, "0x1"},
{token.NUMBER, "0xf"},
{token.NUMBER, "0x42"},
{token.NUMBER, "0x123456789abcDEF"},
{token.NUMBER, "0x" + f100},
{token.NUMBER, "0X0"},
{token.NUMBER, "0X1"},
{token.NUMBER, "0XF"},
{token.NUMBER, "0X42"},
{token.NUMBER, "0X123456789abcDEF"},
{token.NUMBER, "0X" + f100},
{token.NUMBER, "-0"},
{token.NUMBER, "-1"},
{token.NUMBER, "-9"},
{token.NUMBER, "-42"},
{token.NUMBER, "-1234567890"},
{token.NUMBER, "-00"},
{token.NUMBER, "-01"},
{token.NUMBER, "-07"},
{token.NUMBER, "-29"},
{token.NUMBER, "-042"},
{token.NUMBER, "-01234567"},
{token.NUMBER, "-0x0"},
{token.NUMBER, "-0x1"},
{token.NUMBER, "-0xf"},
{token.NUMBER, "-0x42"},
{token.NUMBER, "-0x123456789abcDEF"},
{token.NUMBER, "-0x" + f100},
{token.NUMBER, "-0X0"},
{token.NUMBER, "-0X1"},
{token.NUMBER, "-0XF"},
{token.NUMBER, "-0X42"},
{token.NUMBER, "-0X123456789abcDEF"},
{token.NUMBER, "-0X" + f100},
},
"float": []tokenPair{
{token.FLOAT, "0."},
{token.FLOAT, "1."},
{token.FLOAT, "42."},
{token.FLOAT, "01234567890."},
{token.FLOAT, ".0"},
{token.FLOAT, ".1"},
{token.FLOAT, ".42"},
{token.FLOAT, ".0123456789"},
{token.FLOAT, "0.0"},
{token.FLOAT, "1.0"},
{token.FLOAT, "42.0"},
{token.FLOAT, "01234567890.0"},
{token.FLOAT, "0e0"},
{token.FLOAT, "1e0"},
{token.FLOAT, "42e0"},
{token.FLOAT, "01234567890e0"},
{token.FLOAT, "0E0"},
{token.FLOAT, "1E0"},
{token.FLOAT, "42E0"},
{token.FLOAT, "01234567890E0"},
{token.FLOAT, "0e+10"},
{token.FLOAT, "1e-10"},
{token.FLOAT, "42e+10"},
{token.FLOAT, "01234567890e-10"},
{token.FLOAT, "0E+10"},
{token.FLOAT, "1E-10"},
{token.FLOAT, "42E+10"},
{token.FLOAT, "01234567890E-10"},
{token.FLOAT, "01.8e0"},
{token.FLOAT, "1.4e0"},
{token.FLOAT, "42.2e0"},
{token.FLOAT, "01234567890.12e0"},
{token.FLOAT, "0.E0"},
{token.FLOAT, "1.12E0"},
{token.FLOAT, "42.123E0"},
{token.FLOAT, "01234567890.213E0"},
{token.FLOAT, "0.2e+10"},
{token.FLOAT, "1.2e-10"},
{token.FLOAT, "42.54e+10"},
{token.FLOAT, "01234567890.98e-10"},
{token.FLOAT, "0.1E+10"},
{token.FLOAT, "1.1E-10"},
{token.FLOAT, "42.1E+10"},
{token.FLOAT, "01234567890.1E-10"},
{token.FLOAT, "-0.0"},
{token.FLOAT, "-1.0"},
{token.FLOAT, "-42.0"},
{token.FLOAT, "-01234567890.0"},
{token.FLOAT, "-0e0"},
{token.FLOAT, "-1e0"},
{token.FLOAT, "-42e0"},
{token.FLOAT, "-01234567890e0"},
{token.FLOAT, "-0E0"},
{token.FLOAT, "-1E0"},
{token.FLOAT, "-42E0"},
{token.FLOAT, "-01234567890E0"},
{token.FLOAT, "-0e+10"},
{token.FLOAT, "-1e-10"},
{token.FLOAT, "-42e+10"},
{token.FLOAT, "-01234567890e-10"},
{token.FLOAT, "-0E+10"},
{token.FLOAT, "-1E-10"},
{token.FLOAT, "-42E+10"},
{token.FLOAT, "-01234567890E-10"},
{token.FLOAT, "-01.8e0"},
{token.FLOAT, "-1.4e0"},
{token.FLOAT, "-42.2e0"},
{token.FLOAT, "-01234567890.12e0"},
{token.FLOAT, "-0.E0"},
{token.FLOAT, "-1.12E0"},
{token.FLOAT, "-42.123E0"},
{token.FLOAT, "-01234567890.213E0"},
{token.FLOAT, "-0.2e+10"},
{token.FLOAT, "-1.2e-10"},
{token.FLOAT, "-42.54e+10"},
{token.FLOAT, "-01234567890.98e-10"},
{token.FLOAT, "-0.1E+10"},
{token.FLOAT, "-1.1E-10"},
{token.FLOAT, "-42.1E+10"},
{token.FLOAT, "-01234567890.1E-10"},
},
}
var orderedTokenLists = []string{
"comment",
"operator",
"bool",
"ident",
"heredoc",
"string",
"number",
"float",
}
func TestPosition(t *testing.T) {
// create artifical source code
buf := new(bytes.Buffer)
for _, listName := range orderedTokenLists {
for _, ident := range tokenLists[listName] {
fmt.Fprintf(buf, "\t\t\t\t%s\n", ident.text)
}
}
s := New(buf.Bytes())
pos := token.Pos{"", 4, 1, 5}
s.Scan()
for _, listName := range orderedTokenLists {
for _, k := range tokenLists[listName] {
curPos := s.tokPos
// fmt.Printf("[%q] s = %+v:%+v\n", k.text, curPos.Offset, curPos.Column)
if curPos.Offset != pos.Offset {
t.Fatalf("offset = %d, want %d for %q", curPos.Offset, pos.Offset, k.text)
}
if curPos.Line != pos.Line {
t.Fatalf("line = %d, want %d for %q", curPos.Line, pos.Line, k.text)
}
if curPos.Column != pos.Column {
t.Fatalf("column = %d, want %d for %q", curPos.Column, pos.Column, k.text)
}
pos.Offset += 4 + len(k.text) + 1 // 4 tabs + token bytes + newline
pos.Line += countNewlines(k.text) + 1 // each token is on a new line
s.Scan()
}
}
// make sure there were no token-internal errors reported by scanner
if s.ErrorCount != 0 {
t.Errorf("%d errors", s.ErrorCount)
}
}
func TestNullChar(t *testing.T) {
s := New([]byte("\"\\0"))
s.Scan() // Used to panic
}
func TestComment(t *testing.T) {
testTokenList(t, tokenLists["comment"])
}
func TestOperator(t *testing.T) {
testTokenList(t, tokenLists["operator"])
}
func TestBool(t *testing.T) {
testTokenList(t, tokenLists["bool"])
}
func TestIdent(t *testing.T) {
testTokenList(t, tokenLists["ident"])
}
func TestString(t *testing.T) {
testTokenList(t, tokenLists["string"])
}
func TestNumber(t *testing.T) {
testTokenList(t, tokenLists["number"])
}
func TestFloat(t *testing.T) {
testTokenList(t, tokenLists["float"])
}
func TestWindowsLineEndings(t *testing.T) {
hcl := `// This should have Windows line endings
resource "aws_instance" "foo" {
user_data=<<HEREDOC
test script
HEREDOC
}`
hclWindowsEndings := strings.Replace(hcl, "\n", "\r\n", -1)
literals := []struct {
tokenType token.Type
literal string
}{
{token.COMMENT, "// This should have Windows line endings\r"},
{token.IDENT, `resource`},
{token.STRING, `"aws_instance"`},
{token.STRING, `"foo"`},
{token.LBRACE, `{`},
{token.IDENT, `user_data`},
{token.ASSIGN, `=`},
{token.HEREDOC, "<<HEREDOC\r\n test script\r\nHEREDOC\r\n"},
{token.RBRACE, `}`},
}
s := New([]byte(hclWindowsEndings))
for _, l := range literals {
tok := s.Scan()
if l.tokenType != tok.Type {
t.Errorf("got: %s want %s for %s\n", tok, l.tokenType, tok.String())
}
if l.literal != tok.Text {
t.Errorf("got:\n%v\nwant:\n%v\n", []byte(tok.Text), []byte(l.literal))
}
}
}
func TestRealExample(t *testing.T) {
complexHCL := `// This comes from Terraform, as a test
variable "foo" {
default = "bar"
description = "bar"
}
provider "aws" {
access_key = "foo"
secret_key = "${replace(var.foo, ".", "\\.")}"
}
resource "aws_security_group" "firewall" {
count = 5
}
resource aws_instance "web" {
ami = "${var.foo}"
security_groups = [
"foo",
"${aws_security_group.firewall.foo}"
]
network_interface {
device_index = 0
description = <<EOF
Main interface
EOF
}
network_interface {
device_index = 1
description = <<-EOF
Outer text
Indented text
EOF
}
}`
literals := []struct {
tokenType token.Type
literal string
}{
{token.COMMENT, `// This comes from Terraform, as a test`},
{token.IDENT, `variable`},
{token.STRING, `"foo"`},
{token.LBRACE, `{`},
{token.IDENT, `default`},
{token.ASSIGN, `=`},
{token.STRING, `"bar"`},
{token.IDENT, `description`},
{token.ASSIGN, `=`},
{token.STRING, `"bar"`},
{token.RBRACE, `}`},
{token.IDENT, `provider`},
{token.STRING, `"aws"`},
{token.LBRACE, `{`},
{token.IDENT, `access_key`},
{token.ASSIGN, `=`},
{token.STRING, `"foo"`},
{token.IDENT, `secret_key`},
{token.ASSIGN, `=`},
{token.STRING, `"${replace(var.foo, ".", "\\.")}"`},
{token.RBRACE, `}`},
{token.IDENT, `resource`},
{token.STRING, `"aws_security_group"`},
{token.STRING, `"firewall"`},
{token.LBRACE, `{`},
{token.IDENT, `count`},
{token.ASSIGN, `=`},
{token.NUMBER, `5`},
{token.RBRACE, `}`},
{token.IDENT, `resource`},
{token.IDENT, `aws_instance`},
{token.STRING, `"web"`},
{token.LBRACE, `{`},
{token.IDENT, `ami`},
{token.ASSIGN, `=`},
{token.STRING, `"${var.foo}"`},
{token.IDENT, `security_groups`},
{token.ASSIGN, `=`},
{token.LBRACK, `[`},
{token.STRING, `"foo"`},
{token.COMMA, `,`},
{token.STRING, `"${aws_security_group.firewall.foo}"`},
{token.RBRACK, `]`},
{token.IDENT, `network_interface`},
{token.LBRACE, `{`},
{token.IDENT, `device_index`},
{token.ASSIGN, `=`},
{token.NUMBER, `0`},
{token.IDENT, `description`},
{token.ASSIGN, `=`},
{token.HEREDOC, "<<EOF\nMain interface\nEOF\n"},
{token.RBRACE, `}`},
{token.IDENT, `network_interface`},
{token.LBRACE, `{`},
{token.IDENT, `device_index`},
{token.ASSIGN, `=`},
{token.NUMBER, `1`},
{token.IDENT, `description`},
{token.ASSIGN, `=`},
{token.HEREDOC, "<<-EOF\n\t\t\tOuter text\n\t\t\t\tIndented text\n\t\t\tEOF\n"},
{token.RBRACE, `}`},
{token.RBRACE, `}`},
{token.EOF, ``},
}
s := New([]byte(complexHCL))
for _, l := range literals {
tok := s.Scan()
if l.tokenType != tok.Type {
t.Errorf("got: %s want %s for %s\n", tok, l.tokenType, tok.String())
}
if l.literal != tok.Text {
t.Errorf("got:\n%+v\n%s\n want:\n%+v\n%s\n", []byte(tok.String()), tok, []byte(l.literal), l.literal)
}
}
}
func TestScan_crlf(t *testing.T) {
complexHCL := "foo {\r\n bar = \"baz\"\r\n}\r\n"
literals := []struct {
tokenType token.Type
literal string
}{
{token.IDENT, `foo`},
{token.LBRACE, `{`},
{token.IDENT, `bar`},
{token.ASSIGN, `=`},
{token.STRING, `"baz"`},
{token.RBRACE, `}`},
{token.EOF, ``},
}
s := New([]byte(complexHCL))
for _, l := range literals {
tok := s.Scan()
if l.tokenType != tok.Type {
t.Errorf("got: %s want %s for %s\n", tok, l.tokenType, tok.String())
}
if l.literal != tok.Text {
t.Errorf("got:\n%+v\n%s\n want:\n%+v\n%s\n", []byte(tok.String()), tok, []byte(l.literal), l.literal)
}
}
}
func TestError(t *testing.T) {
testError(t, "\x80", "1:1", "illegal UTF-8 encoding", token.ILLEGAL)
testError(t, "\xff", "1:1", "illegal UTF-8 encoding", token.ILLEGAL)
testError(t, "ab\x80", "1:3", "illegal UTF-8 encoding", token.IDENT)
testError(t, "abc\xff", "1:4", "illegal UTF-8 encoding", token.IDENT)
testError(t, `"ab`+"\x80", "1:4", "illegal UTF-8 encoding", token.STRING)
testError(t, `"abc`+"\xff", "1:5", "illegal UTF-8 encoding", token.STRING)
testError(t, `01238`, "1:6", "illegal octal number", token.NUMBER)
testError(t, `01238123`, "1:9", "illegal octal number", token.NUMBER)
testError(t, `0x`, "1:3", "illegal hexadecimal number", token.NUMBER)
testError(t, `0xg`, "1:3", "illegal hexadecimal number", token.NUMBER)
testError(t, `'aa'`, "1:1", "illegal char", token.ILLEGAL)
testError(t, `"`, "1:2", "literal not terminated", token.STRING)
testError(t, `"abc`, "1:5", "literal not terminated", token.STRING)
testError(t, `"abc`+"\n", "1:5", "literal not terminated", token.STRING)
testError(t, `"${abc`+"\n", "2:1", "literal not terminated", token.STRING)
testError(t, `/*/`, "1:4", "comment not terminated", token.COMMENT)
testError(t, `/foo`, "1:1", "expected '/' for comment", token.COMMENT)
}
func testError(t *testing.T, src, pos, msg string, tok token.Type) {
s := New([]byte(src))
errorCalled := false
s.Error = func(p token.Pos, m string) {
if !errorCalled {
if pos != p.String() {
t.Errorf("pos = %q, want %q for %q", p, pos, src)
}
if m != msg {
t.Errorf("msg = %q, want %q for %q", m, msg, src)
}
errorCalled = true
}
}
tk := s.Scan()
if tk.Type != tok {
t.Errorf("tok = %s, want %s for %q", tk, tok, src)
}
if !errorCalled {
t.Errorf("error handler not called for %q", src)
}
if s.ErrorCount == 0 {
t.Errorf("count = %d, want > 0 for %q", s.ErrorCount, src)
}
}
func testTokenList(t *testing.T, tokenList []tokenPair) {
// create artifical source code
buf := new(bytes.Buffer)
for _, ident := range tokenList {
fmt.Fprintf(buf, "%s\n", ident.text)
}
s := New(buf.Bytes())
for _, ident := range tokenList {
tok := s.Scan()
if tok.Type != ident.tok {
t.Errorf("tok = %q want %q for %q\n", tok, ident.tok, ident.text)
}
if tok.Text != ident.text {
t.Errorf("text = %q want %q", tok.String(), ident.text)
}
}
}
func countNewlines(s string) int {
n := 0
for _, ch := range s {
if ch == '\n' {
n++
}
}
return n
}

View File

@@ -1,96 +0,0 @@
package strconv
import "testing"
type quoteTest struct {
in string
out string
ascii string
}
var quotetests = []quoteTest{
{"\a\b\f\r\n\t\v", `"\a\b\f\r\n\t\v"`, `"\a\b\f\r\n\t\v"`},
{"\\", `"\\"`, `"\\"`},
{"abc\xffdef", `"abc\xffdef"`, `"abc\xffdef"`},
{"\u263a", `"☺"`, `"\u263a"`},
{"\U0010ffff", `"\U0010ffff"`, `"\U0010ffff"`},
{"\x04", `"\x04"`, `"\x04"`},
}
type unQuoteTest struct {
in string
out string
}
var unquotetests = []unQuoteTest{
{`""`, ""},
{`"a"`, "a"},
{`"abc"`, "abc"},
{`"☺"`, "☺"},
{`"hello world"`, "hello world"},
{`"\xFF"`, "\xFF"},
{`"\377"`, "\377"},
{`"\u1234"`, "\u1234"},
{`"\U00010111"`, "\U00010111"},
{`"\U0001011111"`, "\U0001011111"},
{`"\a\b\f\n\r\t\v\\\""`, "\a\b\f\n\r\t\v\\\""},
{`"'"`, "'"},
{`"${file("foo")}"`, `${file("foo")}`},
{`"${file("\"foo\"")}"`, `${file("\"foo\"")}`},
{`"echo ${var.region}${element(split(",",var.zones),0)}"`,
`echo ${var.region}${element(split(",",var.zones),0)}`},
{`"${HH\\:mm\\:ss}"`, `${HH\\:mm\\:ss}`},
{`"${\n}"`, `${\n}`},
}
var misquoted = []string{
``,
`"`,
`"a`,
`"'`,
`b"`,
`"\"`,
`"\9"`,
`"\19"`,
`"\129"`,
`'\'`,
`'\9'`,
`'\19'`,
`'\129'`,
`'ab'`,
`"\x1!"`,
`"\U12345678"`,
`"\z"`,
"`",
"`xxx",
"`\"",
`"\'"`,
`'\"'`,
"\"\n\"",
"\"\\n\n\"",
"'\n'",
`"${"`,
`"${foo{}"`,
"\"${foo}\n\"",
}
func TestUnquote(t *testing.T) {
for _, tt := range unquotetests {
if out, err := Unquote(tt.in); err != nil || out != tt.out {
t.Errorf("Unquote(%#q) = %q, %v want %q, nil", tt.in, out, err, tt.out)
}
}
// run the quote tests too, backward
for _, tt := range quotetests {
if in, err := Unquote(tt.out); in != tt.in {
t.Errorf("Unquote(%#q) = %q, %v, want %q, nil", tt.out, in, err, tt.in)
}
}
for _, s := range misquoted {
if out, err := Unquote(s); out != "" || err != ErrSyntax {
t.Errorf("Unquote(%#q) = %q, %v want %q, %v", s, out, err, "", ErrSyntax)
}
}
}

View File

@@ -1,4 +0,0 @@
foo = [
"1",
"2", # comment
]

View File

@@ -1,6 +0,0 @@
resource = [{
"foo": {
"bar": {},
"baz": [1, 2, "foo"],
}
}]

View File

@@ -1,15 +0,0 @@
// Foo
/* Bar */
/*
/*
Baz
*/
# Another
# Multiple
# Lines
foo = "bar"

View File

@@ -1 +0,0 @@
# Hello

View File

@@ -1,42 +0,0 @@
// This comes from Terraform, as a test
variable "foo" {
default = "bar"
description = "bar"
}
provider "aws" {
access_key = "foo"
secret_key = "bar"
}
provider "do" {
api_key = "${var.foo}"
}
resource "aws_security_group" "firewall" {
count = 5
}
resource aws_instance "web" {
ami = "${var.foo}"
security_groups = [
"foo",
"${aws_security_group.firewall.foo}"
]
network_interface {
device_index = 0
description = "Main network interface"
}
}
resource "aws_instance" "db" {
security_groups = "${aws_security_group.firewall.*.id}"
VPC = "foo"
depends_on = ["aws_instance.web"]
}
output "web_ip" {
value = "${aws_instance.web.private_ip}"
}

View File

@@ -1 +0,0 @@
foo.bar = "baz"

View File

@@ -1 +0,0 @@
foo = [1, 2, "foo"]

View File

@@ -1 +0,0 @@
foo = [1, 2, "foo",]

View File

@@ -1,2 +0,0 @@
foo = "bar"
key = 7

View File

@@ -1,3 +0,0 @@
default = {
"eu-west-1": "ami-b1cf19c6",
}

Some files were not shown because too many files have changed in this diff Show More