Fix the dependency issue (#231)

This commit is contained in:
Robbie Zhang
2018-06-21 12:09:42 -07:00
committed by GitHub
parent 027b76651d
commit 6ec1098bb8
16629 changed files with 74837 additions and 4975021 deletions

View File

@@ -1,21 +0,0 @@
sudo: false
language: go
go:
- 1.7.5
- 1.8
- tip
os:
- linux
- osx
matrix:
allow_failures:
- go: tip
fast_finish: true
script:
- go build
- go test -race -v ./...

View File

@@ -1,452 +0,0 @@
![afero logo-sm](https://cloud.githubusercontent.com/assets/173412/11490338/d50e16dc-97a5-11e5-8b12-019a300d0fcb.png)
A FileSystem Abstraction System for Go
[![Build Status](https://travis-ci.org/spf13/afero.svg)](https://travis-ci.org/spf13/afero) [![Build status](https://ci.appveyor.com/api/projects/status/github/spf13/afero?branch=master&svg=true)](https://ci.appveyor.com/project/spf13/afero) [![GoDoc](https://godoc.org/github.com/spf13/afero?status.svg)](https://godoc.org/github.com/spf13/afero) [![Join the chat at https://gitter.im/spf13/afero](https://badges.gitter.im/Dev%20Chat.svg)](https://gitter.im/spf13/afero?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Overview
Afero is an filesystem framework providing a simple, uniform and universal API
interacting with any filesystem, as an abstraction layer providing interfaces,
types and methods. Afero has an exceptionally clean interface and simple design
without needless constructors or initialization methods.
Afero is also a library providing a base set of interoperable backend
filesystems that make it easy to work with afero while retaining all the power
and benefit of the os and ioutil packages.
Afero provides significant improvements over using the os package alone, most
notably the ability to create mock and testing filesystems without relying on the disk.
It is suitable for use in a any situation where you would consider using the OS
package as it provides an additional abstraction that makes it easy to use a
memory backed file system during testing. It also adds support for the http
filesystem for full interoperability.
## Afero Features
* A single consistent API for accessing a variety of filesystems
* Interoperation between a variety of file system types
* A set of interfaces to encourage and enforce interoperability between backends
* An atomic cross platform memory backed file system
* Support for compositional (union) file systems by combining multiple file systems acting as one
* Specialized backends which modify existing filesystems (Read Only, Regexp filtered)
* A set of utility functions ported from io, ioutil & hugo to be afero aware
# Using Afero
Afero is easy to use and easier to adopt.
A few different ways you could use Afero:
* Use the interfaces alone to define you own file system.
* Wrap for the OS packages.
* Define different filesystems for different parts of your application.
* Use Afero for mock filesystems while testing
## Step 1: Install Afero
First use go get to install the latest version of the library.
$ go get github.com/spf13/afero
Next include Afero in your application.
```go
import "github.com/spf13/afero"
```
## Step 2: Declare a backend
First define a package variable and set it to a pointer to a filesystem.
```go
var AppFs = afero.NewMemMapFs()
or
var AppFs = afero.NewOsFs()
```
It is important to note that if you repeat the composite literal you
will be using a completely new and isolated filesystem. In the case of
OsFs it will still use the same underlying filesystem but will reduce
the ability to drop in other filesystems as desired.
## Step 3: Use it like you would the OS package
Throughout your application use any function and method like you normally
would.
So if my application before had:
```go
os.Open('/tmp/foo')
```
We would replace it with:
```go
AppFs.Open('/tmp/foo')
```
`AppFs` being the variable we defined above.
## List of all available functions
File System Methods Available:
```go
Chmod(name string, mode os.FileMode) : error
Chtimes(name string, atime time.Time, mtime time.Time) : error
Create(name string) : File, error
Mkdir(name string, perm os.FileMode) : error
MkdirAll(path string, perm os.FileMode) : error
Name() : string
Open(name string) : File, error
OpenFile(name string, flag int, perm os.FileMode) : File, error
Remove(name string) : error
RemoveAll(path string) : error
Rename(oldname, newname string) : error
Stat(name string) : os.FileInfo, error
```
File Interfaces and Methods Available:
```go
io.Closer
io.Reader
io.ReaderAt
io.Seeker
io.Writer
io.WriterAt
Name() : string
Readdir(count int) : []os.FileInfo, error
Readdirnames(n int) : []string, error
Stat() : os.FileInfo, error
Sync() : error
Truncate(size int64) : error
WriteString(s string) : ret int, err error
```
In some applications it may make sense to define a new package that
simply exports the file system variable for easy access from anywhere.
## Using Afero's utility functions
Afero provides a set of functions to make it easier to use the underlying file systems.
These functions have been primarily ported from io & ioutil with some developed for Hugo.
The afero utilities support all afero compatible backends.
The list of utilities includes:
```go
DirExists(path string) (bool, error)
Exists(path string) (bool, error)
FileContainsBytes(filename string, subslice []byte) (bool, error)
GetTempDir(subPath string) string
IsDir(path string) (bool, error)
IsEmpty(path string) (bool, error)
ReadDir(dirname string) ([]os.FileInfo, error)
ReadFile(filename string) ([]byte, error)
SafeWriteReader(path string, r io.Reader) (err error)
TempDir(dir, prefix string) (name string, err error)
TempFile(dir, prefix string) (f File, err error)
Walk(root string, walkFn filepath.WalkFunc) error
WriteFile(filename string, data []byte, perm os.FileMode) error
WriteReader(path string, r io.Reader) (err error)
```
For a complete list see [Afero's GoDoc](https://godoc.org/github.com/spf13/afero)
They are available under two different approaches to use. You can either call
them directly where the first parameter of each function will be the file
system, or you can declare a new `Afero`, a custom type used to bind these
functions as methods to a given filesystem.
### Calling utilities directly
```go
fs := new(afero.MemMapFs)
f, err := afero.TempFile(fs,"", "ioutil-test")
```
### Calling via Afero
```go
fs := afero.NewMemMapFs()
afs := &afero.Afero{Fs: fs}
f, err := afs.TempFile("", "ioutil-test")
```
## Using Afero for Testing
There is a large benefit to using a mock filesystem for testing. It has a
completely blank state every time it is initialized and can be easily
reproducible regardless of OS. You could create files to your hearts content
and the file access would be fast while also saving you from all the annoying
issues with deleting temporary files, Windows file locking, etc. The MemMapFs
backend is perfect for testing.
* Much faster than performing I/O operations on disk
* Avoid security issues and permissions
* Far more control. 'rm -rf /' with confidence
* Test setup is far more easier to do
* No test cleanup needed
One way to accomplish this is to define a variable as mentioned above.
In your application this will be set to afero.NewOsFs() during testing you
can set it to afero.NewMemMapFs().
It wouldn't be uncommon to have each test initialize a blank slate memory
backend. To do this I would define my `appFS = afero.NewOsFs()` somewhere
appropriate in my application code. This approach ensures that Tests are order
independent, with no test relying on the state left by an earlier test.
Then in my tests I would initialize a new MemMapFs for each test:
```go
func TestExist(t *testing.T) {
appFS := afero.NewMemMapFs()
// create test files and directories
appFS.MkdirAll("src/a", 0755)
afero.WriteFile(appFS, "src/a/b", []byte("file b"), 0644)
afero.WriteFile(appFS, "src/c", []byte("file c"), 0644)
name := "src/c"
_, err := appFS.Stat(name)
if os.IsNotExist(err) {
t.Errorf("file \"%s\" does not exist.\n", name)
}
}
```
# Available Backends
## Operating System Native
### OsFs
The first is simply a wrapper around the native OS calls. This makes it
very easy to use as all of the calls are the same as the existing OS
calls. It also makes it trivial to have your code use the OS during
operation and a mock filesystem during testing or as needed.
```go
appfs := afero.NewOsFs()
appfs.MkdirAll("src/a", 0755))
```
## Memory Backed Storage
### MemMapFs
Afero also provides a fully atomic memory backed filesystem perfect for use in
mocking and to speed up unnecessary disk io when persistence isnt
necessary. It is fully concurrent and will work within go routines
safely.
```go
mm := afero.NewMemMapFs()
mm.MkdirAll("src/a", 0755))
```
#### InMemoryFile
As part of MemMapFs, Afero also provides an atomic, fully concurrent memory
backed file implementation. This can be used in other memory backed file
systems with ease. Plans are to add a radix tree memory stored file
system using InMemoryFile.
## Network Interfaces
### SftpFs
Afero has experimental support for secure file transfer protocol (sftp). Which can
be used to perform file operations over a encrypted channel.
## Filtering Backends
### BasePathFs
The BasePathFs restricts all operations to a given path within an Fs.
The given file name to the operations on this Fs will be prepended with
the base path before calling the source Fs.
```go
bp := afero.NewBasePathFs(afero.NewOsFs(), "/base/path")
```
### ReadOnlyFs
A thin wrapper around the source Fs providing a read only view.
```go
fs := afero.NewReadOnlyFs(afero.NewOsFs())
_, err := fs.Create("/file.txt")
// err = syscall.EPERM
```
# RegexpFs
A filtered view on file names, any file NOT matching
the passed regexp will be treated as non-existing.
Files not matching the regexp provided will not be created.
Directories are not filtered.
```go
fs := afero.NewRegexpFs(afero.NewMemMapFs(), regexp.MustCompile(`\.txt$`))
_, err := fs.Create("/file.html")
// err = syscall.ENOENT
```
### HttpFs
Afero provides an http compatible backend which can wrap any of the existing
backends.
The Http package requires a slightly specific version of Open which
returns an http.File type.
Afero provides an httpFs file system which satisfies this requirement.
Any Afero FileSystem can be used as an httpFs.
```go
httpFs := afero.NewHttpFs(<ExistingFS>)
fileserver := http.FileServer(httpFs.Dir(<PATH>)))
http.Handle("/", fileserver)
```
## Composite Backends
Afero provides the ability have two filesystems (or more) act as a single
file system.
### CacheOnReadFs
The CacheOnReadFs will lazily make copies of any accessed files from the base
layer into the overlay. Subsequent reads will be pulled from the overlay
directly permitting the request is within the cache duration of when it was
created in the overlay.
If the base filesystem is writeable, any changes to files will be
done first to the base, then to the overlay layer. Write calls to open file
handles like `Write()` or `Truncate()` to the overlay first.
To writing files to the overlay only, you can use the overlay Fs directly (not
via the union Fs).
Cache files in the layer for the given time.Duration, a cache duration of 0
means "forever" meaning the file will not be re-requested from the base ever.
A read-only base will make the overlay also read-only but still copy files
from the base to the overlay when they're not present (or outdated) in the
caching layer.
```go
base := afero.NewOsFs()
layer := afero.NewMemMapFs()
ufs := afero.NewCacheOnReadFs(base, layer, 100 * time.Second)
```
### CopyOnWriteFs()
The CopyOnWriteFs is a read only base file system with a potentially
writeable layer on top.
Read operations will first look in the overlay and if not found there, will
serve the file from the base.
Changes to the file system will only be made in the overlay.
Any attempt to modify a file found only in the base will copy the file to the
overlay layer before modification (including opening a file with a writable
handle).
Removing and Renaming files present only in the base layer is not currently
permitted. If a file is present in the base layer and the overlay, only the
overlay will be removed/renamed.
```go
base := afero.NewOsFs()
roBase := afero.NewReadOnlyFs(base)
ufs := afero.NewCopyOnWriteFs(roBase, afero.NewMemMapFs())
fh, _ = ufs.Create("/home/test/file2.txt")
fh.WriteString("This is a test")
fh.Close()
```
In this example all write operations will only occur in memory (MemMapFs)
leaving the base filesystem (OsFs) untouched.
## Desired/possible backends
The following is a short list of possible backends we hope someone will
implement:
* SSH
* ZIP
* TAR
* S3
# About the project
## What's in the name
Afero comes from the latin roots Ad-Facere.
**"Ad"** is a prefix meaning "to".
**"Facere"** is a form of the root "faciō" making "make or do".
The literal meaning of afero is "to make" or "to do" which seems very fitting
for a library that allows one to make files and directories and do things with them.
The English word that shares the same roots as Afero is "affair". Affair shares
the same concept but as a noun it means "something that is made or done" or "an
object of a particular type".
It's also nice that unlike some of my other libraries (hugo, cobra, viper) it
Googles very well.
## Release Notes
* **0.10.0** 2015.12.10
* Full compatibility with Windows
* Introduction of afero utilities
* Test suite rewritten to work cross platform
* Normalize paths for MemMapFs
* Adding Sync to the file interface
* **Breaking Change** Walk and ReadDir have changed parameter order
* Moving types used by MemMapFs to a subpackage
* General bugfixes and improvements
* **0.9.0** 2015.11.05
* New Walk function similar to filepath.Walk
* MemMapFs.OpenFile handles O_CREATE, O_APPEND, O_TRUNC
* MemMapFs.Remove now really deletes the file
* InMemoryFile.Readdir and Readdirnames work correctly
* InMemoryFile functions lock it for concurrent access
* Test suite improvements
* **0.8.0** 2014.10.28
* First public version
* Interfaces feel ready for people to build using
* Interfaces satisfy all known uses
* MemMapFs passes the majority of the OS test suite
* OsFs passes the majority of the OS test suite
## Contributing
1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request
## Contributors
Names in no particular order:
* [spf13](https://github.com/spf13)
* [jaqx0r](https://github.com/jaqx0r)
* [mbertschler](https://github.com/mbertschler)
* [xor-gate](https://github.com/xor-gate)
## License
Afero is released under the Apache 2.0 license. See
[LICENSE.txt](https://github.com/spf13/afero/blob/master/LICENSE.txt)

View File

@@ -1,699 +0,0 @@
// Copyright © 2014 Steve Francia <spf@spf13.com>.
// Copyright 2009 The Go Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package afero
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"runtime"
"strings"
"syscall"
"testing"
)
var testName = "test.txt"
var Fss = []Fs{&MemMapFs{}, &OsFs{}}
var testRegistry map[Fs][]string = make(map[Fs][]string)
func testDir(fs Fs) string {
name, err := TempDir(fs, "", "afero")
if err != nil {
panic(fmt.Sprint("unable to work with test dir", err))
}
testRegistry[fs] = append(testRegistry[fs], name)
return name
}
func tmpFile(fs Fs) File {
x, err := TempFile(fs, "", "afero")
if err != nil {
panic(fmt.Sprint("unable to work with temp file", err))
}
testRegistry[fs] = append(testRegistry[fs], x.Name())
return x
}
//Read with length 0 should not return EOF.
func TestRead0(t *testing.T) {
for _, fs := range Fss {
f := tmpFile(fs)
defer f.Close()
f.WriteString("Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.")
var b []byte
// b := make([]byte, 0)
n, err := f.Read(b)
if n != 0 || err != nil {
t.Errorf("%v: Read(0) = %d, %v, want 0, nil", fs.Name(), n, err)
}
f.Seek(0, 0)
b = make([]byte, 100)
n, err = f.Read(b)
if n <= 0 || err != nil {
t.Errorf("%v: Read(100) = %d, %v, want >0, nil", fs.Name(), n, err)
}
}
}
func TestOpenFile(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
tmp := testDir(fs)
path := filepath.Join(tmp, testName)
f, err := fs.OpenFile(path, os.O_RDWR|os.O_CREATE, 0600)
if err != nil {
t.Error(fs.Name(), "OpenFile (O_CREATE) failed:", err)
continue
}
io.WriteString(f, "initial")
f.Close()
f, err = fs.OpenFile(path, os.O_WRONLY|os.O_APPEND, 0600)
if err != nil {
t.Error(fs.Name(), "OpenFile (O_APPEND) failed:", err)
continue
}
io.WriteString(f, "|append")
f.Close()
f, err = fs.OpenFile(path, os.O_RDONLY, 0600)
contents, _ := ioutil.ReadAll(f)
expectedContents := "initial|append"
if string(contents) != expectedContents {
t.Errorf("%v: appending, expected '%v', got: '%v'", fs.Name(), expectedContents, string(contents))
}
f.Close()
f, err = fs.OpenFile(path, os.O_RDWR|os.O_TRUNC, 0600)
if err != nil {
t.Error(fs.Name(), "OpenFile (O_TRUNC) failed:", err)
continue
}
contents, _ = ioutil.ReadAll(f)
if string(contents) != "" {
t.Errorf("%v: expected truncated file, got: '%v'", fs.Name(), string(contents))
}
f.Close()
}
}
func TestCreate(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
tmp := testDir(fs)
path := filepath.Join(tmp, testName)
f, err := fs.Create(path)
if err != nil {
t.Error(fs.Name(), "Create failed:", err)
f.Close()
continue
}
io.WriteString(f, "initial")
f.Close()
f, err = fs.Create(path)
if err != nil {
t.Error(fs.Name(), "Create failed:", err)
f.Close()
continue
}
secondContent := "second create"
io.WriteString(f, secondContent)
f.Close()
f, err = fs.Open(path)
if err != nil {
t.Error(fs.Name(), "Open failed:", err)
f.Close()
continue
}
buf, err := ReadAll(f)
if err != nil {
t.Error(fs.Name(), "ReadAll failed:", err)
f.Close()
continue
}
if string(buf) != secondContent {
t.Error(fs.Name(), "Content should be", "\""+secondContent+"\" but is \""+string(buf)+"\"")
f.Close()
continue
}
f.Close()
}
}
func TestMemFileRead(t *testing.T) {
f := tmpFile(new(MemMapFs))
// f := MemFileCreate("testfile")
f.WriteString("abcd")
f.Seek(0, 0)
b := make([]byte, 8)
n, err := f.Read(b)
if n != 4 {
t.Errorf("didn't read all bytes: %v %v %v", n, err, b)
}
if err != nil {
t.Errorf("err is not nil: %v %v %v", n, err, b)
}
n, err = f.Read(b)
if n != 0 {
t.Errorf("read more bytes: %v %v %v", n, err, b)
}
if err != io.EOF {
t.Errorf("error is not EOF: %v %v %v", n, err, b)
}
}
func TestRename(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
tDir := testDir(fs)
from := filepath.Join(tDir, "/renamefrom")
to := filepath.Join(tDir, "/renameto")
exists := filepath.Join(tDir, "/renameexists")
file, err := fs.Create(from)
if err != nil {
t.Fatalf("%s: open %q failed: %v", fs.Name(), to, err)
}
if err = file.Close(); err != nil {
t.Errorf("%s: close %q failed: %v", fs.Name(), to, err)
}
file, err = fs.Create(exists)
if err != nil {
t.Fatalf("%s: open %q failed: %v", fs.Name(), to, err)
}
if err = file.Close(); err != nil {
t.Errorf("%s: close %q failed: %v", fs.Name(), to, err)
}
err = fs.Rename(from, to)
if err != nil {
t.Fatalf("%s: rename %q, %q failed: %v", fs.Name(), to, from, err)
}
file, err = fs.Create(from)
if err != nil {
t.Fatalf("%s: open %q failed: %v", fs.Name(), to, err)
}
if err = file.Close(); err != nil {
t.Errorf("%s: close %q failed: %v", fs.Name(), to, err)
}
err = fs.Rename(from, exists)
if err != nil {
t.Errorf("%s: rename %q, %q failed: %v", fs.Name(), exists, from, err)
}
names, err := readDirNames(fs, tDir)
if err != nil {
t.Errorf("%s: readDirNames error: %v", fs.Name(), err)
}
found := false
for _, e := range names {
if e == "renamefrom" {
t.Error("File is still called renamefrom")
}
if e == "renameto" {
found = true
}
}
if !found {
t.Error("File was not renamed to renameto")
}
_, err = fs.Stat(to)
if err != nil {
t.Errorf("%s: stat %q failed: %v", fs.Name(), to, err)
}
}
}
func TestRemove(t *testing.T) {
for _, fs := range Fss {
x, err := TempFile(fs, "", "afero")
if err != nil {
t.Error(fmt.Sprint("unable to work with temp file", err))
}
path := x.Name()
x.Close()
tDir := filepath.Dir(path)
err = fs.Remove(path)
if err != nil {
t.Errorf("%v: Remove() failed: %v", fs.Name(), err)
continue
}
_, err = fs.Stat(path)
if !os.IsNotExist(err) {
t.Errorf("%v: Remove() didn't remove file", fs.Name())
continue
}
// Deleting non-existent file should raise error
err = fs.Remove(path)
if !os.IsNotExist(err) {
t.Errorf("%v: Remove() didn't raise error for non-existent file", fs.Name())
}
f, err := fs.Open(tDir)
if err != nil {
t.Error("TestDir should still exist:", err)
}
names, err := f.Readdirnames(-1)
if err != nil {
t.Error("Readdirnames failed:", err)
}
for _, e := range names {
if e == testName {
t.Error("File was not removed from parent directory")
}
}
}
}
func TestTruncate(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
f := tmpFile(fs)
defer f.Close()
checkSize(t, f, 0)
f.Write([]byte("hello, world\n"))
checkSize(t, f, 13)
f.Truncate(10)
checkSize(t, f, 10)
f.Truncate(1024)
checkSize(t, f, 1024)
f.Truncate(0)
checkSize(t, f, 0)
_, err := f.Write([]byte("surprise!"))
if err == nil {
checkSize(t, f, 13+9) // wrote at offset past where hello, world was.
}
}
}
func TestSeek(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
f := tmpFile(fs)
defer f.Close()
const data = "hello, world\n"
io.WriteString(f, data)
type test struct {
in int64
whence int
out int64
}
var tests = []test{
{0, 1, int64(len(data))},
{0, 0, 0},
{5, 0, 5},
{0, 2, int64(len(data))},
{0, 0, 0},
{-1, 2, int64(len(data)) - 1},
{1 << 33, 0, 1 << 33},
{1 << 33, 2, 1<<33 + int64(len(data))},
}
for i, tt := range tests {
off, err := f.Seek(tt.in, tt.whence)
if off != tt.out || err != nil {
if e, ok := err.(*os.PathError); ok && e.Err == syscall.EINVAL && tt.out > 1<<32 {
// Reiserfs rejects the big seeks.
// http://code.google.com/p/go/issues/detail?id=91
break
}
t.Errorf("#%d: Seek(%v, %v) = %v, %v want %v, nil", i, tt.in, tt.whence, off, err, tt.out)
}
}
}
}
func TestReadAt(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
f := tmpFile(fs)
defer f.Close()
const data = "hello, world\n"
io.WriteString(f, data)
b := make([]byte, 5)
n, err := f.ReadAt(b, 7)
if err != nil || n != len(b) {
t.Fatalf("ReadAt 7: %d, %v", n, err)
}
if string(b) != "world" {
t.Fatalf("ReadAt 7: have %q want %q", string(b), "world")
}
}
}
func TestWriteAt(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
f := tmpFile(fs)
defer f.Close()
const data = "hello, world\n"
io.WriteString(f, data)
n, err := f.WriteAt([]byte("WORLD"), 7)
if err != nil || n != 5 {
t.Fatalf("WriteAt 7: %d, %v", n, err)
}
f2, err := fs.Open(f.Name())
if err != nil {
t.Fatalf("%v: ReadFile %s: %v", fs.Name(), f.Name(), err)
}
defer f2.Close()
buf := new(bytes.Buffer)
buf.ReadFrom(f2)
b := buf.Bytes()
if string(b) != "hello, WORLD\n" {
t.Fatalf("after write: have %q want %q", string(b), "hello, WORLD\n")
}
}
}
func setupTestDir(t *testing.T, fs Fs) string {
path := testDir(fs)
return setupTestFiles(t, fs, path)
}
func setupTestDirRoot(t *testing.T, fs Fs) string {
path := testDir(fs)
setupTestFiles(t, fs, path)
return path
}
func setupTestDirReusePath(t *testing.T, fs Fs, path string) string {
testRegistry[fs] = append(testRegistry[fs], path)
return setupTestFiles(t, fs, path)
}
func setupTestFiles(t *testing.T, fs Fs, path string) string {
testSubDir := filepath.Join(path, "more", "subdirectories", "for", "testing", "we")
err := fs.MkdirAll(testSubDir, 0700)
if err != nil && !os.IsExist(err) {
t.Fatal(err)
}
f, err := fs.Create(filepath.Join(testSubDir, "testfile1"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 1 content")
f.Close()
f, err = fs.Create(filepath.Join(testSubDir, "testfile2"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 2 content")
f.Close()
f, err = fs.Create(filepath.Join(testSubDir, "testfile3"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 3 content")
f.Close()
f, err = fs.Create(filepath.Join(testSubDir, "testfile4"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 4 content")
f.Close()
return testSubDir
}
func TestReaddirnames(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
testSubDir := setupTestDir(t, fs)
tDir := filepath.Dir(testSubDir)
root, err := fs.Open(tDir)
if err != nil {
t.Fatal(fs.Name(), tDir, err)
}
defer root.Close()
namesRoot, err := root.Readdirnames(-1)
if err != nil {
t.Fatal(fs.Name(), namesRoot, err)
}
sub, err := fs.Open(testSubDir)
if err != nil {
t.Fatal(err)
}
defer sub.Close()
namesSub, err := sub.Readdirnames(-1)
if err != nil {
t.Fatal(fs.Name(), namesSub, err)
}
findNames(fs, t, tDir, testSubDir, namesRoot, namesSub)
}
}
func TestReaddirSimple(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
testSubDir := setupTestDir(t, fs)
tDir := filepath.Dir(testSubDir)
root, err := fs.Open(tDir)
if err != nil {
t.Fatal(err)
}
defer root.Close()
rootInfo, err := root.Readdir(1)
if err != nil {
t.Log(myFileInfo(rootInfo))
t.Error(err)
}
rootInfo, err = root.Readdir(5)
if err != io.EOF {
t.Log(myFileInfo(rootInfo))
t.Error(err)
}
sub, err := fs.Open(testSubDir)
if err != nil {
t.Fatal(err)
}
defer sub.Close()
subInfo, err := sub.Readdir(5)
if err != nil {
t.Log(myFileInfo(subInfo))
t.Error(err)
}
}
}
func TestReaddir(t *testing.T) {
defer removeAllTestFiles(t)
for num := 0; num < 6; num++ {
outputs := make([]string, len(Fss))
infos := make([]string, len(Fss))
for i, fs := range Fss {
testSubDir := setupTestDir(t, fs)
//tDir := filepath.Dir(testSubDir)
root, err := fs.Open(testSubDir)
if err != nil {
t.Fatal(err)
}
defer root.Close()
for j := 0; j < 6; j++ {
info, err := root.Readdir(num)
outputs[i] += fmt.Sprintf("%v Error: %v\n", myFileInfo(info), err)
infos[i] += fmt.Sprintln(len(info), err)
}
}
fail := false
for i, o := range infos {
if i == 0 {
continue
}
if o != infos[i-1] {
fail = true
break
}
}
if fail {
t.Log("Readdir outputs not equal for Readdir(", num, ")")
for i, o := range outputs {
t.Log(Fss[i].Name())
t.Log(o)
}
t.Fail()
}
}
}
type myFileInfo []os.FileInfo
func (m myFileInfo) String() string {
out := "Fileinfos:\n"
for _, e := range m {
out += " " + e.Name() + "\n"
}
return out
}
func TestReaddirAll(t *testing.T) {
defer removeAllTestFiles(t)
for _, fs := range Fss {
testSubDir := setupTestDir(t, fs)
tDir := filepath.Dir(testSubDir)
root, err := fs.Open(tDir)
if err != nil {
t.Fatal(err)
}
defer root.Close()
rootInfo, err := root.Readdir(-1)
if err != nil {
t.Fatal(err)
}
var namesRoot = []string{}
for _, e := range rootInfo {
namesRoot = append(namesRoot, e.Name())
}
sub, err := fs.Open(testSubDir)
if err != nil {
t.Fatal(err)
}
defer sub.Close()
subInfo, err := sub.Readdir(-1)
if err != nil {
t.Fatal(err)
}
var namesSub = []string{}
for _, e := range subInfo {
namesSub = append(namesSub, e.Name())
}
findNames(fs, t, tDir, testSubDir, namesRoot, namesSub)
}
}
func findNames(fs Fs, t *testing.T, tDir, testSubDir string, root, sub []string) {
var foundRoot bool
for _, e := range root {
f, err := fs.Open(filepath.Join(tDir, e))
if err != nil {
t.Error("Open", filepath.Join(tDir, e), ":", err)
}
defer f.Close()
if equal(e, "we") {
foundRoot = true
}
}
if !foundRoot {
t.Logf("Names root: %v", root)
t.Logf("Names sub: %v", sub)
t.Error("Didn't find subdirectory we")
}
var found1, found2 bool
for _, e := range sub {
f, err := fs.Open(filepath.Join(testSubDir, e))
if err != nil {
t.Error("Open", filepath.Join(testSubDir, e), ":", err)
}
defer f.Close()
if equal(e, "testfile1") {
found1 = true
}
if equal(e, "testfile2") {
found2 = true
}
}
if !found1 {
t.Logf("Names root: %v", root)
t.Logf("Names sub: %v", sub)
t.Error("Didn't find testfile1")
}
if !found2 {
t.Logf("Names root: %v", root)
t.Logf("Names sub: %v", sub)
t.Error("Didn't find testfile2")
}
}
func removeAllTestFiles(t *testing.T) {
for fs, list := range testRegistry {
for _, path := range list {
if err := fs.RemoveAll(path); err != nil {
t.Error(fs.Name(), err)
}
}
}
testRegistry = make(map[Fs][]string)
}
func equal(name1, name2 string) (r bool) {
switch runtime.GOOS {
case "windows":
r = strings.ToLower(name1) == strings.ToLower(name2)
default:
r = name1 == name2
}
return
}
func checkSize(t *testing.T, f File, size int64) {
dir, err := f.Stat()
if err != nil {
t.Fatalf("Stat %q (looking for size %d): %s", f.Name(), size, err)
}
if dir.Size() != size {
t.Errorf("Stat %q: size %d want %d", f.Name(), dir.Size(), size)
}
}

View File

@@ -1,15 +0,0 @@
version: '{build}'
clone_folder: C:\gopath\src\github.com\spf13\afero
environment:
GOPATH: C:\gopath
build_script:
- cmd: >-
go version
go env
go get -v github.com/spf13/afero/...
go build github.com/spf13/afero
test_script:
- cmd: go test -race -v github.com/spf13/afero/...

View File

@@ -1,7 +1,6 @@
package afero
import (
"errors"
"os"
"path/filepath"
"runtime"
@@ -9,6 +8,8 @@ import (
"time"
)
var _ Lstater = (*BasePathFs)(nil)
// The BasePathFs restricts all operations to a given path within an Fs.
// The given file name to the operations on this Fs will be prepended with
// the base path before calling the base Fs.
@@ -22,6 +23,16 @@ type BasePathFs struct {
path string
}
type BasePathFile struct {
File
path string
}
func (f *BasePathFile) Name() string {
sourcename := f.File.Name()
return strings.TrimPrefix(sourcename, filepath.Clean(f.path))
}
func NewBasePathFs(source Fs, path string) Fs {
return &BasePathFs{source: source, path: path}
}
@@ -30,7 +41,7 @@ func NewBasePathFs(source Fs, path string) Fs {
// else the given file with the base path prepended
func (b *BasePathFs) RealPath(name string) (path string, err error) {
if err := validateBasePathName(name); err != nil {
return "", err
return name, err
}
bpath := filepath.Clean(b.path)
@@ -52,7 +63,7 @@ func validateBasePathName(name string) error {
// On Windows a common mistake would be to provide an absolute OS path
// We could strip out the base part, but that would not be very portable.
if filepath.IsAbs(name) {
return &os.PathError{Op: "realPath", Path: name, Err: errors.New("got a real OS path instead of a virtual")}
return os.ErrNotExist
}
return nil
@@ -111,14 +122,22 @@ func (b *BasePathFs) OpenFile(name string, flag int, mode os.FileMode) (f File,
if name, err = b.RealPath(name); err != nil {
return nil, &os.PathError{Op: "openfile", Path: name, Err: err}
}
return b.source.OpenFile(name, flag, mode)
sourcef, err := b.source.OpenFile(name, flag, mode)
if err != nil {
return nil, err
}
return &BasePathFile{sourcef, b.path}, nil
}
func (b *BasePathFs) Open(name string) (f File, err error) {
if name, err = b.RealPath(name); err != nil {
return nil, &os.PathError{Op: "open", Path: name, Err: err}
}
return b.source.Open(name)
sourcef, err := b.source.Open(name)
if err != nil {
return nil, err
}
return &BasePathFile{File: sourcef, path: b.path}, nil
}
func (b *BasePathFs) Mkdir(name string, mode os.FileMode) (err error) {
@@ -139,7 +158,23 @@ func (b *BasePathFs) Create(name string) (f File, err error) {
if name, err = b.RealPath(name); err != nil {
return nil, &os.PathError{Op: "create", Path: name, Err: err}
}
return b.source.Create(name)
sourcef, err := b.source.Create(name)
if err != nil {
return nil, err
}
return &BasePathFile{File: sourcef, path: b.path}, nil
}
func (b *BasePathFs) LstatIfPossible(name string) (os.FileInfo, bool, error) {
name, err := b.RealPath(name)
if err != nil {
return nil, false, &os.PathError{Op: "lstat", Path: name, Err: err}
}
if lstater, ok := b.source.(Lstater); ok {
return lstater.LstatIfPossible(name)
}
fi, err := b.source.Stat(name)
return fi, false, err
}
// vim: ts=4 sw=4 noexpandtab nolist syn=go

View File

@@ -1,142 +0,0 @@
package afero
import (
"os"
"path/filepath"
"runtime"
"testing"
)
func TestBasePath(t *testing.T) {
baseFs := &MemMapFs{}
baseFs.MkdirAll("/base/path/tmp", 0777)
bp := NewBasePathFs(baseFs, "/base/path")
if _, err := bp.Create("/tmp/foo"); err != nil {
t.Errorf("Failed to set real path")
}
if fh, err := bp.Create("../tmp/bar"); err == nil {
t.Errorf("succeeded in creating %s ...", fh.Name())
}
}
func TestBasePathRoot(t *testing.T) {
baseFs := &MemMapFs{}
baseFs.MkdirAll("/base/path/foo/baz", 0777)
baseFs.MkdirAll("/base/path/boo/", 0777)
bp := NewBasePathFs(baseFs, "/base/path")
rd, err := ReadDir(bp, string(os.PathSeparator))
if len(rd) != 2 {
t.Errorf("base path doesn't respect root")
}
if err != nil {
t.Error(err)
}
}
func TestRealPath(t *testing.T) {
fs := NewOsFs()
baseDir, err := TempDir(fs, "", "base")
if err != nil {
t.Fatal("error creating tempDir", err)
}
defer fs.RemoveAll(baseDir)
anotherDir, err := TempDir(fs, "", "another")
if err != nil {
t.Fatal("error creating tempDir", err)
}
defer fs.RemoveAll(anotherDir)
bp := NewBasePathFs(fs, baseDir).(*BasePathFs)
subDir := filepath.Join(baseDir, "s1")
realPath, err := bp.RealPath("/s1")
if err != nil {
t.Errorf("Got error %s", err)
}
if realPath != subDir {
t.Errorf("Expected \n%s got \n%s", subDir, realPath)
}
if runtime.GOOS == "windows" {
_, err = bp.RealPath(anotherDir)
if err == nil {
t.Errorf("Expected error")
}
} else {
// on *nix we have no way of just looking at the path and tell that anotherDir
// is not inside the base file system.
// The user will receive an os.ErrNotExist later.
surrealPath, err := bp.RealPath(anotherDir)
if err != nil {
t.Errorf("Got error %s", err)
}
excpected := filepath.Join(baseDir, anotherDir)
if surrealPath != excpected {
t.Errorf("Expected \n%s got \n%s", excpected, surrealPath)
}
}
}
func TestNestedBasePaths(t *testing.T) {
type dirSpec struct {
Dir1, Dir2, Dir3 string
}
dirSpecs := []dirSpec{
dirSpec{Dir1: "/", Dir2: "/", Dir3: "/"},
dirSpec{Dir1: "/", Dir2: "/path2", Dir3: "/"},
dirSpec{Dir1: "/path1/dir", Dir2: "/path2/dir/", Dir3: "/path3/dir"},
dirSpec{Dir1: "C:/path1", Dir2: "path2/dir", Dir3: "/path3/dir/"},
}
for _, ds := range dirSpecs {
memFs := NewMemMapFs()
level1Fs := NewBasePathFs(memFs, ds.Dir1)
level2Fs := NewBasePathFs(level1Fs, ds.Dir2)
level3Fs := NewBasePathFs(level2Fs, ds.Dir3)
type spec struct {
BaseFs Fs
FileName string
}
specs := []spec{
spec{BaseFs: level3Fs, FileName: "f.txt"},
spec{BaseFs: level2Fs, FileName: "f.txt"},
spec{BaseFs: level1Fs, FileName: "f.txt"},
}
for _, s := range specs {
if err := s.BaseFs.MkdirAll(s.FileName, 0755); err != nil {
t.Errorf("Got error %s", err.Error())
}
if _, err := s.BaseFs.Stat(s.FileName); err != nil {
t.Errorf("Got error %s", err.Error())
}
if s.BaseFs == level3Fs {
pathToExist := filepath.Join(ds.Dir3, s.FileName)
if _, err := level2Fs.Stat(pathToExist); err != nil {
t.Errorf("Got error %s (path %s)", err.Error(), pathToExist)
}
} else if s.BaseFs == level2Fs {
pathToExist := filepath.Join(ds.Dir2, ds.Dir3, s.FileName)
if _, err := level1Fs.Stat(pathToExist); err != nil {
t.Errorf("Got error %s (path %s)", err.Error(), pathToExist)
}
}
}
}
}

View File

@@ -205,7 +205,7 @@ func (u *CacheOnReadFs) OpenFile(name string, flag int, perm os.FileMode) (File,
bfi.Close() // oops, what if O_TRUNC was set and file opening in the layer failed...?
return nil, err
}
return &UnionFile{base: bfi, layer: lfi}, nil
return &UnionFile{Base: bfi, Layer: lfi}, nil
}
return u.layer.OpenFile(name, flag, perm)
}
@@ -251,7 +251,7 @@ func (u *CacheOnReadFs) Open(name string) (File, error) {
if err != nil && bfile == nil {
return nil, err
}
return &UnionFile{base: bfile, layer: lfile}, nil
return &UnionFile{Base: bfile, Layer: lfile}, nil
}
func (u *CacheOnReadFs) Mkdir(name string, perm os.FileMode) error {
@@ -286,5 +286,5 @@ func (u *CacheOnReadFs) Create(name string) (File, error) {
bfh.Close()
return nil, err
}
return &UnionFile{base: bfh, layer: lfh}, nil
return &UnionFile{Base: bfh, Layer: lfh}, nil
}

View File

@@ -1,404 +0,0 @@
package afero
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"testing"
"time"
)
var tempDirs []string
func NewTempOsBaseFs(t *testing.T) Fs {
name, err := TempDir(NewOsFs(), "", "")
if err != nil {
t.Error("error creating tempDir", err)
}
tempDirs = append(tempDirs, name)
return NewBasePathFs(NewOsFs(), name)
}
func CleanupTempDirs(t *testing.T) {
osfs := NewOsFs()
type ev struct {
path string
e error
}
errs := []ev{}
for _, x := range tempDirs {
err := osfs.RemoveAll(x)
if err != nil {
errs = append(errs, ev{path: x, e: err})
}
}
for _, e := range errs {
fmt.Println("error removing tempDir", e.path, e.e)
}
if len(errs) > 0 {
t.Error("error cleaning up tempDirs")
}
tempDirs = []string{}
}
func TestUnionCreateExisting(t *testing.T) {
base := &MemMapFs{}
roBase := &ReadOnlyFs{source: base}
ufs := NewCopyOnWriteFs(roBase, &MemMapFs{})
base.MkdirAll("/home/test", 0777)
fh, _ := base.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
fh, err := ufs.OpenFile("/home/test/file.txt", os.O_RDWR, 0666)
if err != nil {
t.Errorf("Failed to open file r/w: %s", err)
}
_, err = fh.Write([]byte("####"))
if err != nil {
t.Errorf("Failed to write file: %s", err)
}
fh.Seek(0, 0)
data, err := ioutil.ReadAll(fh)
if err != nil {
t.Errorf("Failed to read file: %s", err)
}
if string(data) != "#### is a test" {
t.Errorf("Got wrong data")
}
fh.Close()
fh, _ = base.Open("/home/test/file.txt")
data, err = ioutil.ReadAll(fh)
if string(data) != "This is a test" {
t.Errorf("Got wrong data in base file")
}
fh.Close()
fh, err = ufs.Create("/home/test/file.txt")
switch err {
case nil:
if fi, _ := fh.Stat(); fi.Size() != 0 {
t.Errorf("Create did not truncate file")
}
fh.Close()
default:
t.Errorf("Create failed on existing file")
}
}
func TestUnionMergeReaddir(t *testing.T) {
base := &MemMapFs{}
roBase := &ReadOnlyFs{source: base}
ufs := &CopyOnWriteFs{base: roBase, layer: &MemMapFs{}}
base.MkdirAll("/home/test", 0777)
fh, _ := base.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = ufs.Create("/home/test/file2.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = ufs.Open("/home/test")
files, err := fh.Readdirnames(-1)
if err != nil {
t.Errorf("Readdirnames failed")
}
if len(files) != 2 {
t.Errorf("Got wrong number of files: %v", files)
}
}
func TestExistingDirectoryCollisionReaddir(t *testing.T) {
base := &MemMapFs{}
roBase := &ReadOnlyFs{source: base}
overlay := &MemMapFs{}
ufs := &CopyOnWriteFs{base: roBase, layer: overlay}
base.MkdirAll("/home/test", 0777)
fh, _ := base.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
overlay.MkdirAll("home/test", 0777)
fh, _ = overlay.Create("/home/test/file2.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = ufs.Create("/home/test/file3.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = ufs.Open("/home/test")
files, err := fh.Readdirnames(-1)
if err != nil {
t.Errorf("Readdirnames failed")
}
if len(files) != 3 {
t.Errorf("Got wrong number of files in union: %v", files)
}
fh, _ = overlay.Open("/home/test")
files, err = fh.Readdirnames(-1)
if err != nil {
t.Errorf("Readdirnames failed")
}
if len(files) != 2 {
t.Errorf("Got wrong number of files in overlay: %v", files)
}
}
func TestNestedDirBaseReaddir(t *testing.T) {
base := &MemMapFs{}
roBase := &ReadOnlyFs{source: base}
overlay := &MemMapFs{}
ufs := &CopyOnWriteFs{base: roBase, layer: overlay}
base.MkdirAll("/home/test/foo/bar", 0777)
fh, _ := base.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = base.Create("/home/test/foo/file2.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = base.Create("/home/test/foo/bar/file3.txt")
fh.WriteString("This is a test")
fh.Close()
overlay.MkdirAll("/", 0777)
// Opening something only in the base
fh, _ = ufs.Open("/home/test/foo")
list, err := fh.Readdir(-1)
if err != nil {
t.Errorf("Readdir failed %s", err)
}
if len(list) != 2 {
for _, x := range list {
fmt.Println(x.Name())
}
t.Errorf("Got wrong number of files in union: %v", len(list))
}
}
func TestNestedDirOverlayReaddir(t *testing.T) {
base := &MemMapFs{}
roBase := &ReadOnlyFs{source: base}
overlay := &MemMapFs{}
ufs := &CopyOnWriteFs{base: roBase, layer: overlay}
base.MkdirAll("/", 0777)
overlay.MkdirAll("/home/test/foo/bar", 0777)
fh, _ := overlay.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = overlay.Create("/home/test/foo/file2.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = overlay.Create("/home/test/foo/bar/file3.txt")
fh.WriteString("This is a test")
fh.Close()
// Opening nested dir only in the overlay
fh, _ = ufs.Open("/home/test/foo")
list, err := fh.Readdir(-1)
if err != nil {
t.Errorf("Readdir failed %s", err)
}
if len(list) != 2 {
for _, x := range list {
fmt.Println(x.Name())
}
t.Errorf("Got wrong number of files in union: %v", len(list))
}
}
func TestNestedDirOverlayOsFsReaddir(t *testing.T) {
defer CleanupTempDirs(t)
base := NewTempOsBaseFs(t)
roBase := &ReadOnlyFs{source: base}
overlay := NewTempOsBaseFs(t)
ufs := &CopyOnWriteFs{base: roBase, layer: overlay}
base.MkdirAll("/", 0777)
overlay.MkdirAll("/home/test/foo/bar", 0777)
fh, _ := overlay.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = overlay.Create("/home/test/foo/file2.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = overlay.Create("/home/test/foo/bar/file3.txt")
fh.WriteString("This is a test")
fh.Close()
// Opening nested dir only in the overlay
fh, _ = ufs.Open("/home/test/foo")
list, err := fh.Readdir(-1)
fh.Close()
if err != nil {
t.Errorf("Readdir failed %s", err)
}
if len(list) != 2 {
for _, x := range list {
fmt.Println(x.Name())
}
t.Errorf("Got wrong number of files in union: %v", len(list))
}
}
func TestCopyOnWriteFsWithOsFs(t *testing.T) {
defer CleanupTempDirs(t)
base := NewTempOsBaseFs(t)
roBase := &ReadOnlyFs{source: base}
overlay := NewTempOsBaseFs(t)
ufs := &CopyOnWriteFs{base: roBase, layer: overlay}
base.MkdirAll("/home/test", 0777)
fh, _ := base.Create("/home/test/file.txt")
fh.WriteString("This is a test")
fh.Close()
overlay.MkdirAll("home/test", 0777)
fh, _ = overlay.Create("/home/test/file2.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = ufs.Create("/home/test/file3.txt")
fh.WriteString("This is a test")
fh.Close()
fh, _ = ufs.Open("/home/test")
files, err := fh.Readdirnames(-1)
fh.Close()
if err != nil {
t.Errorf("Readdirnames failed")
}
if len(files) != 3 {
t.Errorf("Got wrong number of files in union: %v", files)
}
fh, _ = overlay.Open("/home/test")
files, err = fh.Readdirnames(-1)
fh.Close()
if err != nil {
t.Errorf("Readdirnames failed")
}
if len(files) != 2 {
t.Errorf("Got wrong number of files in overlay: %v", files)
}
}
func TestUnionCacheWrite(t *testing.T) {
base := &MemMapFs{}
layer := &MemMapFs{}
ufs := NewCacheOnReadFs(base, layer, 0)
base.Mkdir("/data", 0777)
fh, err := ufs.Create("/data/file.txt")
if err != nil {
t.Errorf("Failed to create file")
}
_, err = fh.Write([]byte("This is a test"))
if err != nil {
t.Errorf("Failed to write file")
}
fh.Seek(0, os.SEEK_SET)
buf := make([]byte, 4)
_, err = fh.Read(buf)
fh.Write([]byte(" IS A"))
fh.Close()
baseData, _ := ReadFile(base, "/data/file.txt")
layerData, _ := ReadFile(layer, "/data/file.txt")
if string(baseData) != string(layerData) {
t.Errorf("Different data: %s <=> %s", baseData, layerData)
}
}
func TestUnionCacheExpire(t *testing.T) {
base := &MemMapFs{}
layer := &MemMapFs{}
ufs := &CacheOnReadFs{base: base, layer: layer, cacheTime: 1 * time.Second}
base.Mkdir("/data", 0777)
fh, err := ufs.Create("/data/file.txt")
if err != nil {
t.Errorf("Failed to create file")
}
_, err = fh.Write([]byte("This is a test"))
if err != nil {
t.Errorf("Failed to write file")
}
fh.Close()
fh, _ = base.Create("/data/file.txt")
// sleep some time, so we really get a different time.Now() on write...
time.Sleep(2 * time.Second)
fh.WriteString("Another test")
fh.Close()
data, _ := ReadFile(ufs, "/data/file.txt")
if string(data) != "Another test" {
t.Errorf("cache time failed: <%s>", data)
}
}
func TestCacheOnReadFsNotInLayer(t *testing.T) {
base := NewMemMapFs()
layer := NewMemMapFs()
fs := NewCacheOnReadFs(base, layer, 0)
fh, err := base.Create("/file.txt")
if err != nil {
t.Fatal("unable to create file: ", err)
}
txt := []byte("This is a test")
fh.Write(txt)
fh.Close()
fh, err = fs.Open("/file.txt")
if err != nil {
t.Fatal("could not open file: ", err)
}
b, err := ReadAll(fh)
fh.Close()
if err != nil {
t.Fatal("could not read file: ", err)
} else if !bytes.Equal(txt, b) {
t.Fatalf("wanted file text %q, got %q", txt, b)
}
fh, err = layer.Open("/file.txt")
if err != nil {
t.Fatal("could not open file from layer: ", err)
}
fh.Close()
}

View File

@@ -8,6 +8,8 @@ import (
"time"
)
var _ Lstater = (*CopyOnWriteFs)(nil)
// The CopyOnWriteFs is a union filesystem: a read only base file system with
// a possibly writeable layer on top. Changes to the file system will only
// be made in the overlay: Changing an existing file in the base layer which
@@ -76,18 +78,55 @@ func (u *CopyOnWriteFs) Chmod(name string, mode os.FileMode) error {
func (u *CopyOnWriteFs) Stat(name string) (os.FileInfo, error) {
fi, err := u.layer.Stat(name)
if err != nil {
origErr := err
if e, ok := err.(*os.PathError); ok {
err = e.Err
}
if err == os.ErrNotExist || err == syscall.ENOENT || err == syscall.ENOTDIR {
isNotExist := u.isNotExist(err)
if isNotExist {
return u.base.Stat(name)
}
return nil, origErr
return nil, err
}
return fi, nil
}
func (u *CopyOnWriteFs) LstatIfPossible(name string) (os.FileInfo, bool, error) {
llayer, ok1 := u.layer.(Lstater)
lbase, ok2 := u.base.(Lstater)
if ok1 {
fi, b, err := llayer.LstatIfPossible(name)
if err == nil {
return fi, b, nil
}
if !u.isNotExist(err) {
return nil, b, err
}
}
if ok2 {
fi, b, err := lbase.LstatIfPossible(name)
if err == nil {
return fi, b, nil
}
if !u.isNotExist(err) {
return nil, b, err
}
}
fi, err := u.Stat(name)
return fi, false, err
}
func (u *CopyOnWriteFs) isNotExist(err error) bool {
if e, ok := err.(*os.PathError); ok {
err = e.Err
}
if err == os.ErrNotExist || err == syscall.ENOENT || err == syscall.ENOTDIR {
return true
}
return false
}
// Renaming files present only in the base layer is not permitted
func (u *CopyOnWriteFs) Rename(oldname, newname string) error {
b, err := u.isBaseFile(oldname)
@@ -219,7 +258,7 @@ func (u *CopyOnWriteFs) Open(name string) (File, error) {
return nil, fmt.Errorf("BaseErr: %v\nOverlayErr: %v", bErr, lErr)
}
return &UnionFile{base: bfile, layer: lfile}, nil
return &UnionFile{Base: bfile, Layer: lfile}, nil
}
func (u *CopyOnWriteFs) Mkdir(name string, perm os.FileMode) error {

View File

@@ -1,39 +0,0 @@
package afero
import "testing"
func TestCopyOnWrite(t *testing.T) {
var fs Fs
var err error
base := NewOsFs()
roBase := NewReadOnlyFs(base)
ufs := NewCopyOnWriteFs(roBase, NewMemMapFs())
fs = ufs
err = fs.MkdirAll("nonexistent/directory/", 0744)
if err != nil {
t.Error(err)
return
}
_, err = fs.Create("nonexistent/directory/newfile")
if err != nil {
t.Error(err)
return
}
}
func TestCopyOnWriteFileInMemMapBase(t *testing.T) {
base := &MemMapFs{}
layer := &MemMapFs{}
if err := WriteFile(base, "base.txt", []byte("base"), 0755); err != nil {
t.Fatalf("Failed to write file: %s", err)
}
ufs := NewCopyOnWriteFs(base, layer)
_, err := ufs.Stat("base.txt")
if err != nil {
t.Fatal(err)
}
}

View File

@@ -1,112 +0,0 @@
// ©2015 The Go Authors
// Copyright ©2015 Steve Francia <spf@spf13.com>
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package afero
import "testing"
func checkSizePath(t *testing.T, path string, size int64) {
dir, err := testFS.Stat(path)
if err != nil {
t.Fatalf("Stat %q (looking for size %d): %s", path, size, err)
}
if dir.Size() != size {
t.Errorf("Stat %q: size %d want %d", path, dir.Size(), size)
}
}
func TestReadFile(t *testing.T) {
testFS = &MemMapFs{}
fsutil := &Afero{Fs: testFS}
testFS.Create("this_exists.go")
filename := "rumpelstilzchen"
contents, err := fsutil.ReadFile(filename)
if err == nil {
t.Fatalf("ReadFile %s: error expected, none found", filename)
}
filename = "this_exists.go"
contents, err = fsutil.ReadFile(filename)
if err != nil {
t.Fatalf("ReadFile %s: %v", filename, err)
}
checkSizePath(t, filename, int64(len(contents)))
}
func TestWriteFile(t *testing.T) {
testFS = &MemMapFs{}
fsutil := &Afero{Fs: testFS}
f, err := fsutil.TempFile("", "ioutil-test")
if err != nil {
t.Fatal(err)
}
filename := f.Name()
data := "Programming today is a race between software engineers striving to " +
"build bigger and better idiot-proof programs, and the Universe trying " +
"to produce bigger and better idiots. So far, the Universe is winning."
if err := fsutil.WriteFile(filename, []byte(data), 0644); err != nil {
t.Fatalf("WriteFile %s: %v", filename, err)
}
contents, err := fsutil.ReadFile(filename)
if err != nil {
t.Fatalf("ReadFile %s: %v", filename, err)
}
if string(contents) != data {
t.Fatalf("contents = %q\nexpected = %q", string(contents), data)
}
// cleanup
f.Close()
testFS.Remove(filename) // ignore error
}
func TestReadDir(t *testing.T) {
testFS = &MemMapFs{}
testFS.Mkdir("/i-am-a-dir", 0777)
testFS.Create("/this_exists.go")
dirname := "rumpelstilzchen"
_, err := ReadDir(testFS, dirname)
if err == nil {
t.Fatalf("ReadDir %s: error expected, none found", dirname)
}
dirname = ".."
list, err := ReadDir(testFS, dirname)
if err != nil {
t.Fatalf("ReadDir %s: %v", dirname, err)
}
foundFile := false
foundSubDir := false
for _, dir := range list {
switch {
case !dir.IsDir() && dir.Name() == "this_exists.go":
foundFile = true
case dir.IsDir() && dir.Name() == "i-am-a-dir":
foundSubDir = true
}
}
if !foundFile {
t.Fatalf("ReadDir %s: this_exists.go file not found", dirname)
}
if !foundSubDir {
t.Fatalf("ReadDir %s: i-am-a-dir directory not found", dirname)
}
}

27
vendor/github.com/spf13/afero/lstater.go generated vendored Normal file
View File

@@ -0,0 +1,27 @@
// Copyright © 2018 Steve Francia <spf@spf13.com>.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package afero
import (
"os"
)
// Lstater is an optional interface in Afero. It is only implemented by the
// filesystems saying so.
// It will call Lstat if the filesystem iself is, or it delegates to, the os filesystem.
// Else it will call Stat.
// In addtion to the FileInfo, it will return a boolean telling whether Lstat was called or not.
type Lstater interface {
LstatIfPossible(name string) (os.FileInfo, bool, error)
}

View File

@@ -33,8 +33,8 @@ import (
// built-ins from that package.
func Glob(fs Fs, pattern string) (matches []string, err error) {
if !hasMeta(pattern) {
// afero does not support Lstat directly.
if _, err = lstatIfOs(fs, pattern); err != nil {
// Lstat not supported by a ll filesystems.
if _, err = lstatIfPossible(fs, pattern); err != nil {
return nil, nil
}
return []string{pattern}, nil

View File

@@ -1,183 +0,0 @@
// Copyright © 2014 Steve Francia <spf@spf13.com>.
// Copyright 2009 The Go Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package afero
import (
"os"
"path/filepath"
"runtime"
"testing"
)
// contains returns true if vector contains the string s.
func contains(vector []string, s string) bool {
for _, elem := range vector {
if elem == s {
return true
}
}
return false
}
func setupGlobDirRoot(t *testing.T, fs Fs) string {
path := testDir(fs)
setupGlobFiles(t, fs, path)
return path
}
func setupGlobDirReusePath(t *testing.T, fs Fs, path string) string {
testRegistry[fs] = append(testRegistry[fs], path)
return setupGlobFiles(t, fs, path)
}
func setupGlobFiles(t *testing.T, fs Fs, path string) string {
testSubDir := filepath.Join(path, "globs", "bobs")
err := fs.MkdirAll(testSubDir, 0700)
if err != nil && !os.IsExist(err) {
t.Fatal(err)
}
f, err := fs.Create(filepath.Join(testSubDir, "/matcher"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 1 content")
f.Close()
f, err = fs.Create(filepath.Join(testSubDir, "/../submatcher"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 2 content")
f.Close()
f, err = fs.Create(filepath.Join(testSubDir, "/../../match"))
if err != nil {
t.Fatal(err)
}
f.WriteString("Testfile 3 content")
f.Close()
return testSubDir
}
func TestGlob(t *testing.T) {
defer removeAllTestFiles(t)
var testDir string
for i, fs := range Fss {
if i == 0 {
testDir = setupGlobDirRoot(t, fs)
} else {
setupGlobDirReusePath(t, fs, testDir)
}
}
var globTests = []struct {
pattern, result string
}{
{testDir + "/globs/bobs/matcher", testDir + "/globs/bobs/matcher"},
{testDir + "/globs/*/mat?her", testDir + "/globs/bobs/matcher"},
{testDir + "/globs/bobs/../*", testDir + "/globs/submatcher"},
{testDir + "/match", testDir + "/match"},
}
for _, fs := range Fss {
for _, tt := range globTests {
pattern := tt.pattern
result := tt.result
if runtime.GOOS == "windows" {
pattern = filepath.Clean(pattern)
result = filepath.Clean(result)
}
matches, err := Glob(fs, pattern)
if err != nil {
t.Errorf("Glob error for %q: %s", pattern, err)
continue
}
if !contains(matches, result) {
t.Errorf("Glob(%#q) = %#v want %v", pattern, matches, result)
}
}
for _, pattern := range []string{"no_match", "../*/no_match"} {
matches, err := Glob(fs, pattern)
if err != nil {
t.Errorf("Glob error for %q: %s", pattern, err)
continue
}
if len(matches) != 0 {
t.Errorf("Glob(%#q) = %#v want []", pattern, matches)
}
}
}
}
func TestGlobSymlink(t *testing.T) {
defer removeAllTestFiles(t)
fs := &OsFs{}
testDir := setupGlobDirRoot(t, fs)
err := os.Symlink("target", filepath.Join(testDir, "symlink"))
if err != nil {
t.Skipf("skipping on %s", runtime.GOOS)
}
var globSymlinkTests = []struct {
path, dest string
brokenLink bool
}{
{"test1", "link1", false},
{"test2", "link2", true},
}
for _, tt := range globSymlinkTests {
path := filepath.Join(testDir, tt.path)
dest := filepath.Join(testDir, tt.dest)
f, err := fs.Create(path)
if err != nil {
t.Fatal(err)
}
if err := f.Close(); err != nil {
t.Fatal(err)
}
err = os.Symlink(path, dest)
if err != nil {
t.Fatal(err)
}
if tt.brokenLink {
// Break the symlink.
fs.Remove(path)
}
matches, err := Glob(fs, dest)
if err != nil {
t.Errorf("GlobSymlink error for %q: %s", dest, err)
}
if !contains(matches, dest) {
t.Errorf("Glob(%#q) = %#v want %v", dest, matches, dest)
}
}
}
func TestGlobError(t *testing.T) {
for _, fs := range Fss {
_, err := Glob(fs, "[7]")
if err != nil {
t.Error("expected error for bad pattern; got none")
}
}
}

View File

@@ -176,6 +176,9 @@ func (f *File) Read(b []byte) (n int, err error) {
if len(b) > 0 && int(f.at) == len(f.fileData.data) {
return 0, io.EOF
}
if int(f.at) > len(f.fileData.data) {
return 0, io.ErrUnexpectedEOF
}
if len(f.fileData.data)-int(f.at) >= len(b) {
n = len(b)
} else {

View File

@@ -1,154 +0,0 @@
package mem
import (
"testing"
"time"
)
func TestFileDataNameRace(t *testing.T) {
t.Parallel()
const someName = "someName"
const someOtherName = "someOtherName"
d := FileData{
name: someName,
}
if d.Name() != someName {
t.Errorf("Failed to read correct Name, was %v", d.Name())
}
ChangeFileName(&d, someOtherName)
if d.Name() != someOtherName {
t.Errorf("Failed to set Name, was %v", d.Name())
}
go func() {
ChangeFileName(&d, someName)
}()
if d.Name() != someName && d.Name() != someOtherName {
t.Errorf("Failed to read either Name, was %v", d.Name())
}
}
func TestFileDataModTimeRace(t *testing.T) {
t.Parallel()
someTime := time.Now()
someOtherTime := someTime.Add(1 * time.Minute)
d := FileData{
modtime: someTime,
}
s := FileInfo{
FileData: &d,
}
if s.ModTime() != someTime {
t.Errorf("Failed to read correct value, was %v", s.ModTime())
}
SetModTime(&d, someOtherTime)
if s.ModTime() != someOtherTime {
t.Errorf("Failed to set ModTime, was %v", s.ModTime())
}
go func() {
SetModTime(&d, someTime)
}()
if s.ModTime() != someTime && s.ModTime() != someOtherTime {
t.Errorf("Failed to read either modtime, was %v", s.ModTime())
}
}
func TestFileDataModeRace(t *testing.T) {
t.Parallel()
const someMode = 0777
const someOtherMode = 0660
d := FileData{
mode: someMode,
}
s := FileInfo{
FileData: &d,
}
if s.Mode() != someMode {
t.Errorf("Failed to read correct value, was %v", s.Mode())
}
SetMode(&d, someOtherMode)
if s.Mode() != someOtherMode {
t.Errorf("Failed to set Mode, was %v", s.Mode())
}
go func() {
SetMode(&d, someMode)
}()
if s.Mode() != someMode && s.Mode() != someOtherMode {
t.Errorf("Failed to read either mode, was %v", s.Mode())
}
}
func TestFileDataIsDirRace(t *testing.T) {
t.Parallel()
d := FileData{
dir: true,
}
s := FileInfo{
FileData: &d,
}
if s.IsDir() != true {
t.Errorf("Failed to read correct value, was %v", s.IsDir())
}
go func() {
s.Lock()
d.dir = false
s.Unlock()
}()
//just logging the value to trigger a read:
t.Logf("Value is %v", s.IsDir())
}
func TestFileDataSizeRace(t *testing.T) {
t.Parallel()
const someData = "Hello"
const someOtherDataSize = "Hello World"
d := FileData{
data: []byte(someData),
dir: false,
}
s := FileInfo{
FileData: &d,
}
if s.Size() != int64(len(someData)) {
t.Errorf("Failed to read correct value, was %v", s.Size())
}
go func() {
s.Lock()
d.data = []byte(someOtherDataSize)
s.Unlock()
}()
//just logging the value to trigger a read:
t.Logf("Value is %v", s.Size())
//Testing the Dir size case
d.dir = true
if s.Size() != int64(42) {
t.Errorf("Failed to read correct value for dir, was %v", s.Size())
}
}

View File

@@ -1,421 +0,0 @@
package afero
import (
"fmt"
"os"
"path/filepath"
"runtime"
"testing"
"time"
)
func TestNormalizePath(t *testing.T) {
type test struct {
input string
expected string
}
data := []test{
{".", FilePathSeparator},
{"./", FilePathSeparator},
{"..", FilePathSeparator},
{"../", FilePathSeparator},
{"./..", FilePathSeparator},
{"./../", FilePathSeparator},
}
for i, d := range data {
cpath := normalizePath(d.input)
if d.expected != cpath {
t.Errorf("Test %d failed. Expected %q got %q", i, d.expected, cpath)
}
}
}
func TestPathErrors(t *testing.T) {
path := filepath.Join(".", "some", "path")
path2 := filepath.Join(".", "different", "path")
fs := NewMemMapFs()
perm := os.FileMode(0755)
// relevant functions:
// func (m *MemMapFs) Chmod(name string, mode os.FileMode) error
// func (m *MemMapFs) Chtimes(name string, atime time.Time, mtime time.Time) error
// func (m *MemMapFs) Create(name string) (File, error)
// func (m *MemMapFs) Mkdir(name string, perm os.FileMode) error
// func (m *MemMapFs) MkdirAll(path string, perm os.FileMode) error
// func (m *MemMapFs) Open(name string) (File, error)
// func (m *MemMapFs) OpenFile(name string, flag int, perm os.FileMode) (File, error)
// func (m *MemMapFs) Remove(name string) error
// func (m *MemMapFs) Rename(oldname, newname string) error
// func (m *MemMapFs) Stat(name string) (os.FileInfo, error)
err := fs.Chmod(path, perm)
checkPathError(t, err, "Chmod")
err = fs.Chtimes(path, time.Now(), time.Now())
checkPathError(t, err, "Chtimes")
// fs.Create doesn't return an error
err = fs.Mkdir(path2, perm)
if err != nil {
t.Error(err)
}
err = fs.Mkdir(path2, perm)
checkPathError(t, err, "Mkdir")
err = fs.MkdirAll(path2, perm)
if err != nil {
t.Error("MkdirAll:", err)
}
_, err = fs.Open(path)
checkPathError(t, err, "Open")
_, err = fs.OpenFile(path, os.O_RDWR, perm)
checkPathError(t, err, "OpenFile")
err = fs.Remove(path)
checkPathError(t, err, "Remove")
err = fs.RemoveAll(path)
if err != nil {
t.Error("RemoveAll:", err)
}
err = fs.Rename(path, path2)
checkPathError(t, err, "Rename")
_, err = fs.Stat(path)
checkPathError(t, err, "Stat")
}
func checkPathError(t *testing.T, err error, op string) {
pathErr, ok := err.(*os.PathError)
if !ok {
t.Error(op+":", err, "is not a os.PathError")
return
}
_, ok = pathErr.Err.(*os.PathError)
if ok {
t.Error(op+":", err, "contains another os.PathError")
}
}
// Ensure Permissions are set on OpenFile/Mkdir/MkdirAll
func TestPermSet(t *testing.T) {
const fileName = "/myFileTest"
const dirPath = "/myDirTest"
const dirPathAll = "/my/path/to/dir"
const fileMode = os.FileMode(0765)
// directories will also have the directory bit set
const dirMode = fileMode | os.ModeDir
fs := NewMemMapFs()
// Test Openfile
f, err := fs.OpenFile(fileName, os.O_CREATE, fileMode)
if err != nil {
t.Errorf("OpenFile Create failed: %s", err)
return
}
f.Close()
s, err := fs.Stat(fileName)
if err != nil {
t.Errorf("Stat failed: %s", err)
return
}
if s.Mode().String() != fileMode.String() {
t.Errorf("Permissions Incorrect: %s != %s", s.Mode().String(), fileMode.String())
return
}
// Test Mkdir
err = fs.Mkdir(dirPath, dirMode)
if err != nil {
t.Errorf("MkDir Create failed: %s", err)
return
}
s, err = fs.Stat(dirPath)
if err != nil {
t.Errorf("Stat failed: %s", err)
return
}
// sets File
if s.Mode().String() != dirMode.String() {
t.Errorf("Permissions Incorrect: %s != %s", s.Mode().String(), dirMode.String())
return
}
// Test MkdirAll
err = fs.MkdirAll(dirPathAll, dirMode)
if err != nil {
t.Errorf("MkDir Create failed: %s", err)
return
}
s, err = fs.Stat(dirPathAll)
if err != nil {
t.Errorf("Stat failed: %s", err)
return
}
if s.Mode().String() != dirMode.String() {
t.Errorf("Permissions Incorrect: %s != %s", s.Mode().String(), dirMode.String())
return
}
}
// Fails if multiple file objects use the same file.at counter in MemMapFs
func TestMultipleOpenFiles(t *testing.T) {
defer removeAllTestFiles(t)
const fileName = "afero-demo2.txt"
var data = make([][]byte, len(Fss))
for i, fs := range Fss {
dir := testDir(fs)
path := filepath.Join(dir, fileName)
fh1, err := fs.Create(path)
if err != nil {
t.Error("fs.Create failed: " + err.Error())
}
_, err = fh1.Write([]byte("test"))
if err != nil {
t.Error("fh.Write failed: " + err.Error())
}
_, err = fh1.Seek(0, os.SEEK_SET)
if err != nil {
t.Error(err)
}
fh2, err := fs.OpenFile(path, os.O_RDWR, 0777)
if err != nil {
t.Error("fs.OpenFile failed: " + err.Error())
}
_, err = fh2.Seek(0, os.SEEK_END)
if err != nil {
t.Error(err)
}
_, err = fh2.Write([]byte("data"))
if err != nil {
t.Error(err)
}
err = fh2.Close()
if err != nil {
t.Error(err)
}
_, err = fh1.Write([]byte("data"))
if err != nil {
t.Error(err)
}
err = fh1.Close()
if err != nil {
t.Error(err)
}
// the file now should contain "datadata"
data[i], err = ReadFile(fs, path)
if err != nil {
t.Error(err)
}
}
for i, fs := range Fss {
if i == 0 {
continue
}
if string(data[0]) != string(data[i]) {
t.Errorf("%s and %s don't behave the same\n"+
"%s: \"%s\"\n%s: \"%s\"\n",
Fss[0].Name(), fs.Name(), Fss[0].Name(), data[0], fs.Name(), data[i])
}
}
}
// Test if file.Write() fails when opened as read only
func TestReadOnly(t *testing.T) {
defer removeAllTestFiles(t)
const fileName = "afero-demo.txt"
for _, fs := range Fss {
dir := testDir(fs)
path := filepath.Join(dir, fileName)
f, err := fs.Create(path)
if err != nil {
t.Error(fs.Name()+":", "fs.Create failed: "+err.Error())
}
_, err = f.Write([]byte("test"))
if err != nil {
t.Error(fs.Name()+":", "Write failed: "+err.Error())
}
f.Close()
f, err = fs.Open(path)
if err != nil {
t.Error("fs.Open failed: " + err.Error())
}
_, err = f.Write([]byte("data"))
if err == nil {
t.Error(fs.Name()+":", "No write error")
}
f.Close()
f, err = fs.OpenFile(path, os.O_RDONLY, 0644)
if err != nil {
t.Error("fs.Open failed: " + err.Error())
}
_, err = f.Write([]byte("data"))
if err == nil {
t.Error(fs.Name()+":", "No write error")
}
f.Close()
}
}
func TestWriteCloseTime(t *testing.T) {
defer removeAllTestFiles(t)
const fileName = "afero-demo.txt"
for _, fs := range Fss {
dir := testDir(fs)
path := filepath.Join(dir, fileName)
f, err := fs.Create(path)
if err != nil {
t.Error(fs.Name()+":", "fs.Create failed: "+err.Error())
}
f.Close()
f, err = fs.Create(path)
if err != nil {
t.Error(fs.Name()+":", "fs.Create failed: "+err.Error())
}
fi, err := f.Stat()
if err != nil {
t.Error(fs.Name()+":", "Stat failed: "+err.Error())
}
timeBefore := fi.ModTime()
// sorry for the delay, but we have to make sure time advances,
// also on non Un*x systems...
switch runtime.GOOS {
case "windows":
time.Sleep(2 * time.Second)
case "darwin":
time.Sleep(1 * time.Second)
default: // depending on the FS, this may work with < 1 second, on my old ext3 it does not
time.Sleep(1 * time.Second)
}
_, err = f.Write([]byte("test"))
if err != nil {
t.Error(fs.Name()+":", "Write failed: "+err.Error())
}
f.Close()
fi, err = fs.Stat(path)
if err != nil {
t.Error(fs.Name()+":", "fs.Stat failed: "+err.Error())
}
if fi.ModTime().Equal(timeBefore) {
t.Error(fs.Name()+":", "ModTime was not set on Close()")
}
}
}
// This test should be run with the race detector on:
// go test -race -v -timeout 10s -run TestRacingDeleteAndClose
func TestRacingDeleteAndClose(t *testing.T) {
fs := NewMemMapFs()
pathname := "testfile"
f, err := fs.Create(pathname)
if err != nil {
t.Fatal(err)
}
in := make(chan bool)
go func() {
<-in
f.Close()
}()
go func() {
<-in
fs.Remove(pathname)
}()
close(in)
}
// This test should be run with the race detector on:
// go test -run TestMemFsDataRace -race
func TestMemFsDataRace(t *testing.T) {
const dir = "test_dir"
fs := NewMemMapFs()
if err := fs.MkdirAll(dir, 0777); err != nil {
t.Fatal(err)
}
const n = 1000
done := make(chan struct{})
go func() {
defer close(done)
for i := 0; i < n; i++ {
fname := filepath.Join(dir, fmt.Sprintf("%d.txt", i))
if err := WriteFile(fs, fname, []byte(""), 0777); err != nil {
panic(err)
}
if err := fs.Remove(fname); err != nil {
panic(err)
}
}
}()
loop:
for {
select {
case <-done:
break loop
default:
_, err := ReadDir(fs, dir)
if err != nil {
t.Fatal(err)
}
}
}
}
func TestMemFsDirMode(t *testing.T) {
fs := NewMemMapFs()
err := fs.Mkdir("/testDir1", 0644)
if err != nil {
t.Error(err)
}
err = fs.MkdirAll("/sub/testDir2", 0644)
if err != nil {
t.Error(err)
}
info, err := fs.Stat("/testDir1")
if err != nil {
t.Error(err)
}
if !info.IsDir() {
t.Error("should be a directory")
}
if !info.Mode().IsDir() {
t.Error("FileMode is not directory")
}
info, err = fs.Stat("/sub/testDir2")
if err != nil {
t.Error(err)
}
if !info.IsDir() {
t.Error("should be a directory")
}
if !info.Mode().IsDir() {
t.Error("FileMode is not directory")
}
}

View File

@@ -19,6 +19,8 @@ import (
"time"
)
var _ Lstater = (*OsFs)(nil)
// OsFs is a Fs implementation that uses functions provided by the os package.
//
// For details in any method, check the documentation of the os package
@@ -92,3 +94,8 @@ func (OsFs) Chmod(name string, mode os.FileMode) error {
func (OsFs) Chtimes(name string, atime time.Time, mtime time.Time) error {
return os.Chtimes(name, atime, mtime)
}
func (OsFs) LstatIfPossible(name string) (os.FileInfo, bool, error) {
fi, err := os.Lstat(name)
return fi, true, err
}

View File

@@ -60,7 +60,7 @@ func walk(fs Fs, path string, info os.FileInfo, walkFn filepath.WalkFunc) error
for _, name := range names {
filename := filepath.Join(path, name)
fileInfo, err := lstatIfOs(fs, filename)
fileInfo, err := lstatIfPossible(fs, filename)
if err != nil {
if err := walkFn(filename, fileInfo, err); err != nil && err != filepath.SkipDir {
return err
@@ -77,15 +77,13 @@ func walk(fs Fs, path string, info os.FileInfo, walkFn filepath.WalkFunc) error
return nil
}
// if the filesystem is OsFs use Lstat, else use fs.Stat
func lstatIfOs(fs Fs, path string) (info os.FileInfo, err error) {
_, ok := fs.(*OsFs)
if ok {
info, err = os.Lstat(path)
} else {
info, err = fs.Stat(path)
// if the filesystem supports it, use Lstat, else use fs.Stat
func lstatIfPossible(fs Fs, path string) (os.FileInfo, error) {
if lfs, ok := fs.(Lstater); ok {
fi, _, err := lfs.LstatIfPossible(path)
return fi, err
}
return
return fs.Stat(path)
}
// Walk walks the file tree rooted at root, calling walkFn for each file or
@@ -100,7 +98,7 @@ func (a Afero) Walk(root string, walkFn filepath.WalkFunc) error {
}
func Walk(fs Fs, root string, walkFn filepath.WalkFunc) error {
info, err := lstatIfOs(fs, root)
info, err := lstatIfPossible(fs, root)
if err != nil {
return walkFn(root, nil, err)
}

View File

@@ -1,69 +0,0 @@
// Copyright © 2014 Steve Francia <spf@spf13.com>.
// Copyright 2009 The Go Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package afero
import (
"fmt"
"os"
"testing"
)
func TestWalk(t *testing.T) {
defer removeAllTestFiles(t)
var testDir string
for i, fs := range Fss {
if i == 0 {
testDir = setupTestDirRoot(t, fs)
} else {
setupTestDirReusePath(t, fs, testDir)
}
}
outputs := make([]string, len(Fss))
for i, fs := range Fss {
walkFn := func(path string, info os.FileInfo, err error) error {
if err != nil {
t.Error("walkFn err:", err)
}
var size int64
if !info.IsDir() {
size = info.Size()
}
outputs[i] += fmt.Sprintln(path, info.Name(), size, info.IsDir(), err)
return nil
}
err := Walk(fs, testDir, walkFn)
if err != nil {
t.Error(err)
}
}
fail := false
for i, o := range outputs {
if i == 0 {
continue
}
if o != outputs[i-1] {
fail = true
break
}
}
if fail {
t.Log("Walk outputs not equal!")
for i, o := range outputs {
t.Log(Fss[i].Name() + "\n" + o)
}
t.Fail()
}
}

View File

@@ -6,6 +6,8 @@ import (
"time"
)
var _ Lstater = (*ReadOnlyFs)(nil)
type ReadOnlyFs struct {
source Fs
}
@@ -34,6 +36,14 @@ func (r *ReadOnlyFs) Stat(name string) (os.FileInfo, error) {
return r.source.Stat(name)
}
func (r *ReadOnlyFs) LstatIfPossible(name string) (os.FileInfo, bool, error) {
if lsf, ok := r.source.(Lstater); ok {
return lsf.LstatIfPossible(name)
}
fi, err := r.Stat(name)
return fi, false, err
}
func (r *ReadOnlyFs) Rename(o, n string) error {
return syscall.EPERM
}

View File

@@ -1,96 +0,0 @@
package afero
import (
"regexp"
"testing"
)
func TestFilterReadOnly(t *testing.T) {
fs := &ReadOnlyFs{source: &MemMapFs{}}
_, err := fs.Create("/file.txt")
if err == nil {
t.Errorf("Did not fail to create file")
}
// t.Logf("ERR=%s", err)
}
func TestFilterReadonlyRemoveAndRead(t *testing.T) {
mfs := &MemMapFs{}
fh, err := mfs.Create("/file.txt")
fh.Write([]byte("content here"))
fh.Close()
fs := NewReadOnlyFs(mfs)
err = fs.Remove("/file.txt")
if err == nil {
t.Errorf("Did not fail to remove file")
}
fh, err = fs.Open("/file.txt")
if err != nil {
t.Errorf("Failed to open file: %s", err)
}
buf := make([]byte, len("content here"))
_, err = fh.Read(buf)
fh.Close()
if string(buf) != "content here" {
t.Errorf("Failed to read file: %s", err)
}
err = mfs.Remove("/file.txt")
if err != nil {
t.Errorf("Failed to remove file")
}
fh, err = fs.Open("/file.txt")
if err == nil {
fh.Close()
t.Errorf("File still present")
}
}
func TestFilterRegexp(t *testing.T) {
fs := NewRegexpFs(&MemMapFs{}, regexp.MustCompile(`\.txt$`))
_, err := fs.Create("/file.html")
if err == nil {
t.Errorf("Did not fail to create file")
}
// t.Logf("ERR=%s", err)
}
func TestFilterRORegexpChain(t *testing.T) {
rofs := &ReadOnlyFs{source: &MemMapFs{}}
fs := &RegexpFs{re: regexp.MustCompile(`\.txt$`), source: rofs}
_, err := fs.Create("/file.txt")
if err == nil {
t.Errorf("Did not fail to create file")
}
// t.Logf("ERR=%s", err)
}
func TestFilterRegexReadDir(t *testing.T) {
mfs := &MemMapFs{}
fs1 := &RegexpFs{re: regexp.MustCompile(`\.txt$`), source: mfs}
fs := &RegexpFs{re: regexp.MustCompile(`^a`), source: fs1}
mfs.MkdirAll("/dir/sub", 0777)
for _, name := range []string{"afile.txt", "afile.html", "bfile.txt"} {
for _, dir := range []string{"/dir/", "/dir/sub/"} {
fh, _ := mfs.Create(dir + name)
fh.Close()
}
}
files, _ := ReadDir(fs, "/dir")
if len(files) != 2 { // afile.txt, sub
t.Errorf("Got wrong number of files: %#v", files)
}
f, _ := fs.Open("/dir/sub")
names, _ := f.Readdirnames(-1)
if len(names) != 1 {
t.Errorf("Got wrong number of names: %v", names)
}
}

View File

@@ -1,95 +0,0 @@
// Copyright © 2015 Jerry Jacobs <jerry.jacobs@xor-gate.org>.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package sftpfs
import (
"github.com/pkg/sftp"
"os"
)
type File struct {
fd *sftp.File
}
func FileOpen(s *sftp.Client, name string) (*File, error) {
fd, err := s.Open(name)
if err != nil {
return &File{}, err
}
return &File{fd: fd}, nil
}
func FileCreate(s *sftp.Client, name string) (*File, error) {
fd, err := s.Create(name)
if err != nil {
return &File{}, err
}
return &File{fd: fd}, nil
}
func (f *File) Close() error {
return f.fd.Close()
}
func (f *File) Name() string {
return f.fd.Name()
}
func (f *File) Stat() (os.FileInfo, error) {
return f.fd.Stat()
}
func (f *File) Sync() error {
return nil
}
func (f *File) Truncate(size int64) error {
return f.fd.Truncate(size)
}
func (f *File) Read(b []byte) (n int, err error) {
return f.fd.Read(b)
}
// TODO
func (f *File) ReadAt(b []byte, off int64) (n int, err error) {
return 0, nil
}
// TODO
func (f *File) Readdir(count int) (res []os.FileInfo, err error) {
return nil, nil
}
// TODO
func (f *File) Readdirnames(n int) (names []string, err error) {
return nil, nil
}
func (f *File) Seek(offset int64, whence int) (int64, error) {
return f.fd.Seek(offset, whence)
}
func (f *File) Write(b []byte) (n int, err error) {
return f.fd.Write(b)
}
// TODO
func (f *File) WriteAt(b []byte, off int64) (n int, err error) {
return 0, nil
}
func (f *File) WriteString(s string) (ret int, err error) {
return f.fd.Write([]byte(s))
}

View File

@@ -1,129 +0,0 @@
// Copyright © 2015 Jerry Jacobs <jerry.jacobs@xor-gate.org>.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package sftpfs
import (
"os"
"time"
"github.com/pkg/sftp"
"github.com/spf13/afero"
)
// Fs is a afero.Fs implementation that uses functions provided by the sftp package.
//
// For details in any method, check the documentation of the sftp package
// (github.com/pkg/sftp).
type Fs struct {
client *sftp.Client
}
func New(client *sftp.Client) afero.Fs {
return &Fs{client: client}
}
func (s Fs) Name() string { return "sftpfs" }
func (s Fs) Create(name string) (afero.File, error) {
return FileCreate(s.client, name)
}
func (s Fs) Mkdir(name string, perm os.FileMode) error {
err := s.client.Mkdir(name)
if err != nil {
return err
}
return s.client.Chmod(name, perm)
}
func (s Fs) MkdirAll(path string, perm os.FileMode) error {
// Fast path: if we can tell whether path is a directory or file, stop with success or error.
dir, err := s.Stat(path)
if err == nil {
if dir.IsDir() {
return nil
}
return err
}
// Slow path: make sure parent exists and then call Mkdir for path.
i := len(path)
for i > 0 && os.IsPathSeparator(path[i-1]) { // Skip trailing path separator.
i--
}
j := i
for j > 0 && !os.IsPathSeparator(path[j-1]) { // Scan backward over element.
j--
}
if j > 1 {
// Create parent
err = s.MkdirAll(path[0:j-1], perm)
if err != nil {
return err
}
}
// Parent now exists; invoke Mkdir and use its result.
err = s.Mkdir(path, perm)
if err != nil {
// Handle arguments like "foo/." by
// double-checking that directory doesn't exist.
dir, err1 := s.Lstat(path)
if err1 == nil && dir.IsDir() {
return nil
}
return err
}
return nil
}
func (s Fs) Open(name string) (afero.File, error) {
return FileOpen(s.client, name)
}
func (s Fs) OpenFile(name string, flag int, perm os.FileMode) (afero.File, error) {
return nil, nil
}
func (s Fs) Remove(name string) error {
return s.client.Remove(name)
}
func (s Fs) RemoveAll(path string) error {
// TODO have a look at os.RemoveAll
// https://github.com/golang/go/blob/master/src/os/path.go#L66
return nil
}
func (s Fs) Rename(oldname, newname string) error {
return s.client.Rename(oldname, newname)
}
func (s Fs) Stat(name string) (os.FileInfo, error) {
return s.client.Stat(name)
}
func (s Fs) Lstat(p string) (os.FileInfo, error) {
return s.client.Lstat(p)
}
func (s Fs) Chmod(name string, mode os.FileMode) error {
return s.client.Chmod(name, mode)
}
func (s Fs) Chtimes(name string, atime time.Time, mtime time.Time) error {
return s.client.Chtimes(name, atime, mtime)
}

View File

@@ -1,286 +0,0 @@
// Copyright © 2015 Jerry Jacobs <jerry.jacobs@xor-gate.org>.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package afero
import (
"testing"
"os"
"log"
"fmt"
"net"
"flag"
"time"
"io/ioutil"
"crypto/rsa"
_rand "crypto/rand"
"encoding/pem"
"crypto/x509"
"golang.org/x/crypto/ssh"
"github.com/pkg/sftp"
)
type SftpFsContext struct {
sshc *ssh.Client
sshcfg *ssh.ClientConfig
sftpc *sftp.Client
}
// TODO we only connect with hardcoded user+pass for now
// it should be possible to use $HOME/.ssh/id_rsa to login into the stub sftp server
func SftpConnect(user, password, host string) (*SftpFsContext, error) {
/*
pemBytes, err := ioutil.ReadFile(os.Getenv("HOME") + "/.ssh/id_rsa")
if err != nil {
return nil,err
}
signer, err := ssh.ParsePrivateKey(pemBytes)
if err != nil {
return nil,err
}
sshcfg := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{
ssh.Password(password),
ssh.PublicKeys(signer),
},
}
*/
sshcfg := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{
ssh.Password(password),
},
}
sshc, err := ssh.Dial("tcp", host, sshcfg)
if err != nil {
return nil,err
}
sftpc, err := sftp.NewClient(sshc)
if err != nil {
return nil,err
}
ctx := &SftpFsContext{
sshc: sshc,
sshcfg: sshcfg,
sftpc: sftpc,
}
return ctx,nil
}
func (ctx *SftpFsContext) Disconnect() error {
ctx.sftpc.Close()
ctx.sshc.Close()
return nil
}
// TODO for such a weird reason rootpath is "." when writing "file1" with afero sftp backend
func RunSftpServer(rootpath string) {
var (
readOnly bool
debugLevelStr string
debugLevel int
debugStderr bool
rootDir string
)
flag.BoolVar(&readOnly, "R", false, "read-only server")
flag.BoolVar(&debugStderr, "e", true, "debug to stderr")
flag.StringVar(&debugLevelStr, "l", "none", "debug level")
flag.StringVar(&rootDir, "root", rootpath, "root directory")
flag.Parse()
debugStream := ioutil.Discard
if debugStderr {
debugStream = os.Stderr
debugLevel = 1
}
// An SSH server is represented by a ServerConfig, which holds
// certificate details and handles authentication of ServerConns.
config := &ssh.ServerConfig{
PasswordCallback: func(c ssh.ConnMetadata, pass []byte) (*ssh.Permissions, error) {
// Should use constant-time compare (or better, salt+hash) in
// a production setting.
fmt.Fprintf(debugStream, "Login: %s\n", c.User())
if c.User() == "test" && string(pass) == "test" {
return nil, nil
}
return nil, fmt.Errorf("password rejected for %q", c.User())
},
}
privateBytes, err := ioutil.ReadFile("./test/id_rsa")
if err != nil {
log.Fatal("Failed to load private key", err)
}
private, err := ssh.ParsePrivateKey(privateBytes)
if err != nil {
log.Fatal("Failed to parse private key", err)
}
config.AddHostKey(private)
// Once a ServerConfig has been configured, connections can be
// accepted.
listener, err := net.Listen("tcp", "0.0.0.0:2022")
if err != nil {
log.Fatal("failed to listen for connection", err)
}
fmt.Printf("Listening on %v\n", listener.Addr())
nConn, err := listener.Accept()
if err != nil {
log.Fatal("failed to accept incoming connection", err)
}
// Before use, a handshake must be performed on the incoming
// net.Conn.
_, chans, reqs, err := ssh.NewServerConn(nConn, config)
if err != nil {
log.Fatal("failed to handshake", err)
}
fmt.Fprintf(debugStream, "SSH server established\n")
// The incoming Request channel must be serviced.
go ssh.DiscardRequests(reqs)
// Service the incoming Channel channel.
for newChannel := range chans {
// Channels have a type, depending on the application level
// protocol intended. In the case of an SFTP session, this is "subsystem"
// with a payload string of "<length=4>sftp"
fmt.Fprintf(debugStream, "Incoming channel: %s\n", newChannel.ChannelType())
if newChannel.ChannelType() != "session" {
newChannel.Reject(ssh.UnknownChannelType, "unknown channel type")
fmt.Fprintf(debugStream, "Unknown channel type: %s\n", newChannel.ChannelType())
continue
}
channel, requests, err := newChannel.Accept()
if err != nil {
log.Fatal("could not accept channel.", err)
}
fmt.Fprintf(debugStream, "Channel accepted\n")
// Sessions have out-of-band requests such as "shell",
// "pty-req" and "env". Here we handle only the
// "subsystem" request.
go func(in <-chan *ssh.Request) {
for req := range in {
fmt.Fprintf(debugStream, "Request: %v\n", req.Type)
ok := false
switch req.Type {
case "subsystem":
fmt.Fprintf(debugStream, "Subsystem: %s\n", req.Payload[4:])
if string(req.Payload[4:]) == "sftp" {
ok = true
}
}
fmt.Fprintf(debugStream, " - accepted: %v\n", ok)
req.Reply(ok, nil)
}
}(requests)
server, err := sftp.NewServer(channel, channel, debugStream, debugLevel, readOnly, rootpath)
if err != nil {
log.Fatal(err)
}
if err := server.Serve(); err != nil {
log.Fatal("sftp server completed with error:", err)
}
}
}
// MakeSSHKeyPair make a pair of public and private keys for SSH access.
// Public key is encoded in the format for inclusion in an OpenSSH authorized_keys file.
// Private Key generated is PEM encoded
func MakeSSHKeyPair(bits int, pubKeyPath, privateKeyPath string) error {
privateKey, err := rsa.GenerateKey(_rand.Reader, bits)
if err != nil {
return err
}
// generate and write private key as PEM
privateKeyFile, err := os.Create(privateKeyPath)
defer privateKeyFile.Close()
if err != nil {
return err
}
privateKeyPEM := &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(privateKey)}
if err := pem.Encode(privateKeyFile, privateKeyPEM); err != nil {
return err
}
// generate and write public key
pub, err := ssh.NewPublicKey(&privateKey.PublicKey)
if err != nil {
return err
}
return ioutil.WriteFile(pubKeyPath, ssh.MarshalAuthorizedKey(pub), 0655)
}
func TestSftpCreate(t *testing.T) {
os.Mkdir("./test", 0777)
MakeSSHKeyPair(1024, "./test/id_rsa.pub", "./test/id_rsa")
go RunSftpServer("./test/")
time.Sleep(5 * time.Second)
ctx, err := SftpConnect("test", "test", "localhost:2022")
if err != nil {
t.Fatal(err)
}
defer ctx.Disconnect()
var AppFs Fs = SftpFs{
SftpClient: ctx.sftpc,
}
AppFs.MkdirAll("test/dir1/dir2/dir3", os.FileMode(0777))
AppFs.Mkdir("test/foo", os.FileMode(0000))
AppFs.Chmod("test/foo", os.FileMode(0700))
AppFs.Mkdir("test/bar", os.FileMode(0777))
file, err := AppFs.Create("file1")
if err != nil {
t.Error(err)
}
defer file.Close()
file.Write([]byte("hello\t"))
file.WriteString("world!\n")
f1, err := AppFs.Open("file1")
if err != nil {
log.Fatalf("open: %v", err)
}
defer f1.Close()
b := make([]byte, 100)
_, err = f1.Read(b)
fmt.Println(string(b))
// TODO check here if "hello\tworld\n" is in buffer b
}

View File

@@ -21,32 +21,33 @@ import (
// successful read in the overlay will move the cursor position in the base layer
// by the number of bytes read.
type UnionFile struct {
base File
layer File
off int
files []os.FileInfo
Base File
Layer File
Merger DirsMerger
off int
files []os.FileInfo
}
func (f *UnionFile) Close() error {
// first close base, so we have a newer timestamp in the overlay. If we'd close
// the overlay first, we'd get a cacheStale the next time we access this file
// -> cache would be useless ;-)
if f.base != nil {
f.base.Close()
if f.Base != nil {
f.Base.Close()
}
if f.layer != nil {
return f.layer.Close()
if f.Layer != nil {
return f.Layer.Close()
}
return BADFD
}
func (f *UnionFile) Read(s []byte) (int, error) {
if f.layer != nil {
n, err := f.layer.Read(s)
if (err == nil || err == io.EOF) && f.base != nil {
if f.Layer != nil {
n, err := f.Layer.Read(s)
if (err == nil || err == io.EOF) && f.Base != nil {
// advance the file position also in the base file, the next
// call may be a write at this position (or a seek with SEEK_CUR)
if _, seekErr := f.base.Seek(int64(n), os.SEEK_CUR); seekErr != nil {
if _, seekErr := f.Base.Seek(int64(n), os.SEEK_CUR); seekErr != nil {
// only overwrite err in case the seek fails: we need to
// report an eventual io.EOF to the caller
err = seekErr
@@ -54,105 +55,135 @@ func (f *UnionFile) Read(s []byte) (int, error) {
}
return n, err
}
if f.base != nil {
return f.base.Read(s)
if f.Base != nil {
return f.Base.Read(s)
}
return 0, BADFD
}
func (f *UnionFile) ReadAt(s []byte, o int64) (int, error) {
if f.layer != nil {
n, err := f.layer.ReadAt(s, o)
if (err == nil || err == io.EOF) && f.base != nil {
_, err = f.base.Seek(o+int64(n), os.SEEK_SET)
if f.Layer != nil {
n, err := f.Layer.ReadAt(s, o)
if (err == nil || err == io.EOF) && f.Base != nil {
_, err = f.Base.Seek(o+int64(n), os.SEEK_SET)
}
return n, err
}
if f.base != nil {
return f.base.ReadAt(s, o)
if f.Base != nil {
return f.Base.ReadAt(s, o)
}
return 0, BADFD
}
func (f *UnionFile) Seek(o int64, w int) (pos int64, err error) {
if f.layer != nil {
pos, err = f.layer.Seek(o, w)
if (err == nil || err == io.EOF) && f.base != nil {
_, err = f.base.Seek(o, w)
if f.Layer != nil {
pos, err = f.Layer.Seek(o, w)
if (err == nil || err == io.EOF) && f.Base != nil {
_, err = f.Base.Seek(o, w)
}
return pos, err
}
if f.base != nil {
return f.base.Seek(o, w)
if f.Base != nil {
return f.Base.Seek(o, w)
}
return 0, BADFD
}
func (f *UnionFile) Write(s []byte) (n int, err error) {
if f.layer != nil {
n, err = f.layer.Write(s)
if err == nil && f.base != nil { // hmm, do we have fixed size files where a write may hit the EOF mark?
_, err = f.base.Write(s)
if f.Layer != nil {
n, err = f.Layer.Write(s)
if err == nil && f.Base != nil { // hmm, do we have fixed size files where a write may hit the EOF mark?
_, err = f.Base.Write(s)
}
return n, err
}
if f.base != nil {
return f.base.Write(s)
if f.Base != nil {
return f.Base.Write(s)
}
return 0, BADFD
}
func (f *UnionFile) WriteAt(s []byte, o int64) (n int, err error) {
if f.layer != nil {
n, err = f.layer.WriteAt(s, o)
if err == nil && f.base != nil {
_, err = f.base.WriteAt(s, o)
if f.Layer != nil {
n, err = f.Layer.WriteAt(s, o)
if err == nil && f.Base != nil {
_, err = f.Base.WriteAt(s, o)
}
return n, err
}
if f.base != nil {
return f.base.WriteAt(s, o)
if f.Base != nil {
return f.Base.WriteAt(s, o)
}
return 0, BADFD
}
func (f *UnionFile) Name() string {
if f.layer != nil {
return f.layer.Name()
if f.Layer != nil {
return f.Layer.Name()
}
return f.base.Name()
return f.Base.Name()
}
// DirsMerger is how UnionFile weaves two directories together.
// It takes the FileInfo slices from the layer and the base and returns a
// single view.
type DirsMerger func(lofi, bofi []os.FileInfo) ([]os.FileInfo, error)
var defaultUnionMergeDirsFn = func(lofi, bofi []os.FileInfo) ([]os.FileInfo, error) {
var files = make(map[string]os.FileInfo)
for _, fi := range lofi {
files[fi.Name()] = fi
}
for _, fi := range bofi {
if _, exists := files[fi.Name()]; !exists {
files[fi.Name()] = fi
}
}
rfi := make([]os.FileInfo, len(files))
i := 0
for _, fi := range files {
rfi[i] = fi
i++
}
return rfi, nil
}
// Readdir will weave the two directories together and
// return a single view of the overlayed directories
func (f *UnionFile) Readdir(c int) (ofi []os.FileInfo, err error) {
var merge DirsMerger = f.Merger
if merge == nil {
merge = defaultUnionMergeDirsFn
}
if f.off == 0 {
var files = make(map[string]os.FileInfo)
var rfi []os.FileInfo
if f.layer != nil {
rfi, err = f.layer.Readdir(-1)
var lfi []os.FileInfo
if f.Layer != nil {
lfi, err = f.Layer.Readdir(-1)
if err != nil {
return nil, err
}
for _, fi := range rfi {
files[fi.Name()] = fi
}
}
if f.base != nil {
rfi, err = f.base.Readdir(-1)
var bfi []os.FileInfo
if f.Base != nil {
bfi, err = f.Base.Readdir(-1)
if err != nil {
return nil, err
}
for _, fi := range rfi {
if _, exists := files[fi.Name()]; !exists {
files[fi.Name()] = fi
}
}
}
for _, fi := range files {
f.files = append(f.files, fi)
merged, err := merge(lfi, bfi)
if err != nil {
return nil, err
}
f.files = append(f.files, merged...)
}
if c == -1 {
return f.files[f.off:], nil
@@ -174,53 +205,53 @@ func (f *UnionFile) Readdirnames(c int) ([]string, error) {
}
func (f *UnionFile) Stat() (os.FileInfo, error) {
if f.layer != nil {
return f.layer.Stat()
if f.Layer != nil {
return f.Layer.Stat()
}
if f.base != nil {
return f.base.Stat()
if f.Base != nil {
return f.Base.Stat()
}
return nil, BADFD
}
func (f *UnionFile) Sync() (err error) {
if f.layer != nil {
err = f.layer.Sync()
if err == nil && f.base != nil {
err = f.base.Sync()
if f.Layer != nil {
err = f.Layer.Sync()
if err == nil && f.Base != nil {
err = f.Base.Sync()
}
return err
}
if f.base != nil {
return f.base.Sync()
if f.Base != nil {
return f.Base.Sync()
}
return BADFD
}
func (f *UnionFile) Truncate(s int64) (err error) {
if f.layer != nil {
err = f.layer.Truncate(s)
if err == nil && f.base != nil {
err = f.base.Truncate(s)
if f.Layer != nil {
err = f.Layer.Truncate(s)
if err == nil && f.Base != nil {
err = f.Base.Truncate(s)
}
return err
}
if f.base != nil {
return f.base.Truncate(s)
if f.Base != nil {
return f.Base.Truncate(s)
}
return BADFD
}
func (f *UnionFile) WriteString(s string) (n int, err error) {
if f.layer != nil {
n, err = f.layer.WriteString(s)
if err == nil && f.base != nil {
_, err = f.base.WriteString(s)
if f.Layer != nil {
n, err = f.Layer.WriteString(s)
if err == nil && f.Base != nil {
_, err = f.Base.WriteString(s)
}
return n, err
}
if f.base != nil {
return f.base.WriteString(s)
if f.Base != nil {
return f.Base.WriteString(s)
}
return 0, BADFD
}

View File

@@ -20,7 +20,6 @@ import (
"bytes"
"fmt"
"io"
"log"
"os"
"path/filepath"
"strings"
@@ -46,7 +45,7 @@ func WriteReader(fs Fs, path string, r io.Reader) (err error) {
err = fs.MkdirAll(ospath, 0777) // rwx, rw, r
if err != nil {
if err != os.ErrExist {
log.Panicln(err)
return err
}
}
}

View File

@@ -1,450 +0,0 @@
// Copyright ©2015 Steve Francia <spf@spf13.com>
// Portions Copyright ©2015 The Hugo Authors
//
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
package afero
import (
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
)
var testFS = new(MemMapFs)
func TestDirExists(t *testing.T) {
type test struct {
input string
expected bool
}
// First create a couple directories so there is something in the filesystem
//testFS := new(MemMapFs)
testFS.MkdirAll("/foo/bar", 0777)
data := []test{
{".", true},
{"./", true},
{"..", true},
{"../", true},
{"./..", true},
{"./../", true},
{"/foo/", true},
{"/foo", true},
{"/foo/bar", true},
{"/foo/bar/", true},
{"/", true},
{"/some-really-random-directory-name", false},
{"/some/really/random/directory/name", false},
{"./some-really-random-local-directory-name", false},
{"./some/really/random/local/directory/name", false},
}
for i, d := range data {
exists, _ := DirExists(testFS, filepath.FromSlash(d.input))
if d.expected != exists {
t.Errorf("Test %d %q failed. Expected %t got %t", i, d.input, d.expected, exists)
}
}
}
func TestIsDir(t *testing.T) {
testFS = new(MemMapFs)
type test struct {
input string
expected bool
}
data := []test{
{"./", true},
{"/", true},
{"./this-directory-does-not-existi", false},
{"/this-absolute-directory/does-not-exist", false},
}
for i, d := range data {
exists, _ := IsDir(testFS, d.input)
if d.expected != exists {
t.Errorf("Test %d failed. Expected %t got %t", i, d.expected, exists)
}
}
}
func TestIsEmpty(t *testing.T) {
testFS = new(MemMapFs)
zeroSizedFile, _ := createZeroSizedFileInTempDir()
defer deleteFileInTempDir(zeroSizedFile)
nonZeroSizedFile, _ := createNonZeroSizedFileInTempDir()
defer deleteFileInTempDir(nonZeroSizedFile)
emptyDirectory, _ := createEmptyTempDir()
defer deleteTempDir(emptyDirectory)
nonEmptyZeroLengthFilesDirectory, _ := createTempDirWithZeroLengthFiles()
defer deleteTempDir(nonEmptyZeroLengthFilesDirectory)
nonEmptyNonZeroLengthFilesDirectory, _ := createTempDirWithNonZeroLengthFiles()
defer deleteTempDir(nonEmptyNonZeroLengthFilesDirectory)
nonExistentFile := os.TempDir() + "/this-file-does-not-exist.txt"
nonExistentDir := os.TempDir() + "/this/direcotry/does/not/exist/"
fileDoesNotExist := fmt.Errorf("%q path does not exist", nonExistentFile)
dirDoesNotExist := fmt.Errorf("%q path does not exist", nonExistentDir)
type test struct {
input string
expectedResult bool
expectedErr error
}
data := []test{
{zeroSizedFile.Name(), true, nil},
{nonZeroSizedFile.Name(), false, nil},
{emptyDirectory, true, nil},
{nonEmptyZeroLengthFilesDirectory, false, nil},
{nonEmptyNonZeroLengthFilesDirectory, false, nil},
{nonExistentFile, false, fileDoesNotExist},
{nonExistentDir, false, dirDoesNotExist},
}
for i, d := range data {
exists, err := IsEmpty(testFS, d.input)
if d.expectedResult != exists {
t.Errorf("Test %d %q failed exists. Expected result %t got %t", i, d.input, d.expectedResult, exists)
}
if d.expectedErr != nil {
if d.expectedErr.Error() != err.Error() {
t.Errorf("Test %d failed with err. Expected %q(%#v) got %q(%#v)", i, d.expectedErr, d.expectedErr, err, err)
}
} else {
if d.expectedErr != err {
t.Errorf("Test %d failed. Expected error %q(%#v) got %q(%#v)", i, d.expectedErr, d.expectedErr, err, err)
}
}
}
}
func TestReaderContains(t *testing.T) {
for i, this := range []struct {
v1 string
v2 [][]byte
expect bool
}{
{"abc", [][]byte{[]byte("a")}, true},
{"abc", [][]byte{[]byte("b")}, true},
{"abcdefg", [][]byte{[]byte("efg")}, true},
{"abc", [][]byte{[]byte("d")}, false},
{"abc", [][]byte{[]byte("d"), []byte("e")}, false},
{"abc", [][]byte{[]byte("d"), []byte("a")}, true},
{"abc", [][]byte{[]byte("b"), []byte("e")}, true},
{"", nil, false},
{"", [][]byte{[]byte("a")}, false},
{"a", [][]byte{[]byte("")}, false},
{"", [][]byte{[]byte("")}, false}} {
result := readerContainsAny(strings.NewReader(this.v1), this.v2...)
if result != this.expect {
t.Errorf("[%d] readerContains: got %t but expected %t", i, result, this.expect)
}
}
if readerContainsAny(nil, []byte("a")) {
t.Error("readerContains with nil reader")
}
if readerContainsAny(nil, nil) {
t.Error("readerContains with nil arguments")
}
}
func createZeroSizedFileInTempDir() (File, error) {
filePrefix := "_path_test_"
f, e := TempFile(testFS, "", filePrefix) // dir is os.TempDir()
if e != nil {
// if there was an error no file was created.
// => no requirement to delete the file
return nil, e
}
return f, nil
}
func createNonZeroSizedFileInTempDir() (File, error) {
f, err := createZeroSizedFileInTempDir()
if err != nil {
// no file ??
}
byteString := []byte("byteString")
err = WriteFile(testFS, f.Name(), byteString, 0644)
if err != nil {
// delete the file
deleteFileInTempDir(f)
return nil, err
}
return f, nil
}
func deleteFileInTempDir(f File) {
err := testFS.Remove(f.Name())
if err != nil {
// now what?
}
}
func createEmptyTempDir() (string, error) {
dirPrefix := "_dir_prefix_"
d, e := TempDir(testFS, "", dirPrefix) // will be in os.TempDir()
if e != nil {
// no directory to delete - it was never created
return "", e
}
return d, nil
}
func createTempDirWithZeroLengthFiles() (string, error) {
d, dirErr := createEmptyTempDir()
if dirErr != nil {
//now what?
}
filePrefix := "_path_test_"
_, fileErr := TempFile(testFS, d, filePrefix) // dir is os.TempDir()
if fileErr != nil {
// if there was an error no file was created.
// but we need to remove the directory to clean-up
deleteTempDir(d)
return "", fileErr
}
// the dir now has one, zero length file in it
return d, nil
}
func createTempDirWithNonZeroLengthFiles() (string, error) {
d, dirErr := createEmptyTempDir()
if dirErr != nil {
//now what?
}
filePrefix := "_path_test_"
f, fileErr := TempFile(testFS, d, filePrefix) // dir is os.TempDir()
if fileErr != nil {
// if there was an error no file was created.
// but we need to remove the directory to clean-up
deleteTempDir(d)
return "", fileErr
}
byteString := []byte("byteString")
fileErr = WriteFile(testFS, f.Name(), byteString, 0644)
if fileErr != nil {
// delete the file
deleteFileInTempDir(f)
// also delete the directory
deleteTempDir(d)
return "", fileErr
}
// the dir now has one, zero length file in it
return d, nil
}
func TestExists(t *testing.T) {
zeroSizedFile, _ := createZeroSizedFileInTempDir()
defer deleteFileInTempDir(zeroSizedFile)
nonZeroSizedFile, _ := createNonZeroSizedFileInTempDir()
defer deleteFileInTempDir(nonZeroSizedFile)
emptyDirectory, _ := createEmptyTempDir()
defer deleteTempDir(emptyDirectory)
nonExistentFile := os.TempDir() + "/this-file-does-not-exist.txt"
nonExistentDir := os.TempDir() + "/this/direcotry/does/not/exist/"
type test struct {
input string
expectedResult bool
expectedErr error
}
data := []test{
{zeroSizedFile.Name(), true, nil},
{nonZeroSizedFile.Name(), true, nil},
{emptyDirectory, true, nil},
{nonExistentFile, false, nil},
{nonExistentDir, false, nil},
}
for i, d := range data {
exists, err := Exists(testFS, d.input)
if d.expectedResult != exists {
t.Errorf("Test %d failed. Expected result %t got %t", i, d.expectedResult, exists)
}
if d.expectedErr != err {
t.Errorf("Test %d failed. Expected %q got %q", i, d.expectedErr, err)
}
}
}
func TestSafeWriteToDisk(t *testing.T) {
emptyFile, _ := createZeroSizedFileInTempDir()
defer deleteFileInTempDir(emptyFile)
tmpDir, _ := createEmptyTempDir()
defer deleteTempDir(tmpDir)
randomString := "This is a random string!"
reader := strings.NewReader(randomString)
fileExists := fmt.Errorf("%v already exists", emptyFile.Name())
type test struct {
filename string
expectedErr error
}
now := time.Now().Unix()
nowStr := strconv.FormatInt(now, 10)
data := []test{
{emptyFile.Name(), fileExists},
{tmpDir + "/" + nowStr, nil},
}
for i, d := range data {
e := SafeWriteReader(testFS, d.filename, reader)
if d.expectedErr != nil {
if d.expectedErr.Error() != e.Error() {
t.Errorf("Test %d failed. Expected error %q but got %q", i, d.expectedErr.Error(), e.Error())
}
} else {
if d.expectedErr != e {
t.Errorf("Test %d failed. Expected %q but got %q", i, d.expectedErr, e)
}
contents, _ := ReadFile(testFS, d.filename)
if randomString != string(contents) {
t.Errorf("Test %d failed. Expected contents %q but got %q", i, randomString, string(contents))
}
}
reader.Seek(0, 0)
}
}
func TestWriteToDisk(t *testing.T) {
emptyFile, _ := createZeroSizedFileInTempDir()
defer deleteFileInTempDir(emptyFile)
tmpDir, _ := createEmptyTempDir()
defer deleteTempDir(tmpDir)
randomString := "This is a random string!"
reader := strings.NewReader(randomString)
type test struct {
filename string
expectedErr error
}
now := time.Now().Unix()
nowStr := strconv.FormatInt(now, 10)
data := []test{
{emptyFile.Name(), nil},
{tmpDir + "/" + nowStr, nil},
}
for i, d := range data {
e := WriteReader(testFS, d.filename, reader)
if d.expectedErr != e {
t.Errorf("Test %d failed. WriteToDisk Error Expected %q but got %q", i, d.expectedErr, e)
}
contents, e := ReadFile(testFS, d.filename)
if e != nil {
t.Errorf("Test %d failed. Could not read file %s. Reason: %s\n", i, d.filename, e)
}
if randomString != string(contents) {
t.Errorf("Test %d failed. Expected contents %q but got %q", i, randomString, string(contents))
}
reader.Seek(0, 0)
}
}
func TestGetTempDir(t *testing.T) {
dir := os.TempDir()
if FilePathSeparator != dir[len(dir)-1:] {
dir = dir + FilePathSeparator
}
testDir := "hugoTestFolder" + FilePathSeparator
tests := []struct {
input string
expected string
}{
{"", dir},
{testDir + " Foo bar ", dir + testDir + " Foo bar " + FilePathSeparator},
{testDir + "Foo.Bar/foo_Bar-Foo", dir + testDir + "Foo.Bar/foo_Bar-Foo" + FilePathSeparator},
{testDir + "fOO,bar:foo%bAR", dir + testDir + "fOObarfoo%bAR" + FilePathSeparator},
{testDir + "FOo/BaR.html", dir + testDir + "FOo/BaR.html" + FilePathSeparator},
{testDir + "трям/трям", dir + testDir + "трям/трям" + FilePathSeparator},
{testDir + "은행", dir + testDir + "은행" + FilePathSeparator},
{testDir + "Банковский кассир", dir + testDir + "Банковский кассир" + FilePathSeparator},
}
for _, test := range tests {
output := GetTempDir(new(MemMapFs), test.input)
if output != test.expected {
t.Errorf("Expected %#v, got %#v\n", test.expected, output)
}
}
}
// This function is very dangerous. Don't use it.
func deleteTempDir(d string) {
err := os.RemoveAll(d)
if err != nil {
// now what?
}
}
func TestFullBaseFsPath(t *testing.T) {
type dirSpec struct {
Dir1, Dir2, Dir3 string
}
dirSpecs := []dirSpec{
dirSpec{Dir1: "/", Dir2: "/", Dir3: "/"},
dirSpec{Dir1: "/", Dir2: "/path2", Dir3: "/"},
dirSpec{Dir1: "/path1/dir", Dir2: "/path2/dir/", Dir3: "/path3/dir"},
dirSpec{Dir1: "C:/path1", Dir2: "path2/dir", Dir3: "/path3/dir/"},
}
for _, ds := range dirSpecs {
memFs := NewMemMapFs()
level1Fs := NewBasePathFs(memFs, ds.Dir1)
level2Fs := NewBasePathFs(level1Fs, ds.Dir2)
level3Fs := NewBasePathFs(level2Fs, ds.Dir3)
type spec struct {
BaseFs Fs
FileName string
ExpectedPath string
}
specs := []spec{
spec{BaseFs: level3Fs, FileName: "f.txt", ExpectedPath: filepath.Join(ds.Dir1, ds.Dir2, ds.Dir3, "f.txt")},
spec{BaseFs: level3Fs, FileName: "", ExpectedPath: filepath.Join(ds.Dir1, ds.Dir2, ds.Dir3, "")},
spec{BaseFs: level2Fs, FileName: "f.txt", ExpectedPath: filepath.Join(ds.Dir1, ds.Dir2, "f.txt")},
spec{BaseFs: level2Fs, FileName: "", ExpectedPath: filepath.Join(ds.Dir1, ds.Dir2, "")},
spec{BaseFs: level1Fs, FileName: "f.txt", ExpectedPath: filepath.Join(ds.Dir1, "f.txt")},
spec{BaseFs: level1Fs, FileName: "", ExpectedPath: filepath.Join(ds.Dir1, "")},
}
for _, s := range specs {
if actualPath := FullBaseFsPath(s.BaseFs.(*BasePathFs), s.FileName); actualPath != s.ExpectedPath {
t.Errorf("Expected \n%s got \n%s", s.ExpectedPath, actualPath)
}
}
}
}