cache

package standard library
go1.24.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 11, 2025 License: BSD-3-Clause Imports: 30 Imported by: 0

Documentation

Overview

Package cache implements a build artifact cache.

Index

Constants

View Source
const HashSize = 32

HashSize is the number of bytes in a hash.

Variables

View Source
var DebugTest = false

DebugTest is set when GODEBUG=gocachetest=1 is in the environment.

Functions

func DefaultDir

func DefaultDir() (string, bool)

DefaultDir returns the effective GOCACHE setting. It returns "off" if the cache is disabled, and reports whether the effective value differs from GOCACHE.

func FileHash

func FileHash(file string) ([HashSize]byte, error)

FileHash returns the hash of the named file. It caches repeated lookups for a given file, and the cache entry for a file can be initialized using SetFileHash. The hash used by FileHash is not the same as the hash used by NewHash.

func PutBytes added in go1.21.0

func PutBytes(c Cache, id ActionID, data []byte) error

PutBytes stores the given bytes in the cache as the output for the action ID.

func SetFileHash

func SetFileHash(file string, sum [HashSize]byte)

SetFileHash sets the hash returned by FileHash for file.

Types

type ActionID

type ActionID [HashSize]byte

An ActionID is a cache action key, the hash of a complete description of a repeatable computation (command line, environment variables, input file contents, executable contents).

func Subkey

func Subkey(parent ActionID, desc string) ActionID

Subkey returns an action ID corresponding to mixing a parent action ID with a string description of the subkey.

type Cache

type Cache interface {
	// Get returns the cache entry for the provided ActionID.
	// On miss, the error type should be of type *entryNotFoundError.
	//
	// After a successful call to Get, OutputFile(Entry.OutputID) must
	// exist on disk until Close is called (at the end of the process).
	Get(ActionID) (Entry, error)

	// Put adds an item to the cache.
	//
	// The seeker is only used to seek to the beginning. After a call to Put,
	// the seek position is not guaranteed to be in any particular state.
	//
	// As a special case, if the ReadSeeker is of type noVerifyReadSeeker,
	// the verification from GODEBUG=goverifycache=1 is skipped.
	//
	// After a successful call to Put, OutputFile(OutputID) must
	// exist on disk until Close is called (at the end of the process).
	Put(ActionID, io.ReadSeeker) (_ OutputID, size int64, _ error)

	// Close is called at the end of the go process. Implementations can do
	// cache cleanup work at this phase, or wait for and report any errors from
	// background cleanup work started earlier. Any cache trimming in one
	// process should not cause the invariants of this interface to be
	// violated in another process. Namely, a cache trim from one process should
	// not delete an ObjectID from disk that was recently Get or Put from
	// another process. As a rule of thumb, don't trim things used in the last
	// day.
	Close() error

	// OutputFile returns the path on disk where OutputID is stored.
	//
	// It's only called after a successful get or put call so it doesn't need
	// to return an error; it's assumed that if the previous get or put succeeded,
	// it's already on disk.
	OutputFile(OutputID) string

	// FuzzDir returns where fuzz files are stored.
	FuzzDir() string
}

Cache is the interface as used by the cmd/go.

func Default

func Default() Cache

Default returns the default cache to use. It never returns nil.

type DiskCache added in go1.21.0

type DiskCache struct {
	// contains filtered or unexported fields
}

A Cache is a package cache, backed by a file system directory tree.

func Open

func Open(dir string) (*DiskCache, error)

Open opens and returns the cache in the given directory.

It is safe for multiple processes on a single machine to use the same cache directory in a local file system simultaneously. They will coordinate using operating system file locks and may duplicate effort but will not corrupt the cache.

However, it is NOT safe for multiple processes on different machines to share a cache directory (for example, if the directory were stored in a network file system). File locking is notoriously unreliable in network file systems and may not suffice to protect the cache.

func (*DiskCache) Close added in go1.21.0

func (c *DiskCache) Close() error

func (*DiskCache) FuzzDir added in go1.21.0

func (c *DiskCache) FuzzDir() string

FuzzDir returns a subdirectory within the cache for storing fuzzing data. The subdirectory may not exist.

This directory is managed by the internal/fuzz package. Files in this directory aren't removed by the 'go clean -cache' command or by Trim. They may be removed with 'go clean -fuzzcache'.

TODO(#48526): make Trim remove unused files from this directory.

func (*DiskCache) Get added in go1.21.0

func (c *DiskCache) Get(id ActionID) (Entry, error)

Get looks up the action ID in the cache, returning the corresponding output ID and file size, if any. Note that finding an output ID does not guarantee that the saved file for that output ID is still available.

func (*DiskCache) OutputFile added in go1.21.0

func (c *DiskCache) OutputFile(out OutputID) string

OutputFile returns the name of the cache file storing output with the given OutputID.

func (*DiskCache) Put added in go1.21.0

func (c *DiskCache) Put(id ActionID, file io.ReadSeeker) (OutputID, int64, error)

Put stores the given output in the cache as the output for the action ID. It may read file twice. The content of file must not change between the two passes.

func (*DiskCache) PutExecutable added in go1.24.0

func (c *DiskCache) PutExecutable(id ActionID, name string, file io.ReadSeeker) (OutputID, int64, error)

PutExecutable is used to store the output as the output for the action ID into a file with the given base name, with the executable mode bit set. It may read file twice. The content of file must not change between the two passes.

func (*DiskCache) Trim added in go1.21.0

func (c *DiskCache) Trim() error

Trim removes old cache entries that are likely not to be reused.

type Entry

type Entry struct {
	OutputID OutputID
	Size     int64
	Time     time.Time // when added to cache
}

func GetBytes added in go1.21.0

func GetBytes(c Cache, id ActionID) ([]byte, Entry, error)

GetBytes looks up the action ID in the cache and returns the corresponding output bytes. GetBytes should only be used for data that can be expected to fit in memory.

func GetFile added in go1.21.0

func GetFile(c Cache, id ActionID) (file string, entry Entry, err error)

GetFile looks up the action ID in the cache and returns the name of the corresponding data file.

func GetMmap added in go1.21.0

func GetMmap(c Cache, id ActionID) ([]byte, Entry, bool, error)

GetMmap looks up the action ID in the cache and returns the corresponding output bytes. GetMmap should only be used for data that can be expected to fit in memory.

type Hash

type Hash struct {
	// contains filtered or unexported fields
}

A Hash provides access to the canonical hash function used to index the cache. The current implementation uses salted SHA256, but clients must not assume this.

func NewHash

func NewHash(name string) *Hash

NewHash returns a new Hash. The caller is expected to Write data to it and then call Sum.

func (*Hash) Sum

func (h *Hash) Sum() [HashSize]byte

Sum returns the hash of the data written previously.

func (*Hash) Write

func (h *Hash) Write(b []byte) (int, error)

Write writes data to the running hash.

type OutputID

type OutputID [HashSize]byte

An OutputID is a cache output key, the hash of an output of a computation.

func PutNoVerify added in go1.21.0

func PutNoVerify(c Cache, id ActionID, file io.ReadSeeker) (OutputID, int64, error)

PutNoVerify is like Put but disables the verify check when GODEBUG=goverifycache=1 is set. It is meant for data that is OK to cache but that we expect to vary slightly from run to run, like test output containing times and the like.

type ProgCache added in go1.21.0

type ProgCache struct {
	// contains filtered or unexported fields
}

ProgCache implements Cache via JSON messages over stdin/stdout to a child helper process which can then implement whatever caching policy/mechanism it wants.

See https://github.com/golang/go/issues/59719

func (*ProgCache) Close added in go1.21.0

func (c *ProgCache) Close() error

func (*ProgCache) FuzzDir added in go1.21.0

func (c *ProgCache) FuzzDir() string

func (*ProgCache) Get added in go1.21.0

func (c *ProgCache) Get(a ActionID) (Entry, error)

func (*ProgCache) OutputFile added in go1.21.0

func (c *ProgCache) OutputFile(o OutputID) string

func (*ProgCache) Put added in go1.21.0

func (c *ProgCache) Put(a ActionID, file io.ReadSeeker) (_ OutputID, size int64, _ error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL