indexer

package
v0.0.0-...-df3bbb0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 7, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package indexer walks a repository, parses files, and stores the result in the NeuroFS SQLite index.

Index

Constants

View Source
const (
	// ProjectMetaKey is the metadata-table key under which indexer persists
	// the decoded project.Info. Exposed so other packages can read it.
	ProjectMetaKey = "project_info"
)

Variables

This section is empty.

Functions

func BuildChunks

func BuildChunks(filePath, relPath string, lang models.Lang, content string, indexedAt time.Time) []models.Chunk

BuildChunks splits a file into stable, hashable code chunks. Go uses the standard AST for top-level declarations; other languages currently fall back to a whole-file chunk until their AST chunkers land.

func BuildRelations

func BuildRelations(files []models.FileRecord) []models.FileRelation

BuildRelations walks all indexed file records, resolves their imports to target file records, and returns the set of file relationships.

func RequiresEmbeddingReindex

func RequiresEmbeddingReindex(db *storage.DB, provider, model string) (bool, error)

RequiresEmbeddingReindex reports whether db's persisted vectors belong to a different provider/model space than the currently configured client. Comparing vectors across spaces silently corrupts semantic ranking even when their dimensions happen to match.

func RequiresReindex

func RequiresReindex(db *storage.DB) (bool, error)

RequiresReindex reports whether db was built with different parser/chunker semantics. Callers that consume an existing non-empty index (notably retrieval.NewSession) use it to avoid serving stale chunks after a NeuroFS binary upgrade.

func RequiresSourceReindex

func RequiresSourceReindex(cfg *config.Config, db *storage.DB) (bool, error)

RequiresSourceReindex reports whether the indexable working tree differs from the file generations persisted in db. It applies the same traversal, language, size, line-count, and checksum rules as Run, but never mutates the database. Retrieval uses this check before loading a new session so a normal working-tree edit is indexed before its chunks are searched.

Types

type Options

type Options struct {
	// Verbose enables per-file logging via the provided function.
	// If nil, no per-file output is produced.
	Logf func(format string, args ...any)
}

Options configures an indexing run.

type Stats

type Stats struct {
	Discovered int // all files visited
	Skipped    int // unsupported or ignored
	Indexed    int // successfully written to the DB
	Cached     int // files skipped because they are already indexed and unmodified
	Updated    int // existing records refreshed
	Removed    int // stale records deleted
	Symbols    int // total symbols extracted
	Imports    int // total unique imports extracted
	Chunks     int // total chunks extracted
	Errors     int // files that produced errors (skipped)
	Duration   time.Duration
}

Stats summarises what happened during an indexing run.

func Run

func Run(cfg *config.Config, db *storage.DB, opts Options) (Stats, error)

Run indexes the repository rooted at cfg.RepoRoot and stores results in the database at cfg.DBPath. It returns indexing statistics.

type Watcher

type Watcher struct {
	// contains filtered or unexported fields
}

Watcher monitors a repository filesystem for changes and updates the index incrementally.

func NewWatcher

func NewWatcher(cfg *config.Config, db *storage.DB, logf func(format string, args ...any)) (*Watcher, error)

NewWatcher returns a new filesystem watcher for the repository.

func (*Watcher) Close

func (w *Watcher) Close() error

Close shuts down the fsnotify watcher.

func (*Watcher) Start

func (w *Watcher) Start(ctx context.Context) error

Start initiates directory walking, registers all subdirectories to fsnotify, and starts the background listening loop.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL