database

package
v0.65.60 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 11, 2026 License: AGPL-3.0 Imports: 54 Imported by: 0

Documentation

Overview

Package database provides filter utilities for normalizing tag values.

The nostr library optimizes e/p tag values by storing them in binary format (32 bytes + null terminator) rather than hex strings (64 chars). However, filter tags from client queries come as hex strings and don't go through the same binary encoding during unmarshalling.

This file provides utilities to normalize filter tags to match the binary encoding used in stored events, ensuring consistent index lookups and tag comparisons.

Package database provides shared import utilities for events

Index

Constants

View Source
const (
	CompactFormatVersion = 1

	// Tag element type flags
	TagElementRaw          = 0x00 // Raw bytes (varint length + data)
	TagElementPubkeySerial = 0x01 // Pubkey serial reference (5 bytes)
	TagElementEventSerial  = 0x02 // Event ID serial reference (5 bytes)
	TagElementEventIdFull  = 0x03 // Full event ID (32 bytes) - for unknown refs

	// Sanity limits to prevent OOM from corrupt data
	MaxTagsPerEvent     = 10000    // Maximum number of tags in an event
	MaxTagElements      = 100      // Maximum elements in a single tag
	MaxContentLength    = 10 << 20 // 10MB max content
	MaxTagElementLength = 1 << 20  // 1MB max for a single tag element
)
View Source
const (
	// BinaryEncodedLen is the length of a binary-encoded 32-byte hash with null terminator
	BinaryEncodedLen = 33
	// HexEncodedLen is the length of a hex-encoded 32-byte hash
	HexEncodedLen = 64
	// HashLen is the raw length of a hash (pubkey/event ID)
	HashLen = 32
)

Tag binary encoding constants (matching the nostr library)

View Source
const DefaultMaxConcurrentQueries = 3

DefaultMaxConcurrentQueries limits concurrent database queries to prevent memory exhaustion. Each query creates Badger iterators that consume significant memory. With many concurrent connections, unlimited queries can exhaust available memory in seconds. Set very low (3) because each query can internally create many iterators.

View Source
const KindNRCConnection = uint16(30078)

NRC connection event kind - using application-specific data range Kind 30078 is commonly used for app-specific data

View Source
const WriteOpType = 1

WriteOpType is the operation type constant for write operations

Variables

View Source
var (
	ErrTooManyTags        = errors.New("corrupt data: too many tags")
	ErrTooManyTagElems    = errors.New("corrupt data: too many tag elements")
	ErrContentTooLarge    = errors.New("corrupt data: content too large")
	ErrTagElementTooLong  = errors.New("corrupt data: tag element too long")
	ErrUnknownTagElemType = errors.New("corrupt data: unknown tag element type")
)
View Source
var (
	ErrPubkeyNotFound = errors.New("pubkey not found in database")
	ErrEventNotFound  = errors.New("event not found in database")
)

Graph traversal errors

View Source
var (
	// ErrOlderThanExisting is returned when a candidate event is older than an existing replaceable/addressable event.
	ErrOlderThanExisting = errors.New("older than existing event")
	// ErrMissingDTag is returned when a parameterized replaceable event lacks the required 'd' tag.
	ErrMissingDTag = errors.New("event is missing a d tag identifier")
)
View Source
var ErrAliasTaken = &aliasTakenError{}

ErrAliasTaken is returned when attempting to claim an alias already taken by another pubkey.

Functions

func BinaryToHex

func BinaryToHex(binVal []byte) []byte

BinaryToHex converts a 33-byte binary value to 64-character hex string Returns nil if the input is not in binary format

func BuildAddressableEventKey

func BuildAddressableEventKey(pubkey []byte, eventKind uint16, dTagValue []byte) ([]byte, error)

BuildAddressableEventKey builds the key for an AddressableEvent index entry. This is used by both save-event.go (for writing) and deletion (for cleanup).

func CanUsePTagGraph

func CanUsePTagGraph(f *filter.F) bool

CanUsePTagGraph determines if a filter can benefit from p-tag graph optimization.

Requirements: - Filter must have #p tags - Filter should NOT have authors (different index is better for that case) - Optimization works best with kinds filter but is optional

func CheckExpiration

func CheckExpiration(ev *event.E) (expired bool)

func CreateIdHashFromData

func CreateIdHashFromData(data []byte) (i *types2.IdHash, err error)

CreateIdHashFromData creates an IdHash from data that could be hex or binary

func CreatePubHashFromData

func CreatePubHashFromData(data []byte) (p *types2.PubHash, err error)

CreatePubHashFromData creates a PubHash from data that could be hex or binary

func GetCumulativeCompactSavings

func GetCumulativeCompactSavings() int64

GetCumulativeCompactSavings returns total bytes saved across all compact saves.

func GetIndexesForEvent

func GetIndexesForEvent(ev *event.E, serial uint64) (
	idxs [][]byte, err error,
)

GetIndexesForEvent creates all the indexes for an event.E instance as defined in keys.go. It returns a slice of byte slices that can be used to store the event in the database.

func HasDriver

func HasDriver(name string) bool

HasDriver returns true if the named driver is registered.

func HexToBinary

func HexToBinary(hexVal []byte) []byte

HexToBinary converts a 64-character hex string to 33-byte binary format Returns nil if the input is not a valid hex string

func IsAddressableEventQuery

func IsAddressableEventQuery(f *filter.F) bool

IsAddressableEventQuery checks if a filter matches the NIP-33 addressable event query pattern: exactly one kind (30000-39999), one author, and one d-tag. This pattern uniquely identifies a single parameterized replaceable event.

func IsBinaryEncoded

func IsBinaryEncoded(val []byte) bool

IsBinaryEncoded checks if a value field is stored in optimized binary format

func IsBinaryOptimizedTag

func IsBinaryOptimizedTag(key byte) bool

IsBinaryOptimizedTag returns true if the given tag key uses binary encoding

func IsHexString

func IsHexString(data []byte) (isHex bool)

IsHexString checks if the byte slice contains only hex characters

func IsValidHexValue

func IsValidHexValue(b []byte) bool

IsValidHexValue checks if a byte slice is a valid 64-character hex string

func ListDrivers

func ListDrivers() []string

ListDrivers returns a sorted list of registered driver names.

func MarshalCompactEvent

func MarshalCompactEvent(ev *event.E, resolver SerialResolver) (data []byte, err error)

MarshalCompactEvent encodes an event using compact serial references. The resolver is used to look up/create serial mappings for pubkeys and event IDs.

func NewLogger

func NewLogger(logLevel int, label string) (l *logger)

NewLogger creates a new badger logger.

func NormalizeFilter

func NormalizeFilter(f *filter.F) *filter.F

NormalizeFilter normalizes a filter's tags for consistent database queries. This should be called before using a filter for database lookups. The original filter is not modified; a copy with normalized tags is returned.

func NormalizeFilterTag

func NormalizeFilterTag(t *tag.T) *tag.T

NormalizeFilterTag creates a new tag with binary-encoded values for e/p tags. The original tag is not modified.

func NormalizeFilterTags

func NormalizeFilterTags(tags *tag.S) *tag.S

NormalizeFilterTags normalizes all tags in a tag.S, converting e/p hex values to binary. Returns a new tag.S with normalized tags.

func NormalizeTagToHex

func NormalizeTagToHex(val []byte) []byte

NormalizeTagToHex returns the hex representation of a tag value. For binary-encoded values, converts to hex. For hex values, returns as-is.

func NormalizeTagValue

func NormalizeTagValue(key byte, val []byte) []byte

NormalizeTagValue normalizes a tag value for the given key. For e/p tags, hex values are converted to binary format. Other tags are returned unchanged.

func NormalizeTagValueForHash

func NormalizeTagValueForHash(key byte, valueBytes []byte) []byte

NormalizeTagValueForHash normalizes a tag value for consistent hashing. For 'e' and 'p' tags, the nostr library stores values in binary format (32 bytes), but filters from clients come with hex strings (64 chars). This function ensures that filter values are converted to binary to match the stored index format.

This function delegates to NormalizeTagValue from filter_utils.go for consistency.

func RegisterDriver

func RegisterDriver(name, description string, factory DriverFactory)

RegisterDriver registers a database driver with the given name and factory. This is typically called from init() in the driver package.

func RegisterGRPCFactory

func RegisterGRPCFactory(factory func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error))

RegisterGRPCFactory registers the gRPC database factory This is called from the grpc package's init() function

func RegisterNeo4jFactory

func RegisterNeo4jFactory(factory func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error))

RegisterNeo4jFactory registers the neo4j database factory This is called from the neo4j package's init() function

func RegisterWasmDBFactory

func RegisterWasmDBFactory(factory func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error))

RegisterWasmDBFactory registers the wasmdb database factory This is called from the wasmdb package's init() function

func ResetCompactSavingsCounter

func ResetCompactSavingsCounter()

ResetCompactSavingsCounter resets the cumulative savings counter.

func TagValuesMatch

func TagValuesMatch(key byte, eventVal, filterVal []byte) bool

TagValuesMatch compares two tag values, handling both binary and hex encodings. This is useful for post-query tag matching where event values may be binary and filter values may be hex (or vice versa).

func TagValuesMatchUsingTagMethods

func TagValuesMatchUsingTagMethods(eventTag *tag.T, filterVal []byte) bool

TagValuesMatchUsingTagMethods compares an event tag's value with a filter value using the tag.T methods. This leverages the nostr library's ValueHex() method for proper binary/hex conversion.

func TokenHashes

func TokenHashes(content []byte) [][]byte

TokenHashes extracts unique word hashes (8-byte truncated sha256) from content. This is a convenience wrapper around TokenWords that returns only the hashes.

func TrackCompactSaving

func TrackCompactSaving(legacySize, compactSize int)

TrackCompactSaving records bytes saved for a single event. Call this during event save to track cumulative savings.

func UnmarshalCompactEvent

func UnmarshalCompactEvent(data []byte, eventId []byte, resolver SerialResolver) (ev *event.E, err error)

UnmarshalCompactEvent decodes a compact event back to a full event.E. The resolver is used to look up pubkeys and event IDs from serials. The eventId parameter is the full 32-byte event ID (from SerialEventId table).

Types

type AliasClaim

type AliasClaim struct {
	Alias     string    `json:"alias"`
	PubkeyHex string    `json:"pubkey"`
	ClaimedAt time.Time `json:"claimed_at"`
}

AliasClaim represents a claimed email alias.

type AllowedEvent

type AllowedEvent struct {
	ID     string    `json:"id"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

AllowedEvent represents an allowed event entry

type AllowedKind

type AllowedKind struct {
	Kind  int       `json:"kind"`
	Added time.Time `json:"added"`
}

AllowedKind represents an allowed event kind

type AllowedPubkey

type AllowedPubkey struct {
	Pubkey string    `json:"pubkey"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

AllowedPubkey represents an allowed public key entry

type BannedEvent

type BannedEvent struct {
	ID     string    `json:"id"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

BannedEvent represents a banned event entry

type BannedPubkey

type BannedPubkey struct {
	Pubkey string    `json:"pubkey"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

BannedPubkey represents a banned public key entry

type BlacklistedPubkey

type BlacklistedPubkey struct {
	Pubkey string    `json:"pubkey"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

BlacklistedPubkey represents a blacklisted publisher

type BlobDescriptor

type BlobDescriptor struct {
	URL      string     `json:"url"`
	SHA256   string     `json:"sha256"`
	Size     int64      `json:"size"`
	Type     string     `json:"type"`
	Uploaded int64      `json:"uploaded"`
	NIP94    [][]string `json:"nip94,omitempty"`
}

BlobDescriptor represents a blob descriptor as defined in BUD-02

type BlobMetadata

type BlobMetadata struct {
	Pubkey    []byte `json:"pubkey"`
	MimeType  string `json:"mime_type"`
	Uploaded  int64  `json:"uploaded"`
	Size      int64  `json:"size"`
	Extension string `json:"extension"` // File extension (e.g., ".png", ".pdf")
}

BlobMetadata stores metadata about a blob in the database

type BlockedIP

type BlockedIP struct {
	IP     string    `json:"ip"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

BlockedIP represents a blocked IP address entry

type CompactStorageStats

type CompactStorageStats struct {
	// Event counts
	CompactEvents int64 // Number of events in compact format (cmp prefix)
	LegacyEvents  int64 // Number of events in legacy format (evt/sev prefixes)
	TotalEvents   int64 // Total events

	// Storage sizes
	CompactBytes int64 // Total bytes used by compact format
	LegacyBytes  int64 // Total bytes used by legacy format (would be used without compact)

	// Savings
	BytesSaved     int64   // Bytes saved by using compact format
	PercentSaved   float64 // Percentage of space saved
	AverageCompact float64 // Average compact event size
	AverageLegacy  float64 // Average legacy event size (estimated)

	// Serial mappings
	SerialEventIdEntries int64 // Number of sei (serial -> event ID) mappings
	SerialEventIdBytes   int64 // Bytes used by sei mappings
}

CompactStorageStats holds statistics about compact vs legacy storage.

type CuratingACL

type CuratingACL struct {
	*D
}

CuratingACL database operations

func NewCuratingACL

func NewCuratingACL(db *D) *CuratingACL

NewCuratingACL creates a new CuratingACL instance

func (*CuratingACL) BlockIP

func (c *CuratingACL) BlockIP(ip string, duration time.Duration, reason string) error

BlockIP blocks an IP for a specified duration

func (*CuratingACL) CleanupExpiredIPBlocks

func (c *CuratingACL) CleanupExpiredIPBlocks() error

CleanupExpiredIPBlocks removes expired IP blocks

func (*CuratingACL) CleanupOldEventCounts

func (c *CuratingACL) CleanupOldEventCounts(beforeDate string) error

CleanupOldEventCounts removes event counts older than the specified date

func (*CuratingACL) CleanupOldIPEventCounts

func (c *CuratingACL) CleanupOldIPEventCounts(beforeDate string) error

CleanupOldIPEventCounts removes IP event counts older than the specified date

func (*CuratingACL) DeleteEventsForPubkey

func (c *CuratingACL) DeleteEventsForPubkey(pubkeyHex string) (int, error)

DeleteEventsForPubkey deletes all events for a given pubkey Returns the number of events deleted

func (*CuratingACL) GetConfig

func (c *CuratingACL) GetConfig() (CuratingConfig, error)

GetConfig returns the curating configuration

func (*CuratingACL) GetEventCount

func (c *CuratingACL) GetEventCount(pubkey, date string) (int, error)

GetEventCount returns the event count for a pubkey on a specific date

func (*CuratingACL) GetEventsForPubkey

func (c *CuratingACL) GetEventsForPubkey(pubkeyHex string, limit, offset int) ([]EventSummary, int, error)

GetEventsForPubkey fetches events for a pubkey, returning simplified event data limit specifies max events to return, offset is for pagination

func (*CuratingACL) GetIPEventCount

func (c *CuratingACL) GetIPEventCount(ip, date string) (int, error)

GetIPEventCount returns the total event count for an IP on a specific date

func (*CuratingACL) GetIPOffense

func (c *CuratingACL) GetIPOffense(ip string) (*IPOffense, error)

GetIPOffense returns the offense record for an IP

func (*CuratingACL) IncrementEventCount

func (c *CuratingACL) IncrementEventCount(pubkey, date string) (int, error)

IncrementEventCount increments and returns the new event count for a pubkey

func (*CuratingACL) IncrementIPEventCount

func (c *CuratingACL) IncrementIPEventCount(ip, date string) (int, error)

IncrementIPEventCount increments and returns the new event count for an IP

func (*CuratingACL) IsConfigured

func (c *CuratingACL) IsConfigured() (bool, error)

IsConfigured returns true if a configuration event has been set

func (*CuratingACL) IsEventSpam

func (c *CuratingACL) IsEventSpam(eventID string) (bool, error)

IsEventSpam checks if an event is marked as spam

func (*CuratingACL) IsIPBlocked

func (c *CuratingACL) IsIPBlocked(ip string) (bool, time.Time, error)

IsIPBlocked checks if an IP is blocked and returns expiration time

func (*CuratingACL) IsKindAllowed

func (c *CuratingACL) IsKindAllowed(kind int, config *CuratingConfig) bool

IsKindAllowed checks if an event kind is allowed based on config

func (*CuratingACL) IsPubkeyBlacklisted

func (c *CuratingACL) IsPubkeyBlacklisted(pubkey string) (bool, error)

IsPubkeyBlacklisted checks if a pubkey is blacklisted

func (*CuratingACL) IsPubkeyTrusted

func (c *CuratingACL) IsPubkeyTrusted(pubkey string) (bool, error)

IsPubkeyTrusted checks if a pubkey is trusted

func (*CuratingACL) ListBlacklistedPubkeys

func (c *CuratingACL) ListBlacklistedPubkeys() ([]BlacklistedPubkey, error)

ListBlacklistedPubkeys returns all blacklisted pubkeys

func (*CuratingACL) ListBlockedIPs

func (c *CuratingACL) ListBlockedIPs() ([]CuratingBlockedIP, error)

ListBlockedIPs returns all blocked IPs (including expired ones)

func (*CuratingACL) ListSpamEvents

func (c *CuratingACL) ListSpamEvents() ([]SpamEvent, error)

ListSpamEvents returns all spam events

func (*CuratingACL) ListTrustedPubkeys

func (c *CuratingACL) ListTrustedPubkeys() ([]TrustedPubkey, error)

ListTrustedPubkeys returns all trusted pubkeys

func (*CuratingACL) ListUnclassifiedUsers

func (c *CuratingACL) ListUnclassifiedUsers(limit int) ([]UnclassifiedUser, error)

ListUnclassifiedUsers returns users who are neither trusted nor blacklisted sorted by event count descending

func (*CuratingACL) MarkEventAsSpam

func (c *CuratingACL) MarkEventAsSpam(eventID, pubkey, reason string) error

MarkEventAsSpam marks an event as spam

func (*CuratingACL) RecordIPOffense

func (c *CuratingACL) RecordIPOffense(ip, pubkey string) (int, error)

RecordIPOffense records a rate limit violation from an IP for a pubkey Returns the new offense count

func (*CuratingACL) RemoveBlacklistedPubkey

func (c *CuratingACL) RemoveBlacklistedPubkey(pubkey string) error

RemoveBlacklistedPubkey removes a blacklisted pubkey from the database

func (*CuratingACL) RemoveTrustedPubkey

func (c *CuratingACL) RemoveTrustedPubkey(pubkey string) error

RemoveTrustedPubkey removes a trusted pubkey from the database

func (*CuratingACL) SaveBlacklistedPubkey

func (c *CuratingACL) SaveBlacklistedPubkey(pubkey string, reason string) error

SaveBlacklistedPubkey saves a blacklisted pubkey to the database

func (*CuratingACL) SaveConfig

func (c *CuratingACL) SaveConfig(config CuratingConfig) error

SaveConfig saves the curating configuration

func (*CuratingACL) SaveTrustedPubkey

func (c *CuratingACL) SaveTrustedPubkey(pubkey string, note string) error

SaveTrustedPubkey saves a trusted pubkey to the database

func (*CuratingACL) ScanAllPubkeys

func (c *CuratingACL) ScanAllPubkeys() (*ScanResult, error)

ScanAllPubkeys scans the database to find all unique pubkeys and count their events. This populates the event count data needed for the unclassified users list. It uses the SerialPubkey index to find all pubkeys, then counts events for each.

func (*CuratingACL) UnblockIP

func (c *CuratingACL) UnblockIP(ip string) error

UnblockIP removes an IP from the blocked list

func (*CuratingACL) UnmarkEventAsSpam

func (c *CuratingACL) UnmarkEventAsSpam(eventID string) error

UnmarkEventAsSpam removes the spam flag from an event

type CuratingBlockedIP

type CuratingBlockedIP struct {
	IP        string    `json:"ip"`
	Reason    string    `json:"reason"`
	ExpiresAt time.Time `json:"expires_at"`
	Added     time.Time `json:"added"`
}

CuratingBlockedIP represents a temporarily blocked IP with expiration

type CuratingConfig

type CuratingConfig struct {
	DailyLimit     int      `json:"daily_limit"`      // Max events per day for unclassified users
	IPDailyLimit   int      `json:"ip_daily_limit"`   // Max events per day from a single IP (flood protection)
	FirstBanHours  int      `json:"first_ban_hours"`  // IP ban duration for first offense
	SecondBanHours int      `json:"second_ban_hours"` // IP ban duration for second+ offense
	AllowedKinds   []int    `json:"allowed_kinds"`    // Explicit kind numbers
	AllowedRanges  []string `json:"allowed_ranges"`   // Kind ranges like "1000-1999"
	KindCategories []string `json:"kind_categories"`  // Category IDs like "social", "dm"
	ConfigEventID  string   `json:"config_event_id"`  // ID of the config event
	ConfigPubkey   string   `json:"config_pubkey"`    // Pubkey that published config
	ConfiguredAt   int64    `json:"configured_at"`    // Timestamp of config event
}

CuratingConfig represents the configuration for curating ACL mode This is parsed from a kind 30078 event with d-tag "curating-config"

type D

type D struct {
	Logger *logger
	*badger.DB
	// contains filtered or unexported fields
}

D implements the Database interface using Badger as the storage backend

func New

func New(
	ctx context.Context, cancel context.CancelFunc, dataDir, logLevel string,
) (
	d *D, err error,
)

New creates a new Badger database instance with default configuration. This is provided for backward compatibility with existing callers. For full configuration control, use NewWithConfig instead.

func NewWithConfig

func NewWithConfig(
	ctx context.Context, cancel context.CancelFunc, cfg *DatabaseConfig,
) (
	d *D, err error,
)

NewWithConfig creates a new Badger database instance with full configuration. This is the preferred method when you have access to DatabaseConfig.

func (*D) AcquireQuerySlot

func (d *D) AcquireQuerySlot(ctx context.Context) bool

AcquireQuerySlot acquires a slot from the query semaphore to limit concurrent queries. This blocks until a slot is available or the context is cancelled. Returns true if slot was acquired, false if context cancelled.

func (*D) AddInboundRefsToResult

func (d *D) AddInboundRefsToResult(result *GraphResult, depth int, kinds []uint16) error

AddInboundRefsToResult collects inbound references (events that reference discovered items) for events at a specific depth in the result.

For example, if you have a follows graph result and want to find all kind-7 reactions to posts by users at depth 1, this collects those reactions and adds them to result.InboundRefs.

Parameters: - result: The graph result to augment with ref data - depth: The depth at which to collect refs (0 = all depths) - kinds: Event kinds to collect (e.g., [7] for reactions, [6] for reposts)

func (*D) AddNIP43Member

func (d *D) AddNIP43Member(pubkey []byte, inviteCode string) error

AddNIP43Member adds a member to the NIP-43 membership list

func (*D) AddOutboundRefsToResult

func (d *D) AddOutboundRefsToResult(result *GraphResult, depth int, kinds []uint16) error

AddOutboundRefsToResult collects outbound references (events referenced by discovered items).

For example, find all events that posts by users at depth 1 reference (quoted posts, replied-to posts).

func (*D) ArchiveRevokedKey

func (d *D) ArchiveRevokedKey(peer *WireGuardPeer) error

ArchiveRevokedKey stores a revoked keypair for audit purposes.

func (*D) BackfillETagGraph

func (d *D) BackfillETagGraph()

BackfillETagGraph populates e-tag graph indexes (eeg/gee) for all existing events. This enables graph traversal queries for thread/reply discovery.

The migration: 1. Iterates all events in compact storage (cmp prefix) 2. Extracts e-tags from each event 3. For e-tags referencing events we have, creates bidirectional edges:

  • eeg|source|target|kind|direction(out) - forward edge
  • gee|target|kind|direction(in)|source - reverse edge

This is idempotent: running multiple times won't create duplicate edges (BadgerDB overwrites existing keys).

func (*D) BackfillMissingSerialEventIdMappings

func (d *D) BackfillMissingSerialEventIdMappings()

BackfillMissingSerialEventIdMappings finds legacy events that were incorrectly skipped during the v6 compact format migration and creates their SerialEventId mappings. This fixes events whose ID happens to start with byte 0x01, which was mistakenly interpreted as CompactFormatVersion during the original migration.

The v6 migration had this check:

if len(eventData) > 0 && eventData[0] == CompactFormatVersion { continue }

This caused legacy events with IDs starting with 0x01 to be skipped, leaving them without sei mappings and causing "Key not found" errors when fetching.

func (*D) BackfillPubkeyPubkeyGraph

func (d *D) BackfillPubkeyPubkeyGraph()

BackfillPubkeyPubkeyGraph populates pubkey-to-pubkey (noun-noun) graph indexes (ppg/gpp) for all existing events that contain p-tags. This materializes direct pubkey→pubkey edges, collapsing the two-hop pubkey→event→pubkey traversal into a single-hop lookup.

For each event with p-tags, creates bidirectional edges:

  • ppg|author_serial|target_serial|kind|direction(out)|event_serial - forward edge
  • gpp|target_serial|kind|direction(in)|author_serial|event_serial - reverse edge

This is idempotent: running multiple times won't create duplicate edges.

func (*D) CacheEvents

func (d *D) CacheEvents(f *filter.F, events event.S)

CacheEvents stores events for a filter (without subscription ID)

func (*D) CacheMarshaledJSON

func (d *D) CacheMarshaledJSON(f *filter.F, marshaledJSON [][]byte)

CacheMarshaledJSON stores marshaled JSON event envelopes for a filter

func (*D) CheckForDeleted

func (d *D) CheckForDeleted(ev *event.E, admins [][]byte) (err error)

CheckForDeleted checks if the event is deleted, and returns an error with prefix "blocked:" if it is. This function also allows designating admin pubkeys that also may delete the event, normally only the author is allowed to delete an event.

func (*D) ClaimAlias

func (d *D) ClaimAlias(alias, pubkeyHex string) error

ClaimAlias atomically claims an alias for a pubkey.

func (*D) CleanupEphemeralEvents

func (d *D) CleanupEphemeralEvents()

func (*D) CleanupKind3WithoutPTags

func (d *D) CleanupKind3WithoutPTags(ctx context.Context) error

CleanupKind3WithoutPTags scans for kind 3 follow list events that have no p tags and deletes them. This cleanup is needed because the directory spider may have saved malformed events that lost their tags during serialization.

func (*D) CleanupLegacyEventStorage

func (d *D) CleanupLegacyEventStorage()

CleanupLegacyEventStorage removes legacy evt and sev storage entries after compact format migration. This reclaims disk space by removing the old storage format entries once all events have been successfully migrated to cmp format.

The cleanup: 1. Iterates through all cmp entries (compact format) 2. For each serial found in cmp, deletes corresponding evt and sev entries 3. Reports total bytes reclaimed

func (*D) Close

func (d *D) Close() (err error)

Close releases resources and closes the database.

func (*D) CollectRefsForPubkeys

func (d *D) CollectRefsForPubkeys(
	pubkeySerials []*types.Uint40,
	refKinds []uint16,
	eventKinds []uint16,
) (*GraphResult, error)

CollectRefsForPubkeys collects inbound references to events by specific pubkeys. This is useful for "find all reactions to posts by these users" queries.

Parameters: - pubkeySerials: The pubkeys whose events should be checked for refs - refKinds: Event kinds to collect (e.g., [7] for reactions) - eventKinds: Event kinds to check for refs (e.g., [1] for notes)

func (*D) CompactStorageStats

func (d *D) CompactStorageStats() (stats CompactStorageStats, err error)

CompactStorageStats calculates storage statistics for compact event storage. This scans the database to provide accurate metrics on space savings.

func (*D) ConvertSmallEventsToInline

func (d *D) ConvertSmallEventsToInline()

ConvertSmallEventsToInline migrates small events (<=384 bytes) to inline storage. This is a Reiser4-inspired optimization that stores small event data in the key itself, avoiding a second database lookup and improving query performance. Also handles replaceable and addressable events with specialized storage.

func (*D) ConvertToCompactEventFormat

func (d *D) ConvertToCompactEventFormat()

ConvertToCompactEventFormat migrates all existing events to the new compact format. This format uses 5-byte serial references instead of 32-byte IDs/pubkeys, dramatically reducing storage requirements (up to 80% savings on ID/pubkey data).

The migration: 1. Reads each event from legacy storage (evt/sev prefixes) 2. Creates SerialEventId mapping (sei prefix) for event ID lookup 3. Re-encodes the event in compact format 4. Stores in cmp prefix 5. Optionally removes legacy storage after successful migration

func (*D) CountEvents

func (d *D) CountEvents(c context.Context, f *filter.F) (
	count int, approx bool, err error,
)

CountEvents mirrors the initial selection logic of QueryEvents but stops once we have identified candidate event serials (id/pk/ts). It returns the count of those serials. The `approx` flag is always false as requested.

func (*D) CreateNRCConnection

func (d *D) CreateNRCConnection(label string, createdBy []byte) (*NRCConnection, error)

CreateNRCConnection generates a new NRC connection with a random secret. createdBy is the pubkey of the admin creating this connection (can be nil for system-created).

func (*D) DeleteAccessRecord

func (d *D) DeleteAccessRecord(serial uint64) error

DeleteAccessRecord removes the access tracking record for an event. This should be called when an event is deleted.

func (*D) DeleteBlob

func (d *D) DeleteBlob(sha256Hash []byte, pubkey []byte) (err error)

DeleteBlob deletes a blob and its metadata

func (*D) DeleteEvent

func (d *D) DeleteEvent(c context.Context, eid []byte) (err error)

DeleteEvent removes an event from the database identified by `eid`. If noTombstone is false or not provided, a tombstone is created for the event.

func (*D) DeleteEventBySerial

func (d *D) DeleteEventBySerial(
	c context.Context, ser *types.Uint40, ev *event.E,
) (err error)

func (*D) DeleteExpired

func (d *D) DeleteExpired()

func (*D) DeleteInviteCode

func (d *D) DeleteInviteCode(code string) error

DeleteInviteCode removes an invite code (after use)

func (*D) DeleteMarker

func (d *D) DeleteMarker(key string) (err error)

DeleteMarker removes a marker from the database

func (*D) DeleteNRCConnection

func (d *D) DeleteNRCConnection(id string) error

DeleteNRCConnection removes an NRC connection from the database.

func (*D) DeletePaidSubscription

func (d *D) DeletePaidSubscription(pubkeyHex string) error

DeletePaidSubscription removes a paid subscription.

func (*D) DeleteWireGuardPeer

func (d *D) DeleteWireGuardPeer(nostrPubkey []byte) error

DeleteWireGuardPeer removes a WireGuard peer from the database. Note: The sequence number is not recycled to prevent subnet reuse.

func (*D) EventIDHexToSerial

func (d *D) EventIDHexToSerial(eventIDHex string) (*types.Uint40, error)

EventIDHexToSerial converts an event ID hex string to its serial, if it exists. Returns an error if the event is not in the database.

func (*D) EventIdsBySerial

func (d *D) EventIdsBySerial(start uint64, count int) (
	evs []uint64, err error,
)

func (*D) Export

func (d *D) Export(c context.Context, w io.Writer, pubkeys ...[]byte)

Export the complete database of stored events to an io.Writer in line structured minified JSON. Supports both legacy and compact event formats.

func (*D) ExtendBlossomSubscription

func (d *D) ExtendBlossomSubscription(
	pubkey []byte, level string, storageMB int64, days int,
) error

ExtendBlossomSubscription extends or creates a blossom subscription with service level

func (*D) ExtendSubscription

func (d *D) ExtendSubscription(pubkey []byte, days int) error

func (*D) FetchEventBySerial

func (d *D) FetchEventBySerial(ser *types.Uint40) (ev *event.E, err error)

FetchEventBySerial fetches a single event by its serial. This function tries multiple storage formats in order: 1. cmp (compact format with serial references) - newest, most space-efficient 2. sev (small event inline) - legacy Reiser4 optimization 3. evt (traditional separate storage) - legacy fallback

func (*D) FetchEventsBySerials

func (d *D) FetchEventsBySerials(serials []*types.Uint40) (events map[uint64]*event.E, err error)

FetchEventsBySerials fetches multiple events by their serials in a single database transaction. Returns a map of serial uint64 value to event, only including successfully fetched events.

This function tries multiple storage formats in order: 1. cmp (compact format with serial references) - newest, most space-efficient 2. sev (small event inline) - legacy Reiser4 optimization 3. evt (traditional separate storage) - legacy fallback

func (*D) FindEventByAuthorAndKind

func (d *D) FindEventByAuthorAndKind(authorSerial *types.Uint40, kind uint16) (*types.Uint40, error)

FindEventByAuthorAndKind finds the most recent event of a specific kind by an author. This is used to find kind-3 contact lists for follow graph traversal. Returns nil, nil if no matching event is found.

func (*D) FindMentions

func (d *D) FindMentions(pubkey []byte, kinds []uint16) (*GraphResult, error)

FindMentions finds events that mention a pubkey via p-tags. This returns events grouped by depth, where depth represents how the events relate: - Depth 1: Events that directly mention the seed pubkey - Depth 2+: Not typically used for mentions (reserved for future expansion)

The kinds parameter filters which event kinds to include (e.g., [1] for notes only, [1,7] for notes and reactions, etc.)

func (*D) FindMentionsByPubkeys

func (d *D) FindMentionsByPubkeys(pubkeySerials []*types.Uint40, kinds []uint16) (*GraphResult, error)

FindMentionsByPubkeys returns events that mention any of the given pubkeys. Useful for finding mentions across a set of followed accounts.

func (*D) FindMentionsFromHex

func (d *D) FindMentionsFromHex(pubkeyHex string, kinds []uint16) (*GraphResult, error)

FindMentionsFromHex is a convenience wrapper that accepts hex-encoded pubkey.

func (*D) GetAccessLogs

func (d *D) GetAccessLogs(nostrPubkey []byte) (logs []*WireGuardAccessLog, err error)

GetAccessLogs returns access logs for a user.

func (*D) GetAliasByPubkey

func (d *D) GetAliasByPubkey(pubkeyHex string) (string, error)

GetAliasByPubkey returns the alias for a pubkey, or "" if none.

func (*D) GetAliasesByPubkey

func (d *D) GetAliasesByPubkey(pubkeyHex string) ([]string, error)

GetAliasesByPubkey returns all aliases for a pubkey.

func (*D) GetAllAccessLogs

func (d *D) GetAllAccessLogs() (logs []*WireGuardAccessLog, err error)

GetAllAccessLogs returns all access logs (admin view).

func (*D) GetAllNIP43Members

func (d *D) GetAllNIP43Members() ([][]byte, error)

GetAllNIP43Members returns all NIP-43 members

func (*D) GetAllNRCConnections

func (d *D) GetAllNRCConnections() (conns []*NRCConnection, err error)

GetAllNRCConnections returns all NRC connections.

func (*D) GetAllRevokedKeys

func (d *D) GetAllRevokedKeys() (keys []*WireGuardRevokedKey, err error)

GetAllRevokedKeys returns all revoked keys across all users (admin view).

func (*D) GetAllWireGuardPeers

func (d *D) GetAllWireGuardPeers() (peers []*WireGuardPeer, err error)

GetAllWireGuardPeers returns all WireGuard peers.

func (*D) GetBlob

func (d *D) GetBlob(sha256Hash []byte) (data []byte, metadata *BlobMetadata, err error)

GetBlob retrieves blob data by SHA256 hash

func (*D) GetBlobMetadata

func (d *D) GetBlobMetadata(sha256Hash []byte) (metadata *BlobMetadata, err error)

GetBlobMetadata retrieves only metadata for a blob

func (*D) GetBlossomStorageQuota

func (d *D) GetBlossomStorageQuota(pubkey []byte) (quotaMB int64, err error)

GetBlossomStorageQuota returns the current blossom storage quota in MB for a pubkey

func (*D) GetCachedEvents

func (d *D) GetCachedEvents(f *filter.F) (event.S, bool)

GetCachedEvents retrieves cached events for a filter (without subscription ID) Returns nil, false if not found

func (*D) GetCachedJSON

func (d *D) GetCachedJSON(f *filter.F) ([][]byte, bool)

GetCachedJSON retrieves cached marshaled JSON for a filter Returns nil, false if not found

func (*D) GetETagsFromEventSerial

func (d *D) GetETagsFromEventSerial(eventSerial *types.Uint40) ([]*types.Uint40, error)

GetETagsFromEventSerial extracts e-tag event serials from an event by its serial. This is a pure index-based operation - no event decoding required. It scans the eeg (event-event-graph) index for outbound e-tag edges.

func (*D) GetEventAccessInfo

func (d *D) GetEventAccessInfo(serial uint64) (lastAccess int64, accessCount uint32, err error)

GetEventAccessInfo returns access information for an event. Returns (0, 0, nil) if the event has never been accessed.

func (*D) GetEventAuthorSerial

func (d *D) GetEventAuthorSerial(eventSerial *types.Uint40) (*types.Uint40, error)

GetEventAuthorSerial finds the author pubkey serial for an event. Uses the epg (event-pubkey-graph) index with author direction.

func (*D) GetEventIDFromSerial

func (d *D) GetEventIDFromSerial(serial *types.Uint40) (string, error)

GetEventIDFromSerial converts an event serial to its hex ID string.

func (*D) GetEventIdBySerial

func (d *D) GetEventIdBySerial(ser *types.Uint40) (eventId []byte, err error)

GetEventIdBySerial looks up an event ID by its serial number. Uses the SerialEventId index (sei prefix).

func (*D) GetEventsByAuthor

func (d *D) GetEventsByAuthor(authorSerial *types.Uint40, kinds []uint16) ([]*types.Uint40, error)

GetEventsByAuthor finds all events authored by a pubkey. Uses the peg (pubkey-event-graph) index with direction filter for author edges. Optionally filters by event kinds.

func (*D) GetEventsReferencingPubkey

func (d *D) GetEventsReferencingPubkey(pubkeySerial *types.Uint40, kinds []uint16) ([]*types.Uint40, error)

GetEventsReferencingPubkey finds all events that reference a pubkey via p-tags. Uses the peg (pubkey-event-graph) index with direction filter for inbound p-tags. Optionally filters by event kinds.

func (*D) GetFollowersByKindViaGPP

func (d *D) GetFollowersByKindViaGPP(targetSerial *types.Uint40, kind uint16) ([]*types.Uint40, error)

GetFollowersByKindViaGPP returns pubkey serials that reference the target pubkey via the gpp reverse index, filtered to a specific event kind. This enables efficient lookup of e.g. kind-10000 muters or kind-1984 reporters without scanning unrelated relationship types.

Key format: gpp(3)|target(5)|kind(2)|direction(1)|source(5)|event(5) = 21 bytes Prefix used: gpp(3)|target(5)|kind(2) = 10 bytes

func (*D) GetFollowersOfPubkeySerial

func (d *D) GetFollowersOfPubkeySerial(targetSerial *types.Uint40) ([]*types.Uint40, error)

GetFollowersOfPubkeySerial returns the pubkey serials of users who follow a given pubkey. This finds all kind-3 events that have a p-tag referencing the target pubkey.

func (*D) GetFollowersViaGPP

func (d *D) GetFollowersViaGPP(targetSerial *types.Uint40) ([]*types.Uint40, error)

GetFollowersViaGPP returns pubkey serials that reference the target pubkey via the gpp (graph-pubkey-pubkey) reverse index. This is a single prefix scan that replaces the three-hop GetFollowersOfPubkeySerial (find referencing events → get authors → dedup).

Key format: gpp(3)|target(5)|kind(2)|direction(1)|source(5)|event(5) = 21 bytes For kind-3 followers, we can optionally narrow with kind filter.

func (*D) GetFollowsByKindViaPPG

func (d *D) GetFollowsByKindViaPPG(sourceSerial *types.Uint40, kind uint16) ([]*types.Uint40, error)

GetFollowsByKindViaPPG returns pubkey serials that the source pubkey references via the ppg index, filtered to a specific event kind. This restricts the outbound scan to e.g. kind-3 follows only, excluding kind-10000 mutes or kind-1984 reports.

Key format: ppg(3)|source(5)|target(5)|kind(2)|direction(1)|event(5) = 21 bytes Scan prefix: ppg(3)|source(5) = 8 bytes; kind is checked at bytes 13..15 in-loop.

func (*D) GetFollowsFromPubkeySerial

func (d *D) GetFollowsFromPubkeySerial(pubkeySerial *types.Uint40) ([]*types.Uint40, error)

GetFollowsFromPubkeySerial returns the pubkey serials that a user follows. This extracts p-tags from the user's kind-3 contact list event. Returns an empty slice if no kind-3 event is found.

func (*D) GetFollowsViaPPG

func (d *D) GetFollowsViaPPG(sourceSerial *types.Uint40) ([]*types.Uint40, error)

GetFollowsViaPPG returns pubkey serials that the source pubkey references via the ppg (pubkey-pubkey-graph) materialized index. This is a single prefix scan that replaces the two-hop GetFollowsFromPubkeySerial (find kind-3 → extract p-tags).

Key format: ppg(3)|source(5)|target(5)|kind(2)|direction(1)|event(5) = 21 bytes We scan prefix ppg|source and extract target serials at offset 8..13.

func (*D) GetFullIdPubkeyBySerial

func (d *D) GetFullIdPubkeyBySerial(ser *types.Uint40) (
	fidpk *store.IdPkTs, err error,
)

func (*D) GetFullIdPubkeyBySerials

func (d *D) GetFullIdPubkeyBySerials(sers []*types.Uint40) (
	fidpks []*store.IdPkTs, err error,
)

GetFullIdPubkeyBySerials seeks directly to each serial's prefix in the FullIdPubkey index. The input sers slice is expected to be sorted in ascending order, allowing efficient forward-only iteration via a single Badger iterator.

func (*D) GetLeastAccessedEvents

func (d *D) GetLeastAccessedEvents(limit int, minAgeSec int64) (serials []uint64, err error)

GetLeastAccessedEvents returns event serials sorted by coldness. Events with older last access times and lower access counts are returned first. limit: maximum number of events to return minAgeSec: minimum age in seconds since last access (events accessed more recently are excluded)

func (*D) GetMarker

func (d *D) GetMarker(key string) (value []byte, err error)

GetMarker retrieves an arbitrary marker from the database

func (*D) GetNIP43Membership

func (d *D) GetNIP43Membership(pubkey []byte) (*NIP43Membership, error)

GetNIP43Membership retrieves membership details for a pubkey

func (*D) GetNRCAuthorizedSecrets

func (d *D) GetNRCAuthorizedSecrets() (map[string]string, error)

GetNRCAuthorizedSecrets returns a map of derived pubkeys to labels for all connections. This is used by the NRC bridge to authorize incoming connections.

func (*D) GetNRCConnection

func (d *D) GetNRCConnection(id string) (conn *NRCConnection, err error)

GetNRCConnection retrieves an NRC connection by ID.

func (*D) GetNRCConnectionByDerivedPubkey

func (d *D) GetNRCConnectionByDerivedPubkey(derivedPubkey []byte) (*NRCConnection, error)

GetNRCConnectionByDerivedPubkey retrieves an NRC connection by its derived pubkey.

func (*D) GetNRCConnectionURI

func (d *D) GetNRCConnectionURI(conn *NRCConnection, relayPubkey []byte, rendezvousURL string) (string, error)

GetNRCConnectionURI generates the full connection URI for a connection. relayPubkey is the relay's public key (32 bytes). rendezvousURL is the public relay URL.

func (*D) GetNextWGSequence

func (d *D) GetNextWGSequence() (seq uint64, err error)

GetNextWGSequence retrieves and increments the sequence counter using Badger's Sequence.

func (*D) GetOrCreatePubkeySerial

func (d *D) GetOrCreatePubkeySerial(pubkey []byte) (ser *types.Uint40, err error)

GetOrCreatePubkeySerial returns the serial for a pubkey, creating one if it doesn't exist. The pubkey parameter should be 32 bytes (schnorr public key). This function is thread-safe and uses transactions to ensure atomicity.

func (*D) GetOrCreateRelayIdentitySecret

func (d *D) GetOrCreateRelayIdentitySecret() (skb []byte, err error)

GetOrCreateRelayIdentitySecret retrieves the existing relay identity secret key or creates and stores a new one if none exists.

func (*D) GetOrCreateSubnetPool

func (d *D) GetOrCreateSubnetPool(baseNetwork string) (*wireguard.SubnetPool, error)

GetOrCreateSubnetPool creates or restores a subnet pool from the database.

func (*D) GetOrCreateWireGuardPeer

func (d *D) GetOrCreateWireGuardPeer(nostrPubkey []byte, pool *wireguard.SubnetPool) (peer *WireGuardPeer, err error)

GetOrCreateWireGuardPeer retrieves or creates a WireGuard peer. The pool is used for subnet derivation from the sequence number.

func (*D) GetOrCreateWireGuardServerKey

func (d *D) GetOrCreateWireGuardServerKey() (key []byte, err error)

GetOrCreateWireGuardServerKey retrieves or creates the WireGuard server key.

func (*D) GetPTagsFromEventSerial

func (d *D) GetPTagsFromEventSerial(eventSerial *types.Uint40) ([]*types.Uint40, error)

GetPTagsFromEventSerial extracts p-tag pubkey serials from an event by its serial. This is a pure index-based operation - no event decoding required. It scans the epg (event-pubkey-graph) index for p-tag edges.

func (*D) GetPaidSubscription

func (d *D) GetPaidSubscription(pubkeyHex string) (*PaidSubscription, error)

GetPaidSubscription returns the paid subscription for a pubkey.

func (*D) GetPaymentHistory

func (d *D) GetPaymentHistory(pubkey []byte) ([]Payment, error)

func (*D) GetPubkeyByAlias

func (d *D) GetPubkeyByAlias(alias string) (string, error)

GetPubkeyByAlias returns the pubkey for an alias, or "" if not found.

func (*D) GetPubkeyBySerial

func (d *D) GetPubkeyBySerial(ser *types.Uint40) (pubkey []byte, err error)

GetPubkeyBySerial returns the full 32-byte pubkey for a given serial.

func (*D) GetPubkeyHexFromSerial

func (d *D) GetPubkeyHexFromSerial(serial *types.Uint40) (string, error)

GetPubkeyHexFromSerial converts a pubkey serial to its hex string representation.

func (*D) GetPubkeySerial

func (d *D) GetPubkeySerial(pubkey []byte) (ser *types.Uint40, err error)

GetPubkeySerial returns the serial for a pubkey if it exists. Returns an error if the pubkey doesn't have a serial yet.

func (*D) GetReferencingEvents

func (d *D) GetReferencingEvents(targetSerial *types.Uint40, kinds []uint16) ([]*types.Uint40, error)

GetReferencingEvents finds all events that reference a target event via e-tags. Optionally filters by event kinds. Uses the gee (reverse e-tag graph) index.

func (*D) GetRelayIdentitySecret

func (d *D) GetRelayIdentitySecret() (skb []byte, err error)

GetRelayIdentitySecret returns the relay identity secret key bytes if present. If the key is not found, returns (nil, badger.ErrKeyNotFound).

func (*D) GetRevokedKeys

func (d *D) GetRevokedKeys(nostrPubkey []byte) (keys []*WireGuardRevokedKey, err error)

GetRevokedKeys returns all revoked keys for a user.

func (*D) GetSerialById

func (d *D) GetSerialById(id []byte) (ser *types.Uint40, err error)

func (*D) GetSerialsByIds

func (d *D) GetSerialsByIds(ids *tag.T) (
	serials map[string]*types.Uint40, err error,
)

GetSerialsByIds takes a tag.T containing multiple IDs and returns a map of IDs to their corresponding serial numbers. It directly queries the IdPrefix index for matching IDs, which is more efficient than using GetIndexesFromFilter.

func (*D) GetSerialsByIdsWithFilter

func (d *D) GetSerialsByIdsWithFilter(
	ids *tag.T, fn func(ev *event.E, ser *types.Uint40) bool,
) (serials map[string]*types.Uint40, err error)

GetSerialsByIdsWithFilter takes a tag.T containing multiple IDs and returns a map of IDs to their corresponding serial numbers, applying a filter function to each event. The function directly creates ID index prefixes for efficient querying.

func (*D) GetSerialsByRange

func (d *D) GetSerialsByRange(idx Range) (
	sers types.Uint40s, err error,
)

func (*D) GetSerialsFromFilter

func (d *D) GetSerialsFromFilter(f *filter.F) (
	sers types.Uint40s, err error,
)

func (*D) GetSubnetSeed

func (d *D) GetSubnetSeed() (seed []byte, err error)

GetSubnetSeed retrieves the subnet pool seed.

func (*D) GetSubscription

func (d *D) GetSubscription(pubkey []byte) (*Subscription, error)

func (*D) GetThreadParents

func (d *D) GetThreadParents(eventID []byte) (*GraphResult, error)

GetThreadParents finds events that a given event references (its parents/quotes).

func (*D) GetThreadReplies

func (d *D) GetThreadReplies(eventID []byte, kinds []uint16) (*GraphResult, error)

GetThreadReplies finds all direct replies to an event. This is a convenience method that returns events at depth 1 with inbound direction.

func (*D) GetThumbnail

func (d *D) GetThumbnail(key string) (data []byte, err error)

GetThumbnail retrieves a cached thumbnail by key

func (*D) GetTotalBlobStorageUsed

func (d *D) GetTotalBlobStorageUsed(pubkey []byte) (totalMB int64, err error)

GetTotalBlobStorageUsed calculates total storage used by a pubkey in MB

func (*D) GetWireGuardPeer

func (d *D) GetWireGuardPeer(nostrPubkey []byte) (peer *WireGuardPeer, err error)

GetWireGuardPeer retrieves a WireGuard peer by Nostr pubkey.

func (*D) GetWireGuardServerKey

func (d *D) GetWireGuardServerKey() (key []byte, err error)

GetWireGuardServerKey retrieves the WireGuard server private key.

func (*D) HasBlob

func (d *D) HasBlob(sha256Hash []byte) (exists bool, err error)

HasBlob checks if a blob exists

func (*D) HasMarker

func (d *D) HasMarker(key string) (exists bool)

HasMarker checks if a marker exists in the database

func (*D) HealthCheck

func (d *D) HealthCheck(progress io.Writer) (report *HealthReport, err error)

HealthCheck performs a comprehensive health check of the database. It scans all index prefixes and verifies referential integrity.

func (*D) Import

func (d *D) Import(rr io.Reader)

Import a collection of events in line structured minified JSON format (JSONL). This runs synchronously to ensure the reader remains valid during processing. The actual event processing happens after buffering to a temp file, so the caller can close the reader after Import returns.

func (*D) ImportEventsFromReader

func (d *D) ImportEventsFromReader(ctx context.Context, rr io.Reader) error

ImportEventsFromReader imports events from an io.Reader containing JSONL data

func (*D) ImportEventsFromStrings

func (d *D) ImportEventsFromStrings(ctx context.Context, eventJSONs []string, policyManager interface {
	CheckPolicy(action string, ev *event.E, pubkey []byte, remote string) (bool, error)
}) error

ImportEventsFromStrings imports events from a slice of JSON strings with policy filtering

func (*D) IncrementRevokedKeyAccess

func (d *D) IncrementRevokedKeyAccess(nostrPubkey, wgPubkey []byte) error

IncrementRevokedKeyAccess updates the access count for a revoked key.

func (*D) Init

func (d *D) Init(path string) (err error)

Init initializes the database with the given path.

func (*D) InvalidateQueryCache

func (d *D) InvalidateQueryCache()

InvalidateQueryCache clears all entries from the query cache

func (*D) IsAliasTaken

func (d *D) IsAliasTaken(alias string) (bool, error)

IsAliasTaken returns true if the alias is claimed by any pubkey.

func (*D) IsFirstTimeUser

func (d *D) IsFirstTimeUser(pubkey []byte) (bool, error)

IsFirstTimeUser checks if a user is logging in for the first time and marks them as seen

func (*D) IsNIP43Member

func (d *D) IsNIP43Member(pubkey []byte) (isMember bool, err error)

IsNIP43Member checks if a pubkey is a NIP-43 member

func (*D) IsSubscriptionActive

func (d *D) IsSubscriptionActive(pubkey []byte) (bool, error)

func (*D) ListAllBlobUserStats

func (d *D) ListAllBlobUserStats() (stats []*UserBlobStats, err error)

ListAllBlobUserStats returns storage statistics for all users who have uploaded blobs

func (*D) ListAllBlobs

func (d *D) ListAllBlobs() (descriptors []*BlobDescriptor, err error)

ListAllBlobs returns all blob descriptors in the database

func (*D) ListBlobs

func (d *D) ListBlobs(
	pubkey []byte, since, until int64,
) (descriptors []*BlobDescriptor, err error)

ListBlobs lists all blobs for a given pubkey

func (*D) ListPaidSubscriptions

func (d *D) ListPaidSubscriptions() ([]*PaidSubscription, error)

ListPaidSubscriptions returns all paid subscriptions.

func (*D) LogCompactSavings

func (d *D) LogCompactSavings()

LogCompactSavings logs the storage savings achieved by compact format. Call this periodically or after significant operations.

func (*D) LogObsoleteAccess

func (d *D) LogObsoleteAccess(nostrPubkey, wgPubkey []byte, sequence uint32, remoteAddr string) error

LogObsoleteAccess records an access attempt to an obsolete WireGuard address.

func (*D) Path

func (d *D) Path() string

Path returns the path where the database files are stored.

func (*D) ProcessDelete

func (d *D) ProcessDelete(ev *event.E, admins [][]byte) (err error)

func (*D) PubkeyHexToSerial

func (d *D) PubkeyHexToSerial(pubkeyHex string) (*types.Uint40, error)

PubkeyHexToSerial converts a pubkey hex string to its serial, if it exists. Returns an error if the pubkey is not in the database.

func (*D) PublishNIP43MembershipEvent

func (d *D) PublishNIP43MembershipEvent(kind int, pubkey []byte) error

PublishNIP43MembershipEvent publishes membership change events

func (*D) QueryAllVersions

func (d *D) QueryAllVersions(c context.Context, f *filter.F) (
	evs event.S, err error,
)

QueryAllVersions queries events and returns all versions of replaceable events

func (*D) QueryCacheStats

func (d *D) QueryCacheStats() querycache.CacheStats

QueryCacheStats returns statistics about the query cache

func (*D) QueryDeleteEventsByTargetId

func (d *D) QueryDeleteEventsByTargetId(c context.Context, targetEventId []byte) (
	evs event.S, err error,
)

QueryDeleteEventsByTargetId queries for delete events that target a specific event ID

func (*D) QueryEvents

func (d *D) QueryEvents(c context.Context, f *filter.F) (
	evs event.S, err error,
)

func (*D) QueryEventsWithOptions

func (d *D) QueryEventsWithOptions(c context.Context, f *filter.F, includeDeleteEvents bool, showAllVersions bool) (
	evs event.S, err error,
)

func (*D) QueryForAddressableEvent

func (d *D) QueryForAddressableEvent(f *filter.F) (serial *types.Uint40, err error)

QueryForAddressableEvent performs a direct O(1) lookup for a NIP-33 parameterized replaceable event using the AddressableEvent index. Returns the serial if found, nil if not found, or an error.

func (*D) QueryForIds

func (d *D) QueryForIds(c context.Context, f *filter.F) (
	idPkTs []*store.IdPkTs, err error,
)

QueryForIds retrieves a list of IdPkTs based on the provided filter. It supports filtering by ranges and tags but disallows filtering by Ids. Results are sorted by timestamp in reverse chronological order by default. When a search query is present, results are ranked by a 50/50 blend of match count (how many distinct search terms matched) and recency. Returns an error if the filter contains Ids or if any operation fails.

func (*D) QueryForSerials

func (d *D) QueryForSerials(c context.Context, f *filter.F) (
	sers types.Uint40s, err error,
)

QueryForSerials takes a filter and returns the serials of events that match, sorted in reverse chronological order.

func (*D) QueryPTagGraph

func (d *D) QueryPTagGraph(f *filter.F) (sers types.Uint40s, err error)

QueryPTagGraph uses the pubkey graph index for efficient p-tag queries.

This query path is optimized for filters like:

{"#p": ["<pubkey>"], "kinds": [1, 6, 7]}

Performance benefits: - 41% smaller index keys (16 bytes vs 27 bytes) - No hash collisions (exact serial match) - Kind-indexed in key structure - Direction-aware filtering

func (*D) Ready

func (d *D) Ready() <-chan struct{}

Ready returns a channel that closes when the database is ready to serve requests. This allows callers to wait for database warmup to complete.

func (*D) RebuildWordIndexesWithNormalization

func (d *D) RebuildWordIndexesWithNormalization()

RebuildWordIndexesWithNormalization rebuilds all word indexes with unicode normalization applied. This migration: 1. Deletes all existing word indexes (wrd prefix) 2. Re-tokenizes all events with normalizeRune() applied 3. Creates new consolidated indexes where decorative unicode maps to ASCII

After this migration, "ᴅᴇᴀᴛʜ" (small caps) and "𝔇𝔢𝔞𝔱𝔥" (fraktur) will index the same as "death", eliminating duplicate entries and enabling proper search.

func (*D) ReconcileBlobMetadata

func (d *D) ReconcileBlobMetadata() (reconciled int, err error)

ReconcileBlobMetadata scans the blossom directory for blob files that don't have corresponding metadata in the database and creates entries for them. This is useful for recovering from situations where blob files exist but their metadata was lost or never created.

func (*D) RecordEventAccess

func (d *D) RecordEventAccess(serial uint64, connectionID string) error

RecordEventAccess updates access tracking for an event. This increments the access count and updates the last access time. The connectionID is currently not used for deduplication in the database layer, but is passed for potential future use. Deduplication is handled in the higher-level AccessTracker which maintains an in-memory cache.

func (*D) RecordPayment

func (d *D) RecordPayment(
	pubkey []byte, amount int64, invoice, preimage string,
) error

func (*D) ReencodeEventsWithOptimizedTags

func (d *D) ReencodeEventsWithOptimizedTags()

ReencodeEventsWithOptimizedTags re-encodes all events to use the new binary tag format that stores e/p tag values as 33-byte binary (32-byte hash + null) instead of 64-byte hex strings. This reduces memory usage by ~48% for these tags.

func (*D) RegenerateWireGuardPeer

func (d *D) RegenerateWireGuardPeer(nostrPubkey []byte, pool *wireguard.SubnetPool) (peer *WireGuardPeer, err error)

RegenerateWireGuardPeer generates a new keypair for an existing peer. The sequence number (and thus subnet) is preserved. The old keypair is archived for audit purposes.

func (*D) ReleaseQuerySlot

func (d *D) ReleaseQuerySlot()

ReleaseQuerySlot releases a previously acquired query slot.

func (*D) RemoveNIP43Member

func (d *D) RemoveNIP43Member(pubkey []byte) error

RemoveNIP43Member removes a member from the NIP-43 membership list

func (*D) Repair

func (d *D) Repair(ctx context.Context, opts *RepairOptions) (report *RepairReport, err error)

Repair performs database repair operations based on the provided options. It fixes integrity issues found by HealthCheck.

func (*D) RunMigrations

func (d *D) RunMigrations()

func (*D) SaveBlob

func (d *D) SaveBlob(
	sha256Hash []byte, data []byte, pubkey []byte, mimeType string, extension string,
) (err error)

SaveBlob stores a blob with its metadata

func (*D) SaveBlobMetadata

func (d *D) SaveBlobMetadata(
	sha256Hash []byte, size int64, pubkey []byte, mimeType string, extension string,
) (err error)

SaveBlobMetadata stores only the metadata and index for a blob whose file already exists on disk. This is used by the streaming upload path where the file is written during hashing and then renamed into place before this call.

func (*D) SaveBlobReport

func (d *D) SaveBlobReport(sha256Hash []byte, reportData []byte) (err error)

SaveBlobReport stores a report for a blob (BUD-09)

func (*D) SaveEvent

func (d *D) SaveEvent(c context.Context, ev *event.E) (
	replaced bool, err error,
)

SaveEvent saves an event to the database, generating all the necessary indexes.

func (*D) SaveNRCConnection

func (d *D) SaveNRCConnection(conn *NRCConnection) error

SaveNRCConnection stores an NRC connection in the database.

func (*D) SavePaidSubscription

func (d *D) SavePaidSubscription(sub *PaidSubscription) error

SavePaidSubscription saves or updates a paid subscription. Delegates to the PaidACL storage layer.

func (*D) SaveThumbnail

func (d *D) SaveThumbnail(key string, data []byte) error

SaveThumbnail caches a thumbnail with the given key

func (*D) SerialCacheStats

func (d *D) SerialCacheStats() SerialCacheStats

SerialCacheStats returns statistics about the serial cache.

func (*D) SetLogLevel

func (d *D) SetLogLevel(level string)

func (*D) SetMarker

func (d *D) SetMarker(key string, value []byte) (err error)

SetMarker stores an arbitrary marker in the database

func (*D) SetRateLimiter

func (d *D) SetRateLimiter(limiter RateLimiterInterface)

SetRateLimiter sets the rate limiter for controlling memory during import/export

func (*D) SetRelayIdentitySecret

func (d *D) SetRelayIdentitySecret(skb []byte) (err error)

SetRelayIdentitySecret stores the relay identity secret key bytes (expects 32 bytes).

func (*D) SetSubnetSeed

func (d *D) SetSubnetSeed(seed []byte) error

SetSubnetSeed stores the subnet pool seed.

func (*D) SetWireGuardServerKey

func (d *D) SetWireGuardServerKey(key []byte) error

SetWireGuardServerKey stores the WireGuard server private key.

func (*D) StoreEventIdSerial

func (d *D) StoreEventIdSerial(txn *badger.Txn, serial uint64, eventId []byte) error

StoreEventIdSerial stores the mapping from event serial to full event ID. This is called during event save to enable later reconstruction.

func (*D) StoreInviteCode

func (d *D) StoreInviteCode(code string, expiresAt time.Time) error

StoreInviteCode stores an invite code with expiry

func (*D) Sync

func (d *D) Sync() (err error)

Sync flushes the database buffers to disk.

func (*D) TraverseEventEventFromPubkey

func (d *D) TraverseEventEventFromPubkey(seedPubkey []byte, maxDepth int, direction string) (*GraphResult, error)

TraverseEventEventFromPubkey performs BFS traversal of event↔event edges, seeded from events authored by the given pubkey.

func (*D) TraverseFollowers

func (d *D) TraverseFollowers(seedPubkey []byte, maxDepth int) (*GraphResult, error)

TraverseFollowers performs BFS traversal to find who follows the seed pubkey. This is the reverse of TraverseFollows - it finds users whose kind-3 lists contain the target pubkey(s).

At each depth: - Depth 1: Users who directly follow the seed - Depth 2: Users who follow anyone at depth 1 (followers of followers) - etc.

func (*D) TraverseFollowersFromHex

func (d *D) TraverseFollowersFromHex(seedPubkeyHex string, maxDepth int) (*GraphResult, error)

TraverseFollowersFromHex is a convenience wrapper that accepts hex-encoded pubkey.

func (*D) TraverseFollows

func (d *D) TraverseFollows(seedPubkey []byte, maxDepth int) (*GraphResult, error)

TraverseFollows performs BFS traversal of the follow graph starting from a seed pubkey. Returns pubkeys grouped by first-discovered depth (no duplicates across depths).

The traversal works by: 1. Starting with the seed pubkey at depth 0 (not included in results) 2. For each pubkey at the current depth, find their kind-3 contact list 3. Extract p-tags from the contact list to get follows 4. Add new (unseen) follows to the next depth 5. Continue until maxDepth is reached or no new pubkeys are found

Early termination occurs if two consecutive depths yield no new pubkeys.

func (*D) TraverseFollowsFromHex

func (d *D) TraverseFollowsFromHex(seedPubkeyHex string, maxDepth int) (*GraphResult, error)

TraverseFollowsFromHex is a convenience wrapper that accepts hex-encoded pubkey.

func (*D) TraversePubkeyEvent

func (d *D) TraversePubkeyEvent(seedPubkey []byte, maxDepth int, direction string) (*GraphResult, error)

TraversePubkeyEvent performs BFS traversal of pubkey↔event edges. Direction "out" = events authored by/referencing the seed pubkey (peg index). Direction "in" = events that reference the seed pubkey (epg reverse).

func (*D) TraversePubkeyPubkey

func (d *D) TraversePubkeyPubkey(seedPubkey []byte, maxDepth int, direction string) (*GraphResult, error)

TraversePubkeyPubkey performs BFS traversal of the pubkey↔pubkey graph using the ppg/gpp materialized index. This collapses the two-hop pubkey→event→pubkey traversal into a single prefix scan per frontier node.

Direction selects which index to scan:

  • "out": ppg index (who does seed follow/reference?)
  • "in": gpp index (who references seed?)
  • "both": union of both directions at each depth

func (*D) TraversePubkeyPubkeyBaseline

func (d *D) TraversePubkeyPubkeyBaseline(seedPubkey []byte, maxDepth int, direction string) (*GraphResult, error)

TraversePubkeyPubkeyBaseline performs the same BFS as TraversePubkeyPubkey but uses the old multi-hop approach: find kind-3 event → extract p-tags. This provides the benchmark baseline for comparison against the ppg/gpp index.

func (*D) TraverseThread

func (d *D) TraverseThread(seedEventID []byte, maxDepth int, direction string) (*GraphResult, error)

TraverseThread performs BFS traversal of thread structure via e-tags. Starting from a seed event, it finds all replies/references at each depth.

The traversal works bidirectionally: - Forward: Events that the seed references (parents, quoted posts) - Backward: Events that reference the seed (replies, reactions, reposts)

Parameters: - seedEventID: The event ID to start traversal from - maxDepth: Maximum depth to traverse - direction: "both" (default), "inbound" (replies to seed), "outbound" (seed's references)

func (*D) TraverseThreadFromHex

func (d *D) TraverseThreadFromHex(seedEventIDHex string, maxDepth int, direction string) (*GraphResult, error)

TraverseThreadFromHex is a convenience wrapper that accepts hex-encoded event ID.

func (*D) UpdateExpirationTags

func (d *D) UpdateExpirationTags()

func (*D) UpdateNRCConnectionLastUsed

func (d *D) UpdateNRCConnectionLastUsed(id string) error

UpdateNRCConnectionLastUsed updates the last used timestamp for a connection.

func (*D) UpdateWordIndexes

func (d *D) UpdateWordIndexes()

func (*D) ValidateInviteCode

func (d *D) ValidateInviteCode(code string) (valid bool, err error)

ValidateInviteCode checks if an invite code is valid and not expired

func (*D) Wipe

func (d *D) Wipe() (err error)

func (*D) WouldReplaceEvent

func (d *D) WouldReplaceEvent(ev *event.E) (bool, types.Uint40s, error)

WouldReplaceEvent checks if the provided event would replace existing events based on Nostr's replaceable or parameterized replaceable semantics. It returns true if the candidate is newer-or-equal than existing events. If an existing event is newer, it returns (false, nil, ErrOlderThanExisting). If no conflicts exist, it returns (false, nil, nil).

type Database

type Database interface {
	// Core lifecycle methods
	Path() string
	Init(path string) error
	Sync() error
	Close() error
	Wipe() error
	SetLogLevel(level string)
	Ready() <-chan struct{} // Returns a channel that closes when database is ready to serve requests

	// Event storage and retrieval
	SaveEvent(c context.Context, ev *event.E) (exists bool, err error)
	GetSerialsFromFilter(f *filter.F) (serials types.Uint40s, err error)
	WouldReplaceEvent(ev *event.E) (bool, types.Uint40s, error)

	QueryEvents(c context.Context, f *filter.F) (evs event.S, err error)
	QueryAllVersions(c context.Context, f *filter.F) (evs event.S, err error)
	QueryEventsWithOptions(c context.Context, f *filter.F, includeDeleteEvents bool, showAllVersions bool) (evs event.S, err error)
	QueryDeleteEventsByTargetId(c context.Context, targetEventId []byte) (evs event.S, err error)
	QueryForSerials(c context.Context, f *filter.F) (serials types.Uint40s, err error)
	QueryForIds(c context.Context, f *filter.F) (idPkTs []*store.IdPkTs, err error)

	CountEvents(c context.Context, f *filter.F) (count int, approximate bool, err error)

	FetchEventBySerial(ser *types.Uint40) (ev *event.E, err error)
	FetchEventsBySerials(serials []*types.Uint40) (events map[uint64]*event.E, err error)

	GetSerialById(id []byte) (ser *types.Uint40, err error)
	GetSerialsByIds(ids *tag.T) (serials map[string]*types.Uint40, err error)
	GetSerialsByIdsWithFilter(ids *tag.T, fn func(ev *event.E, ser *types.Uint40) bool) (serials map[string]*types.Uint40, err error)
	GetSerialsByRange(idx Range) (serials types.Uint40s, err error)

	GetFullIdPubkeyBySerial(ser *types.Uint40) (fidpk *store.IdPkTs, err error)
	GetFullIdPubkeyBySerials(sers []*types.Uint40) (fidpks []*store.IdPkTs, err error)

	// Event deletion
	DeleteEvent(c context.Context, eid []byte) error
	DeleteEventBySerial(c context.Context, ser *types.Uint40, ev *event.E) error
	DeleteExpired()
	ProcessDelete(ev *event.E, admins [][]byte) error
	CheckForDeleted(ev *event.E, admins [][]byte) error

	// Import/Export
	Import(rr io.Reader)
	Export(c context.Context, w io.Writer, pubkeys ...[]byte)
	ImportEventsFromReader(ctx context.Context, rr io.Reader) error
	ImportEventsFromStrings(ctx context.Context, eventJSONs []string, policyManager interface {
		CheckPolicy(action string, ev *event.E, pubkey []byte, remote string) (bool, error)
	}) error

	// Relay identity
	GetRelayIdentitySecret() (skb []byte, err error)
	SetRelayIdentitySecret(skb []byte) error
	GetOrCreateRelayIdentitySecret() (skb []byte, err error)

	// Markers (metadata key-value storage)
	SetMarker(key string, value []byte) error
	GetMarker(key string) (value []byte, err error)
	HasMarker(key string) bool
	DeleteMarker(key string) error

	// Subscriptions (payment-based access control)
	GetSubscription(pubkey []byte) (*Subscription, error)
	IsSubscriptionActive(pubkey []byte) (bool, error)
	ExtendSubscription(pubkey []byte, days int) error
	RecordPayment(pubkey []byte, amount int64, invoice, preimage string) error
	GetPaymentHistory(pubkey []byte) ([]Payment, error)
	ExtendBlossomSubscription(pubkey []byte, tier string, storageMB int64, daysExtended int) error
	GetBlossomStorageQuota(pubkey []byte) (quotaMB int64, err error)
	IsFirstTimeUser(pubkey []byte) (bool, error)

	// Paid ACL (Lightning payment-gated access)
	SavePaidSubscription(sub *PaidSubscription) error
	GetPaidSubscription(pubkeyHex string) (*PaidSubscription, error)
	DeletePaidSubscription(pubkeyHex string) error
	ListPaidSubscriptions() ([]*PaidSubscription, error)
	ClaimAlias(alias, pubkeyHex string) error
	GetAliasByPubkey(pubkeyHex string) (string, error)
	GetAliasesByPubkey(pubkeyHex string) ([]string, error)
	GetPubkeyByAlias(alias string) (string, error)
	IsAliasTaken(alias string) (bool, error)

	// NIP-43 Invite-based ACL
	AddNIP43Member(pubkey []byte, inviteCode string) error
	RemoveNIP43Member(pubkey []byte) error
	IsNIP43Member(pubkey []byte) (isMember bool, err error)
	GetNIP43Membership(pubkey []byte) (*NIP43Membership, error)
	GetAllNIP43Members() ([][]byte, error)
	StoreInviteCode(code string, expiresAt time.Time) error
	ValidateInviteCode(code string) (valid bool, err error)
	DeleteInviteCode(code string) error
	PublishNIP43MembershipEvent(kind int, pubkey []byte) error

	// Migrations (version tracking for schema updates)
	RunMigrations()

	// Query cache methods
	GetCachedJSON(f *filter.F) ([][]byte, bool)
	CacheMarshaledJSON(f *filter.F, marshaledJSON [][]byte)
	GetCachedEvents(f *filter.F) (event.S, bool)
	CacheEvents(f *filter.F, events event.S)
	InvalidateQueryCache()

	// Access tracking for storage management (garbage collection based on access patterns)
	// RecordEventAccess records an access to an event by a connection.
	// The connectionID is used to deduplicate accesses from the same connection.
	RecordEventAccess(serial uint64, connectionID string) error
	// GetEventAccessInfo returns the last access time and access count for an event.
	GetEventAccessInfo(serial uint64) (lastAccess int64, accessCount uint32, err error)
	// GetLeastAccessedEvents returns event serials sorted by coldness (oldest/lowest access).
	// limit: max events to return, minAgeSec: minimum age in seconds since last access.
	GetLeastAccessedEvents(limit int, minAgeSec int64) (serials []uint64, err error)

	// Utility methods
	EventIdsBySerial(start uint64, count int) (evs []uint64, err error)

	// Blob storage (Blossom)
	SaveBlob(sha256Hash []byte, data []byte, pubkey []byte, mimeType string, extension string) error
	SaveBlobMetadata(sha256Hash []byte, size int64, pubkey []byte, mimeType string, extension string) error
	GetBlob(sha256Hash []byte) (data []byte, metadata *BlobMetadata, err error)
	HasBlob(sha256Hash []byte) (exists bool, err error)
	DeleteBlob(sha256Hash []byte, pubkey []byte) error
	ListBlobs(pubkey []byte, since, until int64) ([]*BlobDescriptor, error)
	ListAllBlobs() ([]*BlobDescriptor, error)
	GetBlobMetadata(sha256Hash []byte) (*BlobMetadata, error)
	GetTotalBlobStorageUsed(pubkey []byte) (totalMB int64, err error)
	SaveBlobReport(sha256Hash []byte, reportData []byte) error
	ListAllBlobUserStats() ([]*UserBlobStats, error)
	ReconcileBlobMetadata() (reconciled int, err error)

	// Thumbnail caching
	GetThumbnail(key string) (data []byte, err error)
	SaveThumbnail(key string, data []byte) error

	// NRC (Nostr Relay Connect) client management
	CreateNRCConnection(label string, createdBy []byte) (*NRCConnection, error)
	GetNRCConnection(id string) (*NRCConnection, error)
	GetNRCConnectionByDerivedPubkey(derivedPubkey []byte) (*NRCConnection, error)
	SaveNRCConnection(conn *NRCConnection) error
	DeleteNRCConnection(id string) error
	GetAllNRCConnections() ([]*NRCConnection, error)
	GetNRCAuthorizedSecrets() (map[string]string, error)
	UpdateNRCConnectionLastUsed(id string) error
	GetNRCConnectionURI(conn *NRCConnection, relayPubkey []byte, rendezvousURL string) (string, error)
}

Database defines the interface that all database implementations must satisfy. This allows switching between different storage backends (badger, neo4j, etc.)

func NewDatabase

func NewDatabase(
	ctx context.Context,
	cancel context.CancelFunc,
	dbType string,
	dataDir string,
	logLevel string,
) (Database, error)

NewDatabase creates a database instance based on the specified type. Supported types: "badger", "neo4j"

func NewDatabaseWithConfig

func NewDatabaseWithConfig(
	ctx context.Context,
	cancel context.CancelFunc,
	dbType string,
	cfg *DatabaseConfig,
) (Database, error)

NewDatabaseWithConfig creates a database instance with full configuration. This is the preferred method when you have access to the app config.

func NewFromDriver

func NewFromDriver(ctx context.Context, cancel context.CancelFunc, driverName string, cfg *DatabaseConfig) (Database, error)

NewFromDriver creates a database using the named driver. Returns an error if the driver is not registered.

type DatabaseConfig

type DatabaseConfig struct {
	// Common settings for all backends
	DataDir  string
	LogLevel string

	// Badger-specific settings
	BlockCacheMB       int           // ORLY_DB_BLOCK_CACHE_MB
	IndexCacheMB       int           // ORLY_DB_INDEX_CACHE_MB
	QueryCacheDisabled bool          // ORLY_QUERY_CACHE_DISABLED - disable query cache to reduce memory usage
	QueryCacheSizeMB   int           // ORLY_QUERY_CACHE_SIZE_MB
	QueryCacheMaxAge   time.Duration // ORLY_QUERY_CACHE_MAX_AGE

	// Serial cache settings for compact event storage
	SerialCachePubkeys  int // ORLY_SERIAL_CACHE_PUBKEYS - max pubkeys to cache (default: 100000)
	SerialCacheEventIds int // ORLY_SERIAL_CACHE_EVENT_IDS - max event IDs to cache (default: 500000)

	// Compression settings
	ZSTDLevel int // ORLY_DB_ZSTD_LEVEL - ZSTD compression level (0=none, 1=fast, 3=default, 9=best)

	// Neo4j-specific settings
	Neo4jURI      string // ORLY_NEO4J_URI
	Neo4jUser     string // ORLY_NEO4J_USER
	Neo4jPassword string // ORLY_NEO4J_PASSWORD

	// Neo4j driver tuning (memory and connection management)
	Neo4jMaxConnPoolSize   int // ORLY_NEO4J_MAX_CONN_POOL - max connection pool size (default: 25)
	Neo4jFetchSize         int // ORLY_NEO4J_FETCH_SIZE - max records per fetch batch (default: 1000)
	Neo4jMaxTxRetrySeconds int // ORLY_NEO4J_MAX_TX_RETRY_SEC - max transaction retry time (default: 30)
	Neo4jQueryResultLimit  int // ORLY_NEO4J_QUERY_RESULT_LIMIT - max results per query (default: 10000, 0=unlimited)

	// gRPC client settings (for remote database)
	GRPCServerAddress  string        // ORLY_GRPC_SERVER - address of remote gRPC database server
	GRPCConnectTimeout time.Duration // ORLY_GRPC_CONNECT_TIMEOUT - connection timeout (default: 10s)
}

DatabaseConfig holds all database configuration options that can be passed to any database backend. Each backend uses the relevant fields for its type. This centralizes configuration instead of having each backend read env vars directly.

type DatabaseSerialResolver

type DatabaseSerialResolver struct {
	// contains filtered or unexported fields
}

DatabaseSerialResolver implements SerialResolver using the database and cache.

func NewDatabaseSerialResolver

func NewDatabaseSerialResolver(db *D, cache *SerialCache) *DatabaseSerialResolver

NewDatabaseSerialResolver creates a new resolver.

func (*DatabaseSerialResolver) GetEventIdBySerial

func (r *DatabaseSerialResolver) GetEventIdBySerial(serial uint64) (eventId []byte, err error)

GetEventIdBySerial implements SerialResolver.

func (*DatabaseSerialResolver) GetEventSerialById

func (r *DatabaseSerialResolver) GetEventSerialById(eventId []byte) (serial uint64, found bool, err error)

GetEventSerialById implements SerialResolver.

func (*DatabaseSerialResolver) GetOrCreatePubkeySerial

func (r *DatabaseSerialResolver) GetOrCreatePubkeySerial(pubkey []byte) (serial uint64, err error)

GetOrCreatePubkeySerial implements SerialResolver.

func (*DatabaseSerialResolver) GetPubkeyBySerial

func (r *DatabaseSerialResolver) GetPubkeyBySerial(serial uint64) (pubkey []byte, err error)

GetPubkeyBySerial implements SerialResolver.

type DriverFactory

type DriverFactory func(ctx context.Context, cancel context.CancelFunc, cfg *DatabaseConfig) (Database, error)

DriverFactory is the signature for database driver factory functions.

func GetDriver

func GetDriver(name string) DriverFactory

GetDriver returns the factory for the named driver, or nil if not found.

type DriverInfo

type DriverInfo struct {
	Name        string
	Description string
	Factory     DriverFactory
}

DriverInfo contains metadata about a registered driver.

func ListDriversWithInfo

func ListDriversWithInfo() []*DriverInfo

ListDriversWithInfo returns information about all registered drivers.

type EventNeedingModeration

type EventNeedingModeration struct {
	ID     string    `json:"id"`
	Reason string    `json:"reason,omitempty"`
	Added  time.Time `json:"added"`
}

EventNeedingModeration represents an event that needs moderation

type EventSummary

type EventSummary struct {
	ID        string `json:"id"`
	Kind      int    `json:"kind"`
	Content   string `json:"content"`
	CreatedAt int64  `json:"created_at"`
}

EventSummary represents a simplified event for display in the UI

type GrapeVineStore

type GrapeVineStore struct {
	*D
}

GrapeVineStore provides database operations for GrapeVine WoT scores. It stores raw JSON blobs without importing the grapevine package to avoid circular deps.

func NewGrapeVineStore

func NewGrapeVineStore(db *D) *GrapeVineStore

NewGrapeVineStore creates a new GrapeVineStore instance.

func (*GrapeVineStore) DeleteScoreSet

func (g *GrapeVineStore) DeleteScoreSet(observerHex string) error

DeleteScoreSet removes all stored scores for an observer. Uses WriteBatch to avoid exceeding Badger's transaction size limit on large graphs.

func (*GrapeVineStore) GetScore

func (g *GrapeVineStore) GetScore(observerHex, targetHex string) ([]byte, error)

GetScore returns the raw JSON for a single score entry, or nil if not found.

func (*GrapeVineStore) GetScoreSet

func (g *GrapeVineStore) GetScoreSet(observerHex string) ([]byte, error)

GetScoreSet returns the raw JSON for a full score set, or nil if not found.

func (*GrapeVineStore) SaveScoreSet

func (g *GrapeVineStore) SaveScoreSet(observerHex string, setData []byte, entries map[string][]byte) error

SaveScoreSet persists a complete score set for an observer. setData is the JSON-marshaled full score set. entries maps target pubkey hex to JSON-marshaled individual score entries. Uses WriteBatch to avoid exceeding Badger's transaction size limit on large graphs.

type GraphAdapter

type GraphAdapter struct {
	// contains filtered or unexported fields
}

GraphAdapter wraps a database instance and implements graph.GraphDatabase interface.

func NewGraphAdapter

func NewGraphAdapter(db *D) *GraphAdapter

NewGraphAdapter creates a new GraphAdapter wrapping the given database.

func (*GraphAdapter) TraverseEventEvent

func (a *GraphAdapter) TraverseEventEvent(seedEventID []byte, maxDepth int, direction string) (graph.GraphResultI, error)

TraverseEventEvent implements graph.GraphDatabase.

func (*GraphAdapter) TraversePubkeyEvent

func (a *GraphAdapter) TraversePubkeyEvent(seedPubkey []byte, maxDepth int, direction string) (graph.GraphResultI, error)

TraversePubkeyEvent implements graph.GraphDatabase.

func (*GraphAdapter) TraversePubkeyPubkey

func (a *GraphAdapter) TraversePubkeyPubkey(seedPubkey []byte, maxDepth int, direction string) (graph.GraphResultI, error)

TraversePubkeyPubkey implements graph.GraphDatabase. Uses the ppg/gpp materialized index for single-hop prefix scans.

func (*GraphAdapter) TraversePubkeyPubkeyBaseline

func (a *GraphAdapter) TraversePubkeyPubkeyBaseline(seedPubkey []byte, maxDepth int, direction string) (graph.GraphResultI, error)

TraversePubkeyPubkeyBaseline implements graph.GraphDatabase. Uses multi-hop NIP-01 queries (no ppg/gpp index) for benchmark comparison.

type GraphResult

type GraphResult struct {
	// PubkeysByDepth maps depth -> pubkeys first discovered at that depth.
	// Each pubkey appears ONLY in the array for the depth where it was first seen.
	// Depth 1 = direct connections, Depth 2 = connections of connections, etc.
	PubkeysByDepth map[int][]string

	// EventsByDepth maps depth -> event IDs discovered at that depth.
	// Used for thread traversal queries.
	EventsByDepth map[int][]string

	// FirstSeenPubkey tracks which depth each pubkey was first discovered.
	// Key is pubkey hex, value is the depth (1-indexed).
	FirstSeenPubkey map[string]int

	// FirstSeenEvent tracks which depth each event was first discovered.
	// Key is event ID hex, value is the depth (1-indexed).
	FirstSeenEvent map[string]int

	// TotalPubkeys is the count of unique pubkeys discovered across all depths.
	TotalPubkeys int

	// TotalEvents is the count of unique events discovered across all depths.
	TotalEvents int

	// InboundRefs tracks inbound references (events that reference discovered items).
	// Structure: kind -> target_id -> []referencing_event_ids
	InboundRefs map[uint16]map[string][]string

	// OutboundRefs tracks outbound references (events referenced by discovered items).
	// Structure: kind -> source_id -> []referenced_event_ids
	OutboundRefs map[uint16]map[string][]string
}

GraphResult contains depth-organized traversal results for graph queries. It tracks pubkeys and events discovered at each depth level, ensuring each entity appears only at the depth where it was first discovered.

func NewGraphResult

func NewGraphResult() *GraphResult

NewGraphResult creates a new initialized GraphResult.

func (*GraphResult) AddEventAtDepth

func (r *GraphResult) AddEventAtDepth(eventIDHex string, depth int) bool

AddEventAtDepth adds an event ID to the result at the specified depth if not already seen. Returns true if the event was added (first time seen), false if already exists.

func (*GraphResult) AddInboundRef

func (r *GraphResult) AddInboundRef(kind uint16, targetIDHex string, referencingEventIDHex string)

AddInboundRef records an inbound reference from a referencing event to a target.

func (*GraphResult) AddOutboundRef

func (r *GraphResult) AddOutboundRef(kind uint16, sourceIDHex string, referencedEventIDHex string)

AddOutboundRef records an outbound reference from a source event to a referenced event.

func (*GraphResult) AddPubkeyAtDepth

func (r *GraphResult) AddPubkeyAtDepth(pubkeyHex string, depth int) bool

AddPubkeyAtDepth adds a pubkey to the result at the specified depth if not already seen. Returns true if the pubkey was added (first time seen), false if already exists.

func (*GraphResult) GetAllEvents

func (r *GraphResult) GetAllEvents() []string

GetAllEvents returns all event IDs discovered across all depths.

func (*GraphResult) GetAllPubkeys

func (r *GraphResult) GetAllPubkeys() []string

GetAllPubkeys returns all pubkeys discovered across all depths.

func (*GraphResult) GetDepthsSorted

func (r *GraphResult) GetDepthsSorted() []int

GetDepthsSorted returns all depths that have pubkeys, sorted ascending.

func (*GraphResult) GetEventDepth

func (r *GraphResult) GetEventDepth(eventIDHex string) int

GetEventDepth returns the depth at which an event was first discovered. Returns 0 if the event was not found.

func (*GraphResult) GetEventDepthsSorted

func (r *GraphResult) GetEventDepthsSorted() []int

GetEventDepthsSorted returns all depths that have events, sorted ascending.

func (*GraphResult) GetEventsAtDepth

func (r *GraphResult) GetEventsAtDepth(depth int) []string

GetEventsAtDepth returns events at a specific depth, or empty slice if none.

func (*GraphResult) GetEventsByDepth

func (r *GraphResult) GetEventsByDepth() map[int][]string

GetEventsByDepth returns the EventsByDepth map for external access.

func (*GraphResult) GetInboundRefs

func (r *GraphResult) GetInboundRefs() map[uint16]map[string][]string

GetInboundRefs returns the InboundRefs map for external access.

func (*GraphResult) GetInboundRefsSorted

func (r *GraphResult) GetInboundRefsSorted(kind uint16) []RefAggregation

GetInboundRefsSorted returns inbound refs for a kind, sorted by count descending.

func (*GraphResult) GetOutboundRefs

func (r *GraphResult) GetOutboundRefs() map[uint16]map[string][]string

GetOutboundRefs returns the OutboundRefs map for external access.

func (*GraphResult) GetOutboundRefsSorted

func (r *GraphResult) GetOutboundRefsSorted(kind uint16) []RefAggregation

GetOutboundRefsSorted returns outbound refs for a kind, sorted by count descending.

func (*GraphResult) GetPubkeyDepth

func (r *GraphResult) GetPubkeyDepth(pubkeyHex string) int

GetPubkeyDepth returns the depth at which a pubkey was first discovered. Returns 0 if the pubkey was not found.

func (*GraphResult) GetPubkeysAtDepth

func (r *GraphResult) GetPubkeysAtDepth(depth int) []string

GetPubkeysAtDepth returns pubkeys at a specific depth, or empty slice if none.

func (*GraphResult) GetPubkeysByDepth

func (r *GraphResult) GetPubkeysByDepth() map[int][]string

GetPubkeysByDepth returns the PubkeysByDepth map for external access.

func (*GraphResult) GetTotalEvents

func (r *GraphResult) GetTotalEvents() int

GetTotalEvents returns the total event count for external access.

func (*GraphResult) GetTotalPubkeys

func (r *GraphResult) GetTotalPubkeys() int

GetTotalPubkeys returns the total pubkey count for external access.

func (*GraphResult) HasEvent

func (r *GraphResult) HasEvent(eventIDHex string) bool

HasEvent returns true if the event has been discovered at any depth.

func (*GraphResult) HasPubkey

func (r *GraphResult) HasPubkey(pubkeyHex string) bool

HasPubkey returns true if the pubkey has been discovered at any depth.

func (*GraphResult) ToDepthArrays

func (r *GraphResult) ToDepthArrays() [][]string

ToDepthArrays converts the result to the response format: array of arrays. Index 0 = depth 1 pubkeys, Index 1 = depth 2 pubkeys, etc. Empty arrays are included for depths with no pubkeys to maintain index alignment.

func (*GraphResult) ToEventDepthArrays

func (r *GraphResult) ToEventDepthArrays() [][]string

ToEventDepthArrays converts event results to the response format: array of arrays. Index 0 = depth 1 events, Index 1 = depth 2 events, etc.

type HealthReport

type HealthReport struct {
	// Scan metadata
	ScanStarted  time.Time
	ScanDuration time.Duration

	// Event counts
	CompactEvents int64 // Events stored in compact format (cmp)
	LegacyEvents  int64 // Events in legacy format (evt)
	SmallEvents   int64 // Small inline events (sev)
	TotalEvents   int64 // Total events
	SerialIdCount int64 // Serial to EventID mappings (sei)

	// Pubkey serial counts
	PubkeySerials int64 // pks entries (pubkey hash -> serial)
	SerialPubkeys int64 // spk entries (serial -> pubkey)

	// Graph edge counts
	EventPubkeyEdges int64 // epg entries
	PubkeyEventEdges int64 // peg entries
	EventEventEdges  int64 // eeg entries
	GraphEventEdges  int64 // gee entries

	// Index counts
	KindIndexes   int64 // kc- entries
	PubkeyIndexes int64 // pc- entries
	TagIndexes    int64 // tc- entries
	WordIndexes   int64 // wrd entries
	IdIndexes     int64 // eid entries

	// Issues found
	MissingSerialEventIds  int64 // cmp entries without corresponding sei
	OrphanedSerialEventIds int64 // sei entries without corresponding cmp
	PubkeySerialMismatches int64 // pks without matching spk or vice versa
	OrphanedIndexes        int64 // Index entries pointing to non-existent events

	// Sample of missing sei serials (for debugging)
	MissingSeiSamples []uint64

	// Health score (0-100)
	HealthScore int
}

HealthReport contains the results of a database health check.

func (*HealthReport) String

func (r *HealthReport) String() string

String returns a human-readable health report.

type IPEventCount

type IPEventCount struct {
	IP        string    `json:"ip"`
	Date      string    `json:"date"`
	Count     int       `json:"count"`
	LastEvent time.Time `json:"last_event"`
}

IPEventCount tracks events from an IP address per day (flood protection)

type IPOffense

type IPOffense struct {
	IP           string    `json:"ip"`
	OffenseCount int       `json:"offense_count"`
	PubkeysHit   []string  `json:"pubkeys_hit"` // Pubkeys that hit rate limit from this IP
	LastOffense  time.Time `json:"last_offense"`
}

IPOffense tracks rate limit violations from IPs

type LRUCache

type LRUCache[K comparable, V any] struct {
	actor.Lifecycle
	// contains filtered or unexported fields
}

LRUCache provides a thread-safe LRU cache with configurable max size. All mutable state is owned by the actor goroutine.

func NewLRUCache

func NewLRUCache[K comparable, V any](maxSize int) *LRUCache[K, V]

NewLRUCache creates a new LRU cache with the given maximum size.

func (*LRUCache[K, V]) Clear

func (c *LRUCache[K, V]) Clear()

Clear removes all entries from the cache.

func (*LRUCache[K, V]) Contains

func (c *LRUCache[K, V]) Contains(key K) bool

Contains returns true if the key exists in the cache without updating LRU order.

func (*LRUCache[K, V]) Delete

func (c *LRUCache[K, V]) Delete(key K)

Delete removes an entry from the cache.

func (*LRUCache[K, V]) Get

func (c *LRUCache[K, V]) Get(key K) (value V, found bool)

Get retrieves a value by key and marks it as recently used.

func (*LRUCache[K, V]) Len

func (c *LRUCache[K, V]) Len() int

Len returns the current number of entries in the cache.

func (*LRUCache[K, V]) MaxSize

func (c *LRUCache[K, V]) MaxSize() int

MaxSize returns the maximum capacity of the cache.

func (*LRUCache[K, V]) Put

func (c *LRUCache[K, V]) Put(key K, value V)

Put adds or updates a value, evicting the LRU entry if at capacity.

func (*LRUCache[K, V]) Shutdown

func (c *LRUCache[K, V]) Shutdown()

Shutdown stops the actor goroutine.

type ManagedACL

type ManagedACL struct {
	*D
}

ManagedACL database operations

func NewManagedACL

func NewManagedACL(db *D) *ManagedACL

NewManagedACL creates a new ManagedACL instance

func (*ManagedACL) GetRelayConfig

func (m *ManagedACL) GetRelayConfig() (ManagedACLConfig, error)

GetRelayConfig returns relay configuration

func (*ManagedACL) IsEventAllowed

func (m *ManagedACL) IsEventAllowed(eventID string) (bool, error)

Check if an event is explicitly allowed

func (*ManagedACL) IsEventBanned

func (m *ManagedACL) IsEventBanned(eventID string) (bool, error)

Check if an event is banned

func (*ManagedACL) IsIPBlocked

func (m *ManagedACL) IsIPBlocked(ip string) (bool, error)

Check if an IP is blocked

func (*ManagedACL) IsKindAllowed

func (m *ManagedACL) IsKindAllowed(kind int) (bool, error)

Check if a kind is allowed

func (*ManagedACL) IsPubkeyAllowed

func (m *ManagedACL) IsPubkeyAllowed(pubkey string) (bool, error)

Check if a pubkey is explicitly allowed

func (*ManagedACL) IsPubkeyBanned

func (m *ManagedACL) IsPubkeyBanned(pubkey string) (bool, error)

Check if a pubkey is banned

func (*ManagedACL) ListAllowedEvents

func (m *ManagedACL) ListAllowedEvents() ([]AllowedEvent, error)

ListAllowedEvents returns all allowed events

func (*ManagedACL) ListAllowedKinds

func (m *ManagedACL) ListAllowedKinds() ([]int, error)

ListAllowedKinds returns all allowed kinds

func (*ManagedACL) ListAllowedPubkeys

func (m *ManagedACL) ListAllowedPubkeys() ([]AllowedPubkey, error)

ListAllowedPubkeys returns all allowed pubkeys

func (*ManagedACL) ListBannedEvents

func (m *ManagedACL) ListBannedEvents() ([]BannedEvent, error)

ListBannedEvents returns all banned events

func (*ManagedACL) ListBannedPubkeys

func (m *ManagedACL) ListBannedPubkeys() ([]BannedPubkey, error)

ListBannedPubkeys returns all banned pubkeys

func (*ManagedACL) ListBlockedIPs

func (m *ManagedACL) ListBlockedIPs() ([]BlockedIP, error)

ListBlockedIPs returns all blocked IPs

func (*ManagedACL) ListEventsNeedingModeration

func (m *ManagedACL) ListEventsNeedingModeration() ([]EventNeedingModeration, error)

ListEventsNeedingModeration returns all events needing moderation

func (*ManagedACL) RemoveAllowedEvent

func (m *ManagedACL) RemoveAllowedEvent(eventID string) error

RemoveAllowedEvent removes an allowed event from the database

func (*ManagedACL) RemoveAllowedKind

func (m *ManagedACL) RemoveAllowedKind(kind int) error

RemoveAllowedKind removes an allowed kind from the database

func (*ManagedACL) RemoveAllowedPubkey

func (m *ManagedACL) RemoveAllowedPubkey(pubkey string) error

RemoveAllowedPubkey removes an allowed pubkey from the database

func (*ManagedACL) RemoveBannedEvent

func (m *ManagedACL) RemoveBannedEvent(eventID string) error

RemoveBannedEvent removes a banned event from the database

func (*ManagedACL) RemoveBannedPubkey

func (m *ManagedACL) RemoveBannedPubkey(pubkey string) error

RemoveBannedPubkey removes a banned pubkey from the database

func (*ManagedACL) RemoveBlockedIP

func (m *ManagedACL) RemoveBlockedIP(ip string) error

RemoveBlockedIP removes a blocked IP from the database

func (*ManagedACL) RemoveEventNeedingModeration

func (m *ManagedACL) RemoveEventNeedingModeration(eventID string) error

RemoveEventNeedingModeration removes an event from moderation queue

func (*ManagedACL) SaveAllowedEvent

func (m *ManagedACL) SaveAllowedEvent(eventID string, reason string) error

SaveAllowedEvent saves an allowed event to the database

func (*ManagedACL) SaveAllowedKind

func (m *ManagedACL) SaveAllowedKind(kind int) error

SaveAllowedKind saves an allowed kind to the database

func (*ManagedACL) SaveAllowedPubkey

func (m *ManagedACL) SaveAllowedPubkey(pubkey string, reason string) error

SaveAllowedPubkey saves an allowed pubkey to the database

func (*ManagedACL) SaveBannedEvent

func (m *ManagedACL) SaveBannedEvent(eventID string, reason string) error

SaveBannedEvent saves a banned event to the database

func (*ManagedACL) SaveBannedPubkey

func (m *ManagedACL) SaveBannedPubkey(pubkey string, reason string) error

SaveBannedPubkey saves a banned pubkey to the database

func (*ManagedACL) SaveBlockedIP

func (m *ManagedACL) SaveBlockedIP(ip string, reason string) error

SaveBlockedIP saves a blocked IP to the database

func (*ManagedACL) SaveEventNeedingModeration

func (m *ManagedACL) SaveEventNeedingModeration(eventID string, reason string) error

SaveEventNeedingModeration saves an event that needs moderation

func (*ManagedACL) SaveRelayConfig

func (m *ManagedACL) SaveRelayConfig(config ManagedACLConfig) error

SaveRelayConfig saves relay configuration

type ManagedACLConfig

type ManagedACLConfig struct {
	RelayName        string `json:"relay_name"`
	RelayDescription string `json:"relay_description"`
	RelayIcon        string `json:"relay_icon"`
}

ManagedACLConfig represents the configuration for managed ACL mode

type NIP43Membership

type NIP43Membership struct {
	Pubkey     []byte
	AddedAt    time.Time
	InviteCode string
}

NIP43Membership represents membership metadata for NIP-43

type NRCAuthorizer

type NRCAuthorizer struct {
	// contains filtered or unexported fields
}

NRCAuthorizer wraps D to implement the NRC authorization interface. This allows the NRC bridge to look up authorized clients from the database.

func NewNRCAuthorizer

func NewNRCAuthorizer(db *D) *NRCAuthorizer

NewNRCAuthorizer creates a new NRC authorizer from a database.

func (*NRCAuthorizer) GetNRCClientByPubkey

func (a *NRCAuthorizer) GetNRCClientByPubkey(derivedPubkey []byte) (id string, label string, found bool, err error)

GetNRCClientByPubkey looks up an authorized client by their derived pubkey. Returns the client ID, label, and whether the client was found.

func (*NRCAuthorizer) UpdateNRCClientLastUsed

func (a *NRCAuthorizer) UpdateNRCClientLastUsed(id string) error

UpdateNRCClientLastUsed updates the last used timestamp for tracking.

type NRCConnection

type NRCConnection struct {
	ID            string `json:"id"`             // Unique identifier (hex of first 8 bytes of secret)
	Label         string `json:"label"`          // Human-readable label (e.g., "Phone", "Laptop")
	Secret        []byte `json:"secret"`         // 32-byte secret for client authentication
	DerivedPubkey []byte `json:"derived_pubkey"` // Pubkey derived from secret (for efficient lookups)
	RendezvousURL string `json:"rendezvous_url"` // WebSocket URL of the rendezvous relay
	CreatedAt     int64  `json:"created_at"`     // Unix timestamp
	LastUsed      int64  `json:"last_used"`      // Unix timestamp of last connection (0 if never)
	CreatedBy     []byte `json:"created_by"`     // Pubkey of admin who created this connection
}

NRCConnection stores an NRC connection configuration in the database.

type NRCEventAuthorizer

type NRCEventAuthorizer struct {
	// contains filtered or unexported fields
}

NRCEventAuthorizer wraps NRCEventStore to implement the NRC authorization interface.

func NewNRCEventAuthorizer

func NewNRCEventAuthorizer(store *NRCEventStore) *NRCEventAuthorizer

NewNRCEventAuthorizer creates a new NRC authorizer from an event store.

func (*NRCEventAuthorizer) GetNRCClientByPubkey

func (a *NRCEventAuthorizer) GetNRCClientByPubkey(derivedPubkey []byte) (id string, label string, found bool, err error)

GetNRCClientByPubkey looks up an authorized client by their derived pubkey.

func (*NRCEventAuthorizer) UpdateNRCClientLastUsed

func (a *NRCEventAuthorizer) UpdateNRCClientLastUsed(id string) error

UpdateNRCClientLastUsed updates the last used timestamp for tracking.

type NRCEventStore

type NRCEventStore struct {
	// contains filtered or unexported fields
}

NRCEventStore provides NRC connection management using events. This works with any Database implementation that supports SaveEvent/QueryEvents.

func NewNRCEventStore

func NewNRCEventStore(db Database, relaySigner *p8k.Signer) *NRCEventStore

NewNRCEventStore creates a new event-based NRC store. relaySigner is used to sign NRC connection events.

func (*NRCEventStore) CreateNRCConnection

func (s *NRCEventStore) CreateNRCConnection(label string, rendezvousURL string, createdBy []byte) (*NRCConnection, error)

CreateNRCConnection generates a new NRC connection with a random secret.

func (*NRCEventStore) DeleteNRCConnection

func (s *NRCEventStore) DeleteNRCConnection(id string) error

DeleteNRCConnection removes an NRC connection by deleting its event.

func (*NRCEventStore) GetAllNRCConnections

func (s *NRCEventStore) GetAllNRCConnections() ([]*NRCConnection, error)

GetAllNRCConnections returns all NRC connections.

func (*NRCEventStore) GetNRCAuthorizedSecrets

func (s *NRCEventStore) GetNRCAuthorizedSecrets() (map[string]string, error)

GetNRCAuthorizedSecrets returns a map of derived pubkeys to labels.

func (*NRCEventStore) GetNRCConnection

func (s *NRCEventStore) GetNRCConnection(id string) (*NRCConnection, error)

GetNRCConnection retrieves an NRC connection by ID.

func (*NRCEventStore) GetNRCConnectionByDerivedPubkey

func (s *NRCEventStore) GetNRCConnectionByDerivedPubkey(derivedPubkey []byte) (*NRCConnection, error)

GetNRCConnectionByDerivedPubkey retrieves an NRC connection by its derived pubkey.

func (*NRCEventStore) GetNRCConnectionURI

func (s *NRCEventStore) GetNRCConnectionURI(conn *NRCConnection, relayPubkey []byte) (string, error)

GetNRCConnectionURI generates the full connection URI for a connection. Uses the rendezvous URL stored in the connection.

func (*NRCEventStore) SaveNRCConnection

func (s *NRCEventStore) SaveNRCConnection(conn *NRCConnection) error

SaveNRCConnection stores an NRC connection as an event.

func (*NRCEventStore) UpdateNRCConnectionLastUsed

func (s *NRCEventStore) UpdateNRCConnectionLastUsed(id string) error

UpdateNRCConnectionLastUsed updates the last used timestamp.

type PaidACL

type PaidACL struct {
	*D
}

PaidACL provides database operations for paid ACL data.

func NewPaidACL

func NewPaidACL(db *D) *PaidACL

NewPaidACL creates a new PaidACL instance.

func (*PaidACL) ClaimAlias

func (p *PaidACL) ClaimAlias(alias, pubkeyHex string) error

ClaimAlias atomically claims an alias for a pubkey. Returns an error if the alias is already taken by another pubkey. One pubkey can have multiple aliases — each claim appends to the list.

func (*PaidACL) DeleteSubscription

func (p *PaidACL) DeleteSubscription(pubkeyHex string) error

DeleteSubscription removes a subscription.

func (*PaidACL) GetAliasByPubkey

func (p *PaidACL) GetAliasByPubkey(pubkeyHex string) (string, error)

GetAliasByPubkey returns the first alias for a pubkey, or "" if none.

func (*PaidACL) GetAliasesByPubkey

func (p *PaidACL) GetAliasesByPubkey(pubkeyHex string) ([]string, error)

GetAliasesByPubkey returns all aliases for a pubkey.

func (*PaidACL) GetPubkeyByAlias

func (p *PaidACL) GetPubkeyByAlias(alias string) (string, error)

GetPubkeyByAlias returns the pubkey for an alias, or "" if not found.

func (*PaidACL) GetSubscription

func (p *PaidACL) GetSubscription(pubkeyHex string) (*PaidSubscription, error)

GetSubscription returns the subscription for a pubkey.

func (*PaidACL) IsAliasTaken

func (p *PaidACL) IsAliasTaken(alias string) (bool, error)

IsAliasTaken returns true if the alias is claimed by any pubkey.

func (*PaidACL) ListSubscriptions

func (p *PaidACL) ListSubscriptions() ([]*PaidSubscription, error)

ListSubscriptions returns all subscriptions.

func (*PaidACL) SaveSubscription

func (p *PaidACL) SaveSubscription(sub *PaidSubscription) error

SaveSubscription saves or updates a subscription.

type PaidSubscription

type PaidSubscription struct {
	PubkeyHex   string    `json:"pubkey"`
	Alias       string    `json:"alias,omitempty"`
	ExpiresAt   time.Time `json:"expires_at"`
	CreatedAt   time.Time `json:"created_at"`
	InvoiceHash string    `json:"invoice_hash,omitempty"`
}

PaidSubscription represents an active paid subscription.

func (*PaidSubscription) IsActive

func (s *PaidSubscription) IsActive() bool

IsActive returns true if the subscription has not expired.

type Payment

type Payment struct {
	Amount    int64     `json:"amount"`
	Timestamp time.Time `json:"timestamp"`
	Invoice   string    `json:"invoice"`
	Preimage  string    `json:"preimage"`
}

Payment represents a recorded payment

type PubkeyEventCount

type PubkeyEventCount struct {
	Pubkey    string    `json:"pubkey"`
	Date      string    `json:"date"` // YYYY-MM-DD format
	Count     int       `json:"count"`
	LastEvent time.Time `json:"last_event"`
}

PubkeyEventCount tracks daily event counts for rate limiting

type Range

type Range struct {
	Start, End []byte
}

func GetIndexesFromFilter

func GetIndexesFromFilter(f *filter.F) (idxs []Range, err error)

GetIndexesFromFilter returns encoded indexes based on the given filter.

An error is returned if any input values are invalid during encoding.

The indexes are designed so that only one table needs to be iterated, being a complete set of combinations of all fields in the event, thus there is no need to decode events until they are to be delivered.

type RateLimiterInterface

type RateLimiterInterface interface {
	IsEnabled() bool
	Wait(ctx context.Context, opType int) (time.Duration, error)
}

RateLimiterInterface defines the minimal interface for rate limiting during import

type RefAggregation

type RefAggregation struct {
	// TargetEventID is the event ID being referenced (for inbound) or referencing (for outbound)
	TargetEventID string

	// TargetAuthor is the author pubkey of the target event (if known)
	TargetAuthor string

	// TargetDepth is the depth at which this target was discovered in the graph
	TargetDepth int

	// RefKind is the kind of the referencing events
	RefKind uint16

	// RefCount is the number of references to/from this target
	RefCount int

	// RefEventIDs is the list of event IDs that reference this target
	RefEventIDs []string
}

RefAggregation represents aggregated reference data for a single target/source.

type RepairOptions

type RepairOptions struct {
	// DryRun if true, only reports what would be fixed without making changes
	DryRun bool

	// FixMissingSei if true, rebuilds missing sei entries from compact events
	FixMissingSei bool

	// RemoveOrphanedSei if true, removes sei entries without corresponding events
	RemoveOrphanedSei bool

	// FixPubkeyMappings if true, fixes inconsistent pubkey serial mappings
	FixPubkeyMappings bool

	// Progress writer for progress updates
	Progress io.Writer
}

RepairOptions configures the repair operation.

func DefaultRepairOptions

func DefaultRepairOptions() *RepairOptions

DefaultRepairOptions returns the default repair options.

type RepairReport

type RepairReport struct {
	// Operation metadata
	Started  time.Time
	Duration time.Duration
	DryRun   bool

	// Events scanned
	CompactEventsScanned int64
	LegacyEventsScanned  int64

	// Repairs performed
	SeiEntriesCreated    int64 // sei mappings rebuilt from compact events
	SeiEntriesRemoved    int64 // orphaned sei entries removed
	PubkeyMappingsFixed  int64 // pubkey serial mappings fixed
	OrphanedIndexesFixed int64 // orphaned index entries removed

	// Errors encountered
	Errors []string
}

RepairReport contains the results of a database repair operation.

func (*RepairReport) String

func (r *RepairReport) String() string

String returns a human-readable repair report.

type ScanResult

type ScanResult struct {
	TotalPubkeys int `json:"total_pubkeys"`
	TotalEvents  int `json:"total_events"`
	Skipped      int `json:"skipped"` // Trusted/blacklisted users skipped
}

ScanResult contains the results of scanning all pubkeys in the database

type SerialCache

type SerialCache struct {
	// contains filtered or unexported fields
}

SerialCache provides LRU caching for pubkey and event ID serial lookups. This is critical for compact event decoding performance since every event requires looking up the author pubkey and potentially multiple tag references.

The cache uses LRU eviction and starts empty, growing on demand up to the configured limits. This provides better memory efficiency than pre-allocation and better hit rates than random eviction.

func NewSerialCache

func NewSerialCache(maxPubkeys, maxEventIds int) *SerialCache

NewSerialCache creates a new serial cache with the specified maximum sizes. The cache starts empty and grows on demand up to these limits.

func (*SerialCache) CacheEventId

func (c *SerialCache) CacheEventId(serial uint64, eventId []byte)

CacheEventId adds an event ID to the cache in both directions.

func (*SerialCache) CachePubkey

func (c *SerialCache) CachePubkey(serial uint64, pubkey []byte)

CachePubkey adds a pubkey to the cache in both directions.

func (*SerialCache) GetEventIdBySerial

func (c *SerialCache) GetEventIdBySerial(serial uint64) (eventId []byte, found bool)

GetEventIdBySerial returns the event ID for a serial from cache.

func (*SerialCache) GetPubkeyBySerial

func (c *SerialCache) GetPubkeyBySerial(serial uint64) (pubkey []byte, found bool)

GetPubkeyBySerial returns the pubkey for a serial from cache.

func (*SerialCache) GetSerialByEventId

func (c *SerialCache) GetSerialByEventId(eventId []byte) (serial uint64, found bool)

GetSerialByEventId returns the serial for an event ID from cache.

func (*SerialCache) GetSerialByPubkey

func (c *SerialCache) GetSerialByPubkey(pubkey []byte) (serial uint64, found bool)

GetSerialByPubkey returns the serial for a pubkey from cache.

func (*SerialCache) Stats

func (c *SerialCache) Stats() SerialCacheStats

Stats returns statistics about the serial cache.

type SerialCacheStats

type SerialCacheStats struct {
	PubkeysCached      int // Number of pubkeys currently cached
	PubkeysMaxSize     int // Maximum pubkey cache size
	EventIdsCached     int // Number of event IDs currently cached
	EventIdsMaxSize    int // Maximum event ID cache size
	PubkeyMemoryBytes  int // Estimated memory usage for pubkey cache
	EventIdMemoryBytes int // Estimated memory usage for event ID cache
	TotalMemoryBytes   int // Total estimated memory usage
}

SerialCacheStats holds statistics about the serial cache.

type SerialResolver

type SerialResolver interface {
	// GetOrCreatePubkeySerial returns the serial for a pubkey, creating one if needed.
	GetOrCreatePubkeySerial(pubkey []byte) (serial uint64, err error)

	// GetPubkeyBySerial returns the full pubkey for a serial.
	GetPubkeyBySerial(serial uint64) (pubkey []byte, err error)

	// GetEventSerialById returns the serial for an event ID, or 0 if not found.
	GetEventSerialById(eventId []byte) (serial uint64, found bool, err error)

	// GetEventIdBySerial returns the full event ID for a serial.
	GetEventIdBySerial(serial uint64) (eventId []byte, err error)
}

SerialResolver is an interface for resolving serials during compact encoding/decoding. This allows the encoder/decoder to look up or create serial mappings.

type SpamEvent

type SpamEvent struct {
	EventID string    `json:"event_id"`
	Pubkey  string    `json:"pubkey"`
	Reason  string    `json:"reason,omitempty"`
	Added   time.Time `json:"added"`
}

SpamEvent represents an event flagged as spam

type Subscription

type Subscription struct {
	TrialEnd       time.Time `json:"trial_end"`
	PaidUntil      time.Time `json:"paid_until"`
	BlossomLevel   string    `json:"blossom_level,omitempty"`   // Service level name (e.g., "basic", "premium")
	BlossomStorage int64     `json:"blossom_storage,omitempty"` // Storage quota in MB
}

Subscription represents a user's subscription status

type TrustedPubkey

type TrustedPubkey struct {
	Pubkey string    `json:"pubkey"`
	Note   string    `json:"note,omitempty"`
	Added  time.Time `json:"added"`
}

TrustedPubkey represents an explicitly trusted publisher

type UnclassifiedUser

type UnclassifiedUser struct {
	Pubkey     string    `json:"pubkey"`
	EventCount int       `json:"event_count"`
	LastEvent  time.Time `json:"last_event"`
}

UnclassifiedUser represents a user who hasn't been trusted or blacklisted

type UserBlobStats

type UserBlobStats struct {
	PubkeyHex      string
	BlobCount      int64
	TotalSizeBytes int64
}

UserBlobStats represents storage statistics for a single user

type WireGuardAccessLog

type WireGuardAccessLog struct {
	NostrPubkey []byte `json:"nostr_pubkey"`  // User's Nostr pubkey
	WGPublicKey []byte `json:"wg_public_key"` // The obsolete public key used
	Sequence    uint32 `json:"sequence"`      // Subnet sequence
	Timestamp   int64  `json:"timestamp"`     // When the access occurred
	RemoteAddr  string `json:"remote_addr"`   // Remote IP address
}

WireGuardAccessLog records an access attempt to an obsolete address.

type WireGuardPeer

type WireGuardPeer struct {
	NostrPubkey  []byte `json:"nostr_pubkey"`   // User's Nostr pubkey (32 bytes)
	WGPrivateKey []byte `json:"wg_private_key"` // WireGuard private key (32 bytes)
	WGPublicKey  []byte `json:"wg_public_key"`  // WireGuard public key (32 bytes)
	Sequence     uint32 `json:"sequence"`       // Sequence number for subnet derivation
	CreatedAt    int64  `json:"created_at"`     // Unix timestamp
}

WireGuardPeer stores WireGuard peer information in the database.

func (*WireGuardPeer) ClientIP

func (p *WireGuardPeer) ClientIP(pool *wireguard.SubnetPool) string

ClientIP returns the derived client IP for this peer's subnet.

func (*WireGuardPeer) ServerIP

func (p *WireGuardPeer) ServerIP(pool *wireguard.SubnetPool) string

ServerIP returns the derived server IP for this peer's subnet.

type WireGuardRevokedKey

type WireGuardRevokedKey struct {
	NostrPubkey  []byte `json:"nostr_pubkey"`   // User's Nostr pubkey (32 bytes)
	WGPublicKey  []byte `json:"wg_public_key"`  // Revoked WireGuard public key (32 bytes)
	Sequence     uint32 `json:"sequence"`       // Sequence number (subnet)
	CreatedAt    int64  `json:"created_at"`     // When the key was originally created
	RevokedAt    int64  `json:"revoked_at"`     // When the key was revoked
	AccessCount  int    `json:"access_count"`   // Number of access attempts since revocation
	LastAccessAt int64  `json:"last_access_at"` // Last access attempt timestamp (0 if never)
}

WireGuardRevokedKey stores a revoked/old WireGuard keypair for audit purposes.

type WordToken

type WordToken struct {
	Word string // normalized lowercase word (e.g., "bitcoin")
	Hash []byte // 8-byte truncated SHA-256
}

WordToken represents a normalized word with its truncated hash for indexing. Used by TokenWords() for NIP-50 word search across database backends.

func TokenWords

func TokenWords(content []byte) []WordToken

TokenWords extracts unique word tokens from content, returning both the normalized word text and its 8-byte truncated SHA-256 hash. Rules: - Unicode-aware: words are sequences of letters or numbers. - Lowercased using unicode case mapping. - Ignore URLs (starting with http://, https://, www., or containing "://"). - Ignore nostr: URIs and #[n] mentions. - Ignore words shorter than 2 runes. - Exclude 64-character hexadecimal strings (likely IDs/pubkeys).

Source Files

  • access_tracking.go
  • blob.go
  • cleanup-kind3.go
  • compact_event.go
  • compact_stats.go
  • count.go
  • curating-acl.go
  • database.go
  • delete-event.go
  • delete-expired.go
  • export.go
  • factory.go
  • fetch-event-by-serial.go
  • fetch-events-by-serials.go
  • filter_utils.go
  • get-fullidpubkey-by-serial.go
  • get-fullidpubkey-by-serials.go
  • get-indexes-for-event.go
  • get-indexes-from-filter.go
  • get-serial-by-id.go
  • get-serials-by-range.go
  • grapevine.go
  • graph-adapter.go
  • graph-follows.go
  • graph-mentions.go
  • graph-pp.go
  • graph-refs.go
  • graph-result.go
  • graph-thread.go
  • graph-traversal.go
  • health.go
  • identity.go
  • import.go
  • import_utils.go
  • interface.go
  • logger.go
  • lrucache.go
  • managed-acl.go
  • markers.go
  • migrations.go
  • nip43.go
  • nrc.go
  • nrc_events.go
  • paid-acl-interface.go
  • paid-acl.go
  • paid-types.go
  • process-delete.go
  • pubkey-serial.go
  • query-addressable.go
  • query-events.go
  • query-for-deleted.go
  • query-for-ids.go
  • query-for-ptag-graph.go
  • query-for-serials.go
  • register_badger.go
  • registry.go
  • repair.go
  • save-event.go
  • serial_cache.go
  • subscriptions.go
  • tokenize.go
  • types.go
  • unicode_normalize.go
  • wireguard.go

Directories

Path Synopsis
Package bufpool provides buffer pools for reducing GC pressure in hot paths.
Package bufpool provides buffer pools for reducing GC pressure in hot paths.
Package grpc provides a gRPC client that implements the database.Database interface.
Package grpc provides a gRPC client that implements the database.Database interface.
Package server provides a shared gRPC database server implementation.
Package server provides a shared gRPC database server implementation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL