Each table below starts with the go test command that produced it. Maintainers refresh the
numbers with make benchupdate, or by commenting /bench-readme (arm64) or
/bench-readme-amd64 on a pull request.
Environment:
goos: darwin
goarch: arm64
pkg: github.com/gofiber/utils/v2
cpu: Apple M2 Pro
EqualFold, IndexFold, ContainsFold, HasPrefixFold, and
HasSuffixFold compare ASCII case-insensitively using the SWAR word
loops: only A..Z/a..z fold, every other byte (including bytes
= 0x80) must match exactly. HasPrefixFold and HasSuffixFold cover
the header checks Fiber otherwise spells as an allocation-prone
ToLower + HasPrefix/HasSuffix pair (authorization schemes such as
Bearer, content-type prefixes such as multipart/form-data, suffixes
such as +json), and like the other Fold helpers they take the needle
as a plain string because call sites pass constant tokens. Their
Benchmark_HasPrefixFold/Benchmark_HasSuffixFold numbers are tracked
on the benchmark charts
and join the catalog above on its next regeneration.
HashFold is the hashing half of a case-insensitive lookup table whose
other half is EqualFold: strings EqualFold reports equal hash equal,
so a table of field or header names can be probed with a key in any case
without first lower-casing a copy of it. It folds each word with
swar.ToLowerWord as it loads it and mixes with wyhash's widening
multiply: keys under eight bytes, as most are, pack into one word and
cost one multiply, with the length in the multiplier so that keys which
pack alike, such as a and aa, stay apart; longer keys take one
multiply per sixteen bytes plus a final one. It takes no seed and is no
defense against collisions planned in advance, so it suits keys of the
program's own, such as the fields of a struct, rather than keys an
attacker picks. Against maphash.String of a strings.ToLower copy,
benchstat over ten runs on an Intel Xeon @ 2.80GHz measured 4.8 ns
against 13.2 ns for a 2-byte key, 4.5 ns against 50.7 ns for a 4-byte
key and 14.7 ns against 184 ns for a 48-byte key, where the copy also
allocates;
go test -run='^$' -bench='^Benchmark_HashFold$' -benchmem -count=10 . > hashfold.txt
followed by benchstat hashfold.txt reproduces that comparison. The
HashFold rows in the amd64 table below, like the table's other rows,
are a single run on the machine its header names.
HTTP dates
AppendHTTPDate and FormatHTTPDate write a time in the RFC 9110
preferred HTTP date format (Mon, 02 Jan 2006 15:04:05 GMT,
net/http.TimeFormat), byte-identical to time.Format with that layout
but without walking a layout string: the calendar fields come from one
civil-date conversion of t.Unix() (Hinnant's era arithmetic, instead of
the three abs() walks behind Date, Clock, and Weekday), and are
written into the fixed-width template with two-digit table lookups.
ParseHTTPDate is the reverse: canonical preferred-format input takes a
strict scalar fast path whose validated fields feed time.Unix directly
— the same Time value time.Date builds, minus its normalization pass —
and everything else — the obsolete RFC 850 and asctime forms, unusual
casing, non-GMT zone names, padding — falls back to time.Parse with
net/http.ParseTime semantics, including its errors. Both directions are
pinned to the time package by a day-by-day sweep over the years
0..9999. Date, Last-Modified, and If-Modified-Since handling sit on
every request/response, which makes these the highest-leverage helpers in
this group for Fiber.
URL escaping
AppendQueryEscape/AppendPathEscape and
AppendQueryUnescape/AppendPathUnescape produce byte-identical results
(and, for the unescape pair, identical url.EscapeError values) to their
net/url counterparts, as append-style, allocation-free single passes.
The escape tables are pinned to net/url.shouldEscape by exhaustive
per-byte tests. Unescaping jumps between escape sites with the vectorized
scans (IndexAny2 when + needs rewriting, bytes.IndexByte otherwise)
and copies clean spans wholesale, so route parameters and query values
without escapes — the common case — cost one scan and one copy. Decoding
never grows the input, so dst may be s[:0] on a common backing array
to unescape in place; escaping can grow the input, so there dst must
not alias s.
AppendPathSegmentsEscape covers a whole multi-segment path in one pass:
it escapes every segment as AppendPathEscape does and keeps the /
between them. The output equals escaping each segment and joining with
/, pinned per byte and by fuzzing; net/url's whole-path escaping
differs, as it also leaves ; and , unescaped. The catalog above
measures it against the per-segment loop a caller writes without it; the
other URL benchmarks are still in the amd64 block below.
JSON string escaping
AppendJSONString appends a value as a double-quoted JSON string,
byte-identical to encoding/json.Marshal of the same string — including
its default HTML escaping (<, >, &), the \ufffd replacement of
invalid UTF-8 bytes, and the U+2028/U+2029 escapes — without any of
Marshal's reflection or allocation. Clean spans are located with a SWAR
scan (the same first-match-mask technique as IndexNonQuotable) and
copied wholesale. This is the building block for hand-rolled JSON hot
paths such as access-log lines and error bodies.
IP address parsing
ParseIPv4 and ParseIPv6 parse addresses into netip.Addr, accepting
exactly the strings netip.ParseAddr accepts for the respective family
(pinned by fuzzing) while reporting failure with a bool instead of a
constructed error. Both take strings or byte slices, so fasthttp-style
callers skip the string conversion entirely; remote-address and
trusted-proxy checks parse an IP on every request in Fiber.
Header key canonicalization
CanonicalHeaderKey Title-Cases an HTTP header key exactly like
net/http.CanonicalHeaderKey, including its return-unchanged guard for
keys containing non-token bytes. Already-canonical keys — the common case
on receive paths — are validated in a single table pass and returned
as-is with zero allocations for strings and byte slices alike. Keys that
do need rewriting cost one allocation; note that for the ~40 header names
in the stdlib's interning table the stdlib returns a cached string
without allocating, so this helper's edge there is time, not allocations.
Control-byte scanning
IndexControl returns the index of the first ASCII control byte — a byte
below 0x20 or DEL (0x7F), the RFC 5234 CTL set — and IndexControlExceptTab
is the same scan with HTAB permitted, which is the byte set an RFC 9110
field value may not contain. Both are SWAR first-match scans
(two words per branch, one overlapping word for the tail) over strings or
byte slices; the HTAB exemption is tested only in words that hold a
control byte, so clean input costs the same either way. They never match
bytes >= 0x80, so unlike
strings.IndexFunc(s, unicode.IsControl) they neither decode UTF-8 nor
flag the C1 range hidden inside it. Fiber spells this check by hand in
half a dozen places — header values before they are echoed, request IDs,
Location values, log fields, Basic auth credentials — and each copy is
one of these two calls.
Byte-separator cuts
CutByte and LastCutByte are strings.Cut/bytes.Cut for a single
byte separator, around its first or last occurrence, generic over strings
and byte slices. They return the two parts and a found flag exactly like
the stdlib pair (including the ""/nil after-part of a miss), cost one
IndexByte/LastIndexByte, and spare []byte callers the []byte{sep}
needle and the length dispatch inside bytes.Index. CutByte beats
strings.Cut; LastCutByte has no stdlib counterpart, so its row is
measured against a hand-written LastIndexByte split, which it trails by
the cost of the call — its value is the generic []byte support and the
readable spelling. Header parameter splits (;), key=value pairs, and
host/port and name/extension splits are the intended call sites.
Host and port splitting
SplitHostPort splits host:port, host%zone:port, [host]:port, and
[host%zone]:port with exactly net.SplitHostPort's acceptance rules
(pinned by fuzzing against it), but reports failure with a bool instead
of constructing a *net.AddrError, and accepts byte slices. Valid input
costs the same as the stdlib — the scan is the same three IndexByte
calls — so the win is on the rejection path, where a malformed Host or
Forwarded value no longer costs an allocation, and on []byte input,
which no longer needs a string conversion first.
List field iteration
SplitTrimSeq returns an iterator over the elements of a list separated
by one byte, with ASCII whitespace trimmed from each element and empty
elements skipped — how HTTP list fields such as Accept-Encoding,
Connection, Vary, and Cache-Control are read (RFC 9110 Section
5.6.1: optional whitespace around the commas, empty elements ignored).
It replaces the strings.SplitSeq + TrimSpace + emptiness-check
triplet, locates each separator with a single IndexByte, works on
strings and byte slices alike, and yields subslices without copying. In
an ordinary caller the iterator inlines and allocates nothing, like
strings.SplitSeq; the benchmark ranges inside a helper function for
that reason, because calls made directly inside a b.Loop body are
never inlined and would show the closure allocations instead.
Duration formatting
AppendDuration renders a time.Duration exactly as d.String() does —
"72h3m0.5s", "1.5ms", "0s" — into the caller's buffer, with the two-digit
table for the field digits; a value with no fraction, the common case,
never touches the fraction digits. d.String() allocates its result and
fmt.Fprintf("%v", d), which Fiber's logger uses for the latency column,
costs several times more on top; an append-style form makes per-request
duration rendering (access logs, Server-Timing) allocation-free.
Decimal formatting
FormatUint/FormatInt (and their 32- and 16-bit variants) and
AppendUint/AppendInt render integers eight digits at a time: a value
below 1e8 becomes eight zero-padded digit lanes through a fixed sequence
of multiplies, shifts, and masks (the SWAR inverse of the 8-digit parse
step in ParseInt), so a 64-bit value costs at most three such
conversions and two divisions, all independent of each other, instead of
one dependent divide-and-store per digit; the first significant digit is
found with a trailing-zero count. Values below 100 come from a
precomputed table, and the output is pinned to strconv by a dense sweep
plus every decimal and lane-group boundary. GetMIME in the same spirit
looks up extensions through a hash table keyed by the extension packed
into one lower-cased word with its length in the top lane, so a hit is a
multiply, a shift, and one slot comparison, with no allocation even for
upper-case input.
These helpers were added on an amd64 machine, so like the simd numbers
their benchmarks are recorded separately from the arm64 catalog above and
join it on its next regeneration. The block also carries the current
numbers for the paths reworked on the same machine (integer formatting,
GetMIME, token-sized EqualFold), whose arm64 rows above predate that
work:
Environment:
goos: linux
goarch: amd64
pkg: github.com/gofiber/utils/v2
cpu: AMD EPYC
The swar package exports the SWAR (SIMD within a register)
building blocks the helpers above are composed from: Load8/Store8,
Broadcast, ZeroLanes, MatchByteMask/MatchRangeMask,
ToLowerWord/ToUpperWord, FirstLane/LastLane, and the WordLen,
Ones, HighBits, and LowSeven constants. They are exported so
downstream packages (Fiber itself, middleware) can fuse their own byte
scans — for example, finding a delimiter while classifying the bytes
before it — without re-deriving the bit tricks. The contracts (unchecked
bounds preconditions, ZeroLanes' approximate mask, the little-endian
lane order) and runnable examples live in the package documentation.
The package benchmarks its primitives against their stdlib counterparts
the same way the helpers above do, measuring the canonical loops composed
from them: Load8/Store8 vs encoding/binary's little-endian
Uint64/PutUint64, a ZeroLanes first-match scan vs
strings.IndexByte, a MatchByteMask+LastLane reverse scan vs
bytes.LastIndexByte, a MatchRangeMask digit scan vs
strings.IndexAny, and an in-place ToLowerWord loop vs
bytes.ToLower. The composed loops are pinned to the stdlib results by a
dedicated test, and the numbers are tracked per commit on the
benchmark charts. Note
that strings.IndexByte is hand-written SIMD assembly and overtakes the
portable SWAR scan on large inputs; the SWAR primitives earn their keep
on short HTTP-sized inputs and on fused scans the stdlib has no single
function for. The package's remaining benchmark, Benchmark_Load8_Fusion,
is an advisory codegen guard rather than an API performance promise, so
it is deliberately not part of the catalog above.
SIMD-accelerated search (package simd)
The simd package is the vector-width counterpart to swar: byte
searching and validation whose scan kernels dispatch to AVX2 assembly (four
32-byte vectors per iteration) on amd64 CPUs for inputs of simd.MinLen
(32) bytes or more, and fall back to portable loops built on the swar
primitives everywhere else, so it builds and behaves identically on every
platform.
simd.Accelerated() reports which mode is active. CPU capability detection
uses golang.org/x/sys/cpu,
which also verifies OS support for saving the vector register state.
The AVX2-backed kernels are the multi-needle scans (Memchr2, Memchr3),
the paired-byte scan (MemchrPair), the class scans
(MemchrDigit/MemchrDigitAt, MemchrWord/MemchrNotWord), the ASCII
scans (IsASCII, FirstNonASCII, CountNonASCII), and the substring
search (Memmem), which
prefilters on the needle's rarest bytes (SelectRareBytes, ByteRank)
before verifying candidates — 2-6 byte needles containing two distinct
values are scanned for their two rarest bytes at the exact relative
distance, longer or single-valued needles for the single rarest byte.
Each kernel classifies four 32-byte vectors per iteration. Where the
result is a position or a yes/no, the four masks are combined, so a
128-byte block costs a single VPMOVMSKB and a single branch, and the
scans that report a position re-extract the four masks in address order on
the (rare) hit path. CountNonASCII is the exception — a count cannot be
recovered from a combined mask, so its block loop accumulates per-lane
counts in vector byte lanes (a signed compare against zero and a byte
subtract per vector) and reduces them once at the end, while the
remaining whole vectors are extracted and population-counted; it is the
one kernel gated on POPCNT in addition to AVX2.
One tier is documented explicitly as unaccelerated:
MemchrInTable/MemchrNotInTable are plain
scalar loops, since an arbitrary 256-entry membership test has no cheap
vector form. There is deliberately no single-needle Memchr:
bytes.IndexByte is already vector-accelerated by the Go runtime. Memmem
delegates to bytes.Index for haystacks of up to 64 bytes, where the
stdlib's brute-force vector kernel is level with the paired prefilter
(Benchmark_Memmem_Prefilter forces the prefilter helpers at every size
to keep that measurable); from 65 bytes bytes.Index switches to a loop
anchored on the needle's first byte, and the paired scan takes over.
Needles that take the single rare-byte scan keep a 128-byte routing point,
since that scan is less selective. Both prefilters bail to bytes.Index
after a bounded number of failed candidate verifications, so even
adversarial inputs — a rare byte everywhere plus a long almost-matching
needle — stay within a constant of the stdlib's O(n+m) (see
Benchmark_Memmem_Adversarial). The
package operates on []byte only; the generic top-level helpers IsASCII,
IndexAny2, and IndexAny3 dispatch into it automatically for inputs of
simd.MinLen+ bytes on amd64, which is where the AVX2 kernels overtake the
SWAR word loops (-35% to -88% ns/op at 32-512B in benchstat -count=10
runs; shorter inputs keep the existing SWAR paths).
The kernels are adapted from the coregex
project's simd package (MIT License, Copyright (c) 2025 Andrey Kolkov and
contributors; see simd/LICENSE), with the legacy-SSE
register moves in the kernels replaced by VEX-encoded ones to avoid AVX-SSE
transition stalls, the scalar tail loops replaced by overlapping vector
rescans at the buffer end (a 63-byte scan previously cost ~3.7x a 64-byte
one), the 32-byte main loops replaced by 4x unrolled 128-byte blocks whose
bound is tested once at the bottom against a precomputed limit pointer, the
\w classifier rewritten from three VPMINUB/VPMAXUB clamp-and-compare
range tests into two VPSHUFB nibble-table lookups (with the block test
reduced to one compare over the four raw class values), the digit range
test reduced from two signed compares and a VPANDN to one biased
VPADDB and one compare, FirstNonASCII and CountNonASCII given
kernels of their own (they were SWAR-only, and the count kernel's block
loop accumulates in vector byte lanes rather than extracting and
population-counting every vector's mask), the 128-byte block loops
placed at fixed offsets inside a 32-byte code window so their throughput
no longer depends on where the linker places each kernel, and the
fallbacks reworked around the stdlib and the swar package as described
above.
Against the pre-unrolling kernels, benchstat -count=10 on the machine
below reports -26.6%/-49.7% ns/op for Memchr2 at 512B/4096B,
-17.6%/-34.3% for Memchr3, -13.4%/-24.6% for MemchrPair,
-39.2%/-50.5% for MemchrDigit, -53.5%/-63.7% for MemchrNotWord,
-42.2%/-65.7% for IsASCII, and -13.7%/-27.1% for Memmem, which rides
the paired scan. Those percentages come from benchstat over ten runs of
each side; the single-run table below is a catalog, not a base-vs-head
comparison, so subtracting its rows from an older copy of it will not
reproduce them.
Two bands see no benefit and are worth stating plainly. Inputs below
simd.MinLen never reach the kernels at all. Inputs of 32-127 bytes do
reach them but skip the 128-byte block loop, so they pay its setup with
nothing to amortize it against: across that band benchstat reports the
class scans clearly ahead (MemchrNotWord -8.6% at 32B, -18.4% at 64B;
IsASCII -5.0%/-4.3%), most rows statistically unchanged, and two small
regressions — MemchrDigit +5.6% at 32B and Memchr3 +3.8% at 64B. Read
those small-input figures against the run's own noise floor: the default
legs call unchanged stdlib code, so their movement bounds what a delta of
that size means. They held within 1% in the run quoted here, but a later
run on the same machine drifted up to 17% on those same rows, so anything
under ~10% at 32-64B should be taken as "no measured change" unless its
control legs are quiet.
On the portable side, pinning each unrolled group of words with a reslice
before loading at constant offsets inside it removes the per-load bounds
check that swar.Load8's own reslice otherwise repeats. Measured with
benchstat -count=10 against the same fallbacks, at 32B/512B/4KiB:
IsASCII -28%/-60%/-63%, CountNonASCII -24%/-44%/-44%,
FirstNonASCII -23%/-37%/-43%, Memchr2 -19%/-23%/-25%,
MemchrDigit -7%/-12%/-17%, Memchr3 -8%/-13%/-18%,
MemchrWord -11%/-8%/-11%, MemchrNotWord -5%/-7%/-6%. The tier a group
adds costs one compare on inputs too short to use it, which shows up as
+3% to +10% on some of the 8B fallback rows — a few tenths of a
nanosecond, and only below the length where the group pays for itself.
utils.IsASCII carries the same rework, so the two copies of that scan
stay in step.
Making the counting kernel finish its own sub-vector remainder (rather
than splitting the input in Go and calling the SWAR loop for the tail)
also removed a second non-inlinable call from every whole-vector input:
CountNonASCII at 32B went from 11.3ns — behind its own fallback — to
6.8ns, level with it, and it leads from 64B up.
A later audit pass reworked the kernels and fallbacks once more;
benchstat -count=10 on the machine below, base vs. head, reports:
MemchrDigit -20% at 512B and -27% at 4KiB from the one-add,
one-compare range test.
MemchrWord -13%/-11% and MemchrNotWord -4%/-10% at 512B/4KiB from
comparing the reduced block once instead of every vector, and their
SWAR fallbacks -36%/-41% at 32B, -40%/-43% at 64B and -43%/-45% at
512B: 15 ALU operations per word instead of 26, from case-folding the
two letter ranges into one test, a one-add underscore test, and a
single shared ASCII gate.
CountNonASCII -15% at 512B and -41% at 4KiB, with 32B and 64B
unchanged: the block loop counts in vector byte lanes (a signed compare
against zero and a byte subtract per vector, flushed through VPSADBW
every 127 blocks) instead of four port-0 VPMOVMSKB extractions per
block. The remaining whole vectors are still extracted and
population-counted, because the vector reduction is a fixed ten-cycle
chain that one to three vectors cannot amortize — a first version that
used the accumulators for those too was 20% slower at 32-64B.
MemchrPair's SWAR fallback -33% at 32B, -41% at 64B, -51% at 512B and
-52% at 4KiB, and MemchrPair itself -30% at 32B, where the dispatch
routes to that fallback: it now scans two words per branch through
pinned three-index windows (no per-load capacity check or pointer
clamp) and finishes with one overlapping word instead of a scalar
tail. Before, at 32B, it ran no faster than a plain scalar loop.
Memmem -62% at 96B: needles that take the paired prefilter are routed
through it from 65-byte haystacks, where bytes.Index leaves its
brute-force vector kernel for a first-byte-anchored loop; 64B and
128B+ are unchanged.
The same pinned-window rework applied to the case-conversion loops in
internal/caseconv, which the strings and bytes packages' ToLower
and ToUpper build on, gives on the same machine -27% to -37% ns/op for
the in-place UnsafeToLower/UnsafeToUpper at 39-64B, -14% to -19% for
the scan-only path that finds nothing to convert, and -5% to -17% for the
allocating conversions, where the allocation dominates. The arm64 catalog
rows above were not regenerated for that change.
Comparing the base and head test binaries also showed the untouched
MemchrPair kernel ~10% slower at 4KiB, purely because the linker had
moved its entry by 32 bytes: on Skylake-derived cores the JCC erratum
mitigation keeps a macro-fused compare-and-branch that crosses or ends on
a 32-byte boundary out of the decoded-uop cache, so the block loop ran
from the legacy decoder. Every block loop is now placed by a BLOCKALIGN
macro at a window offset where all of its fused pairs fit (a few
once-per-call NOPs, visible as +5% on Memchr2 at 512B and nothing at
4KiB), and an amd64 test reads each kernel's code and fails if a later
edit lets a pair straddle a boundary again.
Because the AVX2 kernels only engage on amd64, their benchmark numbers are
recorded separately from the arm64 catalog above:
Environment:
goos: linux
goarch: amd64
pkg: github.com/gofiber/utils/v2/simd
cpu: AMD EPYC
Fiber is an open-source project that runs on donations to pay the bills, e.g., our domain name, hosting, and serverless infrastructure. If you want to support Fiber, please become a GitHub Sponsor.
Tool Sponsors - supporting Fiber with free IDE licenses and AI credits
AddTrailingSlashBytes appends a trailing '/' to b if it does not already end with one.
If the input already ends with '/', the original slice is returned.
A new slice is returned when a '/' is appended. The original slice is never modified.
AddTrailingSlashString appends a trailing '/' to s if it does not already end with one.
If the input already ends with '/', the original string is returned.
A new string is returned only when a '/' needs to be appended.
AppendHTTPDate appends t in the RFC 9110 preferred HTTP date format
("Mon, 02 Jan 2006 15:04:05 GMT", net/http.TimeFormat) to dst and returns
the extended slice. The output is byte-identical to
t.UTC().AppendFormat(dst, http.TimeFormat) and always 29 bytes for the
years 0..9999 that HTTP dates can represent; times outside that range
delegate to time.AppendFormat.
func AppendJSONString[S byteSeq](dst []byte, s S) []byte
AppendJSONString appends s to dst as a double-quoted JSON string and
returns the extended slice. The output is byte-identical to
encoding/json.Marshal of the same string value, including its default
HTML escaping: '"' and '\\' get backslash escapes; control bytes below
0x20 become \b, \f, \n, \r, \t, or \u00XX; '<', '>', '&' become \u00XX; each
invalid UTF-8 byte becomes the six literal characters \ufffd; and the
line separators U+2028/U+2029 become the \u2028 and \u2029 escapes.
Go 1.27's json/v2 writes that replacement raw, so for invalid UTF-8 the two
encodings differ from 1.27 on; both still decode to the same string.
All other bytes, including multi-byte UTF-8 sequences, are copied
verbatim. dst must not alias s: the output is longer than the input (at
minimum by the surrounding quotes), so in-place encoding is impossible
and an aliased dst would overwrite bytes before they are read. Clean
spans are located with a SWAR scan and copied wholesale, so typical log
or header values cost one scan and one copy — with none of
encoding/json.Marshal's reflection or allocation.
func AppendPathEscape[S byteSeq](dst []byte, s S) []byte
AppendPathEscape appends the percent-encoded path-segment form of s to dst
and returns the extended slice. The output is byte-identical to
net/url.PathEscape; the aliasing rule matches AppendQueryEscape.
func AppendPathSegmentsEscape[S byteSeq](dst []byte, s S) []byte
AppendPathSegmentsEscape appends the percent-encoded form of the multi-segment
path s to dst and returns the extended slice. It escapes every segment as
AppendPathEscape does but leaves the '/' between them, so the output matches
escaping each segment separately and joining the results with '/'. Use it for
a value whose slashes are separators; use AppendPathEscape for one segment,
where a slash is data and has to be encoded. Empty and dot segments pass
through unchanged, so clean or reject an untrusted value before appending it
under a fixed prefix. The aliasing rule matches AppendQueryEscape.
func AppendPathUnescape[S byteSeq](dst []byte, s S) ([]byte, error)
AppendPathUnescape appends the decoded form of the path component s to dst
and returns the extended slice, decoding %XX and leaving '+' verbatim,
exactly like net/url.PathUnescape. Error and aliasing behavior match
AppendQueryUnescape.
func AppendQueryEscape[S byteSeq](dst []byte, s S) []byte
AppendQueryEscape appends the percent-encoded form of s to dst and returns
the extended slice. The output is byte-identical to net/url.QueryEscape:
unreserved bytes pass through, space becomes '+', everything else becomes
%XX with uppercase hex. dst must not alias s: escaping can grow the input,
so in-place operation is impossible.
func AppendQueryUnescape[S byteSeq](dst []byte, s S) ([]byte, error)
AppendQueryUnescape appends the decoded form of the query component s to
dst and returns the extended slice, converting '+' to space and %XX to the
byte it encodes, exactly like net/url.QueryUnescape including its
url.EscapeError values for malformed escapes. On error the returned slice
is dst with its original length (its backing array may still have been
reallocated by growth). Decoding never grows the input, so dst may be
s[:0] on a common backing array to decode in place; any other overlap is
invalid. Note that an in-place decode that fails has already overwritten
the prefix of s before the malformed escape with decoded bytes — treat s
as consumed once an in-place decode starts, error or not.
ByteSize returns a human-readable byte string of the form 10M, 12.5K, and so forth.
The unit that results in the smallest number greater than or equal to 1 is always chosen.
Maximum supported input is math.MaxUint64 / 10 (≈ 1844674407370955161).
CanonicalHeaderKey returns the canonical form of the HTTP header key:
the first letter and any letter following a hyphen upper-cased, the rest
lower-cased. The output matches net/http.CanonicalHeaderKey byte for
byte, including its guard: a key containing any byte outside the header
token alphabet is returned unchanged. Unlike the stdlib it accepts byte
slices as well as strings and allocates only when the key actually needs
rewriting — already-canonical keys, the overwhelmingly common case on
receive paths, are validated in one table pass and returned as-is
without the stdlib's canonical-key cache lookup.
func ConvertToBytes(humanReadableString string) int
ConvertToBytes returns integer size of bytes from human-readable string, ex. 42kb, 42M
Decimal units are powers of 1000 (42k = 42000); binary units with an 'i' infix
are powers of 1024 (42Ki = 43008). Note that ByteSize formats with powers of 1024,
so use binary suffixes for a lossless round-trip.
Returns 0 if the string is unrecognized or negative.
func CutByte[S byteSeq](s S, sep byte) (before, after S, found bool)
CutByte is strings.Cut/bytes.Cut for a single-byte separator, generic over
strings and byte slices; a miss returns s, the zero S ("" or nil), and
false. The parts alias s.
GenerateSecureToken generates a cryptographically secure random token encoded in base64.
It uses crypto/rand for randomness and base64.RawURLEncoding for URL-safe output.
If length is less than or equal to 0, it defaults to 32 bytes (256 bits of entropy).
Panics if the random source fails.
GetMIME returns the content-type of a file extension, matched
case-insensitively with or without the leading dot. Unknown extensions map
to MIMEOctetStream; an empty extension yields an empty string.
HasPrefixFold reports whether s begins with prefix, ASCII
case-insensitively: only 'A'..'Z'/'a'..'z' fold, every other byte
(including >= 0x80) must match exactly, mirroring IndexFold. An empty
prefix matches any s. Like the other Fold helpers the needle is a plain
string, since call sites pass constant tokens.
HasSuffixFold reports whether s ends with suffix, ASCII
case-insensitively, under the same folding contract as HasPrefixFold.
An empty suffix matches any s.
HashFold returns a 64-bit hash of s in which ASCII letters are folded to one
case: strings that EqualFold reports equal hash equal. It is the hashing
half of a case-insensitive lookup table whose other half is EqualFold, for
keys such as header or field names, and reads s a word at a time without
writing a folded copy of it anywhere. As in EqualFold, only 'A'..'Z' and
'a'..'z' fold; every other byte, including those >= 0x80, hashes as is.
It is not a cryptographic hash, and it takes no seed, so it offers no
protection against collisions planned in advance: use it to index keys of
the program's own, such as the fields of a struct, and not to store keys an
attacker picks. Its values may change between releases; do not persist them.
IndexAny2 returns the index of the first occurrence in s of either a or b,
or -1 if neither is present. On amd64 CPUs with AVX2, inputs of 32+ bytes
dispatch to package simd instead.
IndexAny3 returns the index of the first occurrence in s of a, b, or c,
or -1 if none is present. On amd64 CPUs with AVX2, inputs of 32+ bytes
dispatch to package simd instead.
IndexControl returns the index of the first ASCII control byte in s (below
0x20 or DEL, the RFC 5234 CTL set), or -1. HTAB counts as a control byte;
bytes >= 0x80 never match, unlike unicode.IsControl's C1 range.
IndexFold returns the index of the first ASCII case-insensitive occurrence
of needle in s, or -1 if absent. An empty needle matches at index 0. Only
'A'..'Z'/'a'..'z' fold; every other byte (including >= 0x80) must match
exactly, so e.g. "no\rcache" does NOT match the needle "no-cache".
The needle is a plain string by design: call sites pass constant tokens,
and a []byte needle would cost its callers a conversion either way.
IndexNonQuotable returns the index of the first byte of s that cannot
appear verbatim inside an RFC 9110 quoted-string — that is, a byte
matching c == '\\' || c == '"' || (c < 0x20 && c != '\t') || c == 0x7f —
or -1 if every byte is quotable. qdtext is HTAB / SP / %x21 / %x23-5B /
%x5D-7E / obs-text, so HTAB and bytes >= 0x80 are quotable. Returning the
index (rather than a bool) lets callers copy the clean prefix and start
escaping exactly at the offending byte. Inputs of 8+ bytes finish with
one overlapping word at n-8, shorter ones byte-wise.
IsASCII reports whether s contains only ASCII bytes (no byte >= 0x80).
It ORs four words per iteration where possible (no index is needed, so
detection can be deferred to one test per 32 bytes), then two, then
word-wise; inputs of 8+ bytes finish with one overlapping word at n-8,
shorter ones byte-wise.
On amd64 CPUs with AVX2, inputs of 32+ bytes dispatch to package simd
instead.
The unrolled loops pin their group of words with a reslice and load at
constant offsets inside it. swar.Load8's own reslice is bounds-checked
against the backing array, which the length-based loop condition does
not imply, so a variable index costs a compare and a branch per load;
pinning the window pays that once per group instead. This mirrors
simd.isASCIIGeneric — the same scan, kept in step so a retune of either
is a retune of both.
IsIPv4 reports whether s is a dotted-decimal IPv4 address. It accepts
exactly what net.ParseIP accepts for the IPv4 case, without the net.IP slice,
so it makes no allocations. It is the validate-only spelling of ParseIPv4,
which accepts the same strings and also returns the address.
IsIPv6 reports whether s is an IPv6 address. It accepts exactly what
net.ParseIP accepts for the IPv6 case, without the net.IP slice, so it makes
no allocations. That excludes a "%zone" suffix, because net.ParseIP rejects
zones.
ParseIPv6 follows netip.ParseAddr instead and does accept zones, so the two
disagree on zoned input by design. Callers relying on IsIPv6 to reject zones
must screen '%' themselves before substituting ParseIPv6.
ParseHTTPDate parses an HTTP date the way net/http.ParseTime does:
the RFC 9110 preferred format ("Mon, 02 Jan 2006 15:04:05 GMT") plus the
obsolete RFC 850 and ANSI C asctime forms, with surrounding ASCII
whitespace tolerated. Canonical preferred-format input takes a fast scalar
path that never calls time.Parse; every other input — legacy formats,
unusual casing, non-GMT zone names, padding — falls back to time.Parse
with byte-for-byte stdlib semantics, including its errors. The returned
time is in time.UTC on the fast path and whatever time.Parse yields on the
fallback; the instants agree in both cases.
ParseIPv4 parses a dotted-decimal IPv4 address into a netip.Addr. It
accepts exactly the IPv4 strings netip.ParseAddr does — four octets 0-255,
no leading zeros, nothing before or after — and reports ok=false for
everything else, including IPv6 forms. It never allocates, so []byte
callers skip both the string conversion and netip's error construction.
IsIPv4 is the validate-only counterpart; it accepts the same strings.
ParseIPv6 parses an IPv6 address into a netip.Addr. It accepts exactly
the IPv6 strings netip.ParseAddr does — 16-bit hex fields, one optional
"::", an optional embedded dotted-decimal IPv4 tail, and an optional
non-empty "%zone" suffix — and reports ok=false for everything else,
including plain IPv4 forms (use ParseIPv4 for those). The parse itself
never allocates; only a present zone is materialized as a string because
netip.Addr stores zones as strings.
IsIPv6 is the validate-only counterpart, but it follows net.ParseIP, which
rejects zones. The two therefore disagree on zoned input by design: prefer
ParseIPv6 when a zone is acceptable, IsIPv6 when it is not.
ParseUint parses a decimal ASCII string or byte slice into a uint64.
It returns the parsed value and nil on success.
If the input contains non-digit characters, it returns 0 and an error.
ParseVendorSpecificContentType check if content type is vendor specific and
if it is parsable to any known types. If its not vendor specific then returns
the original content type.
func SplitHostPort[S byteSeq](hostport S) (host, port S, ok bool)
SplitHostPort splits "host:port", "[host]:port", and their %zone forms
with net.SplitHostPort's exact rules, reporting failure as ok == false
instead of a *net.AddrError; it accepts byte slices as well as strings,
and the parts alias hostport.
SplitTrimSeq iterates over the sep-separated elements of s with ASCII
whitespace trimmed and empty elements skipped, as HTTP list fields are read
(RFC 9110 Section 5.6.1); the elements alias s.
StartTimeStampUpdater launches a background goroutine that updates the cached timestamp every second.
It is safe to call multiple times and from multiple goroutines; only one updater runs at a time.
StopTimeStampUpdater stops the background updater goroutine.
Call this on app shutdown to avoid leaking goroutines.
It is safe to call multiple times and from multiple goroutines.
Trim removes all leading and trailing occurrences of the byte cutset from s.
Unlike strings/bytes.Trim, cutset is a single byte, not a set of characters.
TrimRight removes all trailing occurrences of the byte cutset from s.
Unlike strings/bytes.TrimRight, cutset is a single byte, not a set of characters.
TrimSpace removes leading and trailing whitespace from a string or byte slice.
This is an optimized version that's faster than strings/bytes.TrimSpace for ASCII strings.
It removes the following ASCII whitespace characters: space, tab, newline, carriage return, vertical tab, and form feed.
Walk walks the filesystem rooted at root, calling walkFn for each file or
directory in the filesystem, including root. All errors that arise visiting files
and directories are filtered by walkFn. The files are walked in lexical
order.
CBORUnmarshal parses the CBOR-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an error.
JSONUnmarshal parses the JSON-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an InvalidUnmarshalError.
type MsgPackUnmarshal func(data []byte, v any) error
MsgPackUnmarshal parses the MsgPack-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an InvalidUnmarshalError.
XMLUnmarshal parses the XML-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an InvalidUnmarshalError.