Jul 12, 2026
Benchmark measuring AI agent iterative improvement over 12-hour real-world task interactions
Agents receive executable environments across 134 tasks with multi-level feedback and run for extended sessions while trajectories are recorded. The resulting log-sigmoid scaling relationship between interaction time and performance offers a new evaluation primitive that agent developers and research labs can adopt to move beyond one-shot accuracy metrics.
Environment and dataset of 2.4K verified tasks for training software engineering agents and verifiers
SWE-Gym supplies executable Python repositories, test oracles, and 2.4K real issues drawn from eleven popular projects together with pre-sampled agent trajectories from frontier models. The resulting training substrate enables both reinforcement learning of agents and supervised training of outcome verifiers, delivering new open SWE-Bench records and exposing clear scaling laws that many agent teams can now exploit.
Terminal-Bench 2.0 benchmark suite for evaluating AI agents on containerized terminal tasks
Tasks are packaged as Docker containers and executed through the Harbor framework, which orchestrates concurrent runs, model endpoints, and third-party agents while collecting traces for analysis. The benchmark raises difficulty and validation standards with multi-hour human and LM review per task so frontier labs can reliably measure progress on realistic software-engineering and scientific workflows.
Pi extension enabling model-written JavaScript workflows that orchestrate isolated subagents
The workflow tool parses scripts in a Node VM sandbox and spawns fresh in-memory Pi subagent sessions that support agent, parallel, pipeline, phase, and schema-validated structured output calls. This adapts existing dynamic workflow patterns from other assistants to Pi's tool ecosystem, primarily benefiting developers already embedded in the Pi coding workflow for audits and multi-perspective tasks.
High-performance fused kernels for gated delta rule linear attention on Hopper and Blackwell
FlashQLA implements warp-specialized forward and backward kernels via TileLang that fuse GDN chunked prefill operations while exploiting exponential gate decay for automatic intra-card context parallelism. The library delivers targeted speedups for Qwen-family pretraining and agentic inference workloads but remains narrowly scoped to one linear attention variant rather than becoming a general primitive.
Hybrid edge AI engine delivering OpenAI-compatible APIs for on-device LLM vision and speech inference on mobile
The engine layers a zero-copy C++ graph and ARM NEON kernels over custom rotation-based quantization plus a PyTorch transpiler to run models locally on iOS and Android. Its hybrid cloud handoff and cross-device kernel support give mobile developers a practical path to private low-latency AI without relying on generic inference runtimes.
Log-structured filesystem serving S3 buckets as POSIX filesystems over NFS 9P and NBD block devices
File data is split into compressed encrypted extents packed into immutable segments on object storage while metadata resides in an LSM tree; NFS 9P and NBD servers run inside the same process with optional leader-standby HA. The design delivers near-raw throughput and POSIX compliance without external databases, attracting teams that need reliable object-backed storage in cloud or Kubernetes deployments.
Elixir library defining factories to build and insert test data with Ecto associations
Factories are implemented as functions inside a dedicated module that support sequences, lazy evaluation of attributes, derived values, and automatic merging of overrides before build or insert operations. The approach mirrors well-known factory patterns from other languages yet stays confined to the Elixir testing workflow where it mainly benefits Ecto-based applications.
Terminal-Bench 3 measures AI agents on hard realistic command-line tasks inside containerized environments
Tasks are authored as self-contained container definitions and executed through the Harbor orchestration tool against chosen agents and models. The benchmark advances prior versions by targeting capabilities that remain unsolved by current frontier systems, positioning it as a standard evaluation primitive for labs developing generalist agents.
Short reinforcement learning textbook with PyTorch implementations of algorithms from Monte Carlo to PPO
The repository pairs the book with algorithm implementations under algos/ and a supplementary document of dynamic programming proofs. It follows the standard educational format of existing RL tutorials and targets students or practitioners looking for a compact reference without introducing new methods or widely reusable tooling.
Harbor fork for running CLI coding agents in air-gapped sandboxes with consistent ATIF trajectories
Pier executes Harbor-format tasks inside docker or modal environments while honoring per-agent install scripts and network allowlists so that installed models can operate without external internet access. The changes deliver stricter one-step-per-turn logging plus a dedicated viewer and critique workflow that appeal mainly to research teams building reproducible SWE-agent benchmarks rather than general developers.
Framework for evaluating agents and language models across containerized terminal environments
Harbor launches agents inside Docker or cloud sandboxes from providers such as Daytona and Modal, executing tasks in parallel while capturing full trajectories for reinforcement learning optimization. It unifies several existing benchmark harnesses behind a single CLI and targets the narrow but growing community of agent and terminal-bench researchers.
Legal agent benchmark providing 1660 tasks and execution harness for LLM evaluation on real legal workflows
Tasks include agent instructions, source documents, and detailed rubrics across 24 practice areas, executed via an open harness that supports model adapters, tool use, and all-pass rubric scoring. The benchmark fills a specialized gap in domain-specific agent evaluation that will mainly interest legal technology teams and AI researchers focused on regulated workflows rather than general developer tooling.
Benchmark of 113 long-horizon engineering tasks for evaluating frontier coding agents
Tasks are supplied in the Harbor format with metadata, agent prompts, isolated Docker environments, and program-based verifiers that accept any correct observable behavior. The benchmark supplies a fresh corpus of real open-source work that AI labs and agent developers can use to measure progress beyond shorter or synthetic evaluations.
Lightweight JavaScript runtime built from scratch with a custom MIR-based JIT engine
Ant implements the WinterTC minimum common API using its own Ant Silver engine and a forked MIR backend, shipping as a single 4-8 MB binary with sub-10 ms cold starts. The approach of hand-rolling a full JavaScript engine rather than wrapping V8 or JSC opens a path for constrained environments such as serverless functions, edge nodes, and embedded CLI tools that cannot tolerate the footprint of conventional runtimes.