-
Notifications
You must be signed in to change notification settings - Fork 3.1k
feat(hitl): add human in the loop block #1832
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
* improvement(performance): remove unused source/target indices, add index on snapshot id (#1603) * fix(blog): rename building to blogs with redirect (#1604) * improvement(privacy-policy): updated privacy policy for google (#1602) * updated privacy policy for google * update terms, privacy, and emails to incl address and update verbiage * feat(guardrails): added guardrails block/tools and docs (#1605) * Adding guardrails block * ack PR comments * cleanup checkbox in dark mode * cleanup * fix supabase tools * fix(inference-billing): fix inference billing when stream is true via API, add drag-and-drop functionality to deployed chat (#1606) * fix(inference): fix inference billing when stream is true via API * add drag-and-drop to deployed chat * feat(mistal): added mistral as a provider, updated model prices (#1607) * feat(mistal): added mistral as a provider, updated model prices * remove the ability for a block to reference its own outluts * fixed order of responses for guardrails block * feat(versions): added the ability to rename deployment versions (#1610) * fix(vulns): fix various vulnerabilities and enhanced code security (#1611) * fix(vulns): fix SSRF vulnerabilities * cleanup * cleanup * regen docs * remove unused deps * fix failing tests * cleanup * update deps * regen bun lock
…ad support for more blocks
…hboard improvements
* fix(debug-mode): remove duplicate debug mode flag (#1714) * feat(i18n): update translations (#1709) * improvement(condition): added variable and envvar highlighting for condition input (#1718) * fix(dashboard): add additional context for paginated logs in dashboard, add empty state when selected cell has no data (#1719) * fix(dashboard): add additional context for paginated logs in dashboard, add empty state when selected cell has no data * apps/sim * renaming * remove relative import * feat(tools): added webflow OAuth + tools (#1720) * feat(tools): added webflow OAuth + tools * remove itemId from delete item * remove siteId * added webhook triggers + oauth scopes + site/collection selector * update sample payload for webflow triggers * cleanup * fix discord color * feat(i18n): update translations (#1721) * improvement(schedule): fix UI bug with schedule modal (#1722)
* fix(already-cancelled-sub): UI should allow restoring subscription * restore functionality fixed * fix
…cp improvements, local kb file storage
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Greptile Summary
This PR implements a comprehensive human-in-the-loop (HITL) feature allowing workflows to pause execution and wait for human input before continuing. The implementation includes:
Core Changes:
- New
pause_resumeblock handler that generates pause metadata and executes notification tools - Execution engine modified to detect pause blocks and halt execution while serializing state
PauseResumeManagerclass handling database persistence, queue management, and resume orchestration- Snapshot serialization/restoration system to capture and restore full execution state
Database Schema:
paused_executionstable storing serialized snapshots, pause points, and metadataresume_queuetable managing concurrent resume requests with status tracking- Proper indexes on workflow_id, status, and execution_id for query performance
API & UI:
- Resume API endpoints at
/api/resume/[workflowId]/[executionId]/[contextId]with POST (submit) and GET (status) - React client UI (1463 lines) for dynamic form rendering and submission handling
- Notification tool execution for alerting users about pending approvals
DAG & Execution:
- Enhanced edge construction to wire pause trigger mappings correctly
- Resume execution removes edges from completed pause blocks to prevent re-execution
- Transaction-based concurrency control for queue management
Key Features:
- Support for both API and human operation modes
- Queue system prevents concurrent resumes on same execution
- Preserves loop and parallel execution context across pause/resume
- Self-reference capability allowing pause blocks to reference their own outputs
Confidence Score: 3/5
- This PR requires careful testing before merging due to race condition risks and fire-and-forget error handling
- The implementation is architecturally sound with proper database schema, snapshot serialization, and queue management. However, two critical issues lower confidence: (1) fire-and-forget pattern in resume API loses immediate errors, (2) missing row-level locking could allow race conditions in concurrent resume requests. The extensive changes across 90 files touch core execution logic which needs thorough testing.
apps/sim/app/api/resume/[workflowId]/[executionId]/[contextId]/route.ts(error handling) andapps/sim/lib/workflows/executor/pause-resume-manager.ts(concurrency control)
Important Files Changed
File Analysis
| Filename | Score | Overview |
|---|---|---|
| packages/db/schema.ts | 5/5 | Added two new tables (paused_executions and resume_queue) with proper foreign keys, indexes, and constraints for managing workflow pause/resume state |
| apps/sim/executor/handlers/pause-resume/pause-resume-handler.ts | 4/5 | Implements HITL block handler with notification tool execution, response data parsing, and pause metadata creation; extensive data normalization logic |
| apps/sim/lib/workflows/executor/pause-resume-manager.ts | 3/5 | Core pause/resume orchestration with database persistence, queue management, snapshot restoration, and resume execution; includes transaction-based concurrency control and edge removal logic for resumed blocks |
| apps/sim/executor/execution/engine.ts | 4/5 | Modified execution engine to detect pause metadata, halt execution when pauses occur, handle resume triggers, and build paused execution results with snapshot serialization |
| apps/sim/executor/execution/snapshot-serializer.ts | 5/5 | New file that serializes execution snapshots including DAG structure, state, and trigger IDs for pause persistence |
| apps/sim/executor/dag/construction/edges.ts | 4/5 | Enhanced edge construction to handle pause trigger mappings and wire resume trigger nodes correctly in the DAG |
| apps/sim/app/api/resume/[workflowId]/[executionId]/[contextId]/route.ts | 4/5 | API endpoint for resuming paused executions with queue management, validation, and async execution triggering; includes GET endpoint for pause context details |
| apps/sim/app/resume/[workflowId]/[executionId]/resume-page-client.tsx | 4/5 | Large React client component (1463 lines) providing UI for human-in-the-loop interactions with dynamic form rendering, status polling, and submission handling |
| apps/sim/executor/execution/block-executor.ts | 4/5 | Updated to prepare self-reference for pause/resume blocks, enabling blocks to reference their own outputs during input resolution |
Sequence Diagram
sequenceDiagram
participant Client
participant API as Resume API
participant Manager as PauseResumeManager
participant DB as Database
participant Engine as ExecutionEngine
participant Handler as PauseResumeHandler
Note over Client,Handler: Initial Workflow Execution
Client->>Engine: Execute Workflow
Engine->>Handler: Execute Pause Block
Handler->>Handler: Generate pause metadata
Handler-->>Engine: Return output with _pauseMetadata
Engine->>Engine: Detect pause, halt execution
Engine->>Manager: Serialize snapshot
Manager->>DB: INSERT paused_executions
Manager-->>Client: Return paused result with resume links
Note over Client,Handler: Resume Request
Client->>API: POST /resume/[workflowId]/[executionId]/[contextId]
API->>Manager: enqueueOrStartResume()
Manager->>DB: BEGIN TRANSACTION
Manager->>DB: SELECT paused_execution
Manager->>DB: Check pause_point.resumeStatus
alt Active resume exists
Manager->>DB: INSERT resume_queue (status=pending)
Manager->>DB: Update pause_point (status=queued)
Manager->>DB: COMMIT
Manager-->>API: {status: queued, queuePosition: N}
API-->>Client: Queued response
else No active resume
Manager->>DB: INSERT resume_queue (status=claimed)
Manager->>DB: Update pause_point (status=resuming)
Manager->>DB: COMMIT
Manager-->>API: {status: starting, ...execution_data}
API->>Manager: startResumeExecution() [fire-and-forget]
API-->>Client: Started response
Note over Manager,Engine: Resume Execution (Async)
Manager->>DB: Load execution snapshot
Manager->>Manager: Restore ExecutionSnapshot
Manager->>Manager: Inject resume input into pause block output
Manager->>Manager: Calculate edges to remove
Manager->>Engine: Execute with restored snapshot
Engine->>Engine: Process pending blocks
alt Execution pauses again
Engine-->>Manager: Paused result
Manager->>DB: Persist new pause state
else Execution completes
Engine-->>Manager: Success result
Manager->>DB: Update snapshot after resume
end
Manager->>DB: Mark resume_queue entry completed
Manager->>Manager: processQueuedResumes()
alt Queued resumes exist
Manager->>DB: SELECT next pending resume
Manager->>DB: UPDATE status=claimed
Manager->>Manager: startResumeExecution() [recursive]
end
end
82 files reviewed, 2 comments
* fix(billing): should allow restoring subscription (#1728) * fix(already-cancelled-sub): UI should allow restoring subscription * restore functionality fixed * fix * Add pause resume block * Add db schema * Initial test passes * Tests pass * Execution pauses * Snapshot serializer * Ui checkpoint * Works 1 * Pause resume simple v1 * Hitl block works in parallel branches without timing overlap * Pending status to logs * Pause resume ui link * Big context consolidation * HITL works in loops * Fix parallels * Reference blocks properly * Fix tag dropdown and start block resolution * Filter console logs for hitl block * Fix notifs * Fix logs page * Fix logs page again * Fix * Checkpoint * Cleanup v1 * Refactor v2 * Refactor v3 * Refactor v4 * Refactor v5 * Resume page * Fix variables in loops * Fix var res bugs * Ui changes * Approval block * Hitl works e2e v1 * Fix tets * Row level lock --------- Co-authored-by: Waleed <[email protected]> Co-authored-by: Vikhyath Mondreti <[email protected]>
* fix(billing): should allow restoring subscription (#1728) * fix(already-cancelled-sub): UI should allow restoring subscription * restore functionality fixed * fix * Add pause resume block * Add db schema * Initial test passes * Tests pass * Execution pauses * Snapshot serializer * Ui checkpoint * Works 1 * Pause resume simple v1 * Hitl block works in parallel branches without timing overlap * Pending status to logs * Pause resume ui link * Big context consolidation * HITL works in loops * Fix parallels * Reference blocks properly * Fix tag dropdown and start block resolution * Filter console logs for hitl block * Fix notifs * Fix logs page * Fix logs page again * Fix * Checkpoint * Cleanup v1 * Refactor v2 * Refactor v3 * Refactor v4 * Refactor v5 * Resume page * Fix variables in loops * Fix var res bugs * Ui changes * Approval block * Hitl works e2e v1 * Fix tets * Row level lock --------- Co-authored-by: Waleed <[email protected]> Co-authored-by: Vikhyath Mondreti <[email protected]>
Summary
Adds approval (hitl) block
Type of Change
Testing
Manual
Checklist