Show how to use Temporal as a queue #431
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This example shows how you can use Temporal's event history and workflow input as a queue. It implements a batching mechanism where a workflow continuously listens for incoming signals, as well as a timer, and processes the batch when the timer fires, followed by restarting the timer. The workflow keeps track of the number of incoming events and decides to continue-as-new when there are too many events recorded in the event history, passing the accumulated events in the next workflow's input.
What was changed
Added a new example.
Depending on how complicated the change is, I think it may be worth it to add a Ticker to the workflow API (e.g.
workflow.NewTicker) since we already also have Timer. I think it would at least be a nice addition to the Go SDK (implemented with Timers), if not worthy as a Temporal feature.Why?
This seemed to be a nice pattern in situations where you need some sort of queue to batch operations, but you only have a Temporal cluster available and no event streaming platform. It likely doesn't reach the same performance as say Kafka for extremely high throughput, but it should be good enough for a large amount of use-cases.
I needed to write a similar PoC, so I thought I'd share it here as well.
Checklist
Local Temporal cluster, also explained in readme.