Skip to content

Add e2e tests for Elasticsearch source connector #2593

@hubcio

Description

@hubcio

The Elasticsearch source connector needs integration tests. See the Postgres source tests (#2579) for the testing pattern.

What needs testing

  • Basic flow: documents in ES → messages in Iggy
  • JSON document handling
  • Pagination modes (scroll-based, cursor-based)
  • Timestamp filtering behavior
  • State persistence (last_document_id, last_poll_timestamp)

Implementation notes

  • Reuse the testcontainers ES setup from sink tests
  • Use poll_messages pattern from postgres source tests
  • Test state recovery scenarios

Files to reference

  • core/integration/tests/connectors/postgres/postgres_source.rs
  • core/connectors/sources/elasticsearch_source/src/lib.rs
  • core/connectors/sources/elasticsearch_source/src/state_manager.rs

Acceptance criteria

  • Tests for at least 2 pagination modes
  • State persistence test (stop/restart scenario)
  • Clear failure messages

Metadata

Metadata

Assignees

Labels

connectorsConnectors runtimegood first issueGood for newcomerstestNew testcase, or improvement of existing

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions