Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9228

Reconfigured converters and clients may not be propagated to connector tasks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0, 2.3.1, 2.3.2, 2.4.0
    • 3.7.1, 3.8.0
    • connect
    • None

    Description

      If an existing connector is reconfigured but the only changes are to its converters and/or Kafka clients (enabled as of KIP-458), the changes will not propagate to its tasks unless the connector also generates task configs that differ from the existing task configs. Even after this point, if the connector tasks are reconfigured, they will still not pick up on the new converter and/or Kafka client configs.

      This is because the DistributedHerder only writes new task configurations to the connect config topic if the connector-provided task configs differ from the task configs already in the config topic, and neither of those contain converter or Kafka client configs.

      Attachments

        1. image-2024-10-08-17-03-14-251.png
          319 kB
          Dmitri Pavlov

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-17704 possible race condition in TTL credentials when connectors recycled on single node instance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-16837 Kafka Connect fails on update connector for incorrect previous Config Provider tasks

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-17627 ConfigProvider TTLs do not restart Tasks

          • Major - Major loss of function.
          • Patch Available
          links to

          Web Link GitHub Pull Request #7823

          Web Link GitHub Pull Request #16001

          Web Link GitHub Pull Request #16053

          Activity

            People

              ChrisEgerton Chris Egerton
              ChrisEgerton Chris Egerton
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: