Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-18137

Unloading transaction state incorrectly removes loading partitions

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0
    • None
    • None

    Description

      When there is a become follower transition on a transaction coordinator state partition, we intend to unload the state partition. However, we pass the new epoch to the method that does the unloading. In that method, we create a `TransactionPartitionAndLeaderEpoch` object comprising of the topic partition and the epoch that we use as a key to remove the partition from loading. However, we wouldn't ever expect to see this epoch in that map since we only load on the leader. See the code snippet: https://github.com/apache/kafka/blob/d00f0ecf1a1a082c97564f4b807e7a342472b57a/core/src/main/scala/kafka/coordinator/transaction/TransactionStateManager.scala#L602

      We could have a partition load after the unloading occurs, and that partition will be stuck storing stale state on the broker until it restarts. While this may not immediately cause a correctness issue, we should try to properly clean up state.

      Attachments

        Issue Links

          Activity

            People

              jolshan Justine Olshan
              jolshan Justine Olshan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: