Major Description
The following exception with stack traces is logged many times when state updater is enabled:
01:08:03 INFO [KAFKA] TaskManager - stream-thread [acme-StreamThread-4] Encountered lock exception. Reattempting locking the state in the next iteration.
org.apache.kafka.streams.errors.LockException: stream-thread [acme-StreamThread-4] standby-task [1_15] Failed to lock the state directory for task 1_15
at org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:96)
at org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:114)
at org.apache.kafka.streams.processor.internals.TaskManager.addTaskToStateUpdater(TaskManager.java:1008)
at org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:995)
at org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:911)
at org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1188)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:996)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:711)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670)
The exception is expected since it happens because a lock on the task state directory is not yet been freed by a different stream thread on the same Kafka Streams client after an assignment. But with the state updater acquiring the lock is attempted in each poll iteration which is every 100 ms by default.
One option to reduce the log messages is to reduce the rate at which a lock is attempted to be acquired. The other is to reduce the logging.