Feature/dynamic calibration on start by JakubFara · Pull Request #1695 · luxonis/depthai-core

JakubFara · 2026-02-26T14:56:48Z

AutoCalibration node

Purpose

The auto-calibration of the camera on startup.
The auto-calibration process is invoked only if the stereo pair is present.

New AutoCalibration node

This node has a DynamicCalibration subnode.
Firstly, we check the current calibration; if it is not good, we recalibrate using the DynamicCalibration node.
The auto-calibration is turned off and can be turned on by DEPTHAI_AUTOCALIBRATION=on.

Remarks

It runs only on the host.
The packetization is not involved yet.

Testing

Basic test is added to the tests/src/ondevice_tests/pipeline/node/auto_calibration_test.cpp
Run the HIL tests with the DEPTHAI_AUTOCALIBRATION=on

MaticTonin

Overall looks good, put some general comments on what should be checked before even marking it as testable.

cmake/Depthai/DepthaiDynamicCalibrationConfig.cmake

MaticTonin · 2026-02-26T15:28:46Z

cmake/Depthai/DepthaiDeviceSideConfig.cmake


 # "full commit hash of device side binary"
-set(DEPTHAI_DEVICE_SIDE_COMMIT "5aea90014bbf269c4f2e09243fcd502e28174be7")
+set(DEPTHAI_DEVICE_SIDE_COMMIT "665f8773f7269292eabd2ee562debb2a778776a4")


Check what is latest develop, so we dont downgrade.

MaticTonin · 2026-02-26T15:28:49Z

cmake/Depthai/DepthaiDeviceRVC4Config.cmake


 # "version if applicable"
-set(DEPTHAI_DEVICE_RVC4_VERSION "0.0.1+d5a6e66b0e2a72a400ac6a4cd1dc79eb033fa68a")
+set(DEPTHAI_DEVICE_RVC4_VERSION "0.0.1+5ee962d36af9c2f04ec126aa10cc8083dba61dde")


Check what is latest develop, so we dont downgrade.

include/depthai/pipeline/datatype/AutoCalibrationResult.hpp

bindings/python/src/pipeline/datatype/AutoCalibrationResultBindings.cpp

tests/src/ondevice_tests/pipeline/node/auto_calibration_test.cpp

include/depthai/pipeline/datatype/AutoCalibrationConfig.hpp

MaticTonin · 2026-02-26T15:37:22Z

include/depthai/pipeline/datatype/AutoCalibrationResult.hpp

+        : dataQuality(dataQuality), calibrationConfidence(calibrationConfidence), passed(passed), calibration(calibration) {};
+    virtual ~AutoCalibrationResult();
+
+    double dataQuality;


Done here: 9808e7368

Same comments as above.

include/depthai/pipeline/datatype/AutoCalibrationConfig.hpp

src/pipeline/Pipeline.cpp

src/pipeline/node/AutoCalibration.cpp

moratom · 2026-02-27T06:47:09Z

@JakubFara this doesn't have the packetization right?

moratom

Thanks, exitcting!

Left a few comments/questions.
I think we should add packetization here as well (on the bridge created after Gate node).

src/pipeline/Pipeline.cpp

moratom · 2026-02-27T07:34:33Z

src/pipeline/Pipeline.cpp

+    if(buildInternalQueue) {
+        // Starts pipeline, go through all nodes and start them
+        for(const auto& node : getAllNodes()) {
+            if(node->runOnHost()) {
+                node->buildInternalQueues();
+            }
+        }
+    }


Why do we need this internal queues?
Linking subnodes together without using createOutputQueue() doesn't work?

I have a subnode, and I would like to create subnode->output.createOutputQueue() inside the parent node.
The problem is that if I use :
- parent::buildInternal(){subnode->output.createOutputQueue();}
The parent on the subnode is uninitialized and points to null.
- parent::start(){subnode->output.createOutputQueue();}
The pipeline is already build so I can not create the queue anymore.
If I shift the start() before the pipeline is build it works, but it could cause some issues with other pipelines.

I have changed it to buildStage1, and it does the same.

Could we use the "conventional" way and add inputs to the parent node and then do a normal link between the subnode and the parent node?

The why for my ask is mostly that createOutputQueue() approach won't work if we later mate this node device runnable on RVC4.

Also - pipeline graph will show the structure much nicer in that case as the information gets lost with createOutputQueue() (the information on how the internal connections look like)

I did not realize this option. Fixed here: 9041e3a67 Replace createOutput/InputQueue with linking

include/depthai/pipeline/node/AutoCalibration.hpp

moratom · 2026-02-27T07:37:28Z

src/pipeline/node/AutoCalibration.cpp

+    auto outputCameraLeft = cameraLeft->requestIspOutput(20);
+    auto outputCameraRight = cameraRight->requestIspOutput(20);


Any specific reason to go with 20?
I think going with "default" makes more sense here, to avoid having troubles syncing.

changed here: f94689e Set fps in ISP to default

src/pipeline/node/AutoCalibration.cpp

JakubFara · 2026-02-27T08:54:17Z

#1695 (review)

I’ve identified a bottleneck in the current packetization logic. If the transmission time exceeds the message arrival rate, buffer overflows occur, leading to message loss. This is unintended behavior.
My Solution: Implement adaptive packetization. The system should estimate the arrival time of the next message and dynamically adjust the packet intervals to ensure they are processed within that timeframe.

moratom · 2026-02-27T13:02:58Z

I’ve identified a bottleneck in the current packetization logic. If the transmission time exceeds the message arrival rate, buffer overflows occur, leading to message loss. This is unintended behavior.
My Solution: Implement adaptive packetization. The system should estimate the arrival time of the next message and dynamically adjust the packet intervals to ensure they are processed within that timeframe.

Not sure I understand, XLink streams internally are blocking, where does the buffer overflow happen?
What should happen is XLink creates upstream blockage (stops receiving frames from the input queue).

Can you elaborate on which buffer overflows?

JakubFara · 2026-02-27T13:55:08Z

Can you elaborate on which buffer overflows?

If you send messages slower than they are coming, you have a problem (on the XLlink side, the input is not blocking, or on the sender side if it is blocking). Because of that, I think that the FPS should not be set in the XLinkOut stream. You can simply set it on the Gate side in this case, though.

moratom · 2026-03-03T12:43:25Z

src/pipeline/Pipeline.cpp

    if(isBuild) return;
+    // start ---Add AutoCalibration block---
+    std::unique_lock<std::mutex> lock(pipelineBuildMutex);


Was this intentional?

The idea before was to lock before isBuild is checked, to avoid a race condition here.

I will move it before the check

e3558c285 Add mutex lock on top of the pipeline.build

moratom · 2026-03-03T12:44:07Z

src/pipeline/Pipeline.cpp

    if(isBuild) return;
+    // start ---Add AutoCalibration block---
+    std::unique_lock<std::mutex> lock(pipelineBuildMutex);
+    auto autoCalibtationString = utility::getEnvAs<std::string>("DEPTHAI_AUTOCALIBRATION", "");


We need to add this to the README.md

Done here: 6cc999c34 Add DEPTHAI_AUTOCALIBRATION flag to README

JakubFara requested review from MaticTonin, majvan, moratom and pheec February 26, 2026 14:56

MaticTonin requested changes Feb 26, 2026

View reviewed changes

majvan reviewed Feb 26, 2026

View reviewed changes

include/depthai/pipeline/datatype/AutoCalibrationConfig.hpp Outdated Show resolved Hide resolved

majvan reviewed Feb 26, 2026

View reviewed changes

src/pipeline/Pipeline.cpp Show resolved Hide resolved

majvan reviewed Feb 26, 2026

View reviewed changes

moratom added the testable PR is ready to be tested label Feb 27, 2026

JakubFara force-pushed the feature/dynamicCalibrationOnStart branch 2 times, most recently from 4ef8d97 to 610ef67 Compare February 27, 2026 07:24

moratom reviewed Feb 27, 2026

View reviewed changes

JakubFara force-pushed the feature/dynamicCalibrationOnStart branch 3 times, most recently from 95fcff9 to c0a8b3a Compare February 27, 2026 12:20

JakubFara force-pushed the feature/dynamicCalibrationOnStart branch 3 times, most recently from de12aa5 to 67272b5 Compare March 2, 2026 15:48

moratom reviewed Mar 3, 2026

View reviewed changes

JakubFara and others added 6 commits March 3, 2026 16:01

add DynamicCalibrationWorker

029f899

Add DynamicCalibrationWorker to pipeline::build

caaa35b

Add public Gate constructor

c4af8b1

Bump RVC2 FW

ab39e36

Bump RVC4 FW

a6d0e54

Update the parameters in DCWorker

76bcd78

JakubFara and others added 16 commits March 3, 2026 16:01

Update BYTES_PER_SECOND_LIMIT

8452199

Improve number of Pools

7140b18

Add logging

28646f7

Remove artefact after rebasing

9602b24

Add check if device even exsists

a8a676d

Add tryGetCalibration to calibrationHandler

db08e99

Check that the calibration exists in the handler

d863a13

Lower number of iterations and maxImgPerRecalibration

feb9c69

Fix small typos

fa8c5aa

Add check for correct calibration

0ce13b1

Comment start and end of the AC block in the pipeline::build

cbb166f

Add empty case in the DEPTHAI_AUTOCALIBRATION

df89f50

Add pools only for rvc2

4d1278b

Add DEPTHAI_AUTOCALIBRATION flag to README

c38d287

Add mutex lock on top of the pipeline.build

bfac3eb

Improve mainLoop() checks

c52f725

JakubFara force-pushed the feature/dynamicCalibrationOnStart branch from e206ded to c52f725 Compare March 3, 2026 15:06

JakubFara and others added 5 commits March 3, 2026 16:13

Remove setters and getters from the AutoCalibtationResult/Config

71d12fa

Bump RVC4

7fb87ca

Bump RVC2

66e4243

Fix tests

6ab46a6

Bump DCL to 0.4.0b6

f2d6993

MaticTonin approved these changes Mar 3, 2026

View reviewed changes

MaticTonin added testable PR is ready to be tested and removed testable PR is ready to be tested labels Mar 3, 2026

MaticTonin added 2 commits March 3, 2026 22:10

Fix macOS build

52ed30f

Fix hardcoded FPS values tests

bb0b6c7

MaticTonin added testable PR is ready to be tested and removed testable PR is ready to be tested labels Mar 3, 2026

Improve logging

f6ed8c9

		auto outputCameraLeft = cameraLeft->requestIspOutput(20);
		auto outputCameraRight = cameraRight->requestIspOutput(20);

Conversation

JakubFara commented Feb 26, 2026

AutoCalibration node

Purpose

New AutoCalibration node

Remarks

Testing

Uh oh!

MaticTonin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moratom commented Feb 27, 2026

Uh oh!

moratom left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JakubFara commented Feb 27, 2026

Uh oh!

moratom commented Feb 27, 2026

Uh oh!

JakubFara commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

JakubFara commented Feb 27, 2026 •

edited

Loading