Grid artifact related to SSE instructions

[mdukaczewski]

We are using D435 and have issue with depth images obtained by Python interface.
In the past we used 2.14.0.85 librealsense installed by pip. There are four-part versions on pip.
We got images like this (please notice irregular grid):

We trained our Machine Learning models based on this data.
Now we try to use our models on Nvidia Jetson TX2.
Because of ARM architecture we are not able to install librealsense by pip.
We checkout versions by tags, and compile. There are only three-part versions on github. We tried:

• v2.13.0
• v2.14.0
• v2.14.1

But all those versions are giving us images like this:

What is the cause and what should we do to obtain images like the first one (with grid)?
Thank you!

PS. There is exactly the same python code obtaining both of images shown in this post. We think that the difference is because of different behavior of pyrealsense2.align function: in pip and github version of librealsense.

[dorodnic]

Hi @mdukaczewski
First, this is a rather specific issue, sorry for your trouble.
The grid is an artifact that was first introduced by mistake in #2040 (first released in 2.14.0)
and mostly resolved in #2217 (2.15.0)
As far as I can tell 2.14.1 should still include this bug.
Alternatively, you could try checking out the specific commit:
git checkout 7abb777fec9dba494fabbf5d18dfdf901a1aba0e
To force this issue, all you need is for the following condition in align.cpp to take the second branch.

[mdukaczewski]

@dorodnic thank you for your answer. Unfortunately it looks that grid artifact is related to SSE code path. So we are not able to reproduce it on Jetson device. We are able to reproduce it on x86 though, but it's not sufficient for us.

[dorodnic]

Sorry to hear.
In the non-SSE code, I believe the corresponding point would be align.cpp:32-58
If instead of filling a polygon corresponding to the depth pixel, you always map depth to a single color pixel, something like:

float depth_pixel[2] = { depth_x, depth_y }, depth_point[3], other_point[3], other_pixel[2];
rs2_deproject_pixel_to_point(depth_point, &depth_intrin, depth_pixel, depth);
rs2_transform_point_to_point(other_point, &depth_to_other, depth_point);
rs2_project_point_to_pixel(other_pixel, &other_intrin, other_point);
const int other_x0 = static_cast<int>(other_pixel[0] + 0.5f);
const int other_y0 = static_cast<int>(other_pixel[1] + 0.5f);
transfer_pixel(depth_pixel_index, other_y0 * other_intrin.width + other_x0);

This should restore the bug on resolutions where depth has lower pixel density.