AN13-Various filtering applications of Vzense 3D Time of Flight (ToF) camera

Introduction to four types of filtering in ToF camera

Time filtering: For an in-depth understanding of time filtering, please refer to the technical note “TN10-A comprehensive guide to time filter“.
Confidence filtering: For a deep dive into confidence filtering, please refer to the technical note “TN12-A deep learning into confidence filter“.
Spatial filtering: To quickly grasp the concept of spatial filtering, please refer to the technical note “TN13-A quick guide to understanding spatial filter“.
Flying pixel filtering: To understand flying pixel filtering and how it operates, please refer to the technical note “TN14-Understanding flying pixel filter and how it works“.

The application of filtering in ScepterGUITool

Time filtering: The filtering threshold range is [1, 3], with a default threshold setting of 3. The larger the threshold value, the more significant the filtering effect, and the less jitter in the point cloud. Adjust the position as shown in the figure below:

Confidence filtering: The filtering threshold range is [1, 100], with a default threshold setting of 15. The higher the threshold value, the more pronounced the filtering effect, and the more points with poor signal quality are filtered out. However, excessively high thresholds can result in the loss of depth values in the depth map. Therefore, it is necessary to adjust the threshold to an appropriate value based on the actual scene requirements for application. Adjust the position as shown in the figure below:

Spatial filtering: Filtering can be either enabled or disabled, with it being disabled by default. Enabling filtering can reduce unnecessary noise in the image and achieve an overall smoothing effect, but it may also cause image data distortion and blurriness. The adjustment position is shown in the figure below:

Flying pixel filtering: The filtering threshold range is [1,16], with a default threshold setting of 5. The higher the threshold value, the more pronounced the filtering effect, and the more boundary points are filtered out. However, excessively high thresholds can result in the loss of depth values in the depth map. Therefore, it is necessary to adjust the threshold to an appropriate value based on the actual scene requirements for application. The adjustment position is shown in the figure below:

Filtering implementation in software API

Sample code:
BaseSDK/Windows/Samples/Base/NYX650/ToFFiltersSetGet.

Time filtering

API:

//Set time filter parameter
ScStatus scSetTimeFilterParams(ScDeviceHandle device, ScTimeFilterParams params);

//Get the parameters of the time filter feature
ScStatus scGetTimeFilterParams(ScDeviceHandle device, ScTimeFilterParams* pParams);

Notes:
ScDeviceHandle: The handle of the device.
ScTimeFilterParams: Parameters for time filtering.

typedef struct
{
    int32_t threshold;   /*Range in [1, 6],The larger the value is, the more obvious the 
                           filtering effect is and The smaller the point cloud wobble*/
    bool enable;         //Whether to enable time filter，true is open，false is close
} ScTimeFilterParams;

ScStatus: Return value status. Upon successful invocation, it returns SC_OK; otherwise, it returns another value indicating unsuccessful invocation.

Confidence filtering

API:

//Set the parameters of the confidence filter
ScStatus scSetConfidenceFilterParams(ScDeviceHandle device, ScConfidenceFilterParams params);

//Get the parameters of the confidence filter feature
ScStatus scGetConfidenceFilterParams(ScDeviceHandle device, ScConfidenceFilterParams* pParams);

Notes:
ScDeviceHandle: The handle of the device.
ScConfidenceFilterParams : Parameters for confidence filtering.

typedef struct
{
    int32_t threshold;   /*Range in [1, 100]. The larger the value is, the more obvious the 
                           filtering effect is and the more points are filtered out*/
    bool enable;         //Whether to enable confidence filter，true is open，false is close
} ScConfidenceFilterParams;

ScStatus: Return value status. Upon successful invocation, it returns SC_OK; otherwise, it returns another value indicating unsuccessful invocation.

Spatial filtering

API:

//set enables or disables the spatial filter
ScStatus scSetSpatialFilterEnabled(ScDeviceHandle device, bool bEnabled);

//Returns the boolean value of whether the spatial filter feature is enabled or disabled
ScStatus scGetSpatialFilterEnabled(ScDeviceHandle device, bool* pEnabled);

Notes:
ScDeviceHandle: The handle of the device.
bEnabled: The spatial filtering status parameter.
ScStatus: Return value status. Upon successful invocation, it returns SC_OK; otherwise, it returns another value indicating unsuccessful invocation.

Flying pixel filtering

API:

//Set the parameters of the Flying pixel filter
ScStatus scSetFlyingPixelFilterParams(ScDeviceHandle device, const ScFlyingPixelFilterParams params);
//Get the parameters of the flying pixel filter
ScStatus scGetFlyingPixelFilterParams(ScDeviceHandle device, ScFlyingPixelFilterParams* params);

Notes:
ScDeviceHandle: The handle of the device.
ScFlyingPixelFilterParams: Flying pixel filter parameters.

typedef struct
{
    int32_t threshold;   /*Range in [0, 16]. The larger the value is, the more obvious the filtering effect 
                           is and the more points are filtered out*/
    bool enable;         //Whether to enable confidence filter，true is open，false is close
} ScFlyingPixelFilterParams;

ScStatus: Return value status. Upon successful invocation, it returns SC_OK; otherwise, it returns another value indicating unsuccessful invocation.