Z-Depth Mesh

Usage  |  Controls

Note: this is an experimental node intended for use with depth maps captured by an external depth-sensor device.

This node can be used to generate a triangular mesh from a tracked camera and sequence of depth maps. The triangular mesh is generated at a specific voxel resolution, defined in real-world units. It is therefore important to ensure your camera path and depth maps are defined at the correct scale. The Attach Z-Channel node can be used to combine the RGB and depth clips into a single image stream.

Usage

The following tree shows an example of how the Z-Depth Mesh node can be used to in combination with the Attach Z-Channel node to import z-depth data.

An Auto Match and Camera Solver node are used to generate tracking points with a z-depth value at each frame, read from the depth maps attached to the clip:

Because z-depth data is now available, a z-depth value will be assigned to each tracker in each frame, and used by the Camera Solver node to help track the camera. This will also set a real-world scale according to the units used to define depth in the z-channel input clip.

Lastly, the Z-Depth Mesh node is attached. This node uses the camera motion provided by the Camera Solver node to combine z-depth data from each frame into a triangular mesh, as shown below:

In order to reconstruct a mesh, the overall bound of the scene must first be set. This is achieved by clicking the Edit Bound button and adjusting the bounding box in the Viewer windows using the left mouse button.

Then the Voxel Resolution menu should be set to specify a suitable resolution for the mesh. Note that the amount of memory and time required to build the mesh will be significantly increased for very small resolution voxels. The total number of voxels that will be used during reconstruction is shown in the Cinema window, and this is calculated from the size of the bound and the voxel resolution setting. Voxel counts above 1000 in all directions may take a significant amount of memory and processing time to generate.

Once the initial mesh is built, it will be simplified if necessary to a maximum number of triangles specified in the Max. Triangle Count edit box, and a UV map will also be generated. The mesh will be passed down-stream for further processing. Note that meshes with very high triangle counts are likely to decrease performance of down-stream nodes and increase memory usage significantly.

If texturing is required, a Texture Extraction node can be attached down-stream.

Using the Z-Depth Solver

Alternatively, the Z-Depth Solver node can also be used after tracking the camera to generate depth maps:

These can be combined into a mesh using the same process described above.

Controls

Solver

Voxel Resolution: The voxel resolution used to generate the triangular mesh, assuming one world unit corresponds to one metre. As this value is adjusted, the actual voxel resolution contained within the scene bound is displayed in the Cinema window.

Max. Triangle Count: After the mesh is generated, it will be simplified to this number of triangles if required before the UV map is generated.

Edit Bound: Allow the solver bound to be adjusted in the Viewer windows by clicking a dragging with the left mouser button. Holding the Ctrl key will allow the far side of the bound to be adjusted instead of the nearest face.

Start: Start the process of converting depth maps into a triangular mesh, using the current scene bound and resolution parameters. Conversion will start from the current frame, and continue forwards towards the end of the clip or until the Stop button is clicked in the progress window.

Clear: Delete the mesh and remove the data file from disk.

Display

Viewer proxy: The resolution used to display triangular meshes and point clouds in Viewer windows. For high resolution images, selecting Half, Third or Quarter will increase rendering performance.

Transparency %: The amount of transparency that is used to display a grey-scale depth map in the Cinema window.

Grey-scale gamma: The amount of gamma correction that is used to display a grey-scale depth map in the Cinema window.

Depth cut %: The difference in depth values (as a percentage of the distance between the near and far camera planes) at which triangular mesh edges will be cut when displaying the depth map in Viewer windows. Note that this option is for display purposes only and does not affect the depth map being passed down-stream from the node.

Show depth map: When enabled, a grey-scale depth map will be displayed in the Cinema window if one is available for the current frame.

Show mesh: When enabled, the 3D mesh will be displayed.

Show mesh colour: When enabled, the 3D mesh will be displayed with captured vertex colours.

Show mesh shading: When enabled, the 3D mesh will be shaded using captured surface normals.

Show ground: When enabled, the ground-plane will be displayed.

Show trackers: When enabled, the solved tracking points will be displayed in the Viewer windows.

Show geometry: When enabled, the geometric mesh objects will be displayed in the Viewer windows.

Show frustum: When enabled, the camera frustum and path will be displayed in the Viewer windows.

Show bound: When enabled, the scene bound will be displayed in the Viewer windows.