Z-Depth Tracker

Usage  |  Controls

The Z-Depth Tracker Node can be used to track depth maps from one frame to the next. This tracking can be performed with or without the help of an existing camera track, although for static scenes the results obtained with a camera track will be superior to those obtained without.

As z-depth is tracked, it is written to disk in a floating-point file format. Typically, for a 2048x1556 image, these files occupy about 13Mb of disk space for each frame.

Usage

After z-depth values have been generated for one frame, you can track those values into nearby frames using the tracking tools. This can be useful in situations where z-depth values are required to be temporally consistent from frame-to-frame. Z-Depth values can be tracked forwards or backwards from the current frame.

The spatial and position weights affect how depth values are tracked. When the spatial weight is non-zero, the motion of pixels in the image plane (calculated using optical flow) will be used to track depth values. Tracking with a non-zero position weight should only be done when accurate camera motion is available and the scene does not contain any moving objects, because it assumes that camera positions are accurate from one frame to the next and uses this information to update the depth values from frame-to-frame.

Generally, when tracking depth images for a static scene, the position weight should be large, assuming your camera motion is accurate. For situations where camera motion is less accurate, or not available at all, increasing the Spatial weight may provide improved results.

When tracking z-depth values for a moving camera viewing a static scene containing no moving objects, it is often preferable to use a high position Weight. Conversely, if depth values have been painted by hand for a scene that contains moving objects, or is viewed by a camera that is not tracked accurately, a high spatial Weight should be used because this reduces the reliance on camera data to inform the tracking.

The image below shows a situation where depth values have been tracked using a Spatial Weight of 100 and a Position Weight of 0. The camera in this shot is stationary and the actor's face is
moving. Original image frames are shown as thumbnails.

The initial depth map for this shot was generated using the Z-Depth Edit node.

Masks can be used to specify areas that should be excluded from tracking, or define object boundaries across which no tracking will be performed.

Using Masks

Masks can be used to exclude areas of the image from depth map tracking, or to indicate boundaries between independent objects. The ordering of individual masks is important, because it specifies the relative depth ordering of the objects defined by each mask. Masks at the top of the mask list define objects that are farther away from the camera than masks at the bottom of the list.

When a mask is set to Exclude, pixels covered by the mask will be excluded from depth map tracking. When a mask is set to Boundary, pixels covered by the mask are assumed to belong to an object which will only be tracked into pixels from the same object in other frames.

Controls

Camera

Near plane: The distance of the near plane from the camera. Depth estimates will be generated between the near and far planes, so this value should be set appropriately before solving for any depth maps.

Far plane: The distance of the far plane from the camera. Depth estimates will be generated between the near and far planes, so this value should be set appropriately before solving for any depth maps.

Columns

Name: The name of each active mask.

Colour: The mask overlay colour for each active mask. Double click in this column of the selected mask to change its overlay colour.

State: Exclude or Boundary. When set to Exclude, no depth estimate will be produced for pixels covered by the mask. When set to Boundary, no smoothing will be performed across the edge of the mask. Right-click in this column to change the behaviour of each mask

Controls

Move Closer: Move the selected mask closer towards the camera. Note that the ordering of masks is important only if they overlap.

Move Away: Move the selected mask away from the camera. Note that the ordering of masks is important only if they overlap.

Track

Channels: The image channels to use when tracking z-depth.

Spatial Weight %: The weight given to the spatial motion of pixels (calculated using optical flow) when attempting to track depth values from one frame to the next.

Position Weight %: The weight given to the 3D position of the pixel as viewed from the previous or next frame's camera position when attempting to track depth values from one frame to the next.

<<, <, >, >>: Track depth maps backwards or forwards by one or more frames at a time. Tracking can be stopped by clicking the Stop button or by pressing the Escape key on the keyboard.

Auto-fill missing depths: When enabled an auto-fill algorithm will be used to fill missing depth values for each frame that is tracked. Missing depth values may appear at the edges of the frame during tracking, as existing depth values move out of view and expose areas of the image that do not have depth values associated with them.

Clear: Clear the depth map from the current frame and delete the data file from disk.

Clear All: Clear depth maps from all frames and delete the data files from disk.

Display

Display mode: The method that will be used to display depth maps in Viewer windows. The options are Triangle mesh, which will render a triangular mesh containing every pixel in the image, and Point Cloud which will render a point for every pixel in the image.

Viewer proxy: The resolution used to display triangular meshes and point clouds in Viewer windows. For high resolution images, selecting Half, Third or Quarter will increase rendering performance.

Grey-scale gamma: The amount of gamma correction that is used to display a grey-scale depth map in the Cinema window.

Depth cut %: The difference in depth values (as a percentage of the distance between the near and far camera planes) at which triangular mesh edges will be cut when displaying the depth map in Viewer windows. Note that this option is for display purposes only and does not affect the depth map being passed down-stream from the node.

Transparency: The amount of transparency that is used to display a grey-scale depth map in the Cinema window.

Show ground: When enabled, the ground-plane will be displayed.

Show horizon: When enabled, the horizon line will be displayed.

Show depth map: When enabled, a grey-scale depth map will be displayed in the Cinema window if one is available for the current frame.

Show depth mesh: When enabled, a triangular mesh or point cloud will be displayed in both the Cinema and Viewer windows. If no depth map is available for the current frame, the depth map for the nearest available frame will be displayed.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Clear	Ctrl+L
Clear All	Shift+L
Track Backwards	[
Track Backwards Frame	Shift+{
Track Forwards Frame	Shift+}
Track Forwards	]
Show Ground	Ctrl+G
Show Horizon	Ctrl+H
Show Depth Map	Ctrl+D
Show Depth Mesh	Ctrl+P