Z-Depth Object

Usage  |  Controls

The Z-Depth Object node can be used to outline specific objects in the input clip and change the depth values of those objects using various modifier shapes and image processing techniques. Modified depth maps are then passed down-stream from the node. Note that a camera must be present in order to use this node. If no camera has been tracked, one can be created using the Edit Camera node.

In addition to the spline tools available in the Z-Depth Object node, splines can also be imported from Silhouette¹ and Mocha Pro².

The screenshots below illustrate how several objects can be used to add depth to a scene (in this case, from a static camera position). The left-hand image shows object boundaries overlaid on the background image. Note that for elements such as the chain in the background, only an approximate boundary has been drawn, and the image processing tools available in the Z-Depth Object node have been employed to extract the object of interest. The middle image shows the depth map obtained for the frame, and the right-hand image shows the scene rendered from an alternative view-point.

Usage

Creating objects

Objects are either an X-Spline or a Bezier spline.

Each X-Spline boundary vertex has two control handles: the outer handle is drawn in blue and adjusts the smoothness of the spline, and the inner handle is drawn in white and adjusts the distance between the interior and exterior boundaries. The distance between the inner and outer boundaries can be adjusted for all vertices using the Edge width edit box.

Bezier boundary vertices have left and right control handles which affect the curve of the spline at each vertex. Bezier object boundaries can have their interior and exterior boundaries adjusted independently.

Objects can be created directly in the Z-Depth Object node, or alternately the Roto node can be used and attached to the Z-Depth Object node.

Object hierarchies

Objects can be created hierarchically. Child objects will inherit the tracking transformation from their parent, and have depth and thickness values that are relative to their parent depth. This can be used to create an object hierarchy, where the entire tree of objects can be positioned in 3D space by adjusting the parent depth value, avoiding the need to re-position all child objects.

Child objects can also have their boolean state changed to Subtract, allowing holes to be removed from the parent depth map.

Objects can be re-parented or re-ordered in the list by clicking and dragging with the left mouse button and then releasing the mouse button to drop the object at the desired position.

Drawing object boundaries

Object boundaries are defined using splines, and can be animated by hand or tracked through the clip.

The image below illustrates the different edge types available for each object, along with the inner and outer object boundaries. Note that the adaptive edge type is able to adapt the object shape to the underlying image data without the need to draw an intricate boundary.

Painting object boundaries

Object boundaries can also be painted using the tools available in the node.

Once the object edge has been painted the paint stroke will be converted into an X-Spline with interior and exterior boundaries. These boundaries will be fit to the outside and inside of the paint stroke.

The screenshots below show a paint stroke and the X-Spline generated after conversion.

Editing object boundaries

Object boundaries can be animated manually through the shot, or can be tracked using the tracking tools described below. As keyframes are placed for each vertex, they will be displayed in the scrub bar as dark yellow vertical markers. All boundary vertices are also available for editing in the Curve Editor.

Detail keying

The detail keyer can be used to modify the opacity of pixels inside an object boundary in order to discount pixels that belong to the background. This can be useful in situations where the object is very detailed and would otherwise require a very intricate boundary to be drawn with many holes.

An example is given below, where a chain has been modelled using a simple boundary spline on the left, and the detail keyer has been activated to remove background pixels on the right.

Note that the detail keyer will function best when there is a significant difference between the colour of the foreground object and the colour of the background.

Adjusting object depth and thickness

Once an object boundary has been positioned in at least one frame of the clip, it can also be positioned in 3D space. This must be done to ensure the object is rendered correctly into the depth map that is passed down-stream from the node.

The images below illustrate how a hierarchy of objects (all parented to the actor's head) can be positioned in 3D space by adjusting the depth of the parent object interactively in the perspective viewer window.

By default, objects have zero depth thickness which means each object is represented using a constant depth value which generates a flat plane in 3D space. Increasing the object's thickness allows the depth values to vary within the object. Thickness values can be positive or negative, where positive thickness values extend depth further away from the camera and negative thickness values extend depth towards the camera.

Note that for all modifiers except Planar 3D, an object must have a non-zero depth thickness value before the modifier will have any effect on the depth map.

Ramp:

The Ramp modifier can be used to construct a linear z-gradient in image space. Note that because of perspective projection, this will not translate into a flat planar surface in 3D. The orientation and size of the depth gradient can be changed by adjusting the manipulator.

Radial:

The Radial modifier can be used to construct a circular or elliptical change in depth. The orientation, scale and size of the depth gradient can be changed by adjusting the manipulator.

Roto:

When using the Roto modifier, an additional spline boundary can be drawn and depth will vary smoothly between the inner and outer spline boundaries.

Blobs:

With the Blobs modifier, one or more blobs can be placed on the object, and the scale, position and orientation individually adjusted for each blob.

Corner:

The Corner modifier will create two linear z-depth gradients, much like the corner of a building. Note that because of perspective projection, this will not translate into flat planar surfaces in 3D. The orientation and size of the depth gradients can be changed by
adjusting the manipulator, and the centre-point of the corner can be adjusted by dragging at the mid-point of the central vertical line.

Cylinder:

The Cylinder modifier will create two curved z-depth gradients. The orientation and size of the depth gradients can be changed by adjusting the manipulator, and the centre-point of the corner can be adjusted by dragging at the mid-point of the central vertical line.

Planar 3D:

The Planar 3D modifier works in a different way to the others. This modifier does not manipulate the z-depth values for each pixel directly, but instead generates z-depth values in such a way that the object is represented by a flat planar surface in 3D. This manipulator can be used to adjust the position and orientation of the planar surface. To adjust orientation, hold the 3D Plane Manipulator Toggle keyboard shortcut ('L' by default).

Image-based depth modifiers

In addition to assigning a modifier type, various image processing operations can also be applied to the depth map for each object. The Depth Blur Radius parameter (measured in pixels) can be used to blur the depth map.

When the BG button is not enabled, the blur will only be calculated using pixels already assigned to the object. This will mean that blurring a flat depth map will have any effect. Enabling the BG option will allow background pixels to contribute to the blur, so a flat depth map will become rounded at the edges, as shown in the screenshot below.

When blurring an existing depth map with the BG option enabled, the sign of the blur radius affects how the depth map will be blurred. A positive radius will blur edges towards the object's unmodified depth, and a negative radius will blur edges towards the objects' thickness value, as illustrated in the screenshot below.

The Depth Offset value (in the range -1.0 to 1.0) will offset the object's depth map before blurring. The offset value is relative to the object's thickness, so a depth offset of 1.0 will offset by the entire thickness value.

The Depth Dilate/Erode value can be used to dilate or erode the depth map. A positive value will dilate the depth map towards the thickness point, and a negative value will erode the depth map away from the thickness point.

Finally, the Thickness Falloff curve can be used to modify the transition in depth between the object's depth and thickness points. By default, the curve is a straight line from (0,0) to (1,1).

Image mapping

Image mapping can be used to fetch detail from the background image and apply as a modification to the depth map. This can be used to introduce small-scale variations in depth based on the background pixel colours. For example, the rear wall in the following scene has a fixed depth at each pixel:

The following image illustrates the change in depth applied to the rear wall when image mapping is enabled:

Controls

Mask upstream z-depth: When enabled, the object boundaries will be used to mask the depth map coming from up-stream instead of modifying it. Pixels covered by an object boundary will be passed down-stream, and pixels excluded from all objects will have their depth values removed.

Object List

The object list contains all objects that have been defined in the node and displays the object name which can be edited by double-clicking, the colour which can be changed by double clicking and using the colour selection popup window, the in and out points for the object, as well as the boolean state, depth in the current frame, and the modifier state.

The order in which objects are listed can be changed by clicking and dragging an object using the left mouse button, and then dropping it in the desired position in the list.

Create: Create a new un-named object. If another object is selected, the new object will be created as a child object. Child objects inherit the tracking transformation of their parent, and have depth values relative to the parent object. Child objects can also have their boolean state set to Subtract in order to remove pixels from the parent object.

Delete: Delete the selected object.

In: Set the in-frame for the selected object to the current frame.

Out: Set the out-frame for the selected object to the current frame.

Hide/Show: Hide or show the selected object. Hidden objects do not modify z-depth values down-stream from the node.

Lock/Unlock: Lock or unlock the selected object. Locked objects cannot be modified in any way.

Import: Display a file dialog allowing a Silhouette Shapes XML file or a Nuke 6.2³ Roto file generated by Mocha Pro to be imported. Note that only Nuke Roto files exported by Mocha Pro are currently supported. Additionally, Z-Depth objects exported from another node in the .pfo format can also be imported.

Export: Display a file dialog allowing Z-Depth objects in the .pfo format to be exported. This can be used to move objects to another Z-Depth Object node in another project.

Rand: Randomise the colours used to display the colour overlay for each object.

Edit

Depth: The depth value for the selected object in the current frame. If the selected object is a child, this depth value is relative to the parent depth. When enabled, the C button indicates that the depth value is constant throughout the clip. When enabled, the F button indicates that the object depth is fixed in 3D space rather than being relative to the current camera position. When the C button is not enabled, the depth curve is available for editing in the Curves Editor. When object depth has been fixed in 3D space, the depth curve available in the Curves Editor represents a relative depth compared to the fixed position.

Thickness: The object depth thickness in the current frame. When applying a modifier to an object, the object must have a non-zero depth thickness value. When enabled, the C button indicates that the thickness value is constant throughout the clip. When the C button is not enabled, the thickness curve is available for editing in the Curves Editor.

Edge type: The type of edge for the selected object. Options are Hard which generates a hard-edged object boundary, Soft which generates a soft-edged object boundary that varies between full opacity at the interior object boundary and zero opacity at the exterior object boundary, or Adaptive which varies opacity between the interior and exterior boundaries according to image content. When using Soft or Adaptive edge types, the transition between full opacity at the interior boundary and zero opacity at the exterior boundary can be controlled using the Edge Falloff curve.

Edge width: The distance between the interior and exterior object boundaries when using a Soft or Adaptive edge type. Set updates the interior object boundary. When enabled, the C button indicates that the edge width value for each vertex is constant throughout the clip.

Edge Smoothness %: The smoothness used to transition the adaptive edge opacity from full opacity at the interior boundary to zero opacity at the exterior boundary.

Edge channels: The image channels used to determine the adaptive edge falloff between the inner and outer boundaries for the selected object.

Draw X: Allow the selected object boundary to be drawn in the Cinema window using the left mouse button to place X-Spline vertices. Pressing the Escape key or clicking again on the first vertex will close the boundary and finish drawing.

Draw B: Allow the selected object boundary to be drawn in the Cinema window using the left mouse button to place Bezier vertices. Whilst placing a vertex, hold the left mouse button and drag the cursor outwards to extend the Bezier handles. Pressing the Escape key or clicking again on the first vertex will close the boundary and finish drawing.

Edit: Allow the object boundary to be edited in the Cinema after the selected object has been drawn. Edit a vertex by clicking and dragging the boundary control handles using the left mouse button. The entire object can also be transformed by holding the transform boundary keyboard shortcut (J by default) to display the transformation widget. Multiple vertices can be selected by clicking and dragging the selection marquee using the left mouse button.

Set Default: Store the current settings for depth, thickness, edge type, edge width, edge smoothness, edge channels and edge falloff, and use them as defaults for all new objects.

Add: Add new keyframes to the object at the current frame. When the Edit button is enabled, keyframes will be added to the boundary vertices. When the Edit button is not enabled, keyframes will be added to the object depth and thickness.

R-, Rem, R+: Remove keyframes from the object. R- removes all earlier keyframes, Rem removes the current keyframe, and R+ removes all later keyframes. When the Edit button is enabled, keyframes will be removed from the boundary vertices. When the Edit Boundary button is not enabled, keyframes will be removed from the object depth and thickness.

K-, K+: Move to the previous or next keyframe. When the Edit button is enabled, these buttons will move the previous/next boundary vertex keyframe. When the Edit button is not enabled, these buttons will move to the previous/next object depth and thickness keyframe.

Edge Falloff: The edge opacity falloff for Adaptive and Soft edge types, controlling the transition between full opacity at the interior boundary and zero opacity at the exterior boundary. New points can be inserted into the curve by clicking with the left mouse button, and the curve points and handles can be adjusted by clicking and dragging with the left mouse button (Note: the first and last points of the curve cannot be adjusted). To delete a point, click on it with the left mouse button.

Zero threshold: The threshold at which the object edge transparency will be set to zero, effectively removing pixels from the object between the interior and exterior boundaries (default value is 0.01). Pixels removed from the object do not affect the depth map passed down-stream.

Paint

Brush size: The brush size used to paint around an object outline.

Paint Boundary: Allow an object boundary to be specified by painting around the object using a circular paint brush. After painting around the object boundary, press the Escape key to convert into an X-Spline with interior and exterior boundaries.

Tracking

>>, >, <, <<: Track an object boundary forwards or backwards through the clip. The > and < will track the boundary backwards or forwards by one frame at a time, whilst << and >> will track the boundary backwards or forwards by multiple frames at a time. To halt tracking, press the Escape key or click one of the Stop buttons.

R-, Rem, R+: Remove tracking keyframes. R- removes all earlier tracking keyframes, Rem removes the current tracking keyframe, and R+ removes all later tracking keyframes.

Use pixels %: The percentage of pixels used to estimate the tracking transformation. Reducing this value will eliminate pixels that do not contribute significantly to the tracking result, speeding up the tracking process. Reducing this value too much, however, may decrease tracking accuracy..

Outliers %: The approximate number of outlier pixels (i.e. pixels that should be excluded when tracking) when using the Robust tracking mode described below.

Edit ROI: Adjust the region of interest (ROI) that defines which pixels in the image will be used for tracking. The ROI is displayed as a magenta rectangle in the Cinema window and can be adjusted by clicking and dragging with the left mouse button.

Use incremental tracking: When enabled, the tracking template (i.e. the image that is being matched in each frame) will be updated at every frame. When not enabled, the reference template is always taken from the frame in which tracking was started. Depending on the type of footage being tracked, incremental tracking may perform better because the reference template is able to adapt to changes in image content. However, this can also mean that the object boundary position can drift when tracking over many frames.

Translate: When enabled, the object translation will be updated during tracking.

Rotate: When enabled, the object rotation will be updated during tracking.

Scale: When enabled, the object scale will be updated during tracking.

Shear: When enabled, the object shear will be updated during tracking.

Perspective: When enabled, the object will be tracked using a full 4-point planar perspective transformation, which is suitable for tracking planar surfaces viewed from a moving camera.

Robust: When enabled, object pixels that become occluded whilst tracking or change significantly due to differences in shading will be excluded from the tracking process. The approximate number of excluded pixels should be entered in the Outliers % edit box.

Illumination: When enabled, small changes in image brightness will be compensated for during tracking. Note that the compensation for brightness is assumed to be the same for all pixels inside the mask.

Depth: When enabled, the pixels in the depth map that are closer to the camera than the current layer will be ignored during tracking. This can be useful when multiple objects are being defined. By creating layers in a front-to-back order, occlusions due to depth can be better handled when tracking.

Detail Key

Enable detail keying: Enable detail keying for the selected object.

Threshold %: The threshold used to distinguish foreground colours from the background, with a default of 50%. Increasing this value will remove fewer pixels from the object, and decreasing the value will remove more pixels from the object. When enabled, the C button indicates that the threshold remains constant throughout the entire clip. When the C button is not enabled, the threshold can vary throughout the shot, and is available for editing in the Curves Editor.

Smoothness %: The smoothness of the detail keyer. Increasing this value will attempt to ensure that isolated pixels are not present in the object after the background has been removed.

Erode / Dilate: Erode or dilate the object matte after the background has been removed. The edit boxes specify the number of pixels to erode or dilate the object matte, and the <> button can be used to change the order of operations, performing either the erode or dilate operation first.

Sample BG: Allow sampling of the background colours to be performed, by clicking and dragging with the left mouse button in the Cinema window to draw a selection marquee. Holding the Alt/Option key will allow sampling to be performed using a lasso instead of a rectangular selection areas. As each area is sampled, the previous samples will be removed from the keyer unless the Ctrl key is held.

Clear Samples: Clear all colour sampled from the detail keyer.

Modifier

Mode: The depth modifier applied to the current object. Available options are None, Ramp, Radial, Roto, Blobs, Corner, Cylinder and Planar 3D.

Modify: Allow the current depth modifier to be adjusted by interacting with the depth modifier widget that is displayed in the Cinema window (or in the 3D viewer windows in the case of the Planar 3D depth modifier). When enabled, the C button indicates that the depth modifier remains constant throughout the clip.

Add: Add a new keyframe to the depth modifier at the current frame.

R-, Rem, R+: Remove keyframes from the depth modifier. R- removes all earlier keyframes, Rem removes the current keyframe, and R+ removes all later keyframes.

K-, K+: Move to the previous or next depth modifier keyframe.

Depth offset: The relative offset (in the range -1.0 to 1.0) that is applied to the object's depth map before blurring.

Depth blur radius: The amount of blurring applied to the object's depth map, measured in pixels. When the BG button is enabled, the blur will account for background pixels as well as those assigned to the object which can be used to round-off the object's depth map. The blur radius can be positive or negative, and this influences the direction in which the depth map is rounded off when the BG button is enabled.

Depth dilate/erode: The dilate or erode radius for the object's depth map. Negative values will dilate the depth map (i.e. replace further pixels with nearer ones), and positive values will erode the depth map (i.e. replace nearer pixels with further ones).

Image map: Enable or disable image mapping, which will take pixel colours from the background image and use them to modify the object's depth map.

Channels: The image channels (red, green, blue or alpha) used for image mapping.

Scale / Offset: The scale and offset which is applied to the image intensity before modifying the object's depth map.

Thickness Falloff: The thickness falloff used to modify the object's depth map, controlling the transition between the object's near and far distances. New points can be inserted into the curve by clicking with the left mouse button, and the curve points and handles can be adjusted by clicking and dragging with the left mouse button (Note: the first and last points of the curve cannot be adjusted). To delete a point, click on it with the right mouse button.

Camera

Near plane: The current camera near plane. Far plane: The current camera far plane.

Display

Solver proxy: The resolution at which depth maps will be calculated for each object. Using a reduced resolution can often increase interactive performance when using high resolution clips.

Viewer proxy: The resolution at which the depth map will be rendered in the 3D viewer windows. Using a reduced resolution can often increase rendering speed when displaying high resolution clips.

Transparency: The amount of transparency that is used to display a grey-scale depth map in the Cinema window.

Grey-scale gamma: The amount of gamma correction that is used to display a grey-scale depth map in the Cinema window.

Depth cut %: The difference in depth values (as a percentage of the distance between the near and far camera planes) at which triangular mesh edges will be cut when displaying the depth map in 3D viewer windows. Note that this option is for display purposes only and does not affect the depth map being passed down-stream from the node.

Show ground: When enabled, the ground-plane will be displayed.

Show horizon: When enabled, the horizon line will be displayed.

Show colour: When enabled, objects will be displayed in the Cinema window using a false colour overlay.

Show depth: When enabled, a grey-scale depth map will be displayed in the Cinema window.

Show all objects: When enabled, colour or depth maps will be displayed for all objects, instead of only the selected object.

Interactive update: When enabled, object overlays will be generated for all frames whilst changing frames in the scrub bar. When this option is disabled, object overlays will only be updated once the mouse button is released when scrubbing.

Show input depth: When enabled, the input depth map will be displayed in the Cinema and 3D viewer windows.

Show boundaries: When enabled, exterior object boundaries will be displayed for all objects that are not selected.

Show handles: When enabled, object roto handles will be displayed for the selected object.

Show ROI: When enabled, the region of interest (ROI) will be displayed in the Cinema window.

Bezier vertex context menu

Linear: No handles displayed.

Continuous When enabled, both the left and right handles will be adjusted together. Disabling this option will allow the left and right handles to be adjusted independently.

Mouse controls

Click and hold left mouse button when drawing Bezier spline vertices

Enlarge the Bezier handles for each vertex

Right click Bezier vertex

Display the Bezier vertex context menu

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Create	C
Hide/Show	H
Draw X-Spline	S
Draw Bezier	B
Edit	E
Paint Boundary	P
Modify	O
3D Plane Manipulator Toggle	L
Add Key	Shift+?
Remove Earlier Keys	;
Remove Key	'
Remove Layer Keys	#
Move To Earlier Key	N
Move To Later Key	M
Track Backwards	[
Track Backwards Frame	{
Track Forwards Frame	}
Track Forwards	]
Remove Earlier Tracking Keys	Shift+(
Remove Tracking Key	Shift+
Remove Later Tracking Keys	Shift+)
Show Ground	Ctrl+G
Show Horizon	Ctrl+H
Show Colour	Ctrl+L
Show Depth	Ctrl+D
Show All Objects	Ctrl+A
Show Input Depth	Ctrl+E
Show Boundaries	Ctrl+R
Show Handles	Ctrl+Q
Show ROI	Ctrl+I
Interactive Update	Ctrl+S
Transform Boundary	J
Increase Brush Size	+
Decrease Brush Size	-