Stereo Render

Usage  |  Controls

The Stereo Render Node can be used to render stereoscopic left and right-eye views of the scene from depth maps generated up-stream, as well as analyse the range of disparities present in the stereoscopic image and estimate viewing comfort for each pixel in the image.

The Stereo Render node will take the original image and depth map and generate a left-eye image by moving the camera to the left by half the interocular distance, and a right-eye image by moving the camera to the right by half the interocular distance. The convergence point can be set to ensure zero disparity at the desired location.

Both the left and right-eye cameras will also be generated and passed down-stream. Note that images rendered using these cameras in other applications may not align exactly with the left and right-eye images generated by the Stereo Render node (especially when scales and/or offsets have been applied to the disparity values). This is because the left and right-eye images generated by this node are produced by moving pixels horizontally along each scanline, whereas the left and right-eye cameras are true planar-perspective cameras.

Usage

Defining the stereo camera

The stereo camera used for rendering is defined by two additional parameters to those provided by the input camera. The interocular distance is the distance between the left and right-eye cameras. Each of the left and right-eye cameras is positioned by offsetting the input camera horizontally by half the interocular distance and aiming towards the convergence point. The interocular distance is a constant value that remains fixed for the entire shot. By default, the interocular distance is 65mm.

The convergence point is a point directly in front of the input camera, whose distance from the input camera position is defined as the convergence distance. If required, the convergence distance can be varied throughout the shot, or the cameras can be set to converge at infinity, providing parallel left and right-eye views as shown in the figure below.

Rendering

The Stereo Render node can render stereoscopic images in two ways: First of all the Display Mode menu can be used to select either Left-Eye, Right-Eye, Anaglyph Side-By-Side or Top-And-Bottom to display in the Cinema window. When displaying the red/blue anaglyph render in the Cinema window, the colour saturation of the input image can be controlled using the Saturation % parameter which can make viewing easier for scenes that contain a wide variety of colours.

In-painting

Regions of the image that become exposed whilst rendering the left or right-eye views are in-painted automatically during the rendering process.

Spatial in-painting

All exposed regions are first filled using spatial in-painting. The Edge Dilation and Edge Soften parameters can be used to control this. In situations where the exterior object boundary does not entirely cover the object, artefacts can occur when in-painting. By dilating the object depth map slightly, these artefacts can be reduced as shown in the image below.

The Edge Soften parameter can be used to soften rendered edges slightly at significant depth discontinuities. The difference in depth that must occur to trigger a depth discontinuity is specified by the Depth % parameter, as a percentage of the distance between the near and far camera planes. The image below shows how edge softening can improve the rendered output (in particular on the right-hand side of the actor's neck).

All pixel areas that are in-painted spatially can have a grain pattern rendered over them to reduce visible artefacts. The grain parameters are described in more detail below.

Temporal in-painting

In addition to spatial in-painting, which attempts to fill holes using pixel colours taken from the current frame only, temporal in-painting can be employed to fill holes using pixel data from other frames. Temporal in-painting can be enabled by increasing the Lookahead value above zero. This specifies the number of frames on either side of the current frame that will be examined to see if pixels colours can be used to fill holes in the current frame.

Note that for temporal in-painting to work correctly, the motion of the camera must be known. Enabling temporal in-painting will also increase the time it takes to render each frame

For example, in the screenshot below, the actor is walking from left to right, and when rendering the left eye image pixels on the left side of the actor's body are exposed and require in-painting. Because the stereo disparity in this shot is relatively large, the results of spatial in-painting can be seen relatively easily in the left-hand image. By enabling temporal in-painting, these pixels can be filled more accurately using colours from nearby frames, as shown in the right-hand image.

The Disparity Histogram

The disparity histogram shows a yellow graph at the right-hand edge of the Cinema screen that illustrates the distribution of disparity values that are being used to render the left-eye and right-eye stereoscopic images. The disparity histogram can be used to analyse how image depths are converted into a stereoscopic image and ensure that the disparities generated are distributed correctly for the shot.

The range of disparity values can be both positive and negative, depending on whether pixel have depth values smaller or larger than the convergence distance. The graph shows the number of occurrences of each disparity value in the left-eye image (the disparity values in the right-eye image are identical but reversed). The top of the graph represents the disparity of the most distant pixel in the image, and the bottom of the graph represents the disparity of the nearest pixel. The zero disparity point represents the convergence distance of the stereoscopic camera. Note that the disparity histogram can be represented using a logarithmic scale to make it easier to see the distribution

Stereoscopic Viewer Comfort

The stereoscopic viewer comfort tools can be used to estimate how comfortable it will be to view the stereoscopic image under different viewing conditions than are available when using a computer monitor to display the stereoscopic image.

The Viewer Distance and Screen Width parameters should be set to the viewing settings for which comfort will be estimated. For example, if viewing comfort is to be estimated for a projection screen that is 15 feet wide, and an observer sitting 10 feet away from it, those values should be entered into the screen width and viewer distance edit boxes.

Enabling the Show comfort violations option will display a false-coloured image in the Cinema window, ranging in colour from green (comfortable viewing) to red (uncomfortable viewing). These values are estimated by assuming the observer's interocular distance is fixed at 65mm, and measuring the angle at which the observer's eyes must reach in order to fuse the stereoscopic image at each pixel depth. When the fusion angle exceeds a value generally assumed to indicate visual discomfort, the Cinema overlay is coloured red.

The images below illustrate how factors such as the convergence distance can affect viewing comfort for a scene. In the first image, the convergence distance is set so the left and right-eye cameras converge on the actor. The comfort overlay is shown inset at the bottom-right of the anaglyph image, and indicates that the entire image can be viewed comfortably.

When the convergence distance is moved towards the camera, however, the disparity values for the scene are greatly increased, and the comfort overlay turns red for most of the image. This indicates that the observer's eyes may have trouble fusing the stereoscopic image in the background of the shot.

In cases where viewing comfort exceeds allowable thresholds, the Disparity Scale and Disparity Offset rendering parameters can be used to artificially scale the disparity values to generate a comfortable image, or to offset the disparity values which has the effect of moving the entire scene forwards or backwards from the viewer relative to the display surface.

Controls

Camera

Interocular: The interocular distance (i.e. the distance between the left and right-eye camera views) used to generate the stereoscopic image.

Near plane: The near camera plane. The R button resets incoming value.

Far plane: The far camera plane. The R button resets the far plane to the incoming value.

Convergence: The current convergence distance (i.e. the distance at which the left and right-eye cameras are aimed). When the C button is enabled, the convergence distance is constant throughout the entire shot. The F button will fix the convergence distance at the current position in 3D space.

The convergence curve is available for editing in the Curves Editor when the C button is not enabled. When convergence has been fixed in 3D space, the convergence curve available in the Curves Editor represents a relative distance compared to the fixed position. Finally, to enforce an infinite convergence distance and use parallel cameras, enable the Parallel option.

Finally, convergence can also be adjusted interactively in a Viewer window by holding the Ctrl key and clicking and dragging with the left mouse button.

Export FBX: Export an Autodesk FBX file containing a special stereo camera representing both the left and right-eye viewpoints.

In-paint

Lookahead: The number of frame ahead and behind the current that will be used to in-paint occluded regions. Note that for temporal in-painting to perform best, the path of the camera must be tracked in 3D space because in-painting relies on knowing the camera position for each frame to determine which pixels can be used to fill holes. Increasing this value will mean holes can be filled using data from more distant frames, but will also increase the time it takes to render each frame. Setting this value to zero will mean no temporal in-painting is performed.

Step: The frame step for each lookahead frame. Increasing this value can be used to increase the range of frames used for in-painting without impacting too severely on rendering time. For example, if lookahead is set to 3 and step is set to 1, in-painting frame 10 will use data from the three frames on either side of frame s10 (i.e. frames 7, 8, 9, 11, 12 and 13). If step is set to 2, each lookahead will be doubled, so it will use data from frames 4, 6, 8, 12, 14 and 16. This means the renderer can take data from more distant frames without increasing the number of frames that must be processed.

Edge Dilation: The number of pixels by which the depth map will be dilated before edge in-painting occurs. This can be used to decrease halos around object edges in situations where the depth map is not covering the entire object.

Edge Soften: The amount of softening that will occur along edges that contain significant depth discontinuities which can often cause aliasing artefacts when rendered. The Depth % parameter specifies the relative difference in pixel depth (as a percentage of the distance between the near and far camera planes) that is considered to be a depth discontinuity.

Render

Disparity Scale: The scale factor applied to all pixel disparities during rendering. This can be used to artificially increase or decrease the stereoscopic effect. When enabled, C button indicates that the disparity scale factor is constant throughout the entire clip instead of being allowed to vary from frame-to-frame. When the C button is switched off, the disparity scale curve is available for editing in the Curves Editor.

Disparity Offset: The offset applied to all pixel disparities during rendering. This can be used to artificially move the perceived position of the scene forwards and backwards compared to the viewing surface (i.e. the computer monitor or projection screen). When enabled, C button indicates that the disparity offset is constant throughout the entire clip instead of being allowed to vary from frame-to-frame. When the C button is switched off, the disparity offset curve is available for editing in the Curves Editor.

Red, Green, Blue Grain: The size and intensity of the grain pattern that will be applied to areas that are spatially in-painted. The size represents the approximate size (in pixels) of each grain element, and the intensity percentage represents the amount of colour variation introduced into the image by the film grain.

Grey-scale: When enabled, a single grain pattern will be used to render into the red, green and blue image channels. When disabled, a different pattern will be used for each channel.

Gang RGB: When enabled, adjustments to the grain parameters in a single colour channel will be applied to the other colour channels.

Edit Profile: Display the Grain Profile editing window.

The Grain Profile Editor

The horizontal axis corresponds to pixel value, and the vertical axis to intensity response. By default, the profile is a horizontal line, which means that dark and light pixels use the same amount of grain, corresponding exactly to the parameters. The response can be adjusted so that darker or lighter pixels can use more or less grain as required. Drag the control vertices around with the left mouse button to edit the curve.

To insert a new vertex, click on the curve with the left mouse button. To make a vertex linear, drag the Bezier handles into the vertex, and to expand the handles out of a linear vertex, hold the Shift key and drag with the left mouse button. To remove a vertex, click on it with the right mouse button.

Reset: Reset the profile to its default state. The Channel menu allows the profile to be adjusted separately for each colour channel if required.

Display

Display Mode: The mage that is displayed in the Cinema window. Options are None to display the original clip, Left Eye or Right Eye will display either the left or right-eye stereo render, Anaglyph will display a red/blue anaglyph that can be viewed using suitable red/blue stereoscopic glasses, Side-By-Side will display the left and right-eye images side by side, and Top-And-Bottom will display the left-eye and right-eye images on top of each other.

Viewer proxy: The resolution used to display the triangular mesh in Viewer windows. For high resolution images, selecting Half, Third or Quarter will increase rendering performance.

Transparency: The amount of transparency that is used to display a grey-scale depth map in the Cinema window.

Grey-scale gamma: The amount of gamma correction that is used to display a grey-scale depth map in the Cinema window.

Saturation %: The colour saturation applied when rendering a stereoscopic anaglyph in the Cinema window. Reducing this to zero will render a red/blue anaglyph using grey-scale images for the left and right eyes.

Depth cut %: The difference in depth values (as a percentage of the distance between the near and far camera planes) at which triangular mesh edges will be cut when displaying the depth map in Viewer windows. Note that this option is for display purposes only and does not affect the depth map being passed down-stream from the node.

Show depth map: When enabled, a grey-scale depth map will be displayed in the Cinema window if one is available for the current frame.

Show disparity histogram: When enabled, a graph showing the frequency of different pixel disparities will be displayed at the right-hand edge of the Cinema window.

Log scale: When enabled, the disparity histogram graph will be displayed using a logarithmic scale.

Render stereo display: When enabled, the left and right-eye renders will be displayed on the secondary stereo display screen if one is available.

Cache Z-Depth from upstream: When enabled, z-depth data from up-stream will be cached in the Stereo Render node. This can be useful in situations where z-depth data takes a significant amount of time to calculate. By enabling this option, the data will be cached locally and used to accelerate the rendering process whilst rendering parameters are adjusted. Note that enabling this option will reduce the amount of RAM available for caching rendered stereoscopic image data.

Comfort Zone

Viewer Distance: The viewer distance used to estimate stereoscopic viewer comfort. This is the distance between the display screen and the observer who is viewing the screen.

Screen Width: The size of the display screen used to estimate stereoscopic viewer comfort. This is the width of the display screen on which the stereoscopic image is being shown.

Show Comfort Violations: When enabled, the Cinema window will display a coloured overlay that approximates how comfortably the stereoscopic image can be viewed by an average viewer. The overlay colours can vary from red in regions that are uncomfortable to view, to green in regions that can be viewed comfortably. The size of the display screen and the distance between the screen and observer must be set using the Screen Width and Viewer Distance edit boxes.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Display None	Ctrl+1
Display Left-Eye	Ctrl+2
Display Right-Eye	Ctrl+3
Display Anaglyph	Ctrl+4
Display Side-By-Side	Ctrl+5
Display Top-And-Bottom	Ctrl+6
Display Interlace Left-Eye First	Ctrl+7
Display Interlace Right-Eye First	Ctrl+8
Show Depth Map	Ctrl+D
Show Disparity Histogram	Ctrl+H
Show Comfort Violations	Ctrl+C