Build Stereo Camera

Usage  |  Controls

The Build Stereo Camera node can be used to estimate the relative transformation between the left and right-eye cameras in a stereo pair. This will generate a camera for each eye to pass down-stream to other nodes. Nodes such as the Disparity Solver node (when using a non-zero Stereo Weight value) and the Disparity-To-Depth node require properly calibrated left and right-eye cameras to function correctly.

Note that it is not necessary to build the stereo cameras before using the Disparity Solver node when the Stereo Weight parameter is set to zero.

Usage

The Build Stereo Camera node can function in two ways: If the left-eye of the stereo pair has been tracked using the matchmoving tools available in PFTrack, it can be used to estimate the relative transformation from the left to the right-eye camera and produce a right-eye camera that is positioned correctly to view the 3D scene. This is illustrated in the screenshot below, where the left-eye camera has been tracked first and the Build Stereo Camera node is then used to position the right-eye.

If no camera tracking has been performed, it can be used to estimate the relative transformation from the left to the right-eye camera and produce a pair of cameras that are fixed in 3D space. Such a pair of cameras can still be used with the Disparity Solver and Disparity-To-Depth nodes, as the relative transformation between the left and right-eyes will match the image data. The only difference being the 3D mesh data generated by the Disparity-To-Depth node will not be positioned correctly in 3D space if the camera is moving.

Generally, therefore, it is only necessary to track the left-eye of the stereo pair before using the Build Stereo Camera node if depth values generated by the Disparity-To-Depth node are ultimately to be exported as a 3D mesh instead of as 2D depth maps.

In order to perform tasks such as estimating an unknown focal length, or estimating the unknown translation and rotation values for the stereo camera, PFTrack will generate a set of points in a frame and attempt to match them between the left and right-eye images.

It is important that the points picked in each frame are well distributed over the image area and the scene. If such a set of points cannot be generated automatically, additional points can be placed manually.

In situations where the left-eye camera has not already been tracked and the camera focal length is unknown, it must be estimated before the stereo transformation can be generated.

Controls

Camera

Focal Length: The focal length for the camera at the current frame. The focal length for the left and right-cameras in the stereo pair are assumed to be the same. The displayed units can be changed using the focal length units menu. Note that in order to enter a focal length in a physical unit such as millimetres, the camera film back size must be set correctly. This can be done by changing the camera preset in the Clip Input node.

Estimate: Estimate the camera focal length at the current frame. The focal length can be estimated at multiple frames in the clip if the Variable option is enabled to allow a variable focal length.

Focal Range: Initial estimate of the the minimum and maximum possible focal length in the clip. These values are used when estimating the focal length by clicking the Estimate button and the resulting focal length will be somewhere between the minimum and maximum values. When these edit boxes are empty, no minimum or maximum focal length will be used.

Variable: When this option is enabled, the focal length will be allowed to vary throughout the clip.

Override camera focal length: Override the focal length for the camera using the values displayed in the Focal Length edit box. When this option is disabled, the camera focal length will be the same as the left-eye input camera attached to the first input of the node.

Stereo Transformation

Interocular: Distance between the left and right-eye cameras in the stereo pair. The units are the same as in which the input left-eye camera has been tracked.

Constant: Set the interocular distance to be a constant value for the entire clip. When this option is disabled, the interocular distance is allowed to vary.

Interocular Lock: When enabled, the interocular distance is not allowed to change.

Translation: The translation direction between the left and right-eye cameras in the stereo pair along the X, Y and Z axes. Note that this vector is normalised (i.e. it will always have a length of one unit) and the actual distance between the cameras is controlled using the Interocular distance.

Translation Lock: When enabled, the translation direction will be locked and is not allowed to change.

Rotation: The rotation between the left and right-eye cameras in the stereo pair around the X, Y and Z axes.

Rotation Lock: When enabled, the rotation will be locked and is not allowed to change.

Constant translation and rotation: Translation direction and rotation angles are assumed to be constant throughout the clip. When this option is disabled, the translation and rotation will be allowed to vary for each frame.

Points

Pick Threshold %: The threshold used to pick points from the left and right-eyes when attempting to estimate the stereo transformation between the cameras. The default value is 50%, and decreasing this parameter will mean more points are picked.

P: Preview the point matches in the Cinema window. This shows the points that have been picked in both the left and right-eye images at the current frame. These points will be used to estimate the stereo transformation between the left and right-eye cameras.

Search Range: The horizontal and vertical search range (measured in pixels) that is used to match points between the left and right-eye cameras. These values should be set correctly for a clip to ensure that points are matched accurately over the entire image. The default values are 150 pixels horizontally, and 25 pixels vertically.

Marquee: Select points in the Cinema window by clicking and dragging with the left mouse button to draw a selection marquee. Holding the Shift key will allow multiple sets of points to be selected at the same time, and holding the Ctrl key will allow selection to be performed using a lasso instead of a rectangle.

Delete: Delete all selected points.

Generate: Generate a set of points at the current frame using the current Pick Threshold % and Search Range parameters.

ROI: Adjust the region of interest (ROI) for the current frame using the left mouse button. When generating points using the Generate button, points will only be generated inside the ROI.

Place: Allow a new point to be placed manually in the left and right-eye images. Clicking with the left mouse button in the Cinema window will place a point in one eye and select it. After changing the displayed clip to the other eye (either by using the Current clip menu or the appropriate keyboard shortcut), the point can be placed again by clicking with the left mouse button. This will place the point whilst keeping it selected. To place the point and de-select it (thereby allowing additional points to be placed, hold the Ctrl key whilst clicking with the left mouse button.

Edit: Allow existing points to be edited by clicking and dragging with the left mouse button to adjust the point location in either the left or right-eye images.

Clear: Clear all points from the current frame.

Clear All: Clear points from all frames.

Match Tracker: Allow an existing tracker point to be matched in either the left or right eyes, thereby automatically calculating the interocular distance between the cameras. Clicking the left mouse button will select an existing tracker point. Clicking again with the left mouse button in the right-eye camera frame whilst holding the Ctrl key will allow that tracker point to be placed and will update the estimate of the interocular distance at that frame. Holding the Alt/Option key whilst clicking in the right-eye camera frame will remove the point from the frame.

Solver

Set Keyframe/Unset Keyframe: Place a keyframe at the current frame. Unless a parameter is constant throughout the clip, the stereo interocular, translation and rotation transformation values will be estimated at each keyframe.

Clear Keyframes: Clear all keyframes from the clip.

Point distance: Specify a known distance between two points in a frame and update the Interocular distance between the cameras accordingly. The points must be selected manually using the Marquee tool and it is assumed the stereo transformation is accurate enough to correctly triangulate the 3D point positions.

Solve Frame: Solve for the stereo transformation parameters at the current frame. If no points are available, they will be generated automatically using the current Points parameters. If no keyframe is available at the current frame, one will be set automatically.

Refine Frame: Refine the estimate of the stereo transformation at the current frame. This can be used after points have been placed manually, edited or deleted using the Points controls.

Refine All: Refine the estimate of the stereo transformation at all frames.

Display

Current clip: The left or right-eye of the stereo pair currently displayed in the Cinema.

Points type: The way that points are displayed in the Cinema window. Options are None to display no points, Left/Right-eye matches to show matching points and lines between the left and right-eyes, or Projection errors to colour each point green/yellow/red according to how well it matches between the left and right-eye cameras.

Point size: The scale that points are drawn in the Viewer windows.

Show ground: When enabled, the ground-plane will be displayed.

Show horizon: When enabled, the horizon line will be displayed in the Cinema window.

Show trackers: When enabled, 3D tracker positions will be displayed.

Default Keyboard Shortcuts

Marquee	Shift+M
Delete	Del
Generate	Shift+G
Place	Shift+P
Edit	Shift+E
Match Tracker	Shift+T
Set/Unset Keyframe	Shift+K
Solve Frame	Shift+S
Display None	Ctrl+N
Display Matches	Ctrl+M
Display Errors	Ctrl+E
Show Ground	Ctrl+G
Show Horizon	Ctrl+H
Show Trackers	Ctrl+T
Next Clip	C