Match Camera

Usage  |  Controls

The Match Camera node will automatically track and solve multiple moving cameras into a scene that has been surveyed photographically using the Photo Survey node.

This avoids the need to manually scale and orient each camera path to align it with the scene point cloud, as would be required if the movie camera was tracked and solved separately using the Camera Solver node.

The result of this process is a single scene representation containing a point cloud with photo cameras (from the Photo Survey node), and as many solved and aligned movie cameras as desired.

Feature tracking and camera solving occurs as a single process, and does not require any user intervention or manual feature tracking, although features can be tracked manually up-stream to assist the movie camera solver if required.

When surveying the scene using the Photo Survey node, photographs should be taken to cover as much of the scene viewed by the movie clips as possible. Whilst the solver used to position the movie cameras is able to operate with limited amounts of data, ensuring the Photo Survey point cloud contains many points that are visible to each movie camera will give the best results.

The screenshot below shows a scene surveyed using the Photo Survey node along with two movie clips that have been tracked and solved automatically into the same scene space:

Usage

Before the Match Camera node can be used, the scene must first be surveyed using digital photographs, and the Photo Survey node used to estimate the location of each image and produce a moderately dense point cloud representing details in the scene (see the Photo Survey node documentation for more details).

Once a photographic survey is available, the overall scene scale and orientation can be set once, if desired, using an Orient Camera node immediately below the Photo Survey node, as shown in the screenshot above. Note that this is not a required step, but may help to ensure the scene ground plane is positioned and scaled appropriately.

The survey photos and point cloud should be attached to the first input of the Match Camera node, as shown in the screenshot above. Each movie clip can then be attached to additional inputs.

There is no limit on the number of movie clips that can be attached to a single Match Camera node. Multiple Match Camera nodes can also be used for a single scene, provided they are all attached to the same Photo Survey node to ensure the scale and orientation of the scene remains the same for each.

Initialising the Photo Database

Before a movie camera can be tracked into the scene, the photo collection database must be initialised. This generates the data that will be used to automatically locate the movie camera into the scene space, and may take a few seconds or a few minutes to finish (depending on the number of images used to survey the scene).

Note that the database need only be initialised once for each Match Camera node, but if the point cloud or camera data coming from up-stream is changed, the database must be re-initialised to ensure the movie cameras are matched correctly into the scene.

Tracking and solving movie cameras

The camera and lens parameters for each movie clip can be viewed in the Camera and Auto-Undistort tabs (if automatic lens distortion correction is active).

The camera focal length can be set to a known value, if desired, or can be set to an initialised range to help the solver if the focal length is not known exactly.

Tracking is performed in several stages, the first being an automatic feature tracking process that identifies points in the movie clip that can be tracked from frame-to-frame.

Features will also be selected from the movie clip to try and locate the camera position within the scene generated by the Photo Survey node. Features that are matched will be displayed in the Cinema window, and the corresponding scene points highlighted in the Viewer windows.

Manually tracked features placed up-stream between the Match Camera node and the movie clip node will also be used during tracking to improve the solve. Note that it is not necessary to track any features unless required, but for certain types of scene, placing one or two user-tracked features in specific locations can be used to reduce jitter in the solve, particularly if the movie camera is viewing parts of the scene that were not particularly well covered by the Photo Survey node.

Accounting for lens distortion

In order to get the most accurate estimate of camera motion, it is often important to account for any lens distortion present in the footage. This can be done in several ways:

- Pre-correcting lens distortion the movie clip using the Clip Input node - Shooting calibration grids and building a distortion preset using the Movie Camera Preset editor - Using Automatic distortion correction and specify approximate bounds on the distortion coefficient.

Provided the distortion present in the footage is not too great, the third option can often provide an accurate result, and can be enabled by switching on the Estimate buttons in the Auto-Undistort controls.

Is is important to provide approximate bounds on the distortion coefficient when using automatic estimation by adjusting the minimum and maximum allowable values. Generally, most camera lenses exhibit barrel distortion, which corresponds to positive distortion coefficient in the range 0.0 (no distortion) to 0.3 (fairly high distortion) with a typical value between 0.05 and 0.15.

Note that enabling automatic lens distortion correction can slow down the camera solver, especially when many different focal lengths are used by the cameras and all three parameters are being estimated.

After lens distortion has been removed, the camera's sensor size will be adjusted to account for the new image resolution.

Controls

Clip Menu: This menu allows the current clip to be changed. The first clip will always correspond to the photo collection and point cloud provided by the Photo Survey node. Additional clips are the movie clips that are to be solved into the scene.

Initialize: Initialise the database for the photo collection and point cloud. If the Shift key is held whilst clicking this button, the initialisation process will be performed in the background.

Camera

The camera controls contain information about the current movie camera.

Focal length: The camera focal length for the current camera. Focal length can be set as Known, Unknown or Initialised. If the focal length of the camera is known beforehand,entering the value here can often improve both the speed and accuracy of the solver. Setting focal length to Initialised will allow a minimum and maximum value to be specified in the Focal range edit boxes, and an initial value to be entered into the Focal length edit box. Note that in order to enter a focal length measured in any unit other than Pixels requires that the camera sensor/film back size is set correctly.

Focal range: When Focal length is set to Initialised or Unknown, these edit boxes define the minimum and maximum allowable values of focal length.

Variable focal: When enabled, the camera focal length will be allowed to vary for each frame.

Focal adjust: When using focal lengths set from metadata, this value can be used to allow the focal length to vary slightly from the exact metadata values.

Field of view: The horizontal and vertical field of view for the current camera, measured in Degrees.

Sensor size: The horizontal and vertical sensor/film back size. To change sensor size, edit the camera preset values in the Clip Input node.

Pixel aspect: The current pixel aspect ratio. To change the pixel aspect ratio, edit the camera preset value in the Clip Input node.

Near/Far clip planes: Adjust the near and far clipping planes for the current camera to reduce the number of point cloud points that are displayed in the Cinema window once the camera has been solved. Note that these values do not affect feature tracking or the camera solve, and are for display purposes only.

The Translation and Rotation menus can be used to specify various hints and constraints for how the camera is moving. The first menu is used to control the amount of smoothing that is applied to either the translation or rotation components of motion. Smoothing options are None, Low, Medium and High.

The second menu can be used to specify either a constraint on the motion (for example, 'No Translation'), or indicate that a hint should be used. Translation constraints are as follows:

- No Translation: the camera is not translation at all, and there is no parallax in the shot at all. Note that this option is rarely used, since it refers to the position of the camera's optical centre. Even when mounted on a tripod, the camera will still be translating slightly, since the centre of rotation (the tripod mount point) will not correspond exactly to the camera's optical centre.

- Unknown: A freely moving camera

- Metadata hint: When available, use metadata in the source media to provide a hint to camera translation

- Upstream hint: Use the upstream camera translation as a hint (for example, generated manually using the Edit Camera node).

Rotation constraints are similarly named.

Focal smooth: Specify how smooth the camera focal length changes are for cameras with a variable focal length. Options are None, Low, Medium and High.

Auto-Undistort

Distortion: The current lens distortion coefficient, along with minimum and maximum allowable values (defaults are 0.0 and 0.1). The minimum and maximum values should be specified before the camera positions are solved. Enabling the Estimate option will allow the distortion coefficient to be estimated automatically during the solve.

After solving the camera, the image in the Cinema window will autmatically adjust in size to represent the full undistorted image. If desired, the size of the image can be fixed to match the original image by enabling the Crop to input image size option.

After solving, the actual distortion value calculated for the current frame is displayed to the right:

Handy Tip: if you aren't sure what distortion range to use, try guessing first, solving your camera and then look at the calculated distortion value. If it's at the maximum of your range this means PFTrack tried to increase it more but couldn't.

In this case, adjust the Range control upwards by one setting to increase the maximum allowable value, solve again, and see if that gives better results.

For example, if you set the Range to Moderate (0.1 to 0.2) and then solve your shot and the final distortion estimate is 0.2, try changing the Range to Significant and solving again to see if this gives a better result. You can always undo afterwards if the result isn't any better.

Camera Matching

Start frame: The first frame of the movie clip that will be tracked. If the edit box is empty, the first frame of the clip will be used.

End frame: The last frame of the movie clip that will be tracked. If the edit box is empty, the last frame of the clip will be used.

Tracker count: The number of features that are automatically tracked from frame-to-frame in the movie clip. Decreasing this value may accelerate processing, but may also reduce accuracy. The P button will display a preview of the feature locations and their tracked positions for the current frame.

Feature threshold %: The threshold for the selection of features that are picked from a movie frame and matched to the point cloud to locate the camera in the scene. The P button will display preview points in the Cinema, and the corresponding cloud points will be highlighted in the Viewer windows.

Channels: These options can be used to specify the red/green/blue image channels to use for feature tracking, and also how attached masks are to be used.

Tracking Prediction: The method used to predict tracker motion from one frame to the next:

- None: No tracking prediction will be used

- Using Image: the overall image data motion will be used to predict how individual trackers will move (this is the default)

- Using Metadata: if the metadata is available describing approximate camera motion, this will be used to predict how trackers are moving from frame-to-frame.

- Both: Both metadata and image-based prediction will be used.

Back-track: When enabled, features will be tracked from frame-to-frame in the movie clip between the start and end frames and then back again, ensuring the tracking points exist in as many frames as possible. This will increase processing time but will also increase the length of feature tracks and improve overall accuracy.

Clear Trackers: Clicking this button will remove all trackers from the node.

Edit ROI Allow editing of the region of interest (ROI) in the Cinema window. This specifies the image area inside which points will be detected. When enabled, the ROI can be adjusted by clicking and dragging with the left mouse button in the Cinema window.

Reset: Reset the matching parameters to their default states.

Match Camera: Start the feature tracking and camera solving process for the current movie clip. The Cinema window will be updated to display intermediate results after points are detected and matched in each image. If the Shift key is held whilst clicking this button, the matching process will be performed in the background.

Display

Show ground: When this is enabled, the ground plane will be displayed in the Cinema and Viewer windows.

Show trackers: When this is enabled, tracking points generated in the node will be displayed.

Show point cloud: When this is enabled, the point cloud provided by the Photo Survey node will be displayed in the Cinema and Viewer windows.

Show photo cameras: When this is enabled, the camera positions provided by the Photo Survey node will be displayed in the Viewer windows.

Show movie cameras: When this is enabled, the camera path and frustum for the current movie camera will be displayed in the Viewer windows.

Show frustum: When this is enabled, the near/far clip planes for the current movie camera will be displayed in the Viewer windows.

Show distortion grid: When this is enabled, the distortion grid will be displayed as an overlay in the Cinema window.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Show Ground	Ctrl+G
Show Trackers	Ctrl+T
Show Point Cloud	Ctrl+I
Show Photo Cameras	Ctrl+P
Show Movie Cameras	Ctrl+M
Show Frustum	Ctrl+F
Show Distortion Grid	Ctrl+D