Object Solver

Overview  |  Influencing The Solve  |  Solver Controls  |  The Coverage Panel

Overview

The Object Solver node can be used to estimate the motion of an independently moving rigid object after camera motion has been generated.

The Object Solver node can have one or two inputs and outputs, and it can solve for an object visible in two cameras provided the set of trackers have been tracked in each input clip. Camera motion is never changed during an object solve.

In order to track a moving object, the trackers for that object must be placed in their own tracking group, independent from the main camera group. This should be done in either the User Track, Auto Match or Auto Track node that generated the trackers. If trackers have already been generated in the camera group by mistake, they can quickly be switched to a new group without having to re-track.

The object solving process can be influenced by the user in many ways, such as specifying a pair of initial frames to start from or specifying approximate feature distances from the camera. Error graphs are available to assess which features are not being solved accurately.

Note that an alternative strategy to solving for object motion is to use a Camera Solver node to solve for camera motion relative to a static object, followed by a Make Object node to change that camera motion into object motion. Objects can also be tracked in the Geometry Track node without using tracking points.

The Object Solver works in a similar way to the Camera Solver, and needs at least three trackers to solve for object motion at each frame. Using more trackers will generally produce a better result (provided they are tracked well) and will reduce noise and jitters in the object motion.

Once features are tracked upstream, object motion can be solved for by clicking the Solve Object button.

Note that when tracking an object from a single viewpoint, the overall scale of the object cannot be determined automatically and must be manually adjusted after the object motion has been solved. This is done using the Distance From Camera orientation mode, whilst viewing the scene in a Viewer window.

Influencing The Solve

The Object Solver works by first constructing a partial solution between the two initial frames. In order to get a good overall track, it is important that this initial solution is fairly accurate, so the first step in troubleshooting a problematic shot is to ensure a good initial solution can be obtained. This will then be extended outwards, adding more frames until the entire object path is complete.

There are several steps that can be taken to influence the object solve and refinement process, including:

1. Removing bad trackers

If trackers have not been tracked accurately, or are still present in frames where they should not be visible, this can adversely affect the accuracy of the object solve. These trackers should be corrected or removed entirely before solving.

2. Changing the initial frames

The initial frames should be set so that there is a noticeable amount of parallax in the object motion between the initial frames. There also needs to be at least three trackers common to both initial frames for the solver to run.

Either one or two initial frames can be specified manually. If only one is specified, the other will be estimated automatically.

3. Enter some tracker distances

Entering known (or approximately known) distances of trackers from the camera can be used to help obtain a good initial solution, or to correct situations where foreground and background features are confused.

To correct this, enter two or more tracker distances, one in the background and one in the foreground. Fairly large uncertainty values can be used if the actual tracker distances are not known.

3. Setting hard/soft tracker constraints

If a tracker has been generated automatically (by an Auto Track or Auto Match node), and its tracker path is seen to be very accurate compared to other trackers, it can be enabled as a Hard constraint.

This will mean the camera solver tries harder to ensure that the 3D tracker position matches the tracker path. By default, all trackers generated by the User Track node are set to be hard constraints.

4. Increasing tracker weights

Increasing tracker weight values can be used to help the solver focus more on certain trackers, also reducing their projection error (possibly at the expense of increasing error elsewhere, however).

Solver Controls

Current clip: The clip that is being displayed in the Cinema window.

Current group: The tracker group that will be used to solve for object motion. Note that trackers present in the camera group cannot be used to solve for object motion.

Start/end frames: The start and end frame for the object solve. The S button will store the current frame number as either the start or end frame.

Initial frames: The pair of frames to use to construct the initial solution. The S button will store the current frame number as either the first or second initial frame.

Translation Smooth: This option is used to specify how much smoothing to apply to the object translation. Smoothing options are None, Low, Medium and High.

Rotation Smooth: This option is used to specify how much smoothing to apply to the object rotation. Options are None, Low, Medium and High.

Set initial frames automatically: When enabled, the initial frames will be set automatically by searching for a span of frames that contains enough trackers to solve for object motion.

Solve for initial solution only: When enabled, the object solve will stop once the initial solution has been generated. The initial solution is the object motion and tracker positions for all frames between the two initial frames.

Automatically orient bounding box: If this option is enabled, a bounding box will be fit around the solved tracker positions such that the volume of the bounding box is minimised. The orientation of the bounding box only affects the underlying tracker coordinates, before they are transformed by the object motion path and placed in the scene. It will not affect the projection error of trackers in any way.

Preview: When enabled, a preview of the solution at the initial camera frames will be generated. This can be used when adjusting the initial frames to determine which values perform best.

Show matches: When enabled, only trackers that are present in both initial frames will be displayed in the Cinema window. This can be used when adjusting the initial frames to identify situations where there are enough trackers available to generate an initial solution.

Exhaustive: When enabled, the solver will spend more time adjusting the overall solution each time a new frame is added. This can result in better quality solves, but for long clips can also increase processing time significantly.

Solve Object: Solve for object motion and tracker positions. The object solve can be run in the background by holding the Shift key whilst clicking on the Solve Object button.

Refine Object: Adjust tracker positions and object motion to better match the tracker paths. Refinement can be run multiple times, and a longer refinement can be ran by holding the Shift key whilst clicking on the button.

Solve Trackers: Solve for tracker positions only. This can be used in situations where additional tracker have been created up-stream to estimate new tracker positions without re-solving for object motion.

Display

Show Ground: When enabled, the ground plane will be displayed.

Show Horizon: When enabled, the horizon line will be displayed.

Show Geometry: When enabled, the geometric objects from up-stream will be displayed.

Show Names: When enabled, selected tracker names will be displayed.

Show Info: When enabled, selected trackers will have position and residual error information displayed.

Show Bound: When enabled, an oriented bounding box will be displayed around all tracker positions.

Show Path: When enabled, the path of the object's bounding box origin will be displayed.

Marquee: Allow a tracker selection marquee to be drawn in the Cinema window or in a Viewer window. Holding the Ctrl key whilst drawing will ensure that previous selections are kept. Holding the Shift key will allow a lasso selection to be used instead of a rectangle.

Centre View: When enabled, the Cinema window will be panned so the projection of the first selected tracker is fixed at the centre of the window.

Trackers

The tracker list contains information about all trackers passed into the Object Solver node.

Columns

Name: The tracker name.

Active: Indicates whether the tracker is active in the solve or not.

Hard: Indicates whether the tracker path should be considered as a hard or soft constraint on object motion. By default, automatically generated trackers (from an Auto Track or Auto Match node) are defined as soft constraints, and manually placed trackers (from a User Track node) are defined as hard constraints. The object solver assumes that the path provided by a tracker marked as a hard constraint does not contain errors. Trackers that are marked as soft constraints may have small errors in their tracker paths without affecting the overall object motion.

Weight: The weight given to a particular tracker in the solution. The default value is 1.0, but increasing this will mean the object solver expends more effort to match a 3D tracker position to its path, possibly at the cost of decreased accuracy elsewhere. Changing the tracker weight can often help to lock a solution down onto a particular tracker.

Residual: The residual projection error (measured in pixels) for the tracker in the current frame. The projection error is the difference between the tracker path position and the projection of the 3D tracker point onto the camera plane. Ideally, the projection error should be close to zero for each tracker.

Distance: The distance from the current camera frame to the tracker's 3D position in space. This can be useful to detect when the object solver has failed to determine which trackers are in the foreground and which are in the background.

Initialised: Trackers that are present in the first initial frame can have an initial distance estimate provided to help the object solver, and this value can be entered here. Trackers that are not present in the first initial frame are labelled as N/A.

Uncertainty: For trackers that have an initial distance entered in the Initialised column, the range of uncertainty can be entered here. The uncertainty is a measure of how well the initial distance is known, so for an initial distance D and uncertainty U, the range of possible tracker distance will vary from D-U to D+U. Setting the uncertainty to zero will mean the initial distance must be specified exactly.

Controls

Min/max tracker distance: For trackers that do not have an initial distance and uncertainty, a minimum and maximum distance can be entered here to assist the object solver.

Orientation mode: The mode by which to orient the coordinate system of the object group. Options are:

- Translate origin: Display a translation widget allowing the origin of the group's coordinate system to be adjusted in either the Cinema window or a Viewer window.

- Rotate around origin: Display a rotation widget allowing the orientation of the group's coordinate system to be adjusted in either the Cinema window or a Viewer window.

- Distance from camera: Display a scale widget at the current camera position in a Viewer window. Adjusting the scale widget will scale the object group and move it towards or away from the camera.

- Fly: Translate, rotate or scale the group's coordinate system by holding the Alt/Option key and using the right, middle or left mouse buttons in the Cinema window.

All / None: Select all or none of the trackers from the list.

Activate: Activate all selected trackers in the object solver. Active trackers will contribute to the solution. Trackers can also be activated individually by ticking the Active column in the tracker list.

Deactivate: Deactivate all selected trackers in the object solver. Inactive trackers will not contribute to the solution. Trackers can be deactivated individually by un-ticking the Active column in the tracker list.

Hard: Change all selected trackers to Hard constraints. Trackers that are marked as hard constraints are assumed to have accurate tracker paths that do not contain any errors. Trackers can also be set to hard constraints individually by ticking the Hard column in the tracker list.

Soft: Change all selected trackers to Soft constraints. Trackers that are marked as soft constraints may have small errors in their tracker paths without affecting the overall camera motion. Trackers can also be set to soft constraints individually by un-ticking the Hard column in the tracker list.

Push/Pull: When enabled, initialised tracker distances and uncertainties can be set interactively in a Viewer window. Clicking and dragging with the left mouse button will adjust the initialised tracker distance. Holding the Ctrl key will allow the uncertainty to be changed. Initialised distances are shown as purple line segments. By initialising a distance for several trackers, the camera solver can be helped to obtain a better quality solution.

Error Graph

The error graph plots the projection error (measured in pixels) for each tracker in each frame, along with the average projection error for all trackers visible in a frame. The projection error is the difference between the tracker path position and the projection of the 3D tracker point onto the camera plane. Ideally, the projection error
should be close to zero for each tracker. Selected trackers are shown in yellow, and unselected trackers are shown in blue. The average projection error graph is shown in white. The error graph can be translated and scaled by clicking and dragging with the right or middle mouse buttons.

A tracker may have a large projection error for several reasons. If a tracker is marked as a Hard constraint and has a large projection error in most frames, it often means that the tracker path does not correspond to a fixed point in 3D space. This can be caused by a tracker being positioned over a "virtual corner" (i.e. an image feature that looks like a corner in the image, but is formed by the intersection of edges in the scene at different distances from the camera. As the camera or object moves, the apparent intersection of the edges changes). In these cases, the tracker should probably be de-activated.

If a tracker is marked as a Hard constraint, and is tracked accurately but has a projection error that increases significantly in certain frames, it often indicates that the object position in those frames is incorrect. Adding more trackers to the solution, or providing estimates of tracker distances can often help out here.

Trackers that are marked as Soft constraints may also have an small overall error that increases significantly in certain frames. This can often be caused by the tracker jumping onto a different image feature in certain frames, but because of the soft constraint, these frames do not influence the overall average error by much. These jumps can be removed by trimming the error graph using the Trim button (see below).

Show All: When enabled, error graphs will be shown for all trackers, otherwise graph will only be shown for selected trackers.

Trim: Display a trim line, allowing all trackers whose projection errors are larger than a particular value to be ignored during a camera solve or refinement. The trim line can be moved up and down by dragging it with the mouse. Once activated trimming will remain active even if the Trim button is disabled - the Trim button only controls whether the trim line is displayed or not; not whether trimming itself is active. To disable trimming click the D button next to the Trim button (only available when the trim line is not displayed).

Edit Trim Curve: Toggles between moving the trim line as a whole and editing the shape of the trim line to allow more flexible trimming where a single value for the entire sequence is not sufficient. The trim line can be edited using the standard controls for manipulating a Bezier curve. The R button resets the shape of the trim line.

Fit View: Scale and translate the error graph so all graphs are visible.

Click and drag right mouse button in error graph to pan the graph. Click and drag middle mouse button in error graph to zoom the graph (or use the mouse wheel, holding Alt/Option to zoom vertically instead of horizontally).

The Coverage Panel

The Coverage Panel displays information about the frames in which each tracking point has been tracked:

This can be used to evaluate how well tracking points are distributed throughout the clip, which will help to provide an accurate object solve without any jumps in the motion path.

By default, the Coverage Panel displayed keyframe information showing how tracking points have been positioned and tracked. Each frame in which the tracker is present is shown with a blue square. Light-blue squares indicate where automatically generated tracking points were initially placed, and yellow squares show frames in which manually generated tracking points were keyframed.

Frames in which the tracker is visible but has not been positioned are displayed in dark red. It is important to ensure that tracking points have been positioned in all frames in which they are visible, as this can significantly affect the accuracy of the object solve.

Alternatively, the Coverage Panel can also display the projection error for each tracker by clicking the Errors button. This switches the colour-coding of each indicator to show the error of the solved tracking point in each frame. This is colour-coded to show green for less than 0.5 pixels error, yellow for 1.5 pixels and red is greater than 2.5 pixels.

Coverage Panel controls

The Coverage Panel can be panned horizontally or vertically by clicking and dragging with the right mouse button. Clicking and dragging with the middle mouse button will zoom either horizontally or vertically to increase the number of tracking points and frames that are displayed in the panel.

The mouse wheel can also be used to zoom horizontally, or vertically if the Alt/Option key is held.

Clicking on an indicator with the left mouse button will select the tracking point and display that frame in the Cinema window. Holding the Ctrl key will allow multiple tracking points to be selected.

Double-clicking on an indicator with the left mouse button will select the tracking point and immediately switch to display the node which generated that tracking point. This can be used to quickly jump to a User Track node to manually adjust a tracking point to correct a tracking error.

All: Switch between displaying all trackers, or only those trackers visible in the current frame.

Keys: Display keyframe information, showing where targets have been tracked and manually positioned.

Errors: Display projection error information.

Name: Sort the tracking points by name, in alphabetical order.

Start: Sort the tracking points according to the first frame in which they are tracked.

End: Sort the tracking points according to the last frame in which they are tracked.

Hard: When both Hard and Soft constraint trackers are present, this button can be used to switch between displaying all trackers, or only those marked as a Hard constraint.

Fit: Fit the tracking points display to the window. This will zoom in or out as necessary, displaying as many tracking points and frames as will fit in the viewport.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Set First Initial	Ctrl+A
Set Last Initial	Ctrl+L
Preview	Ctrl+R
Solve Object	Shift+S
Solve Trackers	Shift+T
Refine Object	Shift+R
All/None Trackers	Shift+L
Activate	Shift+A
Deactivate	Shift+D
Hard Constraint	Shift+H
Soft Constraint	Shift+O
Push/Pull	Shift+P
Show Ground	Ctrl+G
Show Horizon	Ctrl+H
Show Geometry	Ctrl+E
Show Names	Ctrl+N
Show Info	Ctrl+I
Show Bound	Ctrl+B
Show Path	Ctrl+T
Marquee	Shift+M
Centre View	Shift+C
All Errors	Shift+E
Trim	Shift+I
Edit Trim Curve	Alt/Option+I
Fit	Shift+F
Next Clip	C