User Track

Usage  |  Controls  |  Example ASCII tracker text file used for import and export

The User Track node can be used to manually generate trackers to use for camera or object solving, and can have multiple inputs and outputs.

Trackers can be placed in multiple clips at the same time, enabling multiple cameras to be solved simultaneously, or helper frames to be used to assist with a camera solve (see the Camera Solver section for further information).

Usage

Tracking groups

All trackers generated by a single User Track node are placed into a tracking group. By default, this group is called Camera and corresponds to tracking the overall camera motion. When tracking an object, the group should be changed to something representative of the object being tracked (for example, "Car" or "Head").

Creating trackers

Trackers should be placed on significant image features such as corners or tracking markers. The more distinct the image feature is compared to the surrounding area, the more easily the feature can be tracked. As new trackers are created, they will be added to the tracker list.

Each tracker has its own set of tracking parameters, and these can be edited using the various menus and edit boxes provided in the UI.

Trackers that are passed into the User Track node can also be fetched for further editing in this node by clicking the Fetch button and selecting one or more trackers from the list.

Tracking and search windows

Trackers have a rectangular window area, the tracking window, that contains the image data to be tracked between frames, and a larger rectangular search area that shows the area that will be searched for the best matching window pixels. The tracking window should be adjusted so it neatly surrounds the image feature. Having a tracking window that is too large compared to the feature it is tracking may cause the tracker to fail, or the tracker position to drift over multiple frames. The search area defines the possible locations of the centre-point of the tracker, rather than the entire tracking window itself.

Note that the size of both the tracking window and search area can directly affect both the accuracy and performance of the tracking algorithm. If the tracking window is too large compared to the feature being tracked (i.e. it contains a lot of pixels that are very similar to the surrounding image area, and not unique to the feature) then the feature may not be tracked accurately, or the time taken to track the feature may increase. However, if the tracking window is too small, it can become difficult to estimate motion from one frame to the next.

The screenshot below shows a search window (the outer square, displayed with short dots) and a tracking window (the inner square, displayed with longer dashes) positioned around a small feature. The tracking window on the left is too large, because the feature being tracked is only occupying a small area in the centre of the window. The window on the right is better, because it provides a tighter bound around the image feature.

Positioning and editing tracking points

To create new tracking points, click the Create button and then click with the left mouse button in the Cinema window. Each click of the mouse button will place a new tracking point at that location.

To adjust the position of tracking points, make sure the Create button is not enabled, and click with the left mouse button in the Cinema window to select a tracking point (or click on the tracking point in the tracker list). With a tracker selected, click and drag with the left mouse button to adjust the position of the tracking point.

To adjust the size of the tracking window or search area, hold the Alt/Option keyboard shortcut whilst clicking and dragging on the corners or edges (Note: the requirement for this keyboard shortcut can be removed by changing the User Preference option if desired

Whilst creating or editing tracking points, the path that the tracker will take during tracking can be previewed by clicking the Preview Track button.

X and Y tracker positions for each tracker are also available in the Curve Editor.

Keyframes

Trackers are flagged as either Keyed, Tracked, or Estimated in each frame.

- Keyed tracker positions are those that are generated by the user, by positioning the tracker manually, and are displayed using yellow markers in the scrub bar.

In addition to frame where you are positioning a tracking point manually, you should create keyframes whenever a tracker comes back into view if it has either been occluded by another object or moved out of frame.

- Tracked positions are generated by tracking the feature.

- Estimated positions are generated for frames where the tracker is visible, but has been neither keyed nor tracked. These are estimated by interpolating the tracker position in-between nearby keyed or tracked positions.

Controlling tracker visibility

When a new tracker is created, it will initially by visible throughout the entire clip. The range of frames in which the selected tracker is visible is shown in red in the scrub bar. After tracking or keyframing, frames will turn a lighter shade of blue.

The following image illustrated this, where a tracker was placed at frame 120, and tracked forwards. The tracker is still visible in earlier and has not been tracked:

Trackers should be hidden from view when the image feature they are tracking either leaves the frame or becomes occluded by another object. This is achieved using the Hide buttons described below. In this example, because the tracker was not visible before frame 120, it is hidden from view:

Once a tracker comes back into view, create a keyframe at that point so you can track throughout the remainder of the clip more easily.

Tracker visibility can also be viewed in the Coverage Panel described below.

The colours used to represent tracked and untracked frames can be changed if desired in the User Preference window.

Dual View

When placing trackers in multiple clips, the Cinema window can be split into two parts, with the right-hand part used as a reference to help when placing trackers. The clip shown in the left part of the split is referred to as the Primary Clip, and the clip on the right as the Secondary Clip.

All interaction, in terms of placing and editing trackers, occurs on the Primary Clip, with the Secondary Clip only used as a reference.

Controls

Primary clip: The primary clip in situations where more than one input has been added to the User Track node. Positioning a tracker in multiple clips allows cameras for all clips to be solved using that tracker, ensuring that cameras are all positioned correctly relative to each other in the scene.

Secondary clip: The secondary clip in situations where more than one input has been added to the User Track node, and the Dual View button has been enabled to help tracker placement.

Current group: The group that trackers will be associated with. To generate trackers for a camera solver, set this menu to Camera. The + button can be used to create a new group for object solving. The additional group can be renamed by typing into the menu box, or deleted by clicking the trash can button on the right.

Tracking Window: The image pattern for the first selected tracker, along with the current x and y tracker position and matching score. A matching score of 1.0 indicates a perfect match of the tracker's reference pattern and the pattern in the current frame. Tracker positions can be adjusted interactively in this window by clicking and dragging with the left mouse button. Holding the Ctrl key will provide finer-grain control over the tracker position. Also, holding down the Alt/Option key while pressing the cursor keys can be used to reposition the tracker. The tracker moves by a default 1/40 of the tracking window size with each cursor key press (adjustable in the preferences) although the cursor key can be held down to move at a faster rate and more than one cursor key can be pressed simultaneously to move diagonally. The C button centres the current tracker around a significant tracking marker.

Trackers

The tracker list contains all trackers that have been generated, along with their window colour and the clips in which they have been positioned. The list can be ordered by clicking on a column header. Trackers can be selected by clicking in the list with the left mouse button. Selected trackers are also highlighted in the Cinema window. Trackers can be renamed by double-clicking in the Name column. Double-clicking in the colour column will display a colour selection window. The selected tracker can also be changed using the Up or Down cursor keys.

Polish: Track selected features forwards and backwards over a small range of frames defined by the Polish range parameter. This can be used to smooth out jumps in trackers when they are re-positioned manually after drifting out of alignment. The new tracking positions that are generated will be blended with the previous tracking positions according to the frame number. Frames that are very close to the current frame will mainly be influenced by the new tracking position, whilst frames that are far away (i.e. those at the extremities of the polish range) will be less affected and retain more of their original track position. Polishing will not overwrite keyframes.

K-, K+: Move to the previous or next keyframe.

<<, <, >, >>: Track selected trackers forwards or backwards through the clip. << or >> track continuously until the start or end of the clip is reached, provided the tracker matching score does not drop below the failure threshold. < or > will track by one frame only.

R-, Remove Key, R+: Remove keyframes from a tracker. The R- button removes all keyframes from selected trackers before the current frame, and R+ removes keyframes after the current frame. Remove Key removes only the current keyframe. Note that removing keyframes does not remove the tracker itself from a frame: in places where the tracker is visible (i.e. not hidden) but no keyframe exists, the tracker position will be linearly interpolated.

H-, Hide, H+: Hide a tracker from frames so it no longer contributes to a camera solve. This is necessary when a tracker is occluded by another moving object, or when the tracker should move out of bounds but instead is tracked to an incorrect position. Hide hides the tracker from the current frame. Frames in which the tracker is visible are displayed in blue or red in the scrub bar. H- or H+ removes a tracker from all previous or future frames, up to the point where a manual keyframe has been positioned.

Channels: The image channels used to match reference patterns. If one or more channels contains significant noise, or does not contain any useful pixel information, disabling these channels can often improve the accuracy of tracking.

Search mode: The algorithm used to perform tracking. The default value is Best Accuracy, which searches over the search window to find the most likely match for the reference pattern, but changing this to Best Speed will mean that tracking is faster but may be less accurate.

Deformation: The deformation of the pattern windows over time during tracking. By enabling Rotate, Scale and/or Skew, tracking windows can warp and deform in order to better match the reference pattern. This will slightly increase the time it takes to track, but can often mean that trackers exist for longer before their matching score drops below the failure threshold.

Window update: The frequency of reference pattern updates during tracking. The default value, Starting Frame, means the reference window is updated at the frame where tracking starts (i.e. the frame where one of the <<, <, >, >> tracking buttons are pressed). Changing this to Every Frame cause the reference pattern to be updated after every frame is tracked. This may result in a feature tracking for longer before its matching score drops below the failure threshold, but could also result in feature drift as small errors accumulate over time. The final option, Above Threshold, will only update the reference window in frames where the matching score is above the Window threshold parameter.

Window threshold: The minimum matching score that a tracker can have before its reference window is updated when Window update mode is set to Above Threshold. If a tracker's matching score is below this value, the reference window will be not be updated.

Failure threshold: The minimum matching score allowable before tracking is terminated. During tracking, a tracker will be rejected if its matching score falls below this value. The matching score for a tracker is displayed in the tracking window.

Tracking offset: The X and Y offset the tracking window from the actual location of the tracker. This is useful in situations where a tracker may be partially obscured, but an nearby area of the image that exhibits the same motion can be found to track instead. By offsetting the tracking window, motion from this nearby area can be used to update the tracker position. The tracking offset can be adjusted interactively in the Cinema window by holding down the Ctrl key whilst moving the tracker using the left mouse button.

Polish range: The number of frames on either side of the current frame that are used to polish a tracker path after a small adjustment.

Auto keyframe: When enabled, keyframes will be created automatically every N frames whilst tracking.

Illumination: When enabled, local image brightness will be taken into account when tracking, so effects such as flicker or lighting changes do not reduce the overall matching score as much.

Blur: When enabled, image data will be blurred slightly before tracking. This has the benefit of limiting the effect that image noise can have on reducing the overall matching score.

Predict: This option is only available when camera metadata describing approximate camera motion is present. When enabled, this approximate motion will be used to predict how the tracker moves from frame-to-frame to assist with the tracking process.

Reset: Reset the tracking parameters back to the current default values.

Set Default: Save the current tracking parameters use them as default values for newly created trackers. This can be used to avoid the need to define the same tracking parameters individually for each tracker. If the Shift key is held whilst clicking the Set Default button, the current tracking parameters will be stored in an XML preset file and will be used as default values for all future User Track nodes.

The preset file will be stored in the user's presets/node/ directory and will be automatically loaded when new User Track nodes are created. If the preset is no longer needed the XML file can be deleted from the directory to remove it from the application.

Errors Graph

The error graph plots the tracking error (measured in pixels) for each tracker in each frame, along with the average tracking error for all trackers visible in a frame. The tracking error is 0.0 when the matching score for a tracker is 1.0, indicating a perfect match. Selected trackers are shown in yellow, and unselected trackers are shown in blue. The average tracking error graph is shown in white. The error graph can be translated and scaled by clicking and dragging with the right or middle mouse buttons.

Individual trackers can be selected from the graph by clicking with the left mouse button. When a tracker is selected, the current frame will change to match the frame number that was clicked in the graph. If the Centre View display option is also enabled, the Cinema window will be panned to display the tracker location in the centre.

All: When enabled, error graphs will be shown for all trackers, otherwise a graph will only be shown for selected trackers.

Fit: Scale and translate the error graph so all graphs are visible.

Acceleration Graph

The acceleration graph plots the acceleration of each tracker in each frame, along with the average acceleration for all trackers visible in a frame. This can be used to frames where a tracker jumps incorrectly from one location to another by looking at the difference between the acceleration for a particular tracker and the average acceleration for all trackers in the frame.

Selected trackers are shown in yellow, and unselected trackers are shown in blue. The average acceleration is shown in white. The graph can be translated and scaled by clicking and dragging with the right or middle mouse buttons.

Individual trackers can be selected from the graph by clicking with the left mouse button. When a tracker is selected, the current frame will change to match the frame number that was clicked in the graph. If the Centre View display option is also enabled, the Cinema window will be panned to display the tracker location in the centre.

All: When enabled, acceleration graphs will be shown for all trackers, otherwise a graph will only be shown for selected trackers.

Fit: Scale and translate the acceleration graph so all graphs are visible.

The Coverage Panel

The Coverage Panel displays information about the frames in which each tracking point has been tracked:

This can be used to evaluate how well tracking points are distributed throughout the clip, which will help to provide an accurate camera solve without any jumps in the camera path.

The Coverage Panel displays each tracker as a horizontal chart, illustrating the frames in which each tracker is visible. The current frame is displayed using white vertical lines.

Each frame in which the tracker is present is indicated by a blue square. Yellow squares indicate frames in which the tracker has been manually positioned rather than tracked.

Frames in which the tracker is visible but has not been positioned are displayed in dark red. It is important to ensure that tracking points have been positioned in all frames in which they are visible, as this can significantly affect the accuracy of the camera solve.

Clicking on an indicator will select the tracking point and display that frame in the Cinema window. Holding the Ctrl key will allow multiple tracking points to be selected.

The panel can be panned horizontally or vertically by clicking and dragging with the right mouse button. Clicking and dragging with the middle mouse button will zoom either horizontally or vertically to increase the number of tracking points and frames that are displayed in the panel.

The mouse wheel can also be used to zoom horizontally, or vertically if the Alt/Option key is held.

Additional controls are also available:

All: Switch between displaying all trackers, or only those trackers visible in the current frame.

Name: Sort the tracking points by name, in alphabetical order.

Start: Sort the tracking points according to the first frame in which they are tracked.

End: Sort the tracking points according to the last frame in which they are tracked.

Fit: Fit the tracking points display to the window. This will zoom in or out as necessary, displaying as many tracking points and frames as will fit in the viewport.

Other Buttons

Create: Create a new tracker at the mouse position by clicking in the Cinema window, provided the mouse is not hovering over an existing tracker. By default, new trackers exist throughout the entire clip.

Fetch: Display a list of all trackers that are being piped into this node, allowing trackers to be taken from up-stream for editing. This is useful in situations where an automatically generated tracker needs to be tracked manually into more frames or edited in some other way, or when footage has been replaced and trackers must be extended into new frames.

Import: Display a file browser allowing an ASCII text file containing tracker locations to be imported. An example ASCII text file is given at the bottom of this document.

Export: Display a file browser allowing an ASCII text file containing tracker locations to be exported. An example ASCII text file is given at the bottom of this document.

Duplicate: Duplicate the currently selected tracker.

Remove: Remove selected trackers from the current clip but not delete them entirely.

Split: Split the selected tracker into two at the current frame. The selected tracker will be terminated at the previous frame, and a new tracker will be generated covering frames from the current frame to the out frame. This can be used in cases where a tracker jumps between two valid feature points to create two separate trackers, each covering a feature point before and after the jump.

Merge: Merge all selected trackers together into a single tracker. For trackers that exist in the same frame, their positions will be averaged during the merging process.

Delete: Delete all selected trackers entirely.

All/None: Select either all or none of the trackers, depending on how many of the trackers are currently selected.

Lock/Unlock: Lock or unlock selected trackers. Locked trackers cannot be adjusted or edited in any way.

Colour: Display a popup window allowing the colour of all selected trackers to be changed at the same time.

Rename: Display a popup window allowing all selected trackers to be renamed at the same time. Trackers will be numbered sequentially starting at 1 using the new name entered in the popup window.

Dual View: Split the Cinema window in two, allowing difference clips to be displayed in the left and right halves (referred to as the Primary and Secondary Clips respectively). Each half of the Cinema window can be panned or zoomed independently.

Sync View: Synchronise the secondary clip frame number to the primary clip, when Dual View is enabled. When Dual View is not enabled, the Secondary Clip frame number can only be changed by holding the Alt/Option key whilst clicking and dragging with the left mouse button in the right-hand half of the Cinema window.

Centre View: When enabled, the image displayed in the Cinema window will be translated so that the first selected tracker is positioned in the centre of the window. This can be useful when checking the motion of one particular tracker to see how well it manages to track an image feature over multiple frames.

Preview Track: When enabled, a preview of how a feature will track is displayed in the Cinema window for ten frames on either side of the current frame. As a tracker is positioned in the image window, the preview track will update dynamically to show the tracker path that will be generated. Tracker paths are colour coded according to the matching score, with red indicating a poor match (i.e. a matching score near the failure threshold), and green indicating a good match (i.e. a matching score near 1.0).

Marquee: When enabled, the left mouse button can be used to draw a selection marquee in the Cinema window to select trackers. Holding the Ctrl key whilst drawing will ensure that previous selections are kept. Holding the Shift key will allow a lasso selection to be used instead of a rectangle.

Show Search: When enabled, the search window will be displayed for all selected trackers.

Show Paths: When enabled, tracker paths will be displayed in the main window.

Show Names: When enabled, tracker names will be displayed in the main window.

Show Info: When enabled, tracker info will be displayed in the main window for all selected trackers.

Show Selected: When enabled, only selected trackers will be displayed in the Cinema window.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Track Backwards	[
Track Backwards Frame	Shift+{
Track Forwards	]
Track Forwards Frame	Shift+}
Centre On Marker	Alt/Option+C
Polish	Shift+B
Previous Keyframe	K
Next Keyframe	L
Remove Backwards	;
Remove Key	'
Remove Forwards	#
Hide Backwards	Shift+:
Hide	Shift+@
Hide Forwards	Shift+~
Create	Shift+W
Remove	Shift+D
Delete	Del
All/None	Shift+A
Duplicate	Shift+U
Lock	Shift+L
Split	Shift+S
Merge	Shift+E
Marquee	Shift+M
Show Selected	Shift+H
Show Search	Shift+R
Show Paths	Shift+P
Show Names	Shift+N
Show Info	Shift+I
Centre View	Shift+C
Preview Track	P
Set Default	Shift+F
Next Primary Clip	C
Next Secondary Clip	V

Example ASCII tracker text file used for import and export

# "Tracker Name"
# clip number
# frame count
# frame, xpos, ypos, matching score

"Tracker0001"
1
4
15 506.41 320.16 1.0000
16 407.83 318.54 0.9956
17 406.44 315.12 0.8854
18 404.22 311.14 0.8774

"Tracker0001"
2
3
66 861.22 196.44 0.9964
67 882.34 185.33 0.9451
68 876.11 179.44 0.9132

"Tracker0002"
1
5
11 274.11 776.34 1.0000
12 277.73 781.45 0.9951
13 281.16 783.44 0.8873
14 285.11 791.62 0.9312
15 288.76 794.12 0.8901