Tracking an Object in PFTrack
Sometimes it is necessary to track motion other than the camera’s, for example to replace moving objects in a shot. In this tutorial you are going to track and solve a prop carried by an actor. You will learn how to use the User Track node to manually track an object and then solve this object into your scene.
– Load a Pre-defined Scene
– Create the Node and Motion Group
– Create the First Tracker
– Track the First Tracker
– Create and Track Additional Trackers
– Track the Back of the Box
– Track the Top of the Box
– Track the Front of the Box
– Set the Object’s Scale
To learn this tutorial you will need to download and use the footage below.
01. Import Clip and Review
Open the File Browser and navigate to the tutorial clip on your hard disk. Then drag and drop the thumbnail into the Tree View. Play through the clip to review. It is an outdoor handheld shot of an actor carrying an object.
02. Solve the Camera
Every scene needs a camera before any object motion can be solved, so you first have to track and solve the camera in this clip.
Create an Auto Track node, change the Search mode to Optical Flow and select Rotate, Scale and Skew in the Deformation menu, then click Auto-Track.
Once tracking is completed, create a Camera Solver node, go to the Distortion tab and check Estimate for Low-order lens distortion and click Solve All to solve for the camera. Select three trackers on the ground and choose X-Z Plane in the Set Plane menu to quickly orient the scene. Then select a suitable tracker and click Set Origin.
If you want to know more about these steps, have a look at the Auto Track and Camera Solve tutorial.
Load a Pre-defined Scene
Alternatively, a scene with a solved camera for this shot is available here. To use this camera, instead of tracking and solving yourself, create an Edit Camera node and connect it to the clip. Click Import Camera and navigate to the PFTPropWalk.fbx file to open it. Then click Okay in the next dialog.
03. Track the Object
Create the Node and Motion Group
Create a User Track node and connect it to your solved scene. Trackers created in this User Track node will be used to solve for the prop’s motion, rather than the camera. To do this, you need to create a new motion group for the object motion by clicking the + button next to the Current group menu. Rename the group to “Box” by directly typing into the field. There are now two motion groups available in the tree, the default Camera group for the moving camera, and the new Box group, which will collect all trackers associated with the motion of the prop.
Create the First Tracker
Click the Create button, then click in the Cinema to create and position the first tracker. Select the top left tracking marker on the prop. Zoom onto that marker if necessary to adjust the tracker more easily. You can fine tune the positioning by clicking inside the tracker and dragging with the mouse. There is a magnified version of the tracker in the panel at the bottom. You can also drag inside this panel to reposition your tracker.
The inner rectangle, the tracking window, contains the pattern that will be tracked. Adjust the tracking window so that it only contains the tracking marker and nothing else. You can do that by dragging the window’s edges in position.
The outer rectangle is the search window. The search window defines the region where PFTrack will search for the pattern in the next frame. Narrow it down so it will be closer to the tracking window. The prop contains the same tracking markers several times, and too wide a search range could result in the tracker jumping to another marker.
Select Rotate, Scale and Skew in the Deformation menu to allow the tracking window to warp in order to match the pattern in the next frame.
When all of this is done, click Set Default, to make these settings (including the size of the tracking window and search window) defaults for all trackers you are going to create in this node.
Track the First Tracker
Click Centre View. With Centre View pressed, the currently selected tracker will always be kept in the centre of the viewport. This makes it easier to follow the tracker and spot potential issues while tracking.
Clicking the > button will track the tracker forward one frame. Clicking >> will track the tracker continuously until the end of the clip, or until the matching score (a quality value that indicates how well the current frame matches the tracker’s reference pattern), is too low. The Failure threshold parameter defines when a matching score is considered too low.
If this happens, you can adjust the positioning and parameters of the tracker before continuing to track. Manually repositioning a tracker often causes jumps in the tracker’s path. These can be smoothed out by clicking Polish, which will re-track the feature over the last 5 frames. The number of frames which will be re-tracked is defined by the Polish range parameter.
Continue tracking once all the adjustments are done.
The frames where the tracking stops depends very much on the initial positioning of the tracker and its tracking window and search window. When tracking does stop before the end of the clip, make sure that the tracker is still in the correct position, reposition and polish if necessary, then continue tracking.
Create and Track Additional Trackers
Jump back to the first frame and create two more trackers for the tracking markers on the side of the box.
The process for each tracker stays the same, track, adjust if necessary, polish if necessary, continue until tracking reaches the end of the clip.
Track the Back of the Box
Jump back to the first frame and place the fourth tracker on the marker at the back of the prop. Due to the angle of the tracking marker, you should adjust the size of the tracking window before starting to track. About halfway through the clip the tracker will be hidden, with frame 59 being the last frame where it is fully visible.
If your tracker has been tracked for longer, you can go back to frame 59 and click and confirm the R+ button. This will delete all tracking keyframes after the current frame.
Despite the tracker not being tracked beyond frame 59, it is still visible and active for the remainder of the clip. This is indicated by the dark red marks in the scrub bar. Dark red indicates frames where the tracker is visible, but not tracked. Blue indicates frames where the tracker is visible and has been tracked. On frame 59, click the H+ button to hide the tracker for the remaining frames. Step forwards a couple of frames to confirm that the tracker is no longer visible in these frames.
Track the Top of the Box
The next step is to create trackers for the top of the box. Go to the first frame and place the tracker in the corner of the first tracking marker.
As this tracking marker is not fully visible, you will have to adjust the tracking window and search window accordingly. Then track to the last frame.
Repeat these steps with the second tracking marker. You will have to initially position it outside the other’s search window and then drag it in position.
During difficult sections it can be helpful to track frame by frame, rather than continuously.
Track the Front of the Box
Whilst at the last frame, place the final tracker on the marker at the front of the box. This marker is only visible in about the last third of the clip. Position it in the last frame and track backwards using the << button. When you have reached frame 69, use the H- button to hide it for the untracked previous frames.
If you run into difficulties tracking the box, or want to compare your trackers, PFTPropWalk_Trackers.txt in the PFTPropWalk_Trackers.zip file contains the tracker locations used in this tutorial. Simply click Import in the User Track node and open the file to load these trackers into your node.
04. Solve the Object
In the same way that a Camera Solver converts tracked features into camera motion, an Object Solver solves for object motion. Create the Object Solver node and make sure it is connected to the User Track node. Note that since the Object solver does not solve cameras, the Box motion group has already been selected as the current group. Leave all settings at their default values and click Solve Object. Remember, you can always find out more about a node by clicking the Help button.
Set the Object’s Scale
Open the perspective view and scrub through the clip to get an impression of the solve. On closer inspection of the moving camera and object, it is clear that the object is too far away from the camera and the origin. At the moment, PFTrack thinks that the object is far away from the camera. This is a mathematical limitation when only using a single camera to solve for object motion.
To correct this, select Distance from camera in the Orientation mode menu. This enables manipulators in the Cinema that let you change the object’s scale and distance from the camera. Drag the manipulators to position the object closer to the origin. Go to a frame where you can get a good idea of the object’s position with respect to the origin and grid, then position it as precisely as you can.
05. Test the Solution
You can now test your solution with test object in the Test Object node. Close the perspective view and create the node. Double click on the Cow_Head entry in the Available objects list to create the test object. Select Place at selected feature as the Interaction mode and set the Positioning mode to Origin, so the head will be placed at its origin, rather than on its bottom edge. Click one of the trackers on the side of the box to place the cow’s head there. Then use the Rotate, Scale and Translate interaction modes to transform it into place.
When you are done, set the Interaction mode to None to hide the manipulators and press play. The test object should behave as if it was attached to the prop.
Create an Export node to export the scene. In the Groups tab you can see that both the camera group and the Box group are available for export. So too are the trackers for both groups. Click Export Scene to write the file to disk.
In this tutorial, you have learned how to add object motion to a scene that already contains a solved camera. To achieve this, you have learned how to create and track manual trackers in PFTrack’s User Track node, and then use an Object Solver to solve for the object’s motion inside the scene.