PFTrack: Auto Track and Camera Solve
In this tutorial, you will learn how to solve for the camera motion in the tutorial clip, and then test the resulting scene so that it is ready for exporting. You are going to solve the motion of the camera in the tutorial clip, using trackers that have been automatically generated by PFTrack.
To learn this tutorial you will need to download and use the footage below.
01. Setting Up a Project
Follow the instructions in the first Getting Started tutorial to create a project and import the tutorial clip.
02. Tracking The Tutorial Clip
The first step in tracking a clip is to play through the it to evaluate the task at hand. This one is a handheld shot in a churchyard.
Stop playback again. Make sure the Clip node is still selected, then create an Auto Track node by either clicking the Create button, and then clicking Auto Track, or by right clicking on the Clip Input node and selecting Auto Track from the menu.
The new node should be automatically connected to it. If it isn’t, drag a line from the Clip Input node’s output connector into the Auto Track node.
As its name suggests, the Auto Track node automatically generates candidate trackers and tracks these throughout the clip. But you can influence how this is done through a set of parameters in the node.
If you want to know more about any of the settings in this node, simply click the Help button to open the node’s help page.
We are going to adjust two parameters for our shot, Search mode and Deformation.
The search mode menu let’s you choose the algorithm used for tracking. The choices are Better Speed, Better Accuracy, and Optical Flow. Select Optical Flow, due to the handheld nature of the shot and the fact that there is motion blur in some of the frames.
You should also change the Deformation setting. This specifies how the tracked pattern can deform over time during tracking. Enable all three, Rotate, Scale, and Skew. Now the tracking window can warp and deform in order to better match the reference pattern, which will increase the life span of the trackers.
Click Auto Track to start tracking. You can now watch PFTrack as it picks features and tracks them throughout the shot. This gives you a chance to spot potential problems during the tracking process.
If you hold the Shift key while clicking the Auto Track button, the auto tracking will run in the background, to let you work on a different clip in the meantime.
You will see that PFTrack performs two tracking passes, the first forward one, then a second, backwards pass. In this second pass, PFTrack will not create any new trackers, but use this pass to increase the length of the existing trackers. As a rule of thumb, tracking points that exist over a long period are often those that provide most benefit to the camera solver.
When tracking has completed, you can scrub through the clip to review the track. All features should have been tracked well in our clip.
03. Solving For The Camera
The next step is to use a Camera Solver node to turn the trackers into camera motion. Click Create and then click on the Camera Solver button to create the node. Make sure it is connected to the Auto Track node.
As with the Auto Track node, there is a number of parameters available to influence how the Camera Solver works. You can find out all about the node and its parameters in the node’s help page, which opens when you click the Help button. The sections “Influencing The Solve” and “Tracking Difficult Shots” contain valuable tips on how to handle tricky clips.
One of those tips is to make sure to account for lens distortion. You can do this by opening the Distortion tab in the node and checking Estimate for Low-order lens distortion and High-order lens distortion. This will automatically estimate the lens distortion during the solve.
Leave all other parameters at their default values and click Solve All to start the camera solve.
PFTrack now does some number crunching to turn the tracker motion into camera motion.
Once completed, you can review our solution in the perspective viewer where we can see our 3D scene.
04. Orienting The Scene
Right click on the Camera Solver node and select Orient Scene from the menu to create the Orient Scene node.
Close the perspective view again by clicking the single cinema viewport button.
You can define the origin of our scene by selecting a tracker and clicking Set Origin.
You can use the Edit mode menu to change the overall orientation of the scene. In Rotate mode, you can use the manipulators in the Cinema to rotate the ground plane. Translate and Scale work in the same way. Select None to hide the manipulators again.
You can also scale the scene by defining a fixed distance between two trackers. For example, select the two track two trackers as shown in the video. Hold the Control key on Windows or Linux, or the Command key on the Mac to multiple select.
If you know that these two points are 15 centimetres away from each other you can type 0.15 in the Distance edit box to scale the scene accordingly.
Now that you have oriented our scene, it is ready for export. But before doing that, you can test it a bit further right inside PFTrack.
05. Testing The Solution
Create a Test Object node and make sure it is connected to the Camera Solver node.
The Test Object node lets you add a geometric object to the scene to test the quality of the solve.
Some default objects are already available and you can also import your own geometry from an obj, fbx or alembic file. Double click on the Cow entry in the Available objects list to add a cow to the scene.
Use the Interaction mode menu to position the cow. It works the same way as the Edit mode in the Orient Scene node, with manipulators for Translate, Rotate and Scale. We are going to use Place at selected feature. This will position our test object at any feature we select by clicking on it with the left mouse button.
Play through the scene. The geometric cow should appear to be standing on the grass as if it was part of the original location.
You can add some more test objects, for example some mushrooms, and place them in the scene the same way as the cow.
In the Display section of the node, turn off Show trackers, Show ground, and Show horizon. This will only leave the test objects in the original clip.
Play back the clip to review one last time. If you are happy with the how the objects are part of the scene, it is ready for export.
06. Exporting The Solution
Create an Export node. This node can be used to export your solution to third-party applications. Most of the available exports are fully customisable Python scripts, so it’s easy to add your own file format if needed.
Depending on the file format, and what you have included in our scene, you can use these tabs to select what should be included or excluded in the exported file.
In your scene, you have a camera in the Cameras tab and the default camera group in the Groups tab, as well as the trackers in the Trackers tab and the test objects in the Objects tab.
Uncheck the Mushroom entries in the Objects tab so to not include them in your exported scene.
Make sure the export Filename and path are correct, then click Export Scene.
This concludes your first camera track in PFTrack. You have learnt how to create a 3D scene from your 2D clip by automatically tracking features throughout the clip and how to prepare your solved camera for export into the VFX pipeline.
Do not delete your project just yet. It is going to be used again in the next tutorial, where you will learn how to solve multiple camera’s into the same scene using photogrammetry.