3.1 Using Geometry to Solve a Camera in PFTrack
This document is part of Level 3. Advanced Matchmoving Strategies in PFTrack of The Pixel Farm Training Academy’s PFTrack course. Find out more and register for the next available live class.
Providing PFTrack with as much information as possible from the set of a shoot can help get a good result more easily. The amount of information you could use can range from which camera and lens has been used, to the distance of specific features from the camera in a frame to a LIDAR scan or a set of photos of the location, to geometric models of structures on set. The Advanced Matchmoving Strategies live class focusses on using geometry to solve cameras in PFTrack.
– Building a Lens Preset from a Grid
– Estimating Focal Length
– Tracking the Object
– Generating Survey Coordinates
– Further Reading
To learn this tutorial you will need to download and use the footage below.
01. Setting up the Clip
We know which camera has been used to clip used in the live class; it is a Canon EOS 7D. The sensor size of this camera is 22.3mm wide and 14.9mm height. To use this information in PFTrack, we create a custom camera preset in the Clip Input node. Be aware that the camera does not use the full sensor height when shooting movies, so the resulting Film back width is different from the one given in the data sheet.
Click Save Preset to store these values in a preset, which you can reuse for other clips from this camera.
02. Setting up the Lens
When using a geometric model to track cameras in PFTrack, it is important to know the focal length in order to accurately align the object to the camera image. To illustrate, the following screenshots show the same geometric model at the same location, but with different focal lengths given to PFTrack.
We do know the camera model used for the shoot, but unfortunately exact information about the lens has been lost over time. However, there are ways in PFTrack to estimate a focal length when detailed lens information isn’t available.
Building a Lens Preset from a Grid
With the clip used in the live class came an image of a checker-board, shot with the same camera/lens combination.
This image can be used in a Build Lens node to create a Lens Preset. Make sure to set the correct film back size in the image’s Clip Input node as well.
In the Build Lens node, match a grid to the checkerboard pattern to let PFTrack determine lens characteristics such as distortion and focal length, which are then saved to disk as a preset.
The node’s reference help page has more information on how to create lens presets in PFTrack. Stored lens presets can be used in the Distortion tab in PFTrack’s solver nodes.
Estimating Focal Length
If no appropriate checker-board image is available, it can be possible to estimate focal length by identifying vanishing lines inside the clip in an Estimate Focal node.
In the Estimate Focal node, align the appropriate axis to straight lines that are parallel to either the X, Y or Z axes of the scene. You will need to identify lines in at least two directions (X, Y or Z) to be able to get an estimate of the focal length.
During the live class, we use the building to identify parallel lines along the x and z axes.
03. Tracking the Camera in the Geometry Track Node
PFTrack’s Geometry Track node can be used to track either the camera or a moving object, using a triangular mesh instead of tracking points. It is especially useful in situations where it is either not possible or not desireable to track using multiple tracking points.
When using the Geometry Track node to track a camera (as opposed to tracking an object), make sure the Moving object checkbox is unchecked before importing the model.
After importing the mesh model into the node, it must be aligned to the camera image in one frame.
From that frame, the model can be tracked forwards and backwards through the clip.
04. Using Geometry with the Survey Solver
Another way of using a geometric model to solve a camera is to generate survey coordinates in the Survey Solver node. Survey coordinates are 3D positions for trackers, which are known before the camera has been solved. There are many ways of obtaining survey coordinates, one of which is by projecting 2D trackers onto the model’s surface.
Tracking the Object
Survey coordinates are generated for trackers, so a number of features need to be tracked on the structure for which a model is available.
Generating Survey Coordinates
To generate survey coordinates for the trackers, the geometric model has to be imported into the Survey Solver node and aligned in one frame with the structure it is representing.
Then survey coordinates can be generated for selected trackers by projecting rays from the camera through the tracker. The tracker’s 3D position is where this ray intersects with the geometric model.
If the model couldn’t be aligned accurately with the camera image (for example if the focal length is unknown), uncertainty values for each tracker can be specified, which will allow the Survey Solver to vary the tracker’s position when solving the camera motion.
Clicking Solve All will start the camera solve.
This document provided an overview over how a geometric model can be used to solve a camera in PFTrack.
3.2 Tracking a Deformable Object with Geometry uses the Geometry Track node to track an object rather than the camera. Furthermore, the geometric object is allowed to deform to capture an actress’ performance.
3.3 Using Survey Data in PFTrack explains the many other ways survey coordinates can be used or generated inside the Survey Solver node.