Solving Nodal Pans in PFTrack

21 Jul, 2016 | PFTRACK, TUTORIALS

An updated version of this tutorial is available as part of our PFTrack Training Course.

Nodal pans, where the camera is mounted on a tripod, provide a specific challenge to 3D camera tracking, due to the lack of parallax in the shot. In this tutorial, you will learn what that means when solving your camera, and how to use reference frames in order to extract 3D information from nodal pans. This tutorial provides an introduction to the User Track node, and demonstrates a technique that can also be used to solve multiple moving cameras into the same scene.

Contents

01. Import And Review Clips

02. Solve Without Reference Frames

03. Track With Reference Still Images

– Import The Stills

– Create The User Track Node

– Create a User Tracker

– Track The User Tracker

– Place The User Tracker In The Reference Images

– Create Additional Trackers

04. Solve The Scene With Reference Images

– Create The Camera Solver

– Set Up The Camera Solver

– Review The Solution

05. Test The Solution

06. Export

07. Conclusion

Tutorial Footage

To learn this tutorial you will need to download and use the footage below.

Footage: PFTChurchyard_Pan.zip

Download

Tutorial Footage

Footage: PFTChurchyard_Reference.zip
Download

01. Import And Review Clips

Open the File Browser, and navigate to the tutorial clip on your hard disk. Then drag and drop the thumbnail into the Tree View. Play through the clip to review.

02. Solve Without Reference Frames

This section establishes the issues that you will encounter when trying to extract 3D data from nodal pans. If you are already familiar with these kinds of problems and just want to know how to use reference frames in PFTrack, you can skip to the next section.

First try and solve the camera the usual way. Drag the thumbnail from the Media Bins into the Tree View, and create an Auto Track node and make sure it is connected to the clip. Leave the settings as they are and click Auto Track. As you look at the automatically created trackers you can notice that their paths are (almost) identical, no matter whether the feature is close to the camera or further away. This indicates a lack of parallax.

A good way to check for parallax is to select one tracker and press the Centre View button. This will keep the selected tracker in the centre of the Cinema. Then scrub forwards and backwards to see how the other trackers move relative to the selected tracker.

When you are done, create a Camera Solver node and connect it to the Auto Track node. Since the camera undergoes very little translation in this clip, which is the cause of the lack of parallax, change the Translation setting to Off-Centre. This setting is tailored for tripod-mounted clips and reflects the fact that in practice the rotation is hardly ever around the true optical centre of the camera. You can click the Help button to open the Camera Solver’s reference help page in the browser for a detailed description of the Translation options. Click Solve All to solve the camera.
Open the perspective view to review this solution and use the left mouse button to rotate the perspective view to see the scene from above. Select a few trackers at different distances from the camera (On Windows or Linux use the Control key to multiple select, on the Mac use the the Command key.) De-select Show info in the Display section to reduce the amount of text. In the perspective view, you can see that the features seem to lie on a semi circle, rather than representing their true distance from the camera. The Camera Solver relies on parallax (the fact that objects further away seem to move more slowly than objects that are close) to extract depth information. The result you are seeing here is due to the lack of parallax in this shot.

03. Track With Reference Still Images

Import The Stills

Open the Media Bins and File Browser and import the still images into your project, then add them to the Tree View.

Create the User Track Node

You are going to use a User Track node to establish the relationship between the movie clip and the still images. Select the clip again, then create the User Track node. Connect the clip and the stills to the new node.

Create a User Tracker

Close the perspective view again, then click the Create button to start creating your user trackers. Click and drag the middle mouse button to zoom onto the rectangular gravestone, then left click to place a tracker on the corner of the white patch. Adjust the size of the tracking window and the search area. You can refine your tracker placement by dragging in the magnified Tracking Window. Make sure Best Accuracy is selected for the Search Mode. Then click Set Default. This will use the same parameters you just set for all future trackers created in this node. Click the Help button to open the User Track’s reference help page if you want to know more about the individual settings. The reference page also includes an example of how to place a good tracker.

Track the User Tracker

Press the Centre View button. This will keep the currently selected tracker in the centre of the Cinema, which makes it easier to judge the quality of the track whilst tracking. Then click the >> button to track the tracker forwards. Depending on your placement of the tracker, tracking may stop before it reaches the end of the clip. This is because the matching score, a quality value that indicates how well the current frame matches the tracker’s reference pattern, is too low. The Failure threshold parameter defines when a matching score is considered too low. If necessary, adjust the positioning of the tracker. When manually repositioning a tracker, click Polish to smooth out jumps. In difficult situations, it sometimes can be helpful to click the > button to track a single frame forwards. Continue tracking until you have reached the end of the clip.

Place the User Tracker in the Reference Images

You can use the Current clip menu to select between the different inputs of the User Track node. Select the first reference image, PFTChurchPan_ReferenceA0, from the menu. Click and drag the right mouse button to pan, and the middle mouse button to zoom to locate the same white patch in this image. Then left click to place the tracker at the same spot. Adjust the placement in the Tracking Window if necessary.

Your trackers table should list one tracker, Tracker0001, with PFTChurchyard_Pan and PFTChurchyard_ReferenceA listed in the Clips column. This means that the same tracker is placed in both the clip and the reference image.

When placing the tracker in different views, you have to make sure that it is selected before left clicking in the Cinema, and that the Create button is not selected. If this is not the case, then you will create a new tracker instead, and your trackers table will look like the image below:
If this has happened to you, then you can select both trackers in the table, and click the Merge button to merge them into one tracker.
Repeat the steps to place the tracker in the second reference image. With Centre View pressed, it is easy to double check the positions in each image when switching between the clips.

Create Additional Trackers

Repeat the steps to place the next tracker at the top of this triangular shaped gravestone and track forwards. Despite that gravestone being no longer visible after frame 44, the tracker will still be active for the rest of the clip. This is indicated by the dark red marks in the scrub bar. Dark red indicates frames where the tracker is visible, but not tracked. Tracked frames are marked blue. On frame 44, click the H+ button to hide the tracker for the remaining frames. Then place the tracker in both reference stills as well.

Place additional trackers as shown in the screenshot above. Don’t forget to place them in both the reference images as well. A minimum of 6 trackers are needed per frame, but it is recommended to have 8 or more.

If you run into difficulties or want to compare your trackers, PFTChurchyard_Trackers.txt in the PFTChurchyard_Trackers.zip file contains the tracker locations used for this tutorial. Simply click Import in the User Track node and open the file to load these trackers into your node. Make sure to have the clip and reference images connected to the node as outlined in this tutorial, with the movie clip in the first input, PFTChurchyard_ReferenceA in the second and PFTChurchyard_ReferenceB in the User Track’s third input before loading the trackers from the text file.

04. Solve The Scene With Reference Images

Create The Camera Solver Node

Create a Camera Solver node and connect all three inputs from the User Track.

Set Up The Camera Solver

Switch to the reference frames in the Current clip menu. You will notice that the Helper checkbox has been automatically checked. This means that these inputs will be used to assist to solve the main camera in the first input. Click the Help button to open the Camera Solver’s reference help page and scroll to the Helper Frames section to read more about using reference images as helpers.

Return to the movie clip and click Solve All.

Review The Solution

Open the perspective view and rotate to view the scene from above. Select trackers at different distances from the camera again (turn off Show info in the Display section to reduce the amount of text in the window). You can see how the trackers more accurately reflect the actual 3D layout of the original scene.

05. Test The Solution

Create a Camera Solver node and connect all three inputs from the User Track.
Change the positioning mode back to Y-min and create and place some more test objects. Then play through the scene to review. The geometric test objects should move as if they are part of the location.

06. Export

The scene is now ready for export so create an Export node. If you connect the Camera Solver’s other outputs, the reference cameras can be included in the scene as well. Click Export Scene to export the scene.

07. Conclusion

In this tutorial you have learned how to use reference still images to make up for the lack of parallax in some shots, for example where the camera has been mounted on a tripod. The resulting scene not only contained the necessary 3D information for the movie camera, but also the camera positions of where the reference images had been shot.

You can use the same technique of placing user trackers in more than one shot to solve multiple movie cameras into the same scene, when the use of photogrammetry is not an option. However, moving cameras cannot be used as helper frames, but would be solved after the main camera.

Related Posts

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This

Share This Post On: