Tracking a 360º Clip with PFTrack’s Photogrammetry Toolset
In this tutorial you will learn how to track and solve a 360º spherical camera into a scene that has been surveyed photographically with a Photo Survey node. By doing so you will learn how to use PFTrack’s familiar tracking toolset to solve 360º footage, instead of the purpose built Spherical Track node, featured in the Tracking 360 footage in PFTrack tutorial.
– Set up the Scene Solver Node
– Track the Front View Camera
– Review the Track
– Match the Other Cameras to the Front View
– Add Test Objects
– Create a New Equirectangular Panorama
01. Set up the Scene
Follow the steps in this tutorial to set up the scene. You should complete the first four sections to have your scene created from the still images and oriented.
When you are done, your tree should look similar to the one in the screenshot above.
02. Import the 360º Clip
Open the File Browser and navigate to your PFTChurch360 clip, then drag and drop the thumbnail into the Tree View. Play through the clip to review. The clip has been shot at the same location as the photos used in the Photo Survey node.
03. The Spherical Track Node
To use equirectangular 360º footage in PFTrack, you will need a Spherical Track node. Create the node and connect it to the clip. The Spherical Track has one input, which requires a 360º equirectangular image. The node’s six outputs represent one camera for each planar face of a cube map generated from the input panorama.
As has been detailed here, you can use the Spherical Track node to track and solve your 360º equirectangular clip. However, in order to solve the clip into the scene created with the Photo Survey node, a different approach has to be used. Instead of tracking inside the Spherical Track node, you will use the Scene Solver node to track one of the cube map views generated by the Spherical Track node.
04. Set up the Cube Map
Create a Spherical Orient node and make sure all six inputs are connected to the corresponding outputs of the Spherical Track node.
By default, the Spherical Orient node shows the front view of the cube map in the Cinema. You can select a different view from the Current clip menu. Open the perspective view and click the Orient button. Use the manipulators in the perspective view to orient the views in such a way, that the church is clearly visible in the front view. Then deselect Orient again.
05. Track the Front Camera
Set up the Scene Solver Node
Select the Orient Scene node and create a Scene Solver node. The Orient Scene node should be connected to the Scene Solver’s first input. First you need to initialise a photo collection database with the Photo Survey point cloud first. Click Initialize to generate the necessary data for the database.
Track the Front View Camera
Now connect the Spherical Orient’s first output to the Scene Solver’s second input. Select the PFTChurch360_Front in the Current clip menu and click Auto Match to start tracking the front camera into the scene.
Review the Track
06. Align the Remaining Cube Map Views
So far, you have only tracked the front view camera into the scene. The other 5 views of the cube map have not been tracked. Create a Spherical Orient node. Connect the Scene Solver’s second output with the first input of the new Spherical Orient node. Connect the remaining outputs of the first Spherical Orient node with the remaining inputs of the new Spherical Orient node. You can see in the perspective window how the remaining 5 cameras from the cubemap are located somewhere in 3D that has no relation to the tracked front camera.
Match the Other Cameras to the Front View
You can align the other 5 cameras to the front view by selecting Front Camera from the Rebuild from menu.
07. Review the Result
Add Test Objects
Create a Test Object node and connect all outputs of the second Spherical Orient node. In the Test Object node, create a Cow_Head object and select Place at selected feature from the Interaction mode menu. Click a feature on the clock tower to position the test object. Use the Scale and Translate modes to adjust positioning, then turn off Show horizon, Show ground and Show trackers and check Bake objects.
Create a New Equirectangular Panorama
Close the perspective window and create a Spherical Render node. Make sure all six of the Test Object node’s output are connected to the corresponding inputs of the new node. Play through the clip to review the new panorama with the baked-in test object.
The scene is now ready to export. You can export the scene with the six cameras created from the cube map plus the still images by connecting the Test Object node’s six outputs and the first output of the Scene Solver node to an Export node. You can also export the new equirectangular panorama by connecting the Spherical Render node’s output to an Export node.
In this tutorial you have learned a new way to track and solve a 360º clip using PFTrack’s photogrammetry toolset. Similar approaches can be used by creating auto or user tracks in combination with a camera or survey solver instead of the Scene Solver.