PFTrack: Photogrammetry

18 May, 2016 | PFTRACK, TUTORIALS

In this tutorial, you will be solving the tutorial clip using PFTrack’s photogrammetry tool set and then add a second camera to the scene.
This will be done by creating a point cloud from a set of still images and then automatically solving cameras into the scene defined by this point cloud.

Tutorial Footage

To learn this tutorial you will need to download and use the footage below.

Footage: PFTChurch1.zip

Download

Footage: PFTChurch2.zip

Download

Footage: PFTChurchStills.zip

Download

01. Importing Still Images

In your project open the File Browser and locate the stills that are available for download on this page. Double click on the thumbnail to review. These 12 images have not been created by a movie camera, but are still photographs of the location taken before the actual movie shoot.

Double click again to return to the File Browser.

By default, PFTrack groups image files that follow a certain naming convention automatically into a single clip. This time, however, you are actually dealing with still images rather than a movie so you can change this default behaviour by clicking the G button on the bottom right of the File Browser.

PFTrack now displays all 12 images individually rather than as a group. Open the Media Bins and drag and drop the whole stills folder into the Media Bins’ left hand column to import the whole folder into our project.

When you’re done click the F button to re-enable the File Browser’s default behaviour of grouping image files.

02. The Image Input Node

The Image Input node is the preferred method of using still images for photogrammetry in PFTrack.. Click Create, then Image Input to create the node.

Open the Media Bins again and select all the still images by either drawing a selection rectangle over them, or clicking on the first image, then shift clicking on the last. Then drag and drop them into the node’s Images list.

You can now step through the images, either by using the usual navigation buttons, or selecting images directly in the list.

The white arrow indicates the orientation of the image; you can override the orientation with these buttons.
There are also options to rotate or flip the images.

03. Photo Survey

You are going to use a Photo Survey node to generate our survey data point cloud from the still images. Right click on the Image Input node and select Photo Survey from the menu.

Start by reading the EXIF data from the image files by clicking Read EXIF. You now have the option to review the EXIF data. Then click Accept. A dialog appears to ask if you want to allow slight deviations from the reported focal length. As the note suggests, this can improve the accuracy of the solution, so confirm by clicking Yes.

Feature Matching

Then start generating the data by clicking Auto Match.

In this first step to generating the survey data, PFTrack steps through all the images to identify features visible in more than one image. You can see the identified features as yellow crosses in the Cinema as the matching process progresses.

Once completed, you can review the matches. After selecting a feature in the Cinema, markers in the scrub bar identify the images that contain the matches of the feature.

You can jump to those images to inspect the match.

The next step is to turn these feature matches into 3D data. Like in the moving camera solve in the earlier tutorial, turn on automatic estimation of lens distortion during the next step to get the most accurate estimate of the camera positions and 3D point cloud.

In the Lens Distortion tab, activate Estimate for both Low-order and High-order lens distortion.

Then click Solve All. This will generate the point cloud and calculate  the camera positions for each image.

Once completed, you will have a 3D scene that you can view in the perspective viewer. It is easy to make out the shape of the church in the point cloud and you can also see the positions where the photos have been taken.

04. Orient The Scene

Like when you solved for the  moving camera in the earlier tutorial, you  are going to use the Orient Scene node to orient the scene. Right click the Photo Survey node and select Orient Scene. Set the origin at the bottom left corner of the church tower.

Split the view into 4 windows so you will have different views of the scene. In the Edit mode menu select Translate, then move the origin into position. Note that the top right window shows the view from the back of the scene, so the bottom left corner is on the right for this view.

You can change this by right clicking in the window whilst holding the Control key on Windows or Linux, or the Command key on the Mac, and selecting a different view from the menu. Select Ortho Front as the new view.

Now select Rotate as the Edit mode and rotate the coordinate system so that the axis align with the walls of the church tower. When you are done, select None as the Edit mode to hide the manipulators, and switch the view back to single window.

05. Solving The Moving Camera

Next you are now going to use a Scene Solver node to track and solve the moving camera into the scene you have just created. Select the Orient Scene node, then right click on it and select Scene Solver.

The survey photos and point cloud should now be connected to the node’s first input. Before you can track the movie clip into the scene, you will have to initialise the point cloud data. Click Initialize to do this.

This has to be done only once per Scene Solver node, but once completed, the node can be used to track and solve a virtually unlimited number of cameras into the scene.

Connect the movie clip to the second input.

And select PFTChurch1 in the Current clip menu.

As with the previous camera solves and when generating the point cloud, turn on lens distortion estimation. In the Lens Distortion tab, check Estimate for Low-order lens distortion.

Then click Auto Match.

In a first step, PFTrack is now tracking features throughout the shot. As with the Auto Track node in the previous tutorial, Track in both directions is checked to increase the average feature length. Once tracking is completed, PFTrack automatically moves on to the next step of solving for the camera motion and aligning the movie camera into the scene.

When this is done, you can review the solution in the perspective view. You can see the movie camera, embedded in the scene along with the still camera positions and the point cloud.

06. Testing The Solution

Create a Test Object node. The new node is automatically connected to Scene Solver’s first output. This output, however, contains the still image and point cloud data, so break the link between the nodes by clicking on it, and connect the Scene Solver’s second output, which contains the movie clip, instead. Then double click the Test Object node to make it the active node again.

In the Test Object node, add a cow to the scene and play back. The cow is sticking in position as desired.

07. Adding a Second Clip

Let’s add a second clip to our scene. Stop playback, then open the File Browser and locate the second tutorial clip. Drag and drop the thumbnail into the tree view.

Remember, you can pan and zoom the tree view with the right and middle mouse button. Connect the clip to the Scene Solver node.

In the Scene Solver node, repeat the steps you took to solve our first clip. Select PFTChurch2 in the Current clip menu, turn on estimate low order lens distortion and click Auto Match.

Once this process is completed, open the perspective view and play through the clip. Your scene now contains two moving cameras. Close the perspective view again when you are done.

Connect the Scene Solver’s third output, which contains the new clip to the Test Object node. As both clips are part of the same scene they share the cow object we’ve created already. Move the cow to the right so it stays in view for longer for the second clip. Then play back to review.

Turn off Show trackers, Show ground and Show horizon and zoom the clip to fit to height. Again, the cow stays in position as it should be.

08. Exporting The Scene

The scene is now ready to export. Select the Scene Solver node, right click and select Export. Move the new node in position and connect both of the Test Object node’s outputs as well.

The exported scene includes three cameras, our test objects as well as the point cloud.

This concludes this introduction  to solving cameras using PFTrack’s photogrammetry tool set.

You have learnt how to create a point cloud from a set of still images and then automatically solve camera’s into the same scene.
The next tutorial will cover how to create a mesh model from the point cloud created in this lesson.

Related Posts

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This

Share This Post On: