Creating a Triangular Mesh in PFTrack

16 Sep, 2016 | PFTRACK, TUTORIALS

An updated version of this tutorial is available as part of our PFTrack Training Course.

In this tutorial you will learn how to create a mesh model from multiple source clips in PFTrack’s Photo Mesh node. After creating the dense triangular mesh, you will simplify its geometry to pass the model downstream to create a texture in the Texture Extraction node and export your solved scene and the textured model.

Contents

01. Set Up the Scene

02. Create the Mesh

– Create the Photo Mesh Node

– Adjust the Bounding Box

– Create Depth Maps

– Review the Depth Maps

– Built the Mesh

– Review the Mesh

03. Using the Mesh

– Export the Full Resolution Mesh

– Simplify the Mesh

– Create a Texture for the Mesh

– Export the Mesh and Texture

Tutorial Footage

Footage: PFTChurchStills.zip

Download

01. Set Up the Scene

Follow the steps in this tutorial to set up the scene with the two clips. You should at least complete the first four sections to have your scene created from the still images and oriented.

When you are done, your tree should look similar to the one in the screenshot.

02. Create the Mesh

The Photo Mesh node is used to create a dense triangular mesh from one or more solved cameras or point clouds. Note that if you are using more than one input, the cameras must be part of the same coordinate system.

Create the Photo Mesh Node

Create a Photo Mesh node and connect both outputs of the Orient Scene node.

Adjust the Bounding Box

The first step towards creating your mesh is to define which part of the scene to use by adjusting the bounding box. By default, the bounding box encompases a majority of the scene, but more often than not you will only be interested in creating a mesh of a specific object.

Select Edit to start editing the bounding box. Split the view into 4 windows to get a better overview of your scene. The white edges define the bounding box. Click and drag with the left mouse button to adjust the nearest, highlighted face of the box. Hold the Control key on Windows or Linux, or the Command key on Mac, to adjust the face opposite the nearest.

Edit your bounding box so that it fits the church structure and doesn’t include much else.

When you’re done, de-select Edit.

Create Depth Maps

Next click Create to create a depth map for each of the 12 still images. This is the first step towards creating the mesh. In the second step, the mesh will be constructed from points from each of the depth maps. As the depth maps are created, you will see them displayed in the perspective and orthogonal windows as well as listed in the Depth Maps table.

Creating depth maps involves complex computing. Even though the algorithms take advantage of all available GPU and CPU resources, it can nevertheless be a time consuming process. When dealing with moving cameras in particular, the Path Sampling parameter can be used to reduce the workload. At its default value 1, a depth map is created for every frame. If you increase this value to 2 for example, a depth map is created for every other frame only. A value of 5 would create a depth map for every fifth frame and so on.

Review the Depth Maps

Once this is done, switch back to two horizontal viewports to examine the depth maps in more detail in the perspective window. As you step through the images you can see the vertices of the current depth map in colour. In some frames, parts of the sky have been picked up as well.

Click this buttondepth_map_confidence_button in the Depth Maps Display section. The mesh is now shown with colours indicating confidence values. Increasing the Min. Confidence setting will hide all depth map pixels below that threshold. You can interactively increase and decrease values in text boxes by clicking and dragging with the left mouse button. Change the value to about 50%. Click the depth_map_coloursbutton to see the vertex colours again.

To prevent the sky being picked up in the first place, you could have masked it out with, for example, a Fuzzy Selection mask.

Build the Mesh

Before building the mesh, you have the option to adjust a separate bounding box for the mesh. This can be useful if you only want to create a mesh from parts of the depth maps. This will not be necessary for this tutorial so you are ready to build the mesh by clicking Create in the Mesh section of the UI.

Review the Mesh

Before taking a closer look at the mesh, turn off the display of both the Mesh and Scene bounding box by de-selecting this buttonsurface_normals in the Display column of both the Scene and the Mesh section. You can also turn on surface normalssurface_normals and turn off vertex coloursvertex_colours to better see the detail in the mesh. Remember that it only took 12 still images to create this mesh and you could expand it and create a more detailed mesh by adding more images.

03. Using the Mesh

You can now export the mesh from within the Photo Mesh node, or simplify it to pass it downstream and use the simplified version in PFTrack.

Export the Full Resolution Mesh

You can now export the full resolution mesh in the Mesh Export tab. The available export formats are FBX, Open Alembic, OBJ and PLY. Click the button to open a file browser to choose the export location and filename, then click Export Mesh to export.

Simplify the Mesh

Please note, this feature is available from PFTrack 2016.09.16.

To pass the mesh downstream in PFTrack, it needs to be simplified first. Switch back to the Simplification tab. The Number of triangles box holds the target number of triangles for the mesh. You can see the current amount of triangles displayed in the top left of the Cinema. Leave the value at its default 100000 and click Simplify.

Once mesh simplification is completed, you can turn Show simplified on and off to switch between viewing the simplified mesh and the full resolution mesh. Only the simplified mesh will be passed down the tracking tree in PFTrack.

Create a Texture for the Mesh

Create a Texture Extraction node and make sure it is connected to the Photo Mesh node. First, you need to generate a UV Map for the mesh. Select the Photo Mesh entry in the Objects table, then choose Custom from the UV Projection menu and click Generate. The Custom setting instructs the node to use the UV map generated during the simplification  process in Photo Mesh.

You can then view and edit the UV map by clicking Edit UV. First you have to un-pack the individual UV charts by clicking the Un-Pack button. This will separate the custom UV map into non-overlapping regions. You can also change the resolution of the texture if desired, then close the window again.

Click Extract to start the texture extraction. The Best frame per triangle entry from the Extract From menu means that for each triangle UV group, PFTrack will use the best of the available still images to generate the texture. Current frame on the other hand would only use the current frame to texture the model. Once the texture extraction is completed, change the Render style in the Display section to Textured so you can see the textured model in the viewers.

Click the Help button to find out more about the Texture Extraction node or any other node in PFTrack.

Export the Mesh and Texture

Create an Export node and connect it to Texture Extraction node. You can see the model listed in the Objects tab, and the texture in the Textures tab. Click Export Scene to export.

Related Posts

PFClean: Auto Dirt Fix

In this tutorial you will learn how to use the Auto Dirt Fix effect in PFClean’s Workbench to automatically remove specks of dirt, dust etc from a clip…

Read More

Unity Project: Utilising PFTrack’s New Photogrammetry Toolset

The month of August was set aside for research and development within the walls of The Pixel Farm. New features have been added to PFTrack’s already powerful toolset, making the Photo Mesh node more versatile and more relevant to an unprecedented number of industries…

Read More

2.3 Using Multiple Views to Solve Nodal Pans

There is one special kind of camera motion, that requires additional views to extract any kind of 3D: Nodal pans. This tutorial explains what nodal pans are and how they are different from other camera movements. After that we will use the same technique shown in the live session to add additional camera views of the same scene in order to obtain a real 3D result.

Read More

ST-Maps in PFTrack

The Pixel Farm have been pioneers in making lens distortion data available for VFX pipelines. For years, the PFBarrel plugin for compositing applications has provided the ability to use PFTrack’s leading lens distortion models in third party software. However, with…

Read More
Share This

Share This Post On: