ST-Maps in PFTrack

10 Jun, 2016 | PFTRACK, TUTORIALS

The Pixel Farm have been pioneers in making lens distortion data available for VFX pipelines. For years, the PFBarrel plugin for compositing applications has provided the ability to use PFTrack’s leading lens distortion models in third party software. 

However, with the wide range of compositing software packages and the various lens distortion models they support growing, a beneficial alternative was needed. That’s why we have introduced the option to export the lens distortion model as a sequence of ST-Maps; an easy and portable way to apply PFTrack’s lens distortion results in a software agnostic way.

Contents

01. So What Are ST-Maps?

02. How Does It Work?

– Solve Your Lens Distortion the Usual Way

– Export the Deformation ST-Maps

– Working With Proxies

03. Using the ST-Maps to Undistort and Redistort your plates

– Using ST-Maps in Nuke

01. So What Are ST-Maps?

The image above shows an example ST-Map. The colours in the red and green channel describe the way the individual pixels are warped to apply or remove lens distortion from an image.

02. How Does It Work?

Solve Your Lens Distortion The Usual Way

You can solve your lens distortion the same way as you always have in PFTrack, either explicitly in an Undistort node, during the camera solve in the Camera Solver, Survey Solver or Photo Survey node, or by using a lens preset. The resulting lens distortion model will be piped down the tree into an Export node.

Exporting the Deformation ST-Maps

In the Export node you can find the new Distortion Export tab. From here you can export ST-Maps for both undistortion and redistortion. Both, the undistortion and redistortion will be formed of a sequence of the same length as the input clip; one ST-Map for each frame. Click Export ST-Maps to write the ST-Map clips to disk.

Working With Proxies

You can use ST-Maps even if you use proxies for tracking. ST-Maps store normalised pixel coordinates, so they will still sample from the correct location in the image after scaling them up or down. For example, if you are tracking half resolution proxies, you will have to double the size of the resulting ST-Maps to use them with the original plates.

03. Using the ST-Maps to Undistort and Redistort Your Plates

Most compositing applications provide ways to read ST-Maps and apply the encoded transformation, thus letting you undistort and redistort your plate using PFTrack’s lens distortion model. Below is an illustration of how to use ST-Maps in Nuke.

Using ST-Maps in Nuke

Use an STMap node to undistort or redistort your original clip. In the example tree above, STMap1 is used to undistort the original footage, using the undistort ST-Maps exported from PFTrack. The output is then connected to STMap2 and lens distortion reapplied using the redistort ST-Maps from PFTrack. The result will be identical to the original input clip.

In the STMap node, make sure to set UV channels to rgb, so that the correct channels will be read.

Related Posts

PFTrack 2017 Texture Maps – The Unity Workflow

Texture maps are used to enhance the features of an object, recreating the effect of a high resolution model after mesh simplification. PFTrack 2017 has seen the introduction of Texture Map Extraction directly from the Photo Mesh node. The updated functionality of the…

Read More

ST-Maps in PFTrack

The Pixel Farm have been pioneers in making lens distortion data available for VFX pipelines. For years, the PFBarrel plugin for compositing applications has provided the ability to use PFTrack’s leading lens distortion models in third party software. However, with…

Read More

Tracking 360º Equirectangular Clips in PFTrack

In this tutorial, you are going to track and solve an equirectangular panorama using PFTrack’s new spherical tracking toolset…

Read More

Data Capture for Photogrammetry: The Spooky Asset Project

This Halloween we were getting into the swing of things, as the winter nights draw closer we felt like spreading some spook! The best way to do this was by using PFTrack’s comprehensive photogrammetry toolset to create some quality assets. The project was a good…

Read More

3 Comments

  1. How can you use the ST-maps in After Effect CC?

    I’ve seen a plugin by RE:maps, guys who do Real Smart Motion Blur.
    But not sure if it uses the same method?

    Is this the only way it could be read in AE or can we use it without purchasing additional plugins to read maps?

    Reply
  2. Hi,

    Is there an automated method for generating a _nuanced_ STmap from a lens grid (checkers image) inside PFT ?

    My understanding is that the current PFT undistort node requires a human to draw several lines on an input image, which is slower and can’t possibly be more accurate than Nuke’s “LensDistortion / Grid Analysis” automatic generation.

    In both cases, the image analysis is of straight lines and generates a distortion node defined by 4 numeric parameters.

    The mathematical description of the distortion, while nice, is not utilizing the full potential of an ST-Map which potentially can describe distortion pixel by pixel, much more accurate than the four numeric parameters.

    So, would you know how can we generate a pixel based nuanced ST-map, that isn’t just calculated from the good old four numeric parameters?

    Please advise,
    Thank you,
    Dani.

    Reply
    • You can use the PFTrack’s Build Lens node to construct a lens model from a calibration grid.

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This

Share This Post On: