ST-Maps in PFTrack

10 Jun, 2016 | KNOWLEDGE BASE, PFTRACK

The Pixel Farm have been pioneers in making lens distortion data available for VFX pipelines. For years, the PFBarrel plugin for compositing applications has provided the ability to use PFTrack’s leading lens distortion models in third party software. 

However, with the wide range of compositing software packages and the various lens distortion models they support growing, a beneficial alternative was needed. That’s why we have introduced the option to export the lens distortion model as a sequence of ST-Maps; an easy and portable way to apply PFTrack’s lens distortion results in a software agnostic way.

Contents

01. So What Are ST-Maps?

02. How Does It Work?

– Solve Your Lens Distortion the Usual Way

– Export the Deformation ST-Maps

– Working With Proxies

03. Using the ST-Maps to Undistort and Redistort your plates

– Using ST-Maps in Nuke

01. So What Are ST-Maps?

The image above shows an example ST-Map. The colours in the red and green channel describe the way the individual pixels are warped to apply or remove lens distortion from an image.

02. How Does It Work?

Solve Your Lens Distortion The Usual Way

You can solve your lens distortion the same way as you always have in PFTrack, either explicitly in an Undistort node, during the camera solve in the Camera Solver, Survey Solver or Photo Survey node, or by using a lens preset. The resulting lens distortion model will be piped down the tree into an Export node.

Exporting the Deformation ST-Maps

In the Export node you can find the new Distortion Export tab. From here you can export ST-Maps for both undistortion and redistortion. Both, the undistortion and redistortion will be formed of a sequence of the same length as the input clip; one ST-Map for each frame. Click Export ST-Maps to write the ST-Map clips to disk.

Working With Proxies

You can use ST-Maps even if you use proxies for tracking. ST-Maps store normalised pixel coordinates, so they will still sample from the correct location in the image after scaling them up or down. For example, if you are tracking half resolution proxies, you will have to double the size of the resulting ST-Maps to use them with the original plates.

03. Using the ST-Maps to Undistort and Redistort Your Plates

Most compositing applications provide ways to read ST-Maps and apply the encoded transformation, thus letting you undistort and redistort your plate using PFTrack’s lens distortion model. Below is an illustration of how to use ST-Maps in Nuke.

Using ST-Maps in Nuke

Use an STMap node to undistort or redistort your original clip. In the example tree above, STMap1 is used to undistort the original footage, using the undistort ST-Maps exported from PFTrack. The output is then connected to STMap2 and lens distortion reapplied using the redistort ST-Maps from PFTrack. The result will be identical to the original input clip.

In the STMap node, make sure to set UV channels to rgb, so that the correct channels will be read.

Related Posts

7 Comments

  1. Has anyone found a good workflow for using these STMaps in After Effects?

    Reply
  2. Is there any way to just force it to export one frame? It’s frustrating that it doesn’t get that if the lens doesn’t change it should only export 1 frame…

    Reply
    • You can do this by using a separate Export node for your ST-Maps. Connect your tree to an Optical Flow node, and that Optical Flow node to an Export node you want to use for exporting distortion. In the Optical Flow node, check “Enable retime” in the bottom right corner, then select “Nearest frame” as the “Render mode” and change the “Frame count” to 1.
      This will retime the clip to the first frame only, which you can then use to export the ST-Map from.

      Reply
  3. Everything is cool if you have access yo nuke, but how can I do this in After Effects?
    Regards.

    Reply
  4. How can you use the ST-maps in After Effect CC?

    I’ve seen a plugin by RE:maps, guys who do Real Smart Motion Blur.
    But not sure if it uses the same method?

    Is this the only way it could be read in AE or can we use it without purchasing additional plugins to read maps?

    Reply
  5. Hi,

    Is there an automated method for generating a _nuanced_ STmap from a lens grid (checkers image) inside PFT ?

    My understanding is that the current PFT undistort node requires a human to draw several lines on an input image, which is slower and can’t possibly be more accurate than Nuke’s “LensDistortion / Grid Analysis” automatic generation.

    In both cases, the image analysis is of straight lines and generates a distortion node defined by 4 numeric parameters.

    The mathematical description of the distortion, while nice, is not utilizing the full potential of an ST-Map which potentially can describe distortion pixel by pixel, much more accurate than the four numeric parameters.

    So, would you know how can we generate a pixel based nuanced ST-map, that isn’t just calculated from the good old four numeric parameters?

    Please advise,
    Thank you,
    Dani.

    Reply
    • You can use the PFTrack’s Build Lens node to construct a lens model from a calibration grid.

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This

Share This Post On: