Photo Input

Overview  |  Source  |  Undistort  |  Position  |  Metadata  |  Camera  |  Camera Model

Overview

The Photo Input node is used to load photos and define the parameters of the cameras that are passed down the tracking tree for photogrammetry processing.

Along with the media thumbnail and the node name (which defaults to that of the loaded media), this icon also displays the number of photos in the node above the thumbnail.

EXIF Metadata

The Photo Input node will only read photos with EXIF metadata defining the camera model and focal length, as provided by digital still cameras. In order to load photos, please ensure that metadata is not removed by any pre-processing operations.

If your image data does not contain suitable EXIF information (for example, if it has been captured with a movie camera, or metadata has been removed from the files) then you should use the Clip Input node to load your media, and the Select Frames node to convert into a suitable set of frames to use with the photogrammetry tools.

User Interface

When a Photo Input node is activated by double-clicking on it with the left mouse button, the PFTrack GUI will switch to display the Cinema Window, a single Viewer Window and the following important areas:

(1) The Photo Input Parameters:

(2) The Metadata window

(3) The Camera Body panel

(4) The Camera Lens panel

The Photo Input Parameters are split into separate sections, selected by clicking the section tab on the left of the panel.

Source: The source media information. This is also where media can be loaded from disk.

Undistort: The Undistort panel where lens distortion correction parameters can be defined.

Position: The GPS Position panel, where GPS data can be examined.

Virtual Camera Model

In order to perform photogrammetry tasks, PFTrack must create a Virtual Camera Model that matches the real camera as closely as possible. The tools available in the Photo Input node must be used to create this virtual camera model before running any photogrammetry calculations. This includes, amongst other things:

- The camera focal length

- The camera sensor size

- The type of lens distortion camera

For photogrammetry, PFTrack requires that the camera focal length be available as metadata in your photos. The other information may also be available as metadata, or may be specified manually.

The following sections describe how photos can be loaded and the parameters of the virtual camera model set:

Source

The source panel is where photos can be loaded, and also lists important information about the source material such as its location on disk, media type and duration.

The Camera Model buttons are used to specify the type of camera model used for the media. Further details are given below in the Camera section.

Loading and managing photos

In order to load photos into the Photo Input node, they must contain metadata, specifically the camera focal length. If the photo files do not contain any focal length measurements, they cannot be loaded into PFTrack.

Load Photos: Clicking this button will display the File Browser where the photos can be located on disk and loaded into the node.

Remove Photo: Once photos have been loaded, clicking this button will remove the current photo from the dataset. Note that all photos must be removed from the node before it can be deleted from the tracking tree. This operation will remove the photo from the Photo Input node, but will not affect any of the original media files on disk.

Remove All: Once photos have been loaded, clicking this button will remove all photos from the dataset. Note that all photos must be removed from the node before it can be deleted from the tracking tree. This operation will remove the photos from the Photo Input node, but will not affect any of the original media files on disk.

Relink Photos: If the location of image files on disk has changed since they were imported, the Cinema window will display Missing Media. Clicking this button will display the File Browser allowing the original source material to be found. Note that the resolution and photo count must be the same as the original photos in order for them to be re-linked.

View All/Current: Clicking this button will switch between showing all photo icons in the Viewer Window, or only the icon for the current photo.

Note: if you have already attached nodes down-stream for photogrammetry processing, removing photos from the Photo Input node may adversely affect the data stored in those nodes. In this case, it is recommended to either remove unwanted photos before processing, or re-process any down-stream nodes after removal.

Undistort

The Undistort panel can be used to specify how lens distortion should be corrected for. It is also the place where an OpenEXR ST-map can be loaded from an external source and applied to the photos if required.

The Lens model menu is used to specify how to correct for lens distortion for this clip:

- None: No lens distortion correction will be applied.

- From Preset: If a camera preset is being used, selecting this option will take the lens distortion model from the camera preset and use it to undistort the photos immediately.

- Automatic: When this option is selected, the amount of lens distortion will be estimated automatically and removed during the photogrammetry calculation process.

- From ST-Map: Selecting this option will allow an OpenEXR ST-map image to be loaded from an external source and applied immediately to all the photos to remove lens distortion.

ST-Map

In order to load an external OpenEXR ST-map from disk, select From ST-Map from the Distortion model menu. The ST-map image can be managed using the following buttons:

- : Open the File Browser to load a single-frame ST-Map image.

- : Clicking this button will remove the ST-Map from the Photo Input node (Note this will not affect the source material file on disk).

The resolution of the undistorted photos will be defined by the resolution of the ST-map image. Note that this will affect the sensor area and field-of-view displayed in the Camera panel.

- ST-map image: This displays the name of the image providing the ST-map, if available.

Framing

After a photo has been undistorted, controls are available to adjust image cropping if required (Note: these options are not available when using an ST-map).

By default, a photo will be undistorted to the same resolution as the original image. This means some pixels will be lost around the boundary of the image. Adjusting the image crop size will allow these pixels to be recovered if desired.

- Image crop: This displays the current width and height of the undistorted image.

- Edit: Selecting this button will allow the image crop area to be adjusted manually in the Cinema window.

- R: Clicking this button will reset the image crop to match the original source material resolution.

- Macroblock: Enabling this control will round the undistorted frame resolution up to the nearest 16x16 macroblock size. This can be useful when exporting undistorted clips in certain formats to ensure the resolution is compatible with the codec.

- Frame aspect: Enabling this control will ensure the original frame aspect ratio is maintained when adjusting the image crop.

The following buttons can be used to fit the image crop to the undistorted image data:

- : Clicking this button will fit horizontally to ensure no empty pixels are visible at the left and right-hand edges of the image.

- : Clicking this button will fit to the full horizontal extents of the undistorted image.

- : Clicking this button will fit vertically to ensure no empty pixels are visible at the top and bottom edges of the image.

- : Clicking this button will fit to the full vertical extents of the undistorted image. Window

Position

The Position panel displays useful information about the GPS position of the current photo. If metadata has been connected (including orientation or height, if available), it will be displayed in the panel, along with the metadata tags being used for each parameter:

To open an external web browser displaying the GPS location on a map, click the button.

Metadata

Metadata is useful information recorded by the camera camera whilst taking photos. For photos it generally comprises of EXIF or XMP data describing the camera focal length and sensor size, and possibly GPS location and camera orientation.

All metadata that can be used in PFTrack for photogrammetry will be displayed in the Active Metadata window at the top-left of the PFTrack interface.

Active Metadata Window

The Active Metadata Window appears at the top-left of the UI, in place of the Workpage window. The top four sections contain important information that PFTrack has identified and is able to use:

Each section displays the name of the camera parameter in the Input column, the metadata value in the Value column, and has a Connect toggle on the right.

(1): Camera Body, containing information related to the camera body and sensor.

(2): Camera Lens, containing information related to the camera lens.

(3): GPS Position, containing information related to GPS position and altitude. This can be toggled between EXIF and XMP depending on the types of metadata available.

(4): IMU Data, containing information captured by IMU devices such as those attached to drones.

If no metadata is available that is relevant to each section, it will be displayed as Not available.

The full list of all metadata in the photo is shown in the Photo Metadata section (5). The important values that PFTrack has identified are highlighted in blue. Individual sections of the Photo Metadata list can be opened and closed using the white arrow buttons to the right of each section title. All photo metadata can be accessed down-stream from the Photo Input node using Python scripting if required.

Further information about how to tell PFTrack which metadata is important for your media is available in the Metadata Tags section.

Camera Body Metadata

Camera body section contains information describing the size of the camera sensor. This information is essential to PFTrack when using camera focal lengths measured in real-world values such as millimeters, as it helps PFTrack to convert a focal length in millimeters into a horizontal or vertical field of view.

The camera sensor information can be found from EXIF metadata in two locations:

1. The EXIF.Photo.FocalPlane tags. If present, these tags provide the sensor size of the camera exactly.

2. The EXIF.Photo.FocalLengthIn35mmFilm tag. If the focal plane tags are not available, the sensor size can be estimated by using the value of the equivalent camera focal length assuming a 35mm film camera.

It's important to remember that although these tags should provide the exact sensor size, it's the responsibility of the particular camera manufacturer to encode this data properly, so the values provided here may not always be accurate, or available at a suitable precision. If in doubt, we recommend checking with the camera documentation or manufacturer to determine the correct sensor size.

Further information about the relationship between focal length and sensor size is available in the Camera section.

Camera Lens Metadata

Camera lens metadata comprises of the focal length. In order to use the camera focal length correctly, PFTrack must also know the camera's sensor size.

Further information about the relationship between focal length and sensor size is available in the Camera section.

GPS Position

GPS position metadata comprises of data specifying the latitude, longitude, and possibly altitude of the camera when the photo was taken. This may come from either EXIF or XMP metadata.

IMU Data

If the photos were captured by a drone or similar device, IMU data may be available that describes either camera position or orientation. This information can often be more accurate than GPS data alone, and is especially useful when altitude above ground level (as opposed to altitude above sea level) has been recorded for each photo.

Connecting Metadata

If metadata is available to use, it must be connected to PFTrack's camera in order to assist with any operations such as photogrammetry. This is achieved by clicking the Connect toggles next to each piece of metadata.

Due to the nature of how metadata is often recorded, the values present in the files may not always be as accurate as is needed for the purposed of photogrammetry. Because of this, the Connect toggle can be set to one of three states:

: When in this state, metadata is not connected and does not affect the camera (this is the default state).

: This state indicates the metadata value is to be used as a Hint to the camera.

: This state indicated the metadata value is to be used as an Exact value in the camera.

Not all states are available for each camera input. Further information on how metadata affects the camera in PFTrack is available in the Camera section.

Camera

In order to perform any photogrammetry tasks, PFTrack must build a Virtual Camera Model that represents (as closely as possible) the real camera that was used to capture the shot.

This virtual camera model can be constructed in various ways, and there are three important areas of the Photo Input UI which area used to define it:

(1): The Camera Model section options.

(2): The Camera Body parameters.

(3): The Camera Lens parameters.

Camera Model

The Camera Model options are used to specify how to build the virtual camera model in PFTrack. There are two options available, each of which provides different levels of control over the camera body and lens parameters:

- : This is the Preset mode, and allows a pre-built camera preset to be selected from the preset menu below the buttons. If no preset is available that can be used for your media (e.g because of frame resolution or aspect ratio differences) then this button will not be available.

- : Enabling this option selects the Metadata mode, and allows the individual camera parameters to be specified using metadata or manually edited by the user. Metadata mode is the default setting.

Underneath the buttons is the Preset Menu where a preset can be chosen when using the Preset mode:

Further information on how to build camera presets is available in the Camera Presets section.

Preset Mode

When PFTrack attempts to calculate a camera position using photogrammetry, it must estimate various camera parameters such as the position and orientation of each photo, along with the camera focal length and lens distortion if necessary. This is a complicated process, and as such it helps to specify as much information about the camera as possible beforehand.

The Preset mode provides the most reliable approach to photogrammetry, because camera presets contain information about the camera lens and distortion that is used directly during the photogrammetry process.

When Preset mode is enabled, camera lens metadata can still be used to describe the focal length when using a zoom lens.

Metadata Mode

If a camera preset is not available, enabling the Metadata mode will allow information about the camera sensor size and focal length to be obtained from metadata or entered manually.

When using the Metadata mode, lens distortion can still be estimated automatically by PFTrack if necessary.

Please note that it is important to ensure data is entered correctly. For example, there is an important link between the camera sensor size, the lens focal length, and the field of view of the virtual camera that PFTrack is using. If either the sensor size or focal length values are incorrect, the field of view of the virtual camera will not match that of the real camera and this may cause problems during the photogrammetry process. Significant errors may in fact prevent the camera being solved entirely.

Camera Body

The Camera Body panel contains information related to the camera sensor size.

When in Preset mode, the camera type and serial number (if available) stored in the camera preset will be displayed, along with the sensor width and height:

When in Metadata mode, the sensor width and sensor height values can be edited manually if required:

The state of the sensor size values are colour-coded to indicate their reliability:

Red: indicates that the size has not been set, or is unknown. These values must be changed before the virtual camera model can be completed.

Orange: when reading values from metadata, this indicates that the value has been estimated, rather than read directly. Values with this colour should be checked to make sure the data is suitable before continuing.

This colour coding is reflected in the virtual camera and frustum displayed in the Viewer Window.

Camera Lens

The Camera Lens panel contains information related to the camera lens, and the virtual camera model that PFTrack is using. The type of lens and focal lengths will be updated automatically using metadata available in the source photos.

When in Preset mode, the lens model and type will be displayed, along with the focal range (Fl range) for zoom lenses, and the focal length (Fl) for prime lenses.

When in Metadata mode, the focal range and focal length will be displayed:

Connected Metadata

When metadata is available in the loaded media, it can be connected to the virtual camera model using the Connect toggles in the Active MetaData Window.

As metadata is connected, the metadata indicators next to each parameter will change according to how the metadata is being used:

: This indicates that no metadata is connected to the camera parameter.

: This indicates that metadata is connected, but is potentially unreliable and therefore only being used as a hint.

: This shows that metadata is connected and the metadata value is being used exactly as specified in the media.