Select Frames

Media Quality  |  Overview  |  The Score Graph  |  Calculating Scores  |  Manual Selection  |  Controls

The Select Frames node can be used to select a number of frames from a movie clip using either a time or motion-based selection algorithm. The frames can then be used with photogrammetry processing nodes such as Photo Survey, Photo Cloud and Photo Mesh.

The movie clip that the Select Frames node will process must be provided by the Clip Input node, and must have a known sensor size and focal length.

The motion of the movie camera can be solved previously using either the Camera Solver or Survey Solver nodes if desired, although this is not a requirement.

An example tree showing the Select Frames node is provided below:

In this case, the Select Frames node is attached after the movie camera has been solved. This means both time and motion-based selection algorithms are available to use. Additionally, after frames have been selected, a Photo Cloud node can be attached directly below because the camera positions are already known.

Alternatively, if the camera has not been solved, the Select Frames node can still be used but only with time-based frame selection:

In this case, the Photo Survey node should be used to solve for the camera positions before attaching any other photogrammetry processing nodes such as Photo Cloud.

Media Quality

For photogrammetry, when using clips shot with a movie camera it is important that good quality images are available that are free from artefacts such as motion blur, defocus and rolling shutter. It is also important that the scene is well-lit and does not contain large moving objects.

Ideally, image quality should be similar to when using a digital stills camera in terms of image sharpness. If many moving objects are visible, they can be masked out using the Roto node before or after frames have been selected.

If the movie clip has already been tracked and solved, it is also important to ensure that the camera field of view is accurate, and there is enough parallax in the shot for the photogrammetry calculations to succeed.

Not all movie clips will be suitable for photogrammetry, and for the best results we recommend using a digital stills camera and the Photo Input node whenever possible.

Overview

The Select Frames node will selected a subset of the frames in a movie clip to use as photographs for photogrammetry processing. As frames are selected, they are scored according to how well they will perform in photogrammetry tasks and displayed in the Score Graph.

For example, widely-spaced frames that do not view the same part of the scene will be scored poorly, whereas frames providing a good overall coverage of the scene will be scored highly.

For long movie clips this will mean processing time is not wasted on redundant frames, whilst also ensuring that the overall quality of the photogrammetry is not compromised.

There are two different ways in which frames can be selected:

1. Motion-Based Frame Selection

Motion-based selection is activated using the button, and is available when the movie camera has been tracked and solved up-stream using either the Camera Solver or Survey Solver nodes.

Motion-based frame selection examines the motion of the solved movie camera and the position of the 3D tracking points. Frames are selected in a way that attempts to ensure any depth maps generated by the Photo Cloud node are of a good quality, whilst not introducing redundant frames that are unlikely to improve the overall quality of the photogrammetry.

2. Time-Based Frame Selection

Time-based selection is activated using the button, and can be used regardless of whether the movie camera has been tracked and solved up-stream. This mode uses a fixed time-period to select frames for photogrammetry.

For example, for a slow moving shot one frame every second may suffice, whereas a faster moving shot may require one frame every 0.5 seconds to ensure a good coverage of the scene.

The Score Graph

The Score Graph is where predicted photogrammetry quality scores are displayed, and are colour-coded accordingly where red represents a low score and green a high score.

When the movie camera has been tracked and solved up-stream, these scores will be calculated automatically, and represent the predicted quality of depth-maps generated by the Photo Cloud node.

For example, the following image shows predicted depth-map quality scores when using time-based frame selection with a time period of 5 seconds. This means one frame is selected every 5 seconds, resulting in a sparse selection of frames. It will therefore be more difficult to estimate depth maps in the Photo Cloud node and the graph scores are low to illustrate this (1).

The selected frames are displayed using camera icons in the Viewer Window, along with photogrammetry scores for each tracking point (2). In this case, each point is coloured according to the number of selected frames that can see the tracking point. Green indicates that the point is visible in enough cameras to reliably estimate depth.

By reducing the time period from 5 seconds to 2 seconds, more frames are selected from the movie clip (3), and the predicted scores are improved (4):

Calculating Scores

If the movie camera has not been solved and 3D tracking points are not available, only time-based frame selection is available. Photogrammetry scores can still be calculated by enabling the button. This process may take a short time to complete, especially for longer shots, so a progress window will be displayed as scores are calculated for each selected frame.

In this case, the scores represent the predicted accuracy of matching frames using the Photo Survey node, rather than the predicted depth-map quality.

Manual Selection

Regardless of whether motion or time-based selection is used, the Select Frames node can also be used to manually add or remove frames as desired.

Manual mode is available when scores are displayed in the Score Graph, and is enabled by clicking the button.

When Manual selection is active, the Score Graph will update dynamically to show how the photogrammetry scores will change if a new frame is added:

This preview can be used to determine good locations to add additional frames if scores in one part of the shot are particularly low. Clicking the button will add the current frame to the graph and update scores accordingly:

Manually added frames are shown with a blue indicator, and can be removed individually by clicking the button, or cleared entirely using the button.

Controls

- . This button is available when the movie camera is not already solved, and enabling it will start the process of calculating photogrammetry scores.

- . Clicking this button will enable Manual mode, where individual frames can be added or removed.

- . This button can be used to add the current frame when in Manual mode.

- . This button is used to remove the current frame when in Manual mode.

- . Clicking this button will remove all manually added frames.

- . This button enables time-based selection mode. The time period can be changed using the edit box, and represents the time separation in seconds between frames.

- . This button enables motion-based selection mode, and is only available if the movie camera has already been tracked and solved upstream. The motion threshold used for frame selection can be adjusted using the slider if required.

- . This button will toggle the View Mode between displaying tracking point predictions for all points, or only those visible in the current frame. This button is only available when the movie camera has already been tracked and solved upstream.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Manual	Shift+M
Add Frame	Shift+A
Remove Frame	Shift+R
View Mode	Shift+V