Image Modelling

Usage  |  Controls

The Image Modelling node can be used to construct 3D polygonal models that match elements viewed by a tracked camera. A simple set of modelling primitives are provided that can be positioned in 3D space to match the image data, or new models can be constructed by connecting 3D tracker or point cloud vertex positions to form a polygon mesh.

The Image Modelling node can have any number of inputs, and these inputs can be either single frames or image sequences, but each must contain a tracked camera, and all cameras must be positioned correctly relative to each other (this is usually done by solving for all camera positions in the same Camera Solve node).

Note that it is important that cameras are tracked accurately, otherwise it may not be possible to position a modelling primitive in such a way that it accurately matches an object in the scene.

Usage

There are two ways to construct a 3D polygonal model: by positioning vertices in the scene and connecting them to form polygons, or by positioning modelling primitives. After they are created, primitives can be passed down-stream for use elsewhere (for example, to be textured and exported along with a camera path).

Splitting the Cinema Window

The main Cinema window can be split into two to assist with moving vertices or primitives. This can be used to easily see where vertices have been positioned in two frames (or two clips) at the same time. Editing can be done in either half of the split window, and clicking the Split button again will remove the split.

Further details of how the split window is used to automatically determine the 3D location of primitive vertices is given below.

Epipolar lines

If the Cinema window has been split, the 3D location of the vertex will be estimated more accurately by searching for a matching feature in the image shown in the other half of the window. A purple guide line is drawn in the other half of the window as the mouse moves, indicating where the matching features is expected to lie. The image is searched along this line, and a pin is created at the best matching location, allowing the 3D location of the vertex to be estimated using triangulation. An example is shown below, where the cursor is in the A half of the window, and the guide line is shown in the B half, along with the estimated pin location.

These guide lines are known as "epipolar lines", and their accuracy relies on there being some distance between the position of the cameras in the A and B halves of the split window. When modelling an object, it is often useful to locate a pair of frames where there is sufficient camera translation to accurately determine the location of the guide line, but where the change in perspective is not so severe that it complicates the task of identifying the matching features along the guide line.

More information on epipolar lines can be found here http://en.wikipedia.org/wiki/Epipolar_line

Modelling Primitives

Pre-built primitives are stored on disk as OBJ files in the primitives folder where the application was installed. Additional OBJ files can be used as modelling primitives by storing them in the user's Documents directory at primitives (the location of this directory can be changed in the preferences window). OBJ files must used a left-handed coordinate with the Y axis as the up direction.

Controls

Current clip: The clip that is being displayed in the main image window.

Polygon Editing

In order to position vertices and create new polygons, a primitive object must first be created, typically an Empty or Trackers/Points primitive.

Draw: Allow new polygons to be drawn in the Cinema window by clicking with the left mouse button to place new vertices and immediately connect them together into polygons. This is equivalent to creating vertices using the Create button and then connecting them together using the Connect button.

Connect: Allow existing vertices to be connected together to form polygons by clicking with the left mouse button in either the Cinema or a Viewer window. To close the polygon, either click on the first vertex again or hold the Shift key whilst clicking on a vertex. Once the polygon is closed, it will be triangulated and the triangles added to the primitive. Triangulation occurs from the point of view of the current camera (or the current viewpoint, if done in a Viewer window).

Triangulate: Automatically triangulate existing vertices in either the Cinema or a Viewer window by clicking and dragging with the left mouse button to select vertices for triangulation. If the Shift key is held, a lasso can be used to select vertices instead of a rectangular marquee.

Vertex Editing

Create: Create new vertices by clicking with the left mouse button in the Cinema window. If a split view is enabled, the 3D position of each vertex will be estimated automatically. Holding the Ctrl key whilst moving the mouse will snap the cursor to the nearest significant corner feature inside a small window around the mouse position. By default, the 3D location of the vertex will be determined by placing it a fixed distance in front of the current camera.

Move: Move and pin existing vertices to other image locations by clicking and dragging with the left mouse button. Clicking the right mouse button whilst hovering over a vertex will display a context menu allowing the vertex to be pinned to other primitives. Holding the Alt/Option will hide epipolar lines whilst creating vertices.

Insert: Insert new vertices into existing triangles by clicking with the left mouse button in either the Cinema or Viewer window. Vertices can be inserted into either the centre of a triangle or an edge.

Marquee: Select vertices by clicking and dragging with the left mouse button in either the Cinema or Viewer window. Holding the Alt/Option key will allow a lasso to be used instead of a rectangular area.

Merge: Merge all selected vertices together, averaging their 3D positions. This can be useful when automatically generated tracking points have been used to track the camera, and multiple tracking points correspond to the same image feature.

General Editing

Erase: Erase vertices or triangles from the primitive by clicking with the left mouse button in either the Cinema or Viewer window. When removing vertices, all triangles that connect to those vertices will also be removed.

Vertices

All / None: Select all or none of the vertices in the current primitive.

Delete: Delete selected vertices (and any relevant triangles) from the current primitive.

Primitives

Create primitives from the top row icons. Creating a "Trackers/Points" primitive will create a primitive without any triangles, but with a set of vertices that match the 3D tracker positions or point cloud vertices being passed into the Image Modelling node. This is useful if many points have already been generated and you want to connect them together to form a set of triangles. When creating a primitive of this type, a popup window will appear allowing you to select a tracker group or point cloud from which to fetch vertex positions. This can be useful when creating geometry for animated tracker groups, or trackers that have been solved using the Mocap Solver.

Edit Mode: The way that a primitive will move when pinning vertices in the image. Options are Rigid, Free or Deform.

Tx, Ty, Tz: Allow the primitive to be translated along either its X, Y or Z axes when pinning vertices in the image.

Rx, Ry, Rz: Allow the primitive to be rotated along either its X, Y or Z axes when pinning vertices in the image.

Sx, Sy, Sz: Allow the primitive to be scaled along either its X, Y or Z axes when pinning vertices in the image.

Translate: Display a translation manipulator, allowing the current primitive to be translated by clicking and dragging with the left mouse button in either the Cinema or Viewer window.

Rotate: Display a rotation manipulator, allowing the current primitive to be rotated by clicking and dragging with the left mouse button in either the Cinema or Viewer window. If the Alt/Option key is held, rotation will be limited to increments of 10 degrees.

Scale: Display a scale manipulator, allowing the current primitive to be scaled by clicking and dragging with the left mouse button in either the Cinema or Viewer window.

Delete: Delete the current primitive.

Duplicate: Duplicate the current primitive and the new copy placed in the same position.

Display

Selected Style: The rendering style used for the selected primitive in the Cinema and Viewer windows, provided that the primitive style is set to Default in the primitive list. Options are Wireframe, Hidden-line, Facet-shaded, Smooth-shaded, Transparent and Textured. When Textured mode is used, the image from the current camera frame will be projected onto the primitive and displayed in the Viewer windows only.

Unselected Style: The style used for rendering all other unselected primitives in the Cinema and Viewer windows, provided that the primitive style is set to Default in the primitive list.

Mesh Colour: Display a colour selection window that allows the colour of the current primitive mesh to be changed. The mesh transparency can also be adjusted by changing the Alpha value in the colour selection window.

Dual View: Split the Cinema window into two sections, displaying the current frame in the right-hand area. This can be used when triangulating vertex positions from different camera positions to form a 3D model. Clicking the button again will revert back to showing a single frame.

The recommended way of setting a split frame is as follows:

1. Select a current clip and move to a frame where vertices that need
to be modelled are visible

2. Click the Split button, copying that clip and frame into the B
half of the Cinema window

3. Change the frame displayed in the A half of the Cinema window by
dragging in the scrub bar with the left mouse button until the camera position has changed significantly, without distorting the overall image data too much.

Alternatively, an entirely separate clip can be positioned in the B half of the window. Please note, however, that if the viewpoint of that camera is very different from that in the A half, the position of the vertex may not be estimated accurately even if the guide line is accurately placed.

Show Ground: When enabled, the ground plane will be displayed in the Cinema and Viewer windows.

Show Horizon: When enabled, the horizon line will be displayed in the Cinema window.

Show Edges: When enabled, triangle edges will be outlined when using the Facet-shaded, Smooth-shaded, Transparent or Textured render styles.

Show Guides: When enabled, purple guide lines will be displayed when moving or creating vertices to help identify where vertices should be placed in different frames.

Show Vertex Names: When enabled, vertex names will be displayed in the Cinema and Viewer windows.

Show Hidden Vertices: When enabled, vertices that are hidden due to back-face culling will be displayed.

Default Keyboard Shortcuts

Keyboard shortcuts can be customised in the Preferences.

Draw Polygon	Shift+D
Connect Vertices	Shift+C
Triangulate	Shift+T
Create Vertices	Shift+V
Move Vertices	Shift+B
Insert Vertices	Shift+I
Marquee	Shift+M
Merge Vertices	Shift+G
Erase	Shift+E
Edit Rigid	Shift+R
Edit Free	Shift+F
Edit Deform	Shift+A
Dual View	Shift+S
Show Ground	Ctrl+G
Show Horizon	Ctrl+H
Show Edges	Ctrl+E
Show Guides	Ctrl+D
Show Vertex Names	Ctrl+N
Show Hidden Vertices	Ctrl+V
Next Clip	C