KeenTools GeoTracker Guide

KeenTools
KeenTools
11 min readOct 29, 2019

--

Last update: ver. 2021.3

GeoTracker is a plugin node for Foundry Nuke that can be used for tracking rigid and deformable 3D models without the usual hassle associated with the tracking job. It makes the tracking process so easy that what usually has been done by a team of tracking specialists, now could be done by one of them in minutes. More than that, with GeoTracker you reduce the pipeline complexity allowing compositing artists to complete this work instead of delegating it to a tracking department. That doesn’t mean of course that GeoTracker is useless for tracking pros — it has many advanced features and can be tweaked extensively allowing maximum precision when it’s needed.

Consider visiting our Knobs Reference page to check what every button does

Setup

You need at least two essential components before you start using GeoTracker: a footage where you want to track the model and the 3D model of the object you want to track. The footage is usually imported into the Nuke script using Read (Image) node. The 3D model can be loaded into Nuke using the built-in ReadGeo node. Also, our free ReadRiggedGeo node may appear useful if ReadGeo doesn’t recognise the format of your file or if you need deformable (rigged) geometry to be tracked.

The geometry may also be created or modified inside Nuke (as well as the image sequence) before being connected to GeoTracker.

Typical GeoTracker setup

There’s the third optional (but still very important) component — the Camera node with the settings of the camera that was used to film the footage. If you know what camera and what lens were used to film the footage, setting up the Camera node can significantly improve the tracking quality. If you don’t know what camera was used, then it’s better to switch on Focal Length Estimation on the main settings tab of GeoTracker. You can either estimate a fixed lens with constant option or a zooming lens with varying option.

It’s also recommended to undistort the footage before connecting it to GeoTracker. It’s important since GeoTracker expects to get the perfect flat picture while in the real world the picture is often affected by lens distortion.

Pre-analysing

To make the tracking process faster we decided to take out image analysing into a dedicated process. So you can not only take a cup of coffee while GeoTracker analyses the video, but also you can automate this process, so all you need for tracking is be ready when you’re ready to work.

Main settings tab of GeoTracker

To launch the analysing process, enter the path for the analysis file (it’s going to be created) and press the Analyse button in the top section of the main GeoTracker settings tab. Hint: you don’t need to enter the file extension, .precalc will be added automatically.

Placing a model

You start tracking with initialisation of the first keyframe: find the frame where you can easily see the object you want to track and press theCenter Geo button on the left side of the second GeoTracker toolbar right over the viewer pane. The model will be placed in the middle of the frame and from this point, you can manipulate it. Here you should know that when you change the position of the model in the frame manually, you create a new keyframe or modify an existing one.

Placing a 3D model into a frame

Keyframes

One important thing to know is that GeoTracker considers keyframes being perfect, so the tracking results are highly dependent on the quality and consistency model’s position across all keyframes. By consistency, we mean that polygons and geometric features of the model are matching surface areas and geometric features of the object in the footage in the same way in every keyframe you have.

Pinning a model

In an ideal case, you’d have just one keyframe, but of course, we don’t live in an ideal world, hence sometimes you end up having a number of keyframes. Here we suggest keeping the number of keyframes as low as possible. Also, when you are in doubt about where to set a keyframe, try to split the movement of the object into logical parts, usually that helps to reduce the number of keyframes.

Keyframe control buttons are placed on the right side of the second GeoTracker toolbar (see the picture below). You can jump between keyframes, create and remove them using the buttons.

GeoTracker toolbars

Tracking

So, as you set up the initial position of the object in the first keyframe, you can proceed to automatic tracking. The tracking buttons are placed on the left side of the first GeoTracker toolbar. They allow you to track forwards and backwards, either continuous and frame-by-frame.

In some cases, you can press the continuous tracking button once and get a clean perfect track, but often you can see the tracking being slightly inaccurate. In that case, you can abort the automatic tracking process any moment (tracking results won’t be lost) and adjust the model position dragging the pins you created before or creating new pins by clicking the mesh. Here again, we’d recommend you to keep the number of pins as low as possible, otherwise, you can end up having some difficulties with changing the model position. And once you adjust the model position, you of course create or modify a keyframe.

Refinement

Once you created your second keyframe you can ask yourself: ‘How do I fix tracking between the two keyframes I have?’ The answer is: using the Refine and Refine All buttons on the first GeoTracker toolbar. The first button will refine tracking results between the two closest keyframes. The second one — will refine all tracking data between all keyframes.

The refinement process tracks the object from both closest keyframes and then merges the two tracks giving you the best results of the two tracks.

The more or less real tracking process would look like this:

  • Place a model in the footage creating the first keyframe
  • Track the model
  • Abort tracking if the track is not perfect
  • Adjust the position of the object somewhere where the track becomes slightly lost, creating a new keyframe
  • Press Refine button to refine the tracking results between the two keyframes you have
  • Continue tracking from the last keyframe you’ve created

Of course, it’s a bit simplified because sometimes you want to adjust some other settings, but more on that later.

Removing keyframes

Sometimes you want to remove a keyframe set before or next to the current frame, or all keyframes before or after the current frame. You can use keyframe and tracking data cleanup buttons in the middle of the first GeoTracker toolbar (see the picture above). Clear Between Keyframes clears all tracking data between the keyframes closest to the current frame on the left and on the right side, leaving the keyframes intact. Clear Backwards removes data and keyframes before the current frame. Clear All clears all data and all keyframes. Clear Forwards clears tracking data and keyframes following the current frame. Note that even when you clear all keyframes and data, the pins you’ve created are being kept intact. If you want to remove the pins — there’s a special button for it on the second toolbar.

Viewport Stabilisation

One of the newest features of GeoTracker is the viewport stabilisation that allows you to control the keyframe consistency keeping during the tracking.

To switch the stabilisation on press the Stabilize_view button on the toolbar. By using this feature, you can retain the position of an object wherever it is, and stabilise it with any zoom level and viewport position.

A short overview of the viewport stabilisation

Moreover, it’s possible to stabilise the object around the selected pins as well. When the stabilisation is on, simply select one or more pins and start the playback. To return to the stabilisation around the object, deselect pins by clicking the left mouse button anywhere in the viewport outside of the object. Stabilisation works with any kind of frame changes, whether it is playback, jumping between keyframes, manual frame-by-frame jumping, or even the tracking process.

Masks

GeoTracker has another optional input called mask. Using this input you can exclude some areas of a frame from tracking. A good example here would be a case when you have some occlusions in front of an object that you’re trying to track. In this case, you can create a roto mask using Nuke’s built-in Roto node, then connect it to the mask input of GeoTracker and choose mask alpha or mask alpha inverted in the Mask section of the main tab of GeoTracker settings. Here’s a suggestion: usually you don’t need to use roto masks to exclude some areas from tracking, so first try tracking without masks, and only when you that see something in the frame is really affecting tracking results — then it makes sense to try using a mask. Also worth mentioning that masks don’t affect pre-analysis, they’re only being used while tracking.

Occlusions are being filtered out automatically

Surface Masks

Surface Masks is a feature built-in in GeoTracker that can help you to exclude some polygons of a model from tracking. It can be reflective or moving parts for example. Here again, we’d recommend you to try tracking without using Surface Masks, and only try them when you clearly see that you can improve tracking by excluding some parts of the 3D model.

To mark a surface to be excluded, press the brush button on the first GeoTracker toolbar and then mark the surfaces of the model with the brush in the Viewer (either in 2D or 3D view). You can set up the radius of the brush and switch off the mask temporarily using the ignore checkbox. The Clear button removes the mask.

User Tracks (Helpers)

You can also use tracks created with Nuke’s built-in Tracker node importing them on the UserTracks tab of GeoTracker. These tracks will be used only if they’re ‘inside’ the geometry in the frame. So if you want to improve the tracking of someone’s face and you’ve tracked an almost invisible freckle with Nuke’s tracker — that can help, but if you’ve tracked something outside of the face — it just won’t be used.

Here once again, we’d like to recommend you to not get crazy with creating many custom tracks with Nuke tracker and then feeding them into GeoTracker — most likely everything you’ll track is already being taken from the GeoTracker analysis file.

Smoothing

Another way of improving tracking quality is smoothing the results. It can help to avoid jitter of motion, transformations, rotations and focal length estimation. On the tab Smoothing, you can find all the controls for it. Note that changes in smoothing have to be made prior to the tracking process. If you’ve already done tracking and want to add smoothing, you can press the Refine or Refine All buttons and then tracking results will be updated.

One important thing to understand here is that smoothing settings are not meant to be global, they affect tracking results only while tracking is happening, so you can use different smoothing settings for different parts of the shot.

Deformable Geometry

Using GeoTracker you can also track deformable geometry. To make it possible you first need to import a 3D model with a skeleton into Nuke. This is possible with our ReadRiggedGeo node. It works just like the built-in ReadGeo node, only it supports models with skeletons as well. After connecting ReadRiggedGeo to GeoTracker, it’s advisable to set up bone constraints to restrict unwanted bone rotation and transformations.

GeoTracker bone constraints setup

When you’re done with bone setup, you can proceed to track in 2D view — the workflow is the same: you create pins by clicking on the mesh and then drag them to corresponding positions setting up your first keyframe, then you launch tracking and refine the results when it’s required.

Deformable 3D model of an excavator being tracked

Exporting Results

The output of the GeoTracker node is translated and rotated the 3D model you’ve passed to its input. Usually, you just use it to connect GeoTracker output to the geo input of the next node in the chain.

Also, you can export geometry transformations as the TransformGeo node. To do this, select TransformGeo in the drop-down of the Export section of the GeoTracker Results tab. Then press theExport button. If you switch on, the exported TransformGeo will be linked to the GeoTracker node and all the changes applied to GeoTracker after exporting will be transferred to the exported TransformGeo node. If you don’t need this kind of behaviour, uncheck Link Output. If you’ve been tracking a rigged geometry, then you have an option to export the TransformGeo node for each bone independently which might be more appropriate for some cases.

In case you need the estimated camera settings, you can export them to a Camera (3D) node. Select Camera in the Export section of the Results tab in GeoTracker’s settings. If you leave the Link Output checkbox on, the exported Camera (3D) node will receive updates if camera settings are changed inside GeoTracker. Note that there’s no point in exporting camera settings if you have a Camera (3D) node connected to the input of GeoTracker— just use that camera.

You might also want to use the camera position across the frames where you’ve pinned the 3D model. In that case, you can export Camera Positions. Then the exported Camera (3D) node will contain all the transformations in the keyframes where you’ve pinned the 3D model, like if the model was static in the default position and only the camera moved around it.

For some special occasions, you might want to export the 3D model you’ve built translated in 3D space in a way so the default camera in the default position would see it fitted in the frame where it was positioned. Then you can use either the Output Transformed Geometry checkbox and the output of GeoTracker node, or export the transformations to a TransformGeo node and then modify the output of GeoTracker with the transformations stored in the exported node. For example, it might be useful when you create textures.

When you have a number of keyframes among which a model is rotating for more than 360°, we recommend using the Unbreak rotation button. The algorithm behind it will go through all of the keyframes you have and will make the rotation continuous. It means that if your model was at 355° at the keyframe #24 and at 10° at the keyframe #25, pressing the button will add 360° to the keyframe #25, so the model will be rotated at 370°. It works backwards as well: 355° after 10° will become -5°.
Considering that the Unbreak Rotation algorithm alters the rotation of object with the closest period (180°), you have to be careful if your model rotates forwards and backwards between keyframes — then the Unbreak Rotation button will most likely make things worse.

GeoTracker is also able to export translations and rotations of bones as a .chan file for further working with rigged geometry. In this case, once you’ve tracked the object, open the Results tab and select all rows for the needed bones. Click the right button, select File, and choose the Export Ascii. In the opened window, you see six channels (rotations (x,y,z) and translations (x,y,z)) for each bone you have. Save the file using the .chan extension for the file name.

Links

Download KeenTools package for Nuke
Follow us: Facebook, Twitter, Instagram, YouTube

GeoTracker in action

--

--