GeoTracker is a plugin for Foundry Nuke that can be used for tracking rigid and deformable 3D models without the usual hassle associated with tracking job. It makes tracking process so easy that what usually have been done by a team of tracking specialists, could be done by one of them in minutes. More than that, with GeoTracker you reduce the pipeline complexity, allowing compositing artists to complete this work instead of delegating it to a tracking department. That doesn’t mean of course that GeoTracker is useless for tracking pros — it has many advanced features and can be tweaked extensively allowing maximum precision when it’s needed.
You need at least two essential components before you start using GeoTracker: a footage where you want to track the model and the 3D model you want to track. The footage is usually imported into the Nuke script using
Read (Image) node. The 3D model can be loaded into Nuke using the built-in
ReadGeo node. Also our free
ReadRiggedGeo node may appear useful if
ReadGeo doesn’t recognise the format of your file or if you need deformable (rigged) geometry to be tracked.
The geometry may also be created or modified inside Nuke (as well as the image sequence) before being connected to GeoTracker.
There’s the third optional (but still very important) component — the
Camera node with the settings of the camera that was used to film the footage where you want to track the model. If you know what camera and what lens were used to film the footage, setting up the
Camera node can significantly improve the tracking quality. If you don’t know what camera was used, then it’s better to switch on
Focal Length Estimation on the main settings tab of GeoTracker. You can either estimate a fixed lens with
constant option or a zooming lens with
It’s also recommended to undistort the footage before connecting it to GeoTracker. It’s important since GeoTracker expects to get the perfect flat picture while in the real world the picture is often affected by lens distortion.
To make tracking process faster we decided to take out image analysing into a dedicated process. So now you can not only take a cup of coffee while GeoTracker analyses the video, but also you can automate this process, so all you need for tracking would be ready when you’re ready to work.
To launch the analysing process, enter the path for the analysis file (it’s going to be created) and press
Analyse button in the top section of main GeoTracker settings tab.
Placing a model
You start tracking with initialisation of the first keyframe: find the frame where you can easily see the object you want to track and press
Center Geo button in the left side of the second GeoTracker toolbar right over the viewer pane. The model will be placed in the middle of the frame and from this point you can manipulate it. Here you should know that when you change the position of the model in the frame manually, you create a new keyframe or modify an existing one.
One important thing to know is that GeoTracker considers keyframes being perfect, so the tracking results are highly dependent on the quality and consistency model’s position across all keyframes. By consistency we mean that polygons and geometric features of the model are matching surface areas and geometric features of the object in the footage in the same way in every keyframe you have.
In an ideal situation you’d have just one keyframe, but of course we don’t live in an ideal world, hence sometimes you end up having a number keyframes. Here we suggest to keep the quantity of keyframes as low as possible. Also, when in doubt where to set a keyframe, try to split the movement of the object into logical parts, usually that helps reducing the quantity of keyframes.
Keyframe control buttons are placed on the right side of the second GeoTracker toolbar (see the picture below). You can jump between them, create and remove keyframes using those buttons.
So, as you set up the initial position of the object in the first keyframe, you can proceed to automatic tracking. The tracking buttons are placed on the left side of the first GeoTracker toolbar. They allow you to track forwards and backwards, either continuous and frame-by-frame.
In some cases you can press the continuous tracking button once and get a clean perfect track, but often you can see the tracking being slightly inaccurate. In that case you can abort the automatic tracking process at any point (tracking results won’t be lost) and adjust the model position dragging pins you created before or creating new pins clicking on the mesh. Here again we’d recommend you to keep the number of pins as low as possible, otherwise you can end up having difficulties with changing the model position. And once you adjust the model position, you of course create or modify a keyframe.
Sometimes you want to remove a keyframe set next or before the current frame, or all keyframes before or after the current frame. You can use keyframe and tracking data cleanup buttons in the middle of the first GeoTracker toolbar (see the picture above).
Clear Between Keyframes clears all tracking data between the keyframes closest to the current frame on the left and on the right side, leaving the keyframes intact.
Clear Backwards removes data and keyframes before the current frame.
Clear All clears all data and all keyframes.
Clear Forwards clears tracking data and keyframes after the current frame. Note that even when you clear all keyframes and data, the pins you’ve set on the models are being kept intact, while model is being reset to the neutral state. If you want to remove the pins — there’s a special button for it on the second toolbar.
GeoTracker has another optional input called
mask. Using this input you can exclude some areas of a frame from tracking. A good example here would be when you have some occlusions in front of an object that you’re trying to track. In this case you can create a roto mask using Nuke’s built-in
Roto node, then connect it to the
mask input of GeoTracker and choose
mask alpha or
mask alpha inverted in the
Mask section of the main tab of GeoTracker settings. Here’s a suggestion: usually you don’t need to use roto masks to exclude some areas from tracking, so first try tracking without masks, and only when you that see something in the frame is really affecting tracking results — then it makes sense to try using a mask. Also worth mentioning that masks don’t affect pre-analysis, they’re only being used while tracking.
Surface Masks is a feature built-in in GeoTracker that can help you to exclude some polygons of a model from tracking. It can be reflective or moving parts for example. Here again we’d recommend you to try tracking without using Surface Masks, and only try them when you clearly see that you can improve tracking by excluding some parts of the 3D model.
To mark a surface to be excluded, press the brush button on the first GeoTracker toolbar and then mark the surfaces of the model with the brush in the
Viewer (either in 2D or 3D view). You can setup the radius of the brush and switch off the mask temporarily using
ignore checkbox. The
Clear button removes the mask.
User Tracks (Helpers)
You can also use tracks created with Nuke’s built-in
Tracker node importing the them on
UserTracks tab of GeoTracker.
Another way of improving tracking quality is smoothing the results. It can help to avoid jitter of motion, transformations, rotations and focal length estimation. On the tab
Smoothing you can find all the controls for it. Note that changes in smoothing have to be made prior to tracking process. If you’ve already done tracking and want to add smoothing, you can press
Refine All button and then tracking results will be updated.
One important thing to understand here is that smoothing settings are not meant to be global, they affect tracking results only while tracking is happening, so you can use different smoothing settings for different parts of the shot.
Using GeoTracker you can also track deformable geometry. To make it possible you first need to import a 3D model with a skeleton into Nuke. This is possible with our ReadRiggedGeo node. It works just like the built-in ReadGeo node, only it supports models with skeletons as well. After connecting ReadRiggedGeo to GeoTracker, it’s advisable to setup bone constraints to restrict unwanted bone rotation and transformations.
When you’re done with bone setup, you can proceed to tracking in 2D view — the workflow is the same: you create pins by clicking on the mesh and then drag them to corresponding positions setting up your first keyframe, then you launch tracking and refine the results when it’s required.
The output of GeoTracker node is translated and rotated 3D model you’ve passed to its input. Usually you just use it connecting GeoTracker output to the
geo input of the next node in the chain.
Also, you can export geometry transformations as
TransformGeo node. To do this, select
TransformGeo in the drop-down of the
Export section of GeoTracker
Results tab. Then press
Export button. If you switch on
Link Output, the exported
TransformGeo will be linked to GeoTracker node and all the changes applied to GeoTracker node after exporting will be transferred to the exported
TransformGeo node. If you don’t need this kind of behaviour, uncheck
In case you need the estimated camera settings, you can export them in a
Camera (3D) node. Select
Camera in the
Export section of the
Results tab in GeoTracker’s settings. If you leave
Link Output checkbox on, the exported
Camera (3D) node will receive updates if camera settings are changed inside GeoTracker. Note that there’s no point in exporting camera settings if you have a
Camera (3D) node connected to the input of GeoTracker— just use that camera.
You might also want to use the camera position across the frames where you’ve pinned the 3D model. In that case you can export
Camera Positions. Then the exported
Camera (3D) node will contain all the transformations in the keyframes where you’ve pinned the 3D model, like if the model was static in the default position and only camera moved around it.
For some special occasions you might want to export the 3D model you’ve built translated in 3D space in a way so the default camera in the default position would see it fitted in the frame where it was positioned. Then you can use either
Output Transformed Geometry checkbox and the output of GeoTracker node, or export the transformations to a
TransformGeo node and then modify the output of GeoTracker with the transformations stored in the exported node. For example, it might be useful when you create textures.
When you have a number of keyframes among which a model is rotating for more than 360°, we recommend using the
Unbreak rotation button. The algorithm behind it will go through all of the keyframes you have and will make the rotation continuous. It means that if your model was at 355° at the keyframe
#24 and at 10° at the keyframe
#25, pressing the button will add 360° to the keyframe
#25, so the model will be rotated at 370°. It works backwards as well: 355° after 10° will become -5°.
Considering that the
Unbreak Rotation algorithm alters the rotation of object with the closest period (180°), you have to be careful if your model rotates forwards and backwards between keyframes — then the
Unbreak Rotation button will most likely make things worse.