Written Guide
Last updated: July 2024
GeoTracker for Nuke allows for quick and stable 3D object tracking in an almost automated way using a matching geometry and just a few keyframes. Extensive controls make it easy to place geometry on top of the target object in the plate, track and adjust its position throughout the footage. GeoTracker supports automatic focal length estimation, tracking with rigged 3D models and exporting results as a camera track.
Check out the Knobs Reference page to learn what every button does
Setup
Let’s create a basic setup for 3D object tracking. You’ll need two external elements: a plate, which is your footage and a 3D model of the object you’re going to track. Load your footage in Nuke with the Read (Image) node and connect it to the bg input of the GeoTracker node. Add the 3D model using the ReadGeo node. Alternatively, you can use our free ReadRiggedGeonode if Nuke can’t recognize the 3D model file format, or if you’re going to track an object with rigged geometry.
Both geometry and image sequence can be created and modified in Nuke prior to connecting them to GeoTracker.
A Few Words About Camera
You can use the cam input for connecting a Camera node in order to specify settings like focal length, sensor size (most importantly, horizontal aperture), etc of the real camera on the set. Knowing these values will significantly improve the tracking accuracy. If you’re unsure which camera and lens were used to film the footage, we highly recommend using the automatic focal length estimation option. Go to GeoTracker Properties > GeoTracker > Focal Length, switch focal length from camera node to fixed focal length and activate theestimate focal length option on the GeoTracker toolbar. For zoom shots choose varying focal length in the dropdown and then activate track with focal length estimation right below it. Now the changes in focal length value will be stored in the focal length keyframes on the timeline.
Important! GeoTracker treats the plate as if it has no lens distortion in it whatsoever. Therefore, getting a stable track in a wide angle shot may be challenging. We highly recommend undistorting your footage before connecting it to GeoTracker.
Pre-analysing
Take a moment to type in a name for the analysis file in GeoTracker Properties and press the analyse button next to it. This will lead to creating an advanced motion vector .precalc file in the specified folder. It will take some time but eventually speed up the tracking, and also let you re-use it in the future when tracking other objects or camera in the same plate.
Pre-analysing your plate is not mandatory, but tracking may be significantly slower without it.
Model Placement
Scroll through the timeline to select the frame in which the target object looks three-dimensional, or, at least, seen in full. Then press the center geo button on the GeoTracker toolbar, which centers the 3D model in the Viewer.
Next step is matching the 3D model with the target object. You can do it by creating pin-points on the surface of your geometry (left-click) and dragging them to the corresponding parts of the 2D object in the plate. The first three pins set the model’s orientation, scale and rotation. You can add more pins to adjust the mesh shape where necessary. To remove a pin, simply right click on it.
Note that the number of pins on the mesh does not have any effect on the actual tracking. We use them only as a tool for aligning the model when placing it in the initial frame or refining the tracking results.
When you set or adjust a pin in a frame, GeoTracker creates a manual keyframe that stores the current position of your model in 3D space. The tracking quality will greatly depend on the accuracy of the model placement. So take your time to match the 3D model with the target object the best you can. There may be two reasons why your model might not sit on the object perfectly: either the model itself is not accurate enough, or the camera focal length value is wrong. Use the correct focal length value or try the estimate focal length option as described in ‘A few words about Camera’ above.
You can jump between keyframes, create new or remove them using the keyframe control buttons on the GeoTracker toolbar.
Tracking
Once the model is in place, you can proceed to tracking. You can track forward or backwards, continuously or frame by frame using the tracking buttons on the GeoTracker toolbar. While tracking, you may notice the mesh slipping away from the target object in some of the frames. If it’s really off, you may pause tracking by pressing Cancel, adjust its position in that frame by dragging pins or creating new ones if necessary, and then track on from that point. You may need to make those adjustments a number of times until the end of the tracking range. Everytime you adjust the mesh position, GeoTracker creates a manual keyframe.
Refining
As you adjust the model position in a frame, GeoTracker creates a new manual keyframe containing the updated translation and rotation data. But you still need to adjust the position of your model everywhere between those manual keyframes. This can be done automatically with the Refinement option. Press Refine to fix the tracking data between the nearest keyframes or Refine all for automatic adjustment throughout the whole range.
Keyframes
The GeoTracker toolbar holds extensive controls for clearing the tracking data in the specific range. You can remove all tracking data between keyframes by pressing Clear Between Keyframes or remove all tracking data and keyframes to the left or right of a certain keyframe by pressing Clear Backwards and Clear Forwards. Clear All removes all keyframes and all tracking data but keeps the pins intact. If you want to remove pins, use the dedicated button on the toolbar.
Make It Stable
Viewport Stabilisation
This feature lets you get more visual control over the tracking accuracy. Simply press the Stabilize_view button on the toolbar. If you want to stabilise the Viewer around a certain pin or a group of pins, just select them when the Viewport Stabilization is on. Now during the playback or tracking, or when switching between the keyframes, the view will be locked to your Geometry, even when you pan or zoom.
Masking
GeoTracker has an additional input for 2D masks which you can use to exclude certain areas in certain moments from tracking. You may need that in case of occlusions like props or other elements getting in front of the object you’re tracking. Create a roto mask using the Nuke’s Roto node, connect it to the mask input of GeoTracker (right side of the node) and select mask alpha or mask alpha inverted in the mask dropdown menu on the GeoTracker Properties pane.
In most cases GeoTracker is able to filter out occlusions automatically. So we suggest trying without a mask first. Only when you see that the element which blocks the view of the target object really makes the 3D model slip away, does it make sense trying to use a mask. Also, note that using masks does not affect the pre-analysis file. They are activated only while tracking.
Surface Mask
Surface mask is the built-in GeoTracker feature that lets you draw a mask right on the surface of the 3D model, thus excluding the selected polygons from tracking. This can be very useful when you need to eliminate things like reflections on glossy surfaces, or hair falling in the face, or when the object is partly covered up. Here again, we recommend trying to track it first without a mask because in most cases GeoTracker is able to snap the 3D model to the target object quite effectively throughout the footage.
To draw a mask on the surface of the 3D model, select the brush on the GeoTracker toolbar and then mark the polygons you want to exclude from tracking. You can do this either in 2D or 3D view. Using controls on the GeoTracker toolbar you can adjust the brush radius, turn the mask on and off with the ignore option and remove the mask with the Clear button.
User Tracks (Helpers)
Another way to ensure stability of your 3D track is using the tracks created with the built-in Tracker node. Already existing tracks can be added to your 3D tracking pipeline in the UserTracks tab on the GeoTracker Properties pane. Note that only trackers located on the target object will count. Those trackers that are located anywhere else in the frame will not be used by GeoTracker.
Don’t go crazy creating too many user tracks. Most likely, all valuable points on the target object are already being used by GeoTracker to recreate its motion in 3D.
Smoothing
Smoothing is an effective way to eliminate annoying jittering that you may encounter in translation and rotation of your 3D model, and also focal length estimation along the track. You can find smoothing settings in the Smoothing tab on the GeoTracker Properties pane. Try different values for Rotation, Translation and Focal Length, depending on the origin of jittering. Remember that you need to set smoothing values before tracking or refining the selected range for smoothing to be applied.
Smoothing settings are not global. They only apply to the current range you’re tracking or refining, which means you can use different smoothing settings for different parts of the shot.
Rigged Geometry
If you need to track an object with a rigged geometry, you can do it by loading a skeleton-containing 3D model using the KeenTools ReadRiggedGeo node. The node is basically the same as the ReadGeo node, but it also supports 3D models with skeletons. After connecting the ReadRiggedGeo node to GeoTracker we suggest setting up bone constraints in the Results tab to restrict unwanted bone rotation and translation.
Rigged geometry tracking workflow is the same as always:
- Select initial frame
- Align geometry in the viewer by creating pins on its surface and dragging them to the matching points on the target object
- Track forward or backwards
- Pause and adjust mesh whenever it’s off
- Refine to automatically update the track between the nearest keyframes
Exporting Results
Results tab
GeoTracker outputs translation and rotation of the 3D model that was fed into its geo input across the tracking frame range. You can simply feed the animated 3D model into the geo input of the next node in the chain. But we recommend using the GeoTracker export options which will give you additional possibilities for utilising your tracking results. Find all export options in the Results tab on the GeoTracker Properties pane. Select the one that suits for your purposes from the Export dropdown and then press the Export button.
In the Results tab you can also see the translation and rotation values for the current frame, as well as the focal length value.
Exporting an Object Track
TransformGeo contains the same data as the GeoTracker node: translation and rotation of the 3D model across the frame range. All adjustments further made to the 3D model position in the GeoTracker node will be automatically transferred to the TransformGeo node as long as the link output option is selected.
When tracking with rigged geometry, you can export each bone animation as a separate TransformGeo node.
You can export your tracking results as an Axis node if you only need an animated null object instead of the 3D model.
Exporting a Camera Track
By selecting Camera you export your camera with the adjusted settings such as focal length.
You can also export your tracking results as a camera track by selecting the tracked_camera option. In that case the 3D model will remain static and the camera will move in the 3D space relevant to it.
Unbreak Rotation
If your model rotates more than 360° between certain keyframes, we recommend using the Unbreak Rotation option. The algorithm behind it will go through all keyframes and make rotation continuous thus avoiding 360 degree jumps. That means that if rotation was 355° in the keyframe #24 and 10° in the keyframe #25, pressing this button will add 360° to keyframe #25, so the resulting rotation of the model will be 370°. It works backwards as well: going from 10° to 355° will make it -5°.
You have to be careful if your model rotates both forward and backwards between the nearest keyframes — then this operation will most likely make things worse.
Export bone animation as .chan file
When tracking with rigged geometry, you can export bone animation as a .chan file. Open the Results tab, select all bones (rows) you want to export, right click on them, choose File and then select Export Ascii. In the popup window you will see rotation x,y,z and translation x,y,z as six channels for each bone you have selected. Save file adding a .chan extension to it.
Links
Download GeoTracker for Nuke
Watch our video tutorial on the basics of geometry tracking.
Follow us: Facebook, Twitter, Instagram, YouTube, Discord
