The tracker works differently than the Nuke one regarding the “Transform” part. For the tracking itself, almost everything is the same. Basically, in Nuke they can only output a CornerPin with exactly 4 points, and they map 1 track to each corner of the CornerPin. For the Transform node they may 1 (translation only), 2, or N points to find the final transformation, however that will never be something other than a [similarity] (https://en.wikipedia.org/wiki/Similarity_(geometry).
In Natron, we offer the possibility to compute a CornerPin with N points, that is an homography.
This is way better because the more tracks you use to compute that CornerPin, the more robust it will be.
An homography is typically used to contain information about a perspective deformation, whereas a similarity is more constrained.
In The Transform tab, that’s what we call “the model”.
Basically the problem we are trying is to fit a model (=SImilarity or Homography) so it is the closest to the correspondences of N points. Each correspondence is the position of a track at the reference frame and its position at the tracked time.
Hence the more correspondences you have (=the more tracks), the more robust the homography is in the region where you tracked features.
The Fitting error parameter in the transform tab, is an indication of how much difference there is in pixels between the reference point on which we applied the computed transformation and the original tracked point. This error is summed across all tracks and squared to give an estimate of the quality of the model found.
For each tracked frame, the correspondences we use to compute the CornerPin are the tracks that are enabled at this frame (i.e: the Enabled parameter is checked at this time) and that have a keyframe on the center (i.e: they successfully tracked).
Basically, when you press Compute, it computes the model (CornerPin/Transform) with all the tracks that meet the aforementioned requirements over all keyframes.
When Compute Transform Automatically is checked, whenever a parameter that has an effect on the output model is changed, this will recompute the Corner/Pin transform over all keyframes again.
The parameters that have an effect on the output model are:
The motion type
The Transform Type (i.e: Similarity or Homography ?)
The Reference frame
Smooth: this can be used to smooth the resulting curve to remove some of the noise in the high frequencies of the CornerPin/Transform. Note that in “Add Jitter” mode, you can increase High frequencies to simulate a camera shake that follows the original camera movements.
Robust model: this is quite complicated, but in short: When trying to find a model that best fits all correspondences, you may have correspondences that are just wrong (bad tracking for example). These bad correspondences are called outliers, and this parameter when checked tells we should not take into account those outliers to compute the final model. In most cases this should be checked. However sometimes, the user may have for example required to compute an homography (i.e CornerPin), but the given tracked points (correspondences) just cannot make-up an homography. In this case, if the parameter were to be checked, it would fail to compute a model. If you uncheck this, it will take into account all the points and compute a model that averages the motion of all correspondences.
Also when Compute Transform Automatically is checked, the model will be computed automatically when the tracking ends.
We cannot compute the model after each track step (i.e: during tracking) because the model at each frame depends on the model at other frames since we may smooth the curve or add jitter.
So all in all it works differently than Nuke, the whole transformation computation can be more robust and happens as a second pass after the tracking is actually done.
One last thing: to compute the CornerPin in the “Transform” tab of the tracker, the to points are computed using the from points as reference.
Basically what happens is that the tracking outputs a transformation matrix at each frame. Then when computing the model, we apply this matrix to the from points at each frame to obtain the to points.
So if you were to change the reference points (i.e: the from points) with the Set to input RoD for example, then you would need to recompute the model at all frames, because the to points would just not be the same.
In the video that’s what I do:
First I disable the CornerPin so that even if my viewer is connected to the Tracker I don’t have any deformation going on, and I set the from points to be the RoD (bounding box) of the Roto shape at the reference frame.
When you export the CornerPin, it just links the parameters of the CornerPin to the ones in the tracker, so if you change something in the tracker transform tab the changes will reflect onto the CornerPin.
Hope this helps, if you have any suggestion to improve the workflow, don’t hesitate.
Basically what the Planar tracker will do in the future is automatically do all the steps for you: it will place markers inside the mask for you, track them and output a cornerPin from the bounding box of the roto shape.