How does tracking and linking actually work?


#1

Hey everyone, I am going crazy here since I just can’t figure out how to use tracking data to actually do something, like “moving” a roto.

I watched a bunch of Nuke Tutorials on that matter but none of them apply since they all seem to have a little “link” button next to the x y transform fields.

So how does this work? where is the magical “this object now moves with this track” button?


#2

Somebody should make a tutorial.

Basically:

  • Track some points inside your mask (shape)
  • In the Transform tab, set the transform to CornerPin and to match-move
  • disable the corner pin and set the from points of the corner pin at the reference frame where you want your object to move in (basically the bounding box of the shape to track)
  • export to corner pin
  • append your corner pin to the Roto node

In a future version we will have a planar tracker that will do that automatically for you in a single click.

To link parameters in Natron, it’s the same as in Nuke except that you drag and drop the widget of a parameter onto another one by holding CTRL


How to use a track made in 2.1?
#3

Thanks for the fast reply, but I still don’t get it.

I am totally at a loss here. Is there any where an example video how it should work, or at least a screenshot?


#4

Here is a small video I made extremely quickly to illustrate what I was saying:


#5

Oh man, thank you soo much, that made a lot of things much much clearer now.

Thank you very much, I appreciate it.


#6

Join us at NatronNation at on Facebook (https://www.facebook.com/groups/NatronNation/). We share and teach most of the needs of Natron so that the developers can keep developing.


#7

Have done that now, I am really not a fan of Facebook but I think natron is worth it.


#8

I understand. I have strict rules on NatronNation. If anyone or anything gets posted that is not pertaining or helping others learn or improve workflow for Natron gets blocked and deleted.


#9

I followed the instructions, and it does indeed work, but I am curious as to what is exactly happening in a few of the steps.

  • Set to Input Rod - this obviously sets the corner pin to the bounding box of whatever element is plugged into the corner pin node, right? Is this setting the reference frame?

  • Compute Transform Automatically, what is this doing?

In Nuke, many times I will export a corner pin matchmove and simply plug it in line after a roto node, and the roto tracks fine. That doesn’t seem to be the case in Natron, right? Something happens with the scaling/antialiasing when I try it.

Just want to understand what’s happening as clearly as I can. :slightly_smiling:


#10

The tracker works differently than the Nuke one regarding the “Transform” part. For the tracking itself, almost everything is the same. Basically, in Nuke they can only output a CornerPin with exactly 4 points, and they map 1 track to each corner of the CornerPin. For the Transform node they may 1 (translation only), 2, or N points to find the final transformation, however that will never be something other than a [similarity] (https://en.wikipedia.org/wiki/Similarity_(geometry).

In Natron, we offer the possibility to compute a CornerPin with N points, that is an homography.

This is way better because the more tracks you use to compute that CornerPin, the more robust it will be.

An homography is typically used to contain information about a perspective deformation, whereas a similarity is more constrained.

In The Transform tab, that’s what we call “the model”.
Basically the problem we are trying is to fit a model (=SImilarity or Homography) so it is the closest to the correspondences of N points. Each correspondence is the position of a track at the reference frame and its position at the tracked time.

Hence the more correspondences you have (=the more tracks), the more robust the homography is in the region where you tracked features.

The Fitting error parameter in the transform tab, is an indication of how much difference there is in pixels between the reference point on which we applied the computed transformation and the original tracked point. This error is summed across all tracks and squared to give an estimate of the quality of the model found.

For each tracked frame, the correspondences we use to compute the CornerPin are the tracks that are enabled at this frame (i.e: the Enabled parameter is checked at this time) and that have a keyframe on the center (i.e: they successfully tracked).

Basically, when you press Compute, it computes the model (CornerPin/Transform) with all the tracks that meet the aforementioned requirements over all keyframes.

When Compute Transform Automatically is checked, whenever a parameter that has an effect on the output model is changed, this will recompute the Corner/Pin transform over all keyframes again.

The parameters that have an effect on the output model are:

  • The motion type

  • The Transform Type (i.e: Similarity or Homography ?)

  • The Reference frame

  • Jitter Period

  • Smooth: this can be used to smooth the resulting curve to remove some of the noise in the high frequencies of the CornerPin/Transform. Note that in “Add Jitter” mode, you can increase High frequencies to simulate a camera shake that follows the original camera movements.

  • Robust model: this is quite complicated, but in short: When trying to find a model that best fits all correspondences, you may have correspondences that are just wrong (bad tracking for example). These bad correspondences are called outliers, and this parameter when checked tells we should not take into account those outliers to compute the final model. In most cases this should be checked. However sometimes, the user may have for example required to compute an homography (i.e CornerPin), but the given tracked points (correspondences) just cannot make-up an homography. In this case, if the parameter were to be checked, it would fail to compute a model. If you uncheck this, it will take into account all the points and compute a model that averages the motion of all correspondences.

Also when Compute Transform Automatically is checked, the model will be computed automatically when the tracking ends.

We cannot compute the model after each track step (i.e: during tracking) because the model at each frame depends on the model at other frames since we may smooth the curve or add jitter.

So all in all it works differently than Nuke, the whole transformation computation can be more robust and happens as a second pass after the tracking is actually done.

One last thing: to compute the CornerPin in the “Transform” tab of the tracker, the to points are computed using the from points as reference.

Basically what happens is that the tracking outputs a transformation matrix at each frame. Then when computing the model, we apply this matrix to the from points at each frame to obtain the to points.

So if you were to change the reference points (i.e: the from points) with the Set to input RoD for example, then you would need to recompute the model at all frames, because the to points would just not be the same.

In the video that’s what I do:

  • First I disable the CornerPin so that even if my viewer is connected to the Tracker I don’t have any deformation going on, and I set the from points to be the RoD (bounding box) of the Roto shape at the reference frame.

  • When you export the CornerPin, it just links the parameters of the CornerPin to the ones in the tracker, so if you change something in the tracker transform tab the changes will reflect onto the CornerPin.

Hope this helps, if you have any suggestion to improve the workflow, don’t hesitate.

Basically what the Planar tracker will do in the future is automatically do all the steps for you: it will place markers inside the mask for you, track them and output a cornerPin from the bounding box of the roto shape.


#11

What I really wish is that the “corner pins” could be the actual tracking points, I tried to overlay an image over another (screen replacement) but I still can’t find a way to snap the corners of the image to the 4 corners that are the tracking points

Or is there a function for that?

Because this is not really what I want:


#12

I suggest you wait for a tutorial, in Natron you don’t manually set each 4 corners to the CornerPin, you can track anything inside the original rectangle and then set the “from” points of the corner pin to the original rectangle.


#13

Awesome, thanks for the tips, Alex! I’ll try really diving into this way of doing things.

ageha, nice footage you’ve got there! :wink: I’ll have a tutorial on this exact stuff posted on my website in the next couple weeks, so keep an eye open!


#14

Yeah the footage is great, I grabbed all of your stuff from your site, helped me a lot in learning how to track stuff. Thanks for that by the way :smiley:


#15

Sure thing! Seeing you post that is the first time I’ve seen anyone using any of that footage out in the real world, hahahaha… kind of exciting.


#16

Hyperfx,

What is the link to this footage tutorial?


#17

Omar, that particular footage I shot a few years ago for a BlenderCookie tutorial series, and you can find it all on my blog www.openvisualfx.com in the Free Stuff tab. The old BCookie tutorial is posted there, as well.


#18

Thanks and Cool Breeze.


#19