This Fuse uses Dimension's motion vectors to auto-stabilize an image or smooth the camera motion.
You first need to analyze your footage by simply playing it back inside Fusion while looking at it in one of the viewers (it's recommended to put this node right behind an EXR loader or an optical flow tool for speed reasons). This determines a transformation matrix by least squares fitting to describe each frame's transformation. If you have moving elements that don't follow the camera's motion you can add a garbage matte. The parts where the matte is black will be excluded by the analyzation.
Check out this intro video on Youtube:
(SmoothCam's GUI has changed since but the basic mode of operation is still valid).
The following inputs appear on the tool's tile in the Flow Editor:
- Input: Source image with optical flow data in its forward vector channels. Required.
- Matte: A black & white mask that defines the area of interest (white) and garbage area (black) when analyzing the vector channel. Use it to exclude objects that move against the camera's motion. Optional.
Analyze Mode In analyze mode, SmoothCam will examine its input image whenever it is rendered or viewed in one of the viewports. Just play back the whole sequence first. The console will print status messages whenever a frame has been analyzed. When these messages stop, SmoothCam is done analyzing the sequence.
Vector Detail: The image's vector channel contains motion data for every pixel. To determine the overall camera motion, SmoothCam will scale this channel down to just a few pixels, averaging vectors in the process. This slider determines how many points are ultimately used to calculate the dominant image transformation. Higher numbers will make the algorithm more robust against glitches in the motion vectors.
Reset Analyzation Data: The Fuse saves its data in a hidden bezier spline (it is visible in the spline editor but mustn't be modified manually!). Whenever you want to start over with the analysis stage, press this button instead, which will delete the stabilization data.
Stabilize/Smooth Mode Once you have finished analyzing the clip, switch to "Stabilize / Smooth".
Reference Frame: This slider defines which frame is used as a reference. The chosen frame will not be transformed. Use the "current" button to set this slider to the current frame.
Smoothing (Number of Averaged Frames): The Smoothing slider defines which operation is performed. A value of 0 will freeze the camera as much as possible. A value of one or more, on the other hand, will smooth the camera motion, removing tiny jitter first and creating an increasingly gentle motion if you increase the smoothing value.
Zoom: Whether you're stabilizing or just smoothing the camera motion - the image transformation that is applied causes black borders to appear in the image. You can choose to fill them (see options below) but you can also choose to get rid of them by scaling the image up by a small amount using the zoom slider.Smoothing allows SmoothCam to handle moving shots where the camera isn't always pointing the same way. Complete stabilization is only recommended for shots that should have been stable in the first place (locked of time-lapse shots or hand-held shots where the camera is locked onto an object).
Transformation Type: Defines what to render. By default, "Basic" mode is enabled which allows you to select the kind of transformation you want to smooth or stabilize: Translation, Rotation, Scaling or any combination of those three. Enable or disable the checkboxes according to your needs.
Full stabilization, however, requires one more operation to smooth camera motion: shear. Since Fusion's SDK doesn't support shearing an image it's recommended to use a PerspectivePositioner tool to do the job. Set the transformation mode to "Full Perspective (External)" and connected the PerspectivePositioner's corners to the SmoothCam's outputs (right-click -> Connect To -> SmoothCam1 -> Unstable Top Left...). This will stabilize/smooth the image better than basic mode.
Edges: This option is only available in "Basic" transformation mode. It works like Fusion's Transform tool and defines how the image borders should be handled. Canvas fills with the image's canvas color, wrap will copy pixels from the opposite side and duplicate will fill the borders with the nearest edge pixel.The coordinates for an external PerspectivePositioner will be available even in "Basic" mode. However, you can save processing power by switching to "Full" (external) mode anyways to prevent the Fuse from rendering an unnecessary output image (performance increase).
Further info: Development History and Discussion on PigsFly