KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

Show us your bits!

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#1

Unread post by AndrewHazelden »

KartaVR's latest update adds an exciting new image warping fuse to Reactor.

Reframe360 Ultra is a DCTL fuse for Fusion and Resolve that allows you to reframe your equirectangular/spherical/LatLong panoramic videos to create unique camera angles. This reframing effect is also called "Overcapture".



The code was ported by @Shem Namo, and Andrew Hazelden from the original open-source Reframe360 Resolve OpenFX plugin by Stefan Sietzen.

Fuse Screenshot

Image

DCTL Fuse Support Requirements

  • An OpenCL, CUDA, or Metal based GPU
  • Fusion Studio 16-17.1+ or Resolve 16-17.1+

Download and Install

Download Reframe360 Ultra today using Reactor's "KartaVR/Tools" category.

Image

Example Comps

There are three example comps included with the toolset to get you started.

Image

You can access these examples by selecting the "Reactor > Tools > Show Comps Folder" menu item. This will open a desktop file browsing window using Explorer (Windows)/Finder (macOS)/Nautilus (Linux). You can then hop into the "KartaVR/Reframe360" folder.

Image

Reframe360 Projection Comparison

The "Reframe360 Projection Comparison.comp" example uses a grid pattern to visualize the distortion applied via the rectilinear and tiny planet projection controls in the Reframe360 Ultra fuse.

Image

Reframe360 Tiny Planet Barrens

The "Reframe360 Tiny Planet Barrens.comp" example project shows a DCTL Fuse-based approach for creating "Tiny Planet" stereographic imagery. It starts with an equirectangular image that is reframed into stereographic output projection. The camera is animated from a wide-angle distant view of the scene down to ground level.

Image

Reframe360 Tiny Planet Barrens

The "Reframe360 Tiny Planet Forest.comp" example project shows a DCTL Fuse-based approach for creating "Tiny Planet" stereographic imagery. It starts with an equirectangular image that is reframed into stereographic output projection. This Fusion example comp also applies a vignetting effect to the tiny planet image and crops the output to the final frame size.

Image

Open Source Software License Term

The Reframe 360 Ultra fuse is Apache 2.0 licensed.

Resolve Effects Template

There is a bonus Resolve Edit page "Effects Template" that can be dragged from the "Effect Library > Toolbox > Effects > Viewer > kvrReframe360Ultra" section right onto a timeline-based video clip. You then have the ability to refine your animation keyframes using the Fusion page Spline view.

Edit Page Usage

Step 1. Install Reactor and add the Reframe360 Ultra fuse package to your Resolve-based system.

Step 2. Open a Resolve video editing timeline in the Edit page.

Step 3. Display the Effects Library tab, and switch to the "Toolbox > Effects > KartaVR > Viewer > kvrReframe360Ultra" section. Drag the "kvrReframe360Ultra" entry onto a video clip in the timeline.

Image

Step 4. Click on the video clip in the timeline and switch to the Inspector's Effects tab. Edit the settings to give the output you desire.

Typically you will adjust the "Field of View" control to zoom out in the scene, and then select if you want a rectilinear or tiny planet output style by sliding one of those two projection sliders all the way to the right. You can rotate the view with the Pitch/Yaw/Roll controls.

Step 5. The viewer window in the Edit page should update in real-time to show your live warped result.

Image

Step 6. If you want to refine the spline animation curves for your keyframe animation, click at the top of the Edit Page Inspector window's Effects section on the small "magic wand" icon. This will bring the currently selected video clip with the Effects Templates applied into a Fusion page compositing session.

Image

STMap / UVPass Generation

@Shem Namo came up with a really neat idea to add a 1-click "Output ST Map" checkbox to the fuse that saves out an instant ST map warping template image.

This approach can help you bake to disk the Tiny Planet warping effect, which can allow you to then apply it to a composite using the Reactor provided STMapper fuse, RE:Vision Effect's RE:Map plugin, or any other external VFX tool's UV pass/ST Map warping toolset; like TouchDesigner if you are doing live stitching and streaming of 360° VR Video.

This output format on the DCTL fuse also allows you to jump back to Fusion 9 (if you just can't live inside of Fusion 17 for too long at one time...) and take the exported ST Map warping template image/image sequence created from Reframe360 with you on the way out the door. :mrgreen:

Image

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#2

Unread post by AndrewHazelden »

Tip: If you are using Reactor in the next month or so and are doing a fresh KartaVR "all packages" based install on your system, you'll benefit from reading this WSL post about the Gitlab rate-limit issue that affects heavy Reactor usage in a single session:

Re: "Rate limit exceed" message from gitlab when installing atoms through Reactor on Resolve/Mac

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#3

Unread post by AndrewHazelden »

Hugh Hou from the YouTube channel CreatorUP published a new episode today that explores the workflow to reframe panoramic 360° videos in DaVinci Resolve 17 with Reactor + KartaVR and the Reframe360 Ultra DCTL Fuse. :popcorn:


User avatar
Millolab
Fusionator
Posts: 1157
Joined: 6 years ago
Location: Milan, Italy
Has thanked: 1 time
Been thanked: 13 times
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#4

Unread post by Millolab »

Amazing!

User avatar
intelligent machine
Fusionista
Posts: 818
Joined: 8 years ago
Location: Austin, Texas, USA
Real name: Sam Treadway
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#5

Unread post by intelligent machine »

He's entertaining!

User avatar
michel-ki
Posts: 1
Joined: 3 years ago

KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#6

Unread post by michel-ki »

When trying to edit/reframe my 360 footage (recorded with Gopro Max in 3k, 60fps, file extension: XXX.360 ) via the
KartaVR plugin on my MacBook Pro 2017 (Davinci Resolve Studio 17), the video seems to be cut in several pieces (attachment #1) and I can't find any way to get a "clear view" (attachment #2) like when opening the file in the Gopro player.
The Video Tutorial for Reframe360 on Windows shows a button in 6min18s which would solve that issue if I were on windows..




PS 1: So far only installed the KartaVR/Viewer/kvrReframe360Ultra from the package, nothing more
PS 2: The original file extension of the footage was ".360" and had to be changed to ".mov" in order to be recognized by Davinci

Any idea what could be the issue
Best and thanks 🙋‍♂️🙋‍♂️

Image

Image

User avatar
SecondMan
Site Admin
Posts: 7929
Joined: 10 years ago
Location: Vancouver, Canada
Mood:
Has thanked: 81 times
Been thanked: 73 times
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#7

Unread post by SecondMan »

@michel-ki welcome to WSL!

Pretty please could you also read this before posting screenshots: Server space, screenshots, and you...

I've adjusted your post for you, so don't worry about it, but keep it in mind for next time? :)

Thanks!

User avatar
FredStork
Posts: 4
Joined: 3 years ago
Real name: Fred Stork

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#8

Unread post by FredStork »

@michel-ki
My guess is that the footage is not stitched and you actually only have half of the image. Use the software that came with the camera to stitch and export the footage before importing it in DaVinci Resolve.

Also make sure you have the correct scaling option (Stretch) I think this is the case but do make sure.
Select your footage in the timeline, go to the inspector and scroll down to "Retime and Scaling", open the section and set "Scaling" to "Stretch"

User avatar
jboive
Posts: 2
Joined: 3 years ago
Real name: Joacim Boive

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#9

Unread post by jboive »

It would be SUPER AWESOME if we could use Smart Reframe along with this!

There are products out now that track you, keep you in the frame, while you ride or walk around - the camera follows you.
Using a360 camera you can achieve the same thing - I usually do this manually using my GoPro 360 camera but it's a pain handling the keyframes manually.

Is there maybe already a way to achieve what I'm talking about? :)


It's already awesome! But this addition would make it truly epic! ;)


/J

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#10

Unread post by AndrewHazelden »

jboive wrote: 3 years agoIt would be SUPER AWESOME if we could use Smart Reframe along with this!
This is possible to a certain degree already with the help of Fusion's built-in 2D tracker node, and expression modifiers which can be applied to XYZ rotation animation channels.

If you look at the KartaVR example stitching projects page, specifically at the VR180 Point Tracking Example, the same general approach shown there can be adapted to work at driving the rotation effect on Fusion's built-in PanoMap node, or KartaVR's Reframe360Ultra fuse via the point tracker keyframe information.

The reason I didn't package this together into a pre-set tool combination is that it would tend to attract people who want to throw 3 hour long (1 million frame duration) handheld VR videos at techniques like this, and I don't have the time to do the hand-holding support on why a simple point tracker is not stable on shots that long with elements coming in and out of the frame all the time... :)

Download the example project file:
VR180_Point_Tracking_Example.zip

BTW SynthEyes Pro + Fusion Studio + 360VR footage is a superb combination if people want the upmost quality in their matchmoving/tracking stage while also liking an easy bridge to bring the camera solve into Fusion with only a few clicks.

Edit: Updated the zip download link to use HTTPS since Firefox prefers that now for allowing downloads to succeed.
Last edited by AndrewHazelden 3 years ago, edited 5 times in total.

User avatar
jboive
Posts: 2
Joined: 3 years ago
Real name: Joacim Boive

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#11

Unread post by jboive »

AndrewHazelden wrote: 3 years ago
jboive wrote: 3 years agoIt would be SUPER AWESOME if we could use Smart Reframe along with this!
This is possible to a certain degree already with the help of Fusion's built-in 2D tracker node, and expression modifiers which can be applied to XYZ rotation animation channels.
Thanks!

I'll check that out!

User avatar
Cudeee
Posts: 1
Joined: 2 years ago

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#12

Unread post by Cudeee »

Hello
Thanks for the great tool!
I need to edit keyframes on the Edit tab. In addition, keyframes need to be anti-aliased. Is it possible to create a layer on which to use standard keyframes (eg "position"). Then use a script to synchronize them with keyframes in "KartaVR Reframe360"?

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#13

Unread post by AndrewHazelden »

Hi @Cudeee,
Cudeee wrote: 2 years agoThanks for the great tool!
I appreciate your feedback.
Cudeee wrote: 2 years ago In addition, keyframes need to be anti-aliased.
If you make the timeline resolution the same resolution as the original source footage, and have the Project/Timeline settings for the "Mismatched Resolution" control set to "Stretch frame to all corners" you will have more detail in the reframed output.

User avatar
dilemma
Posts: 1
Joined: 1 year ago

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#14

Unread post by dilemma »

Hi. Thank you for this great tool.

Is it possible to calculate the picture size after setting the point of view? I would like to ensure not to drop lower than 1080p.

Best regards
Emanuel

User avatar
brogotnorizz
Posts: 1
Joined: 9 months ago

Re: KartaVR | Reframe360 Ultra Fuse for Resolve and Fusion

#15

Unread post by brogotnorizz »

I'm unable to use this tool or even drag it to my clip. It says unknown tool found. anyway I can fix it?
Screenshot 2024-05-20 003207.webp
You do not have the required permissions to view the files attached to this post.