Using PanoTools projects within Natron — Natron 2.5.0 documentation

# Using PanoTools projects within Natron

Those are preliminary notes on using PanoTools or Hugin projects from within Natron.

## Creating a PanoTools (pto) project

### Using the Hugin GUI

- download Hugin - on macOS this can be done using homebrew, by typing the command `brew cask install hugin` in a terminal.

- read the Hugin documentation or start with a Hugin tutorial

- make your panorama

### Using command-line tools

Full details on using command-line tools are given in the Panorama scripting in a nutshell documentation.

On macOS, if Hugin was installed using homebrew, the tools are available in various directories, and they can be added to the PATH using:

    PATH="$PATH:/Applications/Hugin/tools_mac:/Applications/Hugin/Hugin.app/Contents/MacOS:/Applications/Hugin/HuginStitchProject.app/Contents/MacOS"

Here is an example of running the panorama tools from a set of JPEG images (with suffix `.jpg`) placed in the current directory to generate e perspective panorama (other options are described in the documentation):

    pto_gen -o project.pto *.JPG
    cpfind --multirow -o project.pto project.pto

    celeste_standalone -i project.pto -o project.pto
    cpclean -v --output project.pto project.pto

    autooptimiser -a -l -s -m -o project.pto project.pto

    nona -m TIFF_m -o project project.pto

    enblend --save-masks -o panorama.tif project*.tif

The result is:

- A `project.pto` file, which is the Hugin project itself

- The `project*.tif` images, which are images warped to the reference projection.

- The `mask-*.tif` images, which are masks generated by `enblend`.

- The `panorama.tif` image, which is the final panorama.

## Using Hugin/PanoTools parameters in Natron

The `pto` file syntax is described is the PTOptimizer and PTStitcher docs.

This file can be viewed and edited in any text editor.

### Project size

The project size, or format, should be set to the panorama size, which is given in pixels on the `p` line (at the start of the file).

### Distortion correction

The distortion correction parameters for the input images are given in the `i` lines (one for each image).

The values for the `a`, `b`, `c`, `d`, `e`, `g`, `t` parameters should be entered in a LensDistortion node, with Model/`model`=PanoTools and Direction/`direction`=Undistort, placed after the Read node for each input image or video. The script name for these values are `pt_a`, `pt_b`, `pt_c`, `pt_d`, `pt_e`, `pt_g`, `pt_t`.

Note that if all images share the same distortion parameters (this is written as `a=0 b=0 c=0 d=0 e=0 g=0 t=0` on the `i` line), the LensDistortion node can be cloned in Natron (right-click on node, Edit, Clone Nodes).

### Projection

The projection (which can be performed by the Nona tool) can be done using Card3D nodes placed after each LensDistortion node.

Each Card3D node must have its Transform Order/`cardXFormOrder` set to STR, and the Rotation Order/`cardRotOrder` should be left to the default value (ZXY). Be careful, these are neither the Axis nor the Cam Transform Order, which are in the two first groups of the Card3Dnode.

Set the Output Format to Project.

The `r` `p` and `y` values from the `i` line are roll, yaw, pitch angles. Their values should be put in the Rotate parameter of the Card3D (script name is `cardRotate`), using the following convention: `cardRotate.x = p`, `cardRotate.y = -y`, `cardRotate.z = -r`.

The `v` value from the `i` line corresponds to the horizontal field of view, and has to be converted to an aperture value. The Lens-In H.Aperture/`lensInHAperture` param should be set to `2*tan(v*pi/360)`. This expression can be directly typed in the value field, with `v` replaced by the actual value from the corresponding `i` line of the `pto` file.

Setup all the Card3D nodes for each input image that way.

### Camera

In Natron 2, the camera used by the Card3D node is set in the Cam group at the top of the parameters list.

In the Card3D node for the first image, unfold this group, and unfold the Cam Projection group.

The `v` value on the `p` line (usually at the top of the `pto` file) gives the horizontal field of view of the output panorama.

Set the Focal Length/`camfocal` to 1., and set the Horiz. Aperture/`camhaperture` to `2*tan(v*pi/360)`, replacing `v` with its actual value.

You can then copy these two parameters to all the Card3D nodes, or - even better - link these parameters, so that the output camera for the panorama can then be modified: Right-click on the parameter from the first Card3D, Copy Link, then right-click on the same parameter of every other Card3D node, Paste Link.

If you intend to modify the camera orientation later, you can also link the same way the Rotate/`camRotate` parameter (those for the camera, not the card).

Note that when the Natron Card3D node is used in Nuke (where it appears as Card3DOFX), the node has an external Cam input, to which a Camera node may be connected. There may be a similar concept in future versions of Natron.

## Building the panorama in Natron

### First sketch: overlap the images

Make sure that the Output Components parameter in all readers is set to RGBA, so that images have a transparent value outside of their domain.

Now, connect the outpput of the first Card3D to the B input of a Merge node, connect the secont to the A input, the third to the A2 input, etc…

The output of the Merge node should show a first panorama, obtained by overlapping all images.

### Drawing the masks

Add a Roto node after each LensDistortion, before each Card3D.

Only leave connected the A and B inputs to the Merge node to the two first Card3D nodes.

Check Premultiply in each Roto node, check that only the A channel is affected by Roto in its parameters, and start editing the roto mask on the second image, for which the LensDistortion output is connected to the A input of the Merge, while viewing the output of the Merge.

Set the compositing operator of each roto shape to “min” instead of “over”, so that the original image alpha gets masked by the roto shape.

Do not forget to add feather, especially in the overlap area.

Then, reconnect the A2 input to the third Card3D, and start editing its roto mask, always in “min” compositing mode.

Do the same with A3, A4, etc… and you should end up with a full panorama.

## Future work

### Camera response

See Camera response curve and Vig_optimize.

The values `Ra` `Rb` `Rc` `Rd` `Re` on the `i` line encode a color response curve (EMoR). See `EMoRParams`in the hugin source code.

### Exposure and color

See Vignetting and Vig_optimize.

`Eev` encodes the exposure, see `ExposureValue` in the hugin source code.

`Er` and `Eb` encode the red and blue multipliers, see `WhiteBalanceRed` and `WhiteBalanceBlue` in the hugin source code.

### Vignetting

See Vignetting and Vig_optimize.

`Va` is always 1, see `VigCorrMode` in the hugin source code.

`Vb` `Vc` `Vd` encode the degree 2, 4 and 6 coefficients for vignette correction, see `RadialVigCorrCoeff` in the hugin source code.

`Vx` and `Vy` encode the vignetting center shift, see `RadialVigCorrCenterShift` in the hugin source code.

### Blending

- importing enblend masks, using them as roto masks

- executing enblend externally (using RunScript maybe?)