Virtual Zoom

 

Overview
Workflow
Limitations

Virtual cameras can be used as part of MRset or green screen workflows to allow shots to be captured from positions outside of the range of the physical cameras. By reprojecting the filmed action and virtual content to a new viewpoint, wide shots can be captured for small stages with limited space.

A virtual camera can also be used to traverse a large virtual scene, such as a cityscape. It is possible to start by moving through the streets rendering the virtual scene, and end by transitioning into the view of the real camera on an xR stage, capturing the people on the stage within your virtual scene.

 

Requirements for Setup

Virtual cameras are designed to be interchangeable with real cameras as far as possible for a MRset/green screen setup. They should be interacted with in similar ways to a real camera.

  • Virtual cameras appear as an object in the stage, similar to a real camera

  • They can be linked to a ‘parent camera’ which is the real camera providing the filmed content from the stage

  • When no parent camera is selected, the virtual camera should render virtual content only

  • Content can be mapped to the front and back plates of a virtual camera

  • Virtual cameras can be used in MR sets in the same way as a real camera

  • Users can control the position of a virtual camera in the same ways that are possible for a real camera

  • Virtual camera positions can be defined in Global coordinates, or Relative to the parent camera

  • Virtual camera positions can be keyframed using AnimateCameraControl and AnimateCameraPreset layers

Technical Description

Virtual cameras work by doing a planar reprojection of the filmed content from the real camera to the virtual camera’s view:

 

We render the virtual scene from the virtual camera’s viewpoint, add set extension to the outsides of the reprojected plane, and account for the reprojection in the content we send to the screens. This gives us the virtual camera viewpoint:

 

 

Reprojection

Note that reprojection is done slightly differently to a real camera. For a real camera, the set extension is reprojected onto the set extension mesh to match the filmed content, to avoid any discontinuities at the boundaries. For a virtual camera, the filmed content is reprojected using a planar reprojection to match the fixed set extension, to ensure the set extension remains stable when the real camera moves. This means that there may be more obvious discontinuities between filmed content and set extension when using a virtual camera and moving its parent camera than when moving a real camera. This effect will be more obvious the more close up the virtual camera’s view is, so it is recommended not to move the parent camera quickly when using a virtual camera which is only offset by small amount from its parent.

 

 

To set up a virtual camera

  1. Create a new Camera (VirtualCamera) in the Cameras list in the stage

  2. Set the ‘Parent camera’ property to the desired real camera

  3. Add a ‘Live action position marker’ located at the average position of live content on the stage (e.g. people). This will ensure that the live content appears with the correct perspective relative to the virtual content when the virtual camera moves.

  4. ‘Max zoom in factor’ can be set to restrict how much the filmed content can be zoomed into before purely virtual content is displayed. This prevents excessive pixellation when moving the virtual camera towards the live action.

  5. Set the lens settings under Physical->Lens. These can be defined locally in the virtual camera, set to follow the zoom of the parent camera (with an optional scaling applied), or set to follow all the intrinsics of the parent camera (with an optional zoom scale).

To control a virtual camera:

There are various ways for controlling the position of the virtual camera. You can either:

  • Use the 3D object manipulators to move the VirtualCamera object around in the stage

  • Edit the values in the VirtualCamera editor. Note that values can be defined in a Global coordinate system, or Relative to the parent camera.

  • Use a tracking source: A tracking source can be used to control the pose and lens intrinsics for a virtual camera in the same way as a real camera.

  • Keyframe camera animations using AnimateCameraPreset or AnimateCameraControl

To keyframe virtual camera animations:

Virtual camera animations can be keyframed similarly to standard cameras using the AnimateCameraControl or AnimateCameraPreset layers.

 

AnimateCameraControl:

  1. Add an AnimateCameraControl layer

  2. Set the camera to the VirtualCamera you wish to animate

  3. Set the coordinate system to Global or Relative as desired

  4. Keyframe virtual camera properties. In addition to the standard properties for a normal camera, zoom scale can also be keyframed

AnimateCameraPreset:

  1. Add an AnimateCameraPreset layer

  2. Set the camera to the VirtualCamera you wish to animate

  3. Add new VirtualCameraPositions to the Position property

  4. Set the coordinate system to Global or Relative as desired

  5. In addition to the standard properties for a normal camera, the virtual camera’s zoom can be set. Toggle the ‘Animate zoom scale’ option to choose whether to animate zoom scale or global field of view

  6. Keyframe several VirtualCameraPositions to animate the camera between these positions. Positions in Global and Relative coordinates can be mixed, and the camera will move between the coordinate systems.

  • If the virtual camera is significantly off-axis from the real camera, the live objects (e.g. people) will become skewed. The projection can be thought of as looking at a 2D cardboard cutout from a different angle.

  • All live action must be completely within the frame captured by the real camera. If not, it will be cut off during the reprojection.

  • If the lens is heavily distorted, there maybe be distortion artifacts. A radial mask can be set in the editor to remove these effects, which are most pronounced at the edges of the image.

  • If you attempt to move the virtual camera forwards towards the live action, the picture will become progressively more pixellated. This is because you are zooming in on the captured image at fixed resolution. This can be mitigated by setting the ‘Max zoom in factor’.

  • If the virtual camera moves behind the live action, it will render virtual content only.

  • If the parent camera moves quickly, there may be some noticeable discontinuity between filmed content and the set extension. This may be more noticeable if the virtual camera is near to the parent camera. For fast camera moves it is recommended to use a real camera rather than a virtual camera.

Legacy Workflow

Virtual camera animations could previously be keyframed using additional options in the AnimateCameraControl or AnimateCameraPreset layers. The new workflow supercedes this method, however for backwards compatibility the old workflow can be enabled by setting the option switch enableLegacyVirtualCameraWorkflow.