Shooting 360° Footage for Spherical Camera Tracking
We have recently embarked on a 360° adventure, and it has been eye-opening to say the least. Building up to the recent release of spherical tracking in PFTrack, we have been learning all there is to know about the world of 360°, from cameras and filming, to syncing, stitching and of course, tracking.
To show you what can be possible with our new spherical camera tracking technology, we set out on a quest to shoot our very own 360° footage, using a couple of the most cost-effective VR cameras on the market. We’d like to take you behind the scenes to share with you what we learnt along the way; we hope it will help you kick-start your journey into the exciting world of 360° VR possibilities with PFTrack.
We first set out to find the best cost-effective 360° camera for our project. With more and more 360° cameras emerging on the market choosing one can be difficult, but after much research and deliberation we turned our attention to the Ricoh Theta S and the Kodak PIXPRO SP360.
Background on the 360° Cameras
About the Ricoh Theta S
The Ricoh Theta S 360° camera uses a dual UHD sensor and dual fisheye system to capture rectilinear stills and video. What drew us to the camera was its simplicity; not only is the synchronisation of the two cameras done internally (a task which is notoriously painstaking with 360°camera rigs), it also comes with its own stitching technology to stitch the camera footage together. On top of this, wireless controls and live viewing options using the Theta S phone app, make capturing photos and videos as simple as its sleek and unfussy design.
About the Kodak PIXPRO SP360
Like the Ricoh Theta S the Kodak PIXPRO SP360 uses a dual UHD sensor, dual fisheye system to acquire rectilinear stills and video. The Kodak also comes with its own stitching software, as well as a phone app for wireless controls, live viewing and playback. Where the Kodak differs from the Ricoh is that it uses two separate camera bodies back-to-back to shoot 360°, as opposed to being in a self-contained unit. Not only this, the Kodak also offers 4k video resolution; a significant improvement to the Ricoh’s HD offering. You can also get some neat accessories with the Kodak, including a wireless remote control for syncing the two cameras, as well as a purpose made base-mount and monopod.
What we Learnt from Shooting 360°
When you’re filming 360° video, everything around you is being captured. This means there is nowhere to hide, especially if you are the person manually operating the camera. In attempt to tackle this challenge we introduced a monopod (a selfie stick in simpler terms) to our filming. With the Ricoh or Kodak attached to a monopod and raised above the head, the added distance between operator and camera meant they could remain small enough in shot for it not to cause an issue in the footage.
The monopod also had its other benefits, especially when it came to walking over uneven ground with the cameras; with the added stability the effects of rolling shutter could be slightly reduced. As with many camera systems, rolling shutter can be an issue especially when a large amount of movement is involved, and as a result skewing and a “jello” effect can often occur in the footage. Ideal solutions for rolling shutter are hard to come by, especially for VR, where maximising image quality with a small camera system is a primary goal. The Ricoh and Kodak, whilst offering respectable CMOS read reset times, are unfortunately still prone to skew and “jello” when a large amount of movement occurs in shot. We found that the best way to tackle this was with slow, smooth and stabilised movements whilst using the monopod.
Like many camera systems in the similar price bracket as the Ricoh and Kodak, they rely on auto-exposure when shooting video, with no way of manually controlling the settings. The only time this became an issue was when we decided to test transitioning from outdoor to indoor lighting with the cameras.
For both the Kodak and Ricoh, when the rear camera was still set for outside exposure, the front camera was compensating for the lack of indoor lighting. This lead to an exposure discrepancy, which had quite an effect on footage, including the visibility of the stitching line. This problem was eradicated however, when the orientation of the camera was changed; by having the lenses pointing in the opposite direction (left and right as opposed to front and rear), the lighting exposure on the lenses was more evenly dispersed. Therefore the stitch line was almost invisible.
Capturing with the Ricoh: Stitch lines are more visible before the monopod was introduced and camera orientation was changed
Capturing with the Ricoh: After monopod was introduced and camera orientation was changed, stitch lines were almost invisible
Both the Ricoh and Kodak come with their own easy to use stitching software. But stitching footage for many 360° multi-camera rigs does not to come without its issues. In fact, pretty much anyone who has ever filmed 360° video that required stitching together would have experienced parallax: the issue that occurs due to the fact there is a separation between the camera centres. As a result, stitching errors are introduced, and are often worsened with objects close to the camera.
Parallax is particularly exacerbated when the cameras are positioned further away from the central axis plane. As the Ricoh’s cameras were so close together, this didn’t cause too much of an issue with any object more than 3ft away. But as the Kodak’s cameras have a greater distance between them, objects had to be further than 10ft away in order for the effects of parallax to not cause an issue in the footage.
Compared to many other rigs however, the Ricoh and Kodak show far better results thanks to their simplicity. The GoPro system for example, has several cameras with a larger distance from the central axis point, creating a much bigger problem with parallax, which would require a lot of time and patience to fix. On top of this, with the GoPro having multiple cameras means it has multiple stitch lines to contend with, which often creates a greater scope for error in 360° footage.
With the Ricoh, the synchronisation of both cameras is done automatically, and more often than not, successfully too. On the other hand, the synchronisation of the two Kodak cameras must be done manually. This can be a painstaking task for any 360° VR rig without genlock that uses multiple camera heads, so we knew we had to approach this with caution.
The Kodak uses a wireless remote to trigger the recording of the cameras. The biggest problem with the wireless remote however, is that one camera may receive the signal to start recording before the other one, resulting in a synchronisation fault in the footage.
With Kodak’s software there is a way to correct this using audio sync, which compares the audio waveform to match both video layers. However, even when creating a distinct audio spike when filming (we used the sound of a clapper board), the software struggled with matching the two video layers together. If anything, the audio spike seemed to throw the sync off even more!
We soon discovered that unlike professional broadcast systems where the cameras are locked using a timing signal, domestic cameras like Kodak’s have no real way to accurately frame lock with each other, leading to minor variances in frames. Even after using two visual sync markers (a phone light as well as the clapper board) in the hopes of syncing the footage, we soon realised that it would be almost impossible to perfectly match the cameras together. For the purposes of our PFTrack project we did not see this becoming too much of an issue, but in future, an alternative solution would be necessary.
What we loved about the Ricoh is its simplicity, from its ease of use to the automatic synchronisation and stitching with its built in software. What unfortunately lets the Ricoh down is the video resolution. While the photo resolution is impressive, the Ricoh can only shoot video at a maximum of 1920x960, which may not be sufficient depending on the content being produced.
The Kodak on the other hand, has a better video resolution, which could arguably create a better VR viewing experience. But the major downfall is in the complexity of the camera synchronisation and the big parallax issue, which was an overall deal-breaker when selecting footage for our PFTrack project.
Moving the Footage to PFTrack
Moving the footage to PFTrack’s powerful new spherical tracking toolset, it is important to consider that the quality of the camera track depends significantly on the quality of the equirectangular panorama. For the purposes of our PFTrack project, we opted for the footage captured on the Ricoh Theta S; with less stitching and parallax errors than the Kodak, this was much more suitable for PFTrack. The results produced with the Ricoh footage are a real testament to PFTrack’s capabilities, showing how even with footage that is of a lower quality and resolution, good quality tracks can be produced.
Over to You
If you’d like to learn about how to track 360° footage in PFTrack, take a look at our new tutorial detailing how to get started with the powerful new spherical tracking toolset.
We’d also love to know what cameras you’ve been using to shoot 360° footage? Let us know in the comments!