All posts by Jonathan Hassall

Spatial Audio for VR with a Ricoh Theta S Camera and Zoom H2n Audio Recorder

Spatial Audio for VR with a Ricoh Theta S Camera and Zoom H2n Audio Recorder

By Jonathan Hassall on

I had a try adding spa­tial audio to a VR video. In the­o­ry this should add real­ism to a 360 VR video by adding audio that can be processed to play back dif­fer­ent­ly depend­ing on the direc­tion of the viewer.

I updat­ed the Zoom H2n to firmware ver­sion 2.00 as described here https://www.zoom.co.jp/H2n-update-v2, and set it to record to uncom­pressed WAV at 48KHz, 16-bit.

I attached the audio recorder to my Ricoh Theta S cam­era. I ori­en­tat­ed the cam­era so that the record but­ton was fac­ing toward me, and the Zoom H2n’s LCD dis­play was fac­ing away from me. I pressed record on the sound recorder and then the video cam­era. I then need­ed a sound and visu­al indi­ca­tor to be able to syn­chro­nize the two togeth­er in post pro­duc­tion, and click­ing my fin­gers worked perfectly.

I installed the http://www.matthiaskronlachner.com/?p=2015. I cre­at­ed a new project in Adobe Pre­miere, and a new sequence with Audio Mas­ter set to Mul­ti­chan­nel, and 4 adap­tive chan­nels. Next I import­ed the audio and video tracks, and cut them to syn­chro­nize to when I clicked my fin­gers together.

Export­ing was slight­ly more involved. I export­ed two files, one for video and one for audio.

For the video export, I used the fol­low­ing settings:

  • For­mat: H264
  • Width: 2048 Height: 1024
  • Frame Rate: 30
  • Field Order: Progressive
  • Aspect; Square Pix­els (1.0)
  • Pro­file: Main
  • Bitrate: CBR 40Mbps
  • Audio track disabled

For the audio export, I used the fol­low­ing settings:

  • For­mat: Wave­form Audio
  • Audio codec: Uncompressed
  • Sam­ple rate: 48000 Hz
  • Chan­nels: 4 channel
  • Sam­ple Size: 16 bit

I then used FFm­peg to com­bine the two files with the fol­low­ing command:

ffmpeg -i ambisonic_video.mp4 -i ambisonic_audio.wav -channel_layout 4.0 -c:v copy -c:a copy final_video.mov

And final­ly inject­ed 360 meta­da­ta using the 360 Video Meta­da­ta app, mak­ing sure to tick both ‘My video is spher­i­cal (360)’ and ‘My video has spa­tial audio (ambiX ACN/SN3D format).

And final­ly uploaded it to YouTube. It took an extra five hours of wait­ing for the spa­tial audio track to be processed by YouTube. Both the web play­er and native Android and iOS apps appear to sup­port spa­tial audio.

If you have your sound recorder ori­en­tat­ed incor­rect­ly, you can cor­rect it using the plu­g­ins. In my case, I used the Z‑axis rota­tion to effec­tive­ly turn the recorder around.

There are a lot of fas­ci­nat­ing opti­miza­tions and expla­na­tions of ambison­ic and spa­tial audio pro­cess­ing avail­able to read at Wikipedia:

The orig­i­nal in-cam­era audio (Ricoh Theta S records in mono) to com­pare can be viewed here:

Sony Noise Cancelling Headphones as Binaural Microphones

Sony Noise Cancelling Headphones as Binaural Microphones

By Jonathan Hassall on

My Sony smart­phone has an unusu­al TRRRS (Tip-Ring-Ring-Ring-Seal) con­nec­tor, allow­ing it to use very rea­son­ably priced noise can­celling head­phones (Sony MDR-NC31EM) that have an extra micro­phone in each earphone.

I found that the Sony app Sound Recorder allows select­ing record­ing direct­ly from these two micro­phones, and are great for bin­au­r­al record­ing, and I gave it a go walk­ing along a few busy streets. You can lis­ten on YouTube and Soundcloud:

JavaScript ES6 Transpiling with Webpack and Babel

JavaScript ES6 Transpiling with Webpack and Babel

By Jonathan Hassall on

Awe­some to final­ly get to use Web­pack and Babel to tran­spile some ES6 code to vanil­la JavaScript that even Inter­net Explor­er can use:

ES6:

1
2
3
4
5
6
7
8
9
10
11
export func­tion arrowTest() {
var mate­ri­als = [
‘Hydro­gen’,
‘Heli­um’,
‘Lithi­um’,
‘Beryl­li­um’
];

// expect­ed out­put: Array [8, 6, 7, 9]
return mate­ri­als.map(mate­r­i­al => mate­r­i­al.length);
}

Tran­spiled:

1
2
3
4
5
6
7
func­tion arrowTest() {
var mate­ri­als = [‘Hydro­gen’, ‘Heli­um’, ‘Lithi­um’, ‘Beryl­li­um’]; // expect­ed out­put: Array [8, 6, 7, 9]

return mate­ri­als.map(func­tion (mate­r­i­al) {
return mate­r­i­al.length;
});
}

Use­ful links: 

A Droplet for KRPano for Publishing 360 Videos

A Droplet for KRPano for Publishing 360 Videos

By Jonathan Hassall on

Here is the first ver­sion of a sim­ple droplet for con­vert­ing and pub­lish­ing 360 panoram­ic videos. It is intend­ed to be used for the processed out­put file from a Ricoh Theta S that has the stan­dard 1920x960 res­o­lu­tion. It is easy to do man­u­al­ly, but many peo­ple asked for an auto­mat­ic droplet.

It con­ve­nient­ly includes 32-bit and 64-bit ver­sions of FFMPEG for per­form­ing video conversion.

Instruc­tions:

  1. Extract to your KRPano folder.
  2. Drag your MP4 video file to the ‘MAKE PANO (VIDEO FAST) droplet’.
  3. Be patient while your video is encod­ed to var­i­ous formats.
  4. Rename the fin­ished ‘video_x’ fold­er to a name of your choice.

You can down­load the droplet here:

Recent improve­ments include:

  • Adding three vari­a­tions of qual­i­ty, which can be accessed by the view­er in Settings.
  • Improv­ing the qual­i­ty of the default play­back setting.
  • Auto­mat­i­cal­ly switch­ing to the low­est qual­i­ty when used on a mobile device.
  • Using a sin­gle .webm video, as the for­mat is very rarely used, and very time con­sum­ing to encode.
  • Out­puts to a named folder.
Removing JavaScript Debugging in Production with Laravel Elixir

Removing JavaScript Debugging in Production with Laravel Elixir

By Jonathan Hassall on

While using Gulp with Lar­avel’s Elixir, I found while it minifies/uglifies JavaScript on a pro­duc­tion build, it does­n’t strip JavaScript debug­ging. It was also far more time con­sum­ing to imple­ment this as a cus­tom Task or Extension.

Strip­ping debug­ging allows you to freely use Console.debug() and sim­i­lar debug­ging calls in devel­op­ment, which oth­er­wise will reduce the per­for­mance of your JavaScript appli­ca­tion, and in some cas­es make them com­plete­ly unus­able to cer­tain browsers.

So I did it myself, and made a Pull request (Github) with the offi­cial Lar­avel Elixir repos­i­to­ry, which was approved. Nice to give back.

Github Pull Request for Laravel Elixir

Open Bionics robotic hand for amputees

Open Bionics robotic hand for amputees

By Jonathan Hassall on

“A pro­to­type 3D-print­ed robot­ic hand that can be made faster and more cheap­ly than cur­rent alter­na­tives is this year’s UK win­ner of the James Dyson Award.” (BBC News link)

This is a fan­tas­tic idea, which has so much val­ue to peo­ple with­out limbs. Bion­ic pros­thet­ics can cost up to £100,000, and £30,000 for a sin­gle hand.

The 3D-print­ed robot­ic hand in the arti­cle costs £2,000, which is the same price as a pros­thet­ic hook, and offers sim­i­lar func­tion­al­i­ty to the top-of-the-range options.

The design­er gets to devel­op his inter­est in cre­at­ing a prod­uct, while help­ing the esti­mat­ed 11 mil­lion peo­ple who are hand amputees worldwide.

Open Bionics

Rock, Paper Scissors

Rock, Paper Scissors

By Jonathan Hassall on

I lis­tened to an inter­est­ing pod­cast today, explor­ing the game Rock, Paper, Scis­sors, pre­dic­tion pat­terns and game theory.

Being clever isn’t an effec­tive way of win­ning, as peo­ple are rarely per­fect deci­sion mak­ers. The Nash equi­lib­ri­um is only rel­e­vant to domains with per­fect deci­sion mak­ers. So it is impor­tant to con­sid­er the choic­es of others.

In the pod­cast, it was men­tioned that from large datasets:

“The biggest chunk of peo­ple will think one step ahead, the next biggest chunk of peo­ple will think total­ly ran­dom­ly, the next biggest chunk of peo­ple will think two steps ahead, and a small num­ber of peo­ple will go beyond two steps.”

It also con­sid­ers crowds and groups influ­enc­ing decisions.

—

Link to the pod­cast: Rock, Paper, Scis­sors (BBC) 

Rock, Paper, Scissors