Kinect Sesame Street TV

Kinect Sesame Street TV was a blast to work on. Like most, I grew up watching Sesame Street, so it was a real treat to be working on an interactive version of it! Seeing the whole process behind creating branching narrative interactive television was a real learning experience, working alongside game designers, programmers, developers... and Elmo, of course. When I joined the small team, the majority of editing work was already complete on Season 1, so my role as ~Video Artist~ involved compositing, backplate amimation, tweaking edits and creating motion graphics (letter/number animation, shape animations, particles, as well as graphics for promo videos).

I also spent a lot of time working on the interactive Street Story segments: these are existing Sesame segments filmed on Sesame Street with human cast members and muppets, sometimes featuring a celebrity guest. To keep kids engaged during the roughly 7 minute stories, a hidden object is revealed to the child before the segment, and they must find the 12 remaining objects hidden throughout the segment. When they see the object, they point and say picture. When this happens, the object starts to glow and rotate, and Cooper (a completely digital muppet!) pops up, takes a picture of the child pointing using the Kinect camera, and the object counter goes up. At the end, the child is able to view a scrapbook of all these images and see how many objects they were able to find. Pretty fun!

For Season 2, I was responsible for 'hiding' 12 objects in 8 Street Story segments (60 objects in total!) - this required a lot of meticulous camera tracking, rotoscoping (muppet fur!) and compositing. My favourite one was 'Cast Iron Cooks' featuring the hilarious Jason Schwartzman. Below is an example of the composited object (the pumpkin, if you couldn't guess), the glow which appears when found, and a screencap of Cooper popping up and taking a photo of the participant: