Sharing Faces – Seeing yourself reflected in the image of others
Created by Kyle McDonald, “Sharing Faces” uses a megapixel surveillance camera and custom software to match the face locations of the persons looking at the screen. As the person moves, new images are pulled from the database matching the new location and create a mirror-like image of yourself using the images of others.
The project is the longest running project Kyle has worked on. Starting with the initial idea in 2007, and using the working title “Sabine”, the project was inspired by the story from the history of Rome in which the Roman men stole women from neighbouring Sabine families to make their wives. Decades later, the two tribes went to war with each other, with the battle only coming to an end when the Sabine women ran onto the battlefield pleading with everyone, reminding them that they are all literally brothers. Stories like this, about moments of reflection on the nature of our connectedness, really moved me – Kyle tells CAN. They appear throughout history, from Arjunacontemplating the necessity of battle with Krishna in the Bhagavad Gita all the way to Dostoevsky’s Father Zosima in The Brothers Karamazov refusing to fire his shot in the midst of a gun duel.
Initially the piece was meant to match your full-body pose. In 2007, both these things (face tracking and body tracking) were nearly impossible in realtime. Kyle points us to the technical notes from projects like “SeoNang” from 2005. In 2008, Kyle met Taeyoon Choi at Ars Electronica and he mentioned that full-body matching could become too much like puppeteering, turning it into a comedy. He also reminded Kyle that portraiture has a rich history of creating spaces for reflection. Later, Kyle would see Kikko’s “People on PoP” and Bernhard Hopfengärnter’s “Tanzmaschine” which explore a silhouette-based technique for matching, both unsurprisingly fun and absolutely hilarious. Still, he was cautious knowing that even with the good natured portraits of JR’s “Face2Face” in 2007, the photos were seen sometimes as mocking rather than playful. Work like “Hole-in-Space” or even “The Telectroscope” felt closer to what Kyle was looking for.
While trying to solve the problem of face tracking from scratch, Kyle discovered a few other people working with a similar idea; seeing yourself reflected in the image of others – but using mosaics instead of single images. Jeff Han’s “Media Mirror” and the Barbarian Group’s “McLeod Mirror” being excellent examples for what Kyle was trying to achieve. Another “similarity” piece that had the same spirit without resorting to mosaics was Ole Kristensen’s “Stop Motion” in 2008. Then he discovered “ocean_v1” from Masayuki Akamatsu and Wolf Nkole Helzle. It was almost exactly what Kyle was hoping to build, except it manipulated the images to match the viewer, which felt off-limits to him. However, Kyle was determined to make the piece regardless of similarity to what has already been made. It was also really important to him that the piece occurs in two places simultaneously rather than at a single location.
In 2009 Kyle teamed up with Theo Watson to make “Portrait Machine” which was his first chance to flex his computer vision muscles on a piece that would be installed for a longer duration. From this he learned a lot about what was necessary to make “Sabine” a reality.
Jump forward a few years: Kyle became the maintainer of Jason Saragih’s “FaceTracker” library. He wasn’t happy with lower resolution cameras, and traditional hard drives had seek times too slow for this piece to run at 30 fps, but HD capture cards and solid state drives were now common enough to make this installation feasible. Someone pointed him to the ECCV 2010 paper “Being John Malkovich“. Taeyoon and Kyle became good friends and he invited him to propose a piece for the 4th APAP Festival Making Lab. The entire history of this project came back to him, and the relationship between Japan and Korea became the obvious pair that he’s been thinking about all along. Two countries with a deeply connected, complex and unresolved history.
From there, it was a matter of connecting friends at YCAM to those at APAP. Hopefully besides the fleeting connections created by the installation, Kyle was hoping to kick-start the beginning of a lasting connections between the two organisations. The piece itself was coded in less than a week while on site at APAP. It all came together quickly because he was aware of the problems in advance.
The computers database of images with metadata is synchronised every morning, and rebuilt for binned lookup in memory in the first few seconds of the openFrameworks app starting. An image with metadata comes to around 180KB each, making for a total of around 415K photos captured over 8 months filling an archive of around 75gb. Images with similar expression and pose to an existing image are not stored, meaning whoever gets the first unique expression in a unique location gets to “own” that spot. When no one stands in front of the installation, it shows a single point for every image that has been captured. Because the application is quite processor intensive Quad Core Mac Mini is constantly running hot in order to process the 1080p camera at 30 fps, including saving images to disk, finding the closest match from the 75GB database, and pulling it on screen, decompressing it, displaying it. The installation may look simple, like something that could run on any machine, instead it requires quite a lot of resources.
Overall, there are only two things Kyle is somewhat unhappy with. The face overlay on the image is a little too “debug screen” right now, and he’d like to take another pass at it. The harder thing is that he always felt it was essential for the image to be looking the visitor in the eye. This is only possible with the camera literally behind the screen, with a long field of view. Most of the setups for creating this situation are too complex or expensive to replicate in two places, but maybe in the future this will be possible – Kyle.
Full source code for the project is available here.