Meet the World’s First 3-D Interactive Holocaust Survivor
Image by Courtesy of the USC Shoah Foundation
Pinchas Gutter sits in a red chair in front of a black background. He hunches forward and rests his hands on his knees, his elbows bent. He wears a slate sweater vest with matching slacks. His skin tone looks just a bit too orange. He sits mostly still, but every so often he nods his head or shifts in his seat. His chest rises and falls with each breath.
Upon closer inspection, the bags under his eyes and the crow’s feet that curl out from his upper lids become clear. His blue eyes meet yours and it feels like he’s watching you. You can ask him anything and when he responds, he raises his hands to illustrate his answer. When he’s done, his hands return to their original position. He looks out in expectation for the next question.
As the conversation continues, it’s easy to forget that Gutter is only a hologram and whatever he is saying has been previously recorded.
•
The Shoah Foundation at the University of Southern California is currently in the process of piloting an expensive and lofty project called New Dimensions in Technology (NDT), which creates holographic images of Holocaust survivors that can converse with viewers and answer questions about their experiences.
Several years ago, when a designer named Heather Maio found that there was no easy way for young people to learn from and interact with survivors, she brought the concept of using holograms to the Shoah Foundation.
NDT began its first test in March with Pinchas Gutter, a survivor who frequently speaks at colleges across the country. Gutter has been the subject of two documentaries: “The Void: In Search Of Memory Lost” and “Political, Polish Jew: The Story of Pinchas Gutter.” It took him years to open up about his experience of being held in several different concentration camps from ages eight to 13.
“I had too many nightmares, too much suffering,” he said in a phone interview with the Forward.
A jovial-looking man in his early 80s with a thick Polish accent, Gutter found the project was more successful than he initially imagined it would be.
“I thought it was a wonderful idea,” he said. “I couldn’t be happier with the way it evolved.”
Over the course of five days, Gutter sat in what is called a light stage, a room containing a black dome set against a green screen, with 50 cameras attached to it.
For five hours each day, he answered a total of 1,700 questions collected from students and the public, all of them trying to anticipate what a typical teenager might ask.
The executive director of the Shoah Foundation, Stephen Smith, along with his team, set up temporary exhibitions of Gutter’s image at museums to collect queries. They covered a broad range of subjects, such as the war, personal faith and the concentration camps. Gutter asked not to see the questions before the interview process, wanting to answer spontaneously. Questions ran the gamut from “What is your favorite color?” to “Did you find your family?” and “Do you hate the Germans?”
“It was quite exhausting and sometimes frustrating,” Gutter said of the procedure — specifically the cameras’ whirling lights. Still, “I managed to overcome my discomfort,” he said.
One of the highlights of the project is its interactive aspect — you can ask Gutter just about any question, and he will respond. To accomplish this, voice-recognition software and natural-language technology sift through the interviews to locate an appropriate answer. If a student tries to trick the system, or there is a technology glitch, Gutter’s hologram will either ask the person to repeat the question, say, “Let’s get back to the Holocaust” or lead the discussion in a different direction.
“The idea, there, is to carry on the dialogue,” said David Traum, a lead researcher on the project. “It’s beyond what you can get from a static recording or documentary.”
Right now these images can only be viewed at the Illinois Holocaust Museum and Education Center in Skokie, and there are no immediate plans to present the holograms at other museums. Yet despite its goal of educating students and recording stories before the survivors pass away, the project has received mixed responses from industry professionals.
“This is the craziest thing I have ever seen,” said Lawrence Langer, a Holocaust historian who worked with Yale University’s Fortunoff Video Archive for Holocaust Testimonies in the 1980s. “Why do we need these holograms?”
“I don’t want to call it a gimmick, but I believe it is,” he said, adding that he didn’t understand the advantage of seeing a survivor in 3-D.
Others who have worked with the Shoah Foundation believe turning the survivors into an illusion makes it more artificial than lifelike, and could undermine their stories.
Joanne Rudof, the archivist for the Fortunoff Video Archive, was somewhat baffled. “What is the goal of this?” she asked, likening the technology to a video game. “Holograms make it less real,” she said.
Dan Leopard, a professor at St. Mary’s College of California who has studied digital media and education — and worked with the Institute for Creative Technologies, the designers and engineers behind the project — explained that the hologram provides a broader potential audience for the testimonies but falls short of providing an authentic encounter.
“The system doesn’t work enough to give you a full transparency,” he said, referring to the hologram’s preconceived answers. “It gives you a sort of feeling of being in the presence of a puppet or robot, and it distances you from the experience.”
Leopard brought up the “uncanny valley,” a theory introduced more than 40 years ago by Masahiro Mori, a Japanese professor at the Tokyo Institute of Technology. Mori’s theory suggested that our reaction to “human” or lifelike prosthetic ligaments and robots moved from comfort to revulsion. Those interacting with such an imitation — say, shaking its hand — feel an affinity that transitions to an “eerie sensation” upon realizing its artificiality.
“It’s not really alive, but we expect it to be alive,” said Leopard. “That’s a problem. It’s sort of off-putting.”
Both Rudof and Langer agreed that the hologram leans toward the unnatural.
“They’re not getting a response from this quote unquote ‘person,’” said Rudof, who likened the project to IBM’s Watson, a computer hailed for its ability to provide insightful information and answer questions, also using natural language technology. “This is Siri, but artificially embodied.”
“What could be more artificial than that?” asked Langer.
Doing “some soul searching” to address this very issue, Smith asked himself, “Would this turn the survivors into a gimmick?” He decided on a way to ensure it would not: “If we put the content first, then the technology would be purely there to support that content,” he said.
For his part, Gutter didn’t think he looked like an artificial entertainment device at all.
“There are always skeptics,” he said.
•
In March of this year, the Illinois Holocaust Museum and Education Center in Skokie became the first facility to show Gutter’s hologram, beginning with several school groups and then opening it up to the public.
Gutter attended the debut screening, and sat in the back of the auditorium; many people didn’t realize he was present. Students looked at Gutter’s image projected on a 2-D screen on stage. A facilitator relayed each question back through a microphone, as the technology’s current form does not allow for Pinchas’s hologram to hear directly from an audience member.
Gutter said he felt a general empathy from the audience, similar to the kind he receives when he’s speaking on campuses.
Amanda Friedeman, a community liaison at the museum, said she had been skeptical when she’d first heard about the project, but after experiencing it firsthand her doubts dissipated.
“It’s not the same as talking to a survivor, but it’s incredibly close,” she said.
Noah Shenker, author of “Reframing Holocaust Testimony,” believes people can tell the survivors’ testimonies are mediated.
“You’re going to be aware of the technology, even if it’s not a hologram,” he said. The hologram technique can’t replace face-to-face contact, he said, because it cannot provide the “spontaneous exchange” and rapport established between interviewer and subject in taped discussions such as those in the Shoah Foundation’s archives.
“It’s going to be more about the unanticipated paths the students get to take rather than where Pinchas gets to go,” Shenker said.
Friedeman said Pinchas’s image blurred the line between reality and illusion, noting that a number of people in the audience thought they were walking in on a Skype session. During the Q&A, some forgot they were not addressing an actual person. In one session, a woman coughed, then apologized to Gutter’s hologram for having interfered. Traum also witnessed this kind of behavior: Some audience members nodded along with Gutter or said “Excuse me,” if they wanted to interrupt him, Traum said.
And while the technology does occasionally falter, Friedeman said that such minor flaws in fact made Gutter seem even more relatable.
“It’s what makes it feel like a conversation,” Friedeman said.
The Shoah Foundation is currently conducting nearly a dozen more interviews. Although survivors are lining up, eager to participate, Smith said he has had to turn some away because of the high cost of carrying out each interview.
According to those involved with the project like Smith, Traum and Gutter, the project has proven more successful than anticipated, leaving audiences astounded.
“You can’t replace a human being,” said Smith. “It’s never going to be the same, but the early findings are very positive.”
The technology may be “imperfect,” Shenker said, but “at least we can train ourselves to be ethically engaged to these witnesses” to the Holocaust and other genocides.
For some, though, the idea of developing Holocaust stories for the realm of digital media and entertainment has few benefits.
“I don’t think I have to tell you how technology is taking over our lives,” said Langer.
Britta Lokting is the Forward’s culture fellow.
A message from our CEO & publisher Rachel Fishman Feddersen
I hope you appreciated this article. Before you go, I’d like to ask you to please support the Forward’s award-winning, nonprofit journalism during this critical time.
We’ve set a goal to raise $260,000 by December 31. That’s an ambitious goal, but one that will give us the resources we need to invest in the high quality news, opinion, analysis and cultural coverage that isn’t available anywhere else.
If you feel inspired to make an impact, now is the time to give something back. Join us as a member at your most generous level.
— Rachel Fishman Feddersen, Publisher and CEO
