Magical Moments for Teaching

The old man sits in the red wing chair and tells: about escape, camp, cruelty. About family, survival, the future. "We are all just small pieces of history, but we must not take our knowledge to the grave, we must pass it on", says Abba Naor, Holocaust survivor. "What happened once can happen again. To avoid this, we must talk about it."

Teamwork for Virtual Witnesses

That's why Abba Naor still exists virtually: for the project Learning with Digital Certificates (LediZ), the Lithuanian-born man was virtualized as a contemporary witness. Thanks to speech recognition, his alter ego will now be able to answer questions about his survival in the Stutthof and Dachau concentration camps - as if he were asked today, during his lifetime. "The target group of LediZ is young people aged 16 and over and students, they are to be trained with this form of mixed reality", reports Anja Ballis, professor at the Ludwig-Maximilian-University (LMU) Munich, didactician and project manager of LediZ. "The project is interdisciplinary and thrives on the synergy of different subject and knowledge areas.

At LediZ didacticians, educators, historians, linguists, social scientists and computer scientists work together. The Leibniz Supercomputing Centre (LRZ) and service providers such as the design agency Bright White support with technology and experience to preserve for posterity and prepare for teaching reports by contemporary witnesses such as Abba Naor or Eva Umlauf, who spent the first years of her life in concentration camps. "We see the virtualization of eyewitness reports as a great opportunity for the didactic mediation of material, because teachers and students are dealing with the art of questioning, which is all too often neglected in learning," says Ballis. "Up to now, there has been a magical moment in every interview session with the virtual witnesses - when listeners are perceptibly captivated by the answers.“

Cinema Technology + Speech recognition = Dialogue Capability

Magical moments for teaching are still elaborate - and exhausting. Above all, the witnesses are expected to do a lot: For five days, Abba Naor and Eva Umlauf sat down in the red leather armchair for three to four 45-minute rounds of interviews each and answered more than 1000 questions about their lives. They were stereoscopically filmed in the studio of the British design agency and film production company Bright White from York by two RED-Epic cameras: "Placed at eye distance, you later get a three-dimensional impression and you believe the speaking figure is real, when you watch the films with special glasses", Daniel Kolb, specialist at the LRZ for virtual reality and visualization, explains the effect of the technology that is otherwise used in cinema films. "Continuity was important for LediZ, every question should be able to be recorded in the same setting".

For unlike film, the contemporary witness reports of LediZ are to be unwound in dialogue or by questions from viewers. This in turn is controlled by the speech recognition system Google Dialogflow, which processes the audio tracks of the interviews. For this purpose, the questions actually asked were additionally modified semantically and linguistically. "In addition to the 1,000 questions, there are 40,000 variations that we have created using a specially developed linguistic system," explains Ballis. "In this way we increase the fit between questions and answers." Voice recognition now searches for and delivers a content-related answer from contemporary witnesses to every question asked by the audience. "We already achieve 80 percent accuracy with it, but we still want to increase this to over 90 percent," reports Kolb. "It depends on the questions, nested ones, long ones are not understood well."

Witnesses in dialogue

The films from the interviews with Eva Umlauf and Abba Naor can be played in two and three dimensions: 3D glasses allow the audience to immerse themselves in the discussion situation, but the data archived and provided by the LRZ could also be used to develop apps for chats or interaction with contemporary witnesses. In order to ensure that the personal reports can be evaluated and processed by the latest technology in as many ways as possible, they were produced in the highest resolution.

"I still have my doubts about the effect this has on children - isn't that something different than when people answer questions in a lively way", asks Eva Umlauf thoughtfully and gives the answer herself. "When we see an old film that wasn't made in colour or 3D, we are still impressed." The interactive interviews are prototypes. Their effect is currently being researched by the LediZ team. When learners and students interview the virtual contemporary witnesses, scientist Ballis is often touched: "Sometimes students are very nervous when they ask questions. They don't want to ask the wrong questions," observes the didactician during the application. "And often a happy mood is created when the contemporary witnesses give a meaningful answer to their own question."

More witnesses, more knowledge

Although the use of the interactive, virtual experience reports still needs to be researched, Anja Ballis and her colleagues from LediZ, the political scientist Professor Markus Gloe and the historian Professor Michele Barricelli, want to create even more contemporary testimonies and are therefore thinking of other events and social groups: In this way, GDR history could be virtualized, but also the experiences of Roma and Sinti during the Nazi era. Thanks to digital technology, nobody has to take his knowledge to the grave anymore - a fascinating imagination.

The field trial to research the digital teaching materials is currently underway: first Abba Naor visited school classes and seminars with the didacticians*, then the virtual interviews are used in class to compare the effect of humans and their virtual alter egos. At the same time, the researchers are developing, together with the LRZ, a package of mobile computer technology for the use of digital testimonies in schools and other institutions. Last but not least, Professor Anja Ballis and her colleagues from LediZ, the political scientist Professor Markus Gloe and the historian Professor Michele Barricelli are looking for even more witnesses from other, exciting times: For example, GDR history could be virtualised, or the experiences and stories of Roma and Sinti during the Nazi era. Thanks to digital technology, nobody has to take their knowledge to the grave anymore - a fascinating idea.

Abba Naor and his alter ego