Virtual Reality and the oral cavity

This wisdom tooth is a meter tall, but it can be rotated in three dimensions with consummate ease. It can even be thrown – and it faithfully returns to its original position, just like a boomerang. If that sounds suspicious, it’s because (of course!) this tooth “lives” in the world of virtual reality. This particular VR application was developed by PD Dr. Anja Liebermann and Dipl.-Ing. Dr. Kurt Erdelt, and it is designed to enable budding dentists to become intimately acquainted with a world that is inhabited by oral cavities, jaws, gums and teeth. “The tool offers an entirely novel and immersive learning experience,” says Liebermann.

However, its developers had to carry out an arduous expedition of their own, before students of dentistry could navigate this learning environment ‘on the ground’, so to speak. The two pioneers began the project around three years ago, when VR had begun to become an established segment of modern digital technology. – “And the thought just occurred to us one day that VR-based tools would have been useful for us when we were in training.” But at the time, nothing of that sort was within their reach – no infrastructure, no technology, no prior experience. So they decided to take up the challenge themselves, starting from the very bottom. The first steps Dipl.-Ing. Dr. Kurt Erdelt

“This was our first project,” says Dr. Erdelt – a VR headset. But it’s rather disappointing. It can be flipped open, and there are no lenses. “No problem, you simply slot your smartphone in here,” says Erdelt. He and Liebermann went on to develop their first animations for this headset. These featured one tooth, which could be rotated in three dimensions, and a short text. So far, so good – but we can do better than that, its creators thought.

The next step was to incorporate videos. The intrepid pioneers taught themselves how to make films in three dimensions. “We recorded short instructional films using two cameras, which were mounted on a suitable rig,” says Erdelt. This set-up was good enough to produce a passable 3D effect, and they were on their way. “Quite apart from the technical tricks we learned as we went along, we received very useful feedback from students. And all of them, without exception, recommended that the project should be featured in our lectures.” That was a clear indication that Liebermann and Erdelt were on the right track – and it testified to the power of immersive learning on their own part. “Universally deployable“ Professor Dieter Kranzlmüller

Dieter Kranzlmüller Professor of Informatics at LMU and Director of the Leibniz Computer Center, is accustomed to receiving enthusiastic feedback. A specialist in VR, he has taught the subject for many years, and he and his team have provided support for VR projects that originated in many different Faculties. “For example, virtual-reality tools are employed at LMU in such disparate fields as Geophysics and the History of Art,” he says. “VR is universally deployable and has the great advantage it enables one to analyze data in entirely unprecedented ways.

“The user’s immersion in the medium is so complete that, when confronted with the learning environment, our testers were reluctant to jump off the platform from which they observe the virtual goings-on around them,” says Dr. Erdelt. Professor Kranzlmüller tells of a senior colleague who had just donned a VR headset and suddenly lay down on the floor, exclaiming that he had never viewed his data from that perspective before!

Such enthusiasm is understandable, although virtual reality is not all that new. “The first head-mounted displays appeared in the 1960s, and since then interest in virtual reality has come and gone in waves,” says Kranzlmüller. Once its technical feasibility could no longer keep up with the burgeoning expectations of its fans, VR repeatedly dropped out of the limelight. According to Kranzlmüller, this cycle has now been broken. “We have now reached a turning point, because the range of potential applications for the technology continues to grow. One is in the area of digital education, and another current application – interactive conferences made possible by VR – helps to mitigate the effects of climate change by reducing the need for long-distance flights. New medium – new challenges

Because high-end VR systems such as Facebook‘s Oculus Rift and HTC‘s Vive are becoming more accessible, in the ergonomic and the economic sense, Drs. Liebermann and Erdelt decided to take a further step toward a completely immersive learning environment. This presented them with a new set of challenges. First of all, it meant that films were no longer adequate. Programs were now the name of the game. “It cost me an immense amount of time to become familiar with the Unity Engine,” says Erdelt. This developers’ platform forms the basis for the current generation of video games and – it’s freely available for non-commercial use.

On top of that, Liebermann and Erdelt had to decide how users were to find their way through these new worlds. “For many people, virtual reality is still an unfamiliar experience, so the practical side must be easy to master,” says Professor Kranzlmüller. For this reason, ‘the digital dentists’ settled on a concept that required only two buttons on the console. One controls movements, the other interactions. Catching the wave PD Dr. Anja Liebermann

The next thorny problem concerned the final cost of the system. In search of ways to cover the purchasing costs, Drs. Liebermann and Erdelt turned to the Students’ Council of the Faculty of Dentistry. The students’ representatives were immediately convinced by the system’s potential, and arranged for the costs to be covered from tuition fees.

However, in the next phase of the project forced the two developers to look for further sources of financing. “When submitting proposals to grant agencies, we often found ourselves in competition with architects and engineers,” says Dr. Erdelt. “We have received many disappointing rejections. But since we are now able to document our progress with the aid of publications, we hope we will now have more luck.”

The current coronavirus crisis has markedly increased interest in the field of virtual reality as a source of new research tools. “At the moment, we have the feeling that journals are placing greater emphasis on the further development and application of digital tools in education,” says Dr. Liebermann. “But quite apart from the pandemic, we believe that digital teaching tools – and virtual reality in particular – offer ways to meet the needs of our students more effectively.” Indeed, the principal purpose of their VR project has always been to offer students a wider range of options.

This also explains why their virtual learning environment engages all the senses. As students move from inspecting the upper jaw to the lower, they can palpate teeth, read tabulated information about each one, or listen to the audio track which provides more detailed information. Dr. Liebermann is convinced that this approach “makes it possible to cater for the specific learning aptitudes of each student”. And students will in future be able to borrow the VR headsets, making use of the system more flexible. With help from the next funding line, the developers hope to procure a larger number of all-in-one headsets, which did not require the support of high-performance computers. The next level

The project has come a long way in three years – all the way from a single tooth rotating on the screen of a smartphone to a fully interactive learning environment. And the follow-up is already in the implementation phase. “As a rule, our students find their initial encounters with patients very stressful, particularly the planning of prosthetic and sometimes interdisciplinary treatments,” Dr. Liebermann explains. “Our new goal is to reduce stress levels for our students in these daunting situations.”

The idea is that students should be able to upload all the data relevant to the treatment of a given patient into a virtual space. With the aid of the patient’s dental record, and man-sized 3D models of the jaws, they can study X-rays and virtually insert implants in the 3D model, and ultimately decide on the appropriate treatment. In the present implementation, this sort of system is already incorporated, but Liebermann and Erdelt plan to add new features and upgrade the optics. That means still more work – but then, that prospect has never deterred them. You might also be interested in:

Digital Education Logbook The biggest reset in the University’s history Impelled by the Covid-19 pandemic, universities have adopted digital teaching methods virtually overnight. Although a novelty for many, such tools are familiar to others at LMU. Here, three lecturers talk about how they have applied them.

Digital Education Logbook What are serious games good for? For true fans, games like Counter-Strike and League of Legends are a serious business – but that doesn’t mean the games themselves are. Really serious games are used in research, and several LMU projects show how valuable they can be.