According to statistics, about 172 million people worldwide will suffer from anorexia at some point in their lives. There are certain treatments to try to combat it, and some of them are using Virtual Reality (VR) techniques. An example of these types of treatments is the one developed by the University of Barcelona. As a final bachelor’s degree project, I chose to help treat this disease by applying a series of improvements to this method.
The original idea of the treatment is the following: the patient is presented with a virtual room through a set of virtual reality goggles. The room mainly contains a mirror and an avatar, which is modeled after the patient’s real body proportions. When putting on the glasses, the avatar moves like the patient, and after a while, her subconscious ends up associating the virtual body she sees in the mirror with her real body. Then, during the following sessions, the avatar’s weight is increased, making the patient believe that she has gained weight, and with a psychologist by her side, this fear of gaining weight is treated.
For the process to work, the avatar needs to move like the patient, but the way this process is done today, it is not very reliable. Until now, 5 tracking points were used (the head, hands and feet), and the rest (such as elbows or knees) were inferred. This inference, however, was not entirely accurate. Therefore, as the first objective of the TFG, a module was developed for better follow-up.
This first module is a “Full Body Tracking” system that makes use of a single camera to do this tracking. Using a machine learning model of Mediapipe as a base, and from the creation of a model to correct the results, we ended up obtaining a system in Python capable of detecting those points that the original system did not detect. It should be mentioned that in order to calibrate this correction model, a synthetic data set had to be generated using Unreal Engine 5, given the impossibility of finding “ground truth” data that would fit the required model.
Once this system was achieved, it was integrated with Unity to the original system platform. You can see the result in this video.
Unfortunately, there are still several difficulties, especially in terms of occlusion, since the system is not able to see the hidden parts behind. However, by combining the information from the Python module with the VR tracking information, which only tracks the hands and head and not the feet, you get a system that better predicts those points for which you have no information.
Then, once this was done, it was decided as a second objective to create an augmented reality (AR) system with the same purpose as the VR system. As the literature on anorexia treatment with AR is scarce, it seemed like a good idea to make a proposal for a solution in this area. The idea was then to get an AR system that would have the same effect as the aforementioned, but without the need to use all its hardware.
Therefore, for this second objective, we wanted to create a mirror showing the real world, but where the patient was reflected with weight gain. After a few unsuccessful attempts to directly fatten the image of the person, it was decided to use the same avatar as in VR, but based on the entire mirror scene. Eventually, a system was developed where the background is shown as if it were a mirror, but where the person being tracked is erased and replaced by the avatar. To track this person, the system developed in the first objective is used. You can see the result in the following video.
A demonstration of the augmented reality program has been made public. You can find it at the following link.