One of the biggest challenges in AdMiRe is the teleportation process, in which the system needs to get the video captured from the user, apply a serie of operations (such as the Foreground Extraction or the Superresolution) and include the closest representation of the user achieved in the process in the scene. With the pipeline just presented, a small error or underperformance in the streaming capture step will drag the error until the final result, even if we create the best possible image processing algorithms.
For this reason, it is reasonable to make an effort and try to manipulate the receiving input so that the final result is improved. In fact, in AdMiRe we are already working in this module and taking decisions about their functionality. Under the name Wizard, it will have different functions: notify the user of how they can improve some scene aspects (such as light, background, etc.) or their pose in the video, or apply some image processing techniques such as basic filters, or suggest to reposition lights in order to match those of the studio.
On one hand, the user will be advised about how to get the best approach to the desired image quality. This includes: the control of the lights of the room, the pose of the user in the frame, among other things. The module will suggest all of the detected possible improvements to the user through the view. Moreover, the user will be in contact with a person from the control room to finalise the different details and perform the final revision.
On the other hand, if the correction of the scene is not possible by physical methods, we can always complement them through image processing techniques (such as contrast controls, or the amount of illumination in the image, among others). Nevertheless, their use may lead to a reduction of the speed of the process which is not desirable. To tackle this problem, we will convert the frames of the video stream into textures, so that we can make use of shaders, code sent to the GPU that performs faster operations than the CPU. Additionally, it also opens the possibility of increasing the complexity of the process and of applying new plausible features.
