Post-doctoral position (m/f/d): Volumetric Video Capture > Joboolo FR :
Société : FRENCH TOUCH FACTORY Lieu : Paris
Post-doctoralposition (m/f/d) VolumetricVideoCapture with RGB-D cameras Summary of the research themes:
Edge computing system, Graphic Quality Improvement, VolumetricVideo Data encoding and compression Looking forward to welcome you on our team for project ITHACA ITHACA:
Immersion and Training with Holograms for Art, Craft and Audiovisual Funded by the French State (France2030 program), co-funded by the European Commission (Creative Europe).
Context The production of volumetricvideos opens up new markets in various fields such as immersive storytelling, video games, digital training in industry, crafts, and healthcare.
Despite being very costly, it remains limited to large-scale productions.
The goal of ITHACA is to develop capture software based on RGB-D technology (depth cameras) and a post-production platform for affordable volumetricvideo production.
2.
Research A European consortium of four complementary entities:
- French Touch Factory, XR and VolumetricVideo specialist, - Research laboratory Center for Robotics Mines Paris - Virtual and Augmented Reality, - Fraunhofer IPK (Germany), - and Sergio Ochoa VFX Studio (Spain).
For the past year, the consortium has been developing a series of capture, post-production, fusion, and data export tools for volumetricvideo at 30 frames per second (fps).
The success of the project relies on the use of algorithms leveraging the GPU of various devices (PCs, PC VR, standalone MR headsets, Androids, etc.).
Summary:
The processing steps for volumetric data are as follows:
1.
Capture:
Utilize capture devices to record volumetricvideo data.
Transitioning from a wired camera system connected to one or more master PCs to an edge computing approach based on Jetson Nanos.
(By using for instance the new version of cameras ORBBEC Femto Mega or Bolt) [1] 2.
Registration:
Align and synchronize multiple data sources or viewpoints to ensure spatial coherence.
Transitioning from traditional registration algorithms like Iterative Closest Point (ICP) to adaptive algorithms or those utilizing machine learning.
[2] 3.
Post-production:
Apply editing and enhancement techniques to the registered data.
Examples of Point Cloud Filters can be found in the point cloud library (PCL) [3].
AI can also be used to reach a better level of enhancement.
4.
Fusion:
Integrate the registered data sources to create a cohesive volumetric scene.
Transitioning from simple point cloud fusion algorithms to more advanced methods.
[4] 5.
Display:
Present the volumetric content to users through suitable display technologies.
Taking advantage of new display methods such as Neural Radiance Fields (NeRF) or Gaussian splats for real-time hologram display.
[5] 6.
Export:
Prepare the processed data for use in various applications or platforms.
Implementing compression formats for meshes and textures sequences.
[6] 7.
Live Streaming:
Enable real-time transmission of volumetric content for immediate viewing or interaction.
Improving the encoding of volumetric data for live streaming and storage.
[7] The R&D position involves at least one of these steps based on the doctor's interests and specific skills.
Each point has already been studied in the state of the art, and various approaches tailored to specific use cases have been experimented with in the past, both by teams at French Touch Factory or its competitors worldwide, as well as by scientists.
The candidate with a PhD will conduct their research in close collaboration with the teams at French Touch Factory, applying it to real-world cases in sectors such as narrative XR, live performance, video gaming, digital training in crafts, industry, and healthcare many of which involve partners of ITHACA.
The goal is to test and enhance approaches already identified in the state of the art, ensuring they meet genuine business needs and specific use cases.
3.
Requirements â?¢ PhD in Computer Science, Virtual or Augmented Reality, Spatial Computing or similar relevant fields â?¢ Strong knowledge of programming languages, C/C++, C# on Ubuntu environment and Windows, Python, Pytorch â?¢ Motivation for working in a multidisciplinary research project at the crossroad between optimization, computer science and 3D user interfaces â?¢ A good level in English language (written and spoken) â?¢ Additional knowledge in the following areas would be appreciated:
game engine programming (Unity, Unreal), machine learning, shaders, conducting user studies, UX design 4.
Hiring conditions â?¢ 24-month full-time post-doctoral contract â?¢ The successful candidate will be offered a salary commensurate to CV â?¢ Start date:
between May 2024 and January 1st 2025 â?¢ Location:
Paris&Co startup incubator, 157 boulevard Macdonald, 75019, Paris, France â?¢ Telecommuting up to 2 days a week â?¢ 30 days of vacation from the first year of collaboration â?¢ Refill of 75% of the subscription to the public transport â?¢ Stimulating innovation ecosystem (startups, students, researchers) 5.
How to apply Candidates are required to send their applications including a CV with a publication list, a brief motivation letter explaining interests and qualifications regarding the position, prior research/work experience - all within a single pdf file - and contact information of two professional references to f.bouillefrenchtouchfactory.com.
Application deadline:
15/05/2024.
Feel free to contact us for any questions regarding the position.
References [1] https:
//shop.orbbec3d.com/Femto-Mega [2] Colored Point Clouds Registration by Park 2017:
