Hello everyone, I hope you’re doing well. I have a question about DeepAR. Is it possible to track facial movements and create something similar to Animoji? For instance, can we transform a user’s head into a bear and accurately track the movement of their eyes and mouth (facial expressions) in 3D?
I’ve looked into the filters available in the asset store, but it appears that most of them focus on real human facial features. Could you please guide me if this is possible or provide any tutorials for creating Animoji-like effects similar to iOS? Your insights would be greatly appreciated. Thank you!
Thank you for providing the article, it has been quite helpful!
After reviewing it, it’s clear that DeepAR employs rigging for animoji animations.
I’ve taken a closer look at the file both within DeepAR and in a 3D software like Cinema 4D. I’m wondering, does this mean we create joints in the 3D software and then save them? I’m slightly unsure about how to make these joints functional within DeepAR. I’ve attempted rigging my character’s head, but the facial expressions didn’t align with our facial movements. Is there a tutorial available that demonstrates how to establish this connection? Additionally, I’m curious if eye ball movement can be incorporated.
I apologize for the numerous questions, as I’m relatively new to both AR and 3D modeling. Your guidance is greatly appreciated!
Joints need to always be in the same position as we provide in the template (new_head_rig_joints_new_studio.fbx)
You need to affect a big portion of the meshUse multiple joints: eg. for moving the mouth, I would recommend using the jaw joint to move the whole bottom of the face, then 4 mouth joints
Do not move the joints even if the model doesn’t fit then paint the skin weights as if the joints match your model
Hey Jelena, thank you so much for the tips and template.
Additionally, I came across this YouTube video: https://www.youtube.com/shorts/4ch95gGD9Yk where eyelid motion is demonstrated. I noticed a comment mentioning the requirement of JavaScript to achieve this.
From what I gather, creating motions (such as eye blinking) and effects (triggered actions) might indeed necessitate some coding is that right? As I tried out the template shared, but unfortunately, the eyelids didn’t blink as expected.
You can make them as Animations, Studio does offer a blink trigger, but you can also try creating your own script to control blinking if you don’t find the default trigger sensitive enough
