Category: immersion

In this post I I will be summarized my process of working on this funky project in mainly Maya and UE5:

The starting point of this project is a model I built from scratch from last year, inspired mainly by the concept art delivered by the company DNEG. As the design has been highly appreciated through review, therefore I decided to add a story behind it and make it more comprehensive:

Due to my knowledge of using UE5 and Maya, I started blocking the cyberpunk world of this robot directly from Maya and working on its following animation in the same scene. I added an additional dream scene in week 8, and baking alembic cash files with blend shapes from it to the unreal project. For the other model with either rigs or animation, I baked them in FBX files and then imported them into my UE5 file.

I used the advanced skeleton to build up the rigs of the main character, the tattoo robot. However, after completing most of the project, I have been given frequent reviews of adding facial rigs on top.

In the UE5, I have imported my baked animation into the sequence and added all materials on top of each model. Furthermore, I have learnt how to build fire Niagara in week 7 and added that to the sequence as well:

For the lighting, I added all of them inside UE5, and keyed a few of them through the animation sequence:

I have also added the colour grading of the scene through the camera, to make the scene reaching what I expected more:

In the end, After I had rendered all my animation I used the software Premiere Pro to work on its editing and sound design. I used the website ‘story blocks’ recommended by technicians in the digital space and selected a matching soundtrack to the narrative:

I added the dream-like effect on top in the matching dream scene by mainly adjusting the level of exposure and contrast of colours:

I received my final feedback from lecturers, peers and industrial people from Siggraph to make sure I can create better idea based upon the progression of this feedback.

Overall, I really enjoy this project, I have learnt a lot of new knowledge from the weekly workshop and also it allows me to work on an experimental figure in its story. In the future, I want to create more robotic characters and add what I have learnt from this project to their model, rigs and animation.

And in the end, here is my recorded formative presentation for the design proposal of this project, I hope you enjoy watching it:

This week I have made some final changes to my work and let the lecturer checked them before my submission, here is my final render:

I have changed the font of the film to make sure they are much easier for audiences to read and feel the vibe of my creation:

And I start working on the submitting video to make sure I edit all the work from this unit together:

In this week, I have concentrated on adjusting my projects with the feedback I have received.

First, by following the lecturer’s advice, I have made the opening title longer for audiences to read:

In the dream scene, inspired by the dream scene in Rango (2019), I have added bigger scale of Goblin models on the background of my project:

Due to my robot’s lack of any facial rig, therefore its ambition and desire for money in its dream become signs of unreal ‘$’ to deliver to the audiences (a rendered clip below to show the semiotic):

In the workspace, I have removed the shots that were pointed out as crossing the 180-degree rule and added more that were inspired by Toy Story 2 (1999):

In the end, The film shows the interaction between the fixer and Woody, mine shows the close-up of the robotic face after and following up its distance to the mannequin:

Before the tattoo process begins, I have added a shot of the robot moving to the dummy from the latter’s perspective to show their interaction more straightforwardly.

My rendered version of this week’s progression:

This week, I have concentrated on the music design of my project and received a little feedback from the Siggraph event because no lecturer has been arranged for the week:

Due to the feedback from last week from the course leader saying my narrative structure is not strong enough through the moving image, I have tried to add text directly to the short film:

A rendered version of the text been added:

However, as there are big jumps between the text and the film, I have then been given new feedback on turning the text into subtitles and keeping them at the bottom of the film:

A rendered version of turning text to the subtitle:

For the soundtrack of this film, there are three separate parts of the story:

1. The dream scene.
2. The work space.
3. The fire and the emergency.

Hence when I was searching for music on the website Story Blocks, I was trying to find the emotionally matching one to achieve the synthesis of each scene:

My soundtrack list for finding music:

In the end, I put sound effects in necessary place in the timeline, such as the sound for fire, sound for moving robot, and sound for tattooing:

I also use the audio transition in Premiere Pro to help me make the connection of different music more fluently in the scene:

With my finished draft version on Tuesday, I have brought it to the animation event Siggraph in the night and received some valuable feedback from industrial animators:

1. For the design and animation of robots in the future, I can refer more to the movie Wall-E (2008). I could either make the robot fully float on the air or walk by using the support of wheels to combat with the basic rigging.
2. For the dream scene, it’s necessary to check the film Rango (2011), and see how it has been structured in the movie by scaling up and down the mannequin and props:

3. For the process of tattooing in my story, I could check the fixing scene in Toy Story 2 (1999), see how the camera movement has been built between the model and the doctor and apply them to my animation:

4. The shot below in my animation has crossed the 180-degree rule so it needs to be changed and rerendered in the future week:

This week I have focused on the progression of the UE5 assignment and its feedback session, I have finished most of its editing and will be start adding sound design in following weeks:

Within the project, I have built up the opening dream scene this week to indicate the robot’s ambition of being rich, I started working on the scene by modeling cash and coins in Maya and building up their texture in Substance painter:

I have used the reference photos to work on the icon of the money and through the re-topology it brings me a clean topology:

I have also tried to work on the icon in Zbrush, however, I found the result is not ideal and the model is too heavy:

As I was designing the own cash to appear in the robot’s dream, here is what I made and projected:

I prepared the animation of the money in Maya and imported it into UE5 through the baked FBX file and Alembic file separately. Due to the cash having a non-linear bend applied I learnt from the video below, that I couldn’t bake the deformer in to the game engine.

After I had imported everything into a new level of my project in UE5, I decided to give the robot a purple background to indicate its luxurious dream:

However, in the feedback session on Friday morning, my lecturer pointed out that the background built up in UE5 is too protruding therefore she wanted me to build up a proper box as a set, and also add some light spots to make audiences concentrate on the character more. I agreed with her idea, and said it should be more creative:

I used the low poly to work on this box by considering time limitations, due to the robot being a tattoo machine, I experimentally added a layer of skin texture and tattoos on top and related back to the fake skin in its room. To represent its dream and ambition are building up by its reality and ‘function’.

I key the colours and size of the spotlights & skylight in the dream animation sequence to ultimately achieve the desired effect. I keep the purple tone through the lighting:

I have also given the coins some added light through having another light channel 1, therefore they can be more shinning in front of the camera:

In the editing software Premiere Pro, I cut the dream scene with the reality scene at the beginning when the robot sleeps. And because it is a dream, I added an extra layer of effect on top to increase its blurriness and exposure by following an online tutorial below, to build up the dream-like feeling:

During editing, I have also cut off some problematic frames naturally happened in UE5 due to the light changing through shots or camera movement:

In addition, in this week I was also working on the 2D effect of tattoo done by the robot in this animation:

I worked on this specific shot in photoshop by combing the 2D layers with the video document:

I learnt the function from scratch through following this video below:

Overall it’s an intensive week for me, however I do enjoy the progression and always do my backup of files.

By following my plan, I have polished my animation blocking under the narrative of my UE5 assignment. Also, I have learnt how to build up the fire Niagara effect in the scene.

Here is a quick screenshot of the fire from my project:

By learning how to build up the fire, I have checked these two main tutorials online:

Due to the first video needs a specific software called ‘Amber Gen’ to create the effect, hence I moved to the second video which used mainly the started resources included in the game engine:

As the video is about 10 minutes, here I am going to display my notes through study it:

Here is the interface of the flame. To explain more, the flame has three parts: fire, smoke, and debris dropping down. Only the fire and its debris have colour information added to them. As I know my flame starts from frame 1015 in the story and ends up in frame 1350– which is 336 frames and 14 seconds in total, I have changed their lifetime into 14 seconds for adding into the scene.

In the process of making the fire, I found it important to test out the level of the spawn rate and the sprite size to make sure the effect was achieving my expectations. Also, it’s important to add them casting the light by adding a light renderer under the fire part.

Due to I need to add the fire into the scene under a specific frame, and making the animation working, here I have followed this tutorial:

By following the tutorial, I understood how to add life time to my Niagara effect after I have dragged them into my animation sequence:

And so I have used a similar method and built up the animation of each flame on its own in my scene, and here is what shown in my sequence:

As I have noticed I can also key the size of the fire, I let each of them get larger by following the time moving. Also I have added layers of flame effect in my scene by following the time passes, therefore we can see the level of the fire is getting bigger and worse on my cyberpunk human:

Frame 1069:

Frame 1121:

Frame 1278:

Frame 1349:

Moving from building up the fire to the animation part of this week, I have added all those details to the robot:

I have created different quick selections to control different part of the robot:

Here is the graph editor of the camera at the same time in Maya:

Due to I have to add the focal length of the camera manually in UE5 here is my progression:

In some shots I tried to make the background more blurry therefore audiences could focus on the foreground more, although the difference might not be that obvious:

I have also debugged a texture problem of the opacity pass of the robot, for now, audiences are be able to see its empty ink bottles around and the ink materials I put inside:

Next week, I will be focusing on the animation of the robot’s dream at the beginning of the story and doing some explanation by adding music and sentences to the story, the latter will be mainly inspired by old silent films (some examples I am thinking below):

A quick render from this week’s progression:

In this week’s session, we have mainly looked at the control rig in UE5 and the blue print of changing or building up a character’s rig. First of all, we have downloaded the FAB to Ue5 and added its plug-in into our project:

By quickly downloading a paragon from the FAB, we were experimenting with adding UE5’s rig to its mesh.

After that, we were trying to learn the process of editing an existing skeletal in UE5 through the plug-in below, and the control rig added to the project.

Within the control rig, we could change the transformation of different joints, such as changing the root joint to three 0 in its transformation.

Within the control rig, we could also adjust the rig by deleting and adding new rigs within its hierarchy.

As the added rig tends to appear on the middle of the mesh, we can use the transform tool to drag it one by one.

After adjusting the rig, it’s also important to bind skin and check the skin weights in UE5.

After that, we downloaded a control rig pack from the FAB, it has some already rigged characters for us to play and study:

We looked at their blueprints and played with the control rig by setting and controlling the bone and controller, through following the tutorial:

It’s also important to learn the way to change the shape, and side of controllers as different characters are having different shapes:

Always remember to ‘GET THE BONE, and SET THE CONTROLLER’.

After the lesson, my lecturer gave me quick feedback on my assignment. She pointed out some shots in my animation that needed to zoom out a bit, and I needed to review the camera movement after I finished the robot animation.

Also I found some online tutorials might be helpful for me to build fire in my scene:

This week, I have focused on the fracture mode of the UE5 in the session, the blocking of my assignment & its rendering in the game engine.

Here is a render I have made of my own project blocking from UE5, it reflects I have made my own pipeline successful by importing the camera and actors from Maya to the game engine:

Here are the blocking animation of the camera I use and the tattoo robot in the story:

As I am ill, I am taking notes of the recording session this week, and thinking to apply them in the future in my project:

Besides the blocking animation shown on top, I have focused on doing different types of materials in the UE5 to match my story:

1. Skin materials for dummies, cyberpunk humans, creatures:

I have used the UDIM to shorten the working process to save time, organise each model, and use the level of roughness/ metallic to identify the metal and non-metal parts. As all of the above related to the skin, I have added a skin texture to their height map and decreased its level through a filter called ‘contrast luminosity’.

The skin texture is bought from the package in Flipped Normal as a support:

As there are a few ‘naked skin’ in my scene, such as the goblin body and fruits, therefore I have added another layer on top of their basic PBR materials to display the inner light that skin can reflect:

Within the ‘subsurface’ shading mode in UE5, the higher the opacity the lower the effect of the skin scattering happening in the scene, the colour of the latter is based upon the base colour linked to the ‘Subsurface Colour’ in the UE5 material. There will be no skin scattering effect when the opacity is breaching 1 or the above.

In addition, as I have focused on producing those tattoo effects on the model, I have projected the tattoo flash I made when I was an apprentice and added effects to make them look ugly and blurry (as a mess created by the robot in my story):

Before and after the filter ‘Blur Slope’ applied on the ink:

Details of the ‘Blue stencil’ I have added on the first layer of models:

2. Water materials for tattoo inks:

As the design of my story robot is a functional robot for tattooing, it has a circle of ink bottles surrounding its main body, it’s unavoidable to build the water material by following the tutorial above in the game engine.

The shot when its needle is dipping into the ink bottle and preparing to tattoo on the cyberpunk being in my story:

I have produced different colours of the water/ink material through playing with the node parameter at the bottom:

I am glad this week I have finished all the texture attempts in this project, next week, I will concentrate on adding details to the blocking animation I have done within this project.

In this week’s session, we have mainly focused on the material in UE5 and the function of subsequence in the animation sequencer:

In my own time, I have kept focusing on building up my cyberpunk world for my tattoo robot, I first finished some texture of my model in Substance Painter, adding projection and paint stroke to make the whole style more stylized:

As I thought the robot may make mess in its set therefore I add layers of spray on its furniture used in its set:

And also, on top of itself:

I added foliage in the set through the foliage mode in UE5:

As the tattoo robot works on the organic dummy or flesh, I have added these pictures into my reference mainly for this week:

As the organic part always combines with the hard surface part in the cyberpunk world, I have worked on the dummy model all in Zbrush, including the retopology, Z-remesh, projection and UV wrap through polygroups:

Process of showing re-topologizing & polishing on Dummy hard surface models in Zbrush:

Process of showing working on Dummy polygroups and UV wrap in Zbrush:

Process of showing working on model projection in Zbrush:

Process of showing organising UVs and Udims of dummies in Maya after finished my model build-up in Zbrush:

In addition, by making my own alpha maps in Zbrush, I can add specific pattern on top of the dummy as a decoration:

The work I have done this week above has been highly supported by the book Beginner’s guide to Zbrush by 3dtotalpublishing:

in page 153, it indicates how to work on retopology in zbrush by using zsphere:

And I have summarized my own pipeline of working on hard surface in Zbrush after refining the pipeline taught in this book, here is my note:

In the first project of the book has also indicated how to make good polygroups on a human model:

I used all these good references written by professionals to push the assignment work, next week I will be focusing on finishing the model in the set and the blocking of my main character and the camera.

In this blog I will be showing the progression of my ue5 assignment within 2 weeks and a few of my reflection.

First of all the rigging inspiration I have been taken to start working on my robot is shown on the video below:

By understanding how the author used the advanced skeleton to do the car root I have tried the similar logic to add joints in to my creation started with the root to its top:

I have given weigh painting into each specific part of this ‘tattoo robot’ as I have mentioned before, to make sure it can be walking and rotating needle in my film later.

As I was mainly working in Maya for blocking my scene and UV wrapped my model, or keying the blocking camera– the biggest thing I have learnt in the progression is to understand how to import the exact same thing in to the game engine, supported by this video below:

I noticed the main difference between these two software is the units and scale, therefore I need to scale up the mesh I want to import into UE5 by following the system indication.

Therefore, I successfully imported my rough scene and camera into UE5 as shown below, and soon will moving on to adding more models to match the look I want my project to have:

To save time, I have used Udim for my model and enabled the virtual texture in the ue5 setting to allow me apply using udim in this engine:

Here is the outcome of the main building after I had built its simple texture in substance Painter and applied it on top of the engine, I used the material as layers to separate different main parts (for example the transparent window and the building itself) of each model:

In my reference to the cyberpunk building, there are a lot of simple geometries included in the building such as squares and rectangles, therefore I have kept my mesh simple in subdivision level 1 in the scene while adding emission to the necessary parts. Also, this is a 12 weeks project, hence I won’t make them so complicated:

In the week 3 session, there are also a few notes before it has been disturbed by the VR promotion:

G is the short key for seeing all icons.

In the animation sequencer in UE5, there are some updated features with imported meshes:

possessable mesh: won’t disappear by closing the sequencer.

spawnable mesh: going to disappear when closing the sequence, easily to control, having a small icon with it in the sequencer.