I wanted to make a post to share my findings with regards to skin shading and rendering realistic characters in Unreal Engine 4.11. We need some help from Epic on the core skin shader implementation..
Excuse the long post but it’s all related to an attempt at shading and rendering realistic characters in UE4.
Epic we need your help 🙂
Following on from http://ir-ltd.net/next-gen-photometric-scanning/ This type of acquired accurate data is ideal for cross checking with PBR shaders in UE4 or offline renderers. IR’s rig captures the multi lighting data during the scanning session, not after, for solid alignment and comparisons. It’s a 1:1 match to the scan data itself. All this data can be mapped back onto the scan set. Including the backscatter images, which tells us information about translucent areas of skin. Or the specular separated data which can reveal the more oily areas of skin.
Output Data from IR’s Rig
Photometric Normals and Multi Lighting Data
Custom UE4 Skin Shader
We’ve been working with UE4’s awesome new shaders and wanted to share some example results and get an open dialogue going. The results are by no means perfect due to lack of control and real-time rendering limitations. The custom shader changes were made by Charles Greivelding https://www.artstation.com/artist/charlesgreivelding
With real-time rendering we miss many features that offline renderers have, like multiple GI bounce, fine and deep subsurface scattering, true ray traced shadows (soft penumbra) and ray traced reflections.
When trying to render in real-time applications there are many obstacles to over come when trying to match against offline renderers like Vray or Arnold. We have to sacrifice a certain amount of realism but real-time renderers like UE4 and Unity are getting better and better.
We are continually inspired by the work of Jorge Jimenez who IR worked with back in 2013 on “Next Generation Character Rendering” for Activision.
Jorge’s work has inspired many studios since, like Epic, who have integrated his techniques into UE4, like his skin shading solution and eye shading tricks. Which were recently seen in Ninja Theory’s awesome Hellblade demo, featuring Senua.
We need some further guidance from Epic on shader integration and possible core updates. As we have found some limitations with the skin shader, which can hinder realism, some of Jorge’s features are missing. We hope to make a case that might inspire Epic further (or not 🙂
Arnold uses The GGX distribution which is new to Arnold 4.2.12 and generally matches real-world materials better than the default Beckmann distribution. With access to 2x GGX specular controls. Epic uses only 1 GGX specular control.
Some missing controls:
Here is an example of how Maya and Arnold render skin
Notice the tight 1st specular lobe we see on the micro details, combined with the larger rougher bluiesh 2nd specular lobe.
(rendered with Maya and Arnold using http://www.anderslanglands.com/ alshader )
1) Not having access to 2x specular lobes (one white, tight and fine and one blue, large and rough) which is very important for skin shading on micro details
2) No built in back scattering (deep SSS) and multi coloured SSS, for thinner areas of skin like in the ears. For these changes to happen, UE4 needs a core update to unlock these features?
3) Normal maps are also locked at 4K resolution. We can get in 8K colour but not 8K normal maps, which are essential for details. No access to 16k micro normal application.
4) We currently cannot get the Eye shader iris depth to work and the reflections cause darkening errors when compared to default lit or subsurface. Either user error (see node network) or bug
5) We have a wet blur layer (forward hack) but have no access to screen space shadows (light leak at corners of eyes) and no way to control depth blur getting larger at further distances from subject
We’ve since been able to hack together a custom version of UE4.11 with 2x specular lobe control (bad hack), back scattering deep SSS and multiple micro tile blending (which we are still working on)
This means running a customized (50gb!!!) 4.112 version of UE4 to accomplish this. Hopefully this post might inspire some of the devs at Epic to look into further upgrading the skin and to hopefully look further into 2 specular lobe controls. With that we can get much better specular shine on skin, as at the moment it either looks to dry or too wet and we have to battle with the controls.
Example Missing 2x Specular Lobe Control
What UE4 has currently
What is needed
(Image from http://www.iryoku.com/downloads/Next-Generation-Character-Rendering-v6.pptx copyright of Activision)
It was hard to know how to setup the eye shader and hair as no documentation or example files existed. If anyone has any ideas on how to implement the Eye shader network in a more efficient usable manner it would be great to know, we can share what we know currently.
Eye Shader comparison and Layers
Notice reflection errors (white eyelash shadow) no caustics
Correct reflections but no iris depth diffuse and no caustics
(our eye shader uses a custom subsurface profile, with correct reflections but lacks caustics and correct deep inverted iris diffuse shading)
Some of our observations whilst using the eye shader
UE4 is truly incredible and a joy to work with. If this post is of any use we would be happy to share the models, project and custom skin shader with Epic or the market place so others can learn from them.
For eyes and hair we cannot get all the features to work without documentation but we got most things running. Any input from other users would be amazing. Being able to activate caustics and solve the incorrect eye occlusion (white eyelash reflection bug) would be very beneficial. We would like to share what we have so far.
Without the extra correct specular controls it’s very hard to match the skin correctly when compared directly to real-world .exr reference data.
Shader Node Setup
Eye Node Shader
UE4 no doubt have more direct access into the engine’s core shader network, so we can only do so much as artists with the built in skin but it is already incredibly powerful.
Thank you Epic for such great tools, hopefully we can share with you our results so far, like we did with the Infinite head scan 🙂