HDRI with RenderMan (BMRT)
What is a High Dynamic Range Radiance Imaging?
Computer graphics has long been concerned with recreating reality. To this end renderers have been developed to perform algorithms to emulate the way real light behaves. The most complex interaction happens at surfaces and is a combination of the surface properties (the BRDF - bi-directional reflection distribution function) and the nature of the light illuminating the surface (the luminance the surface recieves). To recreate this mathematically takes much computational time or artistic skill in setting up basic point lights and phong surfaces.
Recently there has been a lot of research into physically based modelling and rendering. This means that instead of using a formula to approximate a property of the illuminace (the energy recieved indirectly at the camera) that this is measured, stored and used instead. Much work has been performed to improve BDRF's, such as Steven Westin's BDRF Measurement data, meanwhile others such as Paul Debevec have been working on the lighting problem. My work is an extension of Paul Debevec's and ports his radiance map idea over to the RenderMan world.
So what is a Radiance map? Firstly what is Radiance :o) This gets a little tricky - if you want more details then check out the Radiometry and Photometry FAQ. To generalise greatly if we have a surface we can consider that it gathers light from a sphere surrounding it. Light (measured as energy) from one area of this sphere and arriving at the surface is known as the luminance at that point. The light that is reflected off from the surface and enters the camera, is known as the illuminance and this is what we collect in a radiance map. (PS this is a gross generalisation on my part!).
So what are those shiny balls all about? The shiny balls in question are known as light probes. This is a reflective sphere used as a cheap substitute for a fisheye lens camera. What it allows us to do is generate a very wide angle view of the scene around a point. Once this information is known we only need to know the mapping to relate back to a point in the scene. Coincidentially we will also know an accurate value for the radiance at that point. This allows us to build a virtual set in the computer with light based on the original plate, and coming from the correct direction. This helps us to more seamlessly integrate live background and computer generated foreground elements.
Where has HDRI been used? Besides Paul Debevec's excellent Fiat Lux animation, I have found two other instances of HDRI being used in commercial production. The first was a few scenes in X-men the movie where senator kelly is dying and becomes a pool of water. This was done using Houdini and there is a brief meantion of this on VFX PRo. Oliver James at CFC has also been using a simliar set up on the Guiness Dream Club advert involving several computer generated squirrels.
Lighthgen plugin available for Lightwave 3D. This creates a set of lights in positions and with colours approximating a lightprobe scene. This can help improve render times considerably - espeically with concentrated points of light.
Practical Image-Based Lighting for Surfaces and Fur by Nickson Fong with Deborah Carlson at Centropolis FX available to download. Thanks for kindly letting me put this up on my website.
Setting up the scene
Generating the images required for processing to become a radiance map involves some effort - particularly if you are using film insterad of a digital camera. The basic process uses an inverse of the reciprocity law to remove the influence of the transfer charachteristics of the imaging system, plus some noise filtering. To do this A series of pictures is taken of the light probe at a set stop interval. These pictures need to capture all the information in the scene which means over and under exposing to catch the highlights and shadows at a mid tone exposure level. I had particular difficulties capturing the full range of highlights so it is a good idea to bracket widely and shoot 2 - 4 stops under the lowest exposure needed. This usually means resulting to very slow film or expensive neutral density filters. Image registration is left as an exercise for the reader! (ie I found it a nightmare) but I would suggest looking at the align package that comes with RACAL gien in the links below. Once you have your set of images these need to be fed into the mkhdr program as given in the instructions. An extra step is needed to make it suitable for use with BMRT and that is file type conversion. To export as floating point from mkhdr just comupte with the -fp flag, or if using Sean O'Malley's HDR2TIFF program (thanks Sean I owe you one!). Note that you will probably need to flip the image horizontally before use and always take the pictures in the direction you want the shot to face.
Inside 3D Studio Max create a very large cube that completely encloses your scene. This has to be very big at the mooment due to a normalising bug - I should be able to sort this one out OK soon. Right click on the cube, and choose to convert to NURBS. In the NURBS modifier panel select sub-object mode and select Surface from the Sub-object menu. Then select each surface and flip the normal so they all face inwards. This is needed for Radiance, but has the added bonus that you can view objects easily inside the box. The box also needs to be reotated 90 degrees about the X axis to account for different handedness between Max and RenderMan.
Next enter the material editor and choose a new slot. Choose a MaxMan type surface and then choose the SB_hdr_surface.slc shader. This shader maps the radiance map correctly onto the inside surface of the cube. Turn on Radiance and hit render! You may want to play around with the Radiance settings to achieve the best results. If you want quicker previews then reduce the samples to about 32, the variance to about 0.25 and the max pixel distance to about 20.
The following images were all rendered using Radiance based lighting. Models coutesy of 3D Cafe.
Image created with my own HDR radiance map taken in my back garden near sunrise. Artefacts in image are caused by noise mainly in the red channel due to less than adequate image registration. Cracking is due to non-planar polygons - once subdivision surfaces are working well I will be able to remove these cracks.
Older raytraced images
These images are produced without Radiance lighting, by using conventional ray tracing. This is a brute force approach to the far more refined Radiance model. It could be refined to do monte carlo sampling, but there seems little point when it is built into the rendering engine. I also do not recommend using this technique as it takes forever to render and suffers badly from noise artefacts. For you information to set up for ray tracing surround the scene with a large sphere then use the MM model modifier to change it into an RiSphere primitve. This is more efficient than using NURBS. Create a light and connect up the SB_hdr_light shader to the map parameter (refer to MaxMan help on how to do this). Then on the Sphere again add the MM BMRT modifier and choose to make the object an area light. You can then control the quality with the sampling parameter. You may also want to reduce the sampling rate and increase the number of samples. A set up is included for the brave or the foolish!
The SB_hdr_light shader should also be useable in PRman when connected up to a vast array of lights on a sphere and targetted at the sphere centre. I have not yet tested this combination though. Max Scripts to follow to allow easy setup of this situation.
Left Image is early test. Right image has improved noise by reducing sampling rate size to 0.01 and increasing number of samples to 8. About 512 area light samples used.
Download shader package
The zip file below should contain everything you need to set up and play with high dynamic range lighting with BMRT. You will of course need BMRT and if you want to use 3D Studio Max then a copy of Animal Logic's MaxMan plugin will be essential too. The BMRT only version is free and well supported - I can recommend it. Please refer to the readme file for information on the files included. This is likely to be updated over time as bugs get ironed out and new features added. You can use these shaders for any purpose - all I ask is that I am acknowledged as the original creator and that you post a link back to this page.Simon Bunker's HDRI package 0.1
Links to HDRI related sites
RASCAL RAdiometric Self CALibration - includes software.
Imagemagick image format coversion utility.
How to Display HDR images.
Daniel Maas' version of mkhdr which doesn't need cygwin to run under Windows and also accepts TGA image files.
Astra Image 2.0 Image manipulation/registration software.
Sean O'Malley's HDR2TIFF program to convert existing HDR files to floating point TIFF files.
A Tutorial on removing the photographer from HDRI maps. Uses Panotools.
Tutorial by Marko Dabrovic on creating computer generated lightprobe images.
The Image based lighting discussion board at The Lightbox.
The RenderMan (R) Interface Procedures and RIB Protocol are:
Copyright 1988, 1989, Pixar. All rights reserved.
RenderMan (R) is a registered trademark of Pixar.
BMRT © Copyright 1990-1999 Larry Gritz. All Rights Reserved.
Rendermania © 2000 Simon Bunker. Please see my Disclaimer.