photo

Wenzel Jakob

Realistic Graphics Lab
BC 345
Station 14
CH-1015 Lausanne
Switzerland

About me

I am an assistant professor leading the Realistic Graphics Lab at EPFL's School of Computer and Communication Sciences.

My research is in the area of computer graphics and involves creating synthetic images that reproduce the visual complexity of the tangible world. This entails developing efficient data structures, computational models and measurement methodologies that can be used to model the way in which light interacts with the materials around us.

A second interest of mine is the high dimensional geometry of problems in light transport, and how it can be exploited to develop more efficient simulation methods.

The overarching goal of my work is to improve the accuracy and fidelity of light transport simulations in computer graphics, and to make the underlying models accurate and fast enough for predictive design and manufacturing applications.

I am currently looking for highly motivated students, who are interested in pursuing research in the above areas. Instructions for applying are available on the EDIC program website (please also get in touch via email if you plan to apply). For undegraduate summer internships, please see the Summer@EPFL program.

Bio: I completed my Ph.D. at the Department of Computer Science at Cornell University, where I was advised by Steve Marschner. Before coming to EPFL, I was a Marie Curie postdoctoral fellow at ETH Zürich's Interactive Geometry Lab where I worked with Olga Sorkine-Hornung.

Curriculum Vitae:  PDF

News

Education

Teaching

Upcoming

Physically Based Rendering: From Theory to Implementation (third edition)
Matt Pharr, Greg Humphreys, Wenzel Jakob
Morgan Kaufmann Publishers
Path Space MCMC Methods in Computer Graphics
Path Space MCMC Methods in Computer Graphics
Wenzel Jakob
Proceedings of the Eleventh International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing

Publications

Geometric Tools for Exploring Manifolds of Light Transport Paths
Geometric Tools for Exploring Manifolds of Light Transport Paths (ACM Research Highlight)
Wenzel Jakob, Steve Marschner
In Communications of the ACM (November 2015)
Technical Perspective: The Path to Light Transport (by Frédo Durand)
Abstract
Photorealistic images created using physical simulations of light have become a ubiquitous element of our everyday lives. The most successful techniques for producing such images replicate the key physical phenomena in a detailed software simulation, including the emission of light by sources, transport through space, and scattering in the atmosphere and at the surfaces of objects. Mathematically, this computation involves the approximation of many high-dimensional integrals, one for each pixel of the image, usually using Monte Carlo methods. Although a great deal of progress has been made on rendering algorithms, so that physically based rendering is now routinely used in many applications, commonly occurring situations can still cause these algorithms to become impractically slow, forcing users to make unrealistic scene modifications to obtain satisfactory results.

Light transport is complex because light can flow along a great variety of different paths through a scene, though only a subset of these makes relevant contributes to the final image. The simulation becomes ineffective when it is difficult to find the important paths. Commonly occurring materials like smooth metal or glass surfaces can easily lead to such situations, where only very few lighting paths participate, leading to spiky integrands and poor convergence. How to efficiently handle such cases in general has been a long-standing problem.
BibTeX
@article{Jakob2015Geometric,
author = {Jakob, Wenzel and Marschner, Steve},
title = {Geometric Tools for Exploring Manifolds of Light Transport Paths},
journal = {Commun. ACM},
volume = {58},
number = {11},
month = oct,
year = {2015},
pages = {103--111},
doi = {10.1145/2823402},
publisher = {ACM},
address = {New York, NY, USA},
}
Instant Field-Aligned Meshes
Instant Field-Aligned Meshes
Wenzel Jakob, Marco Tarini, Daniele Panozzo, Olga Sorkine-Hornung
In ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2015)
Abstract
We present a novel approach to isotropically triangulate or quadrangulate surfaces using a unified local smoothing operator that optimizes both the edge orientations and vertex positions in the output mesh. Our algorithm produces meshes with high isotropy while naturally aligning and snapping edges to sharp features. The method is simple to implement and parallelize, and it can process a variety of input surface representations, such as point clouds, range scans and triangle meshes. Our full pipeline executes instantly (less than a second) on meshes with hundreds of thousands of faces, enabling new types of interactive workflows. Since our algorithm avoids any global optimization, and its key steps scale linearly with input size, we are able to process extremely large meshes and point clouds, with sizes exceeding several hundred million elements. To demonstrate the robustness and effectiveness of our method, we triangulate and quadrangulate hundreds of models of varying complexity and provide our cross-platform reference implementation in the supplemental material.
BibTeX
@article{Jakob2015Instant,
author = {Wenzel Jakob and Marco Tarini and Daniele Panozzo and Olga Sorkine-Hornung},
title = {Instant Field-Aligned Meshes},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia)},
volume = {34},
number = {6},
year = {2015},
month = nov,
doi = {10.1145/2816795.2818078}
}
Anisotropic Gaussian Mutations for Metropolis Light Transport through Hessian-Hamiltonian Dynamics
Anisotropic Gaussian Mutations for Metropolis Light Transport through Hessian-Hamiltonian Dynamics
Tzu-Mao Li, Jaakko Lehtinen, Ravi Ramamoorthi, Wenzel Jakob, Fredo Durand
In ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2015)
Abstract
The simulation of light transport in the presence of multi-bounce glossy effects and motion is challenging because the integrand is high dimensional and areas of high-contribution tend to be narrow and hard to sample. We present a Markov Chain Monte Carlo (MCMC) rendering algorithm that extends Metropolis Light Transport by automatically and explicitly adapting to the local shape of the integrand, thereby increasing the acceptance rate. Our algorithm characterizes the local behavior of throughput in path space using its gradient as well as its Hessian. In particular, the Hessian is able to capture the strong anisotropy of the integrand. We obtain the derivatives using automatic differentiation, which makes our solution general and easy to extend to additional sampling dimensions such as time.

However, the resulting second order Taylor expansion is not a proper distribution and cannot be used directly for importance sampling. Instead, we use ideas from Hamiltonian Monte-Carlo and simulate the Hamiltonian dynamics in a flipped version of the Taylor expansion where gravity pulls particles towards the high-contribution region. Whereas such methods usually require numerical integration, we show that our quadratic landscape leads to a closed-form anisotropic Gaussian distribution for the final particle positions, and it results in a standard Metropolis-Hastings algorithm. Our method excels at rendering glossy-to-glossy reflections on small and highly curved surfaces. Furthermore, unlike previous work that derives sampling anisotropy with pen and paper and only considers specific effects such as specular BSDFs, we characterize the local shape of throughput through automatic differentiation. This makes our approach very general. In particular, our method is the first MCMC rendering algorithm that is able to resolve the anisotropy in the time dimension and render difficult moving caustics.
BibTeX
@article{Li2015Anisotropic,
author = {Tzu-Mao Li and Jaakko Lehtinen and Ravi Ramamoorthi and Wenzel Jakob and Frédo Durand},
title = {Anisotropic Gaussian Mutations for Metropolis Light Transport through Hessian-Hamiltonian Dynamics},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia)},
volume = {34},
number = {6},
year = {2015},
month = nov
}
layerlab: A computational toolbox for layered materials
Wenzel Jakob
In SIGGRAPH 2015 Courses: Physically Based Shading in Theory and Practice
Abstract
A layered BSDF model describes the directional reflectance properties of a material whose internal structure consists of a stack of scattering and/or absorbing layers separated by smooth or rough interfaces. The bottom of the stack could be an opaque interface (such as a metal) or a transparent one. Such structural decompositions into layers and interfaces dramatically enlarge the size of the "language" that is available to describe materials, and for this reason they have been the focus of considerable interest in computer graphics in the last years.

In this document, we present layerlab, a Python-based toolbox for computations involving layered materials that implements a model recently proposed by Jakob et al. The purpose of this document is to serve both as a gentle introduction into the underlying theory and as a hands-on tutorial on solving practical layering problems with this tool.
BibTeX
@inproceedings{Jakob2015Layerlab,
author = {Wenzel Jakob},
title = {layerlab: A computational toolbox for layered materials},
booktitle = {SIGGRAPH 2015 Courses},
series = {SIGGRAPH '15},
year = {2015},
publisher = {ACM},
address = {New York, NY, USA},
}
Path-space Motion Estimation and Decomposition for Robust Animation Filtering
Henning Zimmer, Fabrice Rousselle, Wenzel Jakob, Oliver Wang, David Adler, Wojciech Jarosz, Olga Sorkine-Hornung, Alexander Sorkine-Hornung
In Graphics Forum (Proceedings of Eurographics Symposium on Rendering 2015)
Abstract
Renderings of animation sequences with physics-based Monte Carlo light transport simulations are exceedingly costly to generate frame-by-frame, yet much of this computation is highly redundant due to the strong coherence in space, time and among samples. A promising approach pursued in prior work entails subsampling the sequence in space, time, and number of samples, followed by image-based spatio-temporal upsampling and denoising.

These methods can provide significant performance gains, though major issues remain: first, in a multiple scattering simulation, the final pixel color is the composite of many different light transport phenomena, and this conflicting information causes artifacts in image-based methods. Secondly, motion vectors are needed to establish correspondence between the pixels in different frames, but it is unclear how to obtain them for most kinds of light paths (e.g. an object seen through a curved glass panel).

To reduce these ambiguities, we propose a general decomposition framework, where the final pixel color is separated into components corresponding to disjoint subsets of the space of light paths. Each component is accompanied by motion vectors and other auxiliary features such as reflectance and surface normals. The motion vectors of specular paths are computed using a temporal extension of manifold exploration and the remaining components use a specialized variant of optical flow. Our experiments show that this decomposition leads to significant improvements in three image-based applications: denoising, spatial upsampling, and temporal interpolation.
BibTeX
@article{Zimmer2015Path,
author = {Henning Zimmer and Fabrice Rousselle and Wenzel Jakob and Oliver Wang and David Adler and Wojciech Jarosz and Olga Sorkine-Hornung and Alexander Sorkine-Hornung},
title = {Path-space Motion Estimation and Decomposition for Robust Animation Filtering},
journal = {Computer Graphics Forum (Proceedings of Eurographics Symposium on Rendering 2015)},
volume = {34},
number = {4},
year = {2015},
}
Building Volumetric Appearance Models of Fabric Using Micro CT Imaging
Building Volumetric Appearance Models of Fabric Using Micro CT Imaging (ACM Research Highlight)
Shuang Zhao, Wenzel Jakob, Steve Marschner, Kavita Bala
In Communications of the ACM, November 2014
Technical Perspective: The Intricate Dance of Fabric and Light (by Szymon Rusinkiewicz)
Abstract
Cloth is essential to our everyday lives; consequently, visualizing and rendering cloth has been an important area of research in graphics for decades. One important aspect contributing to the rich appearance of cloth is its complex 3D structure. Volumetric algorithms that model this 3D structure can correctly simulate the interaction of light with cloth to produce highly realistic images of cloth. But creating volumetric models of cloth is difficult: writing specialized procedures for each type of material is onerous, and requires significant programmer effort and intuition. Further, the resulting models look unrealistically "perfect" because they lack visually important features like naturally occurring irregularities.

This paper proposes a new approach to acquiring volume models, based on density data from X-ray computed tomography (CT) scans and appearance data from photographs under uncontrolled illumination. To model a material, a CT scan is made, yielding a scalar density volume. This 3D data has micron resolution details about the structure of cloth but lacks all optical information. So we combine this density data with a reference photograph of the cloth sample to infer its optical properties. We show that this approach can easily produce volume appearance models with extreme detail, and at larger scales the distinctive textures and highlights of a range of very different fabrics such as satin and velvet emerge automatically—all based simply on having accurate mesoscale geometry.
BibTeX
@article{Zhao14Building,
author = {Shuang Zhao and Wenzel Jakob and Steve Marschner and Kavita Bala},
title = {Research Higlights: Building Volumetric Appearance Models of Fabric Using Micro CT Imaging},
journal = {Communications of the ACM},
volume = {57},
number = {11}
year = {2014},
month = {nov},
}
A Comprehensive Framework for Rendering Layered Materials
A Comprehensive Framework for Rendering Layered Materials
Wenzel Jakob, Eugene D'Eon, Otto Jakob, Steve Marschner
In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014)
Abstract
We present a general and practical method for computing BSDFs of layered materials. Its ingredients are transport-theoretical models of isotropic or anisotropic scattering layers and smooth or rough boundaries of conductors and dielectrics. Following expansion into a directional basis that supports arbitrary composition, we are able to efficiently and accurately synthesize BSDFs for a great variety of layered structures.

Reflectance models created by our system correctly account for multiple scattering within and between layers, and in the context of a rendering system they are efficient to evaluate and support texturing and exact importance sampling. Although our approach essentially involves tabulating reflectance functions in a Fourier basis, the generated models are compact to store due to the inherent sparsity of our representation, and are accurate even for narrowly peaked functions. While methods for rendering general layered surfaces have been investigated in the past, ours is the first system that supports arbitrary layer structures while remaining both efficient and accurate.

We validate our model by comparing to measurements of real-world examples of layered materials, and we demonstrate an interactive visual design tool that enables easy exploration of the space of layered materials.
BibTeX
@article{Jakob14Comprehensive,
author = {Wenzel Jakob and Eugene D'Eon and Otto Jakob and Steve Marschner},
title = {A Comprehensive Framework for Rendering Layered Materials},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014)},
year = {2014},
volume = {33},
number = {4}
}
Discrete Stochastic Microfacet Models
Discrete Stochastic Microfacet Models
Wenzel Jakob, Miloš Hašan, Ling-Qi Yan, Jason Lawrence, Ravi Ramamoorthi, Steve Marschner
In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014)
Abstract
This paper investigates rendering glittery surfaces, ones which exhibit shifting random patterns of glints as the surface or viewer moves. It applies both to dramatically glittery surfaces that contain mirror-like flakes and also to rough surfaces that exhibit more subtle small scale glitter, without which most glossy surfaces appear too smooth in close-up. These phenomena can in principle be simulated by high-resolution normal maps, but maps with tiny features create severe aliasing problems under narrow-angle illumination. In this paper we present a stochastic model for the effects of random subpixel structures that generates glitter and spatial noise that behave correctly under different illumination conditions and viewing distances, while also being temporally coherent so that they look right in motion. The model is based on microfacet theory, but it replaces the usual continuous microfacet distribution with a discrete distribution of scattering particles on the surface. A novel stochastic hierarchy allows efficient evaluation in the presence of large numbers of random particles, without ever having to consider the particles individually. This leads to a multiscale procedural BRDF that is readily implemented in standard rendering systems, and which converges back to the smooth case in the limit.
BibTeX
@article{Jakob14Discrete,
author = {Wenzel Jakob and Miloš Hašan and Ling-Qi Yan and Jason Lawrence and Ravi Ramamoorthi and Steve Marschner},
title = {Discrete Stochastic Microfacet Models},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014)},
year = {2014},
volume = {33},
number = {4}
}
Rendering Glints on High-Resolution Normal-Mapped Specular Surfaces
Rendering Glints on High-Resolution Normal-Mapped Specular Surfaces
Ling-Qi Yan, Miloš Hašan, Wenzel Jakob, Jason Lawrence, Steve Marschner, Ravi Ramamoorthi
In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014)
Abstract
Rendering a complex specular surface under sharp point lighting is far from easy. Using Monte Carlo point sampling for this purpose is impractical: the energy is concentrated in tiny highlights that take up a minuscule fraction of the pixel. We instead compute the accurate solution that Monte Carlo would eventually converge to, using a completely different deterministic approach with minimal approximations. Our method considers the true distribution of normals on a surface patch seen through a single pixel, which can be highly complicated. This requires computing the probability density of the given normal coming from anywhere on the patch. We show how to evaluate this efficiently, assuming a Gaussian surface patch and Gaussian intrinsic roughness. We also take advantage of hierarchical pruning of position-normal space to quickly find texels that might contribute to a given normal distribution evaluation. Our results show complicated, temporally varying glints from materials such as bumpy plastics, brushed and scratched metals, metallic paint and ocean waves.
BibTeX
@article{Yan14Rendering,
author = {Ling-Qi Yan and Miloš Hašan and Wenzel Jakob and Jason Lawrence and Steve Marschner and Ravi Ramamoorthi},
title = {Rendering Glints on High-Resolution Normal-Mapped Specular Surfaces},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014)},
year = {2014},
volume = {33},
number = {4}
}
Manifold Exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport
Manifold Exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport
Wenzel Jakob and Steve Marschner
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2012)
Abstract
It is a long-standing problem in unbiased Monte Carlo methods for rendering that certain difficult types of light transport paths, particularly those involving viewing and illumination along paths containing specular or glossy surfaces, cause unusably slow convergence. In this paper we introduce Manifold Exploration, a new way of handling specular paths in rendering. It is based on the idea that sets of paths contributing to the image naturally form manifolds in path space, which can be explored locally by a simple equation-solving iteration. This paper shows how to formulate and solve the required equations using only geometric information that is already generally available in ray tracing systems, and how to use this method in in two different Markov Chain Monte Carlo frameworks to accurately compute illumination from general families of paths. The resulting rendering algorithms handle specular, near-specular, glossy, and diffuse surface interactions as well as isotropic or highly anisotropic volume scattering interactions, all using the same fundamental algorithm. An implementation is demonstrated on a range of challenging scenes and evaluated against previous methods.
BibTeX
@article{Jakob2012Manifold,
author = {Jakob, Wenzel and Marschner, Steve},
title = {Manifold Exploration: A Markov Chain Monte Carlo Technique for Rendering Scenes with Difficult Specular Transport},
journal = {ACM Trans. Graph.},
volume = {31},
number = {4},
month = jul,
year = {2012},
pages = {58:1--58:13},
articleno = {58},
publisher = {ACM},
address = {New York, NY, USA}
}
Structure-aware Synthesis for Predictive Woven Fabric Appearance
Structure-aware Synthesis for Predictive Woven Fabric Appearance
Shuang Zhao, Wenzel Jakob, Steve Marschner, and Kavita Bala
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2012)
Abstract
Woven fabrics have a wide range of appearance determined by their small-scale 3D structure. Accurately modeling this structural detail can produce highly realistic renderings of fabrics and is critical for predictive rendering of fabric appearance. But building these yarn-level volumetric models is challenging. Procedural techniques are manually intensive, and fail to capture the naturally arising irregularities which contribute significantly to the overall appearance of cloth. Techniques that acquire the detailed 3D structure of real fabric samples are constrained only to model the scanned samples and cannot represent different fabric designs.

This paper presents a new approach to creating volumetric models of woven cloth, which starts with user-specified fabric designs and produces models that correctly capture the yarn-level structural details of cloth. We create a small database of volumetric exemplars by scanning fabric samples with simple weave structures. To build an output model, our method synthesizes a new volume by copying data from the exemplars at each yarn crossing to match a weave pattern that specifies the desired output structure. Our results demonstrate that our approach generalizes well to complex designs and can produce highly realistic results at both large and small scales.
BibTeX
@article{Zhao2012Structure,
author = {Zhao, Shuang and Jakob, Wenzel and Marschner, Steve and Bala, Kavita},
title = {Structure-aware Synthesis for Predictive Woven Fabric Appearance},
journal = {ACM Trans. Graph.},
volume = {31},
number = {4},
month = jul,
year = {2012},
pages = {75:1--75:10},
articleno = {75},
publisher = {ACM},
address = {New York, NY, USA}
}
Progressive Expectation-Maximization for Hierarchical Volumetric Photon Mapping
Progressive Expectation-Maximization for Hierarchical Volumetric Photon Mapping
Wenzel Jakob, Christian Regg and Wojciech Jarosz
Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering 2011)
Abstract
State-of-the-art density estimation methods for rendering participating media rely on a dense photon representation of the radiance distribution within a scene. A critical bottleneck of such kernel-based approaches is the excessive number of photons that are required in practice to resolve fine illumination details, while controlling the amount of noise. In this paper, we propose a parametric density estimation technique that represents radiance using a hierarchical Gaussian mixture. We efficiently obtain the coefficients of this mixture using a progressive and accelerated form of the Expectation-Maximization algorithm. After this step, we are able to create noise-free renderings of high-frequency illumination using only a few thousand Gaussian terms, where millions of photons are traditionally required. Temporal coherence is trivially supported within this framework, and the compact footprint is also useful in the context of real-time visualization. We demonstrate a hierarchical ray tracing-based implementation, as well as a fast splatting approach that can interactively render animated volume caustics.
BibTeX
@article{Jakob2011Progressive,
author = {Wenzel Jakob and Christian Regg and Wojciech Jarosz},
title = {Progressive Expectation--Maximization for Hierarchical Volumetric Photon Mapping},
journal = {Computer Graphics Forum (Proceedings of EGSR 2011)},
volume = {30},
number = {4},
year = {2011},
month = jun
}
Building Volumetric Appearance Models of Fabric using Micro CT Imaging
Building Volumetric Appearance Models of Fabric using Micro CT Imaging
Shuang Zhao, Wenzel Jakob, Steve Marschner, and Kavita Bala
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2011)
Abstract
The appearance of complex, thick materials like textiles is determined by their 3D structure, and they are incompletely described by surface reflection models alone. While volume scattering can produce highly realistic images of such materials, creating the required volume density models is difficult. Procedural approaches require significant programmer effort and intuition to design specialpurpose algorithms for each material. Further, the resulting models lack the visual complexity of real materials with their naturallyarising irregularities.

This paper proposes a new approach to acquiring volume models, based on density data from X-ray computed tomography (CT) scans and appearance data from photographs under uncontrolled illumination. To model a material, a CT scan is made, resulting in a scalar density volume. This 3D data is processed to extract orientation information and remove noise. The resulting density and orientation fields are used in an appearance matching procedure to define scattering properties in the volume that, when rendered, produce images with texture statistics that match the photographs. As our results show, this approach can easily produce volume appearance models with extreme detail, and at larger scales the distinctive textures and highlights of a range of very different fabrics like satin and velvet emerge automatically -- all based simply on having accurate mesoscale geometry.
BibTeX
@article{Zhao2011Building,
author = {Zhao, Shuang and Jakob, Wenzel and Marschner, Steve and Bala, Kavita},
title = {Building Volumetric Appearance Models of Fabric Using Micro CT Imaging},
journal = {ACM Trans. Graph.},
volume = {30},
number = {4},
month = jul,
year = {2011},
pages = {44:1--44:10},
publisher = {ACM},
address = {New York, NY, USA}
}
Goal-Based Caustics
Goal-Based Caustics
Marios Papas, Wojciech Jarosz, Wenzel Jakob, Szymon Rusinkiewicz, Wojciech Matusik, Tim Weyrich
Computer Graphics Forum (Proceedings of Eurographics 2011)
Abstract
We propose a novel system for designing and manufacturing surfaces that produce desired caustic images when illuminated by a light source. Our system is based on a nonnegative image decomposition using a set of possibly overlapping anisotropic Gaussian kernels. We utilize this decomposition to construct an array of continuous surface patches, each of which focuses light onto one of the Gaussian kernels, either through refraction or reflection. We show how to derive the shape of each continuous patch and arrange them by performing a discrete assignment of patches to kernels in the desired caustic. Our decomposition provides for high fidelity reconstruction of natural images using a small collection of patches. We demonstrate our approach on a wide variety of caustic images by manufacturing physical surfaces with a small number of patches.
BibTeX
@article{Papas2011Goal,
author = {Marios Papas and Wojciech Jarosz and Wenzel Jakob and Szymon Rusinkiewicz and Wojciech Matusik and Tim Weyrich},
title = {Goal-based Caustics},
journal = {Computer Graphics Forum (Proceedings of Eurographics 2011)},
volume = {30},
number = {2},
year = {2011},
month = jun,
pages = {503--511},
doi = {10.1111/j.1467-8659.2011.01876.x}
}
A radiative transfer framework for rendering materials with anisotropic structure
A radiative transfer framework for rendering materials with anisotropic structure
Wenzel Jakob, Adam Arbree, Jonathan T. Moon, Kavita Bala and Steve Marschner
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2010)
Abstract
The radiative transfer framework that underlies all current rendering of volumes is limited to scattering media whose properties are invariant to rotation. Many systems allow for "anisotropic scattering," in the sense that scattered intensity depends on the scattering angle, but the standard equation assumes that the structure of the medium is isotropic. This limitation impedes physics-based rendering of volume models of cloth, hair, skin, and other important volumetric or translucent materials that do have anisotropic structure. This paper presents an end-to-end formulation of physics-based volume rendering of anisotropic scattering structures, allowing these materials to become full participants in global illumination simulations.

We begin with a generalized radiative transfer equation, derived from scattering by oriented non-spherical particles. Within this framework, we propose a new volume scattering model analogous to the well-known family of microfacet surface reflection models; we derive an anisotropic diffusion approximation, including the weak form required for finite element solution and a way to compute the diffusion matrix from the parameters of the scattering model; and we also derive a new anisotropic dipole BSSRDF for anisotropic translucent materials. We demonstrate results from Monte Carlo, finite element, and dipole simulations. All these contributions are readily implemented in existing rendering systems for volumes and translucent materials, and they all reduce to the standard practice in the isotropic case.
BibTeX
@article{Jakob2010Radiative,
author = {Jakob, Wenzel and Arbree, Adam and Moon, Jonathan T. and Bala, Kavita and Marschner, Steve},
title = {A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure},
journal = {ACM Trans. Graph.},
volume = {29},
number = {4},
month = jul,
year = {2010},
pages = {53:1--53:13},
articleno = {53},
numpages = {13},
publisher = {ACM},
address = {New York, NY, USA}
}
Capturing Hair Assemblies Fiber by Fiber
Capturing Hair Assemblies Fiber by Fiber
Wenzel Jakob, Jonathan T. Moon, Steve Marschner
ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2009)
Abstract
Hair models for computer graphics consist of many curves representing individual hair fibers. In current practice these curves are generated by ad hoc random processes, and in close-up views their arrangement appears plainly different from real hair. To begin improving this situation, this paper presents a new method for measuring the detailed arrangement of fibers in a hair assembly. Many macrophotographs with shallow depth of field are taken of a sample of hair, sweeping the plane of focus through the hair's volume. The shallow depth of field helps isolate the fibers and reduces occlusion. Several sweeps are performed with the hair at different orientations, resulting in multiple observations of most of the clearly visible fibers. The images are filtered to detect the fibers, and the resulting feature data from all images is used jointly in a hair growing process to construct smooth curves along the observed fibers. Finally, additional hairs are generated to fill in the unseen volume inside the hair. The method is demonstrated on both straight and wavy hair, with results suitable for realistic close-up renderings. These models provide the first views we know of into the 3D arrangement of hair fibers in real hair assemblies.
BibTeX
@article{Jakob2009Capturing,
author = {Jakob, Wenzel and Moon, Jonathan T. and Marschner, Steve},
title = {Capturing Hair Assemblies Fiber by Fiber},
journal = {ACM Trans. Graph.},
volume = {28},
number = {5},
month = dec,
year = {2009},
pages = {164:1--164:9},
articleno = {164},
numpages = {9},
publisher = {ACM},
address = {New York, NY, USA}
}

Technical reports

An Improved Sampling Routine for the Distribution of Visible Normals on Gaussian Random Surfaces
An Improved Visible Normal Sampling Routine for the Beckmann Distribution
Wenzel Jakob
ETH Zürich, 2014
Abstract
Recently, Heitz and D'Eon proposed a method for importance sampling the distribution of visible normals in the context of microfacet BSDF models. One of their sampling routines internally relies on a discontinuous mapping, which can cause problems in conjunction with Quasi Monte Carlo sampling and Markov Chain Monte Carlo integration. In this report, we develop an alternative method that does not have this drawback.
BibTeX
@techreport{Jakob2014AnImproved,
author = {Jakob, Wenzel},
title = {An Improved Visible Normal Sampling Routine for the Beckmann Distribution},
month = aug,
year = {2014},
institution = {ETH Zürich}
}
Detailed technical report on “A Comprehensive Framework for Rendering Layered Materials”
Detailed technical report on “A Comprehensive Framework for Rendering Layered Materials”
Wenzel Jakob, Eugene D'Eon, Otto Jakob, Steve Marschner
ETH Zürich, 2014

Abstract
We present a general and practical method for computing BSDFs of layered materials. Its ingredients are transport-theoretical models of isotropic or anisotropic scattering layers and smooth or rough boundaries of conductors and dielectrics. Following expansion into a directional basis that supports arbitrary composition, we are able to efficiently and accurately synthesize BSDFs for a great variety of layered structures.

Reflectance models created by our system correctly account for multiple scattering within and between layers, and in the context of a rendering system they are efficient to evaluate and support texturing and exact importance sampling. Although our approach essentially involves tabulating reflectance functions in a Fourier basis, the generated models are compact to store due to the inherent sparsity of our representation, and are accurate even for narrowly peaked functions. While methods for rendering general layered surfaces have been investigated in the past, ours is the first system that supports arbitrary layer structures while remaining both efficient and accurate.

We validate our model by comparing to measurements of real-world examples of layered materials, and we demonstrate an interactive visual design tool that enables easy exploration of the space of layered materials.
BibTeX
@techreport{Jakob2014LayeredTR,
author = {Wenzel Jakob and Eugene D'Eon and Otto Jakob and Steve Marschner},
title = {A Comprehensive Framework for Rendering Layered Materials: Expanded Technical Report},
month = may,
year = {2014},
institution = {ETH Zürich, Department of Computer Science}
}
Numerically stable sampling of the von Mises Fisher distribution on S^2 (and other tricks)
Numerically stable sampling of the von Mises Fisher distribution on S2 (and other tricks)
Wenzel Jakob
Computer Science Department, Cornell University, 2012
Abstract
The von Mises-Fisher (vMF) distribution is a popular all-purpose distribution for statistical inference involving directional data. Many basic operations involving this distribution are prone to severe numerical issues when implemented in finite precision computer arithmetic. There is a surprising lack on information on how these can be circumvented, and hence the purpose of this document is to serve as a collection of numerically-well behaved recipes for common operations.
BibTeX
@techreport{Jakob2012Numerically,
author = {Jakob, Wenzel},
title = {Numerically stable sampling of the von Mises Fisher distribution on $S^2$ (and other tricks)},
month = jul,
year = {2012},
institution = {Cornell University, Department of Computer Science}
}
Detailed technical report on “Manifold Exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport”
Detailed technical report on “Manifold Exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport”
Wenzel Jakob and Steve Marschner
Computer Science Department, Cornell University, 2012

Note: This report has now been superseded by my Ph. D. thesis.

Abstract
It is a long-standing problem in unbiased Monte Carlo methods for rendering that certain difficult types of light transport paths, particularly those involving viewing and illumination along paths containing specular or glossy surfaces, cause unusably slow convergence. In this paper we introduce Manifold Exploration, a new way of handling specular paths in rendering. It is based on the idea that sets of paths contributing to the image naturally form manifolds in path space, which can be explored locally by a simple equation-solving iteration. This paper shows how to formulate and solve the required equations using only geometric information that is already generally available in ray tracing systems, and how to use this method in in two different Markov Chain Monte Carlo frameworks to accurately compute illumination from general families of paths. The resulting rendering algorithms handle specular, near-specular, glossy, and diffuse surface interactions as well as isotropic or highly anisotropic volume scattering interactions, all using the same fundamental algorithm. An implementation is demonstrated on a range of challenging scenes and evaluated against previous methods.
BibTeX
@techreport{Jakob2012ManifoldTR,
author = {Jakob, Wenzel},
title = {Manifold Exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport: Expanded Technical Report},
month = may,
year = {2012},
institution = {Cornell University, Department of Computer Science}
}
Detailed technical report on “A radiative transfer framework for rendering materials with anisotropic structure”
Detailed technical report on “A radiative transfer framework for rendering materials with anisotropic structure”
Wenzel Jakob, Adam Arbree, Jonathan T. Moon, Kavita Bala and Steve Marschner
Computer Science Department, Cornell University, 2010
Abstract
The radiative transfer framework that underlies all current rendering of volumes is limited to scattering media whose properties are invariant to rotation. Many systems allow for "anisotropic scattering," in the sense that scattered intensity depends on the scattering angle, but the standard equation assumes that the structure of the medium is isotropic. This limitation impedes physics-based rendering of volume models of cloth, hair, skin, and other important volumetric or translucent materials that do have anisotropic structure. This paper presents an end-to-end formulation of physics-based volume rendering of anisotropic scattering structures, allowing these materials to become full participants in global illumination simulations.

We begin with a generalized radiative transfer equation, derived from scattering by oriented non-spherical particles. Within this framework, we propose a new volume scattering model analogous to the well-known family of microfacet surface reflection models; we derive an anisotropic diffusion approximation, including the weak form required for finite element solution and a way to compute the diffusion matrix from the parameters of the scattering model; and we also derive a new anisotropic dipole BSSRDF for anisotropic translucent materials. We demonstrate results from Monte Carlo, finite element, and dipole simulations. All these contributions are readily implemented in existing rendering systems for volumes and translucent materials, and they all reduce to the standard practice in the isotropic case.
BibTeX
@techreport{Jakob2012Numerically,
author = {Jakob, Wenzel and Arbree, Adam and Moon, Jonathan T. and Bala, Kavita and Marschner, Steve},
title = {A radiative transfer framework for rendering materials with anisotropic structure: Expanded Technical Report},
month = apr,
year = {2010},
url= {http://hdl.handle.net/1813/14982},
institution = {Cornell University, Department of Computer Science}
}

Theses

Light Transport on Path-Space Manifolds
Light Transport on Path-Space Manifolds
Ph. D. Thesis
Cornell University, Aug 2013
Abstract
The pervasive use of computer-generated graphics in our society has led to strict demands on their visual realism. Generally, users of rendering software want their images to look, in various ways, “real”, which has been a key driving force towards methods that are based on the physics of light transport.

Until recently, industrial practice has relied on a different set of methods that had comparatively little rigorous grounding in physics—but within the last decade, advances in rendering methods and computing power have come together to create a sudden and dramatic shift, in which physics-based methods that were formerly thought impractical have become the standard tool. As a consequence, considerable attention is now devoted towards making these methods as robust as possible.

In this context, robustness refers to an algorithm's ability to process arbitrary input without large increases of the rendering time or degradation of the output image. One particularly challenging aspect of robustness entails simulating the precise interaction of light with all the materials that comprise the input scene. This dissertation focuses on one specific group of materials that has fundamentally been the most important source of difficulties in this process.

Specular materials, such as glass windows, mirrors or smooth coatings (e.g. on finished wood), account for a significant percentage of the objects that surround us every day. It is perhaps surprising, then, that it is not well-understood how they can be accommodated within the theoretical framework that underlies some of the most sophisticated rendering methods available today.

Many of these methods operate using a theoretical framework known as path space integration. But this framework makes no provisions for specular materials: to date, it is not clear how to write down a path space integral involving something as simple as a piece of glass.

Although implementations can in practice still render these materials by side-stepping limitations of the theory, they often suffer from unusably slow convergence; improvements to this situation have been hampered by the lack of a thorough theoretical understanding.

We address these problems by developing a new theory of path-space light transport which, for the first time, cleanly incorporates specular scattering into the standard framework. Most of the results obtained in the analysis of the ideally smooth case can also be generalized to rendering of glossy materials and volumetric scattering so that this dissertation also provides a powerful new set of tools for dealing with them.

The basis of our approach is that each specular material interaction locally collapses the dimension of the space of light paths so that all relevant paths lie on a submanifold of path space. We analyze the high-dimensional differential geometry of this submanifold and use the resulting information to construct an algorithm that is able to “walk” around on it using a simple and efficient equation-solving iteration.

This manifold walking algorithm then constitutes the key operation of a new type of Markov Chain Monte Carlo (MCMC) rendering method that computes lighting through very general families of paths that can involve arbitrary combinations of specular, near-specular, glossy, and diffuse surface interactions as well as isotropic or highly anisotropic volume scattering. We demonstrate our implementation on a range of challenging scenes and evaluate it against previous methods.
BibTeX
@PhdThesis{Jakob2013,
title = "Light Transport on Path-Space Manifolds",
author = "Wenzel Jakob",
school = "Cornell University",
month = aug,
year = "2013"
}
An Analytic Framework for Anisotropic Diffusion in Computer Graphics

An Analytic Framework for Anisotropic Diffusion in Computer Graphics

M. Eng. Project
Cornell University, May 2008

Please refer to the technical report on “A radiative transfer framework for rendering materials with anisotropic structure” for the final version of this writeup.

Accelerating the bidirectional path tracing algorithm using a dedicated intersection processor
Accelerating the bidirectional path tracing algorithm using a dedicated intersection processor
Undergraduate Thesis
Karlsruhe Institute of Technology (KIT), Jul 2007
Abstract
Even with the processing power of today’s computers, the calculation of global illumination solutions for general scenes remains a lengthy undertaking. This project investigates the possibility of accelerating such calculations using a dedicated hardware co-processor. The algorithm of choice, bidirectional path tracing, presents an additional challenge, since its three-dimensional random walks make the use of coherent ray tracing techniques impracticable. A parallel and deeply pipelined ray-triangle intersection processor including a kd-tree traversal unit was developed during the course of the project, and numerous benchmarks were performed.
BibTeX
@mastersthesis{Jakob2007,
document_type = {Undergraduate Thesis},
title = "Accelerating the bidirectional path tracing algorithm using a dedicated intersection processor",
author = "Wenzel Jakob",
school = "Karlsruhe Institute of Technology (KIT)",
month = jul,
year = "2007"
}

Personal and class projects

Mitsuba renderer
Mitsuba Renderer
Mitsuba is a research-oriented rendering system written in cross-platform C++. Its code is extremely modular, consisting of a small set of core libraries and over 100 different plugins that implement functionality ranging from materials and light sources to complete rendering algorithms. A strong emphasis is placed on scalability and parallelism; Mitsuba has been used to render complex scenes using over a thousand networked cores at the same time.
BibTeX
@misc{Mitsuba,
author = {Wenzel Jakob},
year = {2010},
note = {http://www.mitsuba-renderer.org},
title = {Mitsuba renderer}
}
Nori
Nori
Nori is a simple physically based ray tracer for programming assignments in a graduate-level computer graphics course. It is used in the courses CS6630 at Cornell University, Image Synthesis at ETH Zürich, Advanced Computer Graphics at EPFL, and Rendering at Universität Tübingen.
Nori
pybind11
pybind11 is a lightweight header library that exposes C++ types in Python and vice versa, mainly to create Python bindings of existing C++ code. It has a strong focus on scientific computing applications built around SciPy/NumPy.
Nori
nanogui
NanoGUI is a a minimalistic cross-platform widget library for OpenGL 3.x. It supports automatic layout generation, stateful C++11 lambdas callbacks, a variety of useful widget types and Retina-capable rendering on Apple devices. Python bindings are also provided.
Scientific HDR processing pipeline
Scientific HDR processing pipeline
While doing reflectance measurements at Cornell, I found that existing programs to convert RAW exposure series to high dynamic range images did not reliably compute linear radiance values. Their target audience usually consists of photographers, so linearity is not a big constraint: some tools applied nonlinear postprocessing filters or used inaccurate exposure values from the EXIF metadata. Others demosaiced the possibly under- or overexposed input images instead of the final HDR data. The most useful program called dcraw is very difficult to understand or modify. For this reason, I wrote a new scientific HDR merging tool that is suitable for linear measurements and easy to adapt to new projects.
Fluid simulator
Fluid Simulator
This was my final project in the course “Sparse Matrix Computations” taught by Charles Van Loan. It is a projection-method fluid simulator using staggered grids and semi-Lagrangian advection. It solves for the fluid pressure using preconditioned conjugate gradients and a modified incomplete cholesky factor; numerical dampening is reduced using vorticity confinement and the output can be rendered using Mitsuba.
Automatic differentiation library
Automatic Differentiation Library
autodiff.h is a fully templated automatic differentiation library for C++ implemented in a single header file.
xic compiler
xic: an optimizing compiler targeting Intel x86/x86_64 assembly
xic is a cross-platform compiler for a simple object-oriented language co-developed with Owen Arden, Catalin Dumitru, and Danfeng Zhang for a compiler course at Cornell, winning the first prize in the competition. I continued working on it later on to add support for 64 bit code generation. Features include a graph coloring register allocator, register coalescing, partial redundancy elimination, as well as copy and constant propagation.
Final animation project

Eating Healthy

This was my final project for Don Greenberg’s course Computer Animation.

Open Source Hardware Cores

Open Source Hardware Cores

For my undergraduate thesis, I designed a ray tracing co-processor for bidirectional path tracing consisting of several hardware cores developed in VHDL; the underlying designs and test-benches are released here under the GNU GPL:

Z80 Microcomputer

Z80 Microcomputer

This page describes a Z80 microcomputer I designed and built a while ago.

Miscellaneous

Mitsuba development blog

Mitsuba development blog

This blog discusses ongoing developments regarding the Mitsuba renderer project.

My personal blog

My personal blog

This is my personal blog, mostly about travels and my time at Cornell.

Japanese studies

Japanese studies

Since my time at Cornell, I've been studying the Japanese language and writing system, relying on a spaced repetition software called Anki. This involves a daily routine of reviewing a set of automatically selected flash cards, where the trick is that Anki always picks cards one is just about to forget (accessing memories in this way promotes long term memory, making language learning quite a bit easier.) The existing flash card sets for Anki didn't work so well for me, and I ended up making my own. They're available below and updated periodically for the benefit of other Japanese language lerners:

  • Elementary Kanji characters (currently ~1070) with vocabulary and stroke order diagrams: kanji.apkg
  • Basic words, expressisons, and phrases: words.apkg