Ray tracing demystified

For decades, real-time computer graphics have been built on clever illusions. Shadow maps approximate shadows by rendering the scene from the light's perspective. Environment maps fake reflections by storing a static image of the surroundings. Ambient occlusion estimates how exposed each point is to ambient light. Screen-space techniques infer global effects from limited local information. These rasterization tricks are fast and often convincing, but they are fundamentally heuristics - approximations that work well enough in most cases but can break down in edge cases.

Ray tracing takes a radically different approach: instead of faking it, simulate how light actually behaves. Fire mathematical rays from the camera into the scene, determine what they hit, calculate how light interacts with those surfaces, and trace additional rays to simulate reflections, refractions, and shadows. The results can be stunning - physically accurate reflections on curved surfaces, soft shadows with correct penumbras, proper caustics from light focusing through glass, and natural [[global illumination]] where light bounces between surfaces just as it does in the real world.

How light works (and how we simulate it)

In the physical world, light sources emit photons that travel in straight lines until they hit a surface. At the surface, photons might be absorbed (converting light energy to heat), reflected (bouncing off at predictable or scattered angles), or transmitted (passing through transparent materials, possibly bending due to refraction). Eventually, some photons enter your eye, and the pattern of which photons from which directions creates your perception of the scene.

Simulating this directly - shooting photons from light sources and tracking them until they reach the camera - is called forward ray tracing or light tracing. It is physically accurate but computationally disastrous. The vast majority of photons from a light source never reach the camera, so you waste enormous computation tracking irrelevant light paths.

The key insight, attributed to Albrecht Dürer in the 15th century and formalized by Arthur Appel in 1968, is that we can trace rays backward - from the camera into the scene. Since we only care about light that reaches the camera, and light paths are reversible, we can start at the destination and work backward to the sources. This is backward ray tracing, usually just called ray tracing, and it forms the foundation of all modern ray-based rendering.

The basic ray tracing algorithm

The simplest ray tracer works as follows: for each pixel on the screen, construct a [[ray]] originating at the camera and passing through that pixel into the scene. Find the closest surface the ray intersects. At that intersection point, calculate the color by considering direct illumination from light sources (casting shadow rays to check visibility) and, if the surface is reflective or refractive, recursively trace additional rays.

function render(scene, camera):
    for each pixel (x, y):
        ray = camera.generateRay(x, y)
        color = traceRay(ray, scene, depth=0)
        setPixel(x, y, color)

function traceRay(ray, scene, depth):
    if depth > MAX_DEPTH:
        return BLACK  // Prevent infinite recursion
    
    hit = scene.findClosestIntersection(ray)
    
    if no hit:
        return scene.backgroundColor  // Or sky color, environment map, etc.
    
    // Start with emission (for light sources)
    color = hit.material.emission
    
    // Add direct lighting from each light
    for each light in scene.lights:
        shadowRay = Ray(hit.point, directionToLight)
        if not scene.isOccluded(shadowRay):
            color += calculateDirectLighting(hit, light)
    
    // Add reflection if material is reflective
    if hit.material.reflectivity > 0:
        reflectDir = reflect(ray.direction, hit.normal)
        reflectRay = Ray(hit.point, reflectDir)
        color += hit.material.reflectivity * traceRay(reflectRay, scene, depth + 1)
    
    // Add refraction if material is transparent
    if hit.material.transparency > 0:
        refractDir = refract(ray.direction, hit.normal, hit.material.ior)
        refractRay = Ray(hit.point, refractDir)
        color += hit.material.transparency * traceRay(refractRay, scene, depth + 1)
    
    return color

This basic algorithm produces beautiful results for scenes with mirrors and glass. Perfect reflections come naturally - just trace the reflected ray. Refraction through transparent objects with correct bending appears automatically. Shadows are pixel-perfect because we test actual visibility rather than approximating with shadow maps. The recursive structure elegantly handles multiple bounces - light reflecting off a mirror, through glass, off another mirror.

The intersection problem

The computational bottleneck in ray tracing is finding what each ray hits. This operation - ray-scene intersection - dominates the runtime. A naive approach tests every ray against every piece of geometry. For a scene with 1 million triangles and a 1080p image (about 2 million pixels), that is 2 trillion intersection tests per frame. Even at billions of tests per second, that is far too slow for real-time rendering.

The solution is spatial acceleration structures - data structures that organize geometry to rapidly exclude large portions of the scene from consideration. The most common is the [[BVH]] (Bounding Volume Hierarchy). The idea is simple: group nearby triangles into bounding boxes, group those boxes into larger boxes, and so on, forming a tree structure. To trace a ray, first test against the root bounding box. If the ray misses it, we are done - no geometry can possibly be hit. If it hits, descend into child nodes, pruning branches whose bounding boxes the ray misses.

A well-constructed BVH transforms millions of potential triangle tests into dozens of box tests plus a handful of triangle tests for leaves. The ray-box intersection test is extremely fast - just checking if the ray passes through an axis-aligned slab. This reduces average-case complexity from O(n) triangles to O(log n), making ray tracing practical even for complex scenes.

From ray tracing to path tracing

Basic ray tracing handles direct illumination and perfect specular effects (mirrors, glass) beautifully. But the real world has much more complex light transport. Light bounces diffusely between surfaces - a red wall tints nearby white objects pink. Soft shadows arise from area lights, not point lights. Caustics appear when light focuses through curved glass. Ambient occlusion naturally emerges from corners receiving less indirect light.

[[Path tracing]] extends ray tracing to capture these effects through [[Monte Carlo]] integration. Instead of tracing rays only in the reflection direction, we sample the full range of directions that light could arrive from, weighted by the material's bidirectional reflectance distribution function (BRDF). Each random path through the scene contributes a sample to the pixel's color. With enough samples, the average converges to the correct result.

The rendering equation, formulated by James Kajiya in 1986, describes light transport mathematically. It states that the light leaving a point equals the light emitted plus the integral over all incoming directions of incoming light times the BRDF times the cosine of the angle. This integral has no closed-form solution for realistic scenes, but Monte Carlo integration can estimate it by random sampling.

function pathTrace(ray, scene, depth):
    if depth > MAX_DEPTH:
        return BLACK
    
    hit = scene.findClosestIntersection(ray)
    if no hit:
        return scene.getEnvironmentLight(ray.direction)
    
    // Emission from this surface (if it's a light)
    emitted = hit.material.emission
    
    // Sample a random direction based on the BRDF
    (newDirection, pdf) = hit.material.sampleBRDF(ray.direction, hit.normal)
    
    // Evaluate the BRDF for this direction
    brdf = hit.material.evaluateBRDF(ray.direction, newDirection, hit.normal)
    
    // Cosine of angle between new direction and normal
    cosTheta = dot(newDirection, hit.normal)
    
    // Recursively trace and weight by BRDF
    newRay = Ray(hit.point, newDirection)
    incoming = pathTrace(newRay, scene, depth + 1)
    
    return emitted + (brdf * cosTheta * incoming) / pdf

Path tracing is unbiased - given infinite samples, it provably converges to the physically correct image. The catch is variance. With few samples per pixel, the image is extremely noisy, like looking through static. Rendering a clean image might require hundreds or thousands of samples per pixel, making path tracing historically the domain of offline rendering (movies, architectural visualization) rather than real-time graphics.

Making it real-time

Real-time ray tracing became practical through a combination of hardware advances, algorithmic improvements, and clever compromises. NVIDIA's RTX graphics cards, introduced in 2018, include dedicated [[RT Core]]s - fixed-function hardware units that accelerate BVH traversal and ray-triangle intersection by an order of magnitude compared to general-purpose GPU code.

But hardware alone would not be enough. The breakthrough came from AI-powered [[denoising]]. Neural networks trained on pairs of noisy and clean renderings learned to reconstruct plausible detail from sparse samples. Modern denoisers can produce acceptable images from just 1-4 samples per pixel - a 100× or more reduction in required rays. The denoiser essentially fills in what the ray tracer did not have time to compute.

Real-time games also use hybrid rendering - combining traditional rasterization with selective ray tracing. The primary view (what is closest to camera at each pixel) is still determined by rasterization, which remains faster for this task. Ray tracing is then used surgically for effects that benefit most: reflections on shiny surfaces, shadows from important lights, ambient occlusion in corners, global illumination bounces. This hybrid approach captures 90% of the visual improvement at a fraction of the cost of full path tracing.

The future: full path tracing?

Each generation of hardware brings us closer to full real-time path tracing. Some recent games like Cyberpunk 2077's 'Overdrive Mode' and Portal RTX use path tracing for all lighting, though they still require powerful hardware and accept lower framerates. The trajectory is clear - more rays per pixel, more bounces, less reliance on rasterization fallbacks.

The mathematical elegance of path tracing is compelling. Instead of a zoo of specialized techniques for different effects (shadow maps, reflection probes, lightmaps, irradiance volumes), one unified algorithm handles everything. Light just works - bouncing, reflecting, refracting, scattering - because we are simulating what light actually does. The complexity shifts from rendering algorithm design to material modeling and scene optimization.

For now, the practical approach remains hybrid: rasterize what is cheaper to rasterize, ray trace what benefits from ray tracing, and let AI fill in the gaps. But the trend is unmistakably toward more physically accurate simulation. The fixed-function pipeline gave way to programmable shaders; the rasterization pipeline may eventually give way to ray tracing. The vision of photorealistic real-time graphics, long the holy grail of computer graphics, is finally within reach.