Pages

22/02/2013

Pseudo Lens Flare

Lens flare is a photographic artefact, caused by various interactions between a lens and the light passing through it. Although it is an artefact, there are a number of motives for simulating lens flare for use in computer graphics:
  • it increases the perceived brightness and the apparent dynamic range of an image
  • lens flare is ubiquitous in photography, hence its absence in a computer generated images can be conspicuous
  • it can play an important stylistic or dramatic role, or work as part of the gameplay mechanics for video games (think of glare blinding the player)
For real time lens flares, sprite-based techniques have traditionally been the most common approach. Although sprites produce easily controllable and largely realistic results, they have to be placed explicitly and require occlusion data to be displayed correctly. Here I'll describe a simple and relatively cheap screen space process which produces a "pseudo" lens flare from an input colour buffer. It is not physically based, so errs somewhat from photorealism, but can be used as an addition to (or enhancement of) traditional sprite-based effects.

Algorithm

The approach consists of 4 stages:
  1. Downsample/threshold.
  2. Generate lens flare features.
  3. Blur.
  4. Upscale/blend with original image.

1. Downsample/Threshold

Downsampling is key to reducing the cost of subsequent stages. Additionally, we want to select a subset of the brightest pixels in the source image to participate in the lens flare. Using a scale/bias provides a flexible way to achieve this:
   uniform sampler2D uInputTex;

   uniform vec4 uScale;
   uniform vec4 uBias;

   noperspective in vec2 vTexcoord;

   out vec4 fResult;

   void main() {
      fResult = max(vec4(0.0), texture(uInputTex, vTexcoord) + uBias) * uScale;
   }
Adjusting the scale/bias is the main way to tweak the effect; the best settings will be dependant on the dynamic range of the input as well as how subtle you want the result to look. Because of the approximate nature of this technique, subtle is probably better.

2. Feature Generation

Lens flare features tend to pivot around the image centre. To mimic this, we can just flip the result of the previous stage horizontally/vertically. This is easily done at the feature generation stage by flipping the texture coordinates:
   vec2 texcoord = -vTexcoords + vec2(1.0);
Doing this isn't strictly necessary; the rest of the feature generation works perfectly well with or without it. However, the result of flipping the texture coordinates helps to visually separate the lens flare effect from the source image.

GHOSTS

"Ghosts" are the repetitious blobs which mirror bright spots in the input, pivoting around the image centre. The approach I've take to generate these is to get a vector from the current pixel to the centre of the screen, then take a number of samples along this vector.
   uniform sampler2D uInputTex;

   uniform int uGhosts; // number of ghost samples
   uniform float uGhostDispersal; // dispersion factor

   noperspective in vec2 vTexcoord;

   out vec4 fResult;

   void main() {
      vec2 texcoord = -vTexcoord + vec2(1.0);
      vec2 texelSize = 1.0 / vec2(textureSize(uInputTex, 0));
 
   // ghost vector to image centre:
      vec2 ghostVec = (vec2(0.5) - texcoord) * uGhostDispersal;
   
   // sample ghosts:  
      vec4 result = vec4(0.0);
      for (int i = 0; i < uGhosts; ++i) { 
         vec2 offset = fract(texcoord + ghostVec * float(i));
  
         result += texture(uInputTex, offset);
      }
 
      fResult = result;
   }
Note that I use fract() to ensure that the texture coordinates wrap around; you could equally use GL_REPEAT as the texture's/sampler's wrap mode.

Here's the result:
We can improve this by allowing only bright spots from the centre of the source image to generate ghosts. We do this by weighting samples by a falloff from the image centre:
   vec4 result = vec4(0.0);
   for (int i = 0; i < uGhosts; ++i) { 
      vec2 offset = fract(texcoord + ghostVec * float(i));
      
      float weight = length(vec2(0.5) - offset) / length(vec2(0.5));
      weight = pow(1.0 - weight, 10.0);
  
      result += texture(uInputTex, offset) * weight;
   }
The weight function is about as simple as it gets - a linear falloff. The reason we perform the weighting inside the sampling loop is so that bright spots in the centre of the input image can 'cast' ghosts to the edges, but bright spots at the edges can't cast ghosts to the centre.
A final improvement can be made by modulating the ghost colour radially according to a 1D texture:
This is applied after the ghost sampling loop so as to affect the final ghost colour:
   result *= texture(uLensColor, length(vec2(0.5) - texcoord) / length(vec2(0.5)));

HALOS

If we take a vector to the centre of the image, as for the ghost sampling, but fix the vector length, we get a different effect: the source image is radially warped:
We can use this to produce a "halo", weighting the sample to to restrict the contribution of the warped image to a ring, the radius of which is controlled by uHaloWidth:
   // sample halo:
   vec2 haloVec = normalize(ghostVec) * uHaloWidth;
   float weight = length(vec2(0.5) - fract(texcoord + haloVec)) / length(vec2(0.5));
   weight = pow(1.0 - weight, 5.0);
   result += texture(uInputTex, texcoord + haloVec) * weight;

CHROMATIC DISTORTION

Some lens flares exhibit chromatic distortion, caused by the varying refraction of different wavelengths of light. We can simulate this by creating a texture lookup function which fetches the red, green and blue channels separately at slightly different offsets along the sampling vector:

   vec3 textureDistorted(
      in sampler2D tex,
      in vec2 texcoord,
      in vec2 direction, // direction of distortion
      in vec3 distortion // per-channel distortion factor
   ) {
      return vec3(
         texture(tex, texcoord + direction * distortion.r).r,
         texture(tex, texcoord + direction * distortion.g).g,
         texture(tex, texcoord + direction * distortion.b).b
      );
   }
This can be used as a direct replacement for the calls to texture() in the previous listings. I use the following for the direction and distortion parameters:

   vec2 texelSize = 1.0 / vec2(textureSize(uInputTex, 0));
   vec3 distortion = vec3(-texelSize.x * uDistortion, 0.0, texelSize.x * uDistortion);

   vec3 direction = normalize(ghostVec);
Although this is simple it does cost 3x as many texture fetches, although they should all be cache-friendly unless you set uDistortion to some huge value.

That's it for feature generation. Here's the result:

3. Blur

Without applying a blur, the lens flare features (in particular, the ghosts) tend to retain the appearance of the source image. By applying a blur to the lens flare features we attenuate high frequencies and in doing so reduce the coherence with the input image, which helps to sell the effect.
I'll not cover how to achieve the blur here; there are plenty of resources on the web.

4. Upscale/Blend

So now we have our lens flare features, nicely blurred. How do we combine this with the original source image? There are a couple of important considerations to make regarding the overall rendering pipeline:

  • Any post process motion blur or depth of field effect must be applied prior to combining the lens flare, so that the lens flare features don't participate in those effects. Technically the lens flare features would exhibit some motion blur, however it's incompatible with post process motion techniques. As a compromise, you could implement the lens flare using an accumulation buffer.
  • The lens flare should be applied before any tonemapping operation. This makes physical sense, as tonemapping simulates the reaction of the film/CMOS to the incoming light, of which the lens flare is a constituent part.
With this in mind, there are a couple of things we can do at this stage to improve the result:

LENS DIRT

The first is to modulate the lens flare features by a full-resolution "dirt" texture (as used heavily in Battlefield 3):
   uniform sampler2D uInputTex; // source image
   uniform sampler2D uLensFlareTex; // input from the blur stage
   uniform sampler2D uLensDirtTex; // full resolution dirt texture

   noperspective in vec2 vTexcoord;

   out vec4 fResult;

   void main() {
      vec4 lensMod = texture(uLensDirtTex, vTexcoord);
      vec4 lensFlare = texture(uLensFlareTex, vTexcoord) * lensMod;
      fResult = texture(uInputTex, vTexcoord) + lensflare;
   }
The key to this is the lens dirt texture itself. If the contrast is low, the shapes of the lens flare features tend to dominate the result. As the contrast increases, the lens flare features are subdued, giving a different aesthetic appearance, as well as hiding a few of the imperfections.

DIFFRACTION STARBURST

As a further enhancement, we can use a starburst texture in addition to the lens dirt:
As a static texture, the starburst doesn't look very good. We can, however, provide a transformation matrix to the shader which allows us to spin/warp it per frame and produce the dynamic effect we want:

   uniform sampler2D uInputTex; // source image
   uniform sampler2D uLensFlareTex; // input from the blur stage
   uniform sampler2D uLensDirtTex; // full resolution dirt texture

   uniform sampler2D uLensStarTex; // diffraction starburst texture
   uniform mat3 uLensStarMatrix; // transforms texcoords

   noperspective in vec2 vTexcoord;

   out vec4 fResult;

   void main() {
      vec4 lensMod = texture(uLensDirtTex, vTexcoord);

      vec2 lensStarTexcoord = (uLensStarMatrix * vec3(vTexcoord, 1.0)).xy;
      lensMod += texture(uLensStarTex, lensStarTexcoord);

      vec4 lensFlare = texture(uLensFlareTex, vTexcoord) * lensMod;
      fResult = texture(uInputTex, vTexcoord) + lensflare;
   }
The transformation matrix uLensStarMatrix is based on a value derived from the camera's orientation as follows:
   vec3 camx = cam.getViewMatrix().col(0); // camera x (left) vector
   vec3 camz = cam.getViewMatrix().col(1); // camera z (forward) vector
   float camrot = dot(camx, vec3(0,0,1)) + dot(camz, vec3(0,1,0));
There are other ways of obtaining the camrot value; it just needs to change continuously as the camera rotates. The matrix itself is constructed as follows:

   mat3 scaleBias1 = (
      2.0f,   0.0f,  -1.0f,
      0.0f,   2.0f,  -1.0f,
      0.0f,   0.0f,   1.0f,
   );
   mat3 rotation = (
      cos(camrot), -sin(camrot), 0.0f,
      sin(camrot), cos(camrot),  0.0f,
      0.0f,        0.0f,         1.0f
   );
   mat3 scaleBias2 = (
      0.5f,   0.0f,   0.5f,
      0.0f,   0.5f,   0.5f,
      0.0f,   0.0f,   1.0f,
   );

   mat3 uLensStarMatrix = scaleBias2 * rotation * scaleBias1;
The scale and bias matrices are required in order to shift the texture coordinate origin so that we can rotate the starburst around the image centre.

Conclusion

So, that's it! This method demonstrates how a relatively simplistic, image-based post process can produce a decent looking lens flare. It's not quite photorealistic, but when applied subtly can give some lovely results. I've provided a demo implementation.

22 comments:

  1. Great work, Keep it up Mr Chapman. My rendering engine is coming along but im reaching the point where ill need a good bit of help and advice. If i could send you another mail about somethings that would be great, Ill also send a copy of my code over. Your camera movement and materials are awesome. :)

    ReplyDelete
  2. First of all excellent article! In the shader that creates the ghosts the MAX_GHOSTS and texelSize are not used. Is something missing from those?

    ReplyDelete
    Replies
    1. texelSize is used in the chromatic distortion further down, but you're right about MAX_GHOSTS, I've removed it.

      Delete
  3. I'd just like to say this is a fantastic blog. All your posts are very interesting and informative. Please keep it up!

    ReplyDelete
  4. for those interested, i did a three.js implementation for webgl

    https://github.com/jeromeetienne/threex.sslensflare

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Awesome stuff, I need this for Nuke Pleasse!!!!

    ReplyDelete
  7. Is there any other way to apply radial color to the ghost and the halos? 1D texture doesn't seem to be working for me. Great tutorial btw! Thanks for sharing!

    ReplyDelete
  8. Hi John! Amazing tutorial :)

    How do you prevent just about any little white spot - like specular spots - to cast a flare?
    In your video you have teapot with lots of strong specular spots, yet they don't cast flares.

    Great content anyway!

    ReplyDelete
  9. How can I avoid lens flare from head on lights?
    photography

    ReplyDelete
  10. Hello, U write some extraordinarily attractive blogs. I always check back here frequently to see if you have updated
    hawaii aerial photography

    ReplyDelete
  11. I wasn't quite able to reproduce the full effect, but I was able to convert it to a "glare" effect in minecraft. Only around 70 lines and is very light on my GPU. Here is a screenshot.
    http://imgur.com/ua2XuZl
    One good part about it is that its not only limited to the sun/moon, It appears on bright objects (usually would take messing with lightmaps). Thanks for making this.

    ReplyDelete
    Replies
    1. Can I get a look at your code I might be able to fix it

      Delete
  12. Thanks a lot for this excellent tutorial! I liked your camera movements so much.Can you give your camera code?

    ReplyDelete
  13. Super cool man! I made an implementation into Max/MSP/Jitter:
    https://www.youtube.com/watch?v=VeKkJ6Chduw

    ReplyDelete
  14. Can I get a look at your code I might be able to fix it

    ReplyDelete
  15. Thanks a lot for this, I just implemented the most important parts in my compositing software (Nuke) as a little tool, I cutomized the rest to make it a bit more personal and to my tastes.

    ReplyDelete
  16. You were great and everyone received so 먹튀검증 much from your experience and knowledge.

    ReplyDelete
  17. Take a look at my SolarSystem 3D project, there are many things implemented there, including lens flare
    https://github.com/GlebchanskyGit/SolarSystem-3D

    ReplyDelete
  18. Thanks a lot for giving us such a helpful information. You can also visit our website for scdl project help

    ReplyDelete