Lighting a Sphere part the Third - refactoring to use a normal map

So last time we ended up with a sphere with lighting, though it was kinda chunky looking as a result of the lighting being derived from the geometry of the sphere. Now lets see if we can decouple the lighting from the geometry. Currently the lighting is done directly on the texture map based on the normals of each triangle. Thus we are calculating the normals of each triangle in the mesh, and then getting the difference between the normal and the direction of the light, in order to derive the intensity of the lighting.

But it is not necessary to recalculate the normals every frame, since they don’t change, at least in the object coordinates. Thus it makes sense to cache the normals somehow - this is where the normal map comes in. So we can refactor our current example by converting the normals of each surface to a color, where the Red, Green, and Blue channels represent the x, y, & z components of the vector. How do you transform a vector into a color? I did it like this:

        var r:uint = 255*(normal.x + 1)/2;
        var g:uint = 255*(normal.y + 1)/2;
        var b:uint = 255*(normal.z + 1)/2;
        var normalColor:uint = r << 16 | g << 8 | b;

Since the normal is in the range of -1 to 1, we add one to it to put it in the range of 0 - 2, then divide by 2 to get it in the 0 -1 range. Multiplying by 255 gives a 8-bit value. The we shift these over to the right channels for a RGB color using binary operators. So that gives a normal map.. it actually looks quite pretty… click the buttons in the example below to see the normal map and the texture map (which currently just has the shading in it).

flash 10 required

Another advantage of having the normal map in the form of an image, is that we can use PixelBender to process it to produce the lighting/shading from the combintation of the normals and the light direction. All we have to do is the same calculations we were doing before (explained in previous lession) inside the Pixelbender kernel. As it turns out, PixelBender had lots of builtin functions to help with this kind of calculation, and it pretty straightforward. The kernel is really tiny, so here it is in full:

   <languageVersion : 1.0;>

  kernel NormalMap2TextureMap
  <   namespace : "com.dafishinsea";
      vendor : "David Wilhelm";
      version : 1;
      description : "kernel to generate a texture map (shading only) from a normal map";
  >
  {
      input image4 src;
      output pixel4 dst;
      parameter float xr
      <
          minValue: -1.0;
          maxValue: 1.0;
          defaultValue: 0.1;
      >;
      parameter float yr
      <
          minValue: -1.0;
          maxValue: 1.0;
          defaultValue: 0.0;
      >;
      parameter float zr
      <
          minValue: -1.0;
          maxValue: 1.0;
          defaultValue: 0.0;
      >;

      void
      evaluatePixel()
      {
          //get currrent pixel color
          pixel4 p = sampleNearest(src,outCoord());
          //image4 is a 4 element vector of RGBA channels
          //we want a vector of the first three as the normal
          float xd = 2.0*p.r - 1.0;
          float yd = 2.0*p.g - 1.0;
          float zd = 2.0*p.b - 1.0;
          float3 normal = float3(xd, yd, zd);
          float3 light = float3(xr, yr, zr) ;
          float dotp = dot(light, normal);
          //angle between vectors = acos (a dot b)  
          float angle = acos(dotp);
          float clr = angle/3.14159265358;
          dst = pixel4(clr, clr, clr, 1.0);

      }
  }

All this does is convert the normal (color at the source pixel) to a shading color (greyscale) depending on its variance with the lighting direction (which is passed in to the kernel as a parameter – actually 3, for x,y,z components). The huge advantage here is that this process gets called by Actionscript as a ShaderJob, and it runs ansynchronously. The result gets used as the texture for the sphere. I won’t go over the mechanics of using a ShaderJob – check out this article for more info. Or you can just read the code below, for all the gory details.

package com.dafishinsea.tutorials.normalmap {

  import com.bit101.components.*;
  import flash.display.Bitmap;
  import flash.display.BitmapData;
  import flash.display.Shader;
  import flash.display.ShaderJob;
  import flash.display.Shape;
  import flash.display.Sprite;
  import flash.display.StageAlign;
  import flash.display.StageScaleMode;
  import flash.display.TriangleCulling;
  import flash.events.Event;
  import flash.filters.ShaderFilter;
  import flash.geom.Matrix3D;
  import flash.geom.PerspectiveProjection;
  import flash.geom.Point;
  import flash.geom.Rectangle;
  import flash.geom.Utils3D;
  import flash.geom.Vector3D;
  import flash.utils.ByteArray;
  import net.hires.debug.Stats;

  [SWF(backgroundColor="0xffffff", width="800", height="600", frameRate="50")]  
  public class NormalMap3 extends Sprite {

      private var texture:BitmapData = new BitmapData(500,500,false,0xFFFFFF);
      private var textureBmp:Bitmap = new Bitmap(texture);
      private var normalMap:BitmapData = new BitmapData(500,500,false,0xFFFFFF);
      private var normalBmp:Bitmap = new Bitmap(normalMap);
      private var normalMapShp:Shape = new Shape();
      private var vertices:Vector.<Number>;
      private var indices:Vector.<int>;
      private var uvtData:Vector.<Number>;
      private var perspective: PerspectiveProjection;
      private var projectionMatrix : Matrix3D;
      private var rotationMatrix : Matrix3D;
      private var projectedVerts:Vector.<Number>;
      private var focalLength:Number = 50;
      private var container:Sprite;
      private const PI:Number = Math.PI;//half revolution in radians
      private const HALFPI:Number = Math.PI/2;//1/4 revolution in radians
      private const TWOPI:Number = 2*Math.PI;//full revolution in radians
      //x,yz rotation
      private var rx:Number = 60;
      private var ry:Number = 40;
      private var rz:Number = 30;
      private var testBmp:Bitmap;
      private var r:Number = 0;//yrotation of sphere
      private var normalMapShader:Shader;
      private var shaderJob:ShaderJob;
      private var input:ByteArray;
      private var output:ByteArray;
      private var normalMapRdo:PushButton;
      private var textureMapRdo:PushButton;
      private var noMapsRdo:PushButton;

      [Embed(source="NormalMapShader.pbj", mimeType="application/octet-stream")]
      private var NormalMapShader:Class;

      public function NormalMap3() {
          init();
          trace("init");
      }

      private function init():void {
          stage.align = StageAlign.TOP_LEFT;
          stage.scaleMode = StageScaleMode.NO_SCALE;
          vertices = new Vector.<Number>();
          indices = new Vector.<int>();
          uvtData = new Vector.<Number>();
          //set up perspective
          perspective = new PerspectiveProjection();
          perspective.fieldOfView = 50;
          //3D transformation matrix - used to rotate object
          projectionMatrix = perspective.toMatrix3D();
          rotationMatrix = new Matrix3D();//used to keep track of rotation of the sphere
          projectedVerts   = new Vector.<Number>();
          //container  to hold scene
          container = new Sprite();
          container.x = stage.stageWidth/2;
          container.y = stage.stageHeight/2;
          addChild(container);
          createSphere(100, 40, 80);
          createNormalMap();
          createShaderJob();
          addEventListener(Event.ENTER_FRAME, onEnterFrame);
          addEventListener(Event.RENDER, render, false, 0, true);
          addChild(new Stats());
          //texture map
          //controls
          normalMapRdo = new PushButton(this, 500, 30, "Show normal map", onNomalMapBtnClick);
          textureMapRdo = new PushButton(this, 500, 50, "Show texture map", onTextureRdoClick);
          noMapsRdo = new PushButton(this, 500, 10, "Show only sphere", onNoMapsRdoClick);
      }
      protected function onNomalMapBtnClick(event:Event):void
      {
          if(textureBmp.parent == this)
              removeChild(textureBmp);
          addChild(normalBmp);
      }

      protected function onTextureRdoClick(event:Event):void
      {
          if(normalBmp.parent == this)
              removeChild(normalBmp);
          addChild(textureBmp);
      }

      protected function onNoMapsRdoClick(event:Event):void
      {
          if(normalBmp.parent == this)
              removeChild(normalBmp);
          if(textureBmp.parent == this)
              removeChild(textureBmp);
      }

      private function createSphere(radius:Number, rows:int, cols:int):void {

          var lon_incr:Number = TWOPI/cols;
          var lat_incr = PI/rows;
          var lon:Number = 0;//angle of rotation around the y axis, *in radians*
          var lat:Number = 0;//angle of rotation around the x axis
          var x:Number, y:Number, z:Number;
          var vnum:int = 0;
          var ind:int = 0;
          //a full rotation is PI radians
          for(var h:int = 0; h <= rows; ++h)
          {
              lon = 0;
              y = radius*Math.cos(lat);//need to shift angle downwards by 1/4 rev
              for(var v:int = 0; v <= cols; ++v)
              {
                  x = radius*Math.cos(lon)*Math.sin(lat);
                  z = radius*Math.sin(lon)*Math.sin(lat);//seen from above, z = y

                  //add vertex triplet
                  vertices[vnum] = x;
                  vertices[vnum+1] = y;
                  vertices[vnum+2] = z;
                  //uvts
                  uvtData[vnum] = v/cols;
                  uvtData[vnum+1]  = h/rows;
                  uvtData[vnum+2] = 1;

                  vnum+=3;
                  //add indices
                  if(h < rows && v < cols){
                      indices.push(ind, ind+1, ind + cols+1);
                      indices.push(ind + cols+1, ind+1, ind + cols + 2);
                  }
                  ind+=1;
                  lon += lon_incr;
              }
              lat += lat_incr;
          }
      }
      /**
      * create ShaderJob for use later
      */
      private function createShaderJob():void
      {
          normalMapShader = new Shader(new NormalMapShader());
          normalMapShader.data.src.width = normalMap.width;
          normalMapShader.data.src.height = normalMap.height;
          normalMapShader.data.src.input = normalMap;
          
          shaderJob = new ShaderJob(normalMapShader, texture, width, height);
          shaderJob.addEventListener(Event.COMPLETE, shaderJobCompleteHandler);
          shaderJob.start();
      }

      /**
      * shade job complete
      */
      private function shaderJobCompleteHandler(event:Event):void
      {
          //update params and restart job
          var lightDir:Vector3D = rotationMatrix.transformVector(new Vector3D(1,1,1));
          lightDir.normalize();
          normalMapShader.data.xr.value = [lightDir.x];
          normalMapShader.data.yr.value = [lightDir.y];
          normalMapShader.data.zr.value = [lightDir.z];
          shaderJob = new ShaderJob(normalMapShader, texture, width, height);
          shaderJob.addEventListener(Event.COMPLETE, shaderJobCompleteHandler);
          shaderJob.start();
      }

      /**
      * create normal map from geometry to reuse for lighting
      */
      private function createNormalMap():void
      {
          for(var i:int = 0; i < indices.length; i+=3){
              //get the 3 points of the triangle as Vector3D objects
              var pt1:Vector3D = new Vector3D(vertices[indices[i]*3],
                                          vertices[indices[i]*3+1],
                                          vertices[indices[i]*3+2]);
              var pt2:Vector3D = new Vector3D(vertices[indices[i+1]*3],
                                              vertices[indices[i+1]*3+1],
                                              vertices[indices[i+1]*3+2]);
              var pt3:Vector3D = new Vector3D(vertices[indices[i+2]*3],
                                              vertices[indices[i+2]*3+1],
                                              vertices[indices[i+2]*3+2]);
              //subtract the first two from the third
              var d1:Vector3D = pt3.subtract(pt1);
              var d2:Vector3D = pt3.subtract(pt2);
              //get the cross-product of the results to get the normal
              var normal:Vector3D = d1.crossProduct(d2);
              normal.normalize();
              var r:uint = 255*(normal.x + 1)/2;
              var g:uint = 255*(normal.y + 1)/2;
              var b:uint = 255*(normal.z + 1)/2;
              var normalColor:uint = r << 16 | g << 8 | b;
              normalMapShp.graphics.beginFill(normalColor);
              var p1:Point = new Point(uvtData[indices[i]*3]*normalMap.width, uvtData[indices[i]*3+1]*normalMap.height);
              var p2:Point = new Point(uvtData[indices[i+1]*3]*normalMap.width, uvtData[indices[i+1]*3+1]*normalMap.height);
              var p3:Point = new Point(uvtData[indices[i+2]*3]*normalMap.width, uvtData[indices[i+2]*3+1]*normalMap.height);
              normalMapShp.graphics.moveTo(p1.x, p1.y);
              normalMapShp.graphics.lineTo(p2.x, p2.y);
              normalMapShp.graphics.lineTo(p3.x, p3.y);
              normalMapShp.graphics.lineTo(p1.x, p1.y);
              normalMapShp.graphics.endFill();
          }
          normalMap.draw(normalMapShp);

      }
      /**
      * save the normal map -- so I can play with it in PixelBender Toolkit ;)
      */
      private function onSaveBtnClick(click:Event):void
      {
          var savingBmp:SavingBitmap = new SavingBitmap(normalMap);
          savingBmp.encoderType = "png";
          savingBmp.save("normalMap.png");
          
      }

      private function onEnterFrame(event:Event):void {
          //createSphere()
          update();
      }

      private function update():void {
          r+=0.5;
          rotationMatrix = new Matrix3D();
          rotationMatrix.prependRotation(-r, new Vector3D(0,1,0))
          projectionMatrix = perspective.toMatrix3D();
          projectionMatrix.prependTranslation(0.0,0.0,250);
          projectionMatrix.prependRotation(r, new Vector3D(0,1,0))
          stage.invalidate();//trigger render event

      }
      private function render(e:Event):void {
          
          container.graphics.clear();
          Utils3D.projectVectors(projectionMatrix, vertices, projectedVerts, uvtData);
          container.graphics.beginBitmapFill(texture,null, false, false);
          container.graphics.drawTriangles(projectedVerts, indices, uvtData, TriangleCulling.POSITIVE);
          container.graphics.endFill();
      }

      
  }
}

You may have noticed that there is a white blotch in the darkest part of the shadow… this is a bug in the shader, which I have yet to figure out… let me know if you know what it is :)

The next step is to liberate the normal map from the actual geometry, in order to make the lighting more detailed than the geometry… the dramatic conclusion to this series is coming soon!

As usual, code is on GitHub…

NormalMap3.as

Da Fish in Sea

These are the voyages of Captain Observant

Lighting a Sphere Part the Third - Refactoring to Use a Normal Map