Dev►Babylon.js

Babylon.js: Creating a convincing world for your game with custom shaders, height maps and skyboxes

After talking about multi-materials in the previous post, I would like to share with you a more advanced sample. This sample will allow me to introduce you some really powerful features of Babylon.js:

  • Height maps
  • Skyboxes
  • Custom shaders

The result will look like that (using IE11 preview in this case):




Click here if you want a live demonstration (If your browser supports WebGL of course)

This world is composed of a sky, a ground and a reflective/refractive water. So let’s discuss about each of them.

Preparing the web page

First of all we need a simple HTML 5 page with a full page canvas:

<!DOCTYPE html>
<html xmlns="https://www.w3.org/1999/xhtml">
<head>
    <title>Using babylon.js - Test page</title>
    <script src="hand.minified-1.1.0.js"></script>
    <script src="babylon.js"></script>
    <script src="Water/waterMaterial.js"></script>
    <style>
        html, body {
            width: 100%;
            height: 100%;
            padding: ;
            margin: ;
            overflow: hidden;
        }

        #renderCanvas {
            width: 100%;
            height: 100%;
            touch-action: none;
        }
    </style>
</head>
<body>
    <canvas id="renderCanvas"></canvas>
</body>
</html>

The page just needs to reference babylon.js (you can find the latest version here) and hand.js for the touch support (latest version here).

Then you can create a script block right after the canvas element with the following code:

<script>
    if (BABYLON.Engine.isSupported()) {
        var canvas = document.getElementById("renderCanvas");
        var engine = new BABYLON.Engine(canvas, true);
        var scene = new BABYLON.Scene(engine);

        var camera = new BABYLON.ArcRotateCamera("Camera", 0, 0, 10, BABYLON.Vector3.Zero(), scene);
        var sun = new BABYLON.PointLight("Omni0", new BABYLON.Vector3(60, 100, 10), scene);

        camera.setPosition(new BABYLON.Vector3(-40, 40, 0));

        var beforeRenderFunction = function () {
            // Camera
            if (camera.beta < 0.1)
                camera.beta = 0.1;
            else if (camera.beta > (Math.PI / 2) * 0.9)
                camera.beta = (Math.PI / 2) * 0.9;

            if (camera.radius > 50)
                camera.radius = 50;

            if (camera.radius < 5)
                camera.radius = 5;
        };

        camera.attachControl(canvas);

        scene.registerBeforeRender(beforeRenderFunction);

        engine.runRenderLoop(function () {
            scene.render();
        });
    }
</script>

This code creates the engine, the main scene and add a camera and a light (the sun) to it.

The camera is an ArcRotateCamera so you can use your mouse/touch/keyboard to rotate around a central pivot. We just want to limit the amplitude of the camera with beforeRenderFunction function (because we don’t want to go under the ground or beyond the sky). This function is attached to the scene with registerBeforeRender so it will be called before every frame to guarantee our constraints.

Adding a skybox

A skybox is a box with a special material used to simulate the sky:




The material uses a special reflection texture. To create a skybox with Babylon.js you just have to use this code (because skyboxes are already supported by the StandardMaterial):

// Skybox
var skybox = BABYLON.Mesh.CreateBox("skyBox", 1000.0, scene);
var skyboxMaterial = new BABYLON.StandardMaterial("skyBox", scene);
skyboxMaterial.backFaceCulling = false;
skyboxMaterial.reflectionTexture = new BABYLON.CubeTexture("skybox/skybox", scene);
skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
skyboxMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
skyboxMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
skybox.material = skyboxMaterial;

A skybox is just a box with a StandardMaterial. The key point is the CubeTexture used for the reflection channel. A cube texture is composed of 6 textures (one for each face of a cube). Babylon.js will choose the right one depending on the position of the viewer to simulate a continuous sky:


When you want to create a CubeTexture, you have to specify a file scheme based on the following; nx, ny, nz stand for negative (x, y, z) and px, py, pz stand for positive (x, y, z).

Adding the ground and the island

The ground is a simple plane textured with a repetitive bitmap:

To create it, the code is really simple:

var extraGround = BABYLON.Mesh.CreateGround("extraGround", 1000, 1000, 1, scene, false);
var extraGroundMaterial = new BABYLON.StandardMaterial("extraGround", scene);
extraGroundMaterial.diffuseTexture = new BABYLON.Texture("ground.jpg", scene);
extraGroundMaterial.diffuseTexture.uScale = 60;
extraGroundMaterial.diffuseTexture.vScale = 60;
extraGround.position.y = -2.05;
extraGround.material = extraGroundMaterial;

The diffuse texture is scaled by a ratio of 60 in order to repeat it along the ground.

The island also uses a plane but deforms it through an height map.

The height map is a simple map used to define the altitude of every vertex of the plane:

To use it, you have to create a plane with more subdivisions. The plane is then updated using the height map:

var ground = BABYLON.Mesh.CreateGroundFromHeightMap("ground", "heightMap.png", 100, 100, 100, 0, 10, scene, false);
var groundMaterial = new BABYLON.StandardMaterial("ground", scene);
groundMaterial.diffuseTexture = new BABYLON.Texture("ground.jpg", scene);
groundMaterial.diffuseTexture.uScale = 6;
groundMaterial.diffuseTexture.vScale = 6;
groundMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
ground.position.y = -2.0;
ground.material = groundMaterial;

The definition of the CreateGroundFromHeightMap is the following:

function (name, url, width, height, subdivisions, minHeight, maxHeight, scene, updatable)

As you can see the fifth parameter allows you to increase the complexity of your mesh in order to improve the visual quality of it:

The water

The water itself is just a simple plane created with the following code:

var water = BABYLON.Mesh.CreateGround("water", 1000, 1000, 1, scene, false);

The true water lies within the material used to display it. And this time we will not use the good old StandardMaterial.

Indeed, to create a convincing water surface, we need to take in account complex phenomena like refraction and reflection that the StandardMaterial is not intended to handle. To reproduce them, we will use a very powerful feature of Babylon.js: the render target textures. This kind of textures allows you to render a scene into a texture in order to use it later in your own shaders.

Creating an empty custom material

So we need to create our first custom material! To do so, we first need to add a new JavaScript file to our site. This file named waterMaterial.js will contains the following empty anonymous function:

(function() {
})();

Do not forget to reference it in your HTML page:

<script src="Water/waterMaterial.js"></script>

Starting from our empty anonymous function, let’s add a new object called WaterMaterial:

(function() {
    WaterMaterial = function (name, scene, light) {
        this.name = name;
        this.id = name;
        this.light = light;

        this._scene = scene;
        scene.materials.push(this);
    };

    WaterMaterial.prototype = Object.create(BABYLON.Material.prototype);

    // Properties   
    WaterMaterial.prototype.needAlphaBlending = function () {
        return false;
    };

    WaterMaterial.prototype.needAlphaTesting = function () {
        return false;
    };

    // Methods  
    WaterMaterial.prototype.isReady = function (mesh) {
        return true;
    };

    WaterMaterial.prototype.bind = function (world, mesh) {
    };

    WaterMaterial.prototype.dispose = function () {
        this.baseDispose();
    };
})();

This is the minimal code to provide for a custom material:

  1. A constructor to register the material to the scene and get important information (in my case I need the current light to simulate the sun)
  2. Our material MUST retrieve and use the BABYLON.Material prototype
  3. needAlphaBlending: The material must indicate to Babylon.js if it requires alpha blending (in our case we are not based on alpha blending but on render target textures)
  4. needAlphatesting: The material must indicate to Babylon.js if it requires alpha testing (same thing here, we do not need alpha testing)
  5. isReady: Babylon.js will call this function to know if the material is ready to be used
  6. bind: Babylon.js will call this function to activate the shader before rendering objects that use it
  7. dispose: This function allows you to release resources you may have created for your material

The vertex and fragment shaders

The next thing we need to prepare are the shaders themselves. Shaders define the code executed by the GPU to process the vertices sent by the meshes and the pixels produced by these vertices.

For more information about how a GPU works internally and how to create a 3D engine I suggest you to read the excellent series on how to create a 3D soft engine from scratch written by David Rousset:

https://blogs.msdn.com/b/davrous/archive/2013/06/13/tutorial-series-learning-how-to-write-a-3d-soft-engine-from-scratch-in-c-typescript-or-javascript.aspx

The goal of this blog is not to talk about shaders and GLSL but we need them, so here is the vertex shader that we will use for our material:


#ifdef GL_ES
precision mediump float;
#endif

 


// Attributes
attribute vec3 position;
attribute vec3 normal;
attribute vec2 uv;

// Uniforms
uniform vec2 waveData;
uniform mat4 windMatrix;
uniform mat4 world;
uniform mat4 worldViewProjection;

// Normal
varying vec3 vPositionW;
varying vec3 vNormalW;
varying vec4 vUV;
varying vec2 vBumpUV;

void main(void) {
    vec4 outPosition = worldViewProjection * vec4(position, 1.0);
    gl_Position = outPosition;

    vPositionW = vec3(world * vec4(position, 1.0));
    vNormalW = normalize(vec3(world * vec4(normal, 0.0)));

    vUV = outPosition;

    vec2 bumpTexCoord = vec2(windMatrix * vec4(uv, 0.0, 1.0));
    vBumpUV = bumpTexCoord / waveData.x;
}

And here is the fragment shader

#ifdef GL_ES
precision mediump float;
#endif

uniform vec3 vEyePosition;
uniform vec4 vLevels;
uniform vec3 waterColor;
uniform vec2 waveData;

// Lights
varying vec3 vPositionW;
varying vec3 vNormalW;
uniform vec3 vLightPosition;

// Refs
varying vec2 vBumpUV;
varying vec4 vUV;
uniform sampler2D refractionSampler;
uniform sampler2D reflectionSampler;
uniform sampler2D bumpSampler;

void main(void) {
    vec3 viewDirectionW = normalize(vEyePosition - vPositionW);

    // Light
    vec3 lightVectorW = normalize(vLightPosition - vPositionW);

    // Wave
    vec3 bumpNormal = 2.0 * texture2D(bumpSampler, vBumpUV).rgb - 1.0;
    vec2 perturbation = waveData.y * bumpNormal.rg;

    // diffuse
    float ndl = max(0., dot(vNormalW, lightVectorW));

    // Specular
    vec3 angleW = normalize(viewDirectionW + lightVectorW);
    float specComp = dot(normalize(vNormalW), angleW);
    specComp = pow(specComp, 256.);

    // Refraction
    vec2 texCoords;
    texCoords.x = vUV.x / vUV.w / 2.0 + 0.5;
    texCoords.y = vUV.y / vUV.w / 2.0 + 0.5;

    vec3 refractionColor = texture2D(refractionSampler, texCoords + perturbation).rgb;

    // Reflection
    vec3 reflectionColor = texture2D(reflectionSampler, texCoords + perturbation).rgb;

    // Fresnel
    float fresnelTerm = dot(viewDirectionW, vNormalW);
    fresnelTerm = clamp((1.0 - fresnelTerm) * vLevels.y, 0., 1.);

    // Water color

    vec3 finalColor = (waterColor * ndl) * vLevels.x + (1.0 - vLevels.x) * (reflectionColor * fresnelTerm * vLevels.z + 
                                                       (1.0 - fresnelTerm) * refractionColor * vLevels.w) + specComp;


    gl_FragColor = vec4(finalColor, 1.);
}

These shaders use many external variables to achieve the rendering. External variables (defined with the uniform keyword) are used to communicate between your code and the GPU. The GPU will execute the shaders code using the values provided to the external variables by your code:

  • waveData: Defines height and amplitude of waves
  • windMatrix: Defines the direction of the water
  • world: World matrix (Matrix of the current mesh)
  • worldViewProjection: Combined transformation matrix (world x view x projection)
  • vEyePosition: Camera’s position
  • vLightPosition: Sun’s position
  • vLevels: Blending levels of reflection and refraction
  • waterColor: Water’s color
  • refractionSampler: Variable used to read the refraction texture
  • reflectionSampler: Variable used to read the reflection texture
  • bumpSampler: Variable used to read the bump texture (which is used to generate the waves) 

To sum things up, we can decompose the shaders’ work through the following pipeline:

  1. The mesh is transformed by the vertex shader to generate the triangle used to find the pixels that require to be painted:

  1. Diffuse color is computed first (based on the sun position and on the water’s color)

  1. Specular is then added (based on camera’s position):

  1. The refraction texture is then used to simulate the transparency of the water:

  1. The bump texture is then used to add perturbations:

  1. The reflection texture is finally used to add the reflected objects (island and sky):

  1. The cherry on the cake is added by using a Fresnel computation in order to prioritize reflection or refraction depending on the view angle of the camera:



As you can see we start for an almost full reflection to finish to full with a full refraction when we are perpendicular with the ground.

Linking shaders to the material

Now we have created our shaders, we need to link them with the material. We also need to prepare data for the external variables used by the shaders. To do so, let’s update the constructor:

WaterMaterial = function (name, scene, light) {
    this.name = name;
    this.id = name;
    this.light = light;

    this._scene = scene;
    scene.materials.push(this);

    this.bumpTexture = new BABYLON.Texture("Water/bump.png", scene);
    this.bumpTexture.uScale = 2;
    this.bumpTexture.vScale = 2;
    this.bumpTexture.wrapU = BABYLON.Texture.MIRROR_ADDRESSMODE;
    this.bumpTexture.wrapV = BABYLON.Texture.MIRROR_ADDRESSMODE;

    this.reflectionTexture = new BABYLON.MirrorTexture("reflection", 512, scene, true);
    this.refractionTexture = new BABYLON.RenderTargetTexture("refraction", 512, scene, true); 
    this.reflectionTexture.mirrorPlane = new BABYLON.Plane(0, -1, 0, 0);

    this.refractionTexture.onBeforeRender = function() {
        BABYLON.clipPlane = new BABYLON.Plane(0, 1, 0, 0);
    };

    this.refractionTexture.onAfterRender = function() {
        BABYLON.clipPlane = null;
    };

    this.waterColor = new BABYLON.Color3(0.0, 0.3, 0.1);
    this.waterColorLevel = 0.2;
    this.fresnelLevel = 1.0;
    this.reflectionLevel = 0.6;
    this.refractionLevel = 0.8;

    this.waveLength = 0.1;
    this.waveHeight = 0.15;

    this.waterDirection = new BABYLON.Vector2(0, 1.0);

    this._time = 0;
};

The bump texture is a standard texture:

The reflection texture is created from a BABYLON.MirrorTexture which is able to simulate a mirror (exactly what we need!).

The refraction texture is based on a BABYLON.RenderTargetTexture. This kind of texture can receive the rendering of a scene and can then be used as standard texture resource for a shader. With the onBeforeRender and onAfterRender functions you can configure and restore back the scene for your rendering (In this case, I’ve just activated a clipping plane to limit the refraction to the objects over the water).

Then we need to add a new function to the WaterMaterial in order to register these two special textures:

WaterMaterial.prototype.getRenderTargetTextures = function () {
    var results = [];

    results.push(this.reflectionTexture);
    results.push(this.refractionTexture);

    return results;
};

This function is mandatory for us, because we need to have our textures prepared before using our shaders.

The link with the shaders itself is done with the isReady function:

WaterMaterial.prototype.isReady = function (mesh) {
    var engine = this._scene.getEngine();

    if (this.bumpTexture && !this.bumpTexture.isReady) {
        return false;
    }

    this._effect = engine.createEffect("Water/water",
        ["position", "normal", "uv"],
        ["worldViewProjection", "world", "view", "vLightPosition", "vEyePosition", "waterColor", "vLevels", "waveData", "windMatrix"],
        ["reflectionSampler", "refractionSampler", "bumpSampler"],
        "");

    if (!this._effect.isReady()) {
        return false;
    }

    return true;
};

The engine object has a function called createEffect that you can use to compile/link your shaders into a simple object. This function has the following parameters:

  • An array of attributes describing the topology of your vertices
  • An array of uniforms (the external variables) defined by the shaders
  • An array of samplers (the objects used to read textures)
  • An optional define string

The createEffect has an internal cache in order to compile/link your shaders only once (the subsequent calls return directly the cached effect).

Once the effect is created we can use it to transfer values to the shaders within the bind function:

WaterMaterial.prototype.bind = function (world, mesh) {
    this._time += 0.0001 * this._scene.getAnimationRatio();

    this._effect.setMatrix("world", world);
    this._effect.setMatrix("worldViewProjection", world.multiply(this._scene.getTransformMatrix()));
    this._effect.setVector3("vEyePosition", this._scene.activeCamera.position);
    this._effect.setVector3("vLightPosition", this.light.position);
    this._effect.setColor3("waterColor", this.waterColor);
    this._effect.setFloat4("vLevels", this.waterColorLevel, this.fresnelLevel, this.reflectionLevel, this.refractionLevel);
    this._effect.setFloat2("waveData", this.waveLength, this.waveHeight);

    // Textures        
    this._effect.setMatrix("windMatrix", this.bumpTexture._computeTextureMatrix().multiply(
                               BABYLON.Matrix.Translation(this.waterDirection.x * this._time, this.waterDirection.y * this._time, 0)));
    this._effect.setTexture("bumpSampler", this.bumpTexture);
    this._effect.setTexture("reflectionSampler", this.reflectionTexture);
    this._effect.setTexture("refractionSampler", this.refractionTexture);
};

Please note that I use the animated __time_ variable to generate the windMatrix variable in order to simulate waves movement.

Finally do not forget to clean things up when leaving.

WaterMaterial.prototype.dispose = function () {
    if (this.bumpTexture) {
        this.bumpTexture.dispose();
    }

    if (this.groundTexture) {
        this.groundTexture.dispose();
    }

    if (this.snowTexture) {
        this.snowTexture.dispose();
    }
    this.baseDispose();
};

Using our material

Now we have a specific material to simulate the water, we can go back to our HTML page. We have to attach the WaterMaterial to the water object and specify which meshes are used by the reflection and the refraction textures:

// Water
BABYLON.Engine.ShadersRepository = "";
var water = BABYLON.Mesh.CreateGround("water", 1000, 1000, 1, scene, false);
var waterMaterial = new WaterMaterial("water", scene, sun);
waterMaterial.refractionTexture.renderList.push(extraGround);
waterMaterial.refractionTexture.renderList.push(ground);

waterMaterial.reflectionTexture.renderList.push(ground);
waterMaterial.reflectionTexture.renderList.push(skybox);

water.material = waterMaterial;

One important thing to note here: The BABYLON.Engine.ShadersRepository is used to specify the path to shaders’ folder. For this demo I did not use one so we have to set it to empty string.

You are now ready to see your wonderful world with its so cute moving water.

Going further

If you want to go more deeply into babylon.js, here are some useful links: