Elevation - Milestone 2

Processing user-provided elevation for each terrain vertex - 0.0.0-alpha.5 update

How to Elevate Your Terrain!

It's now possible to make non-flat terrains! This is the first plausibly useful update, although it's still missing some essential functionality, such as the texturing system, and querying the terrain for the purposes of placing objects or physics.

The Elevation Function

With this milestone, it is now possible to provide an elevation function to Hello Terrain. This function is dispatched to the GPU and is called for every vertex. The output should be a simple float representing the elevation at that particular point. This will be used by the graph to create an "elevation-field" (here I use the term elevation instead of heightmap, to distinguish from textures or images containing elevation data).

The elevation field is a buffer of floats that determines how vertices will be offset in the vertex shader, and from that we can derive normals for lighting as well.

When does the Elevation Fn run?

The elevation function is part of hello-terrain's compute graph, which dispatches compute tasks only when their inputs change. The elevation function will be ran when it's upstream dependencies change, which are (broadly):

the quadtreeUpdateTask output changes, which fires when you set quadtreeUpdate on the graph.
configuration params such as maxNodes or rootSize, for example.
your elevationFn itself changes.

You should update the quadtreeUpdate function whenever your camera changes position. However, there is no equality checking inside .set(), so doing this always triggers the quadtree tasks to run. Also, because position floats are noisy, it's likely that position would not be exactly the same anyways.

I recommend adding some hysteresis factor or checking in your animation frame, and only set()-ing the quadtreeUpdate param when there are large changes.

useFrame(async ({ camera, gl }) => {
    const cameraHysteresis = 0.05;
    if (
      lastCameraRef.current.distanceToSquared(camera.position) >=
      cameraHysteresis * cameraHysteresis
    ) {
      g.set(quadtreeUpdate, (prev: UpdateParams) => {
        prev.cameraOrigin.x = camera.position.x;
        prev.cameraOrigin.y = camera.position.y;
        prev.cameraOrigin.z = camera.position.z;
        return prev;
      });
      lastCameraRef.current.copy(camera.position);
    }

    await g.run()
])

This way, if you stay still, the elevationFn will not be re-ran, and the values would be cached.

Simple Procedural Terrain

A simple sin-wave example of an elevation function. It takes the worldPosition as an input to modulate the elevation with a sin wave:

const elevation: ElevationCallback = ({ worldPosition }) => {
  const frequency = float(0.3);
  const pos = vec2(worldPosition.x, worldPosition.z).mul(frequency);
  return pos.x.sin().add(pos.y.sin()).mul(float(0.5));
};

And here's the example integrated in a scene:

import { useRef, useMemo, useEffect } from "react";
import { Canvas, extend, useFrame, useThree } from "@react-three/fiber";
import { OrbitControls } from "@react-three/drei";
import * as THREE from "three/webgpu";
import {
  float, vec2, vec3, vec4, Fn,
  normalize, max, dot, normalWorld,
} from "three/tsl";
import {
  terrainGraph,
  TerrainGeometry,
  TerrainMesh,
  innerTileSegments,
  elevationScale,
  elevationFn,
  quadtreeUpdate,
  quadtreeUpdateTask,
  positionNodeTask,
} from "@hello-terrain/three";
import { task } from "@hello-terrain/work";
import type { ElevationCallback, UpdateParams } from "@hello-terrain/three";

// Extend R3F's catalogue with WebGPU-only and custom classes
extend({
  TerrainGeometry,
  TerrainMesh,
  MeshBasicNodeMaterial: THREE.MeshBasicNodeMaterial,
});

function SceneSetup() {
  const { scene } = useThree();
  useEffect(() => {
    scene.background = new THREE.Color("#292929");
  }, [scene]);
  return null;
}

function Terrain({ graph }) {
  const meshRef = useRef(null);
  const materialRef = useRef(null);
  const materialReadyRef = useRef(false);

  // Define the sin-wave elevation function (runs on GPU via TSL)
  useEffect(() => {
    const elevation: ElevationCallback = ({ worldPosition }) => {
      const frequency = float(0.3);
      const pos = vec2(worldPosition.x, worldPosition.z).mul(frequency);
      return pos.x.sin().add(pos.y.sin()).mul(float(0.5));
    };

    // attach the elevation function to the graph
    graph.set(elevationFn, () => elevation);

    // scale the elevation
    graph.set(elevationScale, () => 5);
  }, [graph]);

  // Apply position node + TSL lighting to the material
  useEffect(() => {
    graph.add(
      task((get, work) => {
        const positionNode = get(positionNodeTask);
        const leafSet = get(quadtreeUpdateTask);
        return work(() => {
          const mesh = meshRef.current;
          const material = materialRef.current;
          if (mesh && leafSet?.count !== undefined) {
            mesh.count = leafSet.count;
          }
          if (material && positionNode) {
            material.positionNode = positionNode;

            // Compute lighting entirely in TSL — bypasses the scene
            // this is only necessary when having multiple sandpack scenes
            if (!materialReadyRef.current) {
              material.outputNode = Fn(() => {
                const baseColor = vec3(0.87, 0.57, 0.29);
                const lightDir = normalize(vec3(0.5, 0.8, 0.3));
                const ambient = float(0.3);
                const diff = max(dot(normalWorld, lightDir), float(0));
                return vec4(
                  baseColor.mul(ambient.add(diff.mul(float(0.7)))),
                  float(1)
                );
              })();
              materialReadyRef.current = true;
            }

            material.needsUpdate = true;
          }
        });
      }).displayName("applyPositionNodeTask"),
    );
  }, [graph]);

  // Update camera position and run the graph each frame
  useFrame(async ({ camera, gl }) => {
    graph.set(quadtreeUpdate, (prev: UpdateParams) => {
      prev.cameraOrigin.x = camera.position.x;
      prev.cameraOrigin.y = camera.position.y;
      prev.cameraOrigin.z = camera.position.z;
      return prev;
    });
    await graph.run({ resources: { renderer: gl } });
  });

  return (
    <terrainMesh
      ref={meshRef}
      innerTileSegments={innerTileSegments.get()}
      maxNodes={1024}
    >
      <meshBasicNodeMaterial ref={materialRef} />
    </terrainMesh>
  );
}

export default function App() {
  const graph = useMemo(() => terrainGraph(), []);

  return (
    <Canvas
      style={{ width: "100vw", height: "100vh" }}
      gl={async (props) => {
        const renderer = new THREE.WebGPURenderer({
          ...props,
          antialias: true,
        });
        await renderer.init();
        return renderer;
      }}
      camera={{ position: [0, 30, 60] }}
    >
      <SceneSetup />
      <Terrain graph={graph} />
      <OrbitControls />
    </Canvas>
  );
}

Open Sandbox

Noise-based Elevation

You can implement any noise function as part of the elevation Fn (encouraged!).

A great resource for noises implemented in TSL is threejsroadmap.com

import { useRef, useMemo, useEffect } from "react";
import { Canvas, extend, useFrame, useThree } from "@react-three/fiber";
import { OrbitControls } from "@react-three/drei";
import * as THREE from "three/webgpu";
import {
  float, vec2, vec3, vec4, Fn, Loop,
  normalize, max, dot, normalWorld,
  floor, fract, mix, sin, cos,
} from "three/tsl";
import {
  terrainGraph,
  TerrainGeometry,
  TerrainMesh,
  innerTileSegments,
  elevationScale,
  elevationFn,
  quadtreeUpdate,
  quadtreeUpdateTask,
  positionNodeTask,
} from "@hello-terrain/three";
import { task } from "@hello-terrain/work";
import type { ElevationCallback, UpdateParams } from "@hello-terrain/three";

extend({
  TerrainGeometry,
  TerrainMesh,
  MeshBasicNodeMaterial: THREE.MeshBasicNodeMaterial,
});

// ── TSL noise helpers ──────────────────────────────────────

// Pseudo-random gradient from a 2D lattice point
const randomGradient = Fn(([p]) => {
  const angle = fract(
    sin(dot(p, vec2(127.1, 311.7))).mul(43758.5453)
  ).mul(Math.PI * 2);
  return vec2(cos(angle), sin(angle));
});

// Classic 2D Perlin noise
const perlinNoise = Fn(([p]) => {
  const i = floor(p).toVar();
  const f = fract(p).toVar();
  const u = f.mul(f).mul(float(3).sub(f.mul(2)));

  const g00 = randomGradient(i);
  const g10 = randomGradient(i.add(vec2(1, 0)));
  const g01 = randomGradient(i.add(vec2(0, 1)));
  const g11 = randomGradient(i.add(vec2(1, 1)));

  const d00 = dot(g00, f);
  const d10 = dot(g10, f.sub(vec2(1, 0)));
  const d01 = dot(g01, f.sub(vec2(0, 1)));
  const d11 = dot(g11, f.sub(vec2(1, 1)));

  return mix(mix(d00, d10, u.x), mix(d01, d11, u.x), u.y).add(0.5);
});

// Fractal Brownian Motion — 6 octaves of Perlin noise using TSL Loop
const fbm = Fn(([pos_immutable]) => {
  const p = vec2(pos_immutable).toVar();
  const total = float(0).toVar();
  const amp = float(0.5).toVar();
  const freq = float(1).toVar();

  Loop(6, () => {
    total.addAssign(perlinNoise(p.mul(freq)).mul(amp));
    freq.mulAssign(2.03);
    amp.mulAssign(0.5);
  });

  return total;
});

// ── Scene ──────────────────────────────────────────────────

function SceneSetup() {
  const { scene } = useThree();
  useEffect(() => {
    scene.background = new THREE.Color("#292929");
  }, [scene]);
  return null;
}

function Terrain({ graph }) {
  const meshRef = useRef(null);
  const materialRef = useRef(null);
  const materialReadyRef = useRef(false);

  useEffect(() => {
    const elevation: ElevationCallback = ({ worldPosition }) => {
      const p = vec2(worldPosition.x, worldPosition.z).mul(float(0.05));
      return fbm(p).sub(float(0.3));
    };

    graph.set(elevationFn, () => elevation);
    graph.set(elevationScale, () => 15);
  }, [graph]);

  useEffect(() => {
    graph.add(
      task((get, work) => {
        const positionNode = get(positionNodeTask);
        const leafSet = get(quadtreeUpdateTask);
        return work(() => {
          const mesh = meshRef.current;
          const material = materialRef.current;
          if (mesh && leafSet?.count !== undefined) {
            mesh.count = leafSet.count;
          }
          if (material && positionNode) {
            material.positionNode = positionNode;

            if (!materialReadyRef.current) {
              material.outputNode = Fn(() => {
                const baseColor = vec3(0.42, 0.55, 0.33);
                const lightDir = normalize(vec3(0.5, 0.8, 0.3));
                const ambient = float(0.3);
                const diff = max(dot(normalWorld, lightDir), float(0));
                return vec4(
                  baseColor.mul(ambient.add(diff.mul(float(0.7)))),
                  float(1)
                );
              })();
              materialReadyRef.current = true;
            }

            material.needsUpdate = true;
          }
        });
      }).displayName("applyPositionNodeTask"),
    );
  }, [graph]);

  useFrame(async ({ camera, gl }) => {
    graph.set(quadtreeUpdate, (prev: UpdateParams) => {
      prev.cameraOrigin.x = camera.position.x;
      prev.cameraOrigin.y = camera.position.y;
      prev.cameraOrigin.z = camera.position.z;
      return prev;
    });
    await graph.run({ resources: { renderer: gl } });
  });

  return (
    <terrainMesh
      ref={meshRef}
      innerTileSegments={innerTileSegments.get()}
      maxNodes={1024}
    >
      <meshBasicNodeMaterial ref={materialRef} />
    </terrainMesh>
  );
}

export default function App() {
  const graph = useMemo(() => terrainGraph(), []);

  return (
    <Canvas
      style={{ width: "100vw", height: "100vh" }}
      gl={async (props) => {
        const renderer = new THREE.WebGPURenderer({
          ...props,
          antialias: true,
        });
        await renderer.init();
        return renderer;
      }}
      camera={{ position: [0, 30, 60] }}
    >
      <SceneSetup />
      <Terrain graph={graph} />
      <OrbitControls />
    </Canvas>
  );
}

Open Sandbox

Elevation Function Inputs

The ElevationCallback will be passed some handy params for you to use in your elevation function. The one you will probably most often use is worldPosition. The others can be situationally useful, or for debugging. Notice these are all simply typed as Node (because I don't know how to strictly type TSL...)


interface ElevationParams {
    worldPosition: Node; // vec3: the vertex position in global coordinates
    rootSize: Node; // int: the root size passed as a param to the graph
    rootUV: Node; // vec2: the 2D xy coordinate of this vertex
    tileUV: Node; // vec2: the 2D xy coordinate inside this terrain tile
    tileLevel: Node; // int: subdivision level of this tile (smaller number is larger / lower resolution)
    tileSize: Node; // float: size or width of the tile's edge
    tileOriginVec2: Node; // vec2:  the 2D xy coordinate of the center of the tile in relation to the root tile
    nodeIndex: Node; // int: the index of the tile in the nodeBuffer
}

Creating terrain from heightmaps

The elevation function API makes it trivial to use heightmaps. Load an image and sample it in your elevation callback using rootUV — which maps [0, 1] across the entire root terrain node. It is recommended to use a 16-bit precision format, such EXR or r16 for height data.

const elevation: ElevationCallback = ({ rootUV }) => {
  return texture(heightmap, rootUV).x;
};

import { useRef, useMemo, useEffect } from "react";
import { Canvas, extend, useFrame, useLoader, useThree } from "@react-three/fiber";
import { OrbitControls } from "@react-three/drei";
import * as THREE from "three/webgpu";
import {
  float, vec2, vec3, vec4, Fn, texture,
  normalize, max, dot, normalWorld,
} from "three/tsl";
import { EXRLoader } from "three/examples/jsm/loaders/EXRLoader.js";
import {
  terrainGraph,
  TerrainGeometry,
  TerrainMesh,
  innerTileSegments,
  elevationScale,
  elevationFn,
  quadtreeUpdate,
  quadtreeUpdateTask,
  positionNodeTask,
} from "@hello-terrain/three";
import { task } from "@hello-terrain/work";
import type { ElevationCallback, UpdateParams } from "@hello-terrain/three";

extend({
  TerrainGeometry,
  TerrainMesh,
  MeshBasicNodeMaterial: THREE.MeshBasicNodeMaterial,
});

// ── Scene ──────────────────────────────────────────────────

function SceneSetup() {
  const { scene } = useThree();
  useEffect(() => {
    scene.background = new THREE.Color("#292929");
  }, [scene]);
  return null;
}

function Terrain({ graph }) {
  const meshRef = useRef(null);
  const materialRef = useRef(null);
  const materialReadyRef = useRef(false);

  // Load the EXR heightmap
  const heightmap = useLoader(
    EXRLoader,
    "https://hello-terrain.kenny.wtf/external/everest-2.exr",
  );
  heightmap.wrapS = heightmap.wrapT = THREE.ClampToEdgeWrapping;
  heightmap.minFilter = THREE.LinearFilter;
  heightmap.magFilter = THREE.LinearFilter;

  // Sample the heightmap in the elevation function using rootUV
  useEffect(() => {
    const elevation: ElevationCallback = ({ rootUV }) => {
      return texture(heightmap, rootUV).x;
    };

    graph.set(elevationFn, () => elevation);
    graph.set(elevationScale, () => 30);
  }, [graph, heightmap]);

  useEffect(() => {
    graph.add(
      task((get, work) => {
        const positionNode = get(positionNodeTask);
        const leafSet = get(quadtreeUpdateTask);
        return work(() => {
          const mesh = meshRef.current;
          const material = materialRef.current;
          if (mesh && leafSet?.count !== undefined) {
            mesh.count = leafSet.count;
          }
          if (material && positionNode) {
            material.positionNode = positionNode;

            if (!materialReadyRef.current) {
              material.outputNode = Fn(() => {
                const baseColor = vec3(0.42, 0.55, 0.33);
                const lightDir = normalize(vec3(0.5, 0.8, 0.3));
                const ambient = float(0.3);
                const diff = max(dot(normalWorld, lightDir), float(0));
                return vec4(
                  baseColor.mul(ambient.add(diff.mul(float(0.7)))),
                  float(1)
                );
              })();
              materialReadyRef.current = true;
            }

            material.needsUpdate = true;
          }
        });
      }).displayName("applyPositionNodeTask"),
    );
  }, [graph]);

  useFrame(async ({ camera, gl }) => {
    graph.set(quadtreeUpdate, (prev: UpdateParams) => {
      prev.cameraOrigin.x = camera.position.x;
      prev.cameraOrigin.y = camera.position.y;
      prev.cameraOrigin.z = camera.position.z;
      return prev;
    });
    await graph.run({ resources: { renderer: gl } });
  });

  return (
    <terrainMesh
      ref={meshRef}
      innerTileSegments={innerTileSegments.get()}
      maxNodes={1024}
    >
      <meshBasicNodeMaterial ref={materialRef} />
    </terrainMesh>
  );
}

export default function App() {
  const graph = useMemo(() => terrainGraph(), []);

  return (
    <Canvas
      style={{ width: "100vw", height: "100vh" }}
      gl={async (props) => {
        const renderer = new THREE.WebGPURenderer({
          ...props,
          antialias: true,
        });
        await renderer.init();
        return renderer;
      }}
      camera={{ position: [0, 30, 60] }}
    >
      <SceneSetup />
      <Terrain graph={graph} />
      <OrbitControls />
    </Canvas>
  );
}

Open Sandbox

Sources

heightmaps generated from https://manticorp.github.io/unrealheightmap locations - Badan Jilin Dessert (巴丹吉林沙漠) and Mt. Everest.
sand texture (with modifications) and night sky from https://ambientcg.com/

Next steps, shortfalls

Currently the mesh that is generated right now has some issues.

Apparent pop-in, which will be later resolved with some form of geomorphing (animating in the LODs)
Non-contiguous normals at the tile edges, which makes lighting change abruptly (to be fixed later with stitching tiles)
Perhaps it will be better to hash the quadtree in someway, that way even if the camera position changes, the downstream tasks will not be asked to run if the quadtree resolves to the same shape.

Elevation - Milestone 2

On this page