This chapter describes the glTF loader while it is being built, so unlike the rest of the book it talks about work that is not done yet. Every step gets its own page once it is implemented; this page is the map. Read Rendering concepts first if terms like linear color, BRDF or metallic-roughness are new. The step pages assume you know them.
glTF 2.0 is a JSON document that describes a scene, plus binary buffers with
the vertex data and image files for the textures. A .gltf file is the JSON
with the buffers and images next to it (or embedded as data URIs); a .glb
file packs the JSON and one binary buffer into a single container.
The JSON is a set of flat arrays that reference each other by index:
| glTF array | What it holds | magic-pixels counterpart |
|---|---|---|
scenes |
which nodes are roots | Scene |
nodes |
the hierarchy: translation, rotation, scale, children | Object3D |
meshes |
a list of primitives, each with attributes, indices, material | one Mesh per primitive |
accessors |
a typed view into a buffer: "600 vec3 floats starting at 0" | BufferAttribute |
bufferViews |
a byte range in a buffer, optionally with a stride | (loader internal) |
buffers |
the raw bytes, by URI or in the .glb |
(loader internal) |
materials |
metallic-roughness parameters and texture references | a Material made by createPbrMaterial() |
textures |
image + sampler pairs | Texture |
images |
URI, data URI or a byte range with a MIME type | the image of a Texture |
samplers |
min/mag filter and wrapping | Filter, Wrapping |
cameras |
perspective or orthographic parameters | PerspectiveCamera, OrthographicCamera |
The mapping is close to one-to-one, which is no accident: the magic-pixels scene graph follows three.js, and three.js and glTF grew up together. What is missing on the magic-pixels side falls into three groups, and those groups are the steps below.
1. Quaternions (done). glTF stores node rotations as quaternions, four numbers that describe a rotation axis and angle without the gimbal lock problems of Euler angles. We could convert quaternion to Euler in the loader, but animations (planned for later) interpolate between quaternions, so the quaternion became the real rotation of an Object3D and the Euler angles are derived from it.
2. Typed vertex attributes (done).
BufferAttribute only held Float32Arrays. glTF stores UVs, colors
and joint indices as bytes and shorts, and a "quantized" file even stores
positions as 16 bit integers to halve the file size. WebGL can read those
directly; it just needs to be told the component type and whether to
normalize integers to the 0..1 range. This step teaches the renderer that.
3. Texture extensions (done). Two things.
Images embedded in a .glb come out as an ImageBitmap, which the
Texture type must accept. And color textures in glTF are stored in
the sRGB color space, so the GPU has to convert them to linear values before
lighting math touches them (see
color spaces).
4. Material render state (done). A
Material is shaders plus uniforms. A glTF material can also say
"blend me with what is behind" (alpha mode BLEND) and "render both sides".
That is GPU state, not a shader concern, so the material gets transparent
and side fields and the renderer sets the blend and cull state per draw.
Transparent meshes also have to be drawn last and back to front, which
changes what prepareScene() returns.
5. Lights (done). A physically based material is meaningless without lights. A light is an Object3D subclass so it can be positioned and parented like everything else, and the renderer passes the visible lights to the shader as uniform arrays in view space, the same way it passes the matrices.
6. PBR material. The glTF "metallic-roughness" material is the heart of the loader and the biggest single step. It is a fragment shader that takes base color, metalness, roughness, a normal map, occlusion and emission and computes reflected light for each light in the scene with a BRDF. The page for this step walks through the shader term by term.
7. Loader core. With everything above in place, the loader is mostly bookkeeping: parse the container, turn accessors into attributes, build the node tree, create materials and textures. It is the largest amount of code but the least new theory.
8. Compression. Mesh compression shrinks files several times over. Quantization comes for free with step 2. Meshopt and Draco need a decoder that we do not bundle; the user passes it in and the loader calls it for the buffer views or primitives that are marked as compressed.
9. Documentation. The README and these pages.
Each step page has the same shape:
gltfpack)
and Draco: the two compression
libraries and their glTF extensions.