Magic pixels is a WebGL 3D library. It was originally developed at SinnerSchrader under the name colorful-pixels; the original repository is no longer online, but the last published version is still on npm. Since I left the company, development continues here.
To be honest, there really is no need to build our own 3D library. There are many of these already out there. This is actually a project by Lea, and she decided building something like that anyway, just in order to learn how all this works.
Vector, Matrix classes, plus Float32Array-backed Mat2, Mat3, Mat4 for transformsBox3 and Sphere, cached per geometry and computed in world space for any subtreeObject3D with position, quaternion/rotation, scale, children; Scene, Mesh and the cameras (PerspectiveCamera, OrthographicCamera) are Object3DsRenderer interface at the "render a scene" level, implemented by WebGL2Renderer, which renders a Scene through a Camera and owns all GPU resources (programs, vertex array objects, buffers, textures)modelMatrix, viewMatrix, projectionMatrix, modelViewMatrix, normalMatrix) set by the rendererAmbientLight, DirectionalLight, PointLight), passed to shaders as built-in uniforms in view spacecreatePbrMaterial): the glTF metallic-roughness model with base colour, metallic-roughness, normal, occlusion and emissive maps, alpha modes and an unlit variantRenderTarget, renderer.render(scene, camera, target)) with a colour texture and a depth renderbuffer or texture, optionally floating point, plus fullscreen post-processing passes with a tone-mapping material (createToneMapMaterial)loadGltf, parseGltf) for .gltf and .glb files: nodes, meshes, PBR materials, textures, cameras, KHR_lights_punctual lights, and quantized, meshopt- and Draco-compressed meshes (static models; animations and skinning are not supported yet)NullRenderer that draws nothing, for testing scene code without a GPUMesh contains a BufferGeometry and a Material,Material is what's a RawShaderMaterial in THREE, it has uniform variables, shader sources per language (material.glsl) and a drawModedrawMode is one of DrawMode.TRIANGLES, DrawMode.POINTS, DrawMode.LINES... (plain strings, no GL constants)BufferGeometry API is also similar to three.jsStopwatch class for timing (like performance.now() but with the possibility to start/stop)mix, clamp)npm install magic-pixels.<canvas> element to your DOMScene, add Meshes to it, and a Cameramagic-pixels requires a WebGL2 context. The built-in shaders are written in GLSL ES 3.00; user-written shaders may use either GLSL ES 1.00 or 3.00.
const canvas = document.querySelector('canvas');
const renderer = new WebGL2Renderer(canvas);
renderer.setSize(innerWidth, innerHeight);
WebGL2Renderer implements the Renderer interface (render, setSize, setPixelRatio, dispose).
Code that only needs to render a scene can depend on the interface. For unit tests of scene code
there is a NullRenderer, which needs no canvas and records the frames it was asked to render:
const renderer = new NullRenderer();
renderer.render(scene, camera);
renderer.lastFrame.meshes; // opaque meshes, in tree order
renderer.lastFrame.transparent; // transparent meshes, sorted back to front
renderer.lastFrame.lights; // visible lights, in tree order
// creates a plane geometry of width 2x2 with 3 width segments and 3 height segments
const planeGeometry = createPlaneGeometry(2, 2, 3, 3);
// creates a box geometry of width 1x1x1 with 3 width segments, height segments and depth segments
const boxGeometry = createBoxGeometry(1, 1, 1, 2, 2, 2);
// create a sphere geometry with 16 rings and 16 sides per ring
const sphereGeometry = createSphereGeometry(1, 1, 16, 16);
A material contains shader sources per shading language (material.glsl = { vertex, fragment }),
a drawMode and a uniforms object. A material is plain data: the renderer compiles one program per
distinct pair of shader sources (shared by every material and mesh using it) and uploads the uniforms
on each draw, skipping values that did not change. Just assign to material.uniforms.time = ... (or mutate a Vector in place) and
render. Replacing material.glsl.fragment recompiles the program on the next render.
The setter for a uniform is chosen from the type declared in the shader, so a Vector works for
vec2 and ivec2 alike. A Texture uniform is uploaded on first use and bound to a texture unit
by the renderer.
The default drawMode is DrawMode.TRIANGLES. Draw modes, texture filters and wrapping modes are
plain strings ('triangles', 'linear', 'repeat', ...) rather than GL constants; the DrawMode,
Filter and Wrapping objects list them. See MDN:drawArrays for what the modes mean.
A material also carries render state, all optional:
transparent (default false) blends with SRC_ALPHA, ONE_MINUS_SRC_ALPHA instead of
overwriting the framebuffer, and turns depth writes off unless depthWrite is set explicitly.
Transparent meshes are drawn after opaque ones, sorted back to front.side (default Side.DOUBLE, no culling) restricts drawing to Side.FRONT or Side.BACK
faces.depthTest / depthWrite (default true) control whether a fragment is discarded by what is
already drawn, and whether it writes its own depth.const material = createShaderMaterial(vertexShader, fragmentShader, {
time: 0,
resolution: [800, 600],
color: Color.fromHex('#ff00ff'),
});
// There are some predefined materials:
// just red
const defaultMaterial = createDefaultMaterial();
// just pink
const basicMaterial = createBasicMaterial('#ff00ff');
// the normals
const normalMaterial = createNormalMaterial();
createPbrMaterial(options) is the lit material: the glTF 2.0 metallic-roughness model, shaded by
the lights in the scene. Every option is a glTF material property of the same name and defaults to
the glTF default. Colour factors are linear RGB in 0..1; base colour and emissive maps should be
created with colorSpace: 'srgb', the other maps hold linear data.
const material = createPbrMaterial({
baseColorFactor: [0.8, 0.2, 0.1, 1],
metallicFactor: 0,
roughnessFactor: 0.4,
baseColorMap: albedoTexture, // or { texture, uv: 1 } to sample the `uv1` attribute
normalMap: normalTexture,
emissiveFactor: [0, 0, 0],
alphaMode: AlphaMode.OPAQUE, // or MASK (with alphaCutoff) or BLEND (sets transparent)
doubleSided: false, // true draws both sides and flips the normal on back faces
});
// factors are ordinary uniforms
material.uniforms.roughnessFactor = 0.9;
Which maps and attributes the material uses is baked into the shader when it is created, so adding
a map means creating a new material. Set vertexColors: true to multiply in a color attribute,
tangents: true when the geometry has a tangent attribute for the normal map (otherwise the
tangent is reconstructed from screen-space derivatives), and unlit: true for base colour only
(KHR_materials_unlit). Materials with the same options share one program.
createBasicMaterial(color) is a flat colour and takes no texture. For an unlit textured surface,
for example one with baked-in lighting, use createPbrMaterial({ unlit: true, baseColorMap }): it
skips lighting entirely and is cheap.
const mesh = new Mesh(geometry, material);
mesh.position.set(0, 0, -5);
const scene = new Scene();
scene.add(mesh);
const camera = new PerspectiveCamera(50, innerWidth / innerHeight, 0.1, 100);
// render:
renderer.render(scene, camera);
The renderer clears color and depth before drawing (renderer.autoClear = false to keep the
previous frame, renderer.setClearColor('#202020') to change the color) and draws with depth
testing enabled.
Scene, Mesh and the cameras extend Object3D. Every object has a position, a rotation and a
scale relative to its parent, and a list of children. The rotation is available both as a
quaternion and as Euler angles in rotation (radians, applied in XYZ order); write to either one
and the other follows on the next update (the quaternion wins if both changed). Before each render,
the renderer walks the tree and computes every object's worldMatrix as
parent.worldMatrix × localMatrix.
A child inherits its parent's transform, so it orbits when the parent rotates:
const planet = new Mesh(sphereGeometry, material);
const moon = new Mesh(smallSphereGeometry, material);
planet.add(moon);
moon.position.set(2, 0, 0);
scene.add(planet);
function frame(dt) {
planet.rotation.y += dt; // spins the planet and carries the moon around
}
To orbit the moon without spinning the planet, put an empty Object3D in between as a pivot:
const pivot = new Object3D();
planet.add(pivot);
pivot.add(moon);
pivot.rotation.y += dt; // only the pivot (and the moon with it) rotates
Other useful bits: object.name is a free label (the glTF loader fills it with node names),
object.visible = false hides an object and its children,
object.traverse(callback) visits a subtree, object.lookAt(target) points the object's +Z axis
(for cameras: the viewing direction, -Z) at a point given in the parent's coordinate system.
To drive localMatrix yourself, set matrixAutoUpdate = false and flag changes with
worldMatrixNeedsUpdate = true.
Quaternions are what glTF stores and what animations interpolate. slerp moves between two
rotations along the shortest arc at constant speed:
const from = Quaternion.fromEuler(new Vector(0, 0, 0));
const to = Quaternion.fromAxisAngle(new Vector(1, 1, 0).normalized, Math.PI);
mesh.quaternion.slerpQuaternions(from, to, t); // t from 0 to 1
Lights are Object3Ds: add them to the scene, move them, parent them, hide them. An
AmbientLight adds a flat colour everywhere, a DirectionalLight shines along its own -Z axis
(aim it with lookAt(), like a camera) and a PointLight radiates from its position with
inverse-square falloff, optionally cut off at a range (0, the default, means no cutoff).
Each has a color (a Color or a hex string, taken as linear RGB) and an intensity.
scene.add(new AmbientLight('#202840', 0.5));
const sun = new DirectionalLight('#fff1d6', 0.9);
sun.position.set(5, 5, 5);
sun.lookAt(new Vector(0, 0, 0));
scene.add(sun);
const bulb = new PointLight('#ffb15e', 6, 10); // colour, intensity, range
bulb.position.set(0, 2, 0);
scene.add(bulb);
createPbrMaterial() reads them; so can any shader, through the built-in uniforms described in
the next section. The
lights example has a complete
Lambert shader.
loadGltf(url) fetches a .gltf or .glb file and returns the scene it describes, built from the
same objects as above: Object3D nodes, one Mesh per primitive with a createPbrMaterial()
material, Textures, PerspectiveCamera/OrthographicCamera and KHR_lights_punctual lights.
Relative URIs in the file are resolved against its URL.
const model = await loadGltf('models/helmet.glb');
scene.add(model.scene);
model.scenes; // every scene of the file
model.nodes; // every node, by glTF index (`node.name` is the glTF name)
model.cameras; // cameras found on nodes
model.lights; // lights found on nodes
model.materials; // every material the loader created
model.textures; // every texture the loader created
model.json; // the parsed document, for anything not mapped
parseGltf(data, options) does the same for data already in memory: the bytes of a .glb, the
text or bytes of a .gltf, or its parsed JSON. Both take options: baseUrl (what relative URIs
resolve against), fetch (defaults to the global one; pass your own to load from memory or to
add headers), loadImage (how images are decoded; defaults to createImageBitmap), meshopt
(for EXT_meshopt_compression) and draco (for KHR_draco_mesh_compression). Both accept
either an already-initialized decoder module, decoded on the main thread - meshopt the shape
of MeshoptDecoder from the meshoptimizer package, draco the result of
createDecoderModule() from draco3d or the CDN build's DracoDecoderModule - or a
{ moduleUrl | decoderPath, workerLimit? } to decode in a pool of Web Workers instead, the way
three.js's DRACOLoader does. Neither decoder is bundled by magic-pixels; a file that uses one
of these extensions without the matching option throws, naming it.
const model = await parseGltf(arrayBuffer, {
baseUrl: 'https://example.com/models/',
});
Supported: .gltf with external, data URI or embedded buffers and images, .glb, interleaved
and sparse accessors, quantized attributes (KHR_mesh_quantization), meshopt- and
Draco-compressed meshes (EXT_meshopt_compression, KHR_draco_mesh_compression), every
metallic-roughness material property and alpha mode, KHR_materials_unlit,
KHR_materials_emissive_strength, KHR_lights_punctual (spot lights load as point lights).
Primitives without normals get flat ones. A file that requires an unsupported extension throws;
an optional unsupported extension, animations, skins and morph targets are skipped with a
console.warn.
Attribute locations are fixed by name, so no layout(location = ...) qualifiers are needed and
one geometry works with any program: position is location 0, normal is 1, uv is 2 and
custom attributes get the next free location in the order the renderer first sees them.
The renderer sets the built-in uniforms modelMatrix, viewMatrix, projectionMatrix,
modelViewMatrix (all mat4) and normalMatrix (mat3, the inverse transpose of the model-view
matrix) for every shader that declares them, unless the material defines a uniform of the same
name. The built-in vertex shader uses modelViewMatrix, projectionMatrix and normalMatrix.
The lights of the frame are built-in uniforms as well, in view space with colours premultiplied
by intensity: ambientLightColor (vec3), directionalLightDirections[] (the direction each
light shines in) and directionalLightColors[] (vec3 arrays) with directionalLightCount
(int), and pointLightPositions[], pointLightColors[] (vec3 arrays), pointLightRanges[]
(float array) with pointLightCount. Declare the arrays as large as your shader can handle;
the renderer pads them, drops lights beyond the size and clamps the counts:
#define MAX_POINT_LIGHTS 4
uniform vec3 pointLightPositions[MAX_POINT_LIGHTS];
uniform vec3 pointLightColors[MAX_POINT_LIGHTS];
uniform int pointLightCount;
#version 300 es
precision highp float;
in vec3 position;
in vec3 normal;
in vec2 uv;
uniform mat4 modelViewMatrix;
uniform mat4 projectionMatrix;
uniform mat3 normalMatrix;
out vec2 vUv;
out vec3 vNormal;
void main() {
vUv = uv;
vNormal = normalMatrix * normal;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
const texture = await Texture.fromImageUrl('image.png', {
minFilter: Filter.LINEAR,
magFilter: Filter.LINEAR,
wrapS: Wrapping.REPEAT,
wrapT: Wrapping.REPEAT,
colorSpace: ColorSpace.SRGB, // decode sRGB to linear on sample; default is 'linear'
});
material.uniforms.map = texture;
// for video or canvas textures, flag the texture after the image changed:
texture.needsUpdate = true;
// decode raw bytes (e.g. an image embedded in a binary file) without an <img>:
const fromBytes = await Texture.fromBlob(blob);
A RenderTarget is a texture the renderer can draw into. Its colorAttachment is a normal Texture,
so a material can sample what was rendered. float: true gives a floating point target that can hold
values above 1 (it needs the EXT_color_buffer_float extension; the renderer throws if the browser
does not have it), depth is 'renderbuffer' (the default), 'texture' or 'none'. Change width
and height to resize it.
A fullscreen pass is a mesh that covers the screen, drawn with a material that reads the target. The tone-mapping material converts linear HDR to sRGB, so the scene is drawn without the sRGB conversion:
const target = new RenderTarget(canvas.width, canvas.height, { float: true });
const material = createPbrMaterial({ linearOutput: true }); // linear colour, no sRGB conversion
// ...
const pass = new Scene();
pass.add(
createFullscreenMesh(
createToneMapMaterial({
map: target.colorAttachment,
exposure: 1, // a uniform, can change every frame
toneMapping: ToneMapping.ACES, // or REINHARD, NONE
})
)
);
renderer.render(scene, camera, target); // into the target
renderer.render(pass, camera); // to the canvas; any camera works
createFullscreenMaterial(fragmentShader, uniforms) builds the material for your own passes; the
fragment shader reads in vec2 vUv (0 to 1 across the screen). A Texture also has width and
height, and Texture.empty(width, height) makes one without pixels.
Write into an attribute's data and flag it; the renderer re-uploads it with bufferSubData
on the next render. Adding or removing attributes or calling setIndex bumps
geometry.version, which makes the renderer rebuild its buffers.
const { position } = geometry.attributes;
position.data[0] += 0.1;
position.needsUpdate = true;
// hint for frequently changing data (DYNAMIC_DRAW):
position.dynamic = true;
A geometry measures its position attribute on demand and caches the result:
geometry.computeBoundingBox(); // Box3 with min/max, also in geometry.boundingBox
geometry.computeBoundingSphere(); // Sphere centred on the box, radius of the farthest vertex
geometry.boundingBox = null; // after editing positions: recompute on next use
For a mesh or a whole subtree in world space, computeBoundingBox(object) unions the transformed
geometry boxes and computeBoundingSphere(object) grows a sphere around them. Framing a camera:
const sphere = computeBoundingSphere(model.scene);
const distance = sphere.radius / Math.sin(((camera.fov / 2) * Math.PI) / 180);
The glTF loader fills geometry.boundingBox from the accessor min/max in the file, so loaded
models are measured without a pass over the vertices.
GPU resources live as long as the renderer, or until you dispose them. Rendering an object again after disposing it recreates its resources.
renderer.dispose(geometry); // buffers and VAO of one geometry
renderer.dispose(material); // the program of one material
renderer.dispose(texture); // one texture
renderer.dispose(target); // framebuffer, renderbuffer and textures of one render target
renderer.dispose(); // everything; loses the context
A camera is an Object3D with a projectionMatrix. Place it like any other object; its
viewMatrix (the inverse of its world matrix) is computed when the scene is rendered. A camera can
also be a child of another object, e.g. a pivot to orbit it around a point.
const camera = new PerspectiveCamera(70, innerWidth / innerHeight, 0.01, 100);
camera.position.set(0, 2, 5);
camera.lookAt(new Vector(0, 0, 0));
// on resize:
camera.aspect = innerWidth / innerHeight;
camera.updateProjectionMatrix();
// parallel projection:
const ortho = new OrthographicCamera(-2, 2, 1, -1, 0.1, 100);
The projection helpers perspective, frustum and ortho (returning a Matrix) and their
Mat4 counterparts (Mat4.perspective(...)) are still available if you want to pass matrices
through your own uniforms.
magic-pixels provide basic vector and matrix arithmetics classes.
You can use the mul method on the Matrix class for matrix multiplication.
const a = new Vector(1, 0, 0);
const b = new Vector(0, 1, 0);
const c = a.cross(b);
const d = a.add(b);
Mat2, Mat3 and Mat4 are fixed-size, Float32Array-backed matrices in column-major order
(ready to be uploaded as uniforms). Their operations work in place and allocate nothing, which is
what the scene graph uses per frame. Matrix remains the general-purpose class; convert with
mat4.toMatrix() and Mat4.fromMatrix(matrix).
// identity matrix
const identity = Mat4.identity();
// translate object in space
const translationMatrix = Mat4.translation(tx, ty, tz);
// rotation matrix, composed in place
const DEG = Math.PI / 180;
const rotationMatrix = Mat4.rotX(30 * DEG)
.multiply(Mat4.rotY(45 * DEG))
.multiply(Mat4.rotZ(-5 * DEG));
// translation × rotation × scale, as used by Object3D; the rotation is a
// Quaternion or Euler XYZ angles
const model = new Mat4().compose(position, quaternion, scale);
const fromEuler = new Mat4().compose(position, rotation, scale);
const rotationOnly = Mat4.rotationFromQuaternion(quaternion);
const inverse = model.clone().invert();
// normal matrix for a model(-view) matrix
const normalMatrix = new Mat3().setNormalMatrix(model);
The color helper converts a hex color string to a GLSL-friendly vec3 or vec4 value.
const color = Color.fromHex('#ff00ff');
// returns a Color with {red = 255, green = 0, blue = 255, alpha = 255}
color.toVec3();
// returns [1, 0, 1]
color.toVec4();
// returns [1, 0, 1, 1]
The examples/ folder holds small self-contained demos, one HTML file each; they are deployed
with the docs at learosema.github.io/magic-pixels/examples.
To run them locally, npm run build:examples copies the current bundle next to them, then serve the
folder with any static file server.
Trigger Warning: these examples can cause sickness to people with motion sensitivities.
Releases are cut in two steps, both driven from GitHub:
main runs release-please, which opens or updates a release pull request with the version bump and CHANGELOG.md, based on Conventional Commits (feat: = minor, fix: = patch, feat!: or a BREAKING CHANGE: footer = major). Merging that PR creates the git tag and the GitHub release.npm stage approve.