Learn Creative Coding (#76) - 3D Interaction: Raycasting and Picking

Every 3D scene we've built since episode 62 has been passive. You can orbit the camera and look around, but you can't reach into the scene and touch anything. Objects sit there, doing their thing, completely indifferent to you. The environments from last episode look gorgeous but they're like a painting behind glass -- you observe but never participate.

This episode breaks the glass. We're adding interaction -- the ability to detect what the mouse is pointing at in 3D space, highlight objects on hover, click to trigger actions, drag objects around, and build scenes that respond to you. The core technique is raycasting: shooting an invisible ray from your mouse position into the 3D world and seeing what it hits. Once you have that, everything else is just deciding what happens when the ray intersects something.

Raycasting connects 2D screen space (where your mouse lives) to 3D world space (where your objects live). It's the bridge between you and the scene. And it's simpler than you'd think -- Three.js handles all the projection math. We just need to set it up and decide what to do with the results.

Mouse coordinates to normalized device coordinates

Before we can cast a ray, we need to convert mouse pixel coordinates into the format Three.js expects. The mouse gives us pixels from the top-left corner of the window. Three.js raycasting wants Normalized Device Coordinates (NDC) -- a range from -1 to +1 on both axes, with (0, 0) at the center of the screen, (-1, -1) at the bottom-left, and (1, 1) at the top-right.

const mouse = new THREE.Vector2();

window.addEventListener('mousemove', function (event) {
  // convert pixel coords to NDC
  mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
  mouse.y = -(event.clientY / window.innerHeight) * 2 + 1;
});

The X conversion: divide by window width to get 0..1, multiply by 2 to get 0..2, subtract 1 to get -1..+1. Left edge = -1, right edge = +1.

The Y conversion is the same but negated because screen Y goes downward (0 at top) while NDC Y goes upward (-1 at bottom, +1 at top). That minus sign in front is easy to forget and if you do, hover detection will be vertically flipped -- you'll highlight things on the opposite side of where you're pointing. Ask me how I know. :-)

The Raycaster

Three.js has a built-in Raycaster class that does all the heavy lifting. You give it the camera and the NDC mouse position, and it shoots a ray from the camera through that screen point into the scene. Then you ask it what objects the ray hit:

import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';

const scene = new THREE.Scene();
scene.background = new THREE.Color(0x0a0a14);

const camera = new THREE.PerspectiveCamera(
  60, window.innerWidth / window.innerHeight, 0.1, 100
);
camera.position.set(0, 3, 8);

const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setPixelRatio(window.devicePixelRatio);
document.body.appendChild(renderer.domElement);

const controls = new OrbitControls(camera, renderer.domElement);
controls.enableDamping = true;

scene.add(new THREE.AmbientLight(0x334455, 0.6));
const sun = new THREE.DirectionalLight(0xffeedd, 1.8);
sun.position.set(5, 8, 4);
scene.add(sun);

// create some objects to interact with
const objects = [];

for (let i = 0; i < 20; i++) {
  const geo = Math.random() > 0.5
    ? new THREE.BoxGeometry(0.6, 0.6, 0.6)
    : new THREE.SphereGeometry(0.35, 20, 20);

  const mat = new THREE.MeshStandardMaterial({
    color: new THREE.Color().setHSL(Math.random(), 0.6, 0.45),
    roughness: 0.4,
    metalness: 0.1
  });

  const mesh = new THREE.Mesh(geo, mat);
  mesh.position.set(
    (Math.random() - 0.5) * 8,
    (Math.random() - 0.5) * 4 + 2,
    (Math.random() - 0.5) * 6
  );
  scene.add(mesh);
  objects.push(mesh);
}

// raycaster
const raycaster = new THREE.Raycaster();
const mouse = new THREE.Vector2();

window.addEventListener('mousemove', function (event) {
  mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
  mouse.y = -(event.clientY / window.innerHeight) * 2 + 1;
});

function animate() {
  requestAnimationFrame(animate);

  // update raycaster from camera and mouse position
  raycaster.setFromCamera(mouse, camera);

  // check for intersections with our objects
  const intersects = raycaster.intersectObjects(objects);

  // for now, just log what we hit
  if (intersects.length > 0) {
    console.log('hit:', intersects[0].object.uuid, 'at distance', intersects[0].distance);
  }

  controls.update();
  renderer.render(scene, camera);
}

animate();

raycaster.setFromCamera(mouse, camera) sets up the ray origin and direction based on the camera's projection and the mouse NDC coords. For a perspective camera, the ray starts at the camera position and goes through the corresponding point on the near plane. For an orthographic camera, the ray starts at the mouse's position on the near plane and goes straight forward.

raycaster.intersectObjects(objects) returns an array of intersection results, sorted by distance (closest first). Each result contains:

• object -- the mesh that was hit
• point -- the exact 3D world-space coordinates of the hit
• distance -- distance from the ray origin to the hit point
• face -- which triangle face was hit (has normal vector)
• faceIndex -- index of the hit face in the geometry
• uv -- UV coordinates at the hit point (useful for texture sampling)

If nothing was hit, the array is empty. The closest intersection is always intersects[0].

Hover highlighting

The most basic interaction: change an object's appearance when the mouse hovers over it. Reset it when the mouse moves away. This gives visual feedback that something is interactive -- that affordance of "this thing responds to you."

let hoveredObject = null;
const originalColors = new Map();

function animate() {
  requestAnimationFrame(animate);

  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(objects);

  // reset previous hover
  if (hoveredObject && (!intersects.length || intersects[0].object !== hoveredObject)) {
    hoveredObject.material.emissive.set(0x000000);
    hoveredObject.scale.set(1, 1, 1);
    hoveredObject = null;
  }

  // apply new hover
  if (intersects.length > 0) {
    const hit = intersects[0].object;
    if (hit !== hoveredObject) {
      hoveredObject = hit;
      hoveredObject.material.emissive.set(0x333333);
      hoveredObject.scale.set(1.15, 1.15, 1.15);
    }
  }

  // change cursor
  document.body.style.cursor = hoveredObject ? 'pointer' : 'default';

  controls.update();
  renderer.render(scene, camera);
}

The emissive color adds a glow effect -- the object looks lit from within. Scaling it up slightly makes it "pop" toward the viewer. Together these two effects clearly communicate "I see your mouse, I'm interactive." The cursor change to pointer reinforces that this is something you can click.

The key pattern here is tracking the previously hovered object so you can reset it. Without that tracking, you'd either leave old objects glowing (everything you touch stays highlighted) or you'd reset ALL objects every frame (works but wastes cycles iterating the whole array).

One subtlety: material.emissive modifies the material directly. If multiple objects share the same material instance, hovering one would highlight them all. The code above creates a new material per mesh so this isn't an issue, but if you're optimizing by sharing materials, either clone the material on hover or use a different highlighting approach (like OutlinePass from post-processing, which we covered in ep069).

Click interaction

Clicking takes the hover pattern and adds an action. Raycast on the click event, check what was hit, do something:

window.addEventListener('click', function () {
  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(objects);

  if (intersects.length > 0) {
    const hit = intersects[0].object;
    const hitPoint = intersects[0].point;

    // change color on click
    hit.material.color.setHSL(Math.random(), 0.7, 0.5);

    // animate: spin the clicked object
    const startRotation = hit.rotation.y;
    const targetRotation = startRotation + Math.PI * 2;
    const startTime = performance.now();

    function spinAnimation() {
      const elapsed = (performance.now() - startTime) / 1000;
      const t = Math.min(elapsed / 0.6, 1);
      const eased = 1 - Math.pow(1 - t, 3);  // ease-out cubic
      hit.rotation.y = startRotation + (targetRotation - startRotation) * eased;

      if (t < 1) requestAnimationFrame(spinAnimation);
    }
    spinAnimation();
  }
});

Click an object, it changes color and does a smooth 360 spin with an ease-out curve. The animation is self-contained -- it runs via its own requestAnimationFrame chain and stops when t reaches 1. This means you can click multiple objects and they all spin independently without blocking each other. Simple but effective.

For more dramatic click effects, you could explode the object into particles (grab vertex positions from the geometry, spawn particles at each position, apply outward velocity), play a sound (Web Audio API), remove the object from the scene, replace it with a different mesh, or trigger a physics impulse if you're using Cannon.js from episode 73. The click event is just the trigger -- what you do with it is the creative part.

3D cursor: a sphere that follows the surface

Instead of just highlighting the hovered object, we can render a small sphere at the exact hit point. A 3D cursor that hugs the surface of whatever the mouse points at. This gives spatial awareness -- you can see exactly where in 3D space your mouse is pointing.

const cursorGeo = new THREE.SphereGeometry(0.08, 16, 16);
const cursorMat = new THREE.MeshBasicMaterial({
  color: 0x00ffaa,
  transparent: true,
  opacity: 0.7
});
const cursor3D = new THREE.Mesh(cursorGeo, cursorMat);
cursor3D.visible = false;
scene.add(cursor3D);

// a ring around the cursor for better visibility
const ringGeo = new THREE.RingGeometry(0.12, 0.16, 24);
const ringMat = new THREE.MeshBasicMaterial({
  color: 0x00ffaa,
  side: THREE.DoubleSide,
  transparent: true,
  opacity: 0.5
});
const cursorRing = new THREE.Mesh(ringGeo, ringMat);
cursor3D.add(cursorRing);

function animate() {
  requestAnimationFrame(animate);

  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(objects);

  if (intersects.length > 0) {
    const hit = intersects[0];
    cursor3D.position.copy(hit.point);

    // orient the ring to face the surface normal
    if (hit.face) {
      const normal = hit.face.normal.clone();
      normal.transformDirection(hit.object.matrixWorld);
      cursorRing.lookAt(
        cursor3D.position.x + normal.x,
        cursor3D.position.y + normal.y,
        cursor3D.position.z + normal.z
      );
    }

    cursor3D.visible = true;
  } else {
    cursor3D.visible = false;
  }

  controls.update();
  renderer.render(scene, camera);
}

The cursor sphere sits exactly at the intersection point. The ring orients itself to the surface normal -- so it lays flat against whatever surface you're pointing at. On top of a box it's horizontal. On the side of a box it's vertical. On a sphere it tilts to match the curvature. This normal-aligned ring is what makes the cursor feel spatial rather than just a floating dot.

The hit.face.normal is in the object's local space. transformDirection(hit.object.matrixWorld) converts it to world space so the ring faces the correct direction regardless of the object's rotation or position.

Drag interaction

Dragging is the most complex mouse interaction. The pattern: on mousedown, raycast to find what the mouse hit. If something was hit, start dragging. On mousemove, project the mouse position onto a drag plane (a flat plane in 3D space, parallel to the camera, passing through the original hit point). Move the object to wherever the mouse projects onto that plane. On mouseup, stop dragging.

let isDragging = false;
let dragObject = null;
let dragPlane = new THREE.Plane();
let dragOffset = new THREE.Vector3();
let intersection = new THREE.Vector3();

renderer.domElement.addEventListener('mousedown', function (event) {
  if (event.button !== 0) return;  // left click only

  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(objects);

  if (intersects.length > 0) {
    isDragging = true;
    dragObject = intersects[0].object;

    // create a drag plane: facing the camera, passing through the hit point
    dragPlane.setFromNormalAndCoplanarPoint(
      camera.getWorldDirection(new THREE.Vector3()).negate(),
      intersects[0].point
    );

    // offset: difference between hit point and object center
    // so the object doesn't snap its center to the mouse
    raycaster.ray.intersectPlane(dragPlane, intersection);
    dragOffset.subVectors(dragObject.position, intersection);

    controls.enabled = false;  // disable orbit while dragging
  }
});

renderer.domElement.addEventListener('mousemove', function () {
  if (!isDragging || !dragObject) return;

  raycaster.setFromCamera(mouse, camera);

  if (raycaster.ray.intersectPlane(dragPlane, intersection)) {
    dragObject.position.copy(intersection).add(dragOffset);
  }
});

renderer.domElement.addEventListener('mouseup', function () {
  if (isDragging) {
    isDragging = false;
    dragObject = null;
    controls.enabled = true;  // re-enable orbit
  }
});

The drag plane is critical. Without it, you'd need to decide what depth to move the object to -- the mouse gives you X and Y but not Z. The plane solves this by saying "the object moves on this flat surface in 3D space." By making the plane face the camera (normal = negated camera direction) and pass through the initial hit point, dragging feels natural -- the object stays at roughly the same visual depth and follows the mouse laterally.

The dragOffset prevents the object from snapping its center to the mouse cursor on grab. If you grab the edge of a box, the edge stays under the mouse, not the center. Without this offset, every grab would teleport the object's center to the cursor, which fels jarring.

Disabling OrbitControls during drag is important. Without it, the left mouse button would try to both drag the object and orbit the camera simultaneously, which is chaos. Enable it again on mouseup so you can still look around between drags.

Transform controls: design tool handles

For more precise manipulation, Three.js provides TransformControls -- the 3D equivalent of Figma's transform handles. Colored axis arrows for translation, circles for rotation, boxes for scaling. The same UI paradigm every 3D editor uses (Blender, Unity, etc.).

import { TransformControls } from 'three/addons/controls/TransformControls.js';

const transformControls = new TransformControls(camera, renderer.domElement);
scene.add(transformControls);

// disable orbit when transform is active
transformControls.addEventListener('dragging-changed', function (event) {
  controls.enabled = !event.value;
});

// click to select, attach transform controls
window.addEventListener('click', function () {
  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(objects);

  if (intersects.length > 0) {
    transformControls.attach(intersects[0].object);
  } else {
    transformControls.detach();
  }
});

// switch modes with keyboard
window.addEventListener('keydown', function (event) {
  switch (event.key) {
    case 'g': transformControls.setMode('translate'); break;
    case 'r': transformControls.setMode('rotate'); break;
    case 's': transformControls.setMode('scale'); break;
    case 'Escape': transformControls.detach(); break;
  }
});

Click an object to select it. The transform gizmo appears. Drag the colored handles to move/rotate/scale along specific axes. Press G for translate, R for rotate, S for scale (same as Blender shortcuts). Escape to deselect.

This is overkill for most creative coding pieces but very useful for tools and editors. If you're building an interactive scene editor, level designer, or any application where the user arranges objects in 3D space, TransformControls saves you from reimplementing all that gizmo rendering and axis-constrained dragging yourself.

Physics-based interaction: flicking objects

Combine raycasting with the physics engine from episode 73 and you get direct physical manipulation. Click on an object and apply an impulse at the hit point. The object flies off, spinning realistically based on where you hit it. Hit it center-mass and it goes straight. Hit it off-center and it torques into a spin.

// assuming cannon-es physics world from ep073
// and synced objects array with { body, mesh } pairs

window.addEventListener('click', function () {
  raycaster.setFromCamera(mouse, camera);
  const meshes = objects.map(obj => obj.mesh);
  const intersects = raycaster.intersectObjects(meshes);

  if (intersects.length > 0) {
    const hitMesh = intersects[0].object;
    const hitPoint = intersects[0].point;

    // find the physics body for this mesh
    const pair = objects.find(obj => obj.mesh === hitMesh);
    if (!pair) return;

    // impulse direction: from camera toward hit point
    const impulseDir = new THREE.Vector3();
    impulseDir.subVectors(hitPoint, camera.position).normalize();

    // apply impulse at the hit point (not center of mass)
    const force = new CANNON.Vec3(
      impulseDir.x * 8,
      impulseDir.y * 8 + 3,  // add upward kick
      impulseDir.z * 8
    );

    // contact point relative to body center
    const contactPoint = new CANNON.Vec3(
      hitPoint.x - pair.body.position.x,
      hitPoint.y - pair.body.position.y,
      hitPoint.z - pair.body.position.z
    );

    pair.body.applyImpulse(force, contactPoint);
    pair.body.wakeUp();  // in case it was sleeping
  }
});

applyImpulse(force, relativePoint) is the key. The second parameter is the point of application relative to the body's center of mass. If it's at the center (0, 0, 0), you get pure linear acceleration (the object moves without spinning). If it's off-center, the offset creates torque -- the object spins proportionally to how far from center you hit it. This is exactly how real physics works: kick a ball in the center and it goes straight, kick it off the side and it spins.

The upward + 3 on the Y component makes objects arc upward when clicked instead of just sliding along the ground. It makes the interaction feel more punchy and satisfying.

Proximity interaction: the camera as cursor

Mouse raycasting isn't the only interaction model. Another approach: make objects respond to the camera's proximity. When you move close to an object, it reacts -- glows brighter, moves away, changes color, makes a sound. This creates an exploratory, discovery-based experience where the viewer affects the scene just by moving through it.

const interactionRadius = 3.0;

function animate() {
  requestAnimationFrame(animate);

  for (const mesh of objects) {
    const dist = camera.position.distanceTo(mesh.position);

    if (dist < interactionRadius) {
      // proximity factor: 1 at contact, 0 at edge of radius
      const proximity = 1.0 - (dist / interactionRadius);

      // glow based on proximity
      mesh.material.emissive.setHSL(0.55, 0.6, proximity * 0.3);

      // subtle repulsion: push away from camera
      const away = new THREE.Vector3();
      away.subVectors(mesh.position, camera.position).normalize();
      mesh.position.addScaledVector(away, proximity * 0.02);

      // scale pulse
      const pulse = 1.0 + proximity * 0.15 * Math.sin(performance.now() * 0.005);
      mesh.scale.setScalar(pulse);
    } else {
      // reset when out of range
      mesh.material.emissive.set(0x000000);
      mesh.scale.setScalar(1.0);
    }
  }

  controls.update();
  renderer.render(scene, camera);
}

Objects near the camera glow, pulse, and gently drift away. The closer you get, the stronger the effect. Move away and they calm down and return to neutral. It's like the objects are shy -- they notice you approaching and react. Combined with the environments from last episode (fog, atmosphere, ambient particles), this creates scenes that feel alive and aware of the viewer.

This pattern works especially well with OrbitControls or first-person camera controls where the viewer is actively moving through the scene rather than just looking at it from a fixed position.

Touch and mobile

Everything we've done with mouse events translates directly to touch. Touch events give you screen coordinates just like mouse events, and you convert them to NDC the same way:

renderer.domElement.addEventListener('touchstart', function (event) {
  event.preventDefault();
  const touch = event.touches[0];
  mouse.x = (touch.clientX / window.innerWidth) * 2 - 1;
  mouse.y = -(touch.clientY / window.innerHeight) * 2 + 1;

  // raycast on touch
  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(objects);

  if (intersects.length > 0) {
    handleObjectTap(intersects[0]);
  }
}, { passive: false });

renderer.domElement.addEventListener('touchmove', function (event) {
  event.preventDefault();
  const touch = event.touches[0];
  mouse.x = (touch.clientX / window.innerWidth) * 2 - 1;
  mouse.y = -(touch.clientY / window.innerHeight) * 2 + 1;
}, { passive: false });

event.touches[0] gives the first finger. For multi-touch (two-finger rotate, pinch zoom), OrbitControls already handles that by default. You can also access event.touches[1] for the second finger and compute pinch distance or rotation angle between them for custom gestures.

The { passive: false } and event.preventDefault() are important -- without them the browser will try to scroll the page when you drag on the canvas, which fights with your 3D interaction. On mobile, this conflict between page scrolling and canvas interaction is one of the most common annoyances.

For mobile devices with gyroscopes, Three.js has DeviceOrientationControls that maps the phone's physical orientation to the camera direction. Tilt the phone to look around. Combined with touch-tap raycasting, you get a sort of poor-man's AR experience where you look around by moving the device and interact by tapping.

Creative exercise: interactive 3D gallery

Allez, time to build something. An interactive gallery: floating geometric objects arranged in a loose cloud. Hover to see them glow and pulse. Click to explode them into particles that drift outward, then reform into a new random shape at a new position. Drag to rearrange. A playful 3D sandbox where every object invites interaction.

import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';

const scene = new THREE.Scene();
scene.background = new THREE.Color(0x080812);
scene.fog = new THREE.FogExp2(0x080812, 0.035);

const camera = new THREE.PerspectiveCamera(
  65, window.innerWidth / window.innerHeight, 0.1, 100
);
camera.position.set(0, 2, 12);

const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setPixelRatio(window.devicePixelRatio);
document.body.appendChild(renderer.domElement);

const controls = new OrbitControls(camera, renderer.domElement);
controls.enableDamping = true;

scene.add(new THREE.AmbientLight(0x223344, 0.5));
const keyLight = new THREE.DirectionalLight(0xeeddcc, 1.6);
keyLight.position.set(5, 8, 4);
scene.add(keyLight);

const pointLight = new THREE.PointLight(0x4488ff, 1.0, 20);
pointLight.position.set(-4, 3, -2);
scene.add(pointLight);

// --- gallery objects ---
const galleryObjects = [];
const geometries = [
  () => new THREE.BoxGeometry(0.7, 0.7, 0.7),
  () => new THREE.SphereGeometry(0.4, 24, 24),
  () => new THREE.OctahedronGeometry(0.45),
  () => new THREE.TorusGeometry(0.35, 0.12, 12, 24),
  () => new THREE.IcosahedronGeometry(0.42),
  () => new THREE.TetrahedronGeometry(0.5),
  () => new THREE.ConeGeometry(0.35, 0.7, 16),
  () => new THREE.DodecahedronGeometry(0.4)
];

function createGalleryObject(position) {
  const geoFn = geometries[Math.floor(Math.random() * geometries.length)];
  const hue = Math.random();
  const mesh = new THREE.Mesh(
    geoFn(),
    new THREE.MeshStandardMaterial({
      color: new THREE.Color().setHSL(hue, 0.55, 0.4),
      roughness: 0.35,
      metalness: 0.15,
      emissive: new THREE.Color(0x000000)
    })
  );
  mesh.position.copy(position);
  mesh.userData.baseHue = hue;
  mesh.userData.rotSpeed = (Math.random() - 0.5) * 0.3;
  mesh.userData.bobPhase = Math.random() * Math.PI * 2;
  mesh.userData.bobSpeed = 0.3 + Math.random() * 0.4;
  mesh.userData.baseY = position.y;
  scene.add(mesh);
  galleryObjects.push(mesh);
  return mesh;
}

// place objects in a loose sphere arrangement
for (let i = 0; i < 25; i++) {
  const theta = Math.random() * Math.PI * 2;
  const phi = Math.acos(2 * Math.random() - 1);
  const r = 3 + Math.random() * 4;
  const pos = new THREE.Vector3(
    r * Math.sin(phi) * Math.cos(theta),
    r * Math.sin(phi) * Math.sin(theta) * 0.6 + 2,
    r * Math.cos(phi)
  );
  createGalleryObject(pos);
}

// --- raycasting ---
const raycaster = new THREE.Raycaster();
const mouse = new THREE.Vector2();
let hoveredObj = null;

window.addEventListener('mousemove', function (event) {
  mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
  mouse.y = -(event.clientY / window.innerHeight) * 2 + 1;
});

// --- particle explosion system ---
const particleSystems = [];

function explodeObject(mesh) {
  const positions = mesh.geometry.attributes.position;
  const count = Math.min(positions.count, 200);
  const pPositions = new Float32Array(count * 3);
  const pVelocities = [];

  const worldPos = new THREE.Vector3();
  mesh.getWorldPosition(worldPos);

  for (let i = 0; i < count; i++) {
    const vx = positions.getX(i);
    const vy = positions.getY(i);
    const vz = positions.getZ(i);

    // transform vertex to world space
    const v = new THREE.Vector3(vx, vy, vz);
    v.applyMatrix4(mesh.matrixWorld);

    pPositions[i * 3] = v.x;
    pPositions[i * 3 + 1] = v.y;
    pPositions[i * 3 + 2] = v.z;

    // outward velocity from mesh center
    const dir = new THREE.Vector3().subVectors(v, worldPos).normalize();
    pVelocities.push({
      x: dir.x * (1 + Math.random() * 2),
      y: dir.y * (1 + Math.random() * 2) + 1,
      z: dir.z * (1 + Math.random() * 2)
    });
  }

  const pGeo = new THREE.BufferGeometry();
  pGeo.setAttribute('position', new THREE.BufferAttribute(pPositions, 3));

  const pMat = new THREE.PointsMaterial({
    color: mesh.material.color.clone(),
    size: 0.06,
    transparent: true,
    opacity: 1.0,
    sizeAttenuation: true,
    depthWrite: false,
    blending: THREE.AdditiveBlending
  });

  const points = new THREE.Points(pGeo, pMat);
  scene.add(points);

  particleSystems.push({
    points,
    velocities: pVelocities,
    age: 0,
    maxAge: 2.0
  });
}

function updateParticles(delta) {
  for (let i = particleSystems.length - 1; i >= 0; i--) {
    const ps = particleSystems[i];
    ps.age += delta;

    if (ps.age >= ps.maxAge) {
      scene.remove(ps.points);
      ps.points.geometry.dispose();
      ps.points.material.dispose();
      particleSystems.splice(i, 1);
      continue;
    }

    const positions = ps.points.geometry.attributes.position.array;
    const life = 1.0 - (ps.age / ps.maxAge);

    for (let j = 0; j < ps.velocities.length; j++) {
      const j3 = j * 3;
      const v = ps.velocities[j];

      positions[j3] += v.x * delta;
      positions[j3 + 1] += v.y * delta;
      positions[j3 + 2] += v.z * delta;

      // gravity
      v.y -= 2.0 * delta;

      // drag
      v.x *= 0.98;
      v.z *= 0.98;
    }

    ps.points.geometry.attributes.position.needsUpdate = true;
    ps.points.material.opacity = life;
  }
}

// --- click: explode and respawn ---
window.addEventListener('click', function () {
  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(galleryObjects);

  if (intersects.length > 0) {
    const hit = intersects[0].object;
    const idx = galleryObjects.indexOf(hit);
    if (idx === -1) return;

    // explode
    explodeObject(hit);

    // remove
    scene.remove(hit);
    hit.geometry.dispose();
    hit.material.dispose();
    galleryObjects.splice(idx, 1);

    // respawn at a new random position after a delay
    setTimeout(function () {
      const theta = Math.random() * Math.PI * 2;
      const phi = Math.acos(2 * Math.random() - 1);
      const r = 3 + Math.random() * 4;
      const pos = new THREE.Vector3(
        r * Math.sin(phi) * Math.cos(theta),
        r * Math.sin(phi) * Math.sin(theta) * 0.6 + 2,
        r * Math.cos(phi)
      );
      createGalleryObject(pos);
    }, 2000);
  }
});

// --- drag ---
let isDragging = false;
let dragTarget = null;
const dragPlane = new THREE.Plane();
const dragOffset = new THREE.Vector3();
const dragIntersection = new THREE.Vector3();

renderer.domElement.addEventListener('mousedown', function (event) {
  if (event.button !== 0) return;

  raycaster.setFromCamera(mouse, camera);
  const intersects = raycaster.intersectObjects(galleryObjects);

  if (intersects.length > 0) {
    isDragging = true;
    dragTarget = intersects[0].object;

    dragPlane.setFromNormalAndCoplanarPoint(
      camera.getWorldDirection(new THREE.Vector3()).negate(),
      intersects[0].point
    );

    raycaster.ray.intersectPlane(dragPlane, dragIntersection);
    dragOffset.subVectors(dragTarget.position, dragIntersection);
    controls.enabled = false;
  }
});

renderer.domElement.addEventListener('mousemove', function () {
  if (!isDragging || !dragTarget) return;

  raycaster.setFromCamera(mouse, camera);
  if (raycaster.ray.intersectPlane(dragPlane, dragIntersection)) {
    dragTarget.position.copy(dragIntersection).add(dragOffset);
    dragTarget.userData.baseY = dragTarget.position.y;
  }
});

renderer.domElement.addEventListener('mouseup', function () {
  if (isDragging) {
    isDragging = false;
    dragTarget = null;
    controls.enabled = true;
  }
});

// --- animation loop ---
const clock = new THREE.Clock();

function animate() {
  requestAnimationFrame(animate);
  const delta = clock.getDelta();
  const elapsed = clock.getElapsedTime();

  // hover detection
  if (!isDragging) {
    raycaster.setFromCamera(mouse, camera);
    const intersects = raycaster.intersectObjects(galleryObjects);

    if (hoveredObj && (!intersects.length || intersects[0].object !== hoveredObj)) {
      hoveredObj.material.emissive.set(0x000000);
      hoveredObj = null;
    }

    if (intersects.length > 0 && intersects[0].object !== hoveredObj) {
      hoveredObj = intersects[0].object;
      hoveredObj.material.emissive.setHSL(hoveredObj.userData.baseHue, 0.4, 0.15);
    }

    document.body.style.cursor = hoveredObj ? 'pointer' : 'default';
  }

  // animate gallery objects: gentle rotation and bobbing
  for (const obj of galleryObjects) {
    obj.rotation.y += obj.userData.rotSpeed * delta;
    obj.position.y = obj.userData.baseY +
      Math.sin(elapsed * obj.userData.bobSpeed + obj.userData.bobPhase) * 0.15;
  }

  // update particles
  updateParticles(delta);

  controls.update();
  renderer.render(scene, camera);
}

animate();

window.addEventListener('resize', function () {
  camera.aspect = window.innerWidth / window.innerHeight;
  camera.updateProjectionMatrix();
  renderer.setSize(window.innerWidth, window.innerHeight);
});

25 floating geometric objects, each gently rotating and bobbing. Hover to see them glow in their own color. Click to explode into particles that fly outward, fade, and disappear -- then a new object spawns at a random position 2 seconds later. Drag to rearrange them however you want. The fog adds atmospheric depth. The dual lighting (directional + blue point light) gives the scene a moody gallery feel.

This is the pattern for interactive 3D creative coding pieces: raycasting for detection, visual feedback for hover, action on click, drag for direct manipulation. The same building blocks work whether you're making an art installation, a game, a data visualization, or a scene editor. The interaction model is the same -- only what you do in response to the interaction changes.

Performance notes

Raycasting every frame against every object in the scene can get expensive. Some things to watch for:

intersectObjects checks every object in the array. For scenes with hundreds of objects, filter the array to only include objects that should be interactive. Don't raycast against the ground plane, particles, UI elements, or decorative objects that aren't meant to be clicked.

Complex geometries are slower to raycast against. Each triangle in the mesh needs to be tested for intersection. A sphere with 64x64 segments has ~8000 triangles. A simple SphereGeometry(0.5, 12, 12) has ~288 triangles and raycasts 28x faster. Use simplified collision geometries if you need to raycast against complex meshes -- create an invisible low-poly version and raycast against that instead.

Raycaster has a .far property (default Infinity). Setting it to a reasonable distance (like 50 or 100) skips objects beyond that range, which matters in large scenes.

The intersectObjects second parameter recursive (default false) controls whether it descends into child objects of Group nodes. Set it to true if your interactive objects are nested inside groups.

// performance: limit ray distance and use a filtered list
raycaster.far = 50;
const interactiveObjects = objects.filter(obj => obj.userData.interactive);
const intersects = raycaster.intersectObjects(interactiveObjects);

For really large scenes (thousands of objects), consider spatial partitioning: divide the world into a grid or octree and only raycast against objects in the cells the ray passes through. Three.js doesn't include an octree raycaster out of the box, but the three-mesh-bvh library adds a BVH (Bounding Volume Hierarchy) accelerator that makes raycasting against complex geometry orders of magnitude faster.

What's ahead

We've made the leap from passive scenes to interactive ones. Raycasting connects the 2D screen to the 3D world. From here, objects can respond to you -- glow, move, explode, reform, follow rules you define. Interaction is what turns a visualization into an experience.

Next up we're building a complete procedural 3D world as a mini-project, pulling together everything from this arc -- geometry, materials, lighting, post-processing, physics, environments, and now interaction. One self-contained piece that showcases the whole toolkit.

't Komt erop neer...

• Raycasting shoots an invisible ray from the camera through the mouse position into 3D space. THREE.Raycaster handles the projection math. Call raycaster.setFromCamera(mouseNDC, camera) then raycaster.intersectObjects(array) to get sorted hit results with exact world-space point, face normal, UV coords, and distance
• Mouse pixel coordinates need conversion to Normalized Device Coordinates (NDC): x = (clientX / width) * 2 - 1, y = -(clientY / height) * 2 + 1. The Y is negated because screen Y goes down but NDC Y goes up. Forgetting the negation flips hover detection vertically
• Hover highlighting: track the currently hovered object, set material.emissive for glow and scale for visual pop. Reset the previous hovered object when the ray moves to something else. If objects share material instances, hovering one highlights all of them -- clone materials or use post-processing OutlinePass instead
• Click interaction: raycast on the click event, check intersects[0], trigger whatever action you want. Color changes, spin animations, particle explosions, physics impulses, sound triggers -- the click is just the event, the response is the creative part
• 3D cursor: a small sphere placed at intersects[0].point that follows whatever surface the mouse points at. Orient a ring to the face.normal (transformed to world space via transformDirection(matrixWorld)) for a surface-hugging cursor that gives spatial awareness
• Drag pattern: mousedown raycast to find target, create a drag plane (facing camera, through hit point), compute offset from hit point to object center. Mousemove projects onto the drag plane and moves the object. Mouseup releases. Disable OrbitControls during drag to prevent camera interference
• TransformControls from three/addons gives Blender-style translate/rotate/scale handles. transformControls.attach(mesh) to select, .setMode('translate'|'rotate'|'scale') to switch. Good for editor tools, overkill for art pieces
• Physics-based interaction: combine raycasting with Cannon.js. On click, body.applyImpulse(force, relativeContactPoint) at the hit position. Off-center hits create torque (realistic spin). Center hits give pure linear motion. Wake sleeping bodies with body.wakeUp() after impulse
• Proximity interaction: calculate camera.position.distanceTo(mesh.position) each frame. Objects within range glow, pulse, drift away. Creates exploratory scenes where the viewer affects the world by moving through it rather than clicking
• Touch events map to the same raycasting: event.touches[0].clientX/clientY converted to NDC. Use { passive: false } and preventDefault() to stop page scroll from fighting canvas interaction. OrbitControls handles multi-touch natively for pinch zoom and two-finger rotate
• Performance: filter the raycast array to only interactive objects. Set raycaster.far to a reasonable distance. Use low-poly collision proxies for complex meshes. For very large scenes, consider BVH acceleration via three-mesh-bvh

Sallukes! Thanks for reading.

X

Hive account@femdev