experimenting with MindAR

So, Web AR. Sounds like the future, right? I thought so too. Armed with a shaky understanding of augmented reality, a bunch of tabs open on Stack Overflow, and not nearly enough patience, I decided to dive in. This is a rough-and-tumble account of trying to get AR to work in the browser, or at least, to work somehow.

Spoiler: this isn’t a success story so much as a “here’s what I managed to learn before everything imploded” kind of deal.

My first stop was AR.js. Not because I knew it was the best, but because it was the first thing I stumbled on. I mean, it looked fine! I figured I’d have something working in no time. Spoiler: no time turned into a lot of time.

// Looked promising, didn't deliver
import ARjs from 'ar.js';

const arToolkitSource = new ARjs.Source({
  sourceType: 'webcam'
});

const arToolkitContext = new ARjs.Context({
  cameraParametersUrl: 'camera_para.dat',
  detectionMode: 'mono'
});

I thought this setup would be smooth. It wasn’t. AR.js might work for simpler or older setups, but with SvelteKit? Not so much. I had more troubleshooting tabs open than working AR, so the pivot was inevitable.

I turned to MindAR, which offered a more streamlined approach for pattern and face tracking. And this one? It actually worked—with way less drama.

// Clean, less fiddly
await import('mind-ar/dist/mindar-image-aframe.prod.js');

// Pattern Tracking
<a-scene mindar-image="imageTargetSrc: /targets.mind">
  <a-box position="0 0.5 0" />
</a-scene>

// Face Tracking
<a-scene mindar-face>
  <a-assets>
    <img id="cube-texture" src="/texture.png" crossorigin="anonymous" />
  </a-assets>
  <a-entity mindar-face-target="anchorIndex: 168">
    <a-box src="#cube-texture" />
  </a-entity>
</a-scene>

The setup actually rendered a 3D object on my screen. With MindAR, I could use both pattern and face tracking reliably. And that alone was a victory.

I’ll be honest: I’m not 100% sure why some of this worked, but it did, so here’s what I did to get Web AR running in my browser (sort of).

Basic setup—SvelteKit project, essential dependencies, and fingers crossed.

# Set up a fresh SvelteKit project
npx create-svelte@latest your-ar-project
cd your-ar-project

# AR essentials
npm install mind-ar aframe three @types/three

To test AR on mobile, I first tried SSL certificates with OpenSSL to enable HTTPS. However, this method left me feeling like a monkey with a screwdriver—lots of effort, not a lot of results. Even when HTTPS showed up, it had a strikethrough, and mobile browsers weren’t having it.

Solution: Ngrok. It’s straightforward and doesn’t need certificates. Just install, start up the dev server, and run ngrok in a separate terminal. Voilà—mobile testing without the headache.

// vite.config.ts - For the dev setup
import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vite';

export default defineConfig({
    plugins: [sveltekit()],
    server: {
        host: true,
        port: 5173  // My lucky number now
    }
});

// Mobile testing without SSL
# 1. Install ngrok globally
npm install -g ngrok

# 2. Start the dev server
npm run dev

# 3. In a new terminal
ngrok http 5173

# 4. Use the ngrok-provided HTTPS URL on your phone

Mobile testing with ngrok is super simple and doesn’t even need both devices to be on the same network. Just make sure they both have internet.

You can either track patterns (like images) or track faces. That’s what I figured out, anyway. Here’s what the code looked like, and sometimes it worked.

Pattern tracking seemed straightforward enough with MindAR, but there were issues with flickering. Adding smoothing options didn’t fully solve the flicker, probably due to the low-contrast pattern I used—a gray image on a yellow Post-it Note (note to self: not the best choice).

I din’t manage to figure it out this time round but the whole thing was also not centered? not sure what caused that.

<a-scene
  mindar-image="imageTargetSrc: /targets.mind;
    smoothCount: 20;
    smoothTolerance: 0.5;
    smoothThreshold: 5;"
  embedded
>
  <a-box position="0 0.5 0" src="#texture" />
</a-scene>

Face tracking actually performed way better than I expected. It was faster and more responsive than the solutions I tried with TensorFlow.js or PoseNet. The only downside? This setup only handles one face at a time – I havent explored if multiface is possible.

<a-scene mindar-face
  embedded
  renderer="alpha: true"
  vr-mode-ui="enabled: false"
  device-orientation-permission-ui="enabled: false"
>
  <a-assets>
    <img id="cube-texture" src="/texture.png" crossorigin="anonymous" />
  </a-assets>
  <a-camera position="0 0 0" look-controls="enabled: false"></a-camera>
  <a-entity mindar-face-target="anchorIndex: 168">
    <a-box src="#cube-texture" />
  </a-entity>
</a-scene>

If you’re stuck, here are some things to check (and probably recheck):

• HTTPS enabled? Mobile browsers are serious about this.
• Camera permissions allowed? Gotta click “Allow” to access the AR magic.
• Lighting okay? AR struggles in low light.
• Clean camera lens? Not joking—dust messes with tracking.
• Using the latest browser? Say goodbye to Internet Explorer.
• Patience on hand? (Optional but highly recommended)

So that’s my adventure so far in Web AR. MindAR ultimately saved the project, ngrok saved my sanity, and my Post-it Note pattern didn’t quite save the flicker. If you’re diving in too, hopefully this helps, or at least gives you a better idea of what to expect.

Good luck, and may your AR adventures be less buggy.