Hello from Apple Park!!

I've taken a little break from sharing on socials, but my Vision Pro journey is alive and well over here.

I'm still scrolling around online and rooting for all of you in the visionOS community. Excited for our future 🙏

Excited to share that I'm joining Apple to work on Immersive Analytics!

Big thank you to everyone who's been engaging with my Vision Pro experiments. I'm really grateful to have the opportunity to keep pushing in this direction career-wise 🙏

PSVR2 Sense Controller Input Handling and Haptics in RealityKit PSVR2 Sense Controller Input Handling and Haptics in RealityKit - ImmersiveSenseControllerInputsView.swift

gist.github.com/Matt54/f0bae...

RealityKit experiment #86
(code in the replies)

Here's a simple PSVR2 Sense Controller example that handles the inputs, tracks the aim anchor, and triggers haptics

Logitech Muse Anchor Tracking, Model Spawning, and Haptic Feedback (RealityKit) Logitech Muse Anchor Tracking, Model Spawning, and Haptic Feedback (RealityKit) - ImmersiveStylusSimpleExampleView.swift

gist.github.com/Matt54/553b2...

RealityKit experiment #85
(code in the replies)

I made one more update to my Logitech Muse example to preserve the anchor orientation when spawning the model. Switched to cubes to demonstrate and added a little axis indicator at the aim

Logitech Muse for Apple Vision Pro Tracking, Pressure Sensitive Inputs, and Haptics in RealityKit Logitech Muse for Apple Vision Pro Tracking, Pressure Sensitive Inputs, and Haptics in RealityKit - AppModel.swift

gist.github.com/Matt54/77f73...

RealityKit experiment #84
(code in the replies)

Here's an update to my Logitech Muse example that handles the pressure values for the tip and secondary button

Logitech Muse Laser Pen Scene Measurement (RealityKit & ARKit) Logitech Muse Laser Pen Scene Measurement (RealityKit & ARKit) - ImmersiveLaserPenMeasurementView.swift

gist.github.com/Matt54/ea591...

RealityKit experiment #83
(code in the replies)

This example turns the Logitech Muse into a virtual laser pen for point-to-point measurements. The surface intersection point is determined by raycasting along the stylus' aim-origin vector against the scene reconstruction meshes

Logitech Muse for Apple Vision Pro Tracking, Action Buttons, and Haptics in RealityKit Logitech Muse for Apple Vision Pro Tracking, Action Buttons, and Haptics in RealityKit - AppModel.swift

gist.github.com/Matt54/4b22c...

RealityKit experiment #82
(code in the replies)

Here’s a simple Logitech Muse example that tracks the aim, handles the action buttons, and triggers haptics

PLY to RealityKit with Grid-Based Decimation (Example: HD Skeleton Scan) PLY to RealityKit with Grid-Based Decimation (Example: HD Skeleton Scan) - ModelIOLoader.swift

gist.github.com/Matt54/f001a...

RealityKit experiment #81
(code in the replies)

I found a scan of a human skeleton and wanted to try to visualize it in RealityKit. The PLY file had millions of triangles, so I had to decimate the mesh. It ended up being really satisfying to see the wireframe at different LODs

Thanks Clem! I would love to see that Gaussian Splat example

RealityKit SDF Falling Sand Animation (Metal + LowLevelMesh + Marching Cubes) RealityKit SDF Falling Sand Animation (Metal + LowLevelMesh + Marching Cubes) - EasingCurve.swift

gist.github.com/Matt54/afc92...

RealityKit experiment #80
(gist in the replies)

LowLevelMesh “Falling Sand” dissolution and reassemble effect created with SDF shapes and Marching Cubes. A Metal compute shader uses a spatial sweep to "release" triangles that rigidly start to fall

RealityKit Extruded Text to LowLevelMesh animated by Metal compute shader RealityKit Extruded Text to LowLevelMesh animated by Metal compute shader - ExtrudedTextLowLevelMeshView.swift

gist.github.com/Matt54/44177...

RealityKit experiment #79
(code in the replies)

Animating 3D text by converting the output of MeshResource(extruding:) to a LowLevelMesh and applying a Metal compute shader

The MeshResource was interesting to unpack since it had multiple models/parts plus instance transforms

Thank you! 🙏

RealityKit Explode and Assemble SDF Shape (Metal + LowLevelMesh + Marching Cubes) RealityKit Explode and Assemble SDF Shape (Metal + LowLevelMesh + Marching Cubes) - EasingCurve.swift

gist.github.com/Matt54/87edc...

RealityKit experiment #78
(gist in the replies)

LowLevelMesh explode-and-assemble animation created by meshing an SDF using Marching Cubes and rigidly pushing it’s triangles outward

RealityKit Color-Blending Metaballs (Metal + LowLevelMesh + Marching Cubes) RealityKit Color-Blending Metaballs (Metal + LowLevelMesh + Marching Cubes) - MarchingCubesColorBlobParams.h

gist.github.com/Matt54/bcc31...

RealityKit experiment #77
(gist in the replies)

Color-blending metaballs created from animated sphere SDFs via Marching Cubes. LowLevelMesh vertices store per-vertex color in the Geometry Color vertex attribute, read by ShaderGraphMaterial

Ahhh, well I love that description 😆

Is it the room?

RealityKit Marching Cubes Blob (Metal + LowLevelMesh) RealityKit Marching Cubes Blob (Metal + LowLevelMesh) - MarchingCubesBlobParams.h

gist.github.com/Matt54/1a506...

RealityKit experiment #76
(gist in the replies)

LowLevelMesh blob from meshing the smooth-min of animated sphere SDFs with marching cubes. Glassy ShaderGraphMaterial is based on Apple's "Implementing adjustable material" sample

Haha exactly 😆

gist.github.com/Matt54/aec7d...