Fast feedback
Mac checks catch import failures, gross regressions, visual changes, and CPU-side mistakes before a device queue is used. Their verdict is MAC_PROXY_PASS, not Quest certification.
ForgeFX Simulations / Decision Brief
Quest asset performance monitoring
A verified architecture recommendation for bots optimizing isolated Unity assets that will become part of large interactive simulations, often on standalone Meta Quest 3 VR.
Verified 2026-09-25 · Research only · No benchmark project edits, automation install, hardware purchase, or infrastructure deployment
Decision
Adopt the top-ranked option: hybrid staged checks. Run cheap Mac standalone screening on every changed asset, authoritative Quest 3 APK checks on a leased fixture, and representative integration plus thermal soak nightly or before release. A missing or unauthorized device is NOT TESTED, never PASS.
Why this architecture
Destination authority
Quest checks measure the actual standalone Android/OpenXR/Vulkan stereo path, compositor behavior, stale frames, thermal drift, target memory, and the chosen display cadence.
Bot-friendly rules
Give the bot class and instance budgets plus a representative stress scene. Preserve art quality with fixed hashes; never let the bot lower resolution, shadows, texture quality, foveation, LOD, or physics silently.
Operational truth
The fixture needs one-time human developer-mode, USB/RSA, account, and proximity setup. After that, ADB and scripted testing can operate unattended, subject to device leases and thermal validity.
The three ranked options
01 / Recommended
Hybrid staged
Best balance. Mac PR gate; Quest changed-asset gate; full scene and thermal soak nightly/release.
Tradeoff: two configs can disagree, so the report must label proxy vs authority.
02 / Authority
Quest-only lab
Most faithful. Every asset runs as a native APK on a physical fixture with XR telemetry.
Tradeoff: slow queue, provisioning, USB/RSA, sleep, battery, thermal, and device-drift operations.
03 / Proxy
Mac-only
Cheapest. Native standalone player, fixed settings, moving route, raw frame samples, visual gates.
Tradeoff: cannot certify Quest stereo, compositor, stale frames, SpaceWarp, thermal behavior, or target memory.
External evidence
| Source | Verified evidence | Decision impact |
|---|---|---|
| Meta / OVR Metrics Tool Updated 2026-06-21 | Report mode records CSV and PNG; ADB launches the app and enables CSV. Basic stats include battery, levels, utilization, FPS, stale frames, and App GPU Time. | Use it for device evidence, but do not assume a complete CLI export path. |
| Meta / stats definitions | Stale frames reuse the previous frame; App GPU Time is in microseconds; CPU and GPU work in parallel; SpaceWarp FPS is synthesized. | Do not use average FPS alone, sum CPU+GPU, or call SpaceWarp native FPS. |
| Meta / best practices | Advanced capture adds overhead; first-run shaders and loading skew results; warm repeats are required. | Separate cold/warm lanes and measure observer overhead. |
| Meta / Unity performance | Pipeline state, render/main thread capacity, animation, networking, API, and multithreaded rendering affect Quest results. Link PC-VR is a separate PC target. | Use realistic scenes and keep standalone Quest separate from PCVR Link. |
| Meta / refresh rates | 90 Hz can be requested; thermal events can reduce rates above 72 Hz; thermal behavior can be simulated by documented broadcast. | Record actual refresh and make thermal drift invalid or its own labeled scenario. |
| Meta / CPU-GPU levels | Clock levels trade power and performance; choose the lowest level that sustains target. | Record levels/frequencies; do not compare runs with hidden clock changes. |
| Meta / Scriptable Testing | Developer-only services can set/reset selected device state and auto-sleep behavior, but they prepare a device; they do not run or validate tests. | Full automation begins after one-time human authority and fixture setup. |
| Android / ADB | adb devices -l reports state/model; USB debugging and RSA authorization are prerequisites; -s selects a device. | Preflight read-only availability and use explicit serials. |
| Unity 6 / FrameTimingManager FrameTiming | Results have a four-frame delay; GPU timings may be zero; Android Vulkan/OpenGL ES are supported; timings use documented units/conversion. | Use raw player timings carefully and treat missing GPU samples as invalid. |
| Unity / ProfilerRecorder Performance package | Player/Release access is supported; timing samples are nanoseconds; recorders are disposable. Test Framework builds a development player and warns about VSync/quality configuration. | Instrument release-like builds separately from deep profiling; do not headless-render a rendering test. |
Internal evidence
Grain-bin baseline
The exact ForgeMedia grain bin ran in a macOS standalone IL2CPP player on the M4 at 1920×1080, built-in pipeline, Ultra, uncapped, on actual Unity Terrain.
| Run | Avg FPS | p95 ms | p99 ms | 1% low |
|---|---|---|---|---|
| asset-run1 | 609.46 | 2.342 | 2.956 | 85.85 |
| asset-run2-clean | 648.29 | 2.133 | 2.676 | 105.30 |
Two accepted 30-second runs. A concurrent Unity telemetry process contaminated later candidates, so no third clean repeat was claimed. This is not Quest capability.
What it proves
- Standalone player frame timing is more useful than Editor render callbacks.
- High average FPS can coexist with materially lower tail performance.
- The existing harness saves summaries, not raw per-frame samples.
- The empty-terrain control was shorter and not a same-configuration Quest delta.
- Focus loss and contention must be recorded as invalid, not hidden.
Read the baseline report · Internal #brandon_forgebot context
Recommended workflow
01 / Fixture authority
Provision once
Enable developer mode, accept USB/RSA, provision test account/device, install tools, and mount a repeatable fixture. A headset asleep on a desk is not valid gameplay evidence.
02 / Preflight
Identify exactly
Run adb devices -l; require exactly one device state. Record model, OS, hashed serial, battery/thermal state, and build/settings hashes. No device = NOT TESTED.
03 / Build
Release-like telemetry
Use low-overhead raw telemetry in the release-like APK. Keep a deep-profiling build separate. Compare HUD/telemetry on vs off to quantify observer overhead.
04 / Run
Cold, warm, steady
Separate first-launch shader/load behavior from warm steady state. Use at least five valid steady repetitions for a new baseline, fixed warmup, moving/rotated routes, and realistic stereo workload.
05 / Integrate
Stress and soak
Use multiple instances, LOD transitions, shadows, transparent overdraw, physics, animation, texture residency, and actual scene settings. Run full scene nightly and thermal soak nightly/release.
06 / Preserve
Never rebase silently
Retain raw rows, screenshots, hashes, route IDs, logs, per-run summaries, and the gold baseline. A retry is bounded; repeated invalidity is quarantined.
Metrics and budgets
| Profile | Native budget | Example 20% whole-app guardrail |
|---|---|---|
| 72 Hz | 13.889 ms | 11.111 ms |
| 80 Hz | 12.500 ms | 10.000 ms |
| 90 Hz (provisional) | 11.111 ms | 8.889 ms |
| 96 Hz | 10.417 ms | 8.333 ms |
| 120 Hz | 8.333 ms | 6.667 ms |
Example initial class quotas, pending measured noise and scene allocation: hero/complex interactive ≤0.75 ms GPU delta and ≤0.50 ms main-thread delta; ordinary interactive prop ≤0.25 ms each; repeated small prop ≤0.10 ms per-instance diagnostic delta. These are policy examples, not validated Quest results. Do not add percentiles or extrapolate linearly; the integration stress scene is the authority for combined content.
Verdict states
Thermal acceptance distinction: External heat or an incorrect starting state can invalidate a controlled comparison. However, throttling or missed deadlines caused by the app during the intended sustained-use test are genuine failures, not samples to discard and retry away. Clock changes alone are normal power management, not automatic failures. Missing ADB means headset connectivity is unverified, not proof that no headset is physically attached. The frame-time headroom figures are proposed CPU/GPU workload guards, not limits on refresh-paced frame intervals.
All required metrics, visual/content hashes, and repeated valid runs meet the profile.
Near threshold, high variance, or Mac proxy passed while Quest evidence is pending. Queue stronger validation.
Repeated valid runs exceed a hard frame, stale, CPU/GPU, memory, thermal, or visual threshold.
Focus loss, sleep, wrong hash/settings, contention, missing telemetry, crash, thermal/refresh drift, or incomplete route. Retry once, then quarantine.
No compatible Quest, ADB unavailable, device unauthorized, or no queue lease. Never convert this state to PASS.
Performance pass with missing geometry, wrong materials, reduced texture quality, disabled shadows, or changed camera is invalid.
Verified command boundary
Meta documents these building blocks:
adb devices -l adb -s <serial> install -r <apk> adb -s <serial> shell am start omms://app adb -s <serial> shell am broadcast \ -n com.oculus.ovrmonitormetricsservice/.SettingsBroadcastReceiver \ -a com.oculus.ovrmonitormetricsservice.ENABLE_CSV adb -s <serial> pull <documented-device-path> <local-dir>
ADB availability and device presence were checked read-only on this host: adb is not installed. The OVR page documents CSV capture and later transfer but does not specify one stable universal remote CSV directory or fully unattended metric selection/export CLI. The future runner must verify the path on the pinned device/tool version or use app-instrumented telemetry/logcat. It must not guess.
Uncertain prerequisites
- Destination Unity render pipeline is unknown; current Mac evidence uses built-in pipeline.
- Standalone Quest APK and Quest Link PCVR are separate targets. Link requires a Windows PC GPU and its own budget/profile.
- OVR Metrics Tool version, OS, package, selected stats, export path, and device fixture remain to be pinned.
- Quest actual OpenXR/Vulkan timing, memory, thermal behavior, and observer overhead remain untested.