AI Vision
Multimodal evidence + grounded rule reasoning
Play a sample service-bay video and inspect visual observations, potential rule applicability and cited sources beside the video.
What you can do
- Select one of three service-bay videos and play it in the browser.
- Inspect a sampled frame, its video timestamp and structured car, PPE and work-area observations.
- Follow evidence into a cited rule evaluation and record a review decision for the displayed result.
Analyze a sample video
The interactive demo requires JavaScript and available backend services. The implementation, sources and limitations remain readable below.
Why it exists
A useful visual analysis should make its evidence and interpretation inspectable. This experiment connects observable service-bay conditions with a bounded set of potentially applicable workplace rules.
How it works
Video → sampled JPEG frame → structured visual observations → rule evaluation against maintained source summaries → cited result → human review. The browser samples during playback at six-second intervals, skipping requests while an analysis is in progress.
Architecture and stack
React video playback and canvas capture send a frame to Express /analyze-image. OpenAI GPT-4o returns JSON; the server validates fields, evidence references and rule identifiers, then attaches authoritative source links from its own catalog. Redis bounds requests. There is no TensorFlow or PyTorch pipeline here.
Grounding and applicability
The source catalog contains maintained paraphrases of OSHA 29 CFR 1910.133(a)(1), 1910.132(d)(1) and 1910.22(a)(3), reviewed September 7, 2026. The model reasons over those summaries. Matching a source identifier is deterministic; deciding whether a rule applies is model inference requiring human review. This is not live regulatory retrieval or a complete rules database.
Technical challenges
Occlusion, lighting, perspective and sampled frames limit visual evidence. The UI keeps the exact analyzed frame and timestamp alongside results. Source URLs come from the server catalog, and invalid or unsupported model references fail validation.
Privacy and safety
Only sampled frames are sent to OpenAI through the backend, not the entire video or its audio. The demo offers local sample videos and no user video upload. Review source text and workplace context before acting on a potential issue.
Limitations
A frame cannot establish an entire activity, missing equipment outside the view, workplace jurisdiction or documented hazard assessments. No calibrated confidence, tracking across frames, legal determination or accuracy benchmark is provided. Review selections remain in this page only and reset with a new result.
Implementation lessons
Visual observation, source identity and applicability are separate checks. JSON and citation validation improve inspectability but cannot establish that an observation or compliance conclusion is correct.
Next work
Broader reviewed rule coverage, versioned full source snapshots, persisted human review and labeled evaluation datasets are future work. No measured safety or business impact is claimed.