Principal Machine Learning Engineer
Resideo Inc. (Honeywell Home) · NYSE:REZI▸ Vision understanding & multimodal LLM inference on edge SoCs
▸ DETAILS
- Designed and optimised multimodal large-model inference (LLaMA3, GPT, YOLO, CLIP) on edge SoCs (Oclea) for real-time video understanding and mobile deployment.
- Rewrote and customised backbone operators to support Clavary (.cvimodel) conversion and hardware-accelerated inference on Ambarella CV25 / CV72.
- Implemented model compression, quantisation and knowledge distillation — large models hitting SOTA inference under strict compute and memory constraints.
- Built a general image/video understanding framework and contributed to the Oclea SDK, integrating video pipelines with multimodal LLM capabilities.
- End-to-end ML: train → ONNX → Clavary → on-chip → real-time inference → post-processing.
- Perception-to-action pipelines: real-time triggers for tracking, sensor adjustment and motor actuation from visual scene interpretation.
- Integrated with RESTful APIs and edge-cloud pipelines to coordinate inference results with backend decision systems.
- Yocto-based embedded builds; cross-compilation and chip-specific SDK integrations; CI/CD for embedded reliability.
Multimodal LLMs
C++
Python
Ambarella CV25/CV72
Oclea
Yocto
ONNX
PyTorch
Azure
CI/CD