C.A.R.E. — Companion Autonomous Robotic Entity
Built a hackathon prototype connecting Snap AR Spectacles with a Booster K1 humanoid through ROS 2, live camera streaming, and head-orientation control. My work covered robot control, AR integration, and VLM functionality.
Technical work
Cal Hacks 12.0 · Working assistive-robot prototype
- Developed Booster K1 robot control and Snap AR integration, mapping head orientation to ROS 2 movement commands for an assistive-robot prototype.
- Integrated robot camera streaming and AR interaction with the team’s WebSocket/ngrok communication layer, separating control traffic from video and supporting manual and patrol modes.
- Contributed Gemini VLM and OpenCV functionality for human detection and interaction; collaborated with teammates responsible for SLAM/navigation and the AR communication interface.
Overview
C.A.R.E. bridges augmented reality and robotics to create an assistive robotic companion for individuals with mobility challenges. The system integrates Snap AR Spectacles with the Booster K1 robot, enabling intuitive human-robot interaction through an AR control interface.
Built at Cal Hacks 12.0 in October 2025. Assistance for older people and people with limited mobility motivated the prototype; it was not evaluated as a deployed care system.
What It Does
- AR Control Interface: Users view live robot camera feeds through AR glasses with joystick and HUD overlay for intuitive control
- Dual Operation Modes: Supports both autonomous patrol mode and manual control via head movements
- AI-Powered Detection: Gemini and OpenCV support human detection and interaction in the prototype
- Communication Architecture: ROS2, WebSockets, and ngrok tunneling maintain responsive control with dual channels — low-bandwidth control and high-bandwidth video
- Head Tracking Navigation: User head orientation maps directly to robot movement commands for natural, hands-free control
Target Applications
Proposed uses for further development and validation:
- Elderly and mobility-limited assistance — helping individuals navigate and interact with their environment
- Security and patrol operations — autonomous monitoring with human oversight
- Search and rescue missions — remote exploration in hazardous environments
- Construction and infrastructure inspection — safe inspection of dangerous or hard-to-reach areas
Technologies Used
Technical Challenges & Solutions
- Latency Reduction: Implemented dual WebSocket channels — one for low-bandwidth control signals and another for high-bandwidth video streaming to minimize control lag while maintaining video quality
- Perception Integration: Connected vision-language functionality with the robot interface; the project does not report end-to-end latency or human-following accuracy
- AR Interface Design: Created minimal, intuitive HUD elements that provide essential information without overwhelming the user's field of view
- Sensor Calibration: Developed mapping system to accurately translate AR camera rotation data to robot motor angle commands for precise head-tracking control
Team
Built at Cal Hacks 12.0 by:
- Het Patel — Booster Robot Control, Snap AR Integration, Vision Language Models
- Sunny Deshpande — SLAM and Navigation
- Atharv Mungale — Snap AR Software and Communication
- Vetrivel Balaji — Snap AR Software and Communication