This talk explores how to design agents that can perceive, reason, and act across diverse digital and physical environments.
First, Aravind will present Fara, the latest computer use agent from Microsoft that sets a new SoTA for its size class. Fara adopts a "pixel-in, action-out" formulation: it perceives the computer screen directly through raw screenshots, formulates intermediate reasoning steps, and predicts atomic low-level actions (clicks, scrolls, keystrokes).
This results in robust and human-like behaviours that generalise to the fat tail of web tasks. Next, turning from navigating…

Hosted by Sourabh Patravale, Paras Chopra & Jay Joshi