Transformer architectures have enabled the rapid progress of modern language models, but scaling them efficiently remains a major challenge. As models grow larger, compute cost, memory usage, inference latency, and deployment complexity increase significantly.
In this session, Abhay will discuss practical research directions to improve transformer efficiency in small and medium-sized language models.
The talk will cover architectural experiments, training optimisations, efficient attention mechanisms, memory-efficient techniques, and inference-focused design choices being explored while…

Hosted by Sourabh Patravale, Jay Joshi & Paras Chopra