Scalable Transformer Architectures for Edge Devices

Abstract

The paper focuses on edge-friendly transformer design, including attention compression and memory-aware scheduling for constrained mobile hardware.

Key Highlights

• Memory use dropped significantly under constrained mobile inference budgets
• Latency stayed stable when sequence length increased on edge GPUs and NPUs
• The method favors deployment scenarios where battery life and thermal limits matter

Methodology

1. 1Evaluated attention compression, parameter sharing, and quantization-aware training
2. 2Benchmarked on mobile-class hardware with realistic battery and thermal constraints
3. 3Compared throughput and quality against conventional transformer baselines

Findings

• Smaller attention heads preserved quality better than aggressive full-model compression
• On-device routing improved both responsiveness and privacy posture
• The architecture is suitable for field tools, assistants, and offline analytics