A 35B AI model runs on an iPhone at 20.51 tokens per second
ModelBest reported this speed on an iPhone 18 Pro Max, Pandaily found. Its LES method preloads experts for the next model layer from the phone’s flash storage. The company previously cited over 13 tokens per second on an iPhone 17 Pro Max.
ModelBest has open-sourced the code. If phone makers use LES in commercial devices, running large models on the phone could become a selling point.
@unfiltered_cn
