Actual speed-up
#2
by bonlime - opened
Hi, found your project, can you include actual speed up numbers at least for your own Apple Silicon device? want to understand if it's even worth trying to run it on Macs
Hey, sure coming within the next few days - also another version is coming soon with more speedup. For my M2 and A14 chips it's around 7-9 tok/s on a 4K context...