Launch HN: General Instinct (YC P26) – Frontier models on edge devices — editorial image
Signal Ledger placeholder illustration · Generated
Technology

Launch HN: General Instinct (YC P26) – Frontier models on edge devices

The report

Hey HN, Guanming and Bill here from General Instinct (https://general-instinct. After years of working in robotics, we kept running into the same problem: the best models never fit the hardware we actually had available.

Why it matters

The models that performed best were usually designed around datacenter assumptions: large GPUs, lots of memory bandwidth, and reliable network access. But most physical systems have the opposite constraints.

What sits behind it

That led us down the path of figuring out how much of a frontier model could be preserved while still making it practical to run on edge hardware

Where this fits in Signal Ledger

Related coverage from the Technology desk.

Our take

That led us down the path of figuring out how much of a frontier model could be preserved while still making it practical to run on edge hardware. 5-122B-A10B, a roughly 245 GB BF16 MoE model, into a 48 GiB GGUF.

Source note

Hacker News reporting: https://news.ycombinator.com/item?id=48414869

Read the original reporting