R&D · TRACK 01 / 05 · Shipped

Our own engine,built from scratch.

FISH is a C++ engine that runs GGUF language models on ordinary computers. No llama.cpp underneath: every kernel, the tokenizer and the planner are ours.

Read the notes View the code

What it does.
One system.

KERNELS · LIVE

Reads GGUF directly

A bounds-checked GGUF v2/v3 reader memory-maps the whole file read-only. Every read is checked, so a broken file fails cleanly.

GGUF V2 · V3

Fast on a plain CPU

Activations are quantised to int8 once per matmul and run through AVX2 integer dot products, with a scalar reference kept as the oracle.

AVX2 · INT8

Safe chat templates

Templates are applied structurally in the engine. Text a user types can never become a control token, and BOS is added exactly once.

NO INJECTION

Plans for your machine

A hardware probe measures cores, RAM and disk speed, then picks a context size that fits and decides which layers stay resident.

HW PROBE

How it works

How it works.

Step 1

Map

The GGUF is memory-mapped and its tensors bound by architecture.

Step 2

Plan

The planner sizes the context and residency from the measured machine.

Step 3

Prefill

The prompt runs as a batch, reusing any cached prefix.

Step 4

Decode

Tokens stream out one by one, cancellable at any point.

Spec sheet.

01 / 06

Language

Language.

C++17, exposed through a C ABI

Language
C++17, exposed through a C ABI
Model files
GGUF v2 and v3
Architectures
llama, qwen2, qwen3, phi3, gemma2, plus MoE variants
Weight formats
F16, BF16, Q4–Q8 and K-quants, IQ4, MXFP4
Fast kernels
Q4_0, Q8_0, Q4_K, Q5_K, Q6_K
Larger than RAM
Next layer prefetched while the current one computes

In short.
FISH engine.

Memory-maps the model and keeps weights quantised

int8 activations with AVX2 integer kernels, scalar fallback elsewhere

Chat templates applied inside the engine, so user text cannot inject control tokens

A KV prefix cache makes repeated system prompts free

FROM IDEA TO RELEASE

Builders, not managers.

Say hello, fill in the blanks

Hi, I’m

and I want to talk about

.

Reach me at

Opens your email app, addressed to support@imutechz.in. Nothing is sent from this page.

or just email support@imutechz.in