RESEARCH
Four areas where we push the boundary between what models can do and what practitioners can ship.
01
Distillation and quantization techniques that preserve capability while dramatically reducing compute. Our work on prompt-level compression achieves 60-80% token reduction with minimal semantic loss.
The core insight: most of what LLMs process is structural redundancy. Natural language is full of syntactic scaffolding, filler phrases, and implicit context that models reconstruct anyway. By identifying and compressing these patterns at the prompt level, we reduce token consumption without losing the information that actually drives model behavior.
This research directly produced Wren, our LoRA fine-tuned compression model. Rather than compressing at the model level (quantization, pruning), we compress at the input level, which works with any model regardless of size or architecture.
WREN02
Local inference pipelines for Apple Silicon. MLX-native model serving, speculative decoding, and context caching strategies that make on-device AI practical for development workflows.
We run inference on consumer hardware. Our home lab runs Ollama across 4x Tesla M60 GPUs with 32GB total VRAM, serving models for chat, code generation, and tool use. On the local side, MLX on Apple Silicon handles lightweight models with surprisingly good throughput.
The goal isn't to replace cloud inference but to make local inference viable for the 80% of tasks that don't need frontier models. A 1.5B model running locally with zero latency and zero cost handles a lot of practical development work.
MLX / APPLE SILICON03
Multi-agent orchestration patterns for complex development tasks. Session multiplexing, context isolation, and parallel execution frameworks that scale agent productivity.
Single-agent workflows hit a ceiling fast. Complex tasks require parallel research, specialized tool access, and coordinated execution across multiple contexts. We've built patterns for running 5+ agents simultaneously, each with their own context window, tools, and objectives, converging on a shared outcome.
This work produced Flock, our macOS agent multiplexer. But the architecture patterns are general. We're exploring how multi-agent coordination maps to team workflows, where each agent operates like a specialist with bounded context and clear deliverables.
FLOCK04
Systematic approaches to prompt design. Compression-aware prompting, token-efficient instruction patterns, and automated prompt optimization through empirical measurement.
Most prompt engineering is vibes-based. We measure. Every prompt pattern gets benchmarked for token cost, output quality, and consistency across runs. This empirical approach reveals which techniques actually work versus which just feel right.
Compression-aware prompting is particularly interesting: designing prompts that maintain effectiveness even after compression, ensuring the compressed representation preserves the instruction signals that matter most to the model.
APPLIEDGET INVOLVED
All research ships as open source tools. Star our repos, open issues, or fork and build on what we ship.