---
title: "Local and cost-aware AI"
type: page
description: How Nathan chooses low-cost, local and self-hosted models across projects and work.
dates: "Apr 2026 (approx.) to present"
tags: [local-ai, inference, models, cost]
related: [about.md, projects/aware.md, projects/campus.md, projects/homelab.md, experience/terago.md]
updated: 2026-10-16
---
# Local and cost-aware AI

Most of Nathan's AI work runs on hosted API models, and he deliberately picks low-cost, efficient ones matched to each task, the way a company would, rather than paying for maximum capability. He also runs models locally where the job calls for it: on an edge board, on his laptop, and in an email parser he built at work.

## Where models run

| Where | What runs | Hosted or local | When |
|---|---|---|---|
| Campus | Any OpenAI-compatible endpoint for the agent; embedding search and cross-encoder reranking for retrieval | Either, by config | Aug 2026 to present |
| Homelab (Hermes Agent profiles) | Low-cost hosted API models picked per task, plus llama.cpp as a local provider | Mostly hosted | Jun 2026 to present |
| flow laptop | llama.cpp serving local GGUF models | Local | Apr 2026 (approx.) to present |
| Homelab server | Open WebUI, a Bifrost gateway that routes between local and cloud models, Whisper speech-to-text | Mixed | May 2026 to present |
| AWARE | YOLOv8n, YAMNet, Whisper-tiny, Piper TTS and a small GGUF LLM on an Arduino UNO Q | Local, on-device | Jul 2026 |
| TeraGo | Ollama parser that extracts details from outage emails and opens Jira issues (working, not deployed) | Local | May to Aug 2026 |

## AWARE on-device

Built at Hack the 6ix 2026, AWARE runs every model on one $80 Arduino UNO Q, using the board's Adreno GPU through ONNX Runtime QNN. The LLM only turns plain-English commands into rules through a `create_rule()` call; a deterministic parser compiles them, and a rules engine checks them every 500 ms without the LLM. Details in [AWARE](/knowledgebase/projects/aware.md).

## llama.cpp on flow

His flow laptop runs llama.cpp for local GGUF models. Both Hermes Agent profiles can use it as a provider alongside hosted APIs. Details in [Homelab](/knowledgebase/projects/homelab.md).

## Campus, endpoint-agnostic

Campus talks to any OpenAI-compatible endpoint, so the same code runs against a local server or a hosted API. Retrieval combines embedding search, cross-encoder reranking and a lexical SQL prefilter. Details in [Campus](/knowledgebase/projects/campus.md).

## TeraGo outage-email parser on Ollama

At TeraGo (May to Aug 2026) he built an outage-email parser in Python: a local LLM on Ollama extracts the key details from non-standard provider outage emails and the pipeline opens Jira issues. It works end to end, but TeraGo hasn't deployed it yet. Details in [TeraGo](/knowledgebase/experience/terago.md).