I’m a heavy AI user.
I use AI every day for coding, research, writing, debugging, product thinking, automation, and random experiments. Over time, I started to notice that my AI setup was getting messy.
I had different tools, different model accounts, different keys, different usage limits, and different dashboards.
So I repurposed a small GL.iNet RM10 KVM box into a dedicated local AI Gateway appliance.
What this box does now
The device now acts as a small AI access hub on my local network.
Instead of configuring every app with different model providers and keys, I can point my tools to one local address:
http://device-ip/v1
Behind that one address, I can connect different AI model accounts and route my tools through the box.
Right now, the setup is mainly for models like:
-
GPT-style models
-
Claude
-
Gemini
-
DeepSeek
-
Kimi / GLM-style providers
-
other OpenAI-compatible model services
The idea is simple:
One small box.
One local AI entry point.
Multiple model accounts behind it.
Why I wanted this
For light AI usage, this is probably unnecessary.
But if you use AI a lot, things get messy quickly.
I don’t want to paste my real provider keys into every random app, script, IDE plugin, browser extension, or experiment.
With this box, I can create separate access keys for different tools.
For example:
-
one key for my coding editor
-
one key for my automation scripts
-
one key for testing agents
-
one key for a shared device
-
one key I can revoke anytime
That makes my AI setup much cleaner.
The small screen makes it feel different
The most fun part is the built-in screen.
I redesigned the screen so it shows the things I actually care about:
-
the local AI Gateway address
-
current CPU usage
-
memory usage
-
active requests
-
queued requests
-
today’s token usage
-
total token usage
This turns it from “some service running somewhere” into a real physical AI appliance.
I can just glance at it and know whether my AI gateway is alive, whether requests are flowing, and whether the box is under load.
That sounds small, but I really like it.
The final experience
From my point of view, this is now a tiny AI router on my desk.
My tools connect to the box.
The box connects to different AI model accounts.
I can manage keys, usage, and access in one place.
The screen gives me live status.
It is not trying to run a huge local model by itself.
It is more like a local control layer for the AI services I already use.
That makes it useful for my daily workflow.
What it is not
Just to be clear:
-
it is not a GPU server
-
it is not meant to replace local LLM machines
-
it is not a cloud AI provider
-
it is not a polished commercial product
-
it is not a tutorial post
-
it is not meant for everyone
It is a hardware repurpose project that turned into a useful personal AI tool.
Why use a physical box?
I could run something like this on a server or VPS.
But I like the idea of a dedicated device.
It sits on my desk.
It has a screen.
It has a clear job.
It feels like part of my AI workspace.
For me, that matters.
A lot of AI tools feel invisible and scattered. This makes my AI access layer feel visible, local, and intentional.
I’m curious what other people think
Would you use a small local AI Gateway box like this?
If yes, what would you want it to do?
Some features I’m thinking about:
-
connect multiple AI model accounts
-
one local address for all AI tools
-
separate keys for different apps
-
usage limits per key
-
daily / monthly token tracking
-
cost tracking
-
model health status
-
automatic fallback when one model provider fails
-
local network only mode
-
remote access through Tailscale or similar
-
alerts when usage spikes
-
better screen widgets
-
backup and restore
-
team / family sharing
-
support for local models like Ollama
-
a nicer mobile-friendly dashboard
And what would make this not useful for you?
For example:
-
you would rather run it on an existing server
-
you don’t want another device
-
you need local model inference
-
you need stronger security features
-
you need HTTPS / domain support
-
you need Docker support
-
you only use one AI provider
-
you don’t care about a physical screen
I built this mainly for myself, but after using it, I’m starting to think there may be a real category here:
small physical appliances for personal AI infrastructure.
Not a GPU box.
Not a NAS.
Not a cloud VM.
Just a tiny local AI Gateway that sits on your desk and manages access to the models and tools you already use.
Curious if this feels useful to anyone else, or if I’m just over-nerding the problem.
