A Private AI Box for 4 Cents an Hour

Any hosted chatbot will happily read your notes for you. That’s the catch. Paste them in, and they live on someone else’s servers now.

What I wanted was smaller and more boring: a box of my own that turns my text into vectors, finds the close ones when I ask something, and answers from them. Nothing on it listening to the internet. Nothing leaving it.

By hand, that’s an afternoon. A server, SSH hardening, a firewall, two downloads to check, two systemd units, an API key, and a note to yourself about what you did so future you can undo it.

So I wrote circuits for both halves instead, Ollama for the models and Qdrant for the vectors, and an example that builds the whole box in one Conn3x apply. The full walkthrough is on the Conn3x blog. This is the short version, and the bit about why.

One Apply, One Box

From nothing to a box in under three minutes

git clone https://github.com/conn3xorg/circuit-ollama
cd circuit-ollama/examples/digitalocean
export DIGITALOCEAN_TOKEN=dop_v1_...
conn3x init
conn3x apply -var 'admin_cidrs=["203.0.113.7/32"]'

Put your own address in admin_cidrs (curl -4 https://ifconfig.me will tell you what it is). What comes back is an Ubuntu 24.04 droplet with a cloud firewall that lets in SSH from you and nobody else, and five circuits run on it side by side: baseline, hardening, firewall, Ollama with nomic-embed-text and qwen2.5:0.5b, and Qdrant with a generated API key.

The droplet and everything on it share one state file. They’re planned together, applied together, and destroyed together, which is the whole reason I started Conn3x in the first place. When I ran it from an empty state in London, it made 101 resources in 2 minutes 36 seconds, and the next plan said No changes.

Why Only SSH Is Open

Ollama doesn't ask who you are

Here’s the part that made me careful. Ollama’s API has no authentication at all. Anyone who can reach port 11434 can run your models, pull new ones onto your disk and delete the ones you’ve got. I checked: with Ollama listening on 0.0.0.0 on a droplet, a plain curl from my laptop listed the models. No key, no prompt, nothing.

Qdrant does have a key in this setup. It travels in plain HTTP, though.

So neither of them faces the internet. Both listen on 127.0.0.1, and SSH carries your requests to them:

$(conn3x output -raw tunnel)

That runs ssh -L 11434:127.0.0.1:11434 -L 6333:127.0.0.1:6333 root@<ip>. While it’s open, localhost:11434 on your machine is Ollama on the droplet and localhost:6333 is Qdrant. Two firewalls stand in front of SSH: DigitalOcean’s, and ufw on the droplet in case the first one is ever removed. From outside, ports 11434, 6333 and 6334 didn’t answer, and Qdrant turned away a request without its key with a 401.

There was one snag on the way. The hardening circuit switches off SSH’s TCP forwarding by default, which kills the tunnel stone dead. The example sets AllowTcpForwarding = "local", so ssh -L works and -R is still refused. You can reach in; nobody can use the box to publish a port of their own.

Read the infographic as text

• What one conn3x apply builds: your laptop, an SSH tunnel (ssh -L, one per service), a free DigitalOcean cloud firewall that admits port 22 from admin_cidrs and nothing else, and a droplet where ufw denies all incoming but SSH and sshd takes keys only. The droplet is Ubuntu 24.04, s-2vcpu-4gb, in lon1. Its host key is made by the tls provider, installed by cloud-init and checked from the first connection. Five circuits run side by side: circuit-baseline v0.1.0 (chrony, time zone, network sysctls, key-only SSH), circuit-hardening v0.1.0 (tighter SSH crypto and kernel, AllowTcpForwarding = "local"), circuit-firewall v0.2.0 (ufw, deny all incoming but SSH), circuit-ollama v0.1.1 (Ollama on 127.0.0.1:11434 with nomic-embed-text and qwen2.5:0.5b) and circuit-qdrant v0.1.0 (Qdrant on 127.0.0.1:6333, API key from random_password). Droplet, firewall, key and software are one state file: planned, applied and destroyed together. $(conn3x output -raw tunnel) runs ssh -L 11434:127.0.0.1:11434 -L 6333:127.0.0.1:6333 root@<ip>; while it’s open, localhost:11434 is Ollama and localhost:6333 is Qdrant. From outside, 11434, 6333 and 6334 are unreachable.
• Why only SSH is open: the Ollama API has no authentication, so anyone who can reach 11434 can run your models, pull new ones or delete yours (tested: Ollama on 0.0.0.0 on a droplet, and a plain curl from a laptop listed the models). Qdrant has an API key, but it travels in plain HTTP, so here it only ever travels inside the SSH tunnel; without the key, Qdrant answers 401. Two firewalls: the DigitalOcean cloud firewall and ufw on the box as a backup. circuit-hardening sets AllowTcpForwarding = "local", so ssh -L is allowed and ssh -R is refused: nobody can use the box to publish a port of their own.
• The demo: the note “The spare key to the shed is under the blue flowerpot.” goes into Qdrant as a 768-dimension vector from nomic-embed-text, Cosine distance. The question “Where did I leave the key for the garden shed?” doesn’t contain “flowerpot”. Qdrant returns the note as the top hit with a score of 0.806 (0.8058598 in full), and qwen2.5:0.5b, on CPU, answers “under the blue flowerpot”. After a reboot the same note is found.
• What it costs and how long it takes (DigitalOcean prices, October 2026): $0.03571 an hour, $24 a month for s-2vcpu-4gb, about 4 cents an hour; the cloud firewall is free; the box costs money until you destroy it. From an empty state in lon1, apply built 101 resources in 2 minutes 36 seconds and the second plan said “No changes.” conn3x destroy removed the droplet, firewall and SSH key in 41 seconds. The Ollama release is a 1.5 GB download (it bundles CUDA libraries you won’t use on CPU) and the two models are about 670 MB together. Both run on CPU in 4 GB with room to spare; an 8B model wants about 8 GB of memory and about 5 GB of disk. After a reboot, Ollama, Qdrant and ufw come back and the plan is clean.
• What it doesn’t do: CPU only, with no GPU drivers, CUDA or ROCm installed, and a GPU hasn’t been tested. No web UI, no TLS and no login in front of Ollama, which is why it stays on loopback. Qdrant doesn’t back itself up: snapshots on request through the API, and copying them somewhere is yours. The state file holds the droplet’s private host key and the Qdrant key, so delete it when you’re done, or encrypt state for a box you keep. Needs Conn3x v0.6.7 or later, a DigitalOcean token with write, an SSH key pair, curl and jq. The circuits also run standalone on Ubuntu, Debian, Fedora and Rocky Linux, a home machine included.

The Shed Key

It found the flowerpot

The walkthrough’s test is deliberately small. Store one note:

The spare key to the shed is under the blue flowerpot.

Then ask it something in different words: Where did I leave the key for the garden shed?

Ollama turns both into 768-number vectors, and Qdrant finds the note with a score of 0.806. Notice what isn’t in the question? “Flowerpot.” That’s the point of embeddings: they match meaning, not words. Hand the note and the question to qwen2.5:0.5b, and it answers “under the blue flowerpot”.

That’s retrieval-augmented generation, start to finish. Nothing left your machine except through the tunnel, and nothing left the droplet at all. Every command for it is in the example’s Try it section, and they ran as written in zsh.

What 4 Cents Buys, and What It Doesn’t

The arithmetic, then the limits

The droplet is an s-2vcpu-4gb: $24 a month, or $0.03571 an hour, at DigitalOcean’s prices in October 2026. An hour of poking at it costs about 4 cents, the cloud firewall is free, and both models run on the CPU in 4 GB with room to spare. The apply takes minutes rather than seconds, mind, because the Ollama release is 1.5 GB (it bundles the CUDA libraries) and the two models add about 670 MB.

And the things it won’t do, so you don’t find them yourself:

• It’s CPU only. The circuit doesn’t install GPU drivers, CUDA or ROCm, and I haven’t tested a GPU. Small models and embeddings are fine; big ones are slow. An 8B chat model wants about 8 GB of memory, so set size to a bigger droplet.
• There’s no web UI, no TLS and no login in front of Ollama. That’s exactly why it stays on loopback.
• Qdrant doesn’t back itself up. It writes snapshots when you ask through its API. Copying them somewhere safe is your job.
• The droplet costs money until you remove it.

Removing it is one command, and it took 41 seconds:

conn3x destroy -var 'admin_cidrs=["203.0.113.7/32"]'
rm -f terraform.tfstate terraform.tfstate.backup

Delete the state file too. It holds the droplet’s private host key and the Qdrant key. If you’re keeping the box, turn on state encryption or keep the state somewhere you’d keep a secret.

Feed It Something Better Than a Shed Key

And tell me how it goes

You’ll need Conn3x v0.6.7 or later, a DigitalOcean token with write access, an SSH key pair, curl and jq. Don’t want a droplet? Both circuits work on their own on Ubuntu, Debian, Fedora and Rocky Linux, so you can point them at a machine under your desk instead.

This is the first in a weekly series on what the Conn3x circuits cover. If you feed this one something more interesting than where the shed key lives, start a discussion. I’d like to hear what it found.