By KFSys
System Administrator
I’ve been running a few small LLM agents lately, and the thing that kept biting me wasn’t the model — it was not being able to see what the agent actually did. Which tool did it loop on five times? Where did that run quietly cost me a dollar? Did last week’s prompt tweak make things worse or better? The logs never quite tell you.
The hosted tools for this want your prompts on their servers, and the self-hostable ones (Langfuse and friends) wanted me to stand up Postgres + ClickHouse + Redis + S3 — four moving parts to log a few thousand LLM calls a day. That’s a lot of Droplet for a side project.
So I ended up building a small open-source one, Otterscope, and self-hosting it is genuinely a one-liner. Figured I’d write down how I run it on a DigitalOcean Droplet, because “AI agent observability” sounds heavier than it actually is here.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Accepted Answer
You point your agent’s OpenTelemetry exporter at Otterscope and you get an agent-run-first view: every trace becomes a run, with its steps, tool loops, tokens, and cost. Click a run and you see the actual messages in and out of each model call. There’s a cost table, assertions and LLM-as-judge evals scored onto your real runs, a compare view for “this week vs last,” and webhook alerts when your error rate or spend crosses a line.
The important part for hosting: it’s one static Go binary and one SQLite file. No database to provision, no cache, no object store. That’s the whole reason it fits on the smallest Droplet.
A Basic $6/mo Droplet (1 vCPU, 1 GB RAM) is plenty unless you’re pushing serious volume — SQLite handles this scale comfortably. Spin one up with Ubuntu, add your SSH key, done.
There are two ways to run it. Docker is the fastest; the raw binary with systemd is the leanest.
If you ticked the Docker Marketplace image when creating the Droplet, or just
apt install docker.io, it’s one command:
docker run -d --restart unless-stopped \
-p 8317:8317 -p 4318:4318 \
-v otterscope:/data \
ghcr.io/otterscope/otterscope
8317 is the web UI, 4318 is the standard OTLP ingest port your agents send
to. The otterscope volume keeps your data across restarts.
No Docker, ~15 MB resident:
# grab the linux binary from the releases page, then:
sudo mv otterscope /usr/local/bin/
sudo useradd --system --home /var/lib/otterscope --create-home otterscope
Drop a unit at /etc/systemd/system/otterscope.service:
[Unit]
Description=Otterscope
After=network.target
[Service]
User=otterscope
# -listen/-otlp are set explicitly here because Otterscope binds to
# localhost by default (see the security note below)
ExecStart=/usr/local/bin/otterscope serve -db /var/lib/otterscope/otterscope.db -listen :8317 -otlp :4318
Restart=on-failure
[Install]
WantedBy=multi-user.target
sudo systemctl enable --now otterscope
By default Otterscope binds to 127.0.0.1 — nothing is exposed to the
internet until you decide to. On a Droplet you have two sane choices:
ssh -L 8317:localhost:8317 you@your-droplet, then open
http://localhost:8317. Nothing is public, ever.-listen :8317 -otlp :4318 (as the unit
above does), and put a DigitalOcean Cloud Firewall in front so only
your app servers can reach 4318 and only you can reach 8317. Otterscope
is single-user and has no login yet, so don’t leave 8317 open to the
world.Whatever framework you use — Pydantic AI, the OpenAI Agents SDK, LangGraph, the Vercel AI SDK — it’s the standard OpenTelemetry variable:
export OTEL_EXPORTER_OTLP_ENDPOINT=http://YOUR_DROPLET_IP:4318
Run your agent, refresh the UI, and the run shows up in a couple of seconds. No SDK to install, no code changes if you’re already emitting OTel.
Want to see it populated before wiring a real agent? otterscope sample
seeds a batch of realistic demo runs.
scp the .db file and that’s your backup.-retention flag
if you’d rather prune.That’s the pitch: the observability stack for your agents is a single process sitting next to them, costing about as much as a couple of coffees a month.
Repo and docs are here: https://github.com/otterscope/otterscope
(Apache-2.0). There are per-framework setup guides under docs/frameworks/,
and the Docker image is ghcr.io/otterscope/otterscope.
If you self-host it on a Droplet and hit a rough edge — especially a framework whose traces don’t map cleanly yet — open an issue. Raw payloads are retained, so fixes apply retroactively to data you’ve already collected.
Hi there,
This is a great write-up and a real problem worth solving. The observability gap for small LLM projects is genuinely annoying, most hosted tools are overkill or want your data, and the self-hosted alternatives like Langfuse have a setup cost that does not match a side project.
The one-liner deploy on a $6 Droplet is a good angle. Would love to see the Otterscope repo linked here for anyone who wants to try it.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
Scale up as you grow — whether you're running one virtual machine or ten thousand.

From GPU-powered inference and Kubernetes to managed databases and storage, get everything you need to build, scale, and deploy intelligent applications.
