I Am a Software Developer?

For a long time I never described myself as a software developer. I was a sys admin, a DevOps kind of guy. I kept things running. Developers were the other people, the ones who wrote the software I had to look after.

It wasn't until my days at BuildingIQ that I started to question that.

It turns out I had been writing code all along

Looking back, I had been writing code of some sort since my early teens. First it was 68000 assembly, entered as raw binary. Then Z80 assembly. Then, when Linux arrived, C and C++, because back then you needed to be self-sufficient if you wanted anything to work.

None of it came through the normal channels. No degree, no courses, no bootcamp. Everything was self-taught, learnt by reading, breaking things and fixing them again.

So when people at BuildingIQ asked me about it, I still didn't recognise it. This was despite the fact I had written, and was maintaining, more than 50 Puppet modules along with all the Ruby code that went with them. That is software. I just didn't call it that.

Eventually I gave in.

Once I accepted it, everything got better

Something changed once I stopped fighting the label. Things got better, easier and more efficient. Instead of putting up with outdated system admin tools, I started writing UI apps in VueJS to replace them. I stopped seeing a problem and reaching for a workaround, and started seeing it as something I could build my way out of.

Today my stack is VueJS on the front and Go microservices behind it.

Microservices are not the problem, the old paradigm is

A lot of people give microservices a bad rap. I think that is because most of them are still looking at microservices through the old paradigm.

Remember when VMs became a thing? Everyone did the lift and shift. Take the physical server, turn it into a VM, job done. It worked, but it came with a whole set of issues that didn't go away until the systems were re-architected for what a VM actually was.

Microservices went through the same thing. When I started, it was very much a lift and shift. Containers with fat operating systems underneath them, and all the problems those operating systems bring with them: patching, packages you never asked for, shells and tools sitting there waiting to be used against you.

That is one of the main reasons I settled on Go. A Go service compiles to a single binary. I can take the operating system out of the question entirely and concentrate on the real issues, the code that actually does the work.

Replacing llama.cpp in Marvin

Which brings me to what I have been doing for the last two days: building a replacement for llama.cpp in Marvin's architecture.

This is not because llama.cpp is bad. It isn't. It is just not suitable for my situation.

I build with security in mind. I need to know there isn't a legacy OS sitting under a service that is going to bite me, or another php2shell style hole waiting just around the corner. Every extra layer is something else I have to check, patch and trust.

The result:

• Roughly 6G cut from the set of things in my system I need to check
• A 12.3M CUDA LLM engine for my Qwen models
• CVE scanning is now just a single binary
• No web chat or any other access that isn't needed
• TLS or mTLS for connections inside my cluster network, plus API keys

That is the difference between lift and shift and actually re-architecting. The engine does one job, and there is nothing else in the box.

More to come

There is more to go. I have a few ideas I want to try for running a 27B model on an 8G GPU while keeping a good token rate. If they pan out, that will be its own post.

And yes, after all these years, I think I can finally say it without the question mark. I am a software developer.

Don't Panic, It's Mostly Documentation

This is Marvin, as of 20 October 2026.

If you haven't seen a Gource render before, every dot is a file and every cluster is a folder, with lines tracing how they hang together. It's a quick way to see the shape of a codebase at a glance. And the shape of this one surprised me.

Two galaxies

On the right is the platform: the microservices that make Marvin run. Image generation, the vision model and its proxy, document reading, and all the plumbing that holds them together. This is the part I thought of as "Marvin" when I started in April. It's the part I'd point to if someone asked what I'd built.

On the left is the knowledge base. Rules, procedures, runbooks and governance. How Marvin should behave, what it's allowed to do, how to handle things when they go wrong, and who gets the final say.

The left galaxy is bigger. A lot bigger. There are around 650 files in the knowledge base and around 400 in the entire platform. And right now, the knowledge base is where most of the activity is.

So what is Marvin, really?

I set out to build an AI platform. What I've actually built is a fairly modest set of services and a very large rulebook telling them how to behave.

That isn't an accident. The services are machinery. They take a request and do a thing. The knowledge base is judgement: when to do the thing, when not to, and how to know it was actually done. A few examples:

Questions: "WHEN the user asks a question, the assistant MUST answer it honestly and then stop — a question is never an instruction to continue."

UI testing: "The assistant MUST verify a UI by using it — fill in every field, submit, reload, and confirm every value came back — because a page that renders proves only that it loaded."

Proof: "The assistant MUST NOT report a test passed until the logs of every service the request crossed prove it was handled; missing evidence is a defect, not a pass."

None of these are clever. Each one is there because, at some point, the assistant did exactly the thing the rule now forbids. It took a question as a green light to keep going. It declared a page working because it rendered. It called a test passed with no evidence anyone had handled the request.

Marvin was security-first from day one, and this is where that lives. Not in clever code, but in written-down decisions about what's allowed and what counts as done.

There's also a practical reason the balance tips this way. Code is increasingly cheap to produce. If a service disappeared tomorrow, I could rebuild it. The rules are different. Every one of them exists because I thought something through, or because something went wrong and I didn't want it going wrong again. That's the part that's hard to replace.

How they connect

Notice that the two galaxies don't touch. In the file tree, the rules and the platform are completely separate. The connection only exists at runtime, through the kb-service.

None of the rules are baked into the services themselves. Marvin asks for them.

When a session starts, a single bootstrap call hands Marvin the mandatory rule set. From there, rules are loaded by domain as the work demands: if the job involves git, Marvin searches for everything tagged for git and loads those rules before it touches anything. And if it can't name the document that governs what it's about to do, another rule says it has to search the knowledge base before going any further. No guessing.

Writing works the same way, through one narrow door. New knowledge goes in through a single write tool, and there's a rule about that too: the knowledge base is not a log. Run state and handoff notes don't belong there. Only things worth knowing next time.

Under the hood it's deliberately boring. Plain markdown files, each with a small JSON sidecar for metadata, a full-text search index over the top, and a separate graph that tracks how the services depend on each other.

That separation is why the left galaxy can grow so fast. Because the rules are data rather than code, changing how Marvin behaves usually means editing a markdown file, not redeploying a service.

If you're building your own

When I started, someone told me I wouldn't be any good in AI because I "don't know GPUs." Six months in, I'd say the GPUs were never the hard part. Neither was the code.

If you're building your own Marvin, plan for the rulebook. It'll end up bigger than the robot.

Don't panic. It's mostly documentation.

Running an LLM Is Not an AI Platform

Spend any time in self-hosting circles and you'll see the same post over and over. Someone has installed a model, pointed Continue.dev at it, and now they "run their own AI".

I know that setup well, because I built it. Back in April my first router did exactly that: a couple of models on two GPUs, an endpoint, and Continue.dev in the IDE. I wrote about it in Building a Self-Hosted LLM Router in Go.

It was a good first step. It was also short sighted, and I didn't realise how short sighted until I went further.

What You Actually Have

If you've loaded a model and connected an IDE plugin to it, here is what you have:

• A model that knows nothing about your code beyond what the plugin pastes into the prompt
• A model that knows nothing about your architecture, your conventions, or the decisions you made last year
• A model that can't check its own work
• A model that forgets everything the moment the session ends
• A GPU that sits idle whenever you aren't typing

All the intelligence about how to use the model lives in someone else's plugin. You've built the engine and handed the steering wheel to whoever wrote the extension. When the plugin can't do something, neither can your "AI".

The Model Is the Commodity

Models are getting better and cheaper every month. The one you're running today will be replaced in a few months by something better that you can download in ten minutes. That is exactly why the model is the least valuable part of the setup.

The value is in what surrounds it:

Context. A model that can search your source code and documentation through RAG answers questions about your system, not about systems in general.

Knowledge. Your rules, procedures, runsheets and architecture decisions, stored somewhere the model can reach. This is the difference between an assistant that writes plausible code and one that writes your code.

Routing. One model doing everything is one model doing most things badly. Intent classification, tool calling, embeddings and code generation are different jobs, and smaller specialist models often do them better than one big generalist.

Tools. A model that can read logs, probe services, run builds and look at a rendered page can check its own work. A model that can only emit text can't.

Resilience. What happens when the GPU node goes offline? If the answer is "everything stops", you have a dependency, not a platform.

The Tool Calling Lesson

This is where I learned the most. My April router used Qwen2.5-Coder-7B for agent work. It could produce tool call JSON perfectly. It could not reason about what came back. Call ls, get a file listing, call ls again. Forever.

I added loop detection, next-file hints, JSON schema constraints and deduplication. Every one of them was scaffolding around a gap in the model. The real fix was architectural: give tool calling its own model, standardise tool output before any model sees it, and stop expecting one model to be good at everything.

You don't learn that by pointing a plugin at an endpoint. You learn it by owning the whole loop.

Local Is Not the Same as Secure

"It runs locally, so it's secure" is the other half of the short-sightedness.

Local inference fixes data egress. Your prompts stay on your network. That matters, and it is a big part of why I built Marvin.

But the moment you give a model the ability to act, a new problem appears. Many agent setups hand the model a shell, because a shell is the quickest way to make an agent useful. Now a language model, which can be wrong with total confidence, has arbitrary command execution on your machine.

Marvin has no shell. It has around 90 tools, each with defined inputs, outputs and scope. Deploying to Kubernetes, querying logs, reading a spreadsheet, pushing to git: each is a designed path. That is a trust boundary, and you can't get one from a plugin.

The Economics Don't Work Either

A GPU that only serves autocomplete is idle most of the day. You've bought hardware to save on subscriptions, then used a fraction of it.

Self-hosting only beats paying for tokens when the hardware is doing real work. That means summarising in the background, generating embeddings, routing intent, running agent tasks and indexing knowledge. A system keeps the hardware busy. A single endpoint mostly waits.

What I'm Not Saying

I'm not saying don't run a local model with Continue.dev. Do it. It is the right first step, and it teaches you a lot.

I'm saying don't stop there and call it done. A model behind an endpoint is a component. The platform is everything you build around it: the context, the knowledge, the routing, the tools, the boundaries, and the parts that keep it running when hardware goes away.

That is where the value is. It's also where the interesting engineering is.

Running something similar, or think I've got this wrong? The comments are open.