Ghost in the Pi - Building my Own Ghostwriter

Why?

I have conflicting feelings about generative LLMs. But they're also not going anywhere. I built a small project to help me write blog posts, and I thought I'd share how I did it.

Hardware

Software

Ollama

https://ollama.com/download/linux

You can pick whatever models. For a Raspberry Pi 4, 8GB, I went with phi3:mini. I already like the writing style of the phi3 models, and the mini version is lightweight enough to work on the pi.

Important note that the fetch path is different between linux and macOS. It's /api/{path} for mac, and /v1/{path} for Linux. That one took me a little while to realize; I kept blaming the hardware.

Challenges

Getting the model to not timeout. On a more powerful computer, it's pretty trivial to generate a medium-length blog post. Not so much on the pi.

Enhancements