I need to write up how I’ve been running llama.cpp, but before I do I wanted to capture some notes on how I had to build and run my own Ollama contianer.

There’s a GitHub repo from a user named rjmalagon that has some containers specifically for AMD GPUs. I’ve been running these when I test out Ollama things. But the releases are currently a bit behind the latest Ollama. So I decided to build my own containers that have the latest source code in them. The code in the repo is being updated, it’s just the container releases that seem a bit behind.

If you look at the repo

The readme has instructions for building the images

docker build --build-arg FLAVOR=rocm .

I did add a tag when I build the containers, so my command was

docker build -t joshbressers:ollama-rocm --build-arg FLAVOR=rocm .

It took about 30 minutes, but once it was done, I had a working container. It’s was nice and painless. I now have the latest Ollama code that can run all the released models I want to try.

It’s nice when I don’t have to write a blog post that explains a bunch of extra work I had to do. This was pretty awesome, the work rjmalagon has done is fantastic if you’re running AMD GPUs with Ollama.