Ollama Agentic Workflows

Overview

An agentic workflow platform, built on locally run Ollama models

The problem

With a lot of news about the amount of water used by AI data centres, I wanted to see if one of these agentic workflow platforms could be built and run locally, with no direct reliance on third party data centres

What I built

After creating a couple of dummy companies, I looked at what insights I could get from the data I had created. Relationships, forecasts, what could save me time and effort, and what was in there that I might miss.

I created an agent for each area, with an accompanying Modelfile, setting the model parameters to minimise hallucinations, and repeatedly inform the model never to make up data. This last part was incredibly frustrating. I was right to be frustrated. I repeatedly told the AI never to make anything up, and that "I don't know" is an acceptable answer if it is true. That's on the AI.

Once an additional wrapper had been added so that the prompt was json including instructions, what process to follow, and the user prompt, responses went to 100%, with the agents staying in their lanes and providing accurate data.

At this point a supervisor agent was created. With no direct access to any data, the supervisor was tasked with taking the user prompt, seeing which other agents are required for an answer, and then asking the next agent its question. This happens asynchronously, so in some workflows there can be two or three agents running at the same time, while the supervisor waits for their responses.

When all responses are received, the supervisor verifies that the answers satisfy the original prompt and overall requirements. If not, the agents are asked to improve, and if all is fine, the response comes back to the user.

Users can create agents of their own, create new workflows or use the templates.

All the steps made by the agents are logged, so users can quality check every step of the workflow, and the UI updates in realtime to show each step's status.

The app also contains analytics to show and highlight any historical issues or gaps, and there is a new user tour to show users where everything is.

The UI is still pretty complex and confusing, and it can't be deployed at the moment due to the local requirement of Ollama, but as a PoC app I was really pleased with it

Screenshots

Workflow idle

Workflow in progress

Workflow in progress

Workflow complete

Workflow complete

Agents

Templates

Logs

Logs

Analytics

Analytics

What I learnt

Switching the characters to agents was really easy, and was just a case of adapting the Modelfile and parameters to reduce creativity. An additional wrapper was required, swapping the swting based prompt for a full JSON object, but the Ollama models were far more accurate as soon as this was added.

One issue is that with other non-workflow Ollama models installed, opening the local instance via a Cloudflare tunnel would allow me to deploy the UI, but it would open the full Ollama API on my machine, as the hosting provider I use currently only manages up to Nodejs 20, which means no latest version with server components. This means my apps have to be deployed using a static output, and API calls are made in the client.

As a result this app stays working locally only, on my PC and Surface Pro.