← Return

Project: Context Artist

Art of the Prompt ·

Why

Context artist is my attempt at trying to build a tool for context engineering.1 This is a sport2 where you go beyond just trying to write a decent prompt. I enjoy using Ollama3 and seeing how much mileage I can get out of an open source model, especially to create some kind of magic app glue.4 As I tweaked the full prompt setup5, I realised I was doing lots of tests and changes without keeping track of what works and why.

Ollama

The design of Ollama is: download a model, run the service, call the API. It keeps models hot6 for some time after they're used. You call the API with your context and get a single or streamed response. All context-artist does is wrap this to make the system have managed state.

Design

With the Context Artist, you label the context and can use that label to signify how to couple new input to that context, or use the label to consume metrics and responses. There's also a firehose that allows you to subscribe to everything, which is how monitoring tools like the dashboard work. Because of the variety of reasons to connect to the service, it uses WebSockets.7

Right now I'm going to continue experimenting with this in order to flesh out my personal agent ideas.8

  1. Context engineering is, at the time of writing, a new area related to how the current era of so-called "generative AI" work. Primarily some data like text is provided and this goes through a process called tokenization. This is then used as an input for the model in order to produce some output. There's a lot of research currently underway into how context (content and structure) affects the "accuracy" of the output. ↩
  2. Despite being called "engineering" it doesn't feel like it nor like anything in "STEAM" but feels too competitive to be just a hobby. The noun "artist" is a nod to calling random work "art" like Subway employees being referred to as "sandwich artists" at one time. ↩
  3. Ollama is a tool for running open source models. ↩
  4. One Glue To Rule Them All. The idea here is to make the output of some software transform into the shape of the input of other software, without needing to write an adapter (that is, glue code). This is very useful when either the output or input is some kind of unstructured data, like human speech. ↩
  5. You have a system prompt, example conversation turns, maybe even tools. ↩
  6. I believe this to mean the GPU compute program is loaded, the weights are in memory, and possibly there's a cache for previous runs. I didn't look into this so it's possibly incorrect. ↩
  7. In my opinion, WebSockets makes it much easier to reason about how the service works. If you have streams, you already have a strong argument for them. But the fact that you might have separate tools all streaming for different reasons (send requests, consume responses) plus a "god-mode" dashboard, makes it easier to design for. ↩
  8. If I find a comparable, simpler tool I might end up switching to it. I haven't had time to do research on better alternatives and built this with Claude as an experiment in agent-assisted coding. ↩