Nextcloud for storage, Forgejo for Git, Collabora, Prometheus + Grafana for monitoring - and increasingly, local AI workloads through Ollama. The fun part in this graph is around 21:00. That's Qwen 14B starting to do some actual work. Memory goes from almost nothing to ~6 GB, CPU briefly pushes across multiple cores, CPU temperature shoots up, and the fans immediately … [Read more...] about Been turning an old machine into my own little home cloud
LLM
Context Memory Management is one of the hardest things in LLMs
I have been in touch and working closely with Large Language Models since they became a thing with ChatGPT. I have built a lot of applications so far, mostly in the RAG (Retrieval Augmented Generation) space. First of all this is an amazing piece of tech. It has potential to actually replace a lot of human beings from their jobs. And when I say potential, I mean it will … [Read more...] about Context Memory Management is one of the hardest things in LLMs