Week 2 Updates
This week, I focused on completing a fully functional RAG system. The chatbot is now able to retrieve relevant information based on user queries. A reranking model is used to select the top 10 most relevant chunks (I’m considering reducing this to 5 and lowering the context window for better performance). Metadata is displayed correctly (links and images), and I’ve also started working on a repository-specific chatbot for beagley-ai.
I created a GitHub repository and linked it with GitLab repository with a simple GitHub Action so they both get updated the same time. Since OpenBeagle is down I guess, I’ll be updating these repositories.
I’ll update the document shortly with screenshots from the chatbot responses. In the meantime, I’d like to include your suggested prompts inside this document: BeagleMind Prompts.
Blockers
- The reranking model’s results aren’t always accurate, so I’ve been experimenting with different open source models.
- Setting up the metadata display was a bit challenging, especially formatting the LLM output using Markdown and raw GitHub links.
- For the repository specific chatbot, (I’ll mention in the updates document how does that look like), it still doesnt deliver very accurate response because most of it is images only and there’s not much text for the chatbot to learn from, so I’m figuring out other ways to fix this issue,
I’ll keep you updated.
Next Steps
My goal is to finalize the reranking model and ensure the chatbot is fully functional, with improved metadata display. I’ll also be investigating why the beagley-ai specific chatbot isn’t performing as expected. Finally, I plan to begin integrating a knowledge graph into the architecture.
I’ll update the updates document shortly so you can see the results.