I built a chat client that uses embeddings to cluster messages by topic

Hi HN, I found myself scrolling backwards a lot in my chats with LLMs to follow up on interesting things I had seen earlier in the conversation. So, I built a basic chat client and I added a sidebar that allowed me to jump directly to different messages. The sidebar started getting hard to navigate with a lot of items, so then I thought it'd be nice if the chat history was clustered by topic.

In the end what I settled on was to have an LLM generate summaries of the query and responses, turn those summaries into vector embeddings, and then score the new message against the existing groups. A group's score is the cosine similarity of the new message and the last three messages in each group. The message is added to the best scoring group if it exceeds the threshold and is still similar enough to the first message in the group.

This was a fun project because I've never built a complete website before, so I learned a lot and it was satisfying to get something up and running. The stack itself is Node / Express / Svelte. I also enjoyed thinking about different ways to make the clustering work and trying to improve it.

If you've ever played around with clustering algorithms, you'll know there's definitely some magic numbers involved, so I'm curious to see if other people find it interesting.

Try out a live demo with sample data: https://llmtrail.com/demo

For more technical details, you can read my writeup here: https://doantamphan.com/llmtrail/