llm-based multi agent marketing system

During my Masters thesis at the University of Bristol, I built a multi agent LinkedIn marketing system.

This project earned me the award of "Best Enterprise Project" within the graduating CS class of 2024 at the University of Bristol.

The entire team and their roles are described below. Keep reading to find out how they work.

This "autonomous and self-improving" team of LLM-based agents do the following:

• come up with and new idea
• research the idea with relevant source citations
• write a LinkedIn post (x times based on review)
• review and rate based on 32 relevant metrics (x times based on agent decision)
• check for false info, wrong citations, insensitive wording etc
• make the post on LinkedIn
• collect and review engagement data
• use the collected data and previous rating to improve over next run

All the actions taken by the system run completely without any human intervention. The multi-agent review loop is described above is visualised below.

The research hypothesis was to test - if a multi-agent system is better than a single agent system in this task.

Logically it makes sense but I wanted to rigorously test. The below image simply describes why this might be the case. LLM systems have a certain and unchangeable context window (no. of tokens that can be used for a task). This can have an affect on it the quality of the task execution. In the single agent system, context might be lost - leading to worse task performance.

To build the system described I used 3 main softwares. Airtable to host databases and content. Relevance AI to host agents and custom-coded Python tools for the agents. AWS to host the multi-agent interactions between the agents. The general architecture of the system is described below.

Initially, the plan was to use Microsoft's Autogen or CrewAI, but for speed and simplicity I created my own multi-agent chat framework hosted on AWS. The agent can initiate a chat with any number of agents to work on any task of its choosing.

Each agent has 2 types of tools:

• tools to access to external tools like - Google, LinkedIn, Slack, Airtable
• LLM generation tools - highly specific and well-written mini prompts that act as a form of abstraction from the main system prompt

These tools help each agent provide valuable input to the multi-agent chat that they are part of.

This project was an attempt to see if an entire department could be replaced with AI and automations. Will we enter a future where all knowledge work is going to be done by AI systems? Who knows.

During this project I did a lot of prompt engineering, AI app building and lots of reading academic papers.

If you are interested to learn more about this, send me a message - i love chatting about this project :)