Benchmarking

Objective	Goals	Tools	Models	Expected Output	Agent Output	Tested	Doesn’t work	Somewhat Works	Works Perfect
Writing Blog	Write a 1000-word blog with 2 images.	Searx Toolkit, Read File, Write File, Stable Diffusion Toolkit	GPT-3.5	900-1000 word blog in a .txt file along with 2 images.	350-word blog in a .txt file along with 2 images	Yes	No	Yes	No
Tweeting	Do Twitter Marketing by automatically creating 1 tweet every day about a specific theme.	Twitter Toolkit, Stable Diffusion Toolkit	GPT-4	Tweet with Image	Agent tweeted successfully with a relevant image	Yes	No	Yes	No
Posting Instagram	Do Instagram Marketing by automatically creating 1 post every day about a specific theme.	Instagram Toolkit, Stable Diffusion Toolkit	GPT-4	Instagram Post with Whey Muscle Protein Powder.	Agent posted an image on Instagram with captions and image of generic whey protein from Stable Diffusion	Yes	No	Yes	No
Analytics report	Collect Weekly Analytics Report from Google Analytics and Mail it every Monday at 8 am.	Google Analytics Toolkit, Email Toolkit, Write File	GPT-4	.txt report with the collected data, which is mailed to me.	To be tested	No	No	No	No
Sales prospecting	Create a list of sales managers’ contacts from specific companies in a CSV and send personalized mail.	Tools not built	GPT-4	Maintains a CSV of leads that tracks the mails sent.	To be tested	No	No	No	No
Google Calendar Analysis	Analyze calendar meeting invites for the day and provide an analysis via Email.	Google Calendar Toolkit, Email Toolkit, Write File, Read File	GPT-4	The agent should provide a rundown of meetings and time availability.	The agent got stuck in a loop while making sense of events	Yes	Yes	No	No
Investment Research	Send personalized mails to potential prospects.	Tools not built	GPT-4	To be tested	No	No	No	No	No
Email Summary	Provide a summary of unread mails every workday via Scheduled Agent Run and Recurring Run.	Email Toolkit, Write File, Read File	GPT-4	Mail to me or anyone with a summary of all unread emails.	Agent was able to make a summary of at least 5 emails and send it	Yes	Yes	Yes	No
Email Automatic Reply	Provide automatic draft replies to all unread mails via Scheduled Agent Run and Recurring Run.	Email Toolkit, Write File, Read File	GPT-4	Generate a draft email with an appropriate reply for all recipients.	Agent was able to save three emails as drafts with appropriate replies	Yes	Yes	Yes	No
Automatic Jira Ticket Creation	Create a Jira ticket automatically when a GitHub Issue is created.	Jira Toolkit, GitHub Toolkit	GPT-4	To generate a Jira Ticket with the description when a GitHub Issue is created	Jira tickets are working, but not connected to GitHub Tool	Yes	Yes	No	No