What is a Tool Call? How AI agents take action

Hey, my name is Mahan. I'm the CMO of Naom. My name is Luka and I'm the CEO of Nairon. Luka AI can now write your emails, shape your code, and do a whole bunch of different things that it wasn't capable of doing before. And all of this is largely thanks to tool calls. Correct.

Correct. Probably one of the most advanced topics which we've come to. So about time to jump in. Perfect. So let's start from the foundations and go more in depth from there. So, a tool call is essentially the ability for an LLM to perform an action. Previously, LLMs only had one ability, which was to respond to the user with text. Now, they get to actually perform different actions. They perform actions via APIs. We spoke about them in the previous video. What an LLM does is they decide off of a query from the user whether to go with option A or option B. In this case, option A is to just write some text back to the user.

Option B is to perform a tool call. So in this case, the user asked, hey, can you please, you know, book a meeting for tomorrow? The LLM decided that is an action, not something that I respond with text. In this case, create the tool call. And there's a specific format of how tool calls are set up. So the LLM knows exactly how to execute that with the API largely off of system prompts.

For sure. So the tool calls are actually embedded in the system prompt. When a session actually starts, when you start a chat with an LLM, before you even send a message, the first message, the LLM is given instructions on how to use the tools. So, in this case, send an email message, um, send a database record or query a database, search the web. All of these are tool calls and many of the times your tool calls are actually nested with different providers. So, Gmail will have a uh MCP which will have multiple tool calls for all the actions that you can do on on Gmail. Search the Gmail, create new emails, send email.

All of these are actions that are nested inside the MCP which we'll get to in other videos soon. Um those are actually instructions inside the tool calls that can be executed and actioned. Let's take the example of booking a call on your calendar. Let's actually go step by step on what a tool call would look like when uh when you want that to happen.

Yeah. So, what it would look like is this example over here. We can see you come in and you basically ask the LLM, book a meeting with Sarah tomorrow at 2 p.m. What the LLM does is it picks the first tool call. That's just step number two. After picking the tool call, it actually needs to input the actual details that the tool call is asking for. So the tool call essentially says okay if you want to create a calendar invite you need to give me the date you need to give me the attendee and a bunch of other information. Some are optional and some are required. In this case it's tomorrow 2 p.m. with Sarah. And so what the LM does is it uses the API sends that tool call and then it reverts back to you and it says hey I've done this tool call. I'm done.

Okay. And just to recap, like here is a very basic diagram of what an AI without tool calls would look like versus one that actually has access to the tool calls. Yeah. And when you look at ChatGPT, Jad GPT actually uses tool calls inside the actual interface.

However, you can give it more tool calls and if you had a CRM or you had any other sort of proprietary tech, you can essentially just give chatb to use those tool calls as well. And so, yeah, you're totally right. You could have an AI agent that has no tool calls. Um, and it could just draft emails but in text format. Summarize nodes, explain a concept, but all in text format. Tool calls go beyond the chat interface.

Actually using other tech that exists. Making it multimodal. Of course, the example of booking a call on your calendar makes sense. Now, let's make it more advanced. What would a coding agent look like for example? So what a coding agent does and again when you look at the mechanisms under the hood they're exactly the same. It's just your query is more complex. So when you give a coding agent a goal that you need it to do like implement this dashboard. What the coding agent does is first looks up your codebase. So that's one tool called then identifies where in your codebase does it need to implement the dashboard.

Then it decides what the colors need to look like. Well where do colors go? Colors go in another file. So it writes. So when it writes to a file which means it updates a file it uses another tool call and all of this sort of like thinking and working thinking and working is this loop which you can see here in this diagram where the agent is essentially just asking itself well did I finish the goal and if not then keep working it's reasoning of course and when it comes to access to tools the more the marrier correct well with some caveat there it is a double-edged sword where if you have too little tools the agent might not have enough resources to complete the task.

But sometimes actually in reality if you give it too many tool calls it actually gets confused with all of the possible things that it could do as well as and we spoke about this on context windows in the previous uh videos is that you actually burn a lot of context window and a lot of tokens. You actually burn a lot of your context window and a lot of your tokens by giving it a bunch of tool calls that they don't need. Mhm.

And so what this looks like in reality, and we do this on a day-to-day basis, is we build AI agents. And so with a finance AI agent, I'm for sure not going to give it the CRM tool calls. And likewise with the AI um sales agent, I'm for sure not going to give it the finance tool calls. And so actually having more of a sort of niche approach with your AI agent strategy, and this is a topic which we'll cover in in later videos. Um we can actually have them be more specific. The sort of guidance is consider this like a junior employee.

Imagine on day one you just give him 20 different resources to look at and they could do 20 different things. It'd be extremely tough for them to actually give you the value that you're looking for. Exactly. And this entire video series is essentially building up to how you can effectively use AI agents for your organization. We'll eventually get to a place where we go very very in-depth on on those topics. But for the time being guys, you can subscribe to our newsletter links down in the description below. You can also follow us on LinkedIn. and we're very active on a day-to-day basis. Until then, we'll see you in the next video.