Beyond Conversation: Teaching LLMs to Call Functions for Agentic Tasks

Large language models (LLMs) like ChatGPT can follow instructions because they’ve been trained to do just that. The training process usually happens in three main steps.

DISCLAIMER: This blogpost is based on my experiences as part of Bielik team - Polish Large Language Model (https://huggingface.co/speakleash/Bielik-11B-v2.3-Instruct)

First, the model learns how language works—how words, sentences, and ideas fit together. This is called the base training. It doesn’t yet know how to follow instructions, but it gets really good at understanding and generating human-like text.

Next comes instruction tuning. Here, the model is shown lots of examples where someone gives an instruction, and the model learns to respond helpfully. Because it already understands language well from the first phase, it can start generalizing to new instructions it hasn’t seen before.

After that, there are more advanced training techniques to make the model’s responses even more useful, accurate, and polished.

Now, if we want to build something more powerful—like an AI Agent that can do things for you, such as searching the internet or booking a calendar event—we need to teach the model how to use tools.

Think of a tool as a piece of software that performs a task. Just like humans use apps, LLMs can use tools by calling functions—little blocks of code with parameters that tell the tool exactly what to do.

Let’s say you want to search the web. A developer would create a function like search(query), where the “query” is what you want to look up, like "newest Booker nominations". From the model’s point of view, this is how it “uses Google.”

But here’s the tricky part: the model needs to learn how to recognize that your instruction—“Find the latest Booker Prize nominees”—should trigger that search() function, and it also needs to know how to fill in the right parameter in this case - "latest Booker Prize nominees'.

That’s why we create special datasets that teach the model how to use these tools. We give it lots of examples showing which function to use and how to fill in the blanks, so it can learn to use tools just as naturally as it learns to chat.

How I Created a Dataset to Teach Model to Use Functions

When I started learning how to train language models to use functions (also called function calling), I realized something surprising: even though there’s