Tool Calling

How it works

When you provide a set of tools to the model, it can choose to call one or more of them if it deems it necessary to answer a user's prompt. Instead of returning a text message, the model returns a “tool call” request. Your application then executes the tool with the arguments provided by the model, sends the result back to the model, and the model uses this result to generate its final, user-facing response. PuppyChatter abstracts away this back-and-forth communication, making the process seamless.

Example

This Java code provides a compelling example of a powerful AI feature known as “tool calling” or “function calling,” implemented using the PuppyChatter library. This capability allows a Large Language Model (LLM), like gpt-4o-mini used here, to go beyond generating text and interact with external code to perform specific tasks.

At its core, the code defines two custom functions, or “tools,” for the AI to use: relatedNumber and normalize. The descriptions provided for these tools (“get related number of the two numbers” and “normalize the given number”) act as instructions, helping the AI understand what each tool does and when to use it.

The ToolCallProcessor is the bridge between the AI's intent and the application's logic. When the user asks the AI to perform a task that matches a tool's description—in this case, “First calculate the related number of 1 and 2, and then use the normalize tool to calculate its normalize value”—the AI doesn't just guess the answer. Instead, it sends a request back to the application, asking it to execute the appropriate tool with the necessary arguments.

The PuppyChatter library seamlessly manages this multi-step conversation. It first receives a request from the AI to call relatedNumber(1, 2). The ToolCallProcessor executes this, returning 3. This result is sent back to the AI, which then requests the second tool call, normalize(3). The processor executes this, returning 9. (note that the functions defined in this example are for demonstration only, and have no intention to be mathematically accurate)

Finally, having used the external tools to gather all the necessary information, the AI formulates a final, user-facing response that incorporates these results. This example elegantly demonstrates how developers can augment AI models with custom functionalities, enabling them to perform complex, multi-step tasks and interact with external data and services.

When using the tool calling feature, remember to use a default model that supports it, such as gpt-4o-mini.

public class Test3 {
    public static void main(String[] args) throws Exception{
        OpenrouterPuppyChatter puppyChatter = new OpenrouterPuppyChatter(apiKey, 
            "openai/gpt-4o-mini", null);
        String sessionId = puppyChatter.createSession();
        Gson gson=new Gson();
        puppyChatter.setToolCallProcessor(new ToolCallProcessor() {
            @Override
            public String processToolCallRequest(ToolCallRequest toolCallRequest) {
                Map<String, Object> functionParameters = gson.fromJson(toolCallRequest.getFunction().getArguments(), Map.class);
                if(toolCallRequest.getFunction().getName().equals("relatedNumber")){
                    return String.valueOf(Double.valueOf(""+functionParameters.get("a"))+Double.valueOf(""+functionParameters.get("b")));
                }else{
                    Double input=Double.valueOf(""+functionParameters.get("input"));
                    return String.valueOf(Math.pow(input.intValue(), 2));
                }
            }
        });
        OpenrouterPromptParameters parameters = new OpenrouterPromptParameters("user");
        parameters.setTools(List.of(
            new Tool("relatedNumber", "get related number of the two numbers", FunctionParameters.class),
            new Tool("normalize", "normalize the given number", FunctionParameters2.class)
        ));
        Response response=puppyChatter.bark(sessionId, "model:openai/gpt-4o-mini First, calculate the related numbers for 1 and 2, and then use the normalization tool to compute their normalized values.", parameters);
        System.out.println(response.getMessage());
        // System.out.println(response.getToolCalls().get(0));
        puppyChatter.closeSession(sessionId);
    }

    public static class FunctionParameters {
        @JsonProperty(required = true)
        private int a;
        @JsonProperty(required = true)
        private int b;
        
        public FunctionParameters(int a, int b) {
            this.a = a;
            this.b = b;
        }
        public int getA() {
            return a;
        }
        public void setA(int a) {
            this.a = a;
        }
        public int getB() {
            return b;
        }
        public void setB(int b) {
            this.b = b;
        }

        
    }

    public static class FunctionParameters2 {
        @JsonProperty(required = true)
        private int input;
        
        public FunctionParameters2(int input) {
            this.input = input;
        }

        public int getInput() {
            return input;
        }

        public void setInput(int input) {
            this.input = input;
        }
        
    }
}