spring AI (3) Function Calls

With the foundation of the previous two sections, it's time to step up the intensity and talk about Function Calling.

So what is Function Calling? The official explanation is:

You can register custom Java functions with the OpenAiChatClient and have the OpenAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions. This allows you to connect the LLM capabilities with external tools and APIs. The OpenAI models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.

With the OpenAiChatClient, you can register custom Java functions and have the OpenAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions. This allows you to connect the LLM capabilities with external tools and APIs. The OpenAI models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.

As we all know, OpenAI's ChatGPT cannot access the internet, but with Function Calling, we can provide the model with the latest data.

Imagine adding a 'weather' plugin to ChatGPT, telling GPT the latest weather in Beijing and Shanghai, and having GPT summarize what clothes you should wear. Or adding a 'trending' plugin to summarize the latest hot topics.

Creating Functions#

To create a function, you need to implement the Function interface.

public class MockWeatherService implements Function<MockWeatherService.Request, MockWeatherService.Response> {

    public enum Unit { C, F }
    public record Request(String location, Unit unit) {}
    public record Response(double temp, Unit unit) {}


    public Response apply(Request request) {
        System.err.println("MockWeatherService.apply");
        return new Response(30.0, Unit.C);
    }
}

Here, a fixed value of 30°C is returned, but you can actually call an interface to return real data.

Function Registration#

After creating the function, you need to register it, and there are two ways to do it.

Registering when creating the model#

You can register the function when creating the model, for example:

var openAiApi = new OpenAiApi("https://xxx", "sk-xxx");

OpenAiChatClient chatClient = new OpenAiChatClient(openAiApi, OpenAiChatOptions.builder()
                .withModel("gpt-3.5-turbo-1106")
                .withTemperature(0.4F)
                .withFunctionCallbacks(List.of(
                        FunctionCallbackWrapper.builder(new MockWeatherService())
                                .withName("weather").withDescription("Get the weather in location").build()))
                .build());

In the withFunctionCallbacks() method, you register the function. FunctionCallbackWrapper.builder is used to build the function, and withName("weather").withDescription("Get the weather in location") represents the name and description of the function. It is recommended to use English and describe the function's operation to help the AI determine when to use a certain function.

Dynamic Registration#

Another recommended way is dynamic registration, where you can select the corresponding function based on the prompt.

var promptOptions = OpenAiChatOptions.builder().withFunctionCallbacks(List.of(
                FunctionCallbackWrapper.builder(new MockWeatherService()).withName("weather").withDescription("Get the weather in location").build(),
                FunctionCallbackWrapper.builder(new WbHotService()).withName("wbHot").withDescription("Get the hot list of Weibo").build(),
                FunctionCallbackWrapper.builder(new TodayNews()).withName("todayNews").withDescription("60s watch world news").build())).build();


        ChatResponse response = chatClient.call(new Prompt(message, promptOptions));

        return response.getResult().getOutput().getContent();

Here, three functions are registered:

weather, which is the weather function created above.
wbHot, which calls an interface to get the hot list of Weibo.
todayNews, which calls an interface to get 60s world news.

Based on different prompts, the AI will choose the most appropriate function to call.

Here is an example of the Weibo hot list function:

public class WbHotService implements Function<WbHotService.Request, WbHotService.Response> {

    public record Request(String wb) {}
    public record Response(WbHot wbHot) {}

    /**
     * Applies this function to the given argument.
     *
     * @param request the function argument
     * @return the function result
     */
    @Override
    public Response apply(Request request) {

        System.err.println("微博热榜哦，表哥我进来了哦(*/ω＼*)(*/ω＼*)");

        // Use Hutool to request the interface
        String result = HttpUtil.get("https://api.vvhan.com/api/hotlist/wbHot");
        WbHot bean = JSONUtil.toBean(result, WbHot.class);
        return new Response(bean);
    }

}

In fact, there is no need to convert the JSON returned by the interface into an object. Just return it to the AI and it will handle it automatically.

However, there is currently an issue with Function Calling, which is that the result returned by the function is always processed by the AI.

For example, if I have a function that returns a URL of an image and I tell the AI "show me some pictures", and then call this function, the function is simple and just returns a URL of an image, so there is no need for the AI to process it. ~~What if the AI responds with "Wow, your taste is really weird" after seeing the image? 🤣~~ However, this issue has been raised before ISSUES.

Recommendation#

For specific applications of Function Calling, you can refer to this article:
Spring AI Application - Intelligent Reporter
The complete code is available in this gist.

Spring AI Chat Simple Example
Han Xiaohan API Interface Station - Free API Data Interface Calling Service Platform
Han Xiaohan API Interface Station -