<!--
Thank you so much for your contribution!
Please fill in all the sections below.
Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples.
Please note that PRs with breaking changes or without tests will be
rejected.
Please note that PRs will be reviewed based on the priority of the
issues they address.
We ask for your patience. We are doing our best to review your PR as
quickly as possible.
Please refrain from pinging and asking when it will be reviewed. Thank
you for understanding!
-->
## Issue
<!-- Please specify the ID of the issue this PR is addressing. For
example: "Closes #1234" or "Fixes #1234" -->
Closes #
## Change
<!-- Please describe the changes you made. -->
Support structured output for github models, also remove deprecated
model and add deepSeek models.
## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Checklist for adding new maven module
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the root `pom.xml` and
`langchain4j-bom/pom.xml`
## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreRemovalIT` that
extends from `EmbeddingStoreWithRemovalIT`
## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
Co-authored-by: Dmytro Liubarskyi <ljubarskij@gmail.com>
## Issue
Azure OpenAI integration tests hit rate limits thus blocking CI:
Requests to the ChatCompletions_Create Operation under Azure OpenAI API
version 2025-01-01-preview have
exceeded call rate limit of your current OpenAI S0 pricing tier. Please
retry after 2 seconds.
Please go here: https://aka.ms/oai/quotaincrease if you would like to
further increase the default rate limit.
## Change
- Mark safe unit tests with `@Execution(ExecutionMode.CONCURRENT)`
- Switch to cheaper "gpt-4.1-nano" model in some tests
- Update propmt and condition for json format test to be less strict.
- Added `@EnabledIfEnvironmentVariable(named = "AZURE_OPENAI_KEY",
matches = ".+")` to relevant integraiton tests
- Use Junit Pioneer's
[`@RetryingTest`](https://junit-pioneer.org/docs/retrying-test/)
- Refactoring: Replace repetitive builder patterns with
AzureModelBuilders across IT classes. Set max(completion)Tokens to 100
by default to reduce token consumption and avoid rate limiting
- Enhance formatting
## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Checklist for adding new maven module
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the root `pom.xml` and
`langchain4j-bom/pom.xml`
## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreRemovalIT` that
extends from `EmbeddingStoreWithRemovalIT`
## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
## Issue
Fixes https://github.com/langchain4j/langchain4j/issues/2918
## Change
- Changed `maxRetry` parameter semantics from "max attempts" to "max
retries".
- Changed default value of the `maxRetry` parameter from 3 to 2, but it
does not change the default behaviour. When `maxRetries` parameter is
not specified explicitly, it will attempt to execute up to 3 times (as
it was before).
## Breaking Change
If you do **_not_** specify `maxRetries` parameter explicitly, there is
no breaking change and you do not need to do any changes to your code.
If you specify `maxRetries` parameter explicitly, you will need to
reduce it by 1, example:
```java
// before
OpenAiChatModel.builder()
.apiKey(System.getenv("OPENAI_API_KEY"))
.modelName(GPT_4_O_MINI)
.maxRetries(1)
.build();
// after
OpenAiChatModel.builder()
.apiKey(System.getenv("OPENAI_API_KEY"))
.modelName(GPT_4_O_MINI)
.maxRetries(0)
.build();
```
## General checklist
- [ ] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [X] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Change
Renamed `ChatLanguageModel` into `ChatModel` and
`StreamingChatLanguageModel` into `StreamingChatModel`.
All `chatLanguageModel(...)` methods were renamed into `chatModel(...)`,
all `streamingChatLanguageModel(...)` methods were renamed into
`streamingChatModel(...)`.
`DisabledChatLanguageModel` was renamed into `DisabledChatModel`,
`DisabledStreamingChatLanguageModel` into `DisabledStreamingChatModel`.
### OpenRewrite recipe:
```yml
---
type: specs.openrewrite.org/v1beta/recipe
name: dev.langchain4j.RenameChatModels
recipeList:
- org.openrewrite.java.ChangeType:
oldFullyQualifiedTypeName: dev.langchain4j.model.chat.ChatLanguageModel
newFullyQualifiedTypeName: dev.langchain4j.model.chat.ChatModel
- org.openrewrite.java.ChangeType:
oldFullyQualifiedTypeName: dev.langchain4j.model.chat.StreamingChatLanguageModel
newFullyQualifiedTypeName: dev.langchain4j.model.chat.StreamingChatModel
- org.openrewrite.java.ChangeType:
oldFullyQualifiedTypeName: dev.langchain4j.model.chat.DisabledChatLanguageModel
newFullyQualifiedTypeName: dev.langchain4j.model.chat.DisabledChatModel
- org.openrewrite.java.ChangeType:
oldFullyQualifiedTypeName: dev.langchain4j.model.chat.DisabledStreamingChatLanguageModel
newFullyQualifiedTypeName: dev.langchain4j.model.chat.DisabledStreamingChatModel
- org.openrewrite.java.ChangeMethodName:
methodPattern: dev.langchain4j..* chatLanguageModel(..)
newMethodName: chatModel
- org.openrewrite.java.ChangeMethodName:
methodPattern: dev.langchain4j..* streamingChatLanguageModel(..)
newMethodName: streamingChatModel
```
## General checklist
- [ ] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [x] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Changes
- Removed `JsonSchemaProperty` and `ToolParameters` classes which were
deprecated in previous releases. If you want to specify tool parameters
in `ToolSpecification`, use `JsonObjectSchema`:
```java
ToolSpecification toolSpecification = ToolSpecification.builder()
.name("getWeatherForecast")
.description("Get the weather forecast for a location")
.parameters(JsonObjectSchema.builder()
.addStringProperty("location", "the location to get the weather forecast for")
.required("location")
.build())
.build();
```
- Removed a few methods from `Tokenizer` to estimate token count of
tools (this functionality will not be supported any more)
- `AzureOpenAiStreamingChatModel`: removed the use of `Tokenizer`
methods that estimate token counts for tools (described above), so
estimated `TokenUsage` does not include tools now. This should be fixed
by https://github.com/langchain4j/langchain4j/issues/1068 in the future
## General checklist
- [ ] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
This follows the discussion at
https://github.com/langchain4j/langchain4j/pull/764
Here are my findings:
- New version of the OpenAI models are really good at filtering
violence/self harm/etc so it's very hard to hit the responsible AI
filters.
- To my surprise, other models are also quite good, including the
Mistral ones. For this tests, I'm using PHI 3.5 mini, as usually the
smallest models are less good at everything, which includes filtering.
- I used to test the JSON response with the Azure OpenAI API, which
included some information about the type of filters (violence, self
harm, etc), but I can't access this data with the Azure AI Inference
API. So now I'm testing with the "finish reason", which is less precise
but tells for sure if the content was filtered.
So I'm not super satisfied with this PR, but it works and does some
basic responsible AI tests, so we know the API works.
## Issue
Closes#896Fixes#1480Fixes#1881Fixes#1858Fixes#1865
## Context
Schema for tool parameters in the low-level API (`ChatLanguageModel` +
`ToolSpecification`) is currently specified using `ToolParameters` and
`JsonSchemaProperty` classes. This API works fine for simple cases, but
it does not work well for more complicated cases (e.g., when nested
objects or collections or recursion is required).
Moreover, [Structured
Outputs](https://github.com/langchain4j/langchain4j/pull/1590) feature
is using another API and a lot of logic between tools and structured
outputs is duplicated and out of sync.
## Change
`ToolParameters` and `JsonSchemaProperty` classes are now deprecated in
favour of new `JsonSchemaElement` API (which is used for Structured
Outputs as well). Most of the logic between this 2 features is now
unified.
`JsonSchemaElement` represents a schema for a JSON element. It can be of
such types:
- `JsonArraySchema`
- `JsonBooleanSchema`
- `JsonEnumSchema`
- `JsonIntegerSchema`
- `JsonNumberSchema`
- `JsonObjectSchema`
- `JsonReferenceSchema`
- `JsonStringSchema`
All model providers that support tools (see the list below) were updated
to support the new API. Old API is still supported for backward
compatibility, but will be removed in the future.
`ToolSpecifications` helper class now generates `ToolSpecification`s
using the new API only.
Example of the new API:
```java
ToolSpecification.builder()
.name("weather")
.description("Returns the current weather in the specified city")
.parameters(JsonObjectSchema.builder()
.addStringProperty("city", s -> s.description("The name of the city, e.g., Munich"))
.addEnumProperty("units", TemperatureUnit.class)
.required("city") // the required properties should be specified explicitly
.build())
.build();
```
`JsonObjectSchema` builder has some helper methods to make it easier to
add properties (e.g., `addProperty`, `addStringProperty`,
`addIntegerProperty`, etc.), but properties can also be specified using
a map:
```java
ToolSpecification.builder()
.name("weather")
.description("Returns the current weather in the specified city")
.parameters(JsonObjectSchema.builder()
.properties(Map.of(
"city", JsonStringSchema.builder().description("The name of the city, e.g., Munich").build(),
"units", JsonEnumSchema.builder().enumValues("CELSIUS", "FAHRENHEIT").build()
))
.required("city") // the required properties should be specified explicitly
.build())
.build();
```
### Providers that support new `JsonSchemaElement` for tools
- [x] Amazon Bedrock
- [X] Anthropic
- [X] Azure OpenAI
- [X] DashScope
- [x] GitHub Models
- [X] Google AI Gemini
- [X] Google Vertex AI Gemini
- [x] Jlama
- [x] LocalAI
- [X] Mistral AI
- [X] Ollama
- [X] OpenAI
- [x] Qianfan
- [x] Zhipu AI
### Tested Providers
- [x] Amazon Bedrock
- [x] Anthropic
- [x] Azure OpenAI
- [x] DashScope
- [x] GitHub Models
- [x] Google AI Gemini
- [x] Google Vertex AI Gemini
- [x] Jlama
- [ ] LocalAI
- [x] Mistral AI
- [x] Ollama
- [x] OpenAI
- [x] Qianfan
- [x] Zhipu AI
### Additional Changes
Several bugs have been fixed along the way (see the list of Github
issues on the top).
### Breaking changes
- `ToolSpecification.parameters()` method has been renamed into
`toolParameters()`
### Potentially breaking changes
- `ToolSpecifications` helper class now generates `ToolSpecification`s
with parameters defined using the new API (`JsonObjectSchema`) instead
of the old API (`ToolParameters`).
- If `@Tool` does not have a description (`value` annotation attribute),
`ToolSpecifications` will return `ToolSpecification` with `description()
== null`
## General checklist
- [ ] There are no breaking changes (Most of the use cases should not
break)
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
Dmytro Liubarskyi