Commit Graph

60 Commits

Author SHA1 Message Date
LangChain4j 481426ffb4 change version to 0.37.0-SNAPSHOT 2024-11-13 17:14:47 +01:00
LangChain4j 4d6755161d
Release 0.36.0 (#2104) 2024-11-13 15:48:51 +01:00
renovate[bot] b3c569612a
fix(deps): update dependency io.netty:netty-common to v4.1.115.final [security] (#2089)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [io.netty:netty-common](https://netty.io/)
([source](https://redirect.github.com/netty/netty)) | `4.1.110.Final` ->
`4.1.115.Final` |
[![age](https://developer.mend.io/api/mc/badges/age/maven/io.netty:netty-common/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/maven/io.netty:netty-common/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/maven/io.netty:netty-common/4.1.110.Final/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/io.netty:netty-common/4.1.110.Final/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [io.netty:netty-common](https://netty.io/)
([source](https://redirect.github.com/netty/netty)) | `4.1.111.Final` ->
`4.1.115.Final` |
[![age](https://developer.mend.io/api/mc/badges/age/maven/io.netty:netty-common/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/maven/io.netty:netty-common/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/maven/io.netty:netty-common/4.1.111.Final/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/io.netty:netty-common/4.1.111.Final/4.1.115.Final?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

### GitHub Vulnerability Alerts

####
[CVE-2024-47535](https://redirect.github.com/netty/netty/security/advisories/GHSA-xq3w-v528-46rv)

### Summary

An unsafe reading of environment file could potentially cause a denial
of service in Netty.
When loaded on an Windows application, Netty attemps to load a file that
does not exist. If an attacker creates such a large file, the Netty
application crash.

### Details

When the library netty is loaded in a java windows application, the
library tries to identify the system environnement in which it is
executed.

At this stage, Netty tries to load both `/etc/os-release` and
`/usr/lib/os-release` even though it is in a Windows environment.

<img width="364" alt="1"
src="https://github.com/user-attachments/assets/9466b181-9394-45a3-b0e3-1dcf105def59">

If netty finds this files, it reads them and loads them into memory.

By default :

- The JVM maximum memory size is set to 1 GB,
- A non-privileged user can create a directory at `C:\` and create files
within it.

<img width="340" alt="2"
src="https://github.com/user-attachments/assets/43b359a2-5871-4592-ae2b-ffc40ac76831">

<img width="523" alt="3"
src="https://github.com/user-attachments/assets/ad5c6eed-451c-4513-92d5-ba0eee7715c1">

the source code identified :

https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java

Despite the implementation of the function `normalizeOs()` the source
code not verify the OS before reading `C:\etc\os-release` and
`C:\usr\lib\os-release`.

### PoC

Create a file larger than 1 GB of data in `C:\etc\os-release` or
`C:\usr\lib\os-release` on a Windows environnement and start your Netty
application.

To observe what the application does with the file, the security analyst
used "Process Monitor" from the "Windows SysInternals" suite.
(https://learn.microsoft.com/en-us/sysinternals/)

```
cd C:\etc
fsutil file createnew os-release 3000000000
```

<img width="519" alt="4"
src="https://github.com/user-attachments/assets/39df22a3-462b-4fd0-af9a-aa30077ec08f">

<img width="517" alt="5"
src="https://github.com/user-attachments/assets/129dbd50-fc36-4da5-8eb1-582123fb528f">

The source code used is the Netty website code example : [Echo ‐ the
very basic client and
server](https://netty.io/4.1/xref/io/netty/example/echo/package-summary.html).

The vulnerability was tested on the 4.1.112.Final version.

The security analyst tried the same technique for
`C:\proc\sys\net\core\somaxconn` with a lot of values to impact Netty
but the only things that works is the "larger than 1 GB file" technique.
c0fdb8e9f8/common/src/main/java/io/netty/util/NetUtil.java (L186)

### Impact

By loading the "file larger than 1 GB" into the memory, the Netty
library exceeds the JVM memory limit and causes a crash in the java
Windows application.

This behaviour occurs 100% of the time in both Server mode and Client
mode if the large file exists.

Client mode :

<img width="449" alt="6"
src="https://github.com/user-attachments/assets/f8fe1ed0-1a42-4490-b9ed-dbc9af7804be">

Server mode :

<img width="464" alt="7"
src="https://github.com/user-attachments/assets/b34b42bd-4fbd-4170-b93a-d29ba87b88eb">

somaxconn :

<img width="532" alt="8"
src="https://github.com/user-attachments/assets/0656b3bb-32c6-4ae2-bff7-d93babba08a3">

### Severity

- Attack vector : "Local" because the attacker needs to be on the system
where the Netty application is running.
- Attack complexity : "Low" because the attacker only need to create a
massive file (regardless of its contents).
- Privileges required : "Low" because the attacker requires a user
account to exploit the vulnerability.
- User intercation : "None" because the administrator don't need to
accidentally click anywhere to trigger the vulnerability. Furthermore,
the exploitation works with defaults windows/AD settings.
- Scope : "Unchanged" because only Netty is affected by the
vulnerability.
- Confidentiality : "None" because no data is exposed through exploiting
the vulnerability.
- Integrity : "None" because the explotation of the vulnerability does
not allow editing, deleting or adding data elsewhere.
- Availability : "High" because the exploitation of this vulnerability
crashes the entire java application.

---

### Configuration

📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (no
schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/langchain4j/langchain4j).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS45LjUiLCJ1cGRhdGVkSW5WZXIiOiIzOS45LjUiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-13 10:31:26 +01:00
Konstantin Pavlov 1c481d13e2
Cleanup dependencies (#2059)
## Issue
Dependency cycles removed:
[langchain4j-core]->[langchain4j-parent]->[langchain4j-core]
This cycles prevented project fresh project correctly in Jetbrains Idea.
Cyclyc dependency between this repo and [langchain-embeddings
repo](https://github.com/langchain4j/langchain4j-embeddings) should be
fixed separately.

## Change

- Removed dependency cycles: Removed dependencies to project modules
from `langchain4j-parent`/dependencyManagement to avoid cycles. Added
explicit version number for
langchain4j-core,langchain4j,langchain4j-open-ai.
- Introduced `langchain4j-embeddings.version` for explicit dependency on
another repo.
- Added BOM dependencies to parent pom: junit-bom, mockito-bom. Remved
dependencies, defined in BOMs
- junit.version -> 5.11.3
- Upgraded maven-surefire-plugin, maven-failsafe-plugin. **NB! Renovate
bot is needed to do this automatically**

## General checklist
- [x] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
2024-11-12 09:46:25 +01:00
Konstantin Pavlov e7256f7b81
#1506 refactor enforcer plugin (#1923)
## Issue
Contributes to #1506 

## Change
This pull request involves the moving of the Maven Enforcer Plugin to
langchain4j-parent and the addition of a temporary property to skip
dependency convergence checks across multiple `pom.xml` files.

Enforcer plugin contains following rules enabled by default:
- [Require Maven version
3.8+](https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html)
-
[dependencyConvergence](https://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html)
-
[banDuplicatePomDependencyVersions](https://maven.apache.org/enforcer/enforcer-rules/banDuplicatePomDependencyVersions.html)
- Planned, but failing right now:
[requireUpperBoundDeps](https://maven.apache.org/enforcer/enforcer-rules/requireUpperBoundDeps.html)


The change can be tested locally with `mvn validate` command.

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
2024-10-16 17:38:47 +02:00
LangChain4j c6f4277916
New low-level API to specify tool parameters (#1668)
## Issue
Closes #896
Fixes #1480
Fixes #1881
Fixes #1858
Fixes #1865

## Context
Schema for tool parameters in the low-level API (`ChatLanguageModel` +
`ToolSpecification`) is currently specified using `ToolParameters` and
`JsonSchemaProperty` classes. This API works fine for simple cases, but
it does not work well for more complicated cases (e.g., when nested
objects or collections or recursion is required).

Moreover, [Structured
Outputs](https://github.com/langchain4j/langchain4j/pull/1590) feature
is using another API and a lot of logic between tools and structured
outputs is duplicated and out of sync.


## Change
`ToolParameters` and `JsonSchemaProperty` classes are now deprecated in
favour of new `JsonSchemaElement` API (which is used for Structured
Outputs as well). Most of the logic between this 2 features is now
unified.

`JsonSchemaElement` represents a schema for a JSON element. It can be of
such types:
- `JsonArraySchema`
- `JsonBooleanSchema`
- `JsonEnumSchema`
- `JsonIntegerSchema`
- `JsonNumberSchema`
- `JsonObjectSchema`
- `JsonReferenceSchema`
- `JsonStringSchema`

All model providers that support tools (see the list below) were updated
to support the new API. Old API is still supported for backward
compatibility, but will be removed in the future.

`ToolSpecifications` helper class now generates `ToolSpecification`s
using the new API only.

Example of the new API:
```java
ToolSpecification.builder()
    .name("weather")
    .description("Returns the current weather in the specified city")
    .parameters(JsonObjectSchema.builder()
        .addStringProperty("city", s -> s.description("The name of the city, e.g., Munich"))
        .addEnumProperty("units", TemperatureUnit.class)
        .required("city") // the required properties should be specified explicitly
        .build())
    .build();
```

`JsonObjectSchema` builder has some helper methods to make it easier to
add properties (e.g., `addProperty`, `addStringProperty`,
`addIntegerProperty`, etc.), but properties can also be specified using
a map:
```java
ToolSpecification.builder()
    .name("weather")
    .description("Returns the current weather in the specified city")
    .parameters(JsonObjectSchema.builder()
        .properties(Map.of(
            "city", JsonStringSchema.builder().description("The name of the city, e.g., Munich").build(),
            "units", JsonEnumSchema.builder().enumValues("CELSIUS", "FAHRENHEIT").build()
        ))
        .required("city") // the required properties should be specified explicitly
        .build())
    .build();
```

### Providers that support new `JsonSchemaElement` for tools
- [x] Amazon Bedrock
- [X] Anthropic
- [X] Azure OpenAI
- [X] DashScope
- [x] GitHub Models
- [X] Google AI Gemini
- [X] Google Vertex AI Gemini
- [x] Jlama
- [x] LocalAI
- [X] Mistral AI
- [X] Ollama
- [X] OpenAI
- [x] Qianfan
- [x] Zhipu AI

### Tested Providers
- [x] Amazon Bedrock
- [x] Anthropic
- [x] Azure OpenAI
- [x] DashScope
- [x] GitHub Models
- [x] Google AI Gemini
- [x] Google Vertex AI Gemini
- [x] Jlama
- [ ] LocalAI
- [x] Mistral AI
- [x] Ollama
- [x] OpenAI
- [x] Qianfan
- [x] Zhipu AI

### Additional Changes
Several bugs have been fixed along the way (see the list of Github
issues on the top).

### Breaking changes
- `ToolSpecification.parameters()` method has been renamed into
`toolParameters()`

### Potentially breaking changes
- `ToolSpecifications` helper class now generates `ToolSpecification`s
with parameters defined using the new API (`JsonObjectSchema`) instead
of the old API (`ToolParameters`).
- If `@Tool` does not have a description (`value` annotation attribute),
`ToolSpecifications` will return `ToolSpecification` with `description()
== null`

## General checklist
- [ ] There are no breaking changes (Most of the use cases should not
break)
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
2024-10-16 13:30:58 +02:00
Julien Dubois d2381254d1
Bump com.azure:azure-ai-inference from 1.0.0-beta.1 to 1.0.0-beta.2 (#1872)
This also refactors the pom.xml to use the API version in the parent
pom.xml, and to simplify dependencies.

This fixes an important bug from the Azure AI Inference SDK:
https://github.com/Azure/azure-sdk-for-java/issues/42036
2024-10-07 10:31:32 +02:00
LangChain4j 11855157dd updated version to 0.36.0-SNAPSHOT 2024-09-25 15:23:52 +02:00
LangChain4j 79f03dff36
Release 0.35.0 (#1829) 2024-09-25 13:16:03 +02:00
Julien Dubois d546c64511
Support for GitHub Models using the Azure AI Inference API (#1807)
Fix #1719

This adds GitHub Models (see https://github.com/marketplace/models )
support with the new Azure AI Inference API Java SDK.
2024-09-24 19:18:42 +02:00