Commit Graph

864 Commits

Author SHA1 Message Date
LangChain4j 33a49275b7 Docu: added EmbeddingStore removal methods 2024-05-25 19:59:21 +02:00
LangChain4j 6b96848d2e Docu: added "Deleting Embeddings" column to embedding store comparison table 2024-05-25 19:43:24 +02:00
LangChain4j a1b733d96d bumped version to 0.32.0-SNAPSHOT 2024-05-24 16:25:13 +02:00
jiangsier-xyz 4398395e58
Support Embedding for Large Amounts of Texts (#1142)
fix: #1140

<!-- Thank you so much for your contribution! -->
<!-- Please fill in all the sections below. -->

<!-- Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples. -->
<!-- Please note that PRs with breaking changes will be rejected. -->
<!-- Please note that PRs without tests will be rejected. -->

<!-- Please note that PRs will be reviewed based on the priority of the
issues they address. -->
<!-- We ask for your patience. We are doing our best to review your PR
as quickly as possible. -->
<!-- Please refrain from pinging and asking when it will be reviewed.
Thank you for understanding! -->


## Issue
<!-- Please paste the link to the issue this PR is addressing. For
example: https://github.com/langchain4j/langchain4j/issues/1012 -->


## Change
<!-- Please describe the changes you made. -->


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-24 14:43:13 +02:00
jiangsier-xyz 463a3a3280
Support Blank Strings Tokenization (#1150)
fix: #1147

<!-- Thank you so much for your contribution! -->
<!-- Please fill in all the sections below. -->

<!-- Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples. -->
<!-- Please note that PRs with breaking changes will be rejected. -->
<!-- Please note that PRs without tests will be rejected. -->

<!-- Please note that PRs will be reviewed based on the priority of the
issues they address. -->
<!-- We ask for your patience. We are doing our best to review your PR
as quickly as possible. -->
<!-- Please refrain from pinging and asking when it will be reviewed.
Thank you for understanding! -->


## Issue
<!-- Please paste the link to the issue this PR is addressing. For
example: https://github.com/langchain4j/langchain4j/issues/1012 -->
https://github.com/langchain4j/langchain4j/issues/1147

## Change
<!-- Please describe the changes you made. -->
It seems that Qwen's online tokenizer service doesn't support strings
made up of whitespace characters.

Workaround: when the input is detected to be a string composed of
whitespace characters, add a character to the end of the string before
calling the API. Then, subtract 1 from the result returned by the API to
get the final result.

Additionally, the same test case has been added to
OpenAiTokenizerTest.java.

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-24 14:40:28 +02:00
小宇~ 7639a7558d
Update qianfan.md (#1149)
Update qianfan.md
2024-05-24 14:38:56 +02:00
Yellow-- b6a0c41189
Qianfan Models support http proxy setting(#951) (#1159)
## Issue
https://github.com/langchain4j/langchain4j/issues/951


## Change
- ```QianfanChatModel``` support proxy now
- ```QianfanEmbeddingModel``` support proxy now
- ```QianfanLanguageModel``` support proxy now
- ```QianfanStreamingChatModel``` support proxy now


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [X] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j

Co-authored-by: jianye.zheng <jianye.zheng@xuncetech.com>
2024-05-24 14:27:50 +02:00
jiangsier-xyz 4a325d75b6
Add langchain4j-embeddings-bge-small-zh-v15 into bom (#1154)
And fix a typo error of langchain4j-embeddings-bge-small-en-v15.
2024-05-24 13:59:44 +02:00
二毛 ab9b22cf69
Add image model (#1152)
## Issue
<!-- Please paste the link to the issue this PR is addressing. For
example: https://github.com/langchain4j/langchain4j/issues/1012 -->
#1088 

## Change
<!-- Please describe the changes you made. -->
Add ZhipuAi image model implementation `ZhipuAiImageModel`
Add ZhipuAi image model unit tests `ZhipuAiImageModelIT`

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-24 13:46:40 +02:00
Guillaume Laforge 6293c8f57e
fixes #827 (#1158) 2024-05-24 13:45:29 +02:00
LangChain4j d9cb1e9b81
Release 0.31.0 (#1151) 2024-05-23 17:40:52 +02:00
LangChain4j 358f63c0fa PGVector, remove by ID, same impl as #1020 but on top of last dev. (#1113)
#301: Add support for remove operations in EmbeddingStore (PGVector)
2024-05-23 11:53:42 +02:00
humcqc 13487864c7
PGVector, remove by ID, same impl as #1020 but on top of last dev. (#1113)
same impl as #1020 but on top of last dev.
2024-05-23 11:52:14 +02:00
LangChain4j 240d34df89 Adds an embedding store for Azure Cosmos DB for NoSQL (#1115) 2024-05-23 11:23:55 +02:00
Aayush Kataria 9e382d4f33
Adds an embedding store for Azure Cosmos DB for NoSQL (#1115)
## Issue
This PR add supports for Azure Cosmos DB for NoSQL embedding store.


## Change
- This PR adds an embedding store for Azure Cosmos DB for NoSql. The
test cases and IT test case is also included.


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [x] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-23 11:06:11 +02:00
LangChain4j 2a4576f1ed removed experimental annotations, added deprecations 2024-05-23 10:50:11 +02:00
二毛 c27c127912
Fix: #950 ZhipuAiChatModel does not support tools without parameters (#999)
## Context
Fix: #950

## Checklist
Before submitting this PR, please check the following points:
- [x] I have added unit and integration tests for my change
- [x] All unit and integration tests in the module I have added/changed
are green
2024-05-22 17:33:55 +02:00
Carlos Zela Bueno acdefd34b0
[Documentation] Mistral open mixtral8x22b docs (#979)
Mistral AI documentation updated with new open source model and some
references.
2024-05-22 17:32:43 +02:00
Carlos Zela Bueno 4f307520f7
New Mistral open mixtral8x22b model (#978)
Hi @langchain4j 
Yesterday Mistral AI released a new open source model that supports all
the features known so far.

I added tests and updated code to support it.

Documentation update coming in this
[PR#979](https://github.com/langchain4j/langchain4j/pull/979)
2024-05-22 17:28:33 +02:00
LangChain4j b3d22b499b Add gpt-4o to models enum (#1099) 2024-05-22 17:27:10 +02:00
kuraleta 426cffe475
Add gpt-4o to models enum (#1099) 2024-05-22 17:22:43 +02:00
LangChain4j 31be4a8c5f Jina AI Embedding model integration (#997) 2024-05-22 17:20:49 +02:00
LangChain4j 83606feab9
#866: Support embedding using Cohere (#1145)
## Issue
https://github.com/langchain4j/langchain4j/issues/866


## Change
Copied `CohereEmbeddingModel` from
https://github.com/stephanj/langchain4j-cohere
Kudos to @stephanj for the implementation 🤗 


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [X] There are no breaking changes
- [X] I have added unit and integration tests for my change
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [X] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
2024-05-22 17:18:35 +02:00
LangChain4j f00b48e961 Migrate Anthropic from Gson to Jackson (#1043) 2024-05-22 16:54:17 +02:00
Julien Dubois cea41bd60a
Support GTP-4o with Azure OpenAI (#1144)
GPT-4o on Azure OpenAI was announced at Build yesterday, and it's
already working with LangChain4J!

This shows that the current code works with GPT-4o (all integration
tests are green on my side), with 2 downsides for the moment:

- It's not available in the Sweden Central region, which I typically use
as there are more models there: this is why I'm using East US, and had
to remove GPT 3.5 Turbo which isn't available there (then, GPT-4o
basically replaces both GPT 3.5 Turbo and GPT 4, so it makes sense)
- The CLI commands in the `deploy-azure-openai-models.sh` script
currently complains that the API version isn't correct, as it's not
updated yet. It should be super soon, and otherwise the trick is to
create it using the Azure AI Studio.

Those 2 issues should be very temporary.
2024-05-22 15:40:35 +02:00
LangChain4j 92c5ed0297 Adding jina as scoring model. Feature request: #974 (#1053) 2024-05-22 15:38:30 +02:00
Kevin Smeyers 70d1b4a3b0
Adding jina as scoring model. Feature request: #974 (#1053)
Module added according #973 in order to minimize conflicts.
Implementation in line with cohere reranking

<!-- Thank you so much for your contribution! -->
<!-- Please fill in all the sections below. -->

<!-- Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples. -->
<!-- Please note that PRs with breaking changes will be rejected. -->
<!-- Please note that PRs without tests will be rejected. -->

<!-- Please note that PRs will be reviewed based on the priority of the
issues they address. -->
<!-- We ask for your patience. We are doing our best to review your PR
as quickly as possible. -->
<!-- Please refrain from pinging and asking when it will be reviewed.
Thank you for understanding! -->


## Issue
https://github.com/langchain4j/langchain4j/issues/974


## Change

- Adding Jina ai as a scoring model.
- Default model: jina-reranker-v1-base-en
- Default url: https://api.jina.ai/v1/
- Used Jackson object mapper instead of Gson, as I found it it's
langchain4j intent to move away from Gson
- Tried to keep implementation in line with Cohere reranking module

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)

---------

Co-authored-by: Kevin Smeyers <kevin.smeyers@audlau.be>
Co-authored-by: LangChain4j <langchain4j@gmail.com>
2024-05-22 14:55:02 +02:00
Michał Koźmiński c2a152000b
Freature #1005 - Add streaming API for Bedrock Anthropics (#1006)
## Context
Feature adds Bedrock Antrhopics Streaming capability #1005 

Previous PR https://github.com/langchain4j/langchain4j/pull/679
## Change
Added new streaming model `AbstractBedrockStreamingChatModel`

## Checklist
Before submitting this PR, please check the following points:
- [x] I have added unit and integration tests for my change
- [x] All unit and integration tests in the module I have added/changed
are green
- [x] All unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules are green
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)
(only when a new module is added)

## Checklist for adding new embedding store integration
- [ ] I have added a {NameOfIntegration}EmbeddingStoreIT that extends
from either EmbeddingStoreIT or EmbeddingStoreWithFilteringIT
2024-05-22 14:31:47 +02:00
LangChain4j 050e93b689 Jina AI Embedding model integration (#997) 2024-05-22 13:52:09 +02:00
Pankaj Shukla 0ad92d5b0c
Jina AI Embedding model integration (#997)
## Context
This pr is for integration of jina ai embedding model which is mentioned
in the issue
[973](https://github.com/langchain4j/langchain4j/issues/973)

## Change
1. Since no jina sdk was available for java hence built client for the
same .
2. Added method of embedding generation for both single input and
multiple inputs
3. Default model used _jina-embeddings-v2-base-en_

## Checklist
Before submitting this PR, please check the following points:
- [x] I have added unit and integration tests for my change
- [x] All unit and integration tests in the module I have added/changed
are green
- [x] All unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules are green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)
(only when a new module is added)

## Checklist for adding new embedding store integration
- [ ] I have added a {NameOfIntegration}EmbeddingStoreIT that extends
from either EmbeddingStoreIT or EmbeddingStoreWithFilteringIT
2024-05-22 13:36:16 +02:00
LangChain4j e2239639a9 Ollama: fix ITs 2024-05-22 13:29:01 +02:00
LangChain4j 0484e594e5 Ollama: log requests and responses (#662) 2024-05-22 13:28:49 +02:00
Hashcon 15b58ad756
Ollama: log requests and responses (#662) 2024-05-22 13:19:09 +02:00
LangChain4j 6818e279bf
LLM Observability: Part 1 (#1058)
## Issue
https://github.com/langchain4j/langchain4j/issues/199


## Change
- Added `ModelListener`, `ChatLanguageModelRequest`, and
`ChatLanguageModelResponse` that are compatible (have all the required
attributes) with [OTEL LLM semconv
draft](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/llm-spans.md).
- Added an option to attach multiple
`ModelListener<ChatLanguageModelRequest, ChatLanguageModelResponse>` to
`OpenAiChatModel` and `OpenAiStreamingChatModel` (pilot module).

### `ChatLanguageModelRequest`
```java
public class ChatLanguageModelRequest {

    private final String model;
    private final Double temperature;
    private final Double topP;
    private final Integer maxTokens;
    private final List<ChatMessage> messages;
    private final List<ToolSpecification> toolSpecifications;
}
```

### `ChatLanguageModelResponse`
```java
public class ChatLanguageModelResponse {

    private final String id;
    private final String model;
    private final TokenUsage tokenUsage;
    private final FinishReason finishReason;
    private final AiMessage aiMessage;
}
```

## Example
```java
ModelListener<ChatLanguageModelRequest, ChatLanguageModelResponse> modelListener =
                new ModelListener<ChatLanguageModelRequest, ChatLanguageModelResponse>() {

                    @Override
                    public void onRequest(ChatLanguageModelRequest request) {
                        // handle request
                    }

                    @Override
                    public void onResponse(ChatLanguageModelResponse response, ChatLanguageModelRequest request) {
                        // handle response
                    }

                    @Override
                    public void onError(Throwable error, ChatLanguageModelResponse response, ChatLanguageModelRequest request) {
                        // handle error
                    }
                };

        OpenAiChatModel model = OpenAiChatModel.builder()
                .apiKey(...)
                .listeners(singletonList(modelListener))
                .build();
```

## General checklist
- [X] There are no breaking changes
- [X] I have added unit and integration tests for my change
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
2024-05-22 13:14:11 +02:00
Guillaume Laforge a49ac33519
Gemini and Vertex AI update (#1141)
- support system instructions
- support parallel function calling
- should avoid the allocation issue faced sometimes because of the
lifecycle of the underlying VertexAI client object
2024-05-22 13:00:52 +02:00
Julien Dubois 495344e351
AzureAiSearchEmbeddingStore - add "indexName" to the builder (#1084)
Fix #1062
2024-05-22 09:00:12 +02:00
Steven Pousty 64bf704e54
Instructions for adding base langchain4j dependency (#1138)
<!-- Thank you so much for your contribution! -->
<!-- Please fill in all the sections below. -->

<!-- Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples. -->
<!-- Please note that PRs with breaking changes will be rejected. -->
<!-- Please note that PRs without tests will be rejected. -->

<!-- Please note that PRs will be reviewed based on the priority of the
issues they address. -->
<!-- We ask for your patience. We are doing our best to review your PR
as quickly as possible. -->
<!-- Please refrain from pinging and asking when it will be reviewed.
Thank you for understanding! -->


## Issue
<!-- Please paste the link to the issue this PR is addressing. For
example: https://github.com/langchain4j/langchain4j/issues/1012 -->
None - just a doc issue


## Change
<!-- Please describe the changes you made. -->
The current getting started only shows using an integrations specific
library. At least for the Ollama integration, there are no Services in
the integration jar. A user needs to add the core jar but this is not
mentioned in any docs I could find.

It is demonstrated in the tutorials under the examples repo.

https://github.com/langchain4j/langchain4j-examples/blob/main/tutorials/pom.xml


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [X] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-22 08:32:38 +02:00
Mohamed AIT ABDERRAHMAN 2e5f993837
Rename judge0 engine package (#1136)
## Issue
Related to https://github.com/langchain4j/langchain4j/issues/1066


## Change
Move judge0 code execution engine package from `dev.langchain4j.code` to
`dev.langchain4j.code.judge0`


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-22 08:28:04 +02:00
LangChain4j 504aa173df
Experimental: RAG: SQL database content retriever (#1056)
## Issue
https://github.com/langchain4j/langchain4j/issues/232

## Change
An experimental `SqlDatabaseContentRetriever` has been added.

Simplest usage example:
```java
ContentRetriever contentRetriever = SqlDatabaseContentRetriever.builder()
    .dataSource(dataSource)
    .chatLanguageModel(openAiChatModel)
    .build();
```
In this case SQL dialect and table structure will be determined from the
`DataSource`.

But it can be customized:
```java
ContentRetriever contentRetriever = SqlDatabaseContentRetriever.builder()
    .dataSource(dataSource)
    .sqlDialect("PostgreSQL")
    .databaseStructure(...)
    .promptTemplate(...)
    .chatLanguageModel(openAiChatModel)
    .maxRetries(2)
    .build();
```

See `SqlDatabaseContentRetrieverIT` for a full example.

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [X] There are no breaking changes
- [X] I have added unit and integration tests for my change
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [X] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)
2024-05-21 16:49:02 +02:00
LangChain4j 4a13624eda Tavily Web Search Engine 2024-05-21 16:38:16 +02:00
kuraleta 3457784a29
Tavily Web Search Engine (#676) 2024-05-21 16:10:59 +02:00
LangChain4j 3897e565cc [FEATURE] Google web search integration (#641) 2024-05-21 15:37:30 +02:00
Carlos Zela Bueno 43274ff465
[FEATURE] Google web search integration (#641)
Integrating [Google Custom Search](https://developers.google.com/custom-search) as a `WebSearchEngine` and as a `Tool` for function calling
2024-05-21 14:05:21 +02:00
LangChain4j 788de9fd91
Fix #935 (#1119)
## Issue
Fix #935 

## Change
Replaced `TokenUsage.add(TokanUsage)` with static
`TokenUsage.sum(TokanUsage, TokanUsage)` to better handle cases when one
of the `TokanUsage` is absent (null).

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [X] There are no breaking changes
- [X] I have added unit and integration tests for my change
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
2024-05-21 11:45:50 +02:00
Julien Dubois 0c5a801d45
Upgrade com.azure:azure-storage-blob from 12.25.4 to 12.26.0 (#1131)
Co-authored-by: LangChain4j <langchain4j@gmail.com>
2024-05-21 10:46:55 +02:00
Julien Dubois d1285a08ef
Upgrade com.azure:azure-storage-common from 12.24.4 to 12.25.0. (#1130) 2024-05-21 10:45:53 +02:00
Antonio Goncalves b0f0a7bdef
Adding missing -azure-ai-search and azure-cosmos-mongo-vcore to the BOM (#1127)
I've adding two missing artefacts in the BOM (azure-ai-search and
azure-cosmos-mongo-vcore). And to make it easier to spot, I've sorted
the `pom.xml` files by artifact id
2024-05-21 10:39:29 +02:00
LangChain4j 5f39139783
Fix #1124: preserve metadata during query compression (#1133)
Fixes #1124

This pull request addresses the issue of query metadata being lost
during passing it through the `CompressingQueryTransformer` and
`ExpandingQueryTransformer`.
2024-05-21 10:33:36 +02:00
LangChain4j bc886d1e93 Extract Judge0 code execution engine as module 2024-05-21 08:50:11 +02:00
Mohamed AIT ABDERRAHMAN b27fd4d6ac
Extract Judge0 code execution engine as module (#1051)
## Issue
https://github.com/langchain4j/langchain4j/issues/1048

## Change
I extract these classes as new module
`langchain4j-code-execution-engine-judge0` :
- `Judge0JavaScriptEngine`
- `JavaScriptCodeFixer`
- `Judge0JavaScriptExecutionTool`
- `JavaScriptCodeFixerTest`

and I moved the `com.squareup.okhttp3:okhttp` dependency from the main
module to that new one.

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and integration tests for my change
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [x] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)


## Checklist for adding new model integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [x] I have added my new module in the
[BOM](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-bom/pom.xml)


## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2024-05-21 08:47:25 +02:00