Commit Graph

27 Commits

Author SHA1 Message Date
Dmytro Liubarskyi 2101288ad7
EmbeddingModel: request/response API with per-call parameters, multimodal inputs, and observability (#5735)
## Issue
Closes #1153 — distinguish APIs for embedding queries vs. documents/keys
(adds `EmbeddingInputType.QUERY`/`DOCUMENT` as a per-call parameter,
plus opt-in `embeddingInputType(...)` on
  `EmbeddingStoreContentRetriever` / `EmbeddingStoreIngestor`).

Partially addresses #4019 — adds the multimodal image-embedding API at
the core level (`EmbeddingInput` of `Content` parts) and wires Cohere,
Voyage, Jina, Google (Gemini Embedding 2), and Bedrock Titan; does
  not implement it for `OnnxEmbeddingModel`.

Relates to #5142 — provider-specific / per-call parameters for OpenAI
embeddings (`OpenAiEmbeddingRequestParameters`: `user`,
`encodingFormat`, `customParameters`; e.g. NVIDIA NIM `input_type` via
custom
  parameters).

Relates to #4273 — observability for `EmbeddingModel` via listeners
(`EmbeddingModelListener` + request/response/error contexts, wired
across providers).

  ## Change

Introduces an `EmbeddingModel.embed(EmbeddingRequest) →
EmbeddingResponse` API, structured like `ChatModel`'s request/response
API, so embeddings can carry **per-call parameters** and **multimodal
inputs** and
participate in **observability**. Everything is additive and
`@Experimental`; the existing `embed(String)` / `embed(TextSegment)` /
`embedAll(List)` methods keep working unchanged.

  ### Core (`langchain4j-core`)
- New request/response types: `EmbeddingRequest`, `EmbeddingResponse`,
`EmbeddingResponseMetadata`, `EmbeddingRequestParameters` (+
`DefaultEmbeddingRequestParameters` and typed `EmbeddingParameter<T>`
  tokens), `EmbeddingInput`, `EmbeddingInputType`.
- New default methods on `EmbeddingModel`: `embed(EmbeddingRequest)`,
`doEmbed(...)`, `defaultRequestParameters()`, `supportedParameters()`,
`supportedContentTypes()`, `provider()`, `listeners()`.
- **Strict opt-in / fail-fast:** per-call parameters and content types
are token/type-checked; a request that uses something the model doesn't
declare is rejected with `UnsupportedFeatureException` instead of
being silently ignored. `overrideWith` preserves the provider-specific
parameters subtype (as on the chat side).
- **Multimodal:** an `EmbeddingInput` is an ordered list of `Content`
parts (text/image); models fuse them into one embedding (or
one-per-item, per provider). Modality is auto-detected — no manual flag.
- **Observability:** `EmbeddingModelListener` + request/response/error
contexts (same shape as `ChatModelListener`), fired inline from
`embed(EmbeddingRequest)`. `addListener(...)` still works.
- **RAG opt-in:** `EmbeddingStoreContentRetriever` and
`EmbeddingStoreIngestor` gain an optional `embeddingInputType(...)`
(QUERY / DOCUMENT). Default behavior is unchanged (no input type sent).
- `ModelProvider`: added `COHERE`, `VOYAGE_AI`, `JINA`, with matching
OpenTelemetry `gen_ai.provider.name` mappings (`cohere` is a well-known
OTel value; `voyage_ai` / `jina` are custom, as permitted by the
  spec).

### Providers
- **OpenAI** (dimensions, `user`/`encodingFormat`/custom params),
**Cohere** (Embed v4 multimodal + input types), **Voyage** (multimodal +
input types), **Jina** (CLIP multimodal), **Google AI Gemini** (input
types; **Gemini Embedding 2** multimodal), **Amazon Bedrock Titan**
(multimodal).
- **Google Gen AI** (`langchain4j-google-genai`): input type → SDK
`task_type`, per-call dimensions → `outputDimensionality`, `provider()`,
listeners.
- **Ollama**: text-only — `provider()` + listeners (per-call params
correctly fail fast).
- **In-process models** (ONNX / `AbstractInProcessEmbeddingModel`):
already work via the default `doEmbed→embedAll` bridge (text-only,
image/param requests fail fast); observability via `addListener(...)`.
No
code change (no builders to wire listeners into, no dedicated
`ModelProvider`).
- **Gemini Embedding 2** dropped the `task_type` parameter, so input
types are applied as prompt instructions (`task: search result | query:
…` / `title: none | text: …`) automatically; `gemini-embedding-001`
  still uses `task_type`.
- `modelName` in the response metadata reflects the API-reported model
where the provider returns one (OpenAI/Voyage/Jina), falling back to the
configured name.

  ### Tests
- `AbstractEmbeddingModelIT` — a shared IT base (like
`AbstractChatModelIT`) covering the new API, convenience methods,
listeners, and fail-fast; each provider adds a small
`common/…EmbeddingModelIT` that
parameterizes it and declares its capabilities via `supports*()`
overrides.
- Mock-based unit tests per provider for wire format / routing /
fail-fast (run in CI without keys), plus core value-type and listener
tests.

### Docs
- Embedding-model section in the RAG tutorial (request/response,
multimodal, query-vs-document opt-in), the EmbeddingModel listener
section in the Observability tutorial, the embedding contribution
guidance in
  `CONTRIBUTING.md`, and the six provider integration pages.

  ### Notes
- `EmbeddingResponseMetadata` intentionally has no `finishReason`
(embeddings have no finish reason). No real provider is affected: the
only provider that emits `STOP` (Cloudflare WorkersAI) overrides the
convenience methods directly, and every other provider always returned
`null` here.
- `revapi.json` suppressions were added where the new (non-breaking)
types are exposed in provider APIs.

  ## General checklist
  - [x] There are no breaking changes (API, behaviour)
  - [x] I have added unit and/or integration tests for my change
  - [x] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
  - [x] I have added/updated the documentation
- [ ] I have manually run all the unit and integration tests in the core
and main modules, and they are all green
- [ ] I have added an example in the examples repo (only for "big"
features)
  - [ ] I have added/updated Spring Boot starter(s) (if applicable)

---------

Co-authored-by: agent <agent@langchain4j.dev>
2026-07-09 22:09:34 +02:00
Dmytro Liubarskyi 5726381b65 updated contribution guidelines: how to open a PR 2026-06-29 11:10:02 +02:00
Sahal Hussain 48cc3bab69
spelling (#5002)
<!--
Thank you so much for your contribution!

Please fill in all the sections below.
Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples.
Please note that PRs with breaking changes or without tests will be
rejected.

Please note that PRs will be reviewed based on the priority of the
issues they address.
We ask for your patience. We are doing our best to review your PR as
quickly as possible.
Please refrain from pinging and asking when it will be reviewed. Thank
you for understanding!
-->

## Issue
<!-- Please specify the ID of the issue this PR is addressing. For
example: "Closes #1234" or "Fixes #1234" -->
Closes #5001 

## Change
<!-- Please describe the changes you made. -->
Small correction

We will triage is shortly ->We will triage it shortly
2026-04-23 09:45:37 +02:00
Taras dafdb4065e
[BUG] 4250 - integration tests fix (#4389)
## Issue
Fixes IT failures caused by commit ac8ad4c - per-part media resolution
is not supported by older Gemini models

## Change
This PR rolls back the per-part media resolution feature introduced in
#4337 (commit ac8ad4c).

The issue was that the mediaResolution field was being included in API
requests for all models, but this feature is only supported by Gemini
3.x models and gemini-2.5-computer-use models. Older models like
gemini-2.0-flash-lite reject requests with this field, causing the
error: "Per part media resolution is not supported for this Model."

### Changes made:
- Rolled back per-part media resolution feature
- Added/updated tests for model version detection and media resolution
inclusion/exclusion behavior.


## General checklist
- [X] There are no breaking changes (API, behaviour)
- [X] I have added unit and/or integration tests for my change
- [X] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the core
and main modules, and they are all green
- [ ] I have added/updated the documentation
- [ ] I have added an example in the examples repo (only for "big"
features)
- [ ] I have added/updated Spring Boot starter(s) (if applicable)
2026-01-15 10:17:39 +01:00
Dmytro Liubarskyi ba25afad1f updated contributing guide 2026-01-02 12:12:48 +01:00
Dmytro Liubarskyi 5e257e4a73 cleanup: removed traces of lombok 2026-01-02 12:08:35 +01:00
Dmytro Liubarskyi 3af58c3203
Update CONTRIBUTING.md 2026-01-02 12:06:09 +01:00
Dmytro Liubarskyi d9e0850a7e
Update CONTRIBUTING.md 2025-12-29 11:17:48 +01:00
Dmytro Liubarskyi 298e4076d4
added requirements for common ITs (#3665) 2025-09-10 12:20:08 +02:00
Guillaume Laforge 658ea9431a
Update CONTRIBUTING.md to mention spotless (#3230)
Add a link to suggest using spotless check & apply to enforce the coding
standard.
2025-06-18 15:05:39 +02:00
Dmytro Liubarskyi 8167e50573
Update CONTRIBUTING.md 2025-02-17 10:46:07 +01:00
Martin7-1 b7e7a64d3a
Community document update (#2382)
<!--
Thank you so much for your contribution!

Please fill in all the sections below.
Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples.
Please note that PRs with breaking changes or without tests will be
rejected.

Please note that PRs will be reviewed based on the priority of the
issues they address.
We ask for your patience. We are doing our best to review your PR as
quickly as possible.
Please refrain from pinging and asking when it will be reviewed. Thank
you for understanding!
-->

## Issue
<!-- Please specify the ID of the issue this PR is addressing. For
example: "Closes #1234" or "Fixes #1234" -->
Closes [community
issue#39](https://github.com/langchain4j/langchain4j-community/issues/39)

## Change
<!-- Please describe the changes you made. -->

1. Add configurable parameters in `DashScope` and `ZhiPuAi`.
2. Fix broken example link in the document.
3. How to migrate from main repo to community integrations.
4. Some introductions about community repo in the `intro.md`.
5. CONTRIBUTING.md update: consider adding integrations in community
repo first.

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)


## Checklist for adding new maven module
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the root `pom.xml` and
`langchain4j-bom/pom.xml`


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreRemovalIT` that
extends from `EmbeddingStoreWithRemovalIT`

## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2025-01-17 11:17:19 +01:00
Konstantin Pavlov 5706566222
#2109 Add Spotless Maven Plugin (#2153)
## Issue
Adds format consistency check for changed files. Closes #2109 

## Change

- Add [Spotless Maven plugin](https://github.com/diffplug/spotless) for
code java (eclipse java format) and pom.xml formatting
- Added the `<project.root.dir>` property across multiple POM files to
standardize the directory structure reference. This ensures consistent
project configurations and reduces redundancy in defining the root
project directory. Adjusted existing XML formatting minimally to
accommodate the new property integration.
- Cleanup duplicate settings in pgvector module
- Updated pom.xml files were reformatted according to
[defaults](https://github.com/diffplug/spotless/tree/main/plugin-maven#sortpom)
+ 4 spaces
- Adding linting and formatting commands to the `Makefile`, updating the
`pom.xml` files for better project configuration, and enhancing the
`CONTRIBUTING.md` guidelines.

## General checklist
- [x] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
2024-12-03 16:58:51 +01:00
LangChain4j d9c2f72885
CONTRIBUTING: added positive/negative test cases requirement 2024-11-29 17:56:06 +01:00
LangChain4j 4f5d4da06b
added a note about new integrations and community repo (#2199) 2024-11-27 12:24:55 +01:00
LangChain4j 535c6a8176
Update CONTRIBUTING.md
Retrofit -> Java 11 HTTP Client
2024-11-18 17:00:05 +01:00
Konstantin Pavlov 99dec54a59
#1636 Get rid of Lombok in langchain4j-core (#1987)
## Issue
Contributes to #1636 

## Change
Get rid of Lombok in langchain4j-core: Run Delombok refactoring in
IntelliJ IDEA to remove Lombok annotations and replace them with the
equivalent Java code.

This pull request focuses on removing the Lombok dependency and
replacing it with manually implemented builder patterns across several
classes. Additionally, it includes some minor code improvements.

## General checklist
- [x] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit tests in all modules, and they
are all green
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
2024-10-29 15:05:03 +01:00
LangChain4j 1327cb6588
Update CONTRIBUTING.md 2024-10-18 15:47:47 +02:00
Anush 2a5189a78a
feat: Metadata filtering Qdrant (#1646)
## Description
- Filtering support for Qdrant with tests for the converter.
- Updated metadata storage handling to support complex metadata types.
No longer limited to `Map<String, String>`.

Closes #1600 

## NOTE
In Qdrant,
- Eq, NEq, In, NIn don't allow float and double values. Only integers
and strings.
- LT, GT, LTE, GTE allow only numbers, not alphabets.
- For In and NIn conditions, if the key doesn't exist in the metadata,
it is not matched.
2024-09-12 11:18:28 +02:00
LangChain4j 116b7f484b
added note about lombok 2024-09-09 14:32:34 +02:00
LangChain4j c407ff9d85
Update CONTRIBUTING.md 2024-07-25 09:52:23 +02:00
LangChain4j 39c9dfdbc6
Gson -> Jackson 2024-06-07 12:24:01 +02:00
LangChain4j 6f18b38c93
Update CONTRIBUTING.md 2024-04-26 18:05:29 +02:00
LangChain4j 962dbbe885
Update CONTRIBUTING.md 2024-04-25 11:14:34 +02:00
LangChain4j 608f55bfb3
Update CONTRIBUTING.md 2024-04-18 21:23:09 +02:00
LangChain4j 2e29c47c6e
Update CONTRIBUTING.md 2024-04-18 16:21:54 +02:00
LangChain4j e5a4cb330e
Create CONTRIBUTING.md 2024-04-18 12:00:31 +02:00