Commit Graph

2 Commits

Author SHA1 Message Date
Dmytro Liubarskyi 29878c30e3
Migrate Cohere, HuggingFace, and Workers AI from Retrofit/OkHttp to the HttpClient abstraction (#5780)
## Change
Migrates three more integrations off Retrofit/OkHttp and onto
LangChain4j's own `HttpClient` abstraction
(`dev.langchain4j.http.client`), consistent with the already-migrated
modules (OpenAI, Anthropic,
Mistral, Gemini, Voyage). Each module drops `retrofit`, `okhttp`, and
`converter-jackson`, routes HTTP through the pluggable backend (default
JDK client at runtime), and replaces its hand-rolled logging
  interceptors with the abstraction's `LoggingHttpClient`.

  **`langchain4j-cohere`**
- `CohereClient` rewritten on `HttpClient`; added `CohereJsonUtils`
mirroring the exact ObjectMapper config (`SNAKE_CASE` + `NON_NULL` +
lenient — DTOs have no `@JsonProperty`, so the wire format is
preserved). Endpoints unchanged: `POST embed` (v1 + v2), `POST rerank`;
`Authorization: Bearer` preserved.
- Deleted `CohereApi` + both interceptors. Added
`httpClientBuilder(...)` to `CohereClient`, `CohereEmbeddingModel`, and
`CohereScoringModel`.
- `proxy(...)` (public on `CohereScoringModel` + a deprecated
constructor) can't be expressed through the abstraction, so it now
**throws `UnsupportedOperationException`** pointing users to
`httpClientBuilder(...)` rather than silently ignoring the proxy. The
builder method is `@Deprecated`.

  **`langchain4j-hugging-face`**
- `DefaultHuggingFaceClient` rewritten on `HttpClient`; added
`HuggingFaceJsonUtils` (plain mapper — DTOs self-annotate with
`@JsonNaming`). The `Authorization: Bearer` header from
`ApiKeyInsertingInterceptor`
  is replicated on each request. Endpoints unchanged.
- Deleted `HuggingFaceApi` + `ApiKeyInsertingInterceptor`. The
`HuggingFaceClient` interface and the `HuggingFaceClientFactory` SPI
method are unchanged; `HuggingFaceClientFactory.Input` gains a
**`default
HttpClientBuilder httpClientBuilder()`** (backward-compatible — existing
SPI implementors, e.g. quarkus-langchain4j, inherit the `null` default
and are unaffected). Added `httpClientBuilder(...)` to the
  chat/language/embedding model builders.

  **`langchain4j-workers-ai`**
- `WorkersAiClient` rewritten on `HttpClient`; added
`WorkersAiJsonUtils`. Endpoints/paths/`Authorization` preserved. Image
generation returns raw bytes via `SuccessfulHttpResponse.bodyBytes()`
(no String
round-trip). Cloudflare's `HTTP 200` + `{"success":false}` error
envelope is still surfaced (via a restored `checkSuccess()`), alongside
the automatic `HttpException` for non-2xx.
- Deleted `WorkersAiApi`; added `httpClientBuilder(...)` to all four
model builders. Some internal `.client` plumbing was removed
(`WorkersAiClient` no-arg constructor / `createService` /
`AuthInterceptor`,
and `AbstractWorkersAIModel.processErrors`/`workerAiClient`) —
documented in `revapi.json`.

Each module got a `revapi.json` entry for the intentional
`HttpClientBuilder` exposure (and removals), matching the existing
convention in `langchain4j-chroma`, `langchain4j-open-ai`,
`langchain4j-voyage-ai`,
  etc.

**Behavioral note:** on non-2xx responses the clients now throw
`dev.langchain4j.exception.HttpException` (a `RuntimeException`
subclass) instead of a plain `RuntimeException` with a `"status code:
…"` message
  — the same behavior all other migrated modules already exhibit.

## General checklist
  - [X] There are no breaking changes (API, behaviour)
  - [ ] I have added unit and/or integration tests for my change
  - [ ] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and

[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
2026-07-16 10:19:19 +02:00
Dmytro Liubarskyi 2101288ad7
EmbeddingModel: request/response API with per-call parameters, multimodal inputs, and observability (#5735)
## Issue
Closes #1153 — distinguish APIs for embedding queries vs. documents/keys
(adds `EmbeddingInputType.QUERY`/`DOCUMENT` as a per-call parameter,
plus opt-in `embeddingInputType(...)` on
  `EmbeddingStoreContentRetriever` / `EmbeddingStoreIngestor`).

Partially addresses #4019 — adds the multimodal image-embedding API at
the core level (`EmbeddingInput` of `Content` parts) and wires Cohere,
Voyage, Jina, Google (Gemini Embedding 2), and Bedrock Titan; does
  not implement it for `OnnxEmbeddingModel`.

Relates to #5142 — provider-specific / per-call parameters for OpenAI
embeddings (`OpenAiEmbeddingRequestParameters`: `user`,
`encodingFormat`, `customParameters`; e.g. NVIDIA NIM `input_type` via
custom
  parameters).

Relates to #4273 — observability for `EmbeddingModel` via listeners
(`EmbeddingModelListener` + request/response/error contexts, wired
across providers).

  ## Change

Introduces an `EmbeddingModel.embed(EmbeddingRequest) →
EmbeddingResponse` API, structured like `ChatModel`'s request/response
API, so embeddings can carry **per-call parameters** and **multimodal
inputs** and
participate in **observability**. Everything is additive and
`@Experimental`; the existing `embed(String)` / `embed(TextSegment)` /
`embedAll(List)` methods keep working unchanged.

  ### Core (`langchain4j-core`)
- New request/response types: `EmbeddingRequest`, `EmbeddingResponse`,
`EmbeddingResponseMetadata`, `EmbeddingRequestParameters` (+
`DefaultEmbeddingRequestParameters` and typed `EmbeddingParameter<T>`
  tokens), `EmbeddingInput`, `EmbeddingInputType`.
- New default methods on `EmbeddingModel`: `embed(EmbeddingRequest)`,
`doEmbed(...)`, `defaultRequestParameters()`, `supportedParameters()`,
`supportedContentTypes()`, `provider()`, `listeners()`.
- **Strict opt-in / fail-fast:** per-call parameters and content types
are token/type-checked; a request that uses something the model doesn't
declare is rejected with `UnsupportedFeatureException` instead of
being silently ignored. `overrideWith` preserves the provider-specific
parameters subtype (as on the chat side).
- **Multimodal:** an `EmbeddingInput` is an ordered list of `Content`
parts (text/image); models fuse them into one embedding (or
one-per-item, per provider). Modality is auto-detected — no manual flag.
- **Observability:** `EmbeddingModelListener` + request/response/error
contexts (same shape as `ChatModelListener`), fired inline from
`embed(EmbeddingRequest)`. `addListener(...)` still works.
- **RAG opt-in:** `EmbeddingStoreContentRetriever` and
`EmbeddingStoreIngestor` gain an optional `embeddingInputType(...)`
(QUERY / DOCUMENT). Default behavior is unchanged (no input type sent).
- `ModelProvider`: added `COHERE`, `VOYAGE_AI`, `JINA`, with matching
OpenTelemetry `gen_ai.provider.name` mappings (`cohere` is a well-known
OTel value; `voyage_ai` / `jina` are custom, as permitted by the
  spec).

### Providers
- **OpenAI** (dimensions, `user`/`encodingFormat`/custom params),
**Cohere** (Embed v4 multimodal + input types), **Voyage** (multimodal +
input types), **Jina** (CLIP multimodal), **Google AI Gemini** (input
types; **Gemini Embedding 2** multimodal), **Amazon Bedrock Titan**
(multimodal).
- **Google Gen AI** (`langchain4j-google-genai`): input type → SDK
`task_type`, per-call dimensions → `outputDimensionality`, `provider()`,
listeners.
- **Ollama**: text-only — `provider()` + listeners (per-call params
correctly fail fast).
- **In-process models** (ONNX / `AbstractInProcessEmbeddingModel`):
already work via the default `doEmbed→embedAll` bridge (text-only,
image/param requests fail fast); observability via `addListener(...)`.
No
code change (no builders to wire listeners into, no dedicated
`ModelProvider`).
- **Gemini Embedding 2** dropped the `task_type` parameter, so input
types are applied as prompt instructions (`task: search result | query:
…` / `title: none | text: …`) automatically; `gemini-embedding-001`
  still uses `task_type`.
- `modelName` in the response metadata reflects the API-reported model
where the provider returns one (OpenAI/Voyage/Jina), falling back to the
configured name.

  ### Tests
- `AbstractEmbeddingModelIT` — a shared IT base (like
`AbstractChatModelIT`) covering the new API, convenience methods,
listeners, and fail-fast; each provider adds a small
`common/…EmbeddingModelIT` that
parameterizes it and declares its capabilities via `supports*()`
overrides.
- Mock-based unit tests per provider for wire format / routing /
fail-fast (run in CI without keys), plus core value-type and listener
tests.

### Docs
- Embedding-model section in the RAG tutorial (request/response,
multimodal, query-vs-document opt-in), the EmbeddingModel listener
section in the Observability tutorial, the embedding contribution
guidance in
  `CONTRIBUTING.md`, and the six provider integration pages.

  ### Notes
- `EmbeddingResponseMetadata` intentionally has no `finishReason`
(embeddings have no finish reason). No real provider is affected: the
only provider that emits `STOP` (Cloudflare WorkersAI) overrides the
convenience methods directly, and every other provider always returned
`null` here.
- `revapi.json` suppressions were added where the new (non-breaking)
types are exposed in provider APIs.

  ## General checklist
  - [x] There are no breaking changes (API, behaviour)
  - [x] I have added unit and/or integration tests for my change
  - [x] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
  - [x] I have added/updated the documentation
- [ ] I have manually run all the unit and integration tests in the core
and main modules, and they are all green
- [ ] I have added an example in the examples repo (only for "big"
features)
  - [ ] I have added/updated Spring Boot starter(s) (if applicable)

---------

Co-authored-by: agent <agent@langchain4j.dev>
2026-07-09 22:09:34 +02:00