## Issue
Closes#5670
## Change
`SeleniumDocumentLoader.load(String url, DocumentParser)` encoded the
fetched page content with `pageContent.getBytes()`, using the
platform-default charset. `TextDocumentParser` decodes those bytes as
UTF-8 by default, so on a JVM whose default charset is not UTF-8 (Java
17 target, before JEP 400) non-ASCII page content was corrupted after
parsing.
This encodes explicitly with UTF-8, matching the parser's default
decoding and the repo convention (`Utils.java`, `UriUtils.java` already
use `getBytes(StandardCharsets.UTF_8)`):
```java
documentParser.parse(new ByteArrayInputStream(pageContent.getBytes(StandardCharsets.UTF_8)));
```
The change is limited to that one line plus the `StandardCharsets`
import. `load(String url)` is untouched: it wraps the raw string via
`Document.from(String, Metadata)` and never round-trips through bytes.
Added a credential-free unit test that mocks `WebDriver`, feeds
non-ASCII page content, captures the bytes the parser receives, and
asserts they equal the UTF-8 encoding — locking the contract
deterministically on any default charset.
Series probe: the sibling `PlaywrightDocumentLoader` has the identical
`getBytes()` pattern and is intentionally left for a follow-up PR once
this one merges.
## General checklist
- [X] There are no breaking changes (API, behaviour)
- [X] I have added unit and/or integration tests for my change
- [X] The tests cover both positive and negative cases
<!-- The added unit test asserts UTF-8 bytes (positive path) and the
metadata contract; the platform-default path was the bug being fixed.
-->
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
<!-- New unit test passes. The pre-existing SeleniumDocumentLoaderIT is
Testcontainers/browser-gated and was not run locally. -->
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- N/A — change is isolated to the selenium document-loader module.
-->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
<!-- N/A — bug fix, no doc change; per CONTRIBUTING, docs added after
review. -->
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
<!-- N/A — not a big feature. -->
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
<!-- N/A — no Spring Boot starter for this module. -->
<!-- Checklist for adding new maven module: omitted — no new module. -->
<!-- Checklist for new/changed embedding store integration: omitted —
not an embedding store. -->
### Summary
This PR adds an overloaded method `public Document load(String url)` to
`SeleniumDocumentLoader` to support loading a web document without
requiring a `DocumentParser`. The content is returned as-is (raw HTML or
plain text) and wrapped in a `Document` object with the source URL added
to its metadata.
### Changes
- Added `load(String url)` method that:
- Loads the page using the existing Selenium WebDriver setup.
- Captures the raw page content via `getPageSource()`.
- Returns a `Document` using `Document.from(String, Metadata)`.
- Enhanced Javadoc for:
- `load(String url)`
- Supporting public method `pageContent(String url)`
### Use Case
This overload simplifies use cases where:
- Consumers want raw content without applying parsing logic.
- Parsing will be applied manually or deferred.
- Basic HTML/text capture is sufficient.
Co-authored-by: Harikrishna <harikrishna.gurram@walmart.com>
## Change
- Added `maven-flatten-plugin` to `langchain4j-parent` and
`langchain4j-bom`
- Removed integration-specific dependencies from `langchain4j-parent`'s
`dependencyManagement` section and moved them to the modules where these
dependencies are used
- Explicitly added missing implicit dependencies
- Removed redundant `<maven.compiler.release>` for cassandra, infinispan
and opensearch modules
- Removed redundant license declarations and outdated properties
## General checklist
- [ ] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Change
Removed:
- `ChatMessage.text()`
- `Document.metadata(String)` -> Replaced by:
`Document.metadata().get{Type}(String)`
- `TextSegment.metadata(String)` -> Replaced by:
`TextSegment.metadata().get{Type}(String)`
- `Metadata.get(String)` -> Replaced by: `Metadata.get{Type}(String)`
- `Metadata.add(String, Object)` -> Replaced by: `Metadata.put(String,
{Type})`
- `Metadata.add(String, String)` -> Replaced by: `Metadata.put(String,
{Type})`
- `Metadata.asMap()` -> Replaced by: `Metadata.toMap()`
- `Metadata.from(String, Object)` -> Replaced by: `Metadata.from(String,
String)`
- `Metadata.metadata(String, Object)` -> Replaced by:
`Metadata.metadata(String, String)`
- `RetrievalAugmentor.augment(UserMessage, Metadata)` -> Replaced by:
`RetrievalAugmentor.augment(AugmentationRequest)`
- `ContentInjector.inject(List, UserMessage)` -> Replaced by:
`ContentInjector.inject(List, ChatMessage)`
- `Retriever` -> Replaced by: `ContentRetriever`
- `EmbeddingStoreRetriever` -> Replaced by:
`EmbeddingStoreContentRetriever`
- `Utils.isCollectionEmpty(Collection)` -> Replaced by:
`Utils.isNullOrEmpty(Collection)`
- `JsonObjectSchema.Builder.properties()` -> Replaced by:
`JsonObjectSchema.Builder.addProperties()`
## General checklist
- [ ] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Issue
Streamline maven dependencies
Should fix [this
issue](https://github.com/langchain4j/langchain4j/actions/runs/13666001227/job/38207360909?pr=2640#step:5:16630)
```
2025/03/05 00:52:59,050 3922 [INFO ] [main] netty.implementation.NettyUtility - {"az.sdk.message":"The following Netty versions were found on the classpath and have a mismatch with the versions used by azure-core-http-netty. If your application runs without issue this message can be ignored, otherwise please align the Netty versions used in your application. For more information, see https://aka.ms/azsdk/java/dependency/troubleshoot.","azure-netty-version":"4.1.118.Final","azure-netty-native-version":"2.0.70.Final","classpath-netty-version-io.netty:netty-common":"4.1.115.Final","classpath-netty-version-io.netty:netty-handler":"4.1.110.Final","classpath-netty-version-io.netty:netty-handler-proxy":"4.1.110.Final","classpath-netty-version-io.netty:netty-buffer":"4.1.110.Final","classpath-netty-version-io.netty:netty-codec":"4.1.110.Final","classpath-netty-version-io.netty:netty-codec-http":"4.1.110.Final","classpath-netty-version-io.netty:netty-codec-http2":"4.1.110.Final","classpath-netty-version-io.netty:netty-transport-native-unix-common":"4.1.110.Final","classpath-netty-version-io.netty:netty-transport-native-epoll":"4.1.110.Final","classpath-netty-version-io.netty:netty-transport-native-kqueue":"4.1.118.Final","classpath-native-netty-version-io.netty:netty-tcnative-boringssl-static":"2.0.70.Final"}
```
## Change
This pull request includes the following changes to improve project
dependency management:
- Removed unnecessary and redundant Netty dependencies and exclusions.
- Updated and reorganized dependency versions for consistency, including
new BOM imports for Azure SDK, Netty, and Reactor.
- Eliminated AWS SDK v1 dependencies, transitioning to
`software.amazon.awssdk` BOM for better compatibility and reduced
clutter.
- Removed redundant test dependencies like `assertj-core` and `mockito`
across modules, referring to the parent POM.
- Adjusted `kotlinx-coroutines-test` to use JVM-specific artifact and
standardized dependency version variables.
- Refined POM XML formatting for consistency and clarity (`mvn
spotless:apply`)
## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Checklist for adding new maven module
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the root `pom.xml` and
`langchain4j-bom/pom.xml`
## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreRemovalIT` that
extends from `EmbeddingStoreWithRemovalIT`
## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
---------
Co-authored-by: kpavlov <{ID}+{username}@users.noreply.github.com>
Co-authored-by: Dmytro Liubarskyi <ljubarskij@gmail.com>
## Issue
Adds format consistency check for changed files. Closes#2109
## Change
- Add [Spotless Maven plugin](https://github.com/diffplug/spotless) for
code java (eclipse java format) and pom.xml formatting
- Added the `<project.root.dir>` property across multiple POM files to
standardize the directory structure reference. This ensures consistent
project configurations and reduces redundancy in defining the root
project directory. Adjusted existing XML formatting minimally to
accommodate the new property integration.
- Cleanup duplicate settings in pgvector module
- Updated pom.xml files were reformatted according to
[defaults](https://github.com/diffplug/spotless/tree/main/plugin-maven#sortpom)
+ 4 spaces
- Adding linting and formatting commands to the `Makefile`, updating the
`pom.xml` files for better project configuration, and enhancing the
`CONTRIBUTING.md` guidelines.
## General checklist
- [x] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
## Issue
Dependency cycles removed:
[langchain4j-core]->[langchain4j-parent]->[langchain4j-core]
This cycles prevented project fresh project correctly in Jetbrains Idea.
Cyclyc dependency between this repo and [langchain-embeddings
repo](https://github.com/langchain4j/langchain4j-embeddings) should be
fixed separately.
## Change
- Removed dependency cycles: Removed dependencies to project modules
from `langchain4j-parent`/dependencyManagement to avoid cycles. Added
explicit version number for
langchain4j-core,langchain4j,langchain4j-open-ai.
- Introduced `langchain4j-embeddings.version` for explicit dependency on
another repo.
- Added BOM dependencies to parent pom: junit-bom, mockito-bom. Remved
dependencies, defined in BOMs
- junit.version -> 5.11.3
- Upgraded maven-surefire-plugin, maven-failsafe-plugin. **NB! Renovate
bot is needed to do this automatically**
## General checklist
- [x] There are no breaking changes
- [ ] I have added unit and integration tests for my change
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
github-actions[bot]