Commit Graph

34 Commits

Author SHA1 Message Date
github-actions[bot] a39f132b91 Update versions to 1.19.0-SNAPSHOT and 1.19.0-beta29-SNAPSHOT 2026-07-17 13:42:56 +00:00
github-actions[bot] 66ad5ee6d5 Release versions 1.18.0 and 1.18.0-beta28 2026-07-17 12:34:45 +00:00
Eunbin Son 96a3af9ee3
fix: Use UTF-8 when encoding Playwright page content for the document parser (#5686)
# fix: Use UTF-8 when encoding Playwright page content for the document
parser

## Issue
Closes #5685

## Change
`PlaywrightDocumentLoader.load(String url, DocumentParser
documentParser)` encoded the fetched HTML with `pageContent.getBytes()`,
which uses the JVM's default charset. `TextDocumentParser` decodes those
bytes as UTF-8, so on any JVM whose default charset is not UTF-8 (the
Java 17 target is platform-dependent) non-ASCII content (accents, CJK,
©, emoji) was silently corrupted.

This changes the encoding to `getBytes(StandardCharsets.UTF_8)`,
matching the langchain4j-core convention. It is byte-identical on
UTF-8-default JVMs (backward compatible) and correct on all others. Only
the parser overload is touched; `load(String url)` has no such round
trip.

```java
documentParser.parse(new ByteArrayInputStream(pageContent.getBytes(StandardCharsets.UTF_8)));
```

Also adds the module's first unit test (`PlaywrightDocumentLoaderTest`,
Mockito inherited from parent, no new dependency): it mocks
`Browser`/`Page`, captures the `InputStream` passed to a mocked
`DocumentParser`, and asserts the bytes equal the page content's UTF-8
encoding. This pins the encoding contract on every platform. On a
UTF-8-default CI it is byte-identical to the previous behaviour, so it
is a contract-pin, not a fail-on-bug guard.

## General checklist
- [X] There are no breaking changes (API, behaviour)
- [X] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases <!-- N/A — single
encoding-contract assertion; no meaningful negative case -->
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green <!-- Unit test
passes (1/1). The existing *IT needs a Playwright Testcontainer/Docker
and was not run in the unit phase. -->
- [ ] I have manually run all the unit and integration tests in the core
and main modules <!-- N/A — change is isolated to this module; core/main
untouched -->
- [ ] I have added/updated the documentation <!-- N/A — behaviour fix,
no doc change -->
- [ ] I have added an example in the examples repo <!-- N/A — not a
feature -->
- [ ] I have added/updated Spring Boot starter(s) <!-- N/A — not
applicable -->

<!-- Conditional sections (new maven module / new or changed embedding
store integration) omitted — not applicable to this fix. -->

## Pre-submission notes (honest)
- Unit test: `PlaywrightDocumentLoaderTest` passed (1/1, 0 failures) via
`./mvnw -pl document-loaders/langchain4j-document-loader-playwright -am
clean test` on JDK 25.
- `PlaywrightDocumentLoaderIT` (Testcontainers) was not run — Surefire
runs `*Test`, not `*IT`; it needs Docker. Not affected by this change.
- Spotless: `./mvnw -T12C -Pspotless spotless:check -pl
document-loaders/langchain4j-document-loader-playwright` passes (BUILD
SUCCESS, JDK 17), run from a normal checkout after `spotless:apply`
wrapped one long assignment in the test. It cannot run inside the
isolated git worktree (spotless 2.44.4 JGit cannot resolve a worktree
whose `.git` is a file: "Cannot find git repository in any parent
directory").

## Staleness check (run before submitting)
2026-07-06 10:31:36 +02:00
github-actions[bot] 7d7c3349d7 Update versions to 1.18.0-SNAPSHOT and 1.18.0-beta28-SNAPSHOT 2026-06-26 14:49:40 +00:00
github-actions[bot] 207407aec9 Release versions 1.17.0 and 1.17.0-beta27 2026-06-26 13:13:06 +00:00
github-actions[bot] 01de41d641 Update versions to 1.17.0-SNAPSHOT and 1.17.0-beta27-SNAPSHOT 2026-06-06 06:46:38 +00:00
github-actions[bot] cd836845dd Release versions 1.16.0 and 1.16.0-beta26 2026-06-05 15:46:56 +00:00
github-actions[bot] 6185599e37 Update versions to 1.16.0-SNAPSHOT and 1.16.0-beta26-SNAPSHOT 2026-05-15 16:21:14 +00:00
github-actions[bot] d0e54aa006 Release versions 1.15.0 and 1.15.0-beta25 2026-05-15 15:55:12 +00:00
github-actions[bot] 628ac34c01 Update versions to 1.15.0-SNAPSHOT and 1.15.0-beta25-SNAPSHOT 2026-04-30 18:43:12 +00:00
github-actions[bot] 4917afa297 Release versions 1.14.0 and 1.14.0-beta24 2026-04-30 18:10:30 +00:00
github-actions[bot] 4798a89d66 Update versions to 1.14.0-SNAPSHOT and 1.14.0-beta24-SNAPSHOT 2026-04-09 14:41:27 +00:00
github-actions[bot] 759cd9a236 Release versions 1.13.0 and 1.13.0-beta23 2026-04-09 13:07:30 +00:00
Dmytro Liubarskyi e10abf04d0
Update versions to 1.13.0-SNAPSHOT and 1.13.0-beta23-SNAPSHOT (#4710)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-13 11:39:50 +01:00
Dmytro Liubarskyi c92ea033e4
Update versions to 1.13.0-SNAPSHOT and 1.13.0-beta22-SNAPSHOT (#4666)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-05 17:18:22 +01:00
Dmytro Liubarskyi 336b2accce
Update versions to 1.12.0-SNAPSHOT and 1.12.0-beta20-SNAPSHOT (#4537)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-04 14:27:44 +01:00
Xin Wang 5ea11e8e15
Update Playwright version to 1.57.0 (#4448)
<!--
Thank you so much for your contribution!

Please fill in all the sections below.
Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples.
Please note that PRs with breaking changes or without tests will be
rejected.

Please note that PRs will be reviewed based on the priority of the
issues they address.
We ask for your patience. We are doing our best to review your PR as
quickly as possible.
Please refrain from pinging and asking when it will be reviewed. Thank
you for understanding!
-->

## Issue
<!-- Please specify the ID of the issue this PR is addressing. For
example: "Closes #1234" or "Fixes #1234" -->
Closes #

## Change
<!-- Please describe the changes you made. -->
Upgrade playwright-java to 1.57 to support latest Chromium / Chrome for
Testing.


## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes (API, behaviour)
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)


## Checklist for adding new maven module
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added my new module in the root `pom.xml` and
`langchain4j-bom/pom.xml`


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreRemovalIT` that
extends from `EmbeddingStoreWithRemovalIT`

## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j
2026-01-30 17:22:58 +01:00
Dmytro Liubarskyi c116ec6ec9 cleanup: moved dependency versions to the modules where they are used 2026-01-05 15:52:36 +01:00
Dmytro Liubarskyi 778be1b360
Update versions to 1.11.0-SNAPSHOT and 1.11.0-beta19-SNAPSHOT (#4285)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-24 15:38:05 +01:00
Dmytro Liubarskyi 25d3b3ec87 cleaned up test dependencies 2025-12-10 10:18:30 +01:00
Dmytro Liubarskyi ca6097e35d
Update versions to 1.10.0-SNAPSHOT and 1.10.0-beta18-SNAPSHOT (#4152)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-28 12:21:30 +01:00
Dmytro Liubarskyi bc0801a4df
Update versions to 1.10.0-SNAPSHOT and 1.10.0-beta17-SNAPSHOT (#4140)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-26 17:38:36 +01:00
Dmytro Liubarskyi a473835133
Update versions to 1.9.0-SNAPSHOT and 1.9.0-beta16-SNAPSHOT (#3951)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-10-24 16:51:33 +02:00
Dmytro Liubarskyi 34632c06a2 nex dev iteration 2025-10-02 17:17:35 +02:00
Dmytro Liubarskyi 7add1a1b4e next dev iteration 2025-09-26 16:54:16 +02:00
Dmytro Liubarskyi afee79638b next dev iteration 2025-09-16 15:23:21 +02:00
Dmytro Liubarskyi 7cc34306db next dev iteration 2025-08-29 08:35:38 +02:00
Dmytro Liubarskyi 5b1b2e76d2 next dev iteration 2025-08-07 16:24:25 +02:00
Dmytro Liubarskyi 0a01b49951 next dev iteration 2025-07-29 17:50:28 +02:00
Harikrishna 580f406ae4
[Fix] Fix the testcase should_load_and_parse_html_document (#3430)
Fix should_load_and_parse_html_document
- Extract the document using HtmlToTextDocumentTransformer and make sure
that it doesn't contain the tag <head>

Co-authored-by: Harikrishna <harikrishna.gurram@walmart.com>
2025-07-28 18:44:41 +02:00
Dmytro Liubarskyi c0e76d622f Add support for loading documents with custom DocumentParser (#3386) 2025-07-28 17:33:00 +02:00
Harikrishna 0e7374d643
Add support for loading documents with custom DocumentParser (#3386)
### Summary

This PR introduces an overloaded `load(String url, DocumentParser
parser)` method to allow parsing HTML content using a custom
`DocumentParser` implementation. This enhancement improves flexibility
and decouples the document loading from a fixed parsing strategy.

### Changes Made

- Added new method: `Document load(String url, DocumentParser
documentParser)`
  - Ensures `url` and `documentParser` are non-null
  - Parses the page content using the provided parser
  - Attaches the source URL to the document metadata

- Added test cases for the new method:
  - Loads and parses a real HTML document using `TextDocumentParser`

### Backward Compatibility

- Existing method `load(String url)` remains unchanged.
- No breaking changes.

Co-authored-by: Harikrishna <harikrishna.gurram@walmart.com>
2025-07-25 14:52:14 +02:00
NeatGuyCoding faeac9e061
fix memory leak in load() of PlaywrightDocumentLoader (#3327) 2025-07-09 20:17:34 +03:00
Xin Wang 82b533e2f4
feat: Playwright Support (#3282)
<!--
Thank you so much for your contribution!

Please fill in all the sections below.
Please open the PR as a draft initially. Once it is reviewed and
approved, we will ask you to add documentation and examples.
Please note that PRs with breaking changes or without tests will be
rejected.

Please note that PRs will be reviewed based on the priority of the
issues they address.
We ask for your patience. We are doing our best to review your PR as
quickly as possible.
Please refrain from pinging and asking when it will be reviewed. Thank
you for understanding!
-->

## Issue
<!-- Please specify the ID of the issue this PR is addressing. For
example: "Closes #1234" or "Fixes #1234" -->
Closes https://github.com/langchain4j/langchain4j/issues/3281

## Change
<!-- Please describe the changes you made. -->
Playwright Support

## General checklist
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] There are no breaking changes
- [x] I have added unit and/or integration tests for my change
- [x] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
<!-- Before adding documentation and example(s) (below), please wait
until the PR is reviewed and approved. -->
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)


## Checklist for adding new maven module
<!-- Please double-check the following points and mark them like this:
[X] -->
- [x] I have added my new module in the root `pom.xml` and
`langchain4j-bom/pom.xml`


## Checklist for adding new embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreIT` that extends
from either `EmbeddingStoreIT` or `EmbeddingStoreWithFilteringIT`
- [ ] I have added a `{NameOfIntegration}EmbeddingStoreRemovalIT` that
extends from `EmbeddingStoreWithRemovalIT`

## Checklist for changing existing embedding store integration
<!-- Please double-check the following points and mark them like this:
[X] -->
- [ ] I have manually verified that the
`{NameOfIntegration}EmbeddingStore` works correctly with the data
persisted using the latest released version of LangChain4j

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Dmytro Liubarskyi <ljubarskij@gmail.com>
2025-07-07 14:57:46 +02:00