## Issue
Related to #5541. **This PR intentionally does not close #5541** — the
Kotlin `parseAsync` divergence stays open and is addressed in 2.0 (see
"Follow-up" below).
## Change
`DocumentLoader.load()` forwards the `DocumentSource` metadata onto the
parsed `Document` with `putAll()`. When the parser and the source define
the same key (e.g. a custom `DocumentParser` that sets
`file_name`, or `ApachePdfBoxDocumentParser` /
`ApacheTikaDocumentParser` with `includeMetadata = true`, which copy
keys straight out of the file), the document value is dropped and
nothing is reported.
This PR keeps the existing "source wins" resolution — no behaviour
change, no breaking change — but stops it being silent, and stops batch
loaders losing documents invisibly:
**`langchain4j-core`**
- `DocumentLoader.load()` logs a warning naming the conflicting key
**and both values**, and points at the workaround:
```
Metadata key "file_name" is set both by the document ("report.pdf") and
by the source ("2024-report.pdf").
Keeping the source value and discarding the document value. To control
this, remove the key in your
DocumentParser before returning the Document.
```
- Javadoc on `DocumentLoader.load()` now states the collision behaviour
explicitly instead of just "forwards the source Metadata".
- `Metadata.merge()` reports the conflicting **values**, not only the
key names:
`Metadata keys are not unique. Common keys and their values:
{key2=("value2", "value3")}`
**Batch loaders** (`FileSystemDocumentLoader`,
`ClassPathDocumentLoader`, `AmazonS3DocumentLoader`,
`AzureBlobStorageDocumentLoader`, `GoogleCloudStorageDocumentLoader`,
`TencentCosDocumentLoader`,
`GitHubDocumentLoader`)
All seven swallow per-document exceptions and continue, so a caller can
silently receive a shorter `List<Document>` than expected. Each now logs
a summary when anything was lost:
```
Loaded 9997 of 10000 documents from '/docs'. Skipped 3 that failed to
load and 0 that were blank.
```
`FileSystemDocumentLoader` and `ClassPathDocumentLoader` additionally
pass the throwable to the logger instead of only its message, so
failures get a stack trace like the cloud loaders already did.
## Why not throw on collisions
An earlier revision of this PR made both `DocumentLoader.load()` and the
Kotlin `parseAsync` throw on collisions via `Metadata.merge()`. That
turns out to be the wrong fix for 1.x: every batch loader treats an
exception as "this file is broken, skip it". A perfectly readable
document would then be dropped from the result list purely because of a
metadata key name — replacing a silently discarded *value* with a
silently discarded *document*. The trigger is user data (keys authored
by whoever produced the PDF), not user code, so it is not something the
caller can reliably avoid.
A workaround exists today and needs no new API — decorate the parser to
drop the key:
```java
DocumentParser stripped = inputStream -> {
Document document = myParser.parse(inputStream);
document.metadata().remove(Document.FILE_NAME);
return document;
};
```
## Follow-up (2.0)
Collisions should not be resolved silently, but the fix needs two pieces
that only fit in a major release, and they have to land together:
- **Collisions fail by default**, with an explicit resolution strategy
to opt out (`sourceWins()`, `documentWins()`, or a custom resolver), so
the library never guesses which value was meant.
- **Bulk loading gets a real failure policy**: fail-fast by default,
explicit opt-in to tolerance, and failures returned as **data** rather
than only logged — so "throw" can never mean "document silently
vanishes".
Until then, the Kotlin `parseAsync` extension keeps throwing where the
Java path warns. That divergence is deliberate, and is why #5541 stays
open — better one behaviour change in 2.0 than two in a row.
## Notes for reviewers
- No API changes and no new dependencies; `revapi` is unaffected.
- The only `langchain4j-kotlin` change is a test assertion updated for
the new `Metadata.merge()` message. The Kotlin `parseAsync` main source
is back to `main`.
- `Metadata.merge()`'s exception **message** changed. The exception type
and the conditions that trigger it are unchanged; only assertions on the
exact message text are affected.
- `GoogleCloudStorageDocumentLoader` shows a larger diff than its change
warrants: its imports were already unformatted on `main` and only hit
the spotless ratchet now that the file is touched.
## General checklist
- [X] There are no breaking changes (API, behaviour)
- [X] I have added unit and/or integration tests for my change
- [X] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
<!-- N/A — behaviour is documented in the Javadoc of
DocumentLoader.load() -->
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features) <!-- N/A -->
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable) <!-- N/A -->
---------
Co-authored-by: Dmytro Liubarskyi <ljubarskij@gmail.com>
f11c7a78d2