The design of the Dashscope SDK is evolving towards OpenAI, offering new
fields and specifications. Utilize these latest features to refactor the
implementation of the Qwen models.
I've done integration with Elastic and do some local test to ensure it's
right!（some logic is translated from LangChain Python to Java）.
Elasticsearch do not support `Gson`. So we must have `Jackson`
dependency.
Wrap a method to return the original LLM result, allowing subclasses to
have a chance to obtain more information such as usage (input/output
tokens, finish reason...).
If the text is longer than 510 tokens, it is automatically partitioned
into parts <= 510 tokens long. These partitions are embedded
independently and then weighted (by the number of tokens in each
partition) average embedding is calculated.
Related to https://github.com/langchain4j/langchain4j-embeddings/pull/4
Note that the declaration of license in dashscope-sdk-java:pom.xml is:
<licenses>
<license>
<name>The Apache License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
(There is a "The " prefix in its name.)
Qwen series models are provided by Alibaba Cloud. They are much better
in Asia languages then other LLMs.
DashScope is a model service platform. Qwen models are its primary
supported models. But it also supports other series like LLaMA2, Dolly,
ChatGLM, BiLLa(based on LLaMA)...These may be integrated sometime in the
future.
- Added ChatMemoryStore and InMemoryChatMemoryStore
- Changed MessageWindowChatMemory and TokenWindowChatMemory to use
ChatMemoryStore
- Changed Supplier<ChatMemory> into ChatMemoryProvider
- Small improvements
- did some refactorings
- added javadoc
- fixed NPE in PineconeEmbeddingStoreImpl when adding embedding by id
- PineconeEmbeddingStoreImpl now takes into account minScore and returns
score in EmbeddingMatch
- InMemoryEmbeddingStore now returns score instead of cosine similarity
Some leftovers from an earlier (and now incorrect) CI configuration.
Modules that don't need to comply with the licenses need to deactivate
the relevant plugin on a case-by-case basis.
In-process embeddings are moved to their own repository
(https://github.com/langchain4j/langchain4j-embeddings) due to the size
of the involved files.
Note that once this PR is merged we'd ideally need a release so as to
have the `langchain4j-embeddings` related project depend on that
version.
LangChain4j