LangChain4j 8e4254fc20 make OpenAI tokenizer more precise (#346) ... This PR is a rework of `OpenAiTokenizer`. Added `OpenAiTokenizerIT` with lots of tests to ensure that `OpenAiTokenizer` calculates token usage very close to OpenAI. In most cases calculation is 1:1, in some corner cases the difference is within 5%.		2023-12-12 16:45:16 +01:00
..
src	make OpenAI tokenizer more precise (#346)	2023-12-12 16:45:16 +01:00
pom.xml	reducing duplication of *EmbeddingStoreIT	2023-11-18 19:20:26 +01:00