Input and output guardrails (#2571)
Guardrails support. I also introduced a new `langchain4j-test` module which can be a future place to put in some testing utilities. For now I have several assertj-based assertion classes that can be used for asserting guardrails. (see https://docs.quarkiverse.io/quarkus-langchain4j/dev/guardrails.html#_unit_testing for current details). Additionally, I've introduced a new `integration-tests` where we can build various sets of test suites (think things that use/need `ServiceLoader`s). This way we can test various implementations of things for frameworks that may extend LangChain4j (Quarkus, Spring, etc). I've tried to include as much "genericism" as I can into this so that downstream frameworks can re-use this without having to re-implement. I'll have @geoand / @cescoffier review this too before merging. ## To-do list - [x] Initial scaffolding for being able to define guardrails - [x] Guardrail testing utilities - [x] Wire guardrails into `AiService`s via annotations - [x] Wire guardrails into `AiService`s using the builder - [x] Handle output guardrails on streaming responses - [x] Lots of tests - [x] Documentation Fixes #2549 --------- Co-authored-by: Lize Raes <49833622+LizeRaes@users.noreply.github.com> Co-authored-by: Dmytro Liubarskyi <ljubarskij@gmail.com>
This commit is contained in:
Eric Deandrea
GitHub
parent
0a91cb9a06
commit
023a65c044
|
|
@ -52,6 +52,7 @@ ij_java_blank_lines_after_package = 1
|
|||
ij_java_blank_lines_around_class = 1
|
||||
ij_java_blank_lines_around_field = 0
|
||||
ij_java_blank_lines_around_field_in_interface = 0
|
||||
ij_java_blank_lines_around_field_with_annotations = 0
|
||||
ij_java_blank_lines_around_initializer = 1
|
||||
ij_java_blank_lines_around_method = 1
|
||||
ij_java_blank_lines_around_method_in_interface = 1
|
||||
|
|
@ -116,7 +117,7 @@ ij_java_generate_final_locals = true
|
|||
ij_java_generate_final_parameters = true
|
||||
ij_java_generate_use_type_annotation_before_type = true
|
||||
ij_java_if_brace_force = never
|
||||
ij_java_imports_layout = *, |, javax.**, java.**, |, $*
|
||||
ij_java_imports_layout = $*,|,javax.**,java.**,*
|
||||
ij_java_indent_case_from_switch = true
|
||||
ij_java_insert_inner_class_imports = false
|
||||
ij_java_insert_override_annotation = true
|
||||
|
|
|
|||
|
|
@ -12,8 +12,8 @@ echo "🔍 Scanning all JARs under: $ROOT_DIR (excluding test JARs)"
|
|||
# Get all JAR files first, excluding test JARs
|
||||
jar_files=()
|
||||
while IFS= read -r jar; do
|
||||
# Skip JAR files ending with "-tests.jar"
|
||||
if [[ "$jar" != *"-tests.jar" ]]; then
|
||||
# Skip JAR files ending with "-tests.jar" and anything in the integration-tests directory
|
||||
if [[ "$jar" != *"-tests.jar" ]] && [[ "$jar" != "./integration-tests/"* ]]; then
|
||||
jar_files+=("$jar")
|
||||
fi
|
||||
done < <(find "$ROOT_DIR" -type f -name "*.jar")
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
sidebar_position: 12
|
||||
sidebar_position: 13
|
||||
---
|
||||
|
||||
# Classification
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
sidebar_position: 13
|
||||
sidebar_position: 14
|
||||
---
|
||||
|
||||
# Embedding (Vector) Stores
|
||||
|
|
|
|||
|
|
@ -0,0 +1,539 @@
|
|||
---
|
||||
sidebar_position: 12
|
||||
toc_max_heading_level: 5
|
||||
---
|
||||
|
||||
import useBaseUrl from '@docusaurus/useBaseUrl';
|
||||
import ThemedImage from '@theme/ThemedImage';
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# Guardrails
|
||||
|
||||
Guardrails are mechanisms that let you validate the input and output of the LLM to ensure it meets your expectations. You can do some of the following things with guardrails:
|
||||
- Verify the user input is not out of scope
|
||||
- Ensure the input meets some criteria before calling the LLM (i.e. guard against a [prompt injection attack](https://genai.owasp.org/llmrisk/llm01-prompt-injection/))
|
||||
- Ensure the output format is correct (i.e. it is a JSON document with the correct schema)
|
||||
- Ensure the LLM output is coherent with business rules and constraints (i.e. if this is a chatbot of company X, the response should not contain any reference to a competitor Y).
|
||||
- Detect hallucinations
|
||||
|
||||
Those are just examples. You can do many other things with guardrails.
|
||||
|
||||
:::note
|
||||
Guardrails are only available when using [AI Services](/tutorials/ai-services). They are a higher-level construct that can not be applied to a `ChatModel` or `StreamingChatModel`.
|
||||
:::
|
||||
|
||||
<ThemedImage
|
||||
alt="Guardrails"
|
||||
sources={{
|
||||
light: useBaseUrl('/img/guardrails-light-bg.png'),
|
||||
dark: useBaseUrl('/img/guardrails-dark-bg.png'),
|
||||
}}
|
||||
/>;
|
||||
|
||||
The implementation was originally done in the [Quarkus LangChain4j extension](https://docs.quarkiverse.io/quarkus-langchain4j/dev/) and was backported here.
|
||||
|
||||
## Implementing Guardrails
|
||||
|
||||
Ideally, guardrail implementations should follow the [single responsibility principle](https://en.wikipedia.org/wiki/Single-responsibility_principle), meaning that each guardrail class should validate one thing. Then, chain guardrails together to guard against multiple things.
|
||||
|
||||
The order of guardrails in the chain is important. The first guardrail in the chain to fail will trigger the overall failure. Ensure guardrails that catch the most failures are early in the chain, whereas more specific guardrails that may fail very infrequently are towards the end of the chain.
|
||||
|
||||
Also keep in mind that guardrails can themselves call other services or even invoke other LLM interactions. If these kinds of guardrails have an execution penalty or monetary cost associated with them, make sure you take that into account. You might want to put more expensive guardrails towards the end of the chain.
|
||||
|
||||
:::note
|
||||
The term _expensive_ can mean that something takes some time to execute or has a monetary value associated with it.
|
||||
:::
|
||||
|
||||
## Input Guardrails
|
||||
|
||||
Input guardrails are functions invoked before the LLM is called. Failing an input guardrail prevents the LLM from being called. Input guardrails are the last step prior to calling the LLM. They are invoked _after_ any [RAG](/tutorials/rag) operations have happened.
|
||||
|
||||
### Implementing Input Guardrails
|
||||
|
||||
Input guardrails are implemented by implementing the [`InputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/InputGuardrail.java) interface. The `InputGuardrail` interface has two variants of the `validate` method, at least one of which needs to be implemented:
|
||||
|
||||
```java
|
||||
InputGuardrailResult validate(UserMessage userMessage);
|
||||
InputGuardrailResult validate(InputGuardrailRequest params);
|
||||
```
|
||||
|
||||
The first variant is used for simple guardrails, or when the guardrail only needs access to the [`UserMessage`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/data/message/UserMessage.java).
|
||||
|
||||
The second variant is for more complex guardrails that need more information, such as the chat memory/history, user message template, augmentation results, or variables that were passed to the template. See [`InputGuardrailRequest`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/InputGuardrailRequest.java) for more information.
|
||||
|
||||
Some examples of things you could do:
|
||||
- Check that there are enough documents in the augmentation results
|
||||
- Ensure the user is not asking the same question multiple times
|
||||
- Mitigate potential prompt injection attack
|
||||
|
||||
Input guardrails can be used whether the operation is synchronous or asynchronous/streaming.
|
||||
|
||||
### Input Guardrail Outcomes
|
||||
|
||||
Input guardrails can have the following outcomes. There are helper methods on the `InputGuardrail` interface that can provide the outcomes:
|
||||
|
||||
| Outcome | Helper method on `InputGuardrail` | Description |
|
||||
|:------------------------------------|:--------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| **_success_** | `success()` | - The input is valid.<br/> - The next guardrail in the chain is executed.<br/> - The LLM is called if the last guardrail passes. |
|
||||
| **_success with alternate result_** | `successWith(String)` | Similar to **_success_** except the user message is altered before proceeding to the next step (next guardrail in the chain or calling the LLM). |
|
||||
| **_failure_** | `failure(String)` or `failure(String, Throwable)` | - The input is invalid but the next guardrails in the chain continue to be executed in order to accumulate all possible validation problems.<br/> - The LLM is not called. |
|
||||
| **_fatal_** | `fatal(String)` or `fatal(String, Throwable)` | - The input is invalid and execution is halted with an `InputGuardrailException`.<br/> - The LLM is not called. |
|
||||
|
||||
### Declaring Input Guardrails
|
||||
|
||||
There are several ways to declare input guardrails, listed here in order of precedence:
|
||||
1. [`InputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/InputGuardrail.java) implementation class names or instances set directly on the [`AiServices`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/AiServices.java) builder.
|
||||
2. [`@InputGuardrails` annotations](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/InputGuardrails.java) placed on an individual [AI Service](/tutorials/ai-services) method.
|
||||
3. [`@InputGuardrails` annotation](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/InputGuardrails.java) placed on an [AI Service](/tutorials/ai-services) class.
|
||||
Regardless of how they are declared, input guardrails are always executed in the order they appear in the list.
|
||||
|
||||
#### `AiServices` builder
|
||||
|
||||
[`InputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/InputGuardrail.java) implementation class names or instances set directly on the [`AiServices`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/AiServices.java) builder have the highest precedence, meaning if it is declared in any other ways, the one declared directly on the builder will be the one used.
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
String chat(String question);
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.builder(Assistant.class)
|
||||
.chatModel(chatModel)
|
||||
.inputGuardrailClasses(FirstInputGuardrail.class, SecondInputGuardrail.class)
|
||||
.build();
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
String chat(String question);
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.builder(Assistant.class)
|
||||
.chatModel(chatModel)
|
||||
.inputGuardrails(new FirstInputGuardrail(), new SecondInputGuardrail())
|
||||
.build();
|
||||
```
|
||||
|
||||
In the first scenario, classes that implement `InputGuardrail` are passed. New instances of these classes are created dynamically using reflection.
|
||||
|
||||
:::info
|
||||
The way classes are converted to instances can be customized. For example, frameworks that use dependency injection (like [Quarkus](https://quarkus.io) or [Spring](https://spring.io)) can use [extension points](#extension-points) to provide instances based on how they manage class instances rather than creating new instances via reflection each time.
|
||||
:::
|
||||
|
||||
#### Annotation on individual AI Service methods
|
||||
|
||||
[`@InputGuardrails` annotations](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/InputGuardrails.java) placed on an individual [AI Service](/tutorials/ai-services) methods have the next highest precedence.
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
@InputGuardrails({ FirstInputGuardrail.class, SecondInputGuardrail.class })
|
||||
String chat(String question);
|
||||
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.create(Assistant.class, chatModel);
|
||||
```
|
||||
|
||||
In this example, only the `chat` method has guardrails.
|
||||
- On the `chat` method, `FirstInputGuardrail` is invoked first.
|
||||
- Only if it is successful will the LLM be called.
|
||||
- `SecondInputGuardrail` will only be invoked if `FirstInputGuardrail` does not result in a **_fatal_** result.
|
||||
- Either `FirstInputGuardrail` or `SecondInputGuardrail` could re-write the user message.
|
||||
- If `FirstInputGuardrail` re-writes the user message, then `SecondInputGuardrail` will receive the new user message as input.
|
||||
|
||||
The `doSomethingElse` method does not have any guardrails.
|
||||
|
||||
#### Annotation on the AI Service class
|
||||
|
||||
[`@InputGuardrails` annotation](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/InputGuardrails.java) placed on an [AI Service](/tutorials/ai-services) class has the lowest precedence.
|
||||
|
||||
```java
|
||||
@InputGuardrails({ FirstInputGuardrail.class, SecondInputGuardrail.class })
|
||||
public interface Assistant {
|
||||
String chat(String question);
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.create(Assistant.class, chatModel);
|
||||
```
|
||||
|
||||
In this example, both the `chat` and `doSomethingElse` methods have the guardrails.
|
||||
- Just like in the previous example, `FirstInputGuardrail` is invoked first.
|
||||
- Only if it is successful will the LLM be called.
|
||||
- `SecondInputGuardrail` will only be invoked if `FirstInputGuardrail` does not result in a **_fatal_** result.
|
||||
- Either `FirstInputGuardrail` or `SecondInputGuardrail` could re-write the user message.
|
||||
- If `FirstInputGuardrail` re-writes the user message, then `SecondInputGuardrail` will receive the new user message as input.
|
||||
|
||||
### Unit Testing Input Guardrails
|
||||
|
||||
There are some unit testing utilities based on [AssertJ](https://assertj.github.io/doc/) in the `langchain4j-test` module.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="maven" label="Maven" default>
|
||||
```xml
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="gradleGroovy" label="Gradle (Groovy)">
|
||||
```groovy
|
||||
testImplementation 'dev.langchain4j:langchain4j-test'
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="gradleKotlin" label="Gradle (Kotlin)">
|
||||
```kotlin
|
||||
testImplementation("dev.langchain4j:langchain4j-test")
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
Once you have the dependency, you can perform these kinds of validations:
|
||||
|
||||
```java
|
||||
import static dev.langchain4j.test.guardrail.GuardrailAssertions.assertThat;
|
||||
|
||||
import dev.langchain4j.data.message.UserMessage;
|
||||
import dev.langchain4j.guardrail.GuardrailResult.Result;
|
||||
|
||||
class Tests {
|
||||
MyInputGuardrail inputGuardrail = new MyInputGuardrail();
|
||||
|
||||
@Test
|
||||
void test() {
|
||||
var userMessage = UserMessage.from("Some user message");
|
||||
var result = inputGuardrail.validate(userMessage);
|
||||
|
||||
// These are just some examples of what you can do
|
||||
assertThat(result)
|
||||
.isSuccessful()
|
||||
.hasResult(Result.FATAL)
|
||||
.hasFailures()
|
||||
.hasSingleFailureWithMessage("Prompt injection detected")
|
||||
.assertSingleFailureSatisfied(failure -> assertThat(failure)...)
|
||||
.withFailures().....
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
:::info
|
||||
See the [`GuardrailAssertions`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-test/src/main/java/dev/langchain4j/test/guardrail/GuardrailAssertions.java) and [`InputGuardrailResultAssert`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-test/src/main/java/dev/langchain4j/test/guardrail/InputGuardrailResultAssert.java) classes for more details.
|
||||
:::
|
||||
|
||||
## Output Guardrails
|
||||
|
||||
Output guardrails are functions executed after the LLM has produced its output. Failing an output guardrail allows for more advanced scenarios, such as [retrying](#retry) or [reprompting](#reprompt), to help improve the response. They are invoked _after_ all other operations, including function/tool calls, have happened.
|
||||
|
||||
### Implementing Output Guardrails
|
||||
|
||||
Similar to input guardrails, output guardrails are implemented by implementing the [`OutputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/OutputGuardrail.java) interface. The `OutputGuardrail` interface has two variants of the `validate` method, at least one of which needs to be implemented:
|
||||
|
||||
```java
|
||||
OutputGuardrailResult validate(AiMessage responseFromLLM);
|
||||
OutputGuardrailResult validate(OutputGuardrailRequest params);
|
||||
```
|
||||
|
||||
The first variant is used for simple guardrails, or when the guardrail only needs access to the resulting [`AiMessage`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/data/message/AiMessage.java).
|
||||
|
||||
The second variant is for more complex guardrails that need more information, such as the entire chat response, chat memory/history, user message template, or variables that were passed to the template. See [`OutputGuardrailRequest`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/OutputGuardrailRequest.java) for more information.
|
||||
|
||||
Some examples of things you could do:
|
||||
- Ensure the output format is correct (i.e. it is a JSON document with the correct schema)
|
||||
- Detect an LLM hallucination
|
||||
- Validate that the LLM response contains certain information
|
||||
|
||||
### Output Guardrail Outcomes
|
||||
|
||||
Output guardrails can have the following outcomes. There are helper methods on the `OutputGuardrail` interface that can provide the outcomes:
|
||||
|
||||
| Outcome | Helper method on `OutputGuardrail` | Description |
|
||||
|:---------------------------|:--------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| **_success_** | `success()` | - The output is valid.<br/> - The next guardrail in the chain is executed. If the last guardrail passes the output is returned to the caller. |
|
||||
| **_success with rewrite_** | `successWith(String)` or `successWith(String, Object)` | -Similar to **_success_** except the output isn't valid in its original form and has been rewritten to make it valid.<br/> - The next guardrail is executed against the rewritten output. If the last guardrail passes the output is returned to the caller. |
|
||||
| **_failure_** | `failure(String)` or `failure(String, Throwable)` | - The output is invalid but the next guardrails in the chain continue to be executed in order to accumulate all possible validation problems.<br/> - The validation failure is returned to the user as an `OutputGuardrailException`. |
|
||||
| **_fatal_** | `fatal(String)` or `fatal(String, Throwable)` | The output is invalid and execution is halted with an `OutputGuardrailException` thrown to the caller. |
|
||||
| **_fatal with retry_** | `retry(String)` or `retry(String, Throwable)` | - Similar to **_fatal_** except the LLM is called again with the same prompt and chat history as the original call.<br/> - If the failure persists after a [configurable number of retries](#configuration) then execution is halted with an `OutputGuardrailException` thrown to the caller.<br/> - If the guardrail passes after a retry, the entire chain of guardrails are re-executed from the beginning. |
|
||||
| **_fatal with reprompt_** | `reprompt(String, String)` or `reprompt(String, Throwable, String)` | - Similar to **_fatal with retry_** except the LLM is called again with a new prompt supplied by the guardrail.<br/> - In this situation, the guardrail supplies an additional message to append to the previous user message, then sends a new request to the LLM with the new user message and original chat history.<br/> - If the failure persists after a [configurable number of retries](#configuration) then execution is halted with an `OutputGuardrailException` thrown to the caller.<br/> - If the guardrail passes after a reprompt, the entire chain of guardrails are re-executed from the beginning. |
|
||||
|
||||
### Declaring Output Guardrails
|
||||
|
||||
There are several ways to declare output guardrails, listed here in order of precedence:
|
||||
1. [`OutputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/OutputGuardrail.java) implementation class names or instances set directly on the [`AiServices`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/AiServices.java) builder.
|
||||
2. [`@OutputGuardrails` annotations](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/OutputGuardrails.java) placed on an individual [AI Service](/tutorials/ai-services) method.
|
||||
3. [`@OutputGuardrails` annotation](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/OutputGuardrails.java) placed on an [AI Service](/tutorials/ai-services) class.
|
||||
|
||||
Regardless of how they are declared, output guardrails are always executed in the order they appear in the list.
|
||||
|
||||
#### `AiServices` builder
|
||||
|
||||
[`OutputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/OutputGuardrail.java) implementation class names or instances set directly on the [`AiServices`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/AiServices.java) builder have the highest precedence, meaning if it is declared in any other ways, the one declared on the builder will be the one used.
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
String chat(String question);
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.builder(Assistant.class)
|
||||
.chatModel(chatModel)
|
||||
.outputGuardrailClasses(FirstOutputGuardrail.class, SecondOutputGuardrail.class)
|
||||
.build();
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
String chat(String question);
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.builder(Assistant.class)
|
||||
.chatModel(chatModel)
|
||||
.outputGuardrails(new FirstOutputGuardrail(), new SecondOutputGuardrail())
|
||||
.build();
|
||||
```
|
||||
|
||||
In the first scenario, classes that implement `OutputGuardrail` are passed. New instances of these classes are created dynamically using reflection.
|
||||
|
||||
:::info
|
||||
The way classes are converted to instances can be customized. For example, frameworks that use dependency injection (like [Quarkus](https://quarkus.io) or [Spring](https://spring.io)) can use [extension points](#extension-points) to provide instances based on how they manage class instances rather than creating new instances via reflection each time.
|
||||
:::
|
||||
|
||||
#### Annotation on individual AI Service methods
|
||||
|
||||
[`@OutputGuardrails` annotations](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/OutputGuardrails.java) placed on ndividual [AI Service](/tutorials/ai-services) methods have the next highest precendence.
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
@OutputGuardrails({ FirstOutputGuardrail.class, SecondOutputGuardrail.class })
|
||||
String chat(String question);
|
||||
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.create(Assistant.class, chatModel);
|
||||
```
|
||||
|
||||
In this example, only the `chat` method has guardrails.
|
||||
- On the `chat` method, `FirstOutputGuardrail` is invoked first.
|
||||
- Only if it is successful will the result be returned to the caller. `SecondOutputGuardrail` will only be invoked if `FirstOutputGuardrail` does not result in a **_fatal_**, **_fatal with retry_**, or **_fatal with reprompt_** result.
|
||||
- `SecondOutputGuardrail` will receive the output of `FirstOutputGuardrail`.
|
||||
- If `SecondOutputGuardrail` succeeds after a retry or reprompt, then both `FirstOutputGuardrail` and `SecondOutputGuardrail` are re-executed.
|
||||
|
||||
The `doSomethingElse` method does not have any guardrails.
|
||||
|
||||
#### Annotation on the AI Service class
|
||||
|
||||
[`@OutputGuardrails` annotation](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/guardrail/OutputGuardrails.java) placed on an [AI Service](/tutorials/ai-services) class has the lowest precedence.
|
||||
|
||||
```java
|
||||
@OutputGuardrails({ FirstOutputGuardrail.class, SecondOutputGuardrail.class })
|
||||
public interface Assistant {
|
||||
String chat(String question);
|
||||
String doSomethingElse(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.create(Assistant.class, chatModel);
|
||||
```
|
||||
|
||||
In this example, both the `chat` and `doSomethingElse` methods have the guardrails.
|
||||
- Just like in the previous example, `FirstOutputGuardrail` is invoked first.
|
||||
- Only if it is successful will the result be returned to the caller. `SecondOutputGuardrail` will only be invoked if `FirstOutputGuardrail` does not result in a **_fatal_**, **_fatal with retry_**, or **_fatal with reprompt_** result.
|
||||
- `SecondOutputGuardrail` will receive the output of `FirstOutputGuardrail`.
|
||||
- If `SecondOutputGuardrail` succeeds after a retry or reprompt, then both `FirstOutputGuardrail` and `SecondOutputGuardrail` are re-executed.
|
||||
|
||||
#### Configuration
|
||||
|
||||
Output guardrails have the following additional configuration that can be supplied:
|
||||
|
||||
| Configuration | Description |
|
||||
|:--------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `maxRetries` | - The maximum number of retries for an output guardrail when performing a retry or reprompt.<br/> - Defaults to `2`.<br/> - Set to `0` to disable retries. |
|
||||
|
||||
##### Annotation on individual AI Service methods
|
||||
|
||||
```java
|
||||
public interface MethodLevelAssistant {
|
||||
@OutputGuardrails(
|
||||
value = { FirstOutputGuardrail.class, SecondOutputGuardrail.class },
|
||||
maxRetries = 10
|
||||
)
|
||||
String chat(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.create(MethodLevelAssistant.class, chatModel);
|
||||
```
|
||||
|
||||
##### Annotation on the AI Service class
|
||||
|
||||
```java
|
||||
@OutputGuardrails(
|
||||
value = { FirstOutputGuardrail.class, SecondOutputGuardrail.class },
|
||||
maxRetries = 10
|
||||
)
|
||||
public interface ClassLevelAssistant {
|
||||
String chat(String question);
|
||||
}
|
||||
|
||||
var assistant = AiServices.create(ClassLevelAssistant.class, chatModel);
|
||||
```
|
||||
|
||||
##### `AiServices` builder
|
||||
|
||||
```java
|
||||
public interface Assistant {
|
||||
String chat(String message);
|
||||
}
|
||||
|
||||
var outputGuardrailsConfig = OutputGuardrailsConfig.builder()
|
||||
.maxRetries(10)
|
||||
.build();
|
||||
|
||||
var assistant = AiServices.builder(Assistant.class)
|
||||
.chatModel(chatModel)
|
||||
.outputGuardrailsConfig(outputGuardrailsConfig)
|
||||
.outputGuardrailClasss(FirstOutputGuardrail.class, SecondOutputGuardrail.class)
|
||||
.build();
|
||||
```
|
||||
|
||||
### Output Guardrails on Streaming Responses
|
||||
|
||||
Output guardrails can also work for operations with streaming responses:
|
||||
|
||||
```java
|
||||
public interface StreamingAssistant {
|
||||
@OutputGuardrails({ FirstOutputGuardrail.class, SecondOutputGuardrail.class })
|
||||
TokenStream streamingChat(String message);
|
||||
}
|
||||
```
|
||||
|
||||
In this scenario, the output guardrails will be executed once the entire stream is complete, or more specifically, when `TokenStream.onCompleteResponse` is called. `onPartialResponse` will be buffered and replayed once the guardrails succeed.
|
||||
|
||||
In the situation where a **_retry_** or **_reprompt_** in the chain eventually succeeds, then the entire chain is re-executed _synchronously_. Each guardrail will be re-executed one after the other in the original order. Once the chain completes the result is passed into `TokenStream.onCompleteResponse`.
|
||||
|
||||
### Out-of-the-box Output Guardrails
|
||||
|
||||
There are several common use cases where implementations of an output guardrail are provided by LangChain4j:
|
||||
|
||||
| Guardrail class | Description |
|
||||
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [`JsonExtractorOutputGuardrail`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/JsonExtractorOutputGuardrail.java) | An output guardrail that will check whether or not a response can be successfully deserialized from JSON to an object of a certain type.<br/> - Uses a [Jackson ObjectMapper](https://github.com/FasterXML/jackson-databind) to try and deserialize an object.<br/> - The LLM is reprompted if the response can't be deserialized into the expected object type.<br/> - Can be used as-is, or can be extended and customized (there are several `protected` methods that can be overridden to customize behavior). |
|
||||
|
||||
### Unit Testing Output Guardrails
|
||||
|
||||
There are some unit testing utilities based on [AssertJ](https://assertj.github.io/doc/) in the `langchain4j-test` module.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="maven" label="Maven" default>
|
||||
```xml
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="gradleGroovy" label="Gradle (Groovy)">
|
||||
```groovy
|
||||
testImplementation 'dev.langchain4j:langchain4j-test'
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="gradleKotlin" label="Gradle (Kotlin)">
|
||||
```kotlin
|
||||
testImplementation("dev.langchain4j:langchain4j-test")
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
Once you have the dependency, you can perform these kinds of validations:
|
||||
|
||||
```java
|
||||
import static dev.langchain4j.test.guardrail.GuardrailAssertions.assertThat;
|
||||
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.GuardrailResult.Result;
|
||||
|
||||
class Tests {
|
||||
MyOutputGuardrail outputGuardrail = new MyOutputGuardrail();
|
||||
|
||||
@Test
|
||||
void test() {
|
||||
var aiMessage = AiMessage.from("Some output");
|
||||
var result = outputGuardrail.validate(aiMessage);
|
||||
|
||||
// These are just some examples of what you can do
|
||||
assertThat(result)
|
||||
.isSuccessful()
|
||||
.hasResult(Result.FATAL)
|
||||
.hasFailures()
|
||||
.hasSingleFailureWithMessage("Hallucination detected!")
|
||||
.hasSingleFailureWithMessageAndReprompt("Hallucination detected!", "Please LLM don't hallucinate!")
|
||||
.assertSingleFailureSatisfied(failure -> assertThat(failure)...)
|
||||
.withFailures().....
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
:::info
|
||||
See the [`GuardrailAssertions`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-test/src/main/java/dev/langchain4j/test/guardrail/GuardrailAssertions.java) and [`OutputGuardrailResultAssert`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-test/src/main/java/dev/langchain4j/test/guardrail/OutputGuardrailResultAssert.java) classes for more details.
|
||||
:::
|
||||
|
||||
## Mixing and matching
|
||||
|
||||
You can mix and match input and output guardrails however you like!
|
||||
|
||||
```java
|
||||
public class MyObjectJsonOutputGuardrail extends JsonExtractorOutputGuardrail<MyObject> {
|
||||
public MyObjectJsonOutputGuardrail() {
|
||||
super(MyObject.class);
|
||||
}
|
||||
}
|
||||
|
||||
@InputGuardrails({ FirstInputGuardrail.class, SecondInputGuardrail.class })
|
||||
@OutputGuardrails(value = SomeOutputGuardrail.class, maxRetries = 5)
|
||||
public interface Assistant {
|
||||
String chat(String message);
|
||||
|
||||
@InputGuardrails(PromptInjectionGuardrail.class)
|
||||
@OutputGuardrails(MyObjectJsonOutputGuardrail.class)
|
||||
MyObject chatAndReturnJson(String message);
|
||||
}
|
||||
|
||||
var outputGuardrailsConfig = OutputGuardrailsConfig.builder()
|
||||
.maxRetries(10)
|
||||
.build();
|
||||
|
||||
var assistant = AiServices.builder(Assistant.class)
|
||||
.chatModel(chatModel)
|
||||
.inputGuardrails(new AnotherInputGuardrail())
|
||||
.outputGuardrailsConfig(outputGuardrailsConfig)
|
||||
.build();
|
||||
```
|
||||
|
||||
In this example, all the methods on the `Assistant` have a single input guardrail, `AnotherInputGuardrail`, because it is set on the `AiServices` builder. Additionally, all the output guardrails have a `maxRetries` value == `10`, because the config is also set on the `AiServices` builder.
|
||||
|
||||
The `chat` method has a single output guardrail, `SomeOutputGuardrail`, with a `maxRetries` value == `10`.
|
||||
|
||||
The `chatAndReturnJson` method a single output guardrail, `MyObjectJsonOutputGuardrail` with a `maxRetries` value == `10`.
|
||||
|
||||
## Extension points
|
||||
|
||||
The guardrail system was built in a composable way so it can be extended and reused in other downstream frameworks (such as [Quarkus](https://quarkus.io) or [Spring Boot](https://spring.io/projects/spring-boot)). This section describes some of the extension points or "hooks" that are provided.
|
||||
|
||||
All of these extension points utilize the [Java Service Provider Interface (Java SPI)](https://www.baeldung.com/java-spi).
|
||||
|
||||
| Extension point interface | Purpose |
|
||||
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [`ClassInstanceFactory`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/spi/classloading/ClassInstanceFactory.java) | Provides instanceos of classes.<br/> - Intended to delegate instance creation/retrieval to some other means.<br/> - If not provided, uses reflection to create an instance using the default constructor.<br/> - Other frameworks (like Quarkus or Spring) may use their own bean containers to provide instances of classes. Those frameworks would provide an implementation.<br/> - A Quarkus implementation may look something like [`CDIClassInstanceFactory`](https://github.com/langchain4j/langchain4j/blob/main/integration-tests/integration-tests-class-instance-loader/integration-tests-class-instance-loader-quarkus/src/main/java/com/example/CDIClassInstanceFactory.java)<br/> - A Spring implementation may look something like [`ApplicationContextClassInstanceFactory`](https://github.com/langchain4j/langchain4j/blob/main/integration-tests/integration-tests-class-instance-loader/integration-tests-class-instance-loader-spring/src/main/java/com/example/classes/ApplicationContextClassInstanceFactory.java) |
|
||||
| [`ClassMetadataProviderFactory`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/spi/classloading/ClassMetadataProviderFactory.java) | Provides access to class metadata.<br/> - Used to scan the methods on `AiService` interfaces, and find and process the `@InputGuardrails`/`@OutputGuardrails` annotations.<br/> - [`ReflectionBasedClassMetadataProviderFactory`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/classloading/ReflectionBasedClassMetadataProviderFactory.java) is the default implementation if no others are found, providing class metadata using reflection. |
|
||||
| [`InputGuardrailsConfigBuilderFactory`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/spi/guardrail/config/InputGuardrailsConfigBuilderFactory.java) | - SPI for overriding and/or extending the default [`InputGuardrailsConfigBuilder`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/config/InputGuardrailsConfigBuilder.java)<br/> - Other frameworks may provide their own implementation with extra additional configuration for input guardrails.<br/> - Would also allow other frameworks to drive input guardrail configuration via some other mechanism (i.e. a properties file). |
|
||||
| [`OutputGuardrailsConfigBuilderFactory`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/spi/guardrail/config/OutputGuardrailsConfigBuilderFactory.java) | - SPI for overriding and/or extending the default [`OutputGuardrailsConfigBuilder`](https://github.com/langchain4j/langchain4j/blob/main/langchain4j-core/src/main/java/dev/langchain4j/guardrail/config/OutputGuardrailsConfigBuilder.java)<br/> - Other frameworks may provide their own implementation with extra additional configuration for output guardrails.<br/> - Would also allow other frameworks to drive output guardrail configuration via some other mechanism (i.e. a properties file). |
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 56 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
|
|
@ -0,0 +1,6 @@
|
|||
This contains other full "projects" that can use various LangChain4j features independently but yet aren't necessarily "integration tests".
|
||||
Think of these are separate applications that may be testing some kind of functionality within LangChain4j.
|
||||
|
||||
Think of where `ServiceLoader`s may be invoked - creating a `src/test/META-INF/services` for the service in one of the modules would then override the service being loaded for all tests, which isn't what's intended.
|
||||
|
||||
Instead, we can create isolated projects in here for that.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Some integration tests for the [`ClassInstanceLoader`](../../langchain4j-core/src/main/java/dev/langchain4j/classinstance/ClassInstanceLoader.java).
|
||||
|
|
@ -0,0 +1 @@
|
|||
Some integration tests for the [`ClassInstanceLoader`](../../../langchain4j-core/src/main/java/dev/langchain4j/classinstance/ClassInstanceLoader.java), implemented with Quarkus.
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../../pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-class-instance-loader-quarkus</artifactId>
|
||||
<name>LangChain4j :: Integration Tests :: Class Instance Loader :: Quarkus</name>
|
||||
<description>Tests for the Class Instance Loading abstraction using Quarkus</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<properties>
|
||||
<quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
|
||||
<quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
|
||||
<quarkus.platform.version>3.22.3</quarkus.platform.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>${quarkus.platform.group-id}</groupId>
|
||||
<artifactId>${quarkus.platform.artifact-id}</artifactId>
|
||||
<version>${quarkus.platform.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-arc</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-core</artifactId>
|
||||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-junit5</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>${quarkus.platform.group-id}</groupId>
|
||||
<artifactId>quarkus-maven-plugin</artifactId>
|
||||
<version>${quarkus.platform.version}</version>
|
||||
<extensions>true</extensions>
|
||||
<executions>
|
||||
<execution>
|
||||
<goals>
|
||||
<goal>build</goal>
|
||||
<goal>generate-code</goal>
|
||||
<goal>generate-code-tests</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
package com.example;
|
||||
|
||||
import io.quarkus.runtime.QuarkusApplication;
|
||||
import io.quarkus.runtime.annotations.QuarkusMain;
|
||||
|
||||
@QuarkusMain
|
||||
public class Application implements QuarkusApplication {
|
||||
private final Class1 class1;
|
||||
private final Class2 class2;
|
||||
|
||||
public Application(final Class1 class1, final Class2 class2) {
|
||||
this.class1 = class1;
|
||||
this.class2 = class2;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int run(final String... args) {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
package com.example;
|
||||
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
import io.quarkus.logging.Log;
|
||||
import jakarta.enterprise.inject.spi.CDI;
|
||||
|
||||
public class CDIClassInstanceFactory implements ClassInstanceFactory {
|
||||
@Override
|
||||
public <T> T getInstanceOfClass(Class<T> clazz) {
|
||||
Log.infof("Getting instance of class %s from CDI.", clazz.getName());
|
||||
return CDI.current().select(clazz).get();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package com.example;
|
||||
|
||||
import jakarta.enterprise.context.ApplicationScoped;
|
||||
|
||||
@ApplicationScoped
|
||||
public class Class1 {}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package com.example;
|
||||
|
||||
import jakarta.enterprise.context.ApplicationScoped;
|
||||
|
||||
@ApplicationScoped
|
||||
public class Class2 {}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
package com.example;
|
||||
|
||||
public class Class3 {}
|
||||
|
|
@ -0,0 +1 @@
|
|||
com.example.CDIClassInstanceFactory
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
package dev.langchain4j.classinstance.quarkus;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
|
||||
import com.example.CDIClassInstanceFactory;
|
||||
import com.example.Class1;
|
||||
import com.example.Class2;
|
||||
import com.example.Class3;
|
||||
import dev.langchain4j.classinstance.ClassInstanceLoader;
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
import io.quarkus.test.junit.QuarkusTest;
|
||||
import jakarta.enterprise.inject.UnsatisfiedResolutionException;
|
||||
import jakarta.enterprise.inject.spi.CDI;
|
||||
import java.util.ServiceLoader;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
@QuarkusTest
|
||||
class ClassInstanceLoaderTests {
|
||||
@Test
|
||||
void serviceLoaderFindsCorrectFactory() {
|
||||
assertThat(ServiceLoader.load(ClassInstanceFactory.class).findFirst())
|
||||
.get()
|
||||
.isInstanceOf(CDIClassInstanceFactory.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void loadsClassInstances() {
|
||||
var instance1 = ClassInstanceLoader.getClassInstance(Class1.class);
|
||||
var instance2 = ClassInstanceLoader.getClassInstance(Class1.class);
|
||||
var instance3 = ClassInstanceLoader.getClassInstance(Class2.class);
|
||||
|
||||
assertThat(instance1).isNotNull().isInstanceOf(Class1.class);
|
||||
assertThat(instance2)
|
||||
.isNotNull()
|
||||
.isInstanceOf(Class1.class)
|
||||
.isEqualTo(instance1)
|
||||
.isEqualTo(CDI.current().select(Class1.class).get());
|
||||
assertThat(instance3).isNotNull().isInstanceOf(Class2.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void correctServiceLoader() {
|
||||
assertThatExceptionOfType(UnsatisfiedResolutionException.class)
|
||||
.isThrownBy(() -> ClassInstanceLoader.getClassInstance(Class3.class))
|
||||
.withMessage("No bean found for required type [class %s] and qualifiers [[]]", Class3.class.getName());
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
Some integration tests for the [`ClassInstanceLoader`](../../../langchain4j-core/src/main/java/dev/langchain4j/classinstance/ClassInstanceLoader.java), implemented with Spring.
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../../pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-class-instance-loader-spring</artifactId>
|
||||
<name>LangChain4j :: Integration Tests :: Class Instance Loader :: Spring</name>
|
||||
<description>Tests for the Class Instance Loading abstraction using Spring</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<properties>
|
||||
<slf4j.version>2.0.16</slf4j.version>
|
||||
<logback.version>1.5.16</logback.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<!-- Import dependency management from Spring Boot -->
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-dependencies</artifactId>
|
||||
<version>3.4.5</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-core</artifactId>
|
||||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
package com.example;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
|
||||
@SpringBootApplication
|
||||
public class Application {
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(Application.class, args);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
package com.example;
|
||||
|
||||
import org.springframework.beans.BeansException;
|
||||
import org.springframework.context.ApplicationContext;
|
||||
import org.springframework.context.ApplicationContextAware;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class ApplicationContextProvider implements ApplicationContextAware {
|
||||
private static ApplicationContext context;
|
||||
|
||||
@Override
|
||||
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
|
||||
context = applicationContext;
|
||||
}
|
||||
|
||||
public static ApplicationContext getApplicationContext() {
|
||||
return context;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
package com.example.classes;
|
||||
|
||||
import com.example.ApplicationContextProvider;
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
|
||||
public class ApplicationContextClassInstanceFactory implements ClassInstanceFactory {
|
||||
@Override
|
||||
public <T> T getInstanceOfClass(Class<T> clazz) {
|
||||
return ApplicationContextProvider.getApplicationContext().getBean(clazz);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package com.example.classes;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class Class1 {}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package com.example.classes;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class Class2 {}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
package com.example.classes;
|
||||
|
||||
public class Class3 {}
|
||||
|
|
@ -0,0 +1 @@
|
|||
com.example.classes.ApplicationContextClassInstanceFactory
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
package dev.langchain4j.classinstance.spring;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
|
||||
import com.example.Application;
|
||||
import com.example.classes.ApplicationContextClassInstanceFactory;
|
||||
import com.example.classes.Class1;
|
||||
import com.example.classes.Class2;
|
||||
import com.example.classes.Class3;
|
||||
import dev.langchain4j.classinstance.ClassInstanceLoader;
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
import java.util.ServiceLoader;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.NoSuchBeanDefinitionException;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.context.ApplicationContext;
|
||||
|
||||
@SpringBootTest(classes = Application.class)
|
||||
class ClassInstanceLoaderTests {
|
||||
@Autowired
|
||||
ApplicationContext applicationContext;
|
||||
|
||||
@Test
|
||||
void serviceLoaderFindsCorrectFactory() {
|
||||
assertThat(ServiceLoader.load(ClassInstanceFactory.class).findFirst())
|
||||
.get()
|
||||
.isInstanceOf(ApplicationContextClassInstanceFactory.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void loadsClassInstances() {
|
||||
var instance1 = ClassInstanceLoader.getClassInstance(Class1.class);
|
||||
var instance2 = ClassInstanceLoader.getClassInstance(Class1.class);
|
||||
var instance3 = ClassInstanceLoader.getClassInstance(Class2.class);
|
||||
|
||||
assertThat(instance1).isNotNull().isExactlyInstanceOf(Class1.class);
|
||||
assertThat(instance2)
|
||||
.isNotNull()
|
||||
.isExactlyInstanceOf(Class1.class)
|
||||
.isEqualTo(instance1)
|
||||
.isEqualTo(this.applicationContext.getBean(Class1.class));
|
||||
assertThat(instance3).isNotNull().isExactlyInstanceOf(Class2.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void correctServiceLoader() {
|
||||
assertThatExceptionOfType(NoSuchBeanDefinitionException.class)
|
||||
.isThrownBy(() -> ClassInstanceLoader.getClassInstance(Class3.class))
|
||||
.withMessage("No qualifying bean of type '%s' available", Class3.class.getName());
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-class-instance-loader</artifactId>
|
||||
<name>LangChain4j :: Integration Tests :: Class Instance Loader</name>
|
||||
<description>Tests for the Class Instance Loading abstraction</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-core</artifactId>
|
||||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
package com.example.classloading;
|
||||
|
||||
public final class Classes {
|
||||
private static final Classes INSTANCE = new Classes();
|
||||
|
||||
private Classes() {}
|
||||
|
||||
public static Classes getInstance() {
|
||||
return INSTANCE;
|
||||
}
|
||||
|
||||
public <T> T getInstance(Class<T> clazz) {
|
||||
if (clazz == Class1.class) {
|
||||
return (T) new Class1();
|
||||
}
|
||||
|
||||
if (clazz == Class2.class) {
|
||||
return (T) new Class2();
|
||||
}
|
||||
|
||||
throw new IllegalArgumentException("Unknown class: %s".formatted(clazz.getName()));
|
||||
}
|
||||
|
||||
public static class Class1 {}
|
||||
|
||||
public static class Class2 {}
|
||||
|
||||
public static class Class3 {}
|
||||
}
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
package com.example.classloading;
|
||||
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
|
||||
public class GenericClassInstanceFactory implements ClassInstanceFactory {
|
||||
@Override
|
||||
public <T> T getInstanceOfClass(Class<T> clazz) {
|
||||
return Classes.getInstance().getInstance(clazz);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
com.example.classloading.GenericClassInstanceFactory
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
package dev.langchain4j.classinstance.generic;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
|
||||
import com.example.classloading.Classes;
|
||||
import com.example.classloading.GenericClassInstanceFactory;
|
||||
import dev.langchain4j.classinstance.ClassInstanceLoader;
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
import java.util.ServiceLoader;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class ClassInstanceLoaderTests {
|
||||
@Test
|
||||
void serviceLoaderFindsCorrectFactory() {
|
||||
assertThat(ServiceLoader.load(ClassInstanceFactory.class).findFirst())
|
||||
.get()
|
||||
.isInstanceOf(GenericClassInstanceFactory.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void loadsClassInstances() {
|
||||
var instance1 = ClassInstanceLoader.getClassInstance(Classes.Class1.class);
|
||||
var instance2 = ClassInstanceLoader.getClassInstance(Classes.Class1.class);
|
||||
var instance3 = ClassInstanceLoader.getClassInstance(Classes.Class2.class);
|
||||
|
||||
assertThat(instance1).isNotNull().isExactlyInstanceOf(Classes.Class1.class);
|
||||
assertThat(instance2)
|
||||
.isNotNull()
|
||||
.isExactlyInstanceOf(Classes.Class1.class)
|
||||
.isNotEqualTo(instance1);
|
||||
assertThat(instance3).isNotNull().isExactlyInstanceOf(Classes.Class2.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void correctServiceLoader() {
|
||||
assertThatExceptionOfType(IllegalArgumentException.class)
|
||||
.isThrownBy(() -> ClassInstanceLoader.getClassInstance(Classes.Class3.class))
|
||||
.withMessage("Unknown class: %s", Classes.Class3.class.getName());
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
Some integration tests for the [`ClassMetadataProvider`](../../langchain4j-core/src/main/java/dev/langchain4j/classinstance/ClassMetadataProvider.java).
|
||||
|
|
@ -0,0 +1 @@
|
|||
Some integration tests for the [`ClassMetadataProvider`](../../../langchain4j-core/src/main/java/dev/langchain4j/classinstance/ClassMetadataProvider.java), implemented with Spring.
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../../pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-class-metadata-provider-spring</artifactId>
|
||||
<name>LangChain4j :: Integration Tests :: Class Metadata Provider :: Spring</name>
|
||||
<description>Tests for the Class Metadata Loading abstraction using Spring</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<properties>
|
||||
<slf4j.version>2.0.16</slf4j.version>
|
||||
<logback.version>1.5.16</logback.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<!-- Import dependency management from Spring Boot -->
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-dependencies</artifactId>
|
||||
<version>3.4.5</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j</artifactId>
|
||||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
package com.example;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
|
||||
@SpringBootApplication
|
||||
public class Application {
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(Application.class, args);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
package com.example.classes;
|
||||
|
||||
import dev.langchain4j.Experimental;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Experimental("This is plain and boring!")
|
||||
@Component
|
||||
public class Class1 {
|
||||
public void hello() {}
|
||||
|
||||
@Experimental("Just trying things out")
|
||||
public String goodbye() {
|
||||
return "Goodbye!";
|
||||
}
|
||||
|
||||
public static String wave() {
|
||||
return "Wave!";
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
package com.example.classes;
|
||||
|
||||
import dev.langchain4j.spi.classloading.ClassMetadataProviderFactory;
|
||||
import java.lang.annotation.Annotation;
|
||||
import java.lang.reflect.Method;
|
||||
import java.lang.reflect.Modifier;
|
||||
import java.util.Optional;
|
||||
import java.util.stream.Stream;
|
||||
import org.springframework.core.annotation.AnnotationUtils;
|
||||
import org.springframework.util.ReflectionUtils;
|
||||
|
||||
public class SpringClassMetadataProviderFactory implements ClassMetadataProviderFactory<Method> {
|
||||
@Override
|
||||
public <T extends Annotation> Optional<T> getAnnotation(Method method, Class<T> annotationClass) {
|
||||
return Optional.ofNullable(AnnotationUtils.findAnnotation(method, annotationClass));
|
||||
}
|
||||
|
||||
@Override
|
||||
public <T extends Annotation> Optional<T> getAnnotation(Class<?> clazz, Class<T> annotationClass) {
|
||||
return Optional.ofNullable(AnnotationUtils.findAnnotation(clazz, annotationClass));
|
||||
}
|
||||
|
||||
@Override
|
||||
public Iterable<Method> getNonStaticMethodsOnClass(Class<?> clazz) {
|
||||
return Stream.of(ReflectionUtils.getDeclaredMethods(clazz))
|
||||
.filter(method -> !Modifier.isStatic(method.getModifiers()))
|
||||
.toList();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
com.example.classes.SpringClassMetadataProviderFactory
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
package dev.langchain4j.classinstance.spring;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.Application;
|
||||
import com.example.classes.Class1;
|
||||
import com.example.classes.SpringClassMetadataProviderFactory;
|
||||
import dev.langchain4j.Experimental;
|
||||
import dev.langchain4j.classloading.ClassMetadataProvider;
|
||||
import java.lang.annotation.Target;
|
||||
import java.lang.reflect.Method;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.stream.StreamSupport;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
|
||||
@SpringBootTest(classes = Application.class)
|
||||
class ClassMetadataProviderTests {
|
||||
@Test
|
||||
void serviceLoaderFindsCorrectFactory() {
|
||||
assertThat(ClassMetadataProvider.getClassMetadataProviderFactory())
|
||||
.isInstanceOf(SpringClassMetadataProviderFactory.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void loadsThingsCorrectly() {
|
||||
var factory = ClassMetadataProvider.<Method>getClassMetadataProviderFactory();
|
||||
|
||||
assertThat(factory.getAnnotation(Class1.class, Experimental.class))
|
||||
.get()
|
||||
.extracting(Experimental::value)
|
||||
.isEqualTo("This is plain and boring!");
|
||||
|
||||
assertThat(factory.getAnnotation(Class1.class, Target.class)).isEmpty();
|
||||
|
||||
var methods = factory.getNonStaticMethodsOnClass(Class1.class);
|
||||
|
||||
assertThat(methods).hasSize(2).extracting(Method::getName).containsExactlyInAnyOrder("hello", "goodbye");
|
||||
|
||||
var methodsByName = StreamSupport.stream(methods.spliterator(), false)
|
||||
.collect(Collectors.toMap(Method::getName, method -> method));
|
||||
|
||||
var helloMethod = methodsByName.get("hello");
|
||||
var goodbyeMethod = methodsByName.get("goodbye");
|
||||
|
||||
assertThat(helloMethod).isNotNull();
|
||||
|
||||
assertThat(goodbyeMethod).isNotNull();
|
||||
|
||||
assertThat(factory.getAnnotation(goodbyeMethod, Experimental.class))
|
||||
.get()
|
||||
.extracting(Experimental::value)
|
||||
.isEqualTo("Just trying things out");
|
||||
|
||||
assertThat(factory.getAnnotation(goodbyeMethod, Target.class)).isEmpty();
|
||||
|
||||
assertThat(factory.getAnnotation(helloMethod, Experimental.class)).isEmpty();
|
||||
|
||||
assertThat(factory.getAnnotation(helloMethod, Target.class)).isEmpty();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-class-metadata-provider</artifactId>
|
||||
<name>LangChain4j :: Integration Tests :: Class Metadata Provider</name>
|
||||
<description>Tests for the Class Metadata Loading abstraction</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j</artifactId>
|
||||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
package com.example.classloading;
|
||||
|
||||
import dev.langchain4j.Experimental;
|
||||
|
||||
@Experimental("This is plain and boring!")
|
||||
public class Class1 {
|
||||
public void hello() {}
|
||||
|
||||
@Experimental("Just trying things out")
|
||||
public String goodbye() {
|
||||
return "Goodbye!";
|
||||
}
|
||||
|
||||
public static String wave() {
|
||||
return "Wave!";
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
package com.example.classloading;
|
||||
|
||||
import dev.langchain4j.spi.classloading.ClassMetadataProviderFactory;
|
||||
import java.lang.annotation.Annotation;
|
||||
import java.lang.reflect.Method;
|
||||
import java.lang.reflect.Modifier;
|
||||
import java.util.Optional;
|
||||
import java.util.stream.Stream;
|
||||
|
||||
public class GenericClassMetadataProviderFactory implements ClassMetadataProviderFactory<Method> {
|
||||
@Override
|
||||
public <T extends Annotation> Optional<T> getAnnotation(Method method, Class<T> annotationClass) {
|
||||
return Optional.ofNullable(method.getAnnotation(annotationClass));
|
||||
}
|
||||
|
||||
@Override
|
||||
public <T extends Annotation> Optional<T> getAnnotation(Class<?> clazz, Class<T> annotationClass) {
|
||||
return Optional.ofNullable(clazz.getAnnotation(annotationClass));
|
||||
}
|
||||
|
||||
@Override
|
||||
public Iterable<Method> getNonStaticMethodsOnClass(Class<?> clazz) {
|
||||
return Stream.of(clazz.getDeclaredMethods())
|
||||
.filter(method -> !Modifier.isStatic(method.getModifiers()))
|
||||
.toList();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
com.example.classloading.GenericClassMetadataProviderFactory
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
package dev.langchain4j.classinstance.generic;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.classloading.Class1;
|
||||
import com.example.classloading.GenericClassMetadataProviderFactory;
|
||||
import dev.langchain4j.Experimental;
|
||||
import dev.langchain4j.classloading.ClassMetadataProvider;
|
||||
import java.lang.annotation.Target;
|
||||
import java.lang.reflect.Method;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.stream.StreamSupport;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class ClassMetadataProviderTests {
|
||||
@Test
|
||||
void serviceLoaderFindsCorrectFactory() {
|
||||
assertThat(ClassMetadataProvider.getClassMetadataProviderFactory())
|
||||
.isInstanceOf(GenericClassMetadataProviderFactory.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void loadsThingsCorrectly() {
|
||||
var factory = ClassMetadataProvider.<Method>getClassMetadataProviderFactory();
|
||||
|
||||
assertThat(factory.getAnnotation(Class1.class, Experimental.class))
|
||||
.get()
|
||||
.extracting(Experimental::value)
|
||||
.isEqualTo("This is plain and boring!");
|
||||
|
||||
assertThat(factory.getAnnotation(Class1.class, Target.class)).isEmpty();
|
||||
|
||||
var methods = factory.getNonStaticMethodsOnClass(Class1.class);
|
||||
|
||||
assertThat(methods).hasSize(2).extracting(Method::getName).containsExactlyInAnyOrder("hello", "goodbye");
|
||||
|
||||
var methodsByName = StreamSupport.stream(methods.spliterator(), false)
|
||||
.collect(Collectors.toMap(Method::getName, method -> method));
|
||||
|
||||
var helloMethod = methodsByName.get("hello");
|
||||
var goodbyeMethod = methodsByName.get("goodbye");
|
||||
|
||||
assertThat(helloMethod).isNotNull();
|
||||
|
||||
assertThat(goodbyeMethod).isNotNull();
|
||||
|
||||
assertThat(factory.getAnnotation(goodbyeMethod, Experimental.class))
|
||||
.get()
|
||||
.extracting(Experimental::value)
|
||||
.isEqualTo("Just trying things out");
|
||||
|
||||
assertThat(factory.getAnnotation(goodbyeMethod, Target.class)).isEmpty();
|
||||
|
||||
assertThat(factory.getAnnotation(helloMethod, Experimental.class)).isEmpty();
|
||||
|
||||
assertThat(factory.getAnnotation(helloMethod, Target.class)).isEmpty();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
Some integration tests for Guardrails on AiServices
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-guardrails</artifactId>
|
||||
<name>LangChain4j :: Integration Tests :: Guardrails</name>
|
||||
<description>Tests for Guardrails</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j</artifactId>
|
||||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.junit.jupiter</groupId>
|
||||
<artifactId>junit-jupiter-params</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
package com.example;
|
||||
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import java.util.Map;
|
||||
|
||||
public class InputGuardrailValidation implements InputGuardrail {
|
||||
private static final InputGuardrailValidation INSTANCE = new InputGuardrailValidation();
|
||||
private InputGuardrailRequest params;
|
||||
|
||||
private InputGuardrailValidation() {}
|
||||
|
||||
public static InputGuardrailValidation getInstance() {
|
||||
return INSTANCE;
|
||||
}
|
||||
|
||||
public InputGuardrailResult validate(InputGuardrailRequest params) {
|
||||
this.params = params;
|
||||
return success();
|
||||
}
|
||||
|
||||
public void reset() {
|
||||
this.params = null;
|
||||
}
|
||||
|
||||
public String spyUserMessageTemplate() {
|
||||
return params.requestParams().userMessageTemplate();
|
||||
}
|
||||
|
||||
public String spyUserMessageText() {
|
||||
return params.userMessage().singleText();
|
||||
}
|
||||
|
||||
public Map<String, Object> spyVariables() {
|
||||
return params.requestParams().variables();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
package com.example;
|
||||
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import java.util.Map;
|
||||
|
||||
public class OutputGuardrailValidation implements OutputGuardrail {
|
||||
private static final OutputGuardrailValidation INSTANCE = new OutputGuardrailValidation();
|
||||
private OutputGuardrailRequest params;
|
||||
|
||||
private OutputGuardrailValidation() {}
|
||||
|
||||
public static OutputGuardrailValidation getInstance() {
|
||||
return INSTANCE;
|
||||
}
|
||||
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.params = params;
|
||||
return success();
|
||||
}
|
||||
|
||||
public void reset() {
|
||||
this.params = null;
|
||||
}
|
||||
|
||||
public String spyUserMessageTemplate() {
|
||||
return params.requestParams().userMessageTemplate();
|
||||
}
|
||||
|
||||
public Map<String, Object> spyVariables() {
|
||||
return params.requestParams().variables();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
package com.example;
|
||||
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
import java.lang.reflect.InvocationTargetException;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.ConcurrentMap;
|
||||
|
||||
/**
|
||||
* A factory for providing class instances that are singletons
|
||||
*/
|
||||
public class SingletonClassInstanceFactory implements ClassInstanceFactory {
|
||||
private static final ConcurrentMap<Class<?>, Object> INSTANCES = new ConcurrentHashMap<>(5);
|
||||
|
||||
public static <T> T getInstance(Class<T> clazz) {
|
||||
if (clazz == InputGuardrailValidation.class) {
|
||||
return (T) InputGuardrailValidation.getInstance();
|
||||
}
|
||||
|
||||
if (clazz == OutputGuardrailValidation.class) {
|
||||
return (T) OutputGuardrailValidation.getInstance();
|
||||
}
|
||||
|
||||
return getClassInstance(clazz);
|
||||
}
|
||||
|
||||
public static void clearInstances() {
|
||||
INSTANCES.clear();
|
||||
}
|
||||
|
||||
@Override
|
||||
public <T> T getInstanceOfClass(Class<T> clazz) {
|
||||
return getInstance(clazz);
|
||||
}
|
||||
|
||||
private static <T> T getClassInstance(Class<T> clazz) {
|
||||
return (T) INSTANCES.computeIfAbsent(clazz, SingletonClassInstanceFactory::createNewClassInstance);
|
||||
}
|
||||
|
||||
private static <T> T createNewClassInstance(Class<T> clazz) {
|
||||
try {
|
||||
return clazz.getDeclaredConstructor().newInstance();
|
||||
} catch (InstantiationException
|
||||
| IllegalAccessException
|
||||
| InvocationTargetException
|
||||
| NoSuchMethodException e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
com.example.SingletonClassInstanceFactory
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
|
||||
import dev.langchain4j.service.AiServices;
|
||||
import dev.langchain4j.service.TokenStream;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import java.util.function.Function;
|
||||
import java.util.function.Supplier;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
|
||||
public abstract class BaseGuardrailTests {
|
||||
@BeforeEach
|
||||
void beforeEach() {
|
||||
SingletonClassInstanceFactory.clearInstances();
|
||||
}
|
||||
|
||||
static String execute(Supplier<TokenStream> aiServiceInvocation) throws InterruptedException {
|
||||
var latch = new CountDownLatch(1);
|
||||
var value = new AtomicReference<String>();
|
||||
|
||||
aiServiceInvocation
|
||||
.get()
|
||||
.onError(t -> {
|
||||
throw new RuntimeException(t);
|
||||
})
|
||||
.onPartialResponse(token -> {})
|
||||
.onCompleteResponse(response -> {
|
||||
value.set(response.aiMessage().text());
|
||||
latch.countDown();
|
||||
})
|
||||
.start();
|
||||
|
||||
latch.await(10, TimeUnit.SECONDS);
|
||||
|
||||
return value.get();
|
||||
}
|
||||
|
||||
static <T> T createAiService(Class<T> clazz, Function<AiServices<T>, AiServices<T>> builderCustomizer) {
|
||||
return createAiService(clazz, List.of(), List.of(), builderCustomizer);
|
||||
}
|
||||
|
||||
static <T> T createAiService(Class<T> clazz) {
|
||||
return createAiService(clazz, Function.identity());
|
||||
}
|
||||
|
||||
static <T, I extends InputGuardrail, O extends OutputGuardrail> T createAiService(
|
||||
Class<T> clazz,
|
||||
List<Class<? extends I>> inputGuardrailClasses,
|
||||
List<Class<? extends O>> outputGuardrailClasses) {
|
||||
|
||||
return createAiService(clazz, inputGuardrailClasses, outputGuardrailClasses, Function.identity());
|
||||
}
|
||||
|
||||
static <T, I extends InputGuardrail, O extends OutputGuardrail> T createAiService(
|
||||
Class<T> clazz,
|
||||
List<Class<? extends I>> inputGuardrailClasses,
|
||||
List<Class<? extends O>> outputGuardrailClasses,
|
||||
Function<AiServices<T>, AiServices<T>> builderCustomizer) {
|
||||
|
||||
var builder = AiServices.builder(clazz)
|
||||
.chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(10))
|
||||
.inputGuardrailClasses(inputGuardrailClasses)
|
||||
.outputGuardrailClasses(outputGuardrailClasses);
|
||||
|
||||
return builderCustomizer.apply(builder).build();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.data.message.UserMessage;
|
||||
import dev.langchain4j.model.chat.ChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
|
||||
/**
|
||||
* A {@link ChatModel} that echoes out the {@link UserMessage}
|
||||
*/
|
||||
public class EchoChatModel implements ChatModel {
|
||||
@Override
|
||||
public ChatResponse doChat(ChatRequest chatRequest) {
|
||||
var userMessage = ((UserMessage) chatRequest.messages().get(0)).singleText();
|
||||
|
||||
return ChatResponse.builder().aiMessage(AiMessage.from(userMessage)).build();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,211 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailException;
|
||||
import dev.langchain4j.guardrail.InputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.model.chat.ChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class InputAndOutputGuardrailsTests extends BaseGuardrailTests {
|
||||
MyAiService service = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void ok() {
|
||||
var okIn = SingletonClassInstanceFactory.getInstance(MyOkInputGuardrail.class);
|
||||
var okOut = SingletonClassInstanceFactory.getInstance(MyOkOutputGuardrail.class);
|
||||
assertThat(okIn.getSpy()).isEqualTo(0);
|
||||
assertThat(okOut.getSpy()).isEqualTo(0);
|
||||
service.bothOk("1", "foo");
|
||||
assertThat(okIn.getSpy()).isEqualTo(1);
|
||||
assertThat(okOut.getSpy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void inKo() {
|
||||
var koIn = SingletonClassInstanceFactory.getInstance(MyKoInputGuardrail.class);
|
||||
var okOut = SingletonClassInstanceFactory.getInstance(MyOkOutputGuardrail.class);
|
||||
assertThat(koIn.getSpy()).isEqualTo(0);
|
||||
assertThat(okOut.getSpy()).isEqualTo(0);
|
||||
|
||||
assertThatExceptionOfType(InputGuardrailException.class)
|
||||
.isThrownBy(() -> service.inKo("2", "foo"))
|
||||
.withCauseExactlyInstanceOf(ValidationException.class)
|
||||
.havingRootCause()
|
||||
.withMessage("boom");
|
||||
assertThat(koIn.getSpy()).isEqualTo(1);
|
||||
assertThat(okOut.getSpy()).isEqualTo(0);
|
||||
}
|
||||
|
||||
@Test
|
||||
void outKo() {
|
||||
var okIn = SingletonClassInstanceFactory.getInstance(MyOkInputGuardrail.class);
|
||||
var koOut = SingletonClassInstanceFactory.getInstance(MyKoOutputGuardrail.class);
|
||||
assertThat(okIn.getSpy()).isEqualTo(0);
|
||||
assertThat(koOut.getSpy()).isEqualTo(0);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> service.outKo("2", "foo"))
|
||||
.withCauseExactlyInstanceOf(ValidationException.class)
|
||||
.havingRootCause()
|
||||
.withMessage("boom");
|
||||
assertThat(okIn.getSpy()).isEqualTo(1);
|
||||
assertThat(koOut.getSpy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void retry() {
|
||||
var okIn = SingletonClassInstanceFactory.getInstance(MyOkInputGuardrail.class);
|
||||
var koOutWithRetry = SingletonClassInstanceFactory.getInstance(MyKoWithRetryOutputGuardrail.class);
|
||||
assertThat(okIn.getSpy()).isEqualTo(0);
|
||||
assertThat(koOutWithRetry.getSpy()).isEqualTo(0);
|
||||
service.outKoWithRetry("2", "foo");
|
||||
assertThat(okIn.getSpy()).isEqualTo(1);
|
||||
assertThat(koOutWithRetry.getSpy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void reprompt() {
|
||||
var okIn = SingletonClassInstanceFactory.getInstance(MyOkInputGuardrail.class);
|
||||
var koOutWithReprompt = SingletonClassInstanceFactory.getInstance(MyKoWithRepromprOutputGuardrail.class);
|
||||
assertThat(okIn.getSpy()).isEqualTo(0);
|
||||
assertThat(koOutWithReprompt.getSpy()).isEqualTo(0);
|
||||
service.outKoWithReprompt("2", "foo");
|
||||
assertThat(okIn.getSpy()).isEqualTo(1);
|
||||
assertThat(koOutWithReprompt.getSpy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@InputGuardrails(MyOkInputGuardrail.class)
|
||||
@OutputGuardrails(MyOkOutputGuardrail.class)
|
||||
String bothOk(@MemoryId String id, @UserMessage String message);
|
||||
|
||||
@InputGuardrails(MyKoInputGuardrail.class)
|
||||
@OutputGuardrails(MyOkOutputGuardrail.class)
|
||||
String inKo(@MemoryId String id, @UserMessage String message);
|
||||
|
||||
@InputGuardrails(MyOkInputGuardrail.class)
|
||||
@OutputGuardrails(MyKoOutputGuardrail.class)
|
||||
String outKo(@MemoryId String id, @UserMessage String message);
|
||||
|
||||
@InputGuardrails(MyOkInputGuardrail.class)
|
||||
@OutputGuardrails(MyKoWithRetryOutputGuardrail.class)
|
||||
String outKoWithRetry(@MemoryId String id, @UserMessage String message);
|
||||
|
||||
@InputGuardrails(MyOkInputGuardrail.class)
|
||||
@OutputGuardrails(MyKoWithRepromprOutputGuardrail.class)
|
||||
String outKoWithReprompt(@MemoryId String id, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyOkInputGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger();
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(InputGuardrailRequest params) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyKoInputGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger();
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(InputGuardrailRequest params) {
|
||||
spy.incrementAndGet();
|
||||
return failure("boom", new ValidationException("boom"));
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyOkOutputGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
spy.incrementAndGet();
|
||||
return OutputGuardrailResult.success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyKoOutputGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
spy.incrementAndGet();
|
||||
return failure("boom", new ValidationException("boom"));
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyKoWithRetryOutputGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
if (spy.incrementAndGet() == 1) {
|
||||
return retry("KO");
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyKoWithRepromprOutputGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
if (spy.incrementAndGet() == 1) {
|
||||
return reprompt("KO", "retry");
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyChatModel implements ChatModel {
|
||||
@Override
|
||||
public ChatResponse doChat(ChatRequest chatRequest) {
|
||||
return ChatResponse.builder().aiMessage(AiMessage.from("Hi!")).build();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,136 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailException;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class InputGuardrailChainTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void guardrailChainsAreInvoked() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
aiService.firstOneTwo("1", "foo");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(firstGuardrail.lastAccess()).isLessThan(secondGuardrail.lastAccess());
|
||||
}
|
||||
|
||||
@Test
|
||||
void guardrailOrderIsCorrect() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
aiService.twoAndFirst("1", "foo");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.lastAccess()).isLessThan(firstGuardrail.lastAccess());
|
||||
}
|
||||
|
||||
@Test
|
||||
void failTheChain() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
var failingGuardrail = SingletonClassInstanceFactory.getInstance(FailingGuardrail.class);
|
||||
|
||||
assertThatThrownBy(() -> aiService.failingFirstTwo("1", "foo"))
|
||||
.isInstanceOf(InputGuardrailException.class)
|
||||
.hasCauseInstanceOf(ValidationException.class)
|
||||
.hasRootCauseMessage("boom");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(0);
|
||||
assertThat(failingGuardrail.spy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@InputGuardrails({FirstGuardrail.class, SecondGuardrail.class})
|
||||
String firstOneTwo(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@InputGuardrails({SecondGuardrail.class, FirstGuardrail.class})
|
||||
String twoAndFirst(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@InputGuardrails({FirstGuardrail.class, FailingGuardrail.class, SecondGuardrail.class})
|
||||
String failingFirstTwo(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class FirstGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private final AtomicLong lastAccess = new AtomicLong();
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
lastAccess.set(System.nanoTime());
|
||||
|
||||
try {
|
||||
TimeUnit.MILLISECONDS.sleep(1);
|
||||
} catch (InterruptedException e) {
|
||||
// Ignore me
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public long lastAccess() {
|
||||
return lastAccess.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class SecondGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private volatile AtomicLong lastAccess = new AtomicLong();
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
lastAccess.set(System.nanoTime());
|
||||
try {
|
||||
TimeUnit.MILLISECONDS.sleep(1);
|
||||
} catch (InterruptedException e) {
|
||||
// Ignore me
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public long lastAccess() {
|
||||
return lastAccess.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class FailingGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
if (spy.incrementAndGet() == 1) {
|
||||
return fatal("boom", new ValidationException("boom"));
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.stream.Stream;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.Arguments;
|
||||
import org.junit.jupiter.params.provider.MethodSource;
|
||||
|
||||
class InputGuardrailOnClassAndMethodTest extends BaseGuardrailTests {
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void guardrailsFromTheClassAreInvoked(String testDescription, MyAiService aiService) {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
aiService.hi("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.hi("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
|
||||
assertThat(koGuardrail.spy()).isEqualTo(0);
|
||||
}
|
||||
|
||||
static Stream<Arguments> services() {
|
||||
return Stream.of(
|
||||
Arguments.of("Using AiServices builder", MyAiServiceWithoutClassAnnotations.create()),
|
||||
Arguments.of("Using annotation at class level", MyAiServiceUsingClassAnnotations.create()));
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(OKGuardrail.class)
|
||||
String hi(@MemoryId String mem);
|
||||
}
|
||||
|
||||
public interface MyAiServiceWithoutClassAnnotations extends MyAiService {
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiServiceWithoutClassAnnotations.class,
|
||||
List.of(OKGuardrail.class),
|
||||
List.of(),
|
||||
builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
@InputGuardrails(KOGuardrail.class)
|
||||
public interface MyAiServiceUsingClassAnnotations extends MyAiService {
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiServiceUsingClassAnnotations.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.stream.Stream;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.Arguments;
|
||||
import org.junit.jupiter.params.provider.MethodSource;
|
||||
|
||||
class InputGuardrailOnClassTest extends BaseGuardrailTests {
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void guardrailsFromTheClassAreInvoked(String testDescription, MyAiService aiService) {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
aiService.hi("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.hi("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
static Stream<Arguments> services() {
|
||||
return Stream.of(
|
||||
Arguments.of("Using AiServices builder", MyAiService.create()),
|
||||
Arguments.of("Using annotation at class level", MyAiServiceWithClassAnnotation.create()));
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
String hi(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiService.class,
|
||||
List.of(OKGuardrail.class),
|
||||
List.of(),
|
||||
builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
@InputGuardrails(OKGuardrail.class)
|
||||
public interface MyAiServiceWithClassAnnotation extends MyAiService {
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiServiceWithClassAnnotation.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage ignored) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,207 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.InputGuardrailValidation;
|
||||
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
|
||||
import dev.langchain4j.service.AiServices;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import dev.langchain4j.service.V;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.stream.Stream;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.Arguments;
|
||||
import org.junit.jupiter.params.provider.MethodSource;
|
||||
|
||||
class InputGuardrailPromptTemplateTests {
|
||||
@BeforeEach
|
||||
void beforeEach() {
|
||||
InputGuardrailValidation.getInstance().reset();
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkNoParameters(String testDescription, Assistant aiService) {
|
||||
assertThat(aiService.getJoke()).isEqualTo("Request: Tell me a joke; Response: Hi!");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me a joke");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables()).isEmpty();
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkWithMemoryId(String testDescription, Assistant aiService) {
|
||||
aiService.getAnotherJoke("memory-id-001");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me another joke");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of("arg0", "memory-id-001"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkWithNoMemoryIdAndOneParameter(String testDescription, Assistant aiService) {
|
||||
aiService.sayHiToMyFriendNoMemory("Rambo");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Say hi to my friend {{it}}!");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"arg0", "Rambo",
|
||||
"it", "Rambo"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkWithMemoryIdAndOneParameter(String testDescription, Assistant aiService) {
|
||||
aiService.sayHiToMyFriend("1", "Chuck Norris");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Say hi to my friend {{friend}}!");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"friend", "Chuck Norris",
|
||||
"arg0", "1"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkWithNoMemoryIdAndThreeParameters(String testDescription, Assistant aiService) {
|
||||
aiService.sayHiToMyFriends("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me something about {{topic1}}, {{topic2}}, {{topic3}}!");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"topic1", "Chuck Norris",
|
||||
"topic2", "Jean-Claude Van Damme",
|
||||
"topic3", "Silvester Stallone"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkWithNoMemoryIdAndList(String testDescription, Assistant aiService) {
|
||||
aiService.sayHiToMyFriends(List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"));
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageText())
|
||||
.isEqualTo("Tell me something about [Chuck Norris, Jean-Claude Van Damme, Silvester Stallone]!");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me something about {{it}}!");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"arg0",
|
||||
List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"),
|
||||
"it",
|
||||
"[Chuck Norris, Jean-Claude Van Damme, Silvester Stallone]"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("assistants")
|
||||
void shouldWorkWithMemoryIdAndList(String testDescription, Assistant aiService) {
|
||||
aiService.sayHiToMyFriends(
|
||||
"memory-id-007", List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"));
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageText())
|
||||
.isEqualTo(
|
||||
"Tell me something about [Chuck Norris, Jean-Claude Van Damme, Silvester Stallone]! This is my memory id: memory-id-007");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me something about {{topics}}! This is my memory id: {{memoryId}}");
|
||||
assertThat(InputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"topics",
|
||||
List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"),
|
||||
"memoryId",
|
||||
"memory-id-007"));
|
||||
}
|
||||
|
||||
static Stream<Arguments> assistants() {
|
||||
return Stream.of(
|
||||
Arguments.of("Assistant with class-level annotation", ClassLevelAssistant.create()),
|
||||
Arguments.of("Assistant with method-level annotations", MethodLevelAssistant.create()),
|
||||
Arguments.of("Assistant with builder-style guardrail", Assistant.create()));
|
||||
}
|
||||
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
interface ClassLevelAssistant extends Assistant {
|
||||
static Assistant create() {
|
||||
return Assistant.create(ClassLevelAssistant.class);
|
||||
}
|
||||
}
|
||||
|
||||
interface MethodLevelAssistant extends Assistant {
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@UserMessage("Tell me a joke")
|
||||
@Override
|
||||
String getJoke();
|
||||
|
||||
@UserMessage("Tell me another joke")
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@Override
|
||||
String getAnotherJoke(@MemoryId String memoryId);
|
||||
|
||||
@UserMessage("Say hi to my friend {{it}}!")
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@Override
|
||||
String sayHiToMyFriendNoMemory(String friend);
|
||||
|
||||
@UserMessage("Say hi to my friend {{friend}}!")
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@Override
|
||||
String sayHiToMyFriend(@MemoryId String mem, @V("friend") String friend);
|
||||
|
||||
@UserMessage("Tell me something about {{topic1}}, {{topic2}}, {{topic3}}!")
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@Override
|
||||
String sayHiToMyFriends(@V("topic1") String topic1, @V("topic2") String topic2, @V("topic3") String topic3);
|
||||
|
||||
@UserMessage("Tell me something about {{it}}!")
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@Override
|
||||
String sayHiToMyFriends(List<String> topics);
|
||||
|
||||
@UserMessage("Tell me something about {{topics}}! This is my memory id: {{memoryId}}")
|
||||
@InputGuardrails(InputGuardrailValidation.class)
|
||||
@Override
|
||||
String sayHiToMyFriends(@V("memoryId") @MemoryId String memoryId, @V("topics") List<String> topics);
|
||||
|
||||
static Assistant create() {
|
||||
return Assistant.create(MethodLevelAssistant.class);
|
||||
}
|
||||
}
|
||||
|
||||
interface Assistant {
|
||||
@UserMessage("Tell me a joke")
|
||||
String getJoke();
|
||||
|
||||
@UserMessage("Tell me another joke")
|
||||
String getAnotherJoke(@MemoryId String memoryId);
|
||||
|
||||
@UserMessage("Say hi to my friend {{it}}!")
|
||||
String sayHiToMyFriendNoMemory(String friend);
|
||||
|
||||
@UserMessage("Say hi to my friend {{friend}}!")
|
||||
String sayHiToMyFriend(@MemoryId String mem, @V("friend") String friend);
|
||||
|
||||
@UserMessage("Tell me something about {{topic1}}, {{topic2}}, {{topic3}}!")
|
||||
String sayHiToMyFriends(@V("topic1") String topic1, @V("topic2") String topic2, @V("topic3") String topic3);
|
||||
|
||||
@UserMessage("Tell me something about {{it}}!")
|
||||
String sayHiToMyFriends(List<String> topics);
|
||||
|
||||
@UserMessage("Tell me something about {{topics}}! This is my memory id: {{memoryId}}")
|
||||
String sayHiToMyFriends(@V("memoryId") @MemoryId String memoryId, @V("topics") List<String> topics);
|
||||
|
||||
static <T extends Assistant> T create(Class<T> clazz) {
|
||||
return AiServices.builder(clazz)
|
||||
.chatModel(new MyChatModel())
|
||||
.chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(10))
|
||||
.build();
|
||||
}
|
||||
|
||||
static Assistant create() {
|
||||
return AiServices.builder(Assistant.class)
|
||||
.chatModel(new MyChatModel())
|
||||
.chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(10))
|
||||
.inputGuardrails(List.of(InputGuardrailValidation.getInstance()))
|
||||
.build();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import dev.langchain4j.service.V;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class InputGuardrailRewritingTests extends BaseGuardrailTests {
|
||||
@Test
|
||||
void rewriting() {
|
||||
assertThat(MyAiService.create().test("first prompt", "second prompt"))
|
||||
.hasSize(MessageTruncatingGuardrail.MAX_LENGTH);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Given {{first}} and {{second}} do something")
|
||||
@InputGuardrails(MessageTruncatingGuardrail.class)
|
||||
String test(@V("first") String first, @V("second") String second);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new EchoChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class MessageTruncatingGuardrail implements InputGuardrail {
|
||||
static final int MAX_LENGTH = 20;
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
String text = um.singleText();
|
||||
return successWith(text.substring(0, MAX_LENGTH));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class InputGuardrailTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void guardrailsAreInvoked() {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
aiService.hi("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.hi("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void guardrailCanThrowValidationException() {
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(0);
|
||||
assertThatThrownBy(() -> aiService.ko("1")).hasCauseExactlyInstanceOf(ValidationException.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(1);
|
||||
assertThatThrownBy(() -> aiService.ko("1")).hasCauseExactlyInstanceOf(ValidationException.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(OKGuardrail.class)
|
||||
String hi(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(KOGuardrail.class)
|
||||
String ko(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO", new ValidationException("KO"));
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,234 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
import static org.assertj.core.data.Index.atIndex;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.InputGuardrail;
|
||||
import dev.langchain4j.guardrail.InputGuardrailException;
|
||||
import dev.langchain4j.guardrail.InputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult;
|
||||
import dev.langchain4j.memory.ChatMemory;
|
||||
import dev.langchain4j.model.chat.ChatModel;
|
||||
import dev.langchain4j.model.chat.StreamingChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import dev.langchain4j.model.chat.response.StreamingChatResponseHandler;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.TokenStream;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class InputGuardrailValidationTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void ok() {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
aiService.ok("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.ok("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void ko() {
|
||||
assertThatThrownBy(() -> aiService.ko("2"))
|
||||
.isInstanceOf(InputGuardrailException.class)
|
||||
.hasMessageContaining("KO");
|
||||
}
|
||||
|
||||
@Test
|
||||
void okStreaming() throws InterruptedException {
|
||||
var latch = new CountDownLatch(1);
|
||||
var text = new AtomicReference<String>();
|
||||
var partialResponses = new ArrayList<String>();
|
||||
|
||||
aiService
|
||||
.okStream("1")
|
||||
.onError(t -> latch.countDown())
|
||||
.onPartialResponse(partialResponses::add)
|
||||
.onCompleteResponse(response -> {
|
||||
text.set(response.aiMessage().text());
|
||||
latch.countDown();
|
||||
})
|
||||
.start();
|
||||
|
||||
latch.await(10, TimeUnit.SECONDS);
|
||||
|
||||
assertThat(String.join(" ", text.get())).isEqualTo("Streaming hi !");
|
||||
assertThat(String.join(" ", partialResponses)).isEqualTo(text.get());
|
||||
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void koStreaming() {
|
||||
assertThatExceptionOfType(InputGuardrailException.class)
|
||||
.isThrownBy(() -> aiService.koStream("2"))
|
||||
.withMessageContaining("KO");
|
||||
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void fatalException() {
|
||||
assertThatExceptionOfType(InputGuardrailException.class)
|
||||
.isThrownBy(() -> aiService.fatal("5"))
|
||||
.withMessageContaining("Fatal");
|
||||
|
||||
var fatal = SingletonClassInstanceFactory.getInstance(KOFatalGuardrail.class);
|
||||
assertThat(fatal.spy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void memoryCheck() {
|
||||
var memoryCheck = SingletonClassInstanceFactory.getInstance(MemoryCheck.class);
|
||||
aiService.test("1", "foo");
|
||||
assertThat(memoryCheck.spy()).isEqualTo(1);
|
||||
|
||||
aiService.test("1", "bar");
|
||||
assertThat(memoryCheck.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(OKGuardrail.class)
|
||||
String ok(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(KOGuardrail.class)
|
||||
String ko(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(OKGuardrail.class)
|
||||
TokenStream okStream(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(KOGuardrail.class)
|
||||
TokenStream koStream(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@InputGuardrails(KOFatalGuardrail.class)
|
||||
String fatal(@MemoryId String mem);
|
||||
|
||||
@InputGuardrails(MemoryCheck.class)
|
||||
String test(@MemoryId String name, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel())
|
||||
.streamingChatModel(new MyStreamingChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOFatalGuardrail implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(dev.langchain4j.data.message.UserMessage um) {
|
||||
spy.incrementAndGet();
|
||||
throw new IllegalArgumentException("Fatal");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MemoryCheck implements InputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public InputGuardrailResult validate(InputGuardrailRequest params) {
|
||||
spy.incrementAndGet();
|
||||
var messages = Optional.ofNullable(params.requestParams().chatMemory())
|
||||
.map(ChatMemory::messages)
|
||||
.orElseGet(List::of);
|
||||
|
||||
if (messages.isEmpty()) {
|
||||
assertThat(params.userMessage().singleText()).isEqualTo("foo");
|
||||
}
|
||||
|
||||
if (messages.size() == 2) {
|
||||
assertThat(messages)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isInstanceOf(dev.langchain4j.data.message.UserMessage.class)
|
||||
.extracting(m -> ((dev.langchain4j.data.message.UserMessage) m).singleText())
|
||||
.isEqualTo("foo"),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isInstanceOf(AiMessage.class)
|
||||
.extracting(m -> ((AiMessage) m).text())
|
||||
.isEqualTo("Hi!"),
|
||||
atIndex(1));
|
||||
|
||||
assertThat(params.userMessage().singleText()).isEqualTo("bar");
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyChatModel implements ChatModel {
|
||||
@Override
|
||||
public ChatResponse doChat(ChatRequest chatRequest) {
|
||||
return ChatResponse.builder().aiMessage(AiMessage.from("Hi!")).build();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyStreamingChatModel implements StreamingChatModel {
|
||||
@Override
|
||||
public void doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler) {
|
||||
handler.onPartialResponse("Streaming hi");
|
||||
handler.onPartialResponse("!");
|
||||
handler.onCompleteResponse(ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Streaming hi !"))
|
||||
.build());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.data.message.ChatMessageType;
|
||||
import dev.langchain4j.model.chat.ChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
|
||||
public class MyChatModel implements ChatModel {
|
||||
private static String getUserMessage(ChatRequest chatRequest) {
|
||||
return chatRequest.messages().stream()
|
||||
.filter(message -> message.type() == ChatMessageType.USER)
|
||||
.findFirst()
|
||||
.map(chatMessage -> ((dev.langchain4j.data.message.UserMessage) chatMessage).singleText())
|
||||
.orElseThrow(() -> new IllegalArgumentException("No user message found"));
|
||||
}
|
||||
|
||||
@Override
|
||||
public ChatResponse doChat(ChatRequest chatRequest) {
|
||||
return ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Request: %s; Response: Hi!".formatted(getUserMessage(chatRequest))))
|
||||
.build();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,283 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.atIndex;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.data.message.ChatMessage;
|
||||
import dev.langchain4j.data.message.ChatMessageType;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.memory.ChatMemory;
|
||||
import dev.langchain4j.model.chat.StreamingChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import dev.langchain4j.model.chat.response.StreamingChatResponseHandler;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.TokenStream;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailChainOnStreamedResponseTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void guardrailChainsAreInvoked() throws InterruptedException {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
|
||||
var value = execute(() -> aiService.firstOneTwo("1", "foo"));
|
||||
assertThat(value).isEqualTo("Hi! World! ");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(firstGuardrail.lastAccess()).isLessThan(secondGuardrail.lastAccess());
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void guardrailOrderIsCorrect() throws InterruptedException {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
|
||||
var value = execute(() -> aiService.twoAndFirst("1", "foo"));
|
||||
assertThat(value).isEqualTo("Hi! World! ");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.lastAccess()).isLessThan(firstGuardrail.lastAccess());
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryRestartsTheChain() throws InterruptedException {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
var failingGuardrail = SingletonClassInstanceFactory.getInstance(FailingGuardrail.class);
|
||||
var value = execute(() -> aiService.failingFirstTwo("1", "foo"));
|
||||
assertThat(value).isEqualTo("Hi! World! ");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(2);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(firstGuardrail.lastAccess()).isLessThan(secondGuardrail.lastAccess());
|
||||
assertThat(failingGuardrail.spy()).isEqualTo(2);
|
||||
assertThat(failingGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@OutputGuardrails({FirstGuardrail.class, SecondGuardrail.class})
|
||||
TokenStream firstOneTwo(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails({SecondGuardrail.class, FirstGuardrail.class})
|
||||
TokenStream twoAndFirst(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails({FirstGuardrail.class, FailingGuardrail.class, SecondGuardrail.class})
|
||||
TokenStream failingFirstTwo(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiService.class, builder -> builder.streamingChatModel(new MyStreamingChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class FirstGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private volatile AtomicLong lastAccess = new AtomicLong();
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
lastAccess.set(System.nanoTime());
|
||||
try {
|
||||
TimeUnit.MILLISECONDS.sleep(1);
|
||||
} catch (InterruptedException e) {
|
||||
// Ignore me
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public long lastAccess() {
|
||||
return lastAccess.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class SecondGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private volatile AtomicLong lastAccess = new AtomicLong();
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
lastAccess.set(System.nanoTime());
|
||||
try {
|
||||
TimeUnit.MILLISECONDS.sleep(1);
|
||||
} catch (InterruptedException e) {
|
||||
// Ignore me
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public long lastAccess() {
|
||||
return lastAccess.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class FailingGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
if (spy.incrementAndGet() == 1) {
|
||||
return reprompt("Retry", "Retry");
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyStreamingChatModel implements StreamingChatModel {
|
||||
@Override
|
||||
public void doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler) {
|
||||
handler.onPartialResponse("Hi!");
|
||||
handler.onPartialResponse(" ");
|
||||
handler.onPartialResponse("World!");
|
||||
handler.onCompleteResponse(ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Hi! World! "))
|
||||
.build());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,535 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
import static org.assertj.core.api.Assertions.atIndex;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.data.message.ChatMessage;
|
||||
import dev.langchain4j.data.message.ChatMessageType;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.memory.ChatMemory;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailChainTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void guardrailChainsAreInvoked() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
aiService.firstOneTwo("1", "foo");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(firstGuardrail.lastAccess()).isLessThan(secondGuardrail.lastAccess());
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void guardrailOrderIsCorrect() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
aiService.twoAndFirst("1", "foo");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(secondGuardrail.lastAccess()).isLessThan(firstGuardrail.lastAccess());
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryRepromptRestartsTheChain() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondGuardrail.class);
|
||||
var failingGuardrail = SingletonClassInstanceFactory.getInstance(FailingGuardrail.class);
|
||||
aiService.failingFirstTwo("1", "foo");
|
||||
assertThat(firstGuardrail.spy()).isEqualTo(3);
|
||||
assertThat(secondGuardrail.spy()).isEqualTo(1);
|
||||
assertThat(firstGuardrail.lastAccess()).isLessThan(secondGuardrail.lastAccess());
|
||||
assertThat(failingGuardrail.spy()).isEqualTo(3);
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(failingGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void rewritesTheOutputTwiceInTheChain() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstRewritingGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(SecondRewritingGuardrail.class);
|
||||
assertThat(aiService.rewritingSuccess("1", "foo")).isEqualTo("Request: foo; Response: Hi!,1,2");
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void repromptAfterRewriteIsNotAllowed() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstRewritingGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(RepromptingGuardrail.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> aiService.repromptAfterRewrite("1", "foo"))
|
||||
.withMessageContaining("Retry or reprompt is not allowed after a rewritten output");
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void rewritesTheOutputWithAResult() {
|
||||
var firstGuardrail = SingletonClassInstanceFactory.getInstance(FirstRewritingGuardrail.class);
|
||||
var secondGuardrail = SingletonClassInstanceFactory.getInstance(RewritingGuardrailWithResult.class);
|
||||
assertThat(aiService.rewritingSuccessWithResult("1", "foo")).isSameAs(RewritingGuardrailWithResult.RESULT);
|
||||
assertThat(firstGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
assertThat(secondGuardrail.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@OutputGuardrails({FirstGuardrail.class, SecondGuardrail.class})
|
||||
String firstOneTwo(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails({SecondGuardrail.class, FirstGuardrail.class})
|
||||
String twoAndFirst(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails(
|
||||
value = {FirstGuardrail.class, FailingGuardrail.class, SecondGuardrail.class},
|
||||
maxRetries = 3)
|
||||
String failingFirstTwo(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails({FirstRewritingGuardrail.class, SecondRewritingGuardrail.class})
|
||||
String rewritingSuccess(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails({FirstRewritingGuardrail.class, RepromptingGuardrail.class})
|
||||
String repromptAfterRewrite(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails({FirstRewritingGuardrail.class, RewritingGuardrailWithResult.class})
|
||||
String rewritingSuccessWithResult(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class FirstGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private volatile AtomicLong lastAccess = new AtomicLong();
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
lastAccess.set(System.nanoTime());
|
||||
|
||||
try {
|
||||
TimeUnit.MILLISECONDS.sleep(1);
|
||||
} catch (InterruptedException e) {
|
||||
// Ignore me
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public long lastAccess() {
|
||||
return lastAccess.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class SecondGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private volatile AtomicLong lastAccess = new AtomicLong();
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
lastAccess.set(System.nanoTime());
|
||||
|
||||
try {
|
||||
TimeUnit.MILLISECONDS.sleep(1);
|
||||
} catch (InterruptedException e) {
|
||||
// Ignore me
|
||||
}
|
||||
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public long lastAccess() {
|
||||
return lastAccess.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class FailingGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
int v = spy.incrementAndGet();
|
||||
|
||||
if (v == 1) {
|
||||
return reprompt("Reprompt", "Reprompt");
|
||||
} else if (v == 2) {
|
||||
return retry("Retry");
|
||||
}
|
||||
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class FirstRewritingGuardrail implements OutputGuardrail {
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
String text = responseFromLLM.text();
|
||||
return successWith(text + ",1");
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class SecondRewritingGuardrail implements OutputGuardrail {
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
var text = responseFromLLM.text();
|
||||
return successWith(text + ",2");
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RewritingGuardrailWithResult implements OutputGuardrail {
|
||||
static final String RESULT = String.valueOf(1_000);
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
var text = responseFromLLM.text();
|
||||
return successWith(text + ",2", RESULT);
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RepromptingGuardrail implements OutputGuardrail {
|
||||
private boolean firstCall = true;
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
if (firstCall) {
|
||||
firstCall = false;
|
||||
String text = responseFromLLM.text();
|
||||
return reprompt("Wrong message", text + ", " + text);
|
||||
}
|
||||
|
||||
return success();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailOnClassAndMethodTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void properGuardrailsAreInvoked() {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
aiService.hi("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.hi("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(0);
|
||||
}
|
||||
|
||||
@OutputGuardrails(KOGuardrail.class)
|
||||
public interface MyAiService {
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
String hi(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,102 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.stream.Stream;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.Arguments;
|
||||
import org.junit.jupiter.params.provider.MethodSource;
|
||||
|
||||
class OutputGuardrailOnClassTests extends BaseGuardrailTests {
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void correctGuardrailsAreInvoked(String testDescription, MyAiService aiService) {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
aiService.hi("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.hi("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(0);
|
||||
}
|
||||
|
||||
static Stream<Arguments> services() {
|
||||
return Stream.of(
|
||||
Arguments.of("AiService using builder", MyAiService.create()),
|
||||
Arguments.of("AiService using annotation at class level", MyAiServiceWithClassAnnotation.create()),
|
||||
Arguments.of(
|
||||
"AiService using annotation at class and method level",
|
||||
MyAiServiceWithClassAndMethodAnnotation.create()));
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
String hi(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiService.class,
|
||||
List.of(),
|
||||
List.of(OKGuardrail.class),
|
||||
builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
public interface MyAiServiceWithClassAnnotation extends MyAiService {
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiServiceWithClassAnnotation.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
@OutputGuardrails(KOGuardrail.class)
|
||||
public interface MyAiServiceWithClassAndMethodAnnotation extends MyAiService {
|
||||
@Override
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
String hi(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiServiceWithClassAndMethodAnnotation.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,159 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
import static org.assertj.core.api.Assertions.fail;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.model.chat.StreamingChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import dev.langchain4j.model.chat.response.StreamingChatResponseHandler;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.TokenStream;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.ArrayList;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailOnStreamedResponseTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void outputGuardrailsAreInvoked() throws InterruptedException {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
execute(() -> aiService.ok("1"));
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
execute(() -> aiService.ok("2"));
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void guardrailCanThrowValidationException() {
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(0);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> execute(() -> aiService.ko("1")))
|
||||
.withCauseExactlyInstanceOf(ValidationException.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(1);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> execute(() -> aiService.ko("1")))
|
||||
.withCauseExactlyInstanceOf(ValidationException.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void streamingPartialResponsesBuffered() {
|
||||
var okPartials = new ArrayList<String>();
|
||||
var okComplete = new AtomicReference<ChatResponse>();
|
||||
|
||||
this.aiService
|
||||
.ok("1")
|
||||
.onPartialResponse(okPartials::add)
|
||||
.onError(t -> fail(t.getMessage()))
|
||||
.onCompleteResponse(okComplete::set)
|
||||
.start();
|
||||
|
||||
assertThat(okPartials).hasSize(3).containsExactly("Hi!", " ", "World!");
|
||||
assertThat(okComplete.get())
|
||||
.isNotNull()
|
||||
.extracting(m -> m.aiMessage().text())
|
||||
.isEqualTo("Hi! World! ");
|
||||
}
|
||||
|
||||
@Test
|
||||
void streamingPartialResponsesNotShownOnError() {
|
||||
var repromptPartials = new ArrayList<String>();
|
||||
var repromptComplete = new AtomicReference<ChatResponse>();
|
||||
|
||||
assertThatExceptionOfType(OutputGuardrailException.class).isThrownBy(() -> this.aiService
|
||||
.reprompt("1")
|
||||
.onPartialResponse(repromptPartials::add)
|
||||
.onError(t -> fail(t.getMessage()))
|
||||
.onCompleteResponse(repromptComplete::set)
|
||||
.start());
|
||||
|
||||
assertThat(repromptPartials).isEmpty();
|
||||
assertThat(repromptComplete.get()).isNull();
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
TokenStream ok(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(KOGuardrail.class)
|
||||
TokenStream ko(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(RepromptGuardrail.class)
|
||||
TokenStream reprompt(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiService.class, builder -> builder.streamingChatModel(new MyStreamingChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RepromptGuardrail implements OutputGuardrail {
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
return reprompt("reorompt", "reprompt");
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
if (responseFromLLM.text().length() > 3) { // Accumulated response.
|
||||
return failure("KO", new ValidationException("KO"));
|
||||
} else { // Chunk, do not fail on the first chunk
|
||||
if (responseFromLLM.text().contains("Hi!")) {
|
||||
return success();
|
||||
} else {
|
||||
return failure("KO", new ValidationException("KO"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyStreamingChatModel implements StreamingChatModel {
|
||||
@Override
|
||||
public void doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler) {
|
||||
handler.onPartialResponse("Hi!");
|
||||
handler.onPartialResponse(" ");
|
||||
handler.onPartialResponse("World!");
|
||||
handler.onCompleteResponse(ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Hi! World! "))
|
||||
.build());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,199 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.model.chat.StreamingChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import dev.langchain4j.model.chat.response.StreamingChatResponseHandler;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.TokenStream;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailOnStreamedResponseValidationTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void ok() throws InterruptedException {
|
||||
assertThat(execute(() -> aiService.ok("1"))).isEqualTo("Hi! World! ");
|
||||
}
|
||||
|
||||
@Test
|
||||
void ko() {
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> execute(() -> aiService.ko("2")))
|
||||
.withMessageContaining("KO");
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryOk() throws InterruptedException {
|
||||
var retry = SingletonClassInstanceFactory.getInstance(RetryingGuardrail.class);
|
||||
|
||||
assertThat(execute(() -> aiService.retry("3"))).isEqualTo("Hi! World! ");
|
||||
assertThat(retry.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void fetryFail() {
|
||||
var retryFail = SingletonClassInstanceFactory.getInstance(RetryingButFailGuardrail.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> execute(() -> aiService.retryButFail("4")))
|
||||
.withMessageContaining("maximum number of retries");
|
||||
assertThat(retryFail.spy()).isEqualTo(3);
|
||||
}
|
||||
|
||||
@Test
|
||||
void fatalException() {
|
||||
var fatal = SingletonClassInstanceFactory.getInstance(KOFatalGuardrail.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> execute(() -> aiService.fatal("5")))
|
||||
.withMessageContaining("Fatal");
|
||||
assertThat(fatal.spy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void rewritingWhileStreaming() throws InterruptedException {
|
||||
var rewriting = SingletonClassInstanceFactory.getInstance(RewritingGuardrail.class);
|
||||
assertThat(execute(() -> aiService.rewriting("1"))).isEqualTo("Hi! World! ,1");
|
||||
assertThat(rewriting.spy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
TokenStream ok(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(KOGuardrail.class)
|
||||
TokenStream ko(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(value = RetryingGuardrail.class, maxRetries = 3)
|
||||
TokenStream retry(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(value = RetryingButFailGuardrail.class, maxRetries = 3)
|
||||
TokenStream retryButFail(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(KOFatalGuardrail.class)
|
||||
TokenStream fatal(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails({RewritingGuardrail.class})
|
||||
TokenStream rewriting(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(
|
||||
MyAiService.class, builder -> builder.streamingChatModel(new MyStreamingChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RetryingGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
int v = spy.incrementAndGet();
|
||||
if (v >= 2) {
|
||||
return OutputGuardrailResult.success();
|
||||
}
|
||||
return retry("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RetryingButFailGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return retry("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOFatalGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
throw new IllegalArgumentException("Fatal");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RewritingGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
String text = responseFromLLM.text();
|
||||
return successWith(text + ",1");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyStreamingChatModel implements StreamingChatModel {
|
||||
@Override
|
||||
public void doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler) {
|
||||
handler.onPartialResponse("Hi!");
|
||||
handler.onPartialResponse(" ");
|
||||
handler.onPartialResponse("World!");
|
||||
handler.onCompleteResponse(ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Hi! World! "))
|
||||
.build());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,197 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import com.example.OutputGuardrailValidation;
|
||||
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
|
||||
import dev.langchain4j.service.AiServices;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import dev.langchain4j.service.V;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.stream.Stream;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.Arguments;
|
||||
import org.junit.jupiter.params.provider.MethodSource;
|
||||
|
||||
class OutputGuardrailPromptTemplateTests extends BaseGuardrailTests {
|
||||
@BeforeEach
|
||||
void beforeEach() {
|
||||
OutputGuardrailValidation.getInstance().reset();
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkNoParameters(String testDescription, MyAiService aiService) {
|
||||
aiService.getJoke();
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me a joke");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables()).isEmpty();
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkWithMemoryId(String testDescription, MyAiService aiService) {
|
||||
aiService.getAnotherJoke("memory-id-001");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me another joke");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of("arg0", "memory-id-001"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkWithNoMemoryIdAndOneParameter(String testDescription, MyAiService aiService) {
|
||||
aiService.sayHiToMyFriendNoMemory("Rambo");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Say hi to my friend {{it}}!");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"arg0", "Rambo",
|
||||
"it", "Rambo"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkWithMemoryIdAndOneParameter(String testDescription, MyAiService aiService) {
|
||||
aiService.sayHiToMyFriend("1", "Chuck Norris");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Say hi to my friend {{friend}}!");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"friend", "Chuck Norris",
|
||||
"arg0", "1"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkWithNoMemoryIdAndThreeParameters(String testDescription, MyAiService aiService) {
|
||||
aiService.sayHiToMyFriends("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me something about {{topic1}}, {{topic2}}, {{topic3}}!");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"topic1", "Chuck Norris",
|
||||
"topic2", "Jean-Claude Van Damme",
|
||||
"topic3", "Silvester Stallone"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkWithNoMemoryIdAndList(String testDescription, MyAiService aiService) {
|
||||
aiService.sayHiToMyFriends(List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"));
|
||||
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me something about {{it}}!");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"arg0",
|
||||
List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"),
|
||||
"it",
|
||||
"[Chuck Norris, Jean-Claude Van Damme, Silvester Stallone]"));
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("services")
|
||||
void shouldWorkWithMemoryIdAndList(String testDescription, MyAiService aiService) {
|
||||
aiService.sayHiToMyFriends(
|
||||
"memory-id-007", List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"));
|
||||
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyUserMessageTemplate())
|
||||
.isEqualTo("Tell me something about {{topics}}! This is my memory id: {{memoryId}}");
|
||||
assertThat(OutputGuardrailValidation.getInstance().spyVariables())
|
||||
.containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"topics",
|
||||
List.of("Chuck Norris", "Jean-Claude Van Damme", "Silvester Stallone"),
|
||||
"memoryId",
|
||||
"memory-id-007"));
|
||||
}
|
||||
|
||||
static Stream<Arguments> services() {
|
||||
return Stream.of(
|
||||
Arguments.of("AiService with class-level annotation", ClassLevelAiService.create()),
|
||||
Arguments.of("AiService with method-level annotations", MethodLevelAiService.create()),
|
||||
Arguments.of("AiService with builder-style guardrails", MyAiService.create()));
|
||||
}
|
||||
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
public interface ClassLevelAiService extends MyAiService {
|
||||
static MyAiService create() {
|
||||
return createAiService(ClassLevelAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public interface MethodLevelAiService extends MyAiService {
|
||||
@Override
|
||||
@UserMessage("Tell me a joke")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String getJoke();
|
||||
|
||||
@Override
|
||||
@UserMessage("Tell me another joke")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String getAnotherJoke(@MemoryId String memoryId);
|
||||
|
||||
@Override
|
||||
@UserMessage("Say hi to my friend {{it}}!")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String sayHiToMyFriendNoMemory(String friend);
|
||||
|
||||
@Override
|
||||
@UserMessage("Say hi to my friend {{friend}}!")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String sayHiToMyFriend(@MemoryId String mem, @V("friend") String friend);
|
||||
|
||||
@Override
|
||||
@UserMessage("Tell me something about {{topic1}}, {{topic2}}, {{topic3}}!")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String sayHiToMyFriends(@V("topic1") String topic1, @V("topic2") String topic2, @V("topic3") String topic3);
|
||||
|
||||
@Override
|
||||
@UserMessage("Tell me something about {{it}}!")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String sayHiToMyFriends(List<String> topics);
|
||||
|
||||
@Override
|
||||
@UserMessage("Tell me something about {{topics}}! This is my memory id: {{memoryId}}")
|
||||
@OutputGuardrails(OutputGuardrailValidation.class)
|
||||
String sayHiToMyFriends(@V("memoryId") @MemoryId String memoryId, @V("topics") List<String> topics);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MethodLevelAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Tell me a joke")
|
||||
String getJoke();
|
||||
|
||||
@UserMessage("Tell me another joke")
|
||||
String getAnotherJoke(@MemoryId String memoryId);
|
||||
|
||||
@UserMessage("Say hi to my friend {{it}}!")
|
||||
String sayHiToMyFriendNoMemory(String friend);
|
||||
|
||||
@UserMessage("Say hi to my friend {{friend}}!")
|
||||
String sayHiToMyFriend(@MemoryId String mem, @V("friend") String friend);
|
||||
|
||||
@UserMessage("Tell me something about {{topic1}}, {{topic2}}, {{topic3}}!")
|
||||
String sayHiToMyFriends(@V("topic1") String topic1, @V("topic2") String topic2, @V("topic3") String topic3);
|
||||
|
||||
@UserMessage("Tell me something about {{it}}!")
|
||||
String sayHiToMyFriends(List<String> topics);
|
||||
|
||||
@UserMessage("Tell me something about {{topics}}! This is my memory id: {{memoryId}}")
|
||||
String sayHiToMyFriends(@V("memoryId") @MemoryId String memoryId, @V("topics") List<String> topics);
|
||||
|
||||
static MyAiService create() {
|
||||
return AiServices.builder(MyAiService.class)
|
||||
.chatModel(new MyChatModel())
|
||||
.chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(10))
|
||||
.outputGuardrails(OutputGuardrailValidation.getInstance())
|
||||
.build();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,124 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.SystemMessage;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailRepromptingRetryTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void ok() {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OkGuardrail.class);
|
||||
aiService.ok("1", "foo");
|
||||
assertThat(okGuardrail.getSpy()).isEqualTo(1);
|
||||
|
||||
aiService.ok("1", "bar");
|
||||
assertThat(okGuardrail.getSpy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryFailing() {
|
||||
var retryGuardrail = SingletonClassInstanceFactory.getInstance(RetryGuardrail.class);
|
||||
assertThatThrownBy(() -> aiService.retry("1", "foo"))
|
||||
.isInstanceOf(OutputGuardrailException.class)
|
||||
.hasMessageContaining("maximum number of retries");
|
||||
assertThat(retryGuardrail.getSpy()).isEqualTo(5);
|
||||
}
|
||||
|
||||
@Test
|
||||
void noRetry() {
|
||||
var retryGuardrail = SingletonClassInstanceFactory.getInstance(RetryGuardrail.class);
|
||||
retryGuardrail.reset();
|
||||
assertThatThrownBy(() -> aiService.noRetry("2", "foo"))
|
||||
.isInstanceOf(OutputGuardrailException.class)
|
||||
.hasMessageContaining("maximum number of retries");
|
||||
assertThat(retryGuardrail.getSpy()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void repromptingFailing() {
|
||||
var repromptingGuardrail = SingletonClassInstanceFactory.getInstance(RepromptingGuardrail.class);
|
||||
assertThatThrownBy(() -> aiService.reprompting("1", "foo"))
|
||||
.isInstanceOf(OutputGuardrailException.class)
|
||||
.hasMessageContaining("maximum number of retries");
|
||||
assertThat(repromptingGuardrail.getSpy())
|
||||
.isEqualTo(dev.langchain4j.guardrail.config.OutputGuardrailsConfig.MAX_RETRIES_DEFAULT);
|
||||
}
|
||||
|
||||
@SystemMessage("Say Hi!")
|
||||
public interface MyAiService {
|
||||
@OutputGuardrails(OkGuardrail.class)
|
||||
String ok(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails(value = RetryGuardrail.class, maxRetries = 5)
|
||||
String retry(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails(value = RetryGuardrail.class, maxRetries = 0)
|
||||
String noRetry(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails(RepromptingGuardrail.class)
|
||||
String reprompting(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OkGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RetryGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
spy.incrementAndGet();
|
||||
return retry("Retry");
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public void reset() {
|
||||
this.spy.set(0);
|
||||
}
|
||||
}
|
||||
|
||||
public static class RepromptingGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
int v = spy.incrementAndGet();
|
||||
return reprompt("Retry", "reprompt");
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,264 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
import static org.assertj.core.api.Assertions.atIndex;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.data.message.ChatMessage;
|
||||
import dev.langchain4j.data.message.ChatMessageType;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailRequest;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.memory.ChatMemory;
|
||||
import dev.langchain4j.model.chat.ChatModel;
|
||||
import dev.langchain4j.model.chat.request.ChatRequest;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.SystemMessage;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailRepromptingTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void repromptingOkAfterOneRetry() {
|
||||
var repromptingOne = SingletonClassInstanceFactory.getInstance(RepromptingOne.class);
|
||||
aiService.one("1", "foo");
|
||||
assertThat(repromptingOne.getSpy()).isEqualTo(2);
|
||||
assertThat(repromptingOne.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void repromptingOkAfterTwoRetries() {
|
||||
var repromptingTwo = SingletonClassInstanceFactory.getInstance(RepromptingTwo.class);
|
||||
aiService.two("2", "foo");
|
||||
assertThat(repromptingTwo.getSpy()).isEqualTo(3);
|
||||
assertThat(repromptingTwo.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@Test
|
||||
void repromptingFailing() {
|
||||
var repromptingFailed = SingletonClassInstanceFactory.getInstance(RepromptingFailed.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class).isThrownBy(() -> aiService.fail("3", "foo"));
|
||||
assertThat(repromptingFailed.getSpy()).isEqualTo(3);
|
||||
assertThat(repromptingFailed.chatMemory())
|
||||
.isNotNull()
|
||||
.extracting(ChatMemory::messages)
|
||||
.satisfies(messages -> assertThat(messages)
|
||||
.isNotNull()
|
||||
.hasSize(2)
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.USER),
|
||||
atIndex(0))
|
||||
.satisfies(
|
||||
message -> assertThat(message)
|
||||
.isNotNull()
|
||||
.extracting(ChatMessage::type)
|
||||
.isEqualTo(ChatMessageType.AI),
|
||||
atIndex(1)));
|
||||
}
|
||||
|
||||
@SystemMessage("Say Hi!")
|
||||
public interface MyAiService {
|
||||
@OutputGuardrails(RepromptingOne.class)
|
||||
String one(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails(value = RepromptingTwo.class, maxRetries = 3)
|
||||
String two(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
@OutputGuardrails(value = RepromptingFailed.class, maxRetries = 3)
|
||||
String fail(@MemoryId String mem, @UserMessage String message);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class RepromptingOne implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
return OutputGuardrail.super.validate(params);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
if (spy.incrementAndGet() == 1) {
|
||||
return reprompt("Retry", "Retry");
|
||||
}
|
||||
|
||||
return success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RepromptingTwo implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
int v = spy.incrementAndGet();
|
||||
var messages = params.requestParams().chatMemory().messages();
|
||||
|
||||
if (v == 1) {
|
||||
ChatMessage last = messages.get(messages.size() - 1);
|
||||
assertThat(last).isInstanceOfSatisfying(AiMessage.class, am -> assertThat(am.text())
|
||||
.isEqualTo("Nope"));
|
||||
assertThat(params.responseFromLLM().aiMessage().text()).isEqualTo("Nope");
|
||||
return reprompt("Retry", "Retry");
|
||||
}
|
||||
|
||||
if (v == 2) {
|
||||
// Check that it's NOT in memory
|
||||
ChatMessage last = messages.get(messages.size() - 1);
|
||||
ChatMessage beforeLast = messages.get(messages.size() - 2);
|
||||
|
||||
assertThat(last).isInstanceOfSatisfying(AiMessage.class, am -> assertThat(am.text())
|
||||
.isEqualTo("Nope"));
|
||||
assertThat(params.responseFromLLM().aiMessage().text()).isEqualTo("Hello");
|
||||
assertThat(beforeLast)
|
||||
.isInstanceOfSatisfying(
|
||||
dev.langchain4j.data.message.UserMessage.class,
|
||||
um -> assertThat(um.singleText()).isEqualTo("foo"));
|
||||
|
||||
return reprompt("Retry", "Retry");
|
||||
}
|
||||
|
||||
if (v != 3) {
|
||||
throw new IllegalArgumentException("Unexpected call");
|
||||
}
|
||||
|
||||
return success();
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RepromptingFailed implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
private final AtomicReference<ChatMemory> chatMemory = new AtomicReference<>();
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
this.chatMemory.set(params.requestParams().chatMemory());
|
||||
int v = spy.incrementAndGet();
|
||||
var messages = params.requestParams().chatMemory().messages();
|
||||
|
||||
if (v == 1) {
|
||||
ChatMessage last = messages.get(messages.size() - 1);
|
||||
assertThat(last).isInstanceOfSatisfying(AiMessage.class, am -> assertThat(am.text())
|
||||
.isEqualTo("Nope"));
|
||||
return reprompt("Retry", "Retry Once");
|
||||
}
|
||||
|
||||
if (v == 2) {
|
||||
// Check that it's NOT in memory
|
||||
ChatMessage last = messages.get(messages.size() - 1);
|
||||
ChatMessage beforeLast = messages.get(messages.size() - 2);
|
||||
|
||||
assertThat(last).isInstanceOfSatisfying(AiMessage.class, am -> assertThat(am.text())
|
||||
.isEqualTo("Nope"));
|
||||
assertThat(beforeLast)
|
||||
.isInstanceOfSatisfying(
|
||||
dev.langchain4j.data.message.UserMessage.class,
|
||||
um -> assertThat(um.singleText()).isEqualTo("foo"));
|
||||
return reprompt("Retry", "Retry Twice");
|
||||
}
|
||||
|
||||
return reprompt("Retry", "Retry Again");
|
||||
}
|
||||
|
||||
public int getSpy() {
|
||||
return spy.get();
|
||||
}
|
||||
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class MyChatModel implements ChatModel {
|
||||
@Override
|
||||
public ChatResponse doChat(ChatRequest chatRequest) {
|
||||
var messages = chatRequest.messages();
|
||||
var last = messages.get(messages.size() - 1);
|
||||
|
||||
if (last instanceof dev.langchain4j.data.message.UserMessage um) {
|
||||
if ("foo".equals(um.singleText())) {
|
||||
return ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Nope"))
|
||||
.build();
|
||||
}
|
||||
|
||||
if (um.singleText().contains("Retry")) {
|
||||
return ChatResponse.builder()
|
||||
.aiMessage(AiMessage.from("Hello"))
|
||||
.build();
|
||||
}
|
||||
}
|
||||
|
||||
throw new IllegalArgumentException("Unexpected message: " + messages);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void outputGuardrailsAreInvoked() {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
assertThat(okGuardrail.spy()).isEqualTo(0);
|
||||
aiService.hi("1");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(1);
|
||||
aiService.hi("2");
|
||||
assertThat(okGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void guardrailCanThrowValidationException() {
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(0);
|
||||
assertThatThrownBy(() -> aiService.ko("1")).hasCauseExactlyInstanceOf(ValidationException.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(1);
|
||||
assertThatThrownBy(() -> aiService.ko("1")).hasCauseExactlyInstanceOf(ValidationException.class);
|
||||
assertThat(koGuardrail.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
String hi(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(KOGuardrail.class)
|
||||
String ko(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO", new ValidationException("KO"));
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,159 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
|
||||
|
||||
import com.example.SingletonClassInstanceFactory;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrail;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailException;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult;
|
||||
import dev.langchain4j.service.MemoryId;
|
||||
import dev.langchain4j.service.UserMessage;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class OutputGuardrailValidationTests extends BaseGuardrailTests {
|
||||
MyAiService aiService = MyAiService.create();
|
||||
|
||||
@Test
|
||||
void ok() {
|
||||
var okGuardrail = SingletonClassInstanceFactory.getInstance(OKGuardrail.class);
|
||||
aiService.ok("1");
|
||||
assertThat(okGuardrail.spy()).isOne();
|
||||
}
|
||||
|
||||
@Test
|
||||
void ko() {
|
||||
var koGuardrail = SingletonClassInstanceFactory.getInstance(KOGuardrail.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> aiService.ko("2"))
|
||||
.withMessageContaining("KO");
|
||||
|
||||
assertThat(koGuardrail.spy()).isOne();
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryOk() {
|
||||
var retry = SingletonClassInstanceFactory.getInstance(RetryingGuardrail.class);
|
||||
aiService.retry("3");
|
||||
assertThat(retry.spy()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void retryFail() {
|
||||
var retryFail = SingletonClassInstanceFactory.getInstance(RetryingButFailGuardrail.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> aiService.retryButFail("4"))
|
||||
.withMessageContaining("maximum number of retries");
|
||||
assertThat(retryFail.spy()).isEqualTo(3);
|
||||
}
|
||||
|
||||
@Test
|
||||
void fatalException() {
|
||||
var fatal = SingletonClassInstanceFactory.getInstance(KOFatalGuardrail.class);
|
||||
assertThatExceptionOfType(OutputGuardrailException.class)
|
||||
.isThrownBy(() -> aiService.fatal("5"))
|
||||
.withMessageContaining("Fatal");
|
||||
assertThat(fatal.spy()).isOne();
|
||||
}
|
||||
|
||||
public interface MyAiService {
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(OKGuardrail.class)
|
||||
String ok(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(KOGuardrail.class)
|
||||
String ko(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(RetryingGuardrail.class)
|
||||
String retry(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(value = RetryingButFailGuardrail.class, maxRetries = 3)
|
||||
String retryButFail(@MemoryId String mem);
|
||||
|
||||
@UserMessage("Say Hi!")
|
||||
@OutputGuardrails(KOFatalGuardrail.class)
|
||||
String fatal(@MemoryId String mem);
|
||||
|
||||
static MyAiService create() {
|
||||
return createAiService(MyAiService.class, builder -> builder.chatModel(new MyChatModel()));
|
||||
}
|
||||
}
|
||||
|
||||
public static class OKGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return success();
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
return failure("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RetryingGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
int v = spy.incrementAndGet();
|
||||
if (v == 2) {
|
||||
return OutputGuardrailResult.success();
|
||||
}
|
||||
return retry("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class RetryingButFailGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
int v = spy.incrementAndGet();
|
||||
return retry("KO");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
|
||||
public static class KOFatalGuardrail implements OutputGuardrail {
|
||||
private final AtomicInteger spy = new AtomicInteger(0);
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
spy.incrementAndGet();
|
||||
throw new IllegalArgumentException("Fatal");
|
||||
}
|
||||
|
||||
public int spy() {
|
||||
return spy.get();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
package dev.langchain4j.service.guardrail;
|
||||
|
||||
public class ValidationException extends RuntimeException {
|
||||
public ValidationException(String message) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-parent</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
<relativePath>../langchain4j-parent/pom.xml</relativePath>
|
||||
</parent>
|
||||
|
||||
<artifactId>langchain4j-integration-tests-parent</artifactId>
|
||||
<packaging>pom</packaging>
|
||||
<name>LangChain4j :: Integration Tests</name>
|
||||
<description>Parent POM for all integration tests</description>
|
||||
|
||||
<licenses>
|
||||
<license>
|
||||
<name>Apache-2.0</name>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
|
||||
<distribution>repo</distribution>
|
||||
<comments>A business-friendly OSS license</comments>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<modules>
|
||||
<module>integration-tests-class-instance-loader</module>
|
||||
<module>integration-tests-class-instance-loader/integration-tests-class-instance-loader-spring</module>
|
||||
<module>integration-tests-class-instance-loader/integration-tests-class-instance-loader-quarkus</module>
|
||||
<module>integration-tests-class-metadata-provider</module>
|
||||
<module>integration-tests-class-metadata-provider/integration-tests-class-metadata-provider-spring</module>
|
||||
<module>integration-tests-guardrails</module>
|
||||
</modules>
|
||||
|
||||
<properties>
|
||||
<!-- Don't want to actually install, deploy, or generate sources/javadocs for any of the artifacts under here -->
|
||||
<maven.deploy.file.skip>true</maven.deploy.file.skip>
|
||||
<maven.install.skip>true</maven.install.skip>
|
||||
<maven.javadoc.skip>true</maven.javadoc.skip>
|
||||
<maven.source.skip>true</maven.source.skip>
|
||||
<enforcer.skipRules>requireUpperBoundDeps</enforcer.skipRules>
|
||||
</properties>
|
||||
</project>
|
||||
|
|
@ -31,6 +31,12 @@
|
|||
<version>1.1.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-test</artifactId>
|
||||
<version>1.1.0-beta7-SNAPSHOT</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>dev.langchain4j</groupId>
|
||||
<artifactId>langchain4j-http-client</artifactId>
|
||||
|
|
@ -564,10 +570,10 @@
|
|||
<executions>
|
||||
<execution>
|
||||
<id>flatten</id>
|
||||
<phase>process-resources</phase>
|
||||
<goals>
|
||||
<goal>flatten</goal>
|
||||
</goals>
|
||||
<phase>process-resources</phase>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>flatten.clean</id>
|
||||
|
|
|
|||
|
|
@ -42,7 +42,6 @@
|
|||
<version>${jspecify.version}</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
<!-- test dependencies -->
|
||||
|
||||
<dependency>
|
||||
|
|
@ -91,6 +90,30 @@
|
|||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
|
||||
<!-- We want to use some of the things inside the langchain4j-test module -->
|
||||
<!-- But we can't depend on the module directly because it would introduce a circular dependency -->
|
||||
<!-- The langchain4j-test module depends on this module -->
|
||||
<!-- So instead we will just add the source of that module to the test source of this module -->
|
||||
<plugin>
|
||||
<groupId>org.codehaus.mojo</groupId>
|
||||
<artifactId>build-helper-maven-plugin</artifactId>
|
||||
<version>3.6.0</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<goals>
|
||||
<goal>add-test-source</goal>
|
||||
</goals>
|
||||
<phase>generate-test-sources</phase>
|
||||
<configuration>
|
||||
<sources>
|
||||
<source>${project.basedir}/../langchain4j-test/src/main/java</source>
|
||||
</sources>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-source-plugin</artifactId>
|
||||
|
|
@ -143,9 +166,12 @@
|
|||
<rules>
|
||||
<rule>
|
||||
<excludes>
|
||||
<exclude>dev.langchain4j.classinstance</exclude>
|
||||
<exclude>dev.langchain4j.data.document</exclude>
|
||||
<exclude>dev.langchain4j.data.text</exclude>
|
||||
<exclude>dev.langchain4j.exception</exclude>
|
||||
<exclude>dev.langchain4j.guardrail</exclude>
|
||||
<exclude>dev.langchain4j.guardrail.config</exclude>
|
||||
<exclude>dev.langchain4j.internal</exclude>
|
||||
<exclude>dev.langchain4j.model.chat</exclude>
|
||||
<exclude>dev.langchain4j.model.chat.listener</exclude>
|
||||
|
|
|
|||
|
|
@ -1,14 +1,28 @@
|
|||
package dev.langchain4j;
|
||||
|
||||
import java.lang.annotation.Target;
|
||||
|
||||
import static java.lang.annotation.ElementType.CONSTRUCTOR;
|
||||
import static java.lang.annotation.ElementType.METHOD;
|
||||
import static java.lang.annotation.ElementType.PACKAGE;
|
||||
import static java.lang.annotation.ElementType.TYPE;
|
||||
|
||||
import java.lang.annotation.Documented;
|
||||
import java.lang.annotation.Inherited;
|
||||
import java.lang.annotation.Retention;
|
||||
import java.lang.annotation.RetentionPolicy;
|
||||
import java.lang.annotation.Target;
|
||||
|
||||
/**
|
||||
* Indicates that a class/constructor/method is experimental and might change in the future.
|
||||
*/
|
||||
@Target({TYPE, CONSTRUCTOR, METHOD})
|
||||
@Inherited
|
||||
@Documented
|
||||
@Retention(RetentionPolicy.RUNTIME)
|
||||
@Target({TYPE, CONSTRUCTOR, METHOD, PACKAGE})
|
||||
public @interface Experimental {
|
||||
/**
|
||||
* Describes why the annotated element is experimental
|
||||
*
|
||||
* @return The experimental description
|
||||
*/
|
||||
String value() default "This feature is experimental and the API is subject to change";
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,45 @@
|
|||
package dev.langchain4j.classinstance;
|
||||
|
||||
import dev.langchain4j.spi.classloading.ClassInstanceFactory;
|
||||
import java.lang.reflect.InvocationTargetException;
|
||||
import java.util.ServiceLoader;
|
||||
|
||||
/**
|
||||
* Utility class for creating and retrieving instances of specified class types.
|
||||
* This class provides a mechanism to delegate instance creation to a factory, if available,
|
||||
* or fallback to direct instantiation using the no-argument constructor.
|
||||
* <p>
|
||||
* This is useful in scenarios where dependency injection frameworks or other managed
|
||||
* object factories might need to be leveraged for object creation.
|
||||
* </p>
|
||||
*/
|
||||
public final class ClassInstanceLoader {
|
||||
private ClassInstanceLoader() {}
|
||||
|
||||
/**
|
||||
* Retrieves an instance of the specified class type. This method first attempts to obtain the instance
|
||||
* through a {@link ClassInstanceFactory}, if available. If no factory is present, it creates a new
|
||||
* instance of the class using its no-argument constructor.
|
||||
*
|
||||
* @param <T> the type of the class
|
||||
* @param clazz the class object representing the type whose instance is to be created
|
||||
* @return an instance of the specified class type
|
||||
*/
|
||||
public static <T> T getClassInstance(Class<T> clazz) {
|
||||
return ServiceLoader.load(ClassInstanceFactory.class)
|
||||
.findFirst()
|
||||
.map(classInstanceFactory -> classInstanceFactory.getInstanceOfClass(clazz))
|
||||
.orElseGet(() -> createNewClassInstance(clazz));
|
||||
}
|
||||
|
||||
private static <T> T createNewClassInstance(Class<T> clazz) {
|
||||
try {
|
||||
return clazz.getDeclaredConstructor().newInstance();
|
||||
} catch (InstantiationException
|
||||
| IllegalAccessException
|
||||
| InvocationTargetException
|
||||
| NoSuchMethodException e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,262 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.Internal;
|
||||
import dev.langchain4j.guardrail.GuardrailResult.Failure;
|
||||
import dev.langchain4j.guardrail.config.GuardrailsConfig;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Abstract base class for {@link GuardrailExecutor}s.
|
||||
* @param <C>
|
||||
* The type of {@link GuardrailsConfig} to use for configuration
|
||||
* @param <P>
|
||||
* The type of {@link GuardrailRequest} to validate
|
||||
* @param <R>
|
||||
* The type of {@link GuardrailResult} to return
|
||||
* @param <G>
|
||||
* The type of {@link Guardrail}s being executed
|
||||
* @param <F>
|
||||
* The type of {@link Failure} to return
|
||||
*/
|
||||
@Internal
|
||||
public abstract sealed class AbstractGuardrailExecutor<
|
||||
C extends GuardrailsConfig,
|
||||
P extends GuardrailRequest<P>,
|
||||
R extends GuardrailResult<R>,
|
||||
G extends Guardrail<P, R>,
|
||||
F extends Failure>
|
||||
implements GuardrailExecutor<C, P, R, G> permits InputGuardrailExecutor, OutputGuardrailExecutor {
|
||||
|
||||
private final C config;
|
||||
private final List<G> guardrails;
|
||||
|
||||
protected AbstractGuardrailExecutor(C config, List<G> guardrails) {
|
||||
ensureNotNull(config, "config");
|
||||
this.config = config;
|
||||
this.guardrails = Optional.ofNullable(guardrails).orElseGet(List::of);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a failure result from some {@link Failure}s.
|
||||
* @param failures The failures
|
||||
* @return A {@link GuardrailResult} containing the failures
|
||||
*/
|
||||
protected abstract R createFailure(List<F> failures);
|
||||
|
||||
/**
|
||||
* Creates a success result.
|
||||
* @return A {@link GuardrailResult} representing success
|
||||
*/
|
||||
protected abstract R createSuccess();
|
||||
|
||||
/**
|
||||
* Creates a {@link GuardrailException} using the provided message and optional cause.
|
||||
*
|
||||
* @param message The detailed message for the exception.
|
||||
* @param cause The underlying cause of the exception, or null if no cause is available.
|
||||
* @return A new instance of {@link GuardrailException} constructed with the provided message and cause.
|
||||
*/
|
||||
protected abstract GuardrailException createGuardrailException(String message, Throwable cause);
|
||||
|
||||
@Override
|
||||
public C config() {
|
||||
return this.config;
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<G> guardrails() {
|
||||
return this.guardrails;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates a guardrail against a set of params.
|
||||
* <p>
|
||||
* If any kind of {@link Exception} is thrown during validation, it will be wrapped in a {@link GuardrailException}.
|
||||
* </p>
|
||||
* @param params The {@link GuardrailRequest} to validate
|
||||
* @param guardrail The {@link Guardrail} to evaluate against
|
||||
* @throws GuardrailException If any kind of {@link Exception} is thrown during validation
|
||||
* @return The {@link GuardrailResult} of the validation
|
||||
*/
|
||||
protected R validate(P params, G guardrail) {
|
||||
ensureNotNull(params, "params");
|
||||
ensureNotNull(guardrail, "guardrail");
|
||||
|
||||
try {
|
||||
return guardrail.validate(params).validatedBy(guardrail.getClass());
|
||||
} catch (Exception e) {
|
||||
throw createGuardrailException(e.getMessage(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handles a fatal result.
|
||||
* @param accumulatedResult The accumulated result
|
||||
* @param result The fatal result
|
||||
* @return The fatal result, possibly wrapped/modified in some way
|
||||
*/
|
||||
protected R handleFatalResult(R accumulatedResult, R result) {
|
||||
return result;
|
||||
}
|
||||
|
||||
protected R executeGuardrails(P params) {
|
||||
ensureNotNull(params, "params");
|
||||
|
||||
var accumulatedResult = createSuccess();
|
||||
var accumulatedParams = params;
|
||||
|
||||
for (var guardrail : this.guardrails) {
|
||||
if (guardrail != null) {
|
||||
var result = validate(accumulatedParams, guardrail);
|
||||
|
||||
if (result.isFatal()) {
|
||||
// Fatal result, so stop right here and don't do any more processing
|
||||
return handleFatalResult(accumulatedResult, result);
|
||||
}
|
||||
|
||||
if (result.hasRewrittenResult()) {
|
||||
accumulatedParams = accumulatedParams.withText(result.successfulText());
|
||||
}
|
||||
|
||||
accumulatedResult = composeResult(accumulatedResult, result);
|
||||
}
|
||||
}
|
||||
|
||||
return accumulatedResult;
|
||||
}
|
||||
|
||||
protected R composeResult(R oldResult, R newResult) {
|
||||
if (oldResult.isSuccess()) {
|
||||
return newResult;
|
||||
}
|
||||
|
||||
if (newResult.isSuccess()) {
|
||||
return oldResult;
|
||||
}
|
||||
|
||||
var failures = new ArrayList<F>(oldResult.failures());
|
||||
failures.addAll(newResult.failures());
|
||||
|
||||
return createFailure(failures);
|
||||
}
|
||||
|
||||
/**
|
||||
* A generic abstract builder class for creating instances of {@link GuardrailExecutor}.
|
||||
*
|
||||
* @param <C>
|
||||
* The type of {@link GuardrailsConfig} to use for configuration
|
||||
* @param <P>
|
||||
* The type of {@link GuardrailRequest} to validate
|
||||
* @param <R>
|
||||
* The type of {@link GuardrailResult} to return
|
||||
* @param <G>
|
||||
* The type of {@link Guardrail}s being executed
|
||||
*
|
||||
* This class is sealed to restrict subclassing to only specific permitted classes, such as
|
||||
* {@link InputGuardrailExecutor.InputGuardrailExecutorBuilder} and
|
||||
* {@link OutputGuardrailExecutor.OutputGuardrailExecutorBuilder}.
|
||||
*
|
||||
* It provides methods to configure and manage the guardrails and their associated configurations,
|
||||
* eventually culminating in the construction of a specific {@link GuardrailExecutor}.
|
||||
*/
|
||||
public abstract static sealed class GuardrailExecutorBuilder<
|
||||
C extends GuardrailsConfig,
|
||||
R extends GuardrailResult<R>,
|
||||
P extends GuardrailRequest<P>,
|
||||
G extends Guardrail<P, R>,
|
||||
B extends GuardrailExecutorBuilder<C, R, P, G, B>>
|
||||
permits InputGuardrailExecutor.InputGuardrailExecutorBuilder,
|
||||
OutputGuardrailExecutor.OutputGuardrailExecutorBuilder {
|
||||
|
||||
private final C defaultConfig;
|
||||
private C config;
|
||||
private List<G> guardrails = new ArrayList<>();
|
||||
|
||||
protected GuardrailExecutorBuilder(C defaultConfig) {
|
||||
this.defaultConfig = ensureNotNull(defaultConfig, "defaultConfig");
|
||||
}
|
||||
|
||||
/**
|
||||
* Constructs and returns an instance of {@link GuardrailExecutor}.
|
||||
*
|
||||
* This method finalizes the building process, using the configuration and guardrails
|
||||
* provided, to create a fully-formed {@link GuardrailExecutor} instance. The returned
|
||||
* instance enables execution of guardrails on given parameters.
|
||||
*
|
||||
* @return A fully initialized instance of {@link GuardrailExecutor}, ready to validate
|
||||
* interactions based on the configured guardrails and parameters.
|
||||
*/
|
||||
public abstract GuardrailExecutor<C, P, R, G> build();
|
||||
|
||||
/**
|
||||
* Retrieves the current configuration instance used by this builder.
|
||||
*
|
||||
* @return The configuration set in the builder.
|
||||
*/
|
||||
protected C config() {
|
||||
return (this.config != null) ? this.config : this.defaultConfig;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieves the list of guardrails configured in the builder.
|
||||
* Guardrails are validation rules applied to interactions with the model, ensuring that inputs or outputs
|
||||
* meet required conditions for safety and correctness.
|
||||
*
|
||||
* @return A list containing the configured guardrails.
|
||||
*/
|
||||
protected List<G> guardrails() {
|
||||
return this.guardrails;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the configuration for the guardrail executor builder.
|
||||
*
|
||||
* @param config The configuration instance to be set, which implements {@link GuardrailsConfig}.
|
||||
* This can be null if no specific configuration is required.
|
||||
* @return The updated instance of the builder, allowing for method chaining.
|
||||
*/
|
||||
public B config(C config) {
|
||||
this.config = config;
|
||||
return (B) this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Updates the list of guardrails for the builder. The provided guardrails will replace
|
||||
* the current list of guardrails in the builder. If the provided list is null, all
|
||||
* existing guardrails will be cleared.
|
||||
*
|
||||
* @param guardrails A list of guardrails to be set for the builder. It can be null,
|
||||
* in which case the current list of guardrails will be cleared.
|
||||
* @return The updated instance of the builder, allowing for method chaining.
|
||||
*/
|
||||
public B guardrails(List<G> guardrails) {
|
||||
this.guardrails.clear();
|
||||
|
||||
if (guardrails != null) {
|
||||
this.guardrails.addAll(guardrails);
|
||||
}
|
||||
|
||||
return (B) this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Updates the builder with the specified guardrails. This method accepts
|
||||
* a variadic array of guardrails, which will be used to replace the current
|
||||
* set of guardrails in the builder. If the input is null, the existing
|
||||
* guardrails will remain unchanged.
|
||||
*
|
||||
* @param guardrails An optional array of guardrails to be set for the builder.
|
||||
* Null values are accepted and will not clear existing guardrails.
|
||||
* @return The updated instance of the builder, allowing for method chaining.
|
||||
*/
|
||||
public B guardrails(G... guardrails) {
|
||||
Optional.ofNullable(guardrails).map(List::of).ifPresent(this::guardrails);
|
||||
|
||||
return (B) this;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
/**
|
||||
* A guardrail is a rule that is applied when interacting with an LLM either to the input (the user message) or to the
|
||||
* output of the model to ensure that they are safe and meet the expectations of the model.
|
||||
*
|
||||
* @param <P>
|
||||
* The type of the {@link GuardrailRequest}
|
||||
* @param <R>
|
||||
* The type of the {@link GuardrailResult}
|
||||
*/
|
||||
public interface Guardrail<P extends GuardrailRequest, R extends GuardrailResult<R>> {
|
||||
/**
|
||||
* Validate the interaction between the model and the user in one of the two directions.
|
||||
*
|
||||
* @param params
|
||||
* The parameters of the request or the response to be validated
|
||||
*
|
||||
* @return The result of the validation
|
||||
*/
|
||||
R validate(P params);
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import dev.langchain4j.exception.LangChain4jException;
|
||||
|
||||
/**
|
||||
* Exception thrown when an input or output guardrail validation fails.
|
||||
* <p>
|
||||
* This class is not intended to be used within guardrail implementations. It is for the framework only.
|
||||
* </p>
|
||||
* @see InputGuardrailException
|
||||
* @see OutputGuardrailException
|
||||
*/
|
||||
public sealed class GuardrailException extends LangChain4jException
|
||||
permits InputGuardrailException, OutputGuardrailException {
|
||||
protected GuardrailException(String message) {
|
||||
super(message);
|
||||
}
|
||||
|
||||
protected GuardrailException(String message, Throwable cause) {
|
||||
super(message, cause);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import dev.langchain4j.guardrail.config.GuardrailsConfig;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Represents a mechanism to execute a set of guardrails on given parameters.
|
||||
* This interface defines the contract for validating interactions (input or output)
|
||||
* using multiple guardrails.
|
||||
*
|
||||
* @param <C>
|
||||
* The type of {@link GuardrailsConfig} to use for configuration
|
||||
* @param <P>
|
||||
* The type of {@link GuardrailRequest} to validate
|
||||
* @param <R>
|
||||
* The type of {@link GuardrailResult} to return
|
||||
* @param <G>
|
||||
* The type of {@link Guardrail}s being executed
|
||||
*/
|
||||
public sealed interface GuardrailExecutor<
|
||||
C extends GuardrailsConfig,
|
||||
P extends GuardrailRequest,
|
||||
R extends GuardrailResult<R>,
|
||||
G extends Guardrail<P, R>>
|
||||
permits AbstractGuardrailExecutor {
|
||||
|
||||
/**
|
||||
* The {@link GuardrailsConfig} to use for configuration of the guardrail execution
|
||||
* @return The {@link GuardrailsConfig} to use for configuration of the guardrail execution
|
||||
*/
|
||||
C config();
|
||||
|
||||
/**
|
||||
* Retrieves the guardrails associated with the implementation.
|
||||
* @return The guardrails which can be used for validating inputs or outputs against predefined rules.
|
||||
*/
|
||||
List<G> guardrails();
|
||||
|
||||
/**
|
||||
* Executes the provided guardrails on the given parameters.
|
||||
* @param params The {@link GuardrailRequest} to validate
|
||||
* @return The {@link GuardrailResult} of the validation
|
||||
*/
|
||||
R execute(P params);
|
||||
}
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
/**
|
||||
* Represents the parameter passed to {@link Guardrail#validate(GuardrailRequest)}} in order to validate an interaction
|
||||
* between a user and the LLM.
|
||||
*/
|
||||
public sealed interface GuardrailRequest<P extends GuardrailRequest<P>>
|
||||
permits InputGuardrailRequest, OutputGuardrailRequest {
|
||||
|
||||
/**
|
||||
* Retrieves the common parameters that are shared across guardrail checks.
|
||||
*
|
||||
* @return an instance of {@code GuardrailRequestParams} containing shared parameters such as chat memory,
|
||||
* user message template, and additional variables.
|
||||
*/
|
||||
GuardrailRequestParams requestParams();
|
||||
|
||||
/**
|
||||
* Recreate this guardrail param with the given input or output text.
|
||||
*
|
||||
* @param text
|
||||
* The text of the rewritten param.
|
||||
*
|
||||
* @return A clone of this guardrail params with the given input or output text.
|
||||
*/
|
||||
P withText(String text);
|
||||
}
|
||||
|
|
@ -0,0 +1,134 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.memory.ChatMemory;
|
||||
import dev.langchain4j.rag.AugmentationResult;
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
* Represents the common parameters shared across guardrail checks when validating interactions
|
||||
* between a user and a language model. This class encapsulates the chat memory, user message
|
||||
* template, and additional variables required for guardrail processing.
|
||||
*/
|
||||
public final class GuardrailRequestParams {
|
||||
private final ChatMemory chatMemory;
|
||||
private final AugmentationResult augmentationResult;
|
||||
private final String userMessageTemplate;
|
||||
private final Map<String, Object> variables;
|
||||
|
||||
private GuardrailRequestParams(Builder builder) {
|
||||
this.chatMemory = builder.chatMemory;
|
||||
this.augmentationResult = builder.augmentationResult;
|
||||
this.userMessageTemplate = ensureNotNull(builder.userMessageTemplate, "userMessageTemplate");
|
||||
this.variables = ensureNotNull(builder.variables, "variables");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the chat memory.
|
||||
*
|
||||
* @return the chat memory, may be null
|
||||
*/
|
||||
public ChatMemory chatMemory() {
|
||||
return chatMemory;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the augmentation result.
|
||||
*
|
||||
* @return the augmentation result, may be null
|
||||
*/
|
||||
public AugmentationResult augmentationResult() {
|
||||
return augmentationResult;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the user message template.
|
||||
*
|
||||
* @return the user message template, never null
|
||||
*/
|
||||
public String userMessageTemplate() {
|
||||
return userMessageTemplate;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the variables.
|
||||
*
|
||||
* @return the variables, never null
|
||||
*/
|
||||
public Map<String, Object> variables() {
|
||||
return variables;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new builder for {@link GuardrailRequestParams}.
|
||||
*
|
||||
* @return a new builder
|
||||
*/
|
||||
public static Builder builder() {
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder for {@link GuardrailRequestParams}.
|
||||
*/
|
||||
public static class Builder {
|
||||
private ChatMemory chatMemory;
|
||||
private AugmentationResult augmentationResult;
|
||||
private String userMessageTemplate;
|
||||
private Map<String, Object> variables;
|
||||
|
||||
/**
|
||||
* Sets the chat memory.
|
||||
*
|
||||
* @param chatMemory the chat memory
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder chatMemory(ChatMemory chatMemory) {
|
||||
this.chatMemory = chatMemory;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the augmentation result.
|
||||
*
|
||||
* @param augmentationResult the augmentation result
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder augmentationResult(AugmentationResult augmentationResult) {
|
||||
this.augmentationResult = augmentationResult;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the user message template.
|
||||
*
|
||||
* @param userMessageTemplate the user message template
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder userMessageTemplate(String userMessageTemplate) {
|
||||
this.userMessageTemplate = userMessageTemplate;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the variables.
|
||||
*
|
||||
* @param variables the variables
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder variables(Map<String, Object> variables) {
|
||||
this.variables = variables;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a new {@link GuardrailRequestParams}.
|
||||
*
|
||||
* @return a new {@link GuardrailRequestParams}
|
||||
*/
|
||||
public GuardrailRequestParams build() {
|
||||
return new GuardrailRequestParams(this);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,155 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* The result of the validation of an interaction between a user and the LLM.
|
||||
*
|
||||
* @param <GR>
|
||||
* The type of guardrail result to expect
|
||||
*
|
||||
* @see InputGuardrailResult
|
||||
* @see OutputGuardrailResult
|
||||
*/
|
||||
public sealed interface GuardrailResult<GR extends GuardrailResult<GR>>
|
||||
permits InputGuardrailResult, OutputGuardrailResult {
|
||||
/**
|
||||
* The possible results of a guardrails validation.
|
||||
*/
|
||||
enum Result {
|
||||
/**
|
||||
* A successful validation.
|
||||
*/
|
||||
SUCCESS,
|
||||
/**
|
||||
* A successful validation with a specific result.
|
||||
*/
|
||||
SUCCESS_WITH_RESULT,
|
||||
/**
|
||||
* A failed validation not preventing the subsequent validations eventually registered to be evaluated.
|
||||
*/
|
||||
FAILURE,
|
||||
/**
|
||||
* A fatal failed validation, blocking the evaluation of any other validations eventually registered.
|
||||
*/
|
||||
FATAL
|
||||
}
|
||||
|
||||
/**
|
||||
* The message and the cause of the failure of a single validation.
|
||||
*/
|
||||
sealed interface Failure permits InputGuardrailResult.Failure, OutputGuardrailResult.Failure {
|
||||
/**
|
||||
* Build a failure from a specific {@link Guardrail} class
|
||||
*/
|
||||
Failure withGuardrailClass(Class<? extends Guardrail> guardrailClass);
|
||||
|
||||
/**
|
||||
* The failure message
|
||||
*/
|
||||
String message();
|
||||
|
||||
/**
|
||||
* The cause of the failure
|
||||
*/
|
||||
Throwable cause();
|
||||
|
||||
/**
|
||||
* The {@link Guardrail} class
|
||||
*/
|
||||
Class<? extends Guardrail> guardrailClass();
|
||||
|
||||
/**
|
||||
* The string representation of the failure
|
||||
* @return A string representation of the failure
|
||||
*/
|
||||
default String asString() {
|
||||
var guardrailName =
|
||||
Optional.ofNullable(guardrailClass()).map(Class::getName).orElse("");
|
||||
|
||||
return "The guardrail %s failed with this message: %s".formatted(guardrailName, message());
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The result of the guardrail
|
||||
*/
|
||||
Result result();
|
||||
|
||||
/**
|
||||
* @return The list of failures eventually resulting from a set of validations.
|
||||
*/
|
||||
<F extends Failure> List<F> failures();
|
||||
|
||||
/**
|
||||
* The message of the successful result
|
||||
*/
|
||||
String successfulText();
|
||||
|
||||
/**
|
||||
* Whether or not the result is successful, but the result was re-written, potentially due to re-prompting
|
||||
*/
|
||||
default boolean hasRewrittenResult() {
|
||||
return result() == Result.SUCCESS_WITH_RESULT;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether or not the result is considered fatal
|
||||
*/
|
||||
default boolean isFatal() {
|
||||
return result() == Result.FATAL;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether or not the result is considered successful
|
||||
*/
|
||||
default boolean isSuccess() {
|
||||
var result = result();
|
||||
return (result == Result.SUCCESS) || (result == Result.SUCCESS_WITH_RESULT);
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the exception from the first failure
|
||||
*/
|
||||
default Throwable getFirstFailureException() {
|
||||
return !isSuccess()
|
||||
? failures().stream()
|
||||
.map(Failure::cause)
|
||||
.filter(Objects::nonNull)
|
||||
.findFirst()
|
||||
.orElse(null)
|
||||
: null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The {@link Guardrail} class which performed this validation
|
||||
*/
|
||||
default GR validatedBy(Class<? extends Guardrail> guardrailClass) {
|
||||
ensureNotNull(guardrailClass, "guardrailClass");
|
||||
|
||||
if (!isSuccess()) {
|
||||
var failures = failures();
|
||||
|
||||
if (failures.size() != 1) {
|
||||
throw new IllegalArgumentException();
|
||||
}
|
||||
|
||||
failures.set(0, failures.get(0).withGuardrailClass(guardrailClass));
|
||||
}
|
||||
|
||||
return (GR) this;
|
||||
}
|
||||
|
||||
default String asString() {
|
||||
if (isSuccess()) {
|
||||
return hasRewrittenResult() ? "Success with '%s'".formatted(successfulText()) : "Success";
|
||||
}
|
||||
|
||||
return failures().stream().map(Failure::toString).collect(Collectors.joining(", "));
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,120 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.data.message.UserMessage;
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult.Failure;
|
||||
|
||||
/**
|
||||
* An input guardrail is a rule that is applied to the input of the model to ensure that the input (i.e. the user
|
||||
* message and parameters) is safe and meets the expectations of the model.
|
||||
* <p>
|
||||
* Input guardrails are either successful or failed. A successful guardrail means that the input is valid and can be sent to
|
||||
* the model. A failed guardrail means that the input is invalid and cannot be sent to the model.
|
||||
* </p>
|
||||
* <p>
|
||||
* A failed guardrail will stop further processing of any other input guardrails.
|
||||
* </p>
|
||||
*/
|
||||
public interface InputGuardrail extends Guardrail<InputGuardrailRequest, InputGuardrailResult> {
|
||||
/**
|
||||
* Validates the {@code user message} that will be sent to the LLM.
|
||||
* <p>
|
||||
*
|
||||
* @param userMessage
|
||||
* the response from the LLM
|
||||
*/
|
||||
default InputGuardrailResult validate(UserMessage userMessage) {
|
||||
return failure("Validation not implemented");
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates the input that will be sent to the LLM.
|
||||
* <p>
|
||||
* Unlike {@link #validate(UserMessage)}, this method allows to access the memory and the augmentation result (in
|
||||
* the case of a RAG).
|
||||
* <p>
|
||||
* Implementation must not attempt to write to the memory or the augmentation result.
|
||||
*
|
||||
* @param params
|
||||
* the parameters, including the user message, the memory, and the augmentation result.
|
||||
*/
|
||||
@Override
|
||||
default InputGuardrailResult validate(InputGuardrailRequest params) {
|
||||
ensureNotNull(params, "params");
|
||||
return validate(params.userMessage());
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a successful result without any successful text
|
||||
*
|
||||
* @return The result of a successful input guardrail validation.
|
||||
*/
|
||||
default InputGuardrailResult success() {
|
||||
return InputGuardrailResult.success();
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a successful result with specific success text
|
||||
*
|
||||
* @return The result of a successful input guardrail validation with a specific text.
|
||||
*
|
||||
* @param successfulText
|
||||
* The text of the successful result.
|
||||
*/
|
||||
default InputGuardrailResult successWith(String successfulText) {
|
||||
return InputGuardrailResult.successWith(successfulText);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
*
|
||||
* @return The result of a failed input guardrail validation.
|
||||
*/
|
||||
default InputGuardrailResult failure(String message) {
|
||||
return new InputGuardrailResult(new Failure(message), false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param cause
|
||||
* The exception that caused this failure.
|
||||
*
|
||||
* @return The result of a failed input guardrail validation.
|
||||
*/
|
||||
default InputGuardrailResult failure(String message, Throwable cause) {
|
||||
return new InputGuardrailResult(new Failure(message, cause), false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
*
|
||||
* @return The result of a failed input guardrail validation.
|
||||
*/
|
||||
default InputGuardrailResult fatal(String message) {
|
||||
return new InputGuardrailResult(new Failure(message), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param cause
|
||||
* The exception that caused this failure.
|
||||
*
|
||||
* @return The result of a failed input guardrail validation.
|
||||
*/
|
||||
default InputGuardrailResult fatal(String message, Throwable cause) {
|
||||
return new InputGuardrailResult(new Failure(message, cause), true);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
/**
|
||||
* Exception thrown when an input guardrail validation fails.
|
||||
* <p>
|
||||
* This class is not intended to be thrown within guardrail implementations. It is for the framework only. It is ok to catch it.
|
||||
* </p>
|
||||
*/
|
||||
public final class InputGuardrailException extends GuardrailException {
|
||||
public InputGuardrailException(String message) {
|
||||
super(message);
|
||||
}
|
||||
|
||||
public InputGuardrailException(String message, Throwable cause) {
|
||||
super(message, cause);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,101 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import dev.langchain4j.guardrail.InputGuardrailResult.Failure;
|
||||
import dev.langchain4j.guardrail.config.InputGuardrailsConfig;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* The {@link GuardrailExecutor} for {@link InputGuardrail}s.
|
||||
*/
|
||||
public non-sealed class InputGuardrailExecutor
|
||||
extends AbstractGuardrailExecutor<
|
||||
InputGuardrailsConfig, InputGuardrailRequest, InputGuardrailResult, InputGuardrail, Failure> {
|
||||
|
||||
protected InputGuardrailExecutor(InputGuardrailsConfig config, List<InputGuardrail> guardrails) {
|
||||
super(config, guardrails);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a failure result from some {@link Failure}s.
|
||||
* @param failures The failures
|
||||
* @return A {@link InputGuardrailResult} containing the failures
|
||||
*/
|
||||
@Override
|
||||
protected InputGuardrailResult createFailure(List<Failure> failures) {
|
||||
return new InputGuardrailResult(failures, false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a success result.
|
||||
* @return A {@link InputGuardrailResult} representing success
|
||||
*/
|
||||
@Override
|
||||
protected InputGuardrailResult createSuccess() {
|
||||
return InputGuardrailResult.success();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected InputGuardrailException createGuardrailException(String message, Throwable cause) {
|
||||
return new InputGuardrailException(message, cause);
|
||||
}
|
||||
|
||||
/**
|
||||
* Execeutes the {@link InputGuardrail}s on the given {@link InputGuardrailRequest}.
|
||||
*
|
||||
* @param params The {@link InputGuardrailRequest} to validate
|
||||
* @return The {@link InputGuardrailResult} of the validation
|
||||
*/
|
||||
@Override
|
||||
public InputGuardrailResult execute(InputGuardrailRequest params) {
|
||||
var result = executeGuardrails(params);
|
||||
|
||||
if (!result.isSuccess()) {
|
||||
throw new InputGuardrailException(result.toString(), result.getFirstFailureException());
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates and returns a new builder for {@link InputGuardrailExecutor}.
|
||||
*
|
||||
* This builder allows for constructing and configuring an {@link InputGuardrailExecutor}
|
||||
* instance, enabling customization of parameters such as the configuration and input guardrails.
|
||||
*
|
||||
* @return An {@link InputGuardrailExecutorBuilder} used to create {@link InputGuardrailExecutor} instances
|
||||
*/
|
||||
public static InputGuardrailExecutorBuilder builder() {
|
||||
return new InputGuardrailExecutorBuilder();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder class for constructing instances of {@link InputGuardrailExecutor}.
|
||||
*
|
||||
* This builder allows configuration of an {@link InputGuardrailExecutor} by specifying the associated configuration
|
||||
* type ({@link InputGuardrailsConfig}) and the input guardrails to be executed.
|
||||
*
|
||||
* Extends {@link GuardrailExecutorBuilder} for the specific types:
|
||||
* - Configuration type: {@link InputGuardrailsConfig}
|
||||
* - Result type: {@link InputGuardrailResult}
|
||||
* - Parameter type: {@link InputGuardrailRequest}
|
||||
* - Guardrail type: {@link InputGuardrail}
|
||||
*
|
||||
* Provides the {@code build()} method to create an {@link InputGuardrailExecutor} instance.
|
||||
*/
|
||||
public static non-sealed class InputGuardrailExecutorBuilder
|
||||
extends GuardrailExecutorBuilder<
|
||||
InputGuardrailsConfig,
|
||||
InputGuardrailResult,
|
||||
InputGuardrailRequest,
|
||||
InputGuardrail,
|
||||
InputGuardrailExecutorBuilder> {
|
||||
public InputGuardrailExecutorBuilder() {
|
||||
super(InputGuardrailsConfig.builder().build());
|
||||
}
|
||||
|
||||
@Override
|
||||
public InputGuardrailExecutor build() {
|
||||
return new InputGuardrailExecutor(config(), guardrails());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.data.message.ContentType;
|
||||
import dev.langchain4j.data.message.TextContent;
|
||||
import dev.langchain4j.data.message.UserMessage;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Represents the parameter passed to {@link InputGuardrail#validate(InputGuardrailRequest)}.
|
||||
*/
|
||||
public final class InputGuardrailRequest implements GuardrailRequest<InputGuardrailRequest> {
|
||||
private final UserMessage userMessage;
|
||||
private final GuardrailRequestParams commonParams;
|
||||
|
||||
private InputGuardrailRequest(Builder builder) {
|
||||
this.userMessage = ensureNotNull(builder.userMessage, "userMessage");
|
||||
this.commonParams = ensureNotNull(builder.commonParams, "requestParams");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the user message.
|
||||
*
|
||||
* @return the user message
|
||||
*/
|
||||
public UserMessage userMessage() {
|
||||
return userMessage;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the common parameters shared between types of guardrails.
|
||||
*
|
||||
* @return the common parameters
|
||||
*/
|
||||
@Override
|
||||
public GuardrailRequestParams requestParams() {
|
||||
return commonParams;
|
||||
}
|
||||
|
||||
@Override
|
||||
public InputGuardrailRequest withText(String text) {
|
||||
return new Builder()
|
||||
.userMessage(rewriteUserMessage(text))
|
||||
.commonParams(this.commonParams)
|
||||
.build();
|
||||
}
|
||||
|
||||
public UserMessage rewriteUserMessage(String text) {
|
||||
if (Objects.isNull(this.userMessage) || Objects.isNull(text)) {
|
||||
return this.userMessage;
|
||||
}
|
||||
|
||||
var rewrittenContent = this.userMessage.contents().stream()
|
||||
.map(c -> (c.type() == ContentType.TEXT) ? new TextContent(text) : c)
|
||||
.toList();
|
||||
|
||||
return Objects.nonNull(this.userMessage.name())
|
||||
? UserMessage.from(this.userMessage.name(), rewrittenContent)
|
||||
: UserMessage.from(rewrittenContent);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new builder for {@link InputGuardrailRequest}.
|
||||
*
|
||||
* @return a new builder
|
||||
*/
|
||||
public static Builder builder() {
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder for {@link InputGuardrailRequest}.
|
||||
*/
|
||||
public static class Builder {
|
||||
private UserMessage userMessage;
|
||||
private GuardrailRequestParams commonParams;
|
||||
|
||||
/**
|
||||
* Sets the user message.
|
||||
*
|
||||
* @param userMessage the user message
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder userMessage(UserMessage userMessage) {
|
||||
this.userMessage = userMessage;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the common parameters.
|
||||
*
|
||||
* @param commonParams the common parameters
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder commonParams(GuardrailRequestParams commonParams) {
|
||||
this.commonParams = commonParams;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a new {@link InputGuardrailRequest}.
|
||||
*
|
||||
* @return a new {@link InputGuardrailRequest}
|
||||
*/
|
||||
public InputGuardrailRequest build() {
|
||||
return new InputGuardrailRequest(this);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,164 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.data.message.UserMessage;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* The result of the validation of an {@link InputGuardrail}
|
||||
*/
|
||||
public final class InputGuardrailResult implements GuardrailResult<InputGuardrailResult> {
|
||||
private static final InputGuardrailResult SUCCESS = new InputGuardrailResult();
|
||||
|
||||
private final Result result;
|
||||
private final String successfulText;
|
||||
private final List<Failure> failures;
|
||||
|
||||
private InputGuardrailResult(Result result, String successfulText, List<Failure> failures) {
|
||||
this.result = ensureNotNull(result, "result");
|
||||
this.successfulText = successfulText;
|
||||
this.failures = Optional.ofNullable(failures).orElseGet(List::of);
|
||||
}
|
||||
|
||||
private InputGuardrailResult() {
|
||||
this(Result.SUCCESS, null, Collections.emptyList());
|
||||
}
|
||||
|
||||
InputGuardrailResult(List<Failure> failures, boolean fatal) {
|
||||
this(fatal ? Result.FATAL : Result.FAILURE, null, failures);
|
||||
}
|
||||
|
||||
InputGuardrailResult(Failure failure, boolean fatal) {
|
||||
this(new ArrayList<>(List.of(failure)), fatal);
|
||||
}
|
||||
|
||||
private InputGuardrailResult(String successfulText) {
|
||||
this(Result.SUCCESS_WITH_RESULT, successfulText, Collections.emptyList());
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets a successful input guardrail result
|
||||
*/
|
||||
public static InputGuardrailResult success() {
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a successful result with specific success text
|
||||
*
|
||||
* @return The result of a successful input guardrail validation with a specific text.
|
||||
*
|
||||
* @param successfulText
|
||||
* The text of the successful result.
|
||||
*/
|
||||
public static InputGuardrailResult successWith(String successfulText) {
|
||||
return (successfulText == null) ? success() : new InputGuardrailResult(successfulText);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Result result() {
|
||||
return result;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String successfulText() {
|
||||
return successfulText;
|
||||
}
|
||||
|
||||
@Override
|
||||
@SuppressWarnings("unchecked")
|
||||
public <F extends GuardrailResult.Failure> List<F> failures() {
|
||||
return (List<F>) failures;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return asString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the {@link UserMessage} computed from the combination of the original {@link UserMessage} in the {@link InputGuardrailRequest}
|
||||
* and this result
|
||||
* @param params The input guardrail params
|
||||
* @return A {@link UserMessage} computed from the combination of the original {@link UserMessage} in the {@link InputGuardrailRequest}
|
||||
* * and this result
|
||||
*/
|
||||
public UserMessage userMessage(InputGuardrailRequest params) {
|
||||
return hasRewrittenResult() ? params.rewriteUserMessage(successfulText()) : params.userMessage();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean equals(Object o) {
|
||||
if (this == o) return true;
|
||||
if (o == null || getClass() != o.getClass()) return false;
|
||||
InputGuardrailResult that = (InputGuardrailResult) o;
|
||||
return result == that.result
|
||||
&& Objects.equals(successfulText, that.successfulText)
|
||||
&& Objects.equals(failures, that.failures);
|
||||
}
|
||||
|
||||
@Override
|
||||
public int hashCode() {
|
||||
return Objects.hash(result, successfulText, failures);
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents an input guardrail failure
|
||||
*/
|
||||
public static final class Failure implements GuardrailResult.Failure {
|
||||
private final String message;
|
||||
private final Throwable cause;
|
||||
private final Class<? extends Guardrail> guardrailClass;
|
||||
|
||||
Failure(String message, Throwable cause, Class<? extends Guardrail> guardrailClass) {
|
||||
this.message = ensureNotNull(message, "message");
|
||||
this.cause = cause;
|
||||
this.guardrailClass = guardrailClass;
|
||||
}
|
||||
|
||||
Failure(String message) {
|
||||
this(message, null, null);
|
||||
}
|
||||
|
||||
Failure(String message, Throwable cause) {
|
||||
this(message, cause, null);
|
||||
}
|
||||
|
||||
/**
|
||||
* Adds a guardrail class name to a failure
|
||||
*
|
||||
* @param guardrailClass
|
||||
* The guardrail class
|
||||
*/
|
||||
@Override
|
||||
public Failure withGuardrailClass(Class<? extends Guardrail> guardrailClass) {
|
||||
ensureNotNull(guardrailClass, "guardrailClass");
|
||||
return new Failure(this.message, this.cause, guardrailClass);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String message() {
|
||||
return message;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Throwable cause() {
|
||||
return cause;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Class<? extends Guardrail> guardrailClass() {
|
||||
return guardrailClass;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return asString();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,139 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import java.util.Optional;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
/**
|
||||
* An {@link OutputGuardrail} that will check whether or not a response can be successfully deserialized to an object
|
||||
* of type {@code T} from JSON
|
||||
* <p>
|
||||
* If deserialization fails, the LLM will be reprompted with {@link #getInvalidJsonReprompt(AiMessage, String)}, which
|
||||
* defaults to {@link #DEFAULT_REPROMPT_PROMPT}.
|
||||
* </p>
|
||||
*
|
||||
* @param <T> The type of object that the class should deserialize from JSON
|
||||
*/
|
||||
public class JsonExtractorOutputGuardrail<T> implements OutputGuardrail {
|
||||
/**
|
||||
* The default message to use when reprompting
|
||||
*/
|
||||
public static final String DEFAULT_REPROMPT_MESSAGE = "Invalid JSON";
|
||||
|
||||
/**
|
||||
* The default prompt to append to the LLM during a reprompt
|
||||
*/
|
||||
public static final String DEFAULT_REPROMPT_PROMPT =
|
||||
"Make sure you return a valid JSON object following the specified format";
|
||||
|
||||
private static final Logger LOGGER = LoggerFactory.getLogger(JsonExtractorOutputGuardrail.class);
|
||||
private final ObjectMapper objectMapper;
|
||||
private Class<T> outputClass;
|
||||
private TypeReference<T> outputType;
|
||||
|
||||
public JsonExtractorOutputGuardrail(ObjectMapper objectMapper, Class<T> outputClass) {
|
||||
this.objectMapper = ensureNotNull(objectMapper, "objectMapper");
|
||||
this.outputClass = ensureNotNull(outputClass, "outputClass");
|
||||
}
|
||||
|
||||
public JsonExtractorOutputGuardrail(ObjectMapper objectMapper, TypeReference<T> outputType) {
|
||||
this.objectMapper = ensureNotNull(objectMapper, "objectMapper");
|
||||
this.outputType = ensureNotNull(outputType, "outputType");
|
||||
}
|
||||
|
||||
public JsonExtractorOutputGuardrail(Class<T> outputClass) {
|
||||
this(new ObjectMapper(), outputClass);
|
||||
}
|
||||
|
||||
public JsonExtractorOutputGuardrail(TypeReference<T> outputType) {
|
||||
this(new ObjectMapper(), outputType);
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
var llmResponse = ensureNotNull(responseFromLLM, "responseFromLLM").text();
|
||||
LOGGER.debug("LLM output: {}", llmResponse);
|
||||
|
||||
return deserialize(llmResponse).map(r -> successWith(llmResponse, r)).orElseGet(() -> {
|
||||
LOGGER.debug("LLM output contained invalid JSON. Attempting to trim non-JSON");
|
||||
var json = trimNonJson(llmResponse);
|
||||
|
||||
LOGGER.debug("Attempting to deserialize trimmed JSON: {}", json);
|
||||
return deserialize(json)
|
||||
.map(r -> successWith(json, r))
|
||||
.orElseGet(() -> invokeInvalidJson(responseFromLLM, json));
|
||||
});
|
||||
}
|
||||
|
||||
protected String trimNonJson(String llmResponse) {
|
||||
var jsonMapStart = llmResponse.indexOf('{');
|
||||
var jsonListStart = llmResponse.indexOf('[');
|
||||
|
||||
if ((jsonMapStart < 0) && (jsonListStart < 0)) {
|
||||
return "";
|
||||
}
|
||||
|
||||
var isJsonMap = (jsonMapStart >= 0) && ((jsonMapStart < jsonListStart) || (jsonListStart < 0));
|
||||
var jsonStart = isJsonMap ? jsonMapStart : jsonListStart;
|
||||
var jsonEnd = isJsonMap ? llmResponse.lastIndexOf('}') : llmResponse.lastIndexOf(']');
|
||||
|
||||
return (jsonEnd >= 0) && (jsonStart < jsonEnd) ? llmResponse.substring(jsonStart, jsonEnd + 1) : "";
|
||||
}
|
||||
|
||||
protected OutputGuardrailResult invokeInvalidJson(AiMessage aiMessage, String json) {
|
||||
LOGGER.debug("Found invalid JSON for aiMessage = {} and json = {}", aiMessage, json);
|
||||
return reprompt(getInvalidJsonMessage(aiMessage, json), getInvalidJsonReprompt(aiMessage, json));
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates a message indicating that the provided JSON is invalid.
|
||||
*
|
||||
* @param aiMessage the AI message associated with the invalid JSON. This parameter is not used.
|
||||
* @param json the JSON that failed validation. This parameter is not used.
|
||||
* @return a default message indicating that the JSON is invalid.
|
||||
*/
|
||||
protected String getInvalidJsonMessage(
|
||||
@SuppressWarnings("unused") AiMessage aiMessage, @SuppressWarnings("unused") String json) {
|
||||
return DEFAULT_REPROMPT_MESSAGE;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates a reprompt message indicating that the provided JSON is invalid.
|
||||
* <p>
|
||||
* This message is appended to the user message from the previous request.
|
||||
* </p>
|
||||
*
|
||||
* @param aiMessage the AI message associated with the invalid JSON. This parameter is not used.
|
||||
* @param json the JSON input that failed validation. This parameter is not used.
|
||||
* @return a reprompt message indicating that the JSON is invalid.
|
||||
*/
|
||||
protected String getInvalidJsonReprompt(
|
||||
@SuppressWarnings("unused") AiMessage aiMessage, @SuppressWarnings("unused") String json) {
|
||||
return DEFAULT_REPROMPT_PROMPT;
|
||||
}
|
||||
|
||||
/**
|
||||
* Tries to deserialize the provided LLM response string into an object of type T using the configured {@link ObjectMapper}.
|
||||
* If deserialization fails, an empty Optional is returned.
|
||||
*
|
||||
* @param llmResponse the JSON-formatted response string to be deserialized
|
||||
* @return an Optional containing the deserialized object if successful, or an empty Optional if deserialization fails
|
||||
*/
|
||||
protected Optional<T> deserialize(String llmResponse) {
|
||||
try {
|
||||
var obj = (this.outputClass != null)
|
||||
? this.objectMapper.readValue(llmResponse, this.outputClass)
|
||||
: this.objectMapper.readValue(llmResponse, this.outputType);
|
||||
|
||||
return Optional.ofNullable(obj);
|
||||
} catch (JsonProcessingException e) {
|
||||
return Optional.empty();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,183 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import java.util.Arrays;
|
||||
|
||||
/**
|
||||
* An output guardrail is a rule that is applied to the output of the model to ensure that the output is safe and meets
|
||||
* the expectations.
|
||||
* <p>
|
||||
* In the case of reprompting, the reprompt message is added to the LLM context and the request is retried.
|
||||
* <p>
|
||||
* The maximum number of retries is configurable, defaulting to {@link dev.langchain4j.guardrail.config.OutputGuardrailsConfig#MAX_RETRIES_DEFAULT}.
|
||||
*/
|
||||
public interface OutputGuardrail extends Guardrail<OutputGuardrailRequest, OutputGuardrailResult> {
|
||||
/**
|
||||
* Validates the response from the LLM.
|
||||
*
|
||||
* @param responseFromLLM
|
||||
* the response from the LLM
|
||||
*/
|
||||
default OutputGuardrailResult validate(AiMessage responseFromLLM) {
|
||||
return failure("Validation not implemented");
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates the response from the LLM.
|
||||
* <p>
|
||||
* Unlike {@link #validate(AiMessage)}, this method allows to access the memory and the augmentation result (in the
|
||||
* case of a RAG).
|
||||
* <p>
|
||||
* Implementation must not attempt to write to the memory or the augmentation result.
|
||||
*
|
||||
* @param params
|
||||
* the parameters, including the response from the LLM, the memory, and the augmentation result.
|
||||
*/
|
||||
@Override
|
||||
default OutputGuardrailResult validate(OutputGuardrailRequest params) {
|
||||
return validate(params.responseFromLLM().aiMessage());
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a successful result without any successful text
|
||||
*
|
||||
* @return The result of a successful output guardrail validation.
|
||||
*/
|
||||
default OutputGuardrailResult success() {
|
||||
return OutputGuardrailResult.success();
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a successful result with specific success text
|
||||
*
|
||||
* @return The result of a successful output guardrail validation with a specific text.
|
||||
*
|
||||
* @param successfulText
|
||||
* The text of the successful result.
|
||||
*/
|
||||
default OutputGuardrailResult successWith(String successfulText) {
|
||||
return OutputGuardrailResult.successWith(successfulText);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @return The result of a successful output guardrail validation with a specific text.
|
||||
*
|
||||
* @param successfulText
|
||||
* The text of the successful result.
|
||||
* @param successfulResult
|
||||
* The object generated by this successful result.
|
||||
*/
|
||||
default OutputGuardrailResult successWith(String successfulText, Object successfulResult) {
|
||||
return OutputGuardrailResult.successWith(successfulText, successfulResult);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
*
|
||||
* @return The result of a failed output guardrail validation.
|
||||
*/
|
||||
default OutputGuardrailResult failure(String message) {
|
||||
return new OutputGuardrailResult(new OutputGuardrailResult.Failure(message), false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param cause
|
||||
* The exception that caused this failure.
|
||||
*
|
||||
* @return The result of a failed output guardrail validation.
|
||||
*/
|
||||
default OutputGuardrailResult failure(String message, Throwable cause) {
|
||||
return new OutputGuardrailResult(new OutputGuardrailResult.Failure(message, cause), false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
*
|
||||
* @return The result of a fatally failed output guardrail validation, blocking the evaluation of any other
|
||||
* subsequent validation.
|
||||
*/
|
||||
default OutputGuardrailResult fatal(String message) {
|
||||
return new OutputGuardrailResult(Arrays.asList(new OutputGuardrailResult.Failure(message)), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a fatal failure
|
||||
*
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param cause
|
||||
* The exception that caused this failure.
|
||||
*
|
||||
* @return The result of a fatally failed output guardrail validation, blocking the evaluation of any other
|
||||
* subsequent validation.
|
||||
*/
|
||||
default OutputGuardrailResult fatal(String message, Throwable cause) {
|
||||
return new OutputGuardrailResult(Arrays.asList(new OutputGuardrailResult.Failure(message, cause)), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
*
|
||||
* @return The result of a fatally failed output guardrail validation, blocking the evaluation of any other
|
||||
* subsequent validation and triggering a retry with the same user prompt.
|
||||
*/
|
||||
default OutputGuardrailResult retry(String message) {
|
||||
return new OutputGuardrailResult(Arrays.asList(new OutputGuardrailResult.Failure(message, null, true)), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param cause
|
||||
* The exception that caused this failure.
|
||||
*
|
||||
* @return The result of a fatally failed output guardrail validation, blocking the evaluation of any other
|
||||
* subsequent validation and triggering a retry with the same user prompt.
|
||||
*/
|
||||
default OutputGuardrailResult retry(String message, Throwable cause) {
|
||||
return new OutputGuardrailResult(Arrays.asList(new OutputGuardrailResult.Failure(message, cause, true)), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param reprompt
|
||||
* The new prompt to be used for the retry.
|
||||
*
|
||||
* @return The result of a fatally failed output guardrail validation, blocking the evaluation of any other
|
||||
* subsequent validation and triggering a retry with a new user prompt.
|
||||
*/
|
||||
default OutputGuardrailResult reprompt(String message, String reprompt) {
|
||||
return new OutputGuardrailResult(
|
||||
Arrays.asList(new OutputGuardrailResult.Failure(message, null, true, reprompt)), true);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param message
|
||||
* A message describing the failure.
|
||||
* @param cause
|
||||
* The exception that caused this failure.
|
||||
* @param reprompt
|
||||
* The new prompt to be used for the retry.
|
||||
*
|
||||
* @return The result of a fatally failed output guardrail validation, blocking the evaluation of any other
|
||||
* subsequent validation and triggering a retry with a new user prompt.
|
||||
*/
|
||||
default OutputGuardrailResult reprompt(String message, Throwable cause, String reprompt) {
|
||||
return new OutputGuardrailResult(
|
||||
Arrays.asList(new OutputGuardrailResult.Failure(message, cause, true, reprompt)), true);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
/**
|
||||
* Exception thrown when an output guardrail validation fails.
|
||||
* <p>
|
||||
* This class is not intended to be thrown within guardrail implementations. It is for the framework only. It is ok to catch it.
|
||||
* </p>
|
||||
*/
|
||||
public final class OutputGuardrailException extends GuardrailException {
|
||||
public OutputGuardrailException(String message) {
|
||||
super(message);
|
||||
}
|
||||
|
||||
public OutputGuardrailException(String message, Throwable cause) {
|
||||
super(message, cause);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,169 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import dev.langchain4j.data.message.UserMessage;
|
||||
import dev.langchain4j.guardrail.OutputGuardrailResult.Failure;
|
||||
import dev.langchain4j.guardrail.config.OutputGuardrailsConfig;
|
||||
import dev.langchain4j.memory.ChatMemory;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* The {@link GuardrailExecutor} for {@link OutputGuardrail}s.
|
||||
* <p>
|
||||
* When executing output guardrails, if any {@link OutputGuardrail} triggers a reprompt or retry,
|
||||
* the new response has to go back through the entire chain of output guardrails to ensure the new response
|
||||
* passes all the output guardrails.
|
||||
* </p>
|
||||
*/
|
||||
public non-sealed class OutputGuardrailExecutor
|
||||
extends AbstractGuardrailExecutor<
|
||||
OutputGuardrailsConfig, OutputGuardrailRequest, OutputGuardrailResult, OutputGuardrail, Failure> {
|
||||
|
||||
public static final String MAX_RETRIES_MESSAGE_TEMPLATE =
|
||||
"""
|
||||
Output validation failed. The guardrails have reached the maximum number of retries.
|
||||
Guardrail messages:
|
||||
|
||||
%s
|
||||
""";
|
||||
|
||||
protected OutputGuardrailExecutor(OutputGuardrailsConfig config, List<OutputGuardrail> guardrails) {
|
||||
super(config, guardrails);
|
||||
}
|
||||
|
||||
/**
|
||||
* Executes the {@link OutputGuardrail}s on the given {@link OutputGuardrailRequest}.
|
||||
*
|
||||
* @param params The {@link OutputGuardrailRequest} to validate
|
||||
* @return The {@link OutputGuardrailResult} of the validation
|
||||
*/
|
||||
@Override
|
||||
public OutputGuardrailResult execute(OutputGuardrailRequest params) {
|
||||
OutputGuardrailResult result = null;
|
||||
var accumulatedParams = params;
|
||||
var attempt = 0;
|
||||
var maxAttempts = config().maxRetries();
|
||||
|
||||
if (maxAttempts == 0) {
|
||||
maxAttempts = 1;
|
||||
} else if (maxAttempts < 0) {
|
||||
maxAttempts = OutputGuardrailsConfig.MAX_RETRIES_DEFAULT;
|
||||
}
|
||||
|
||||
while (attempt < maxAttempts) {
|
||||
result = executeGuardrails(accumulatedParams);
|
||||
|
||||
if (result.isSuccess()) {
|
||||
return result;
|
||||
}
|
||||
|
||||
// Not successful
|
||||
if (!result.isRetry()) {
|
||||
// Not any kind of retry, so just stop here
|
||||
throw new OutputGuardrailException(result.toString(), result.getFirstFailureException());
|
||||
}
|
||||
|
||||
// If we get here we know it is some kind of retry
|
||||
// We don't want to add intermediary UserMessages to the memory
|
||||
var chatMessages = Optional.ofNullable(
|
||||
accumulatedParams.requestParams().chatMemory())
|
||||
.map(ChatMemory::messages)
|
||||
.orElseGet(ArrayList::new);
|
||||
result.getReprompt().map(UserMessage::from).ifPresent(chatMessages::add);
|
||||
|
||||
// Re-execute the request with the appended message
|
||||
// But don't add it or the resulting message to the memory
|
||||
var response = accumulatedParams.chatExecutor().execute(chatMessages);
|
||||
|
||||
attempt++;
|
||||
accumulatedParams = OutputGuardrailRequest.builder()
|
||||
.responseFromLLM(response)
|
||||
.chatExecutor(accumulatedParams.chatExecutor())
|
||||
.requestParams(accumulatedParams.requestParams())
|
||||
.build();
|
||||
}
|
||||
|
||||
if (attempt == maxAttempts) {
|
||||
var failureMessages = result.failures().stream()
|
||||
.map(GuardrailResult.Failure::message)
|
||||
.collect(Collectors.joining(System.lineSeparator()));
|
||||
|
||||
throw new OutputGuardrailException(MAX_RETRIES_MESSAGE_TEMPLATE.formatted(failureMessages));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a failure result from some {@link Failure}s.
|
||||
* @param failures The failures
|
||||
* @return A {@link OutputGuardrailResult} containing the failures
|
||||
*/
|
||||
@Override
|
||||
protected OutputGuardrailResult createFailure(List<Failure> failures) {
|
||||
return OutputGuardrailResult.failure(failures);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a success result.
|
||||
* @return A {@link OutputGuardrailResult} representing success
|
||||
*/
|
||||
@Override
|
||||
protected OutputGuardrailResult createSuccess() {
|
||||
return OutputGuardrailResult.success();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected OutputGuardrailException createGuardrailException(String message, Throwable cause) {
|
||||
return new OutputGuardrailException(message, cause);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected OutputGuardrailResult handleFatalResult(
|
||||
OutputGuardrailResult accumulatedResult, OutputGuardrailResult result) {
|
||||
return accumulatedResult.hasRewrittenResult() ? result.blockRetry() : result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new instance of {@link OutputGuardrailExecutorBuilder}.
|
||||
* The builder is used to construct and configure instances of {@link OutputGuardrailExecutorBuilder}.
|
||||
* @return A new {@link OutputGuardrailExecutorBuilder} instance.
|
||||
*/
|
||||
public static OutputGuardrailExecutorBuilder builder() {
|
||||
return new OutputGuardrailExecutorBuilder();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder class for constructing instances of {@link OutputGuardrailExecutor}.
|
||||
*
|
||||
* This builder allows configuration of an {@link OutputGuardrailExecutor} by specifying the associated configuration
|
||||
* type ({@link OutputGuardrailsConfig}) and the output guardrails to be executed.
|
||||
*
|
||||
* Extends {@link GuardrailExecutorBuilder} for the specific types:
|
||||
* - Configuration type: {@link OutputGuardrailsConfig}
|
||||
* - Result type: {@link OutputGuardrailResult}
|
||||
* - Parameter type: {@link OutputGuardrailRequest}
|
||||
* - Guardrail type: {@link OutputGuardrail}
|
||||
*
|
||||
* Provides the {@code build()} method to create an {@link OutputGuardrailExecutor} instance.
|
||||
*/
|
||||
public static non-sealed class OutputGuardrailExecutorBuilder
|
||||
extends GuardrailExecutorBuilder<
|
||||
OutputGuardrailsConfig,
|
||||
OutputGuardrailResult,
|
||||
OutputGuardrailRequest,
|
||||
OutputGuardrail,
|
||||
OutputGuardrailExecutorBuilder> {
|
||||
|
||||
protected OutputGuardrailExecutorBuilder() {
|
||||
super(OutputGuardrailsConfig.builder().build());
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailExecutor build() {
|
||||
return new OutputGuardrailExecutor(config(), guardrails());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,134 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.model.chat.ChatExecutor;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Represents the parameter passed to {@link OutputGuardrail#validate(OutputGuardrailRequest)}.
|
||||
*/
|
||||
public final class OutputGuardrailRequest implements GuardrailRequest<OutputGuardrailRequest> {
|
||||
private final ChatResponse responseFromLLM;
|
||||
private final ChatExecutor chatExecutor;
|
||||
private final GuardrailRequestParams requestParams;
|
||||
|
||||
private OutputGuardrailRequest(Builder builder) {
|
||||
this.responseFromLLM = ensureNotNull(builder.responseFromLLM, "responseFromLLM");
|
||||
this.requestParams = ensureNotNull(builder.requestParams, "requestParams");
|
||||
this.chatExecutor = ensureNotNull(builder.chatExecutor, "chatExecutor");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the response from the LLM.
|
||||
*
|
||||
* @return the response from the LLM
|
||||
*/
|
||||
public ChatResponse responseFromLLM() {
|
||||
return responseFromLLM;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the chat executor.
|
||||
*
|
||||
* @return the chat executor
|
||||
*/
|
||||
public ChatExecutor chatExecutor() {
|
||||
return chatExecutor;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the common parameters that are shared across guardrail checks.
|
||||
*
|
||||
* @return an instance of {@code GuardrailRequestParams} containing shared parameters
|
||||
*/
|
||||
@Override
|
||||
public GuardrailRequestParams requestParams() {
|
||||
return requestParams;
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailRequest withText(String text) {
|
||||
ensureNotNull(text, "text");
|
||||
|
||||
var aiMessage = Optional.ofNullable(this.responseFromLLM.aiMessage().toolExecutionRequests())
|
||||
.filter(t -> !t.isEmpty())
|
||||
.map(t -> new AiMessage(text, t))
|
||||
.orElseGet(() -> new AiMessage(text));
|
||||
|
||||
var chatResponse = ChatResponse.builder()
|
||||
.aiMessage(aiMessage)
|
||||
.metadata(this.responseFromLLM.metadata())
|
||||
.build();
|
||||
|
||||
return builder()
|
||||
.responseFromLLM(chatResponse)
|
||||
.chatExecutor(this.chatExecutor)
|
||||
.requestParams(this.requestParams)
|
||||
.build();
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new builder for {@link OutputGuardrailRequest}.
|
||||
*
|
||||
* @return a new builder
|
||||
*/
|
||||
public static Builder builder() {
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder for {@link OutputGuardrailRequest}.
|
||||
*/
|
||||
public static class Builder {
|
||||
private ChatResponse responseFromLLM;
|
||||
private ChatExecutor chatExecutor;
|
||||
private GuardrailRequestParams requestParams;
|
||||
|
||||
private Builder() {}
|
||||
|
||||
/**
|
||||
* Sets the response from the LLM.
|
||||
*
|
||||
* @param responseFromLLM the response from the LLM
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder responseFromLLM(ChatResponse responseFromLLM) {
|
||||
this.responseFromLLM = responseFromLLM;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the chat executor.
|
||||
*
|
||||
* @param chatExecutor the chat executor
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder chatExecutor(ChatExecutor chatExecutor) {
|
||||
this.chatExecutor = chatExecutor;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the common parameters.
|
||||
*
|
||||
* @param requestParams the common parameters
|
||||
* @return this builder
|
||||
*/
|
||||
public Builder requestParams(GuardrailRequestParams requestParams) {
|
||||
this.requestParams = requestParams;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a new {@link OutputGuardrailRequest}.
|
||||
*
|
||||
* @return a new {@link OutputGuardrailRequest}
|
||||
*/
|
||||
public OutputGuardrailRequest build() {
|
||||
return new OutputGuardrailRequest(this);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,290 @@
|
|||
package dev.langchain4j.guardrail;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
import dev.langchain4j.data.message.AiMessage;
|
||||
import dev.langchain4j.model.chat.response.ChatResponse;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.stream.Stream;
|
||||
|
||||
/**
|
||||
* The result of the validation of an {@link OutputGuardrail}
|
||||
*/
|
||||
public final class OutputGuardrailResult implements GuardrailResult<OutputGuardrailResult> {
|
||||
private static final OutputGuardrailResult SUCCESS = new OutputGuardrailResult();
|
||||
|
||||
private final Result result;
|
||||
private final String successfulText;
|
||||
private final Object successfulResult;
|
||||
private final List<Failure> failures;
|
||||
|
||||
private OutputGuardrailResult(
|
||||
Result result, String successfulText, Object successfulResult, List<Failure> failures) {
|
||||
this.result = ensureNotNull(result, "result");
|
||||
this.successfulText = successfulText;
|
||||
this.successfulResult = successfulResult;
|
||||
this.failures = Optional.ofNullable(failures).orElseGet(List::of);
|
||||
}
|
||||
|
||||
private OutputGuardrailResult() {
|
||||
this(Result.SUCCESS, null, null, Collections.emptyList());
|
||||
}
|
||||
|
||||
private OutputGuardrailResult(String successfulText) {
|
||||
this(Result.SUCCESS_WITH_RESULT, successfulText, null, Collections.emptyList());
|
||||
}
|
||||
|
||||
private OutputGuardrailResult(String successfulText, Object successfulResult) {
|
||||
this(Result.SUCCESS_WITH_RESULT, successfulText, successfulResult, Collections.emptyList());
|
||||
}
|
||||
|
||||
OutputGuardrailResult(List<Failure> failures, boolean fatal) {
|
||||
this(fatal ? Result.FATAL : Result.FAILURE, null, null, failures);
|
||||
}
|
||||
|
||||
OutputGuardrailResult(Failure failure, boolean fatal) {
|
||||
// Using Stream.of().collect() here because we need a mutable list
|
||||
this(Stream.of(failure).collect(Collectors.toList()), fatal);
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets a successful output guardrail result
|
||||
*/
|
||||
public static OutputGuardrailResult success() {
|
||||
return SUCCESS;
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a successful result with specific success text
|
||||
*
|
||||
* @return The result of a successful output guardrail validation with a specific text.
|
||||
*
|
||||
* @param successfulText
|
||||
* The text of the successful result.
|
||||
*/
|
||||
public static OutputGuardrailResult successWith(String successfulText) {
|
||||
return (successfulText == null) ? success() : new OutputGuardrailResult(successfulText);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param successfulText
|
||||
* The text of the successful result.
|
||||
* @param successfulResult
|
||||
* The object generated by this successful result.
|
||||
* @return The result of a successful output guardrail validation with a specific text.
|
||||
*/
|
||||
public static OutputGuardrailResult successWith(String successfulText, Object successfulResult) {
|
||||
return new OutputGuardrailResult(successfulText, successfulResult);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a non-fatal failure
|
||||
*
|
||||
* @param failures A list of {@link Failure}s
|
||||
*
|
||||
* @return The result of a failed output guardrail validation.
|
||||
*/
|
||||
public static OutputGuardrailResult failure(List<Failure> failures) {
|
||||
return new OutputGuardrailResult(failures, false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether or not the guardrail is forcing a retry
|
||||
*/
|
||||
public boolean isRetry() {
|
||||
return !isSuccess() && this.failures.stream().anyMatch(Failure::retry);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether or not the guardrail is forcing a reprompt
|
||||
*/
|
||||
public boolean isReprompt() {
|
||||
return !isSuccess()
|
||||
&& this.failures.stream()
|
||||
.map(Failure::reprompt)
|
||||
.filter(Objects::nonNull)
|
||||
.count()
|
||||
> 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Block all retries for this result
|
||||
*/
|
||||
public OutputGuardrailResult blockRetry() {
|
||||
this.failures.set(0, this.failures.get(0).blockRetry());
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the reprompt message
|
||||
*/
|
||||
public Optional<String> getReprompt() {
|
||||
return !isSuccess()
|
||||
? this.failures.stream()
|
||||
.map(Failure::reprompt)
|
||||
.filter(Objects::nonNull)
|
||||
.findFirst()
|
||||
: Optional.empty();
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return asString();
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean equals(Object o) {
|
||||
if (this == o) return true;
|
||||
if (o == null || getClass() != o.getClass()) return false;
|
||||
OutputGuardrailResult that = (OutputGuardrailResult) o;
|
||||
return result == that.result
|
||||
&& Objects.equals(successfulText, that.successfulText)
|
||||
&& Objects.equals(successfulResult, that.successfulResult)
|
||||
&& Objects.equals(failures, that.failures);
|
||||
}
|
||||
|
||||
@Override
|
||||
public int hashCode() {
|
||||
return Objects.hash(result, successfulText, successfulResult, failures);
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the response computed from the combination of the original {@link ChatResponse} in the {@link OutputGuardrailRequest}
|
||||
* and this result
|
||||
* @param request The output guardrail request
|
||||
* @param <T> The type of response
|
||||
* @return A response computed from the combination of the original {@link ChatResponse} in the {@link OutputGuardrailRequest}
|
||||
* and this result
|
||||
*/
|
||||
public <T> T response(OutputGuardrailRequest request) {
|
||||
return (T) Optional.ofNullable(successfulResult).orElseGet(() -> createResponse(request));
|
||||
}
|
||||
|
||||
private ChatResponse createResponse(OutputGuardrailRequest params) {
|
||||
var response = params.responseFromLLM();
|
||||
var aiMessage = response.aiMessage();
|
||||
var newAiMessage = aiMessage;
|
||||
|
||||
if (hasRewrittenResult()) {
|
||||
newAiMessage = aiMessage.hasToolExecutionRequests()
|
||||
? AiMessage.from(successfulText(), aiMessage.toolExecutionRequests())
|
||||
: AiMessage.from(successfulText());
|
||||
}
|
||||
|
||||
return response.toBuilder().aiMessage(newAiMessage).build();
|
||||
}
|
||||
|
||||
@Override
|
||||
public Result result() {
|
||||
return result;
|
||||
}
|
||||
|
||||
@Override
|
||||
@SuppressWarnings("unchecked")
|
||||
public <F extends GuardrailResult.Failure> List<F> failures() {
|
||||
return (List<F>) failures;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String successfulText() {
|
||||
return successfulText;
|
||||
}
|
||||
|
||||
public Object successfulResult() {
|
||||
return successfulResult;
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents an output guardrail failure
|
||||
*/
|
||||
public static final class Failure implements GuardrailResult.Failure {
|
||||
private final String message;
|
||||
private final Throwable cause;
|
||||
private final Class<? extends Guardrail> guardrailClass;
|
||||
private final boolean retry;
|
||||
private final String reprompt;
|
||||
|
||||
Failure(
|
||||
String message,
|
||||
Throwable cause,
|
||||
Class<? extends Guardrail> guardrailClass,
|
||||
boolean retry,
|
||||
String reprompt) {
|
||||
this.message = ensureNotNull(message, "message");
|
||||
this.cause = cause;
|
||||
this.guardrailClass = guardrailClass;
|
||||
this.retry = retry;
|
||||
this.reprompt = reprompt;
|
||||
}
|
||||
|
||||
Failure(String message) {
|
||||
this(message, null);
|
||||
}
|
||||
|
||||
Failure(String message, Throwable cause) {
|
||||
this(message, cause, false);
|
||||
}
|
||||
|
||||
Failure(String message, Throwable cause, boolean retry) {
|
||||
this(message, cause, null, retry, null);
|
||||
}
|
||||
|
||||
Failure(String message, Throwable cause, boolean retry, String reprompt) {
|
||||
this(message, cause, null, retry, reprompt);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Failure withGuardrailClass(Class<? extends Guardrail> guardrailClass) {
|
||||
ensureNotNull(guardrailClass, "guardrailClass");
|
||||
return new Failure(message(), cause(), guardrailClass, this.retry, this.reprompt);
|
||||
}
|
||||
|
||||
@Override
|
||||
public String message() {
|
||||
return message;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Throwable cause() {
|
||||
return cause;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Class<? extends Guardrail> guardrailClass() {
|
||||
return guardrailClass;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a failure from this failure that blocks retries
|
||||
*/
|
||||
public Failure blockRetry() {
|
||||
return this.retry
|
||||
? new Failure(
|
||||
"Retry or reprompt is not allowed after a rewritten output",
|
||||
cause(),
|
||||
this.guardrailClass,
|
||||
false,
|
||||
this.reprompt)
|
||||
: this;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return asString();
|
||||
}
|
||||
|
||||
public boolean retry() {
|
||||
return retry;
|
||||
}
|
||||
|
||||
public String reprompt() {
|
||||
return reprompt;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
package dev.langchain4j.guardrail.config;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
/**
|
||||
* The default implementation of {@link InputGuardrailsConfig} for this library if no other libraries provide their own implementations.
|
||||
*/
|
||||
final class DefaultInputGuardrailsConfig implements InputGuardrailsConfig {
|
||||
DefaultInputGuardrailsConfig(Builder builder) {
|
||||
ensureNotNull(builder, "builder");
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets a builder instance for building {@link DefaultInputGuardrailsConfig} instances.
|
||||
* @return The builder instance for building {@link DefaultInputGuardrailsConfig} instances.
|
||||
*/
|
||||
static Builder builder() {
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder for {@link DefaultInputGuardrailsConfig} instances.
|
||||
*/
|
||||
static class Builder implements InputGuardrailsConfigBuilder {
|
||||
@Override
|
||||
public InputGuardrailsConfig build() {
|
||||
return new DefaultInputGuardrailsConfig(this);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
package dev.langchain4j.guardrail.config;
|
||||
|
||||
import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
|
||||
|
||||
/**
|
||||
* The default implementation of {@link OutputGuardrailsConfig} for this library if no other libraries provide their own implementations.
|
||||
*/
|
||||
final class DefaultOutputGuardrailsConfig implements OutputGuardrailsConfig {
|
||||
private final int maxRetries;
|
||||
|
||||
DefaultOutputGuardrailsConfig(Builder builder) {
|
||||
ensureNotNull(builder, "builder");
|
||||
this.maxRetries = builder.maxRetries;
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets a builder instance for building {@link DefaultOutputGuardrailsConfig} instances.
|
||||
* @return The builder instance for building {@link DefaultOutputGuardrailsConfig} instances.
|
||||
*/
|
||||
static Builder builder() {
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
@Override
|
||||
public int maxRetries() {
|
||||
return this.maxRetries;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder for {@link DefaultOutputGuardrailsConfig} instances.
|
||||
*/
|
||||
static class Builder implements OutputGuardrailsConfigBuilder {
|
||||
private int maxRetries = MAX_RETRIES_DEFAULT;
|
||||
|
||||
@Override
|
||||
public Builder maxRetries(int maxRetries) {
|
||||
this.maxRetries = maxRetries;
|
||||
return this;
|
||||
}
|
||||
|
||||
@Override
|
||||
public OutputGuardrailsConfig build() {
|
||||
return new DefaultOutputGuardrailsConfig(this);
|
||||
}
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue