dnrops
/
open-webui
mirror of https://github.com/open-webui/open-webui
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into feature-external-db-reconnect

This commit is contained in:
perf3ct 2024-06-16 09:03:57 -07:00
parent ad32a2ef3c 162643a4b1
commit 75d713057c
No known key found for this signature in database
GPG Key ID: 569C4EEC436F5232
206 changed files with 21988 additions and 7009 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/pull_request_template.md vendored
View File

@ -11,7 +11,7 @@
- [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [ ] **Testing:** Have you written and run sufficient tests for validating the changes?
- [ ] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [ ] **Label:** To cleary categorize this pull request, assign a relevant label to the pull request title, using one of the following:
- [ ] **Prefix:** To cleary categorize this pull request, prefix the pull request title, using one of the following:
- **BREAKING CHANGE**: Significant changes that may affect compatibility
- **build**: Changes that affect the build system or external dependencies
- **ci**: Changes to our continuous integration processes or workflows

6
.github/workflows/docker-build.yaml vendored
View File

@ -70,8 +70,10 @@ jobs:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-${{ matrix.platform }}-
latest=false
- name: Build Docker image (latest)
uses: docker/build-push-action@v5
@ -158,8 +160,10 @@ jobs:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-cuda-${{ matrix.platform }}-
latest=false
- name: Build Docker image (cuda)
uses: docker/build-push-action@v5
@ -247,8 +251,10 @@ jobs:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-ollama-${{ matrix.platform }}-
latest=false
- name: Build Docker image (ollama)
uses: docker/build-push-action@v5

1
.github/workflows/release-pypi.yml vendored
View File

@ -4,7 +4,6 @@ on:
push:
branches:
- main # or whatever branch you want to use
- dev
jobs:
release:

184
CHANGELOG.md
View File

@ -5,6 +5,190 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.3.4] - 2024-06-12
### Fixed
- **🔒 Mixed Content with HTTPS Issue**: Resolved a problem where mixed content (HTTP and HTTPS) was causing security warnings and blocking resources on HTTPS sites.
- **🔍 Web Search Issue**: Addressed the problem where web search functionality was not working correctly. The 'ENABLE_RAG_LOCAL_WEB_FETCH' option has been reintroduced to restore proper web searching capabilities.
- **💾 RAG Template Not Being Saved**: Fixed an issue where the RAG template was not being saved correctly, ensuring your custom templates are now preserved as expected.
## [0.3.3] - 2024-06-12
### Added
- **🛠️ Native Python Function Calling**: Introducing native Python function calling within Open WebUI. We’ve also included a built-in code editor to seamlessly develop and integrate function code within the 'Tools' workspace. With this, you can significantly enhance your LLM’s capabilities by creating custom RAG pipelines, web search tools, and even agent-like features such as sending Discord messages.
- **🌐 DuckDuckGo Integration**: Added DuckDuckGo as a web search provider, giving you more search options.
- **🌏 Enhanced Translations**: Improved translations for Vietnamese and Chinese languages, making the interface more accessible.
### Fixed
- **🔗 Web Search URL Error Handling**: Fixed the issue where a single URL error would disrupt the data loading process in Web Search mode. Now, such errors will be handled gracefully to ensure uninterrupted data loading.
- **🖥️ Frontend Responsiveness**: Resolved the problem where the frontend would stop responding if the backend encounters an error while downloading a model. Improved error handling to maintain frontend stability.
- **🔧 Dependency Issues in pip**: Fixed issues related to pip installations, ensuring all dependencies are correctly managed to prevent installation errors.
## [0.3.2] - 2024-06-10
### Added
- **🔍 Web Search Query Status**: The web search query will now persist in the results section to aid in easier debugging and tracking of search queries.
- **🌐 New Web Search Provider**: We have added Serply as a new option for web search providers, giving you more choices for your search needs.
- **🌏 Improved Translations**: We've enhanced translations for Chinese and Portuguese.
### Fixed
- **🎤 Audio File Upload Issue**: The bug that prevented audio files from being uploaded in chat input has been fixed, ensuring smooth communication.
- **💬 Message Input Handling**: Improved the handling of message inputs by instantly clearing images and text after sending, along with immediate visual indications when a response message is loading, enhancing user feedback.
- **⚙️ Parameter Registration and Validation**: Fixed the issue where parameters were not registering in certain cases and addressed the problem where users were unable to save due to invalid input errors.
## [0.3.1] - 2024-06-09
### Fixed
- **💬 Chat Functionality**: Resolved the issue where chat functionality was not working for specific models.
## [0.3.0] - 2024-06-09
### Added
- **📚 Knowledge Support for Models**: Attach documents directly to models from the models workspace, enhancing the information available to each model.
- **🎙️ Hands-Free Voice Call Feature**: Initiate voice calls without needing to use your hands, making interactions more seamless.
- **📹 Video Call Feature**: Enable video calls with supported vision models like Llava and GPT-4o, adding a visual dimension to your communications.
- **🎛️ Enhanced UI for Voice Recording**: Improved user interface for the voice recording feature, making it more intuitive and user-friendly.
- **🌐 External STT Support**: Now support for external Speech-To-Text services, providing more flexibility in choosing your STT provider.
- **⚙️ Unified Settings**: Consolidated settings including document settings under a new admin settings section for easier management.
- **🌑 Dark Mode Splash Screen**: A new splash screen for dark mode, ensuring a consistent and visually appealing experience for dark mode users.
- **📥 Upload Pipeline**: Directly upload pipelines from the admin settings > pipelines section, streamlining the pipeline management process.
- **🌍 Improved Language Support**: Enhanced support for Chinese and Ukrainian languages, better catering to a global user base.
### Fixed
- **🛠️ Playground Issue**: Fixed the playground not functioning properly, ensuring a smoother user experience.
- **🔥 Temperature Parameter Issue**: Corrected the issue where the temperature value '0' was not being passed correctly.
- **📝 Prompt Input Clearing**: Resolved prompt input textarea not being cleared right away, ensuring a clean slate for new inputs.
- **✨ Various UI Styling Issues**: Fixed numerous user interface styling problems for a more cohesive look.
- **👥 Active Users Display**: Fixed active users showing active sessions instead of actual users, now reflecting accurate user activity.
- **🌐 Community Platform Compatibility**: The Community Platform is back online and fully compatible with Open WebUI.
### Changed
- **📝 RAG Implementation**: Updated the RAG (Retrieval-Augmented Generation) implementation to use a system prompt for context, instead of overriding the user's prompt.
- **🔄 Settings Relocation**: Moved Models, Connections, Audio, and Images settings to the admin settings for better organization.
- **✍️ Improved Title Generation**: Enhanced the default prompt for title generation, yielding better results.
- **🔧 Backend Task Management**: Tasks like title generation and search query generation are now managed on the backend side and controlled only by the admin.
- **🔍 Editable Search Query Prompt**: You can now edit the search query generation prompt, offering more control over how queries are generated.
- **📏 Prompt Length Threshold**: Set the prompt length threshold for search query generation from the admin settings, giving more customization options.
- **📣 Settings Consolidation**: Merged the Banners admin setting with the Interface admin setting for a more streamlined settings area.
## [0.2.5] - 2024-06-05
### Added
- **👥 Active Users Indicator**: Now you can see how many people are currently active and what they are running. This helps you gauge when performance might slow down due to a high number of users.
- **🗂️ Create Ollama Modelfile**: The option to create a modelfile for Ollama has been reintroduced in the Settings > Models section, making it easier to manage your models.
- **⚙️ Default Model Setting**: Added an option to set the default model from Settings > Interface. This feature is now easily accessible, especially convenient for mobile users as it was previously hidden.
- **🌐 Enhanced Translations**: We've improved the Chinese translations and added support for Turkmen and Norwegian languages to make the interface more accessible globally.
### Fixed
- **📱 Mobile View Improvements**: The UI now uses dvh (dynamic viewport height) instead of vh (viewport height), providing a better and more responsive experience for mobile users.
## [0.2.4] - 2024-06-03
### Added
- **👤 Improved Account Pending Page**: The account pending page now displays admin details by default to avoid confusion. You can disable this feature in the admin settings if needed.
- **🌐 HTTP Proxy Support**: We have enabled the use of the 'http_proxy' environment variable in OpenAI and Ollama API calls, making it easier to configure network settings.
- **❓ Quick Access to Documentation**: You can now easily access Open WebUI documents via a question mark button located at the bottom right corner of the screen (available on larger screens like PCs).
- **🌍 Enhanced Translation**: Improvements have been made to translations.
### Fixed
- **🔍 SearxNG Web Search**: Fixed the issue where the SearxNG web search functionality was not working properly.
## [0.2.3] - 2024-06-03
### Added
- **📁 Export Chat as JSON**: You can now export individual chats as JSON files from the navbar menu by navigating to 'Download > Export Chat'. This makes sharing specific conversations easier.
- **✏️ Edit Titles with Double Click**: Double-click on titles to rename them quickly and efficiently.
- **🧩 Batch Multiple Embeddings**: Introduced 'RAG_EMBEDDING_OPENAI_BATCH_SIZE' to process multiple embeddings in a batch, enhancing performance for large datasets.
- **🌍 Improved Translations**: Enhanced the translation quality across various languages for a better user experience.
### Fixed
- **🛠️ Modelfile Migration Script**: Fixed an issue where the modelfile migration script would fail if an invalid modelfile was encountered.
- **💬 Zhuyin Input Method on Mac**: Resolved an issue where using the Zhuyin input method in the Web UI on a Mac caused text to send immediately upon pressing the enter key, leading to incorrect input.
- **🔊 Local TTS Voice Selection**: Fixed the issue where the selected local Text-to-Speech (TTS) voice was not being displayed in settings.
## [0.2.2] - 2024-06-02
### Added
- **🌊 Mermaid Rendering Support**: We've included support for Mermaid rendering. This allows you to create beautiful diagrams and flowcharts directly within Open WebUI.
- **🔄 New Environment Variable 'RESET_CONFIG_ON_START'**: Introducing a new environment variable: 'RESET_CONFIG_ON_START'. Set this variable to reset your configuration settings upon starting the application, making it easier to revert to default settings.
### Fixed
- **🔧 Pipelines Filter Issue**: We've addressed an issue with the pipelines where filters were not functioning as expected.
## [0.2.1] - 2024-06-02
### Added
- **🖱️ Single Model Export Button**: Easily export models with just one click using the new single model export button.
- **🖥️ Advanced Parameters Support**: Added support for 'num_thread', 'use_mmap', and 'use_mlock' parameters for Ollama.
- **🌐 Improved Vietnamese Translation**: Enhanced Vietnamese language support for a better user experience for our Vietnamese-speaking community.
### Fixed
- **🔧 OpenAI URL API Save Issue**: Corrected a problem preventing the saving of OpenAI URL API settings.
- **🚫 Display Issue with Disabled Ollama API**: Fixed the display bug causing models to appear in settings when the Ollama API was disabled.
### Changed
- **💡 Versioning Update**: As a reminder from our previous update, version 0.2.y will focus primarily on bug fixes, while major updates will be designated as 0.x from now on for better version tracking.
## [0.2.0] - 2024-06-01
### Added
- **🔧 Pipelines Support**: Open WebUI now includes a plugin framework for enhanced customization and functionality (https://github.com/open-webui/pipelines). Easily add custom logic and integrate Python libraries, from AI agents to home automation APIs.
- **🔗 Function Calling via Pipelines**: Integrate function calling seamlessly through Pipelines.
- **⚖️ User Rate Limiting via Pipelines**: Implement user-specific rate limits to manage API usage efficiently.
- **📊 Usage Monitoring with Langfuse**: Track and analyze usage statistics with Langfuse integration through Pipelines.
- **🕒 Conversation Turn Limits**: Set limits on conversation turns to manage interactions better through Pipelines.
- **🛡️ Toxic Message Filtering**: Automatically filter out toxic messages to maintain a safe environment using Pipelines.
- **🔍 Web Search Support**: Introducing built-in web search capabilities via RAG API, allowing users to search using SearXNG, Google Programmatic Search Engine, Brave Search, serpstack, and serper. Activate it effortlessly by adding necessary variables from Document settings > Web Params.
- **🗂️ Models Workspace**: Create and manage model presets for both Ollama/OpenAI API. Note: The old Modelfiles workspace is deprecated.
- **🛠️ Model Builder Feature**: Build and edit all models with persistent builder mode.
- **🏷️ Model Tagging Support**: Organize models with tagging features in the models workspace.
- **📋 Model Ordering Support**: Effortlessly organize models by dragging and dropping them into the desired positions within the models workspace.
- **📈 OpenAI Generation Stats**: Access detailed generation statistics for OpenAI models.
- **📅 System Prompt Variables**: New variables added: '{{CURRENT_DATE}}' and '{{USER_NAME}}' for dynamic prompts.
- **📢 Global Banner Support**: Manage global banners from admin settings > banners.
- **🗃️ Enhanced Archived Chats Modal**: Search and export archived chats easily.
- **📂 Archive All Button**: Quickly archive all chats from settings > chats.
- **🌐 Improved Translations**: Added and improved translations for French, Croatian, Cebuano, and Vietnamese.
### Fixed
- **🔍 Archived Chats Visibility**: Resolved issue with archived chats not showing in the admin panel.
- **💬 Message Styling**: Fixed styling issues affecting message appearance.
- **🔗 Shared Chat Responses**: Corrected the issue where shared chat response messages were not readonly.
- **🖥️ UI Enhancement**: Fixed the scrollbar overlapping issue with the message box in the user interface.
### Changed
- **💾 User Settings Storage**: User settings are now saved on the backend, ensuring consistency across all devices.
- **📡 Unified API Requests**: The API request for getting models is now unified to '/api/models' for easier usage.
- **🔄 Versioning Update**: Our versioning will now follow the format 0.x for major updates and 0.x.y for patches.
- **📦 Export All Chats (All Users)**: Moved this functionality to the Admin Panel settings for better organization and accessibility.
### Removed
- **🚫 Bundled LiteLLM Support Deprecated**: Migrate your LiteLLM config.yaml to a self-hosted LiteLLM instance. LiteLLM can still be added via OpenAI Connections. Download the LiteLLM config.yaml from admin settings > database > export LiteLLM config.yaml.
## [0.1.125] - 2024-05-19
### Added

77
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,77 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contribute to a positive environment for our community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address, without their explicit permission
- **Spamming of any kind**
- Aggressive sales tactics targeting our community members are strictly prohibited. You can mention your product if it's relevant to the discussion, but under no circumstances should you push it forcefully
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies within all community spaces and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, spamming, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at hello@openwebui.com. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Temporary Ban
**Community Impact**: Any violation of community standards, including but not limited to inappropriate language, unprofessional behavior, harassment, or spamming.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 2. Permanent Ban
**Community Impact**: Repeated or severe violations of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

2
Dockerfile
View File

@ -38,7 +38,6 @@ ARG USE_OLLAMA
ARG USE_CUDA_VER
ARG USE_EMBEDDING_MODEL
ARG USE_RERANKING_MODEL
ARG BUILD_HASH
ARG UID
ARG GID
@ -154,6 +153,7 @@ HEALTHCHECK CMD curl --silent --fail http://localhost:8080/health | jq -e '.stat
USER $UID:$GID
ARG BUILD_HASH
ENV WEBUI_BUILD_VERSION=${BUILD_HASH}
CMD [ "bash", "start.sh"]

27
README.md
View File

@ -11,7 +11,7 @@
[![Discord](https://img.shields.io/badge/Discord-Open_WebUI-blue?logo=discord&logoColor=white)](https://discord.gg/5rJgQTnV4s)
[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/tjbck)
Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI designed to operate entirely offline. It supports various LLM runners, including Ollama and OpenAI-compatible APIs. For more information, be sure to check out our [Open WebUI Documentation](https://docs.openwebui.com/).
Open WebUI is an [extensible](https://github.com/open-webui/pipelines), feature-rich, and user-friendly self-hosted WebUI designed to operate entirely offline. It supports various LLM runners, including Ollama and OpenAI-compatible APIs. For more information, be sure to check out our [Open WebUI Documentation](https://docs.openwebui.com/).
![Open WebUI Demo](./demo.gif)
@ -19,7 +19,9 @@ Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI d
- 🚀 **Effortless Setup**: Install seamlessly using Docker or Kubernetes (kubectl, kustomize or helm) for a hassle-free experience with support for both `:ollama` and `:cuda` tagged images.
- 🤝 **OpenAI API Integration**: Effortlessly integrate OpenAI-compatible APIs for versatile conversations alongside Ollama models. Customize the OpenAI API URL to link with **LMStudio, GroqCloud, Mistral, OpenRouter, and more**.
- 🤝 **Ollama/OpenAI API Integration**: Effortlessly integrate OpenAI-compatible APIs for versatile conversations alongside Ollama models. Customize the OpenAI API URL to link with **LMStudio, GroqCloud, Mistral, OpenRouter, and more**.
- 🧩 **Pipelines, Open WebUI Plugin Support**: Seamlessly integrate custom logic and Python libraries into Open WebUI using [Pipelines Plugin Framework](https://github.com/open-webui/pipelines). Launch your Pipelines instance, set the OpenAI URL to the Pipelines URL, and explore endless possibilities. [Examples](https://github.com/open-webui/pipelines/tree/main/examples) include **Function Calling**, User **Rate Limiting** to control access, **Usage Monitoring** with tools like Langfuse, **Live Translation with LibreTranslate** for multilingual support, **Toxic Message Filtering** and much more.
- 📱 **Responsive Design**: Enjoy a seamless experience across Desktop PC, Laptop, and Mobile devices.
@ -27,11 +29,15 @@ Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI d
- ✒️🔢 **Full Markdown and LaTeX Support**: Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction.
- 🧩 **Model Builder**: Easily create Ollama models via the Web UI. Create and add custom characters/agents, customize chat elements, and import models effortlessly through [Open WebUI Community](https://openwebui.com/) integration.
- 🎤📹 **Hands-Free Voice/Video Call**: Experience seamless communication with integrated hands-free voice and video call features, allowing for a more dynamic and interactive chat environment.
- 🛠️ **Model Builder**: Easily create Ollama models via the Web UI. Create and add custom characters/agents, customize chat elements, and import models effortlessly through [Open WebUI Community](https://openwebui.com/) integration.
- 🐍 **Native Python Function Calling Tool**: Enhance your LLMs with built-in code editor support in the tools workspace. Bring Your Own Function (BYOF) by simply adding your pure Python functions, enabling seamless integration with LLMs.
- 📚 **Local RAG Integration**: Dive into the future of chat interactions with groundbreaking Retrieval Augmented Generation (RAG) support. This feature seamlessly integrates document interactions into your chat experience. You can load documents directly into the chat or add files to your document library, effortlessly accessing them using the `#` command before a query.
- 🔍 **Web Search for RAG**: Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, and `serper`, and inject the results directly into your chat experience.
- 🔍 **Web Search for RAG**: Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, `serper`, and `Serply` and inject the results directly into your chat experience.
- 🌐 **Web Browsing Capability**: Seamlessly integrate websites into your chat experience using the `#` command followed by a URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions.
@ -144,10 +150,19 @@ docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/wa
In the last part of the command, replace `open-webui` with your container name if it is different.
### Moving from Ollama WebUI to Open WebUI
Check our Migration Guide available in our [Open WebUI Documentation](https://docs.openwebui.com/migration/).
### Using the Dev Branch 🌙
> [!WARNING]
> The `:dev` branch contains the latest unstable features and changes. Use it at your own risk as it may have bugs or incomplete features.
If you want to try out the latest bleeding-edge features and are okay with occasional instability, you can use the `:dev` tag like this:
```bash
docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:dev
```
## What's Next? 🌟
Discover upcoming features on our roadmap in the [Open WebUI Documentation](https://docs.openwebui.com/roadmap/).

266
backend/apps/audio/main.py
View File

@ -17,13 +17,12 @@ from fastapi.middleware.cors import CORSMiddleware
from faster_whisper import WhisperModel
from pydantic import BaseModel
import uuid
import requests
import hashlib
from pathlib import Path
import json
from constants import ERROR_MESSAGES
from utils.utils import (
decode_token,
@ -41,10 +40,15 @@ from config import (
WHISPER_MODEL_DIR,
WHISPER_MODEL_AUTO_UPDATE,
DEVICE_TYPE,
AUDIO_OPENAI_API_BASE_URL,
AUDIO_OPENAI_API_KEY,
AUDIO_OPENAI_API_MODEL,
AUDIO_OPENAI_API_VOICE,
AUDIO_STT_OPENAI_API_BASE_URL,
AUDIO_STT_OPENAI_API_KEY,
AUDIO_TTS_OPENAI_API_BASE_URL,
AUDIO_TTS_OPENAI_API_KEY,
AUDIO_STT_ENGINE,
AUDIO_STT_MODEL,
AUDIO_TTS_ENGINE,
AUDIO_TTS_MODEL,
AUDIO_TTS_VOICE,
AppConfig,
)
@ -61,10 +65,17 @@ app.add_middleware(
)
app.state.config = AppConfig()
app.state.config.OPENAI_API_BASE_URL = AUDIO_OPENAI_API_BASE_URL
app.state.config.OPENAI_API_KEY = AUDIO_OPENAI_API_KEY
app.state.config.OPENAI_API_MODEL = AUDIO_OPENAI_API_MODEL
app.state.config.OPENAI_API_VOICE = AUDIO_OPENAI_API_VOICE
app.state.config.STT_OPENAI_API_BASE_URL = AUDIO_STT_OPENAI_API_BASE_URL
app.state.config.STT_OPENAI_API_KEY = AUDIO_STT_OPENAI_API_KEY
app.state.config.STT_ENGINE = AUDIO_STT_ENGINE
app.state.config.STT_MODEL = AUDIO_STT_MODEL
app.state.config.TTS_OPENAI_API_BASE_URL = AUDIO_TTS_OPENAI_API_BASE_URL
app.state.config.TTS_OPENAI_API_KEY = AUDIO_TTS_OPENAI_API_KEY
app.state.config.TTS_ENGINE = AUDIO_TTS_ENGINE
app.state.config.TTS_MODEL = AUDIO_TTS_MODEL
app.state.config.TTS_VOICE = AUDIO_TTS_VOICE
# setting device type for whisper model
whisper_device_type = DEVICE_TYPE if DEVICE_TYPE and DEVICE_TYPE == "cuda" else "cpu"
@ -74,41 +85,101 @@ SPEECH_CACHE_DIR = Path(CACHE_DIR).joinpath("./audio/speech/")
SPEECH_CACHE_DIR.mkdir(parents=True, exist_ok=True)
class OpenAIConfigUpdateForm(BaseModel):
url: str
key: str
model: str
speaker: str
class TTSConfigForm(BaseModel):
OPENAI_API_BASE_URL: str
OPENAI_API_KEY: str
ENGINE: str
MODEL: str
VOICE: str
class STTConfigForm(BaseModel):
OPENAI_API_BASE_URL: str
OPENAI_API_KEY: str
ENGINE: str
MODEL: str
class AudioConfigUpdateForm(BaseModel):
tts: TTSConfigForm
stt: STTConfigForm
from pydub import AudioSegment
from pydub.utils import mediainfo
def is_mp4_audio(file_path):
"""Check if the given file is an MP4 audio file."""
if not os.path.isfile(file_path):
print(f"File not found: {file_path}")
return False
info = mediainfo(file_path)
if (
info.get("codec_name") == "aac"
and info.get("codec_type") == "audio"
and info.get("codec_tag_string") == "mp4a"
):
return True
return False
def convert_mp4_to_wav(file_path, output_path):
"""Convert MP4 audio file to WAV format."""
audio = AudioSegment.from_file(file_path, format="mp4")
audio.export(output_path, format="wav")
print(f"Converted {file_path} to {output_path}")
@app.get("/config")
async def get_openai_config(user=Depends(get_admin_user)):
async def get_audio_config(user=Depends(get_admin_user)):
return {
"OPENAI_API_BASE_URL": app.state.config.OPENAI_API_BASE_URL,
"OPENAI_API_KEY": app.state.config.OPENAI_API_KEY,
"OPENAI_API_MODEL": app.state.config.OPENAI_API_MODEL,
"OPENAI_API_VOICE": app.state.config.OPENAI_API_VOICE,
"tts": {
"OPENAI_API_BASE_URL": app.state.config.TTS_OPENAI_API_BASE_URL,
"OPENAI_API_KEY": app.state.config.TTS_OPENAI_API_KEY,
"ENGINE": app.state.config.TTS_ENGINE,
"MODEL": app.state.config.TTS_MODEL,
"VOICE": app.state.config.TTS_VOICE,
},
"stt": {
"OPENAI_API_BASE_URL": app.state.config.STT_OPENAI_API_BASE_URL,
"OPENAI_API_KEY": app.state.config.STT_OPENAI_API_KEY,
"ENGINE": app.state.config.STT_ENGINE,
"MODEL": app.state.config.STT_MODEL,
},
}
@app.post("/config/update")
async def update_openai_config(
form_data: OpenAIConfigUpdateForm, user=Depends(get_admin_user)
async def update_audio_config(
form_data: AudioConfigUpdateForm, user=Depends(get_admin_user)
):
if form_data.key == "":
raise HTTPException(status_code=400, detail=ERROR_MESSAGES.API_KEY_NOT_FOUND)
app.state.config.TTS_OPENAI_API_BASE_URL = form_data.tts.OPENAI_API_BASE_URL
app.state.config.TTS_OPENAI_API_KEY = form_data.tts.OPENAI_API_KEY
app.state.config.TTS_ENGINE = form_data.tts.ENGINE
app.state.config.TTS_MODEL = form_data.tts.MODEL
app.state.config.TTS_VOICE = form_data.tts.VOICE
app.state.config.OPENAI_API_BASE_URL = form_data.url
app.state.config.OPENAI_API_KEY = form_data.key
app.state.config.OPENAI_API_MODEL = form_data.model
app.state.config.OPENAI_API_VOICE = form_data.speaker
app.state.config.STT_OPENAI_API_BASE_URL = form_data.stt.OPENAI_API_BASE_URL
app.state.config.STT_OPENAI_API_KEY = form_data.stt.OPENAI_API_KEY
app.state.config.STT_ENGINE = form_data.stt.ENGINE
app.state.config.STT_MODEL = form_data.stt.MODEL
return {
"status": True,
"OPENAI_API_BASE_URL": app.state.config.OPENAI_API_BASE_URL,
"OPENAI_API_KEY": app.state.config.OPENAI_API_KEY,
"OPENAI_API_MODEL": app.state.config.OPENAI_API_MODEL,
"OPENAI_API_VOICE": app.state.config.OPENAI_API_VOICE,
"tts": {
"OPENAI_API_BASE_URL": app.state.config.TTS_OPENAI_API_BASE_URL,
"OPENAI_API_KEY": app.state.config.TTS_OPENAI_API_KEY,
"ENGINE": app.state.config.TTS_ENGINE,
"MODEL": app.state.config.TTS_MODEL,
"VOICE": app.state.config.TTS_VOICE,
},
"stt": {
"OPENAI_API_BASE_URL": app.state.config.STT_OPENAI_API_BASE_URL,
"OPENAI_API_KEY": app.state.config.STT_OPENAI_API_KEY,
"ENGINE": app.state.config.STT_ENGINE,
"MODEL": app.state.config.STT_MODEL,
},
}
@ -125,13 +196,21 @@ async def speech(request: Request, user=Depends(get_verified_user)):
return FileResponse(file_path)
headers = {}
headers["Authorization"] = f"Bearer {app.state.config.OPENAI_API_KEY}"
headers["Authorization"] = f"Bearer {app.state.config.TTS_OPENAI_API_KEY}"
headers["Content-Type"] = "application/json"
try:
body = body.decode("utf-8")
body = json.loads(body)
body["model"] = app.state.config.TTS_MODEL
body = json.dumps(body).encode("utf-8")
except Exception as e:
pass
r = None
try:
r = requests.post(
url=f"{app.state.config.OPENAI_API_BASE_URL}/audio/speech",
url=f"{app.state.config.TTS_OPENAI_API_BASE_URL}/audio/speech",
data=body,
headers=headers,
stream=True,
@ -181,41 +260,110 @@ def transcribe(
)
try:
filename = file.filename
file_path = f"{UPLOAD_DIR}/{filename}"
ext = file.filename.split(".")[-1]
id = uuid.uuid4()
filename = f"{id}.{ext}"
file_dir = f"{CACHE_DIR}/audio/transcriptions"
os.makedirs(file_dir, exist_ok=True)
file_path = f"{file_dir}/{filename}"
print(filename)
contents = file.file.read()
with open(file_path, "wb") as f:
f.write(contents)
f.close()
whisper_kwargs = {
"model_size_or_path": WHISPER_MODEL,
"device": whisper_device_type,
"compute_type": "int8",
"download_root": WHISPER_MODEL_DIR,
"local_files_only": not WHISPER_MODEL_AUTO_UPDATE,
}
if app.state.config.STT_ENGINE == "":
whisper_kwargs = {
"model_size_or_path": WHISPER_MODEL,
"device": whisper_device_type,
"compute_type": "int8",
"download_root": WHISPER_MODEL_DIR,
"local_files_only": not WHISPER_MODEL_AUTO_UPDATE,
}
log.debug(f"whisper_kwargs: {whisper_kwargs}")
log.debug(f"whisper_kwargs: {whisper_kwargs}")
try:
model = WhisperModel(**whisper_kwargs)
except:
log.warning(
"WhisperModel initialization failed, attempting download with local_files_only=False"
try:
model = WhisperModel(**whisper_kwargs)
except:
log.warning(
"WhisperModel initialization failed, attempting download with local_files_only=False"
)
whisper_kwargs["local_files_only"] = False
model = WhisperModel(**whisper_kwargs)
segments, info = model.transcribe(file_path, beam_size=5)
log.info(
"Detected language '%s' with probability %f"
% (info.language, info.language_probability)
)
whisper_kwargs["local_files_only"] = False
model = WhisperModel(**whisper_kwargs)
segments, info = model.transcribe(file_path, beam_size=5)
log.info(
"Detected language '%s' with probability %f"
% (info.language, info.language_probability)
)
transcript = "".join([segment.text for segment in list(segments)])
transcript = "".join([segment.text for segment in list(segments)])
data = {"text": transcript.strip()}
return {"text": transcript.strip()}
# save the transcript to a json file
transcript_file = f"{file_dir}/{id}.json"
with open(transcript_file, "w") as f:
json.dump(data, f)
print(data)
return data
elif app.state.config.STT_ENGINE == "openai":
if is_mp4_audio(file_path):
print("is_mp4_audio")
os.rename(file_path, file_path.replace(".wav", ".mp4"))
# Convert MP4 audio file to WAV format
convert_mp4_to_wav(file_path.replace(".wav", ".mp4"), file_path)
headers = {"Authorization": f"Bearer {app.state.config.STT_OPENAI_API_KEY}"}
files = {"file": (filename, open(file_path, "rb"))}
data = {"model": "whisper-1"}
print(files, data)
r = None
try:
r = requests.post(
url=f"{app.state.config.STT_OPENAI_API_BASE_URL}/audio/transcriptions",
headers=headers,
files=files,
data=data,
)
r.raise_for_status()
data = r.json()
# save the transcript to a json file
transcript_file = f"{file_dir}/{id}.json"
with open(transcript_file, "w") as f:
json.dump(data, f)
print(data)
return data
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"External: {res['error']['message']}"
except:
error_detail = f"External: {e}"
raise HTTPException(
status_code=r.status_code if r != None else 500,
detail=error_detail,
)
except Exception as e:
log.exception(e)

602
backend/apps/ollama/main.py
View File

@ -29,6 +29,8 @@ import time
from urllib.parse import urlparse
from typing import Optional, List, Union
from starlette.background import BackgroundTask
from apps.webui.models.models import Models
from apps.webui.models.users import Users
from constants import ERROR_MESSAGES
@ -39,8 +41,6 @@ from utils.utils import (
get_admin_user,
)
from utils.models import get_model_id_from_custom_model_id
from config import (
SRC_LOG_LEVELS,
@ -75,9 +75,6 @@ app.state.config.OLLAMA_BASE_URLS = OLLAMA_BASE_URLS
app.state.MODELS = {}
REQUEST_POOL = []
# TODO: Implement a more intelligent load balancing mechanism for distributing requests among multiple backend instances.
# Current implementation uses a simple round-robin approach (random.choice). Consider incorporating algorithms like weighted round-robin,
# least connections, or least response time for better resource utilization and performance optimization.
@ -132,20 +129,10 @@ async def update_ollama_api_url(form_data: UrlUpdateForm, user=Depends(get_admin
return {"OLLAMA_BASE_URLS": app.state.config.OLLAMA_BASE_URLS}
@app.get("/cancel/{request_id}")
async def cancel_ollama_request(request_id: str, user=Depends(get_current_user)):
if user:
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
return True
else:
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
async def fetch_url(url):
timeout = aiohttp.ClientTimeout(total=5)
try:
async with aiohttp.ClientSession(timeout=timeout) as session:
async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session:
async with session.get(url) as response:
return await response.json()
except Exception as e:
@ -154,6 +141,45 @@ async def fetch_url(url):
return None
async def cleanup_response(
response: Optional[aiohttp.ClientResponse],
session: Optional[aiohttp.ClientSession],
):
if response:
response.close()
if session:
await session.close()
async def post_streaming_url(url: str, payload: str):
r = None
try:
session = aiohttp.ClientSession(trust_env=True)
r = await session.post(url, data=payload)
r.raise_for_status()
return StreamingResponse(
r.content,
status_code=r.status,
headers=dict(r.headers),
background=BackgroundTask(cleanup_response, response=r, session=session),
)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = await r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status if r else 500,
detail=error_detail,
)
def merge_models_lists(model_lists):
merged_models = {}
@ -246,54 +272,57 @@ async def get_ollama_tags(
@app.get("/api/version")
@app.get("/api/version/{url_idx}")
async def get_ollama_versions(url_idx: Optional[int] = None):
if app.state.config.ENABLE_OLLAMA_API:
if url_idx == None:
if url_idx == None:
# returns lowest version
tasks = [
fetch_url(f"{url}/api/version")
for url in app.state.config.OLLAMA_BASE_URLS
]
responses = await asyncio.gather(*tasks)
responses = list(filter(lambda x: x is not None, responses))
# returns lowest version
tasks = [
fetch_url(f"{url}/api/version") for url in app.state.config.OLLAMA_BASE_URLS
]
responses = await asyncio.gather(*tasks)
responses = list(filter(lambda x: x is not None, responses))
if len(responses) > 0:
lowest_version = min(
responses,
key=lambda x: tuple(
map(int, re.sub(r"^v|-.*", "", x["version"]).split("."))
),
)
if len(responses) > 0:
lowest_version = min(
responses,
key=lambda x: tuple(
map(int, re.sub(r"^v|-.*", "", x["version"]).split("."))
),
)
return {"version": lowest_version["version"]}
return {"version": lowest_version["version"]}
else:
raise HTTPException(
status_code=500,
detail=ERROR_MESSAGES.OLLAMA_NOT_FOUND,
)
else:
raise HTTPException(
status_code=500,
detail=ERROR_MESSAGES.OLLAMA_NOT_FOUND,
)
url = app.state.config.OLLAMA_BASE_URLS[url_idx]
r = None
try:
r = requests.request(method="GET", url=f"{url}/api/version")
r.raise_for_status()
return r.json()
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
else:
url = app.state.config.OLLAMA_BASE_URLS[url_idx]
r = None
try:
r = requests.request(method="GET", url=f"{url}/api/version")
r.raise_for_status()
return r.json()
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return {"version": False}
class ModelNameForm(BaseModel):
@ -313,65 +342,7 @@ async def pull_model(
# Admin should be able to pull models from any source
payload = {**form_data.model_dump(exclude_none=True), "insecure": True}
def get_request():
nonlocal url
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
yield json.dumps({"id": request_id, "done": False}) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
log.warning("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/api/pull",
data=json.dumps(payload),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return await post_streaming_url(f"{url}/api/pull", json.dumps(payload))
class PushModelForm(BaseModel):
@ -399,50 +370,9 @@ async def push_model(
url = app.state.config.OLLAMA_BASE_URLS[url_idx]
log.debug(f"url: {url}")
r = None
def get_request():
nonlocal url
nonlocal r
try:
def stream_content():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
r = requests.request(
method="POST",
url=f"{url}/api/push",
data=form_data.model_dump_json(exclude_none=True).encode(),
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return await post_streaming_url(
f"{url}/api/push", form_data.model_dump_json(exclude_none=True).encode()
)
class CreateModelForm(BaseModel):
@ -461,53 +391,9 @@ async def create_model(
url = app.state.config.OLLAMA_BASE_URLS[url_idx]
log.info(f"url: {url}")
r = None
def get_request():
nonlocal url
nonlocal r
try:
def stream_content():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
r = requests.request(
method="POST",
url=f"{url}/api/create",
data=form_data.model_dump_json(exclude_none=True).encode(),
stream=True,
)
r.raise_for_status()
log.debug(f"r: {r}")
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return await post_streaming_url(
f"{url}/api/create", form_data.model_dump_json(exclude_none=True).encode()
)
class CopyModelForm(BaseModel):
@ -797,66 +683,9 @@ async def generate_completion(
url = app.state.config.OLLAMA_BASE_URLS[url_idx]
log.info(f"url: {url}")
r = None
def get_request():
nonlocal form_data
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
if form_data.stream:
yield json.dumps({"id": request_id, "done": False}) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
log.warning("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/api/generate",
data=form_data.model_dump_json(exclude_none=True).encode(),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return await post_streaming_url(
f"{url}/api/generate", form_data.model_dump_json(exclude_none=True).encode()
)
class ChatMessage(BaseModel):
@ -897,7 +726,6 @@ async def generate_chat_completion(
model_info = Models.get_model_by_id(model_id)
if model_info:
print(model_info)
if model_info.base_model_id:
payload["model"] = model_info.base_model_id
@ -906,44 +734,77 @@ async def generate_chat_completion(
if model_info.params:
payload["options"] = {}
payload["options"]["mirostat"] = model_info.params.get("mirostat", None)
payload["options"]["mirostat_eta"] = model_info.params.get(
"mirostat_eta", None
)
payload["options"]["mirostat_tau"] = model_info.params.get(
"mirostat_tau", None
)
payload["options"]["num_ctx"] = model_info.params.get("num_ctx", None)
if model_info.params.get("mirostat", None):
payload["options"]["mirostat"] = model_info.params.get("mirostat", None)
payload["options"]["repeat_last_n"] = model_info.params.get(
"repeat_last_n", None
)
payload["options"]["repeat_penalty"] = model_info.params.get(
"frequency_penalty", None
)
if model_info.params.get("mirostat_eta", None):
payload["options"]["mirostat_eta"] = model_info.params.get(
"mirostat_eta", None
)
payload["options"]["temperature"] = model_info.params.get(
"temperature", None
)
payload["options"]["seed"] = model_info.params.get("seed", None)
if model_info.params.get("mirostat_tau", None):
payload["options"]["stop"] = (
[
bytes(stop, "utf-8").decode("unicode_escape")
for stop in model_info.params["stop"]
]
if model_info.params.get("stop", None)
else None
)
payload["options"]["mirostat_tau"] = model_info.params.get(
"mirostat_tau", None
)
payload["options"]["tfs_z"] = model_info.params.get("tfs_z", None)
if model_info.params.get("num_ctx", None):
payload["options"]["num_ctx"] = model_info.params.get("num_ctx", None)
payload["options"]["num_predict"] = model_info.params.get(
"max_tokens", None
)
payload["options"]["top_k"] = model_info.params.get("top_k", None)
if model_info.params.get("repeat_last_n", None):
payload["options"]["repeat_last_n"] = model_info.params.get(
"repeat_last_n", None
)
payload["options"]["top_p"] = model_info.params.get("top_p", None)
if model_info.params.get("frequency_penalty", None):
payload["options"]["repeat_penalty"] = model_info.params.get(
"frequency_penalty", None
)
if model_info.params.get("temperature", None) is not None:
payload["options"]["temperature"] = model_info.params.get(
"temperature", None
)
if model_info.params.get("seed", None):
payload["options"]["seed"] = model_info.params.get("seed", None)
if model_info.params.get("stop", None):
payload["options"]["stop"] = (
[
bytes(stop, "utf-8").decode("unicode_escape")
for stop in model_info.params["stop"]
]
if model_info.params.get("stop", None)
else None
)
if model_info.params.get("tfs_z", None):
payload["options"]["tfs_z"] = model_info.params.get("tfs_z", None)
if model_info.params.get("max_tokens", None):
payload["options"]["num_predict"] = model_info.params.get(
"max_tokens", None
)
if model_info.params.get("top_k", None):
payload["options"]["top_k"] = model_info.params.get("top_k", None)
if model_info.params.get("top_p", None):
payload["options"]["top_p"] = model_info.params.get("top_p", None)
if model_info.params.get("use_mmap", None):
payload["options"]["use_mmap"] = model_info.params.get("use_mmap", None)
if model_info.params.get("use_mlock", None):
payload["options"]["use_mlock"] = model_info.params.get(
"use_mlock", None
)
if model_info.params.get("num_thread", None):
payload["options"]["num_thread"] = model_info.params.get(
"num_thread", None
)
if model_info.params.get("system", None):
# Check if the payload already has a system message
@ -981,73 +842,18 @@ async def generate_chat_completion(
print(payload)
r = None
def get_request():
nonlocal payload
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
if payload.get("stream", None):
yield json.dumps({"id": request_id, "done": False}) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
log.warning("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/api/chat",
data=json.dumps(payload),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
log.exception(e)
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return await post_streaming_url(f"{url}/api/chat", json.dumps(payload))
# TODO: we should update this part once Ollama supports other types
class OpenAIChatMessageContent(BaseModel):
type: str
model_config = ConfigDict(extra="allow")
class OpenAIChatMessage(BaseModel):
role: str
content: str
content: Union[str, OpenAIChatMessageContent]
model_config = ConfigDict(extra="allow")
@ -1075,7 +881,6 @@ async def generate_openai_chat_completion(
model_info = Models.get_model_by_id(model_id)
if model_info:
print(model_info)
if model_info.base_model_id:
payload["model"] = model_info.base_model_id
@ -1132,68 +937,7 @@ async def generate_openai_chat_completion(
url = app.state.config.OLLAMA_BASE_URLS[url_idx]
log.info(f"url: {url}")
r = None
def get_request():
nonlocal payload
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
if payload.get("stream"):
yield json.dumps(
{"request_id": request_id, "done": False}
) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
log.warning("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/v1/chat/completions",
data=json.dumps(payload),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
return await post_streaming_url(f"{url}/v1/chat/completions", json.dumps(payload))
@app.get("/v1/models")
@ -1305,7 +1049,7 @@ async def download_file_stream(
timeout = aiohttp.ClientTimeout(total=600) # Set the timeout
async with aiohttp.ClientSession(timeout=timeout) as session:
async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session:
async with session.get(file_url, headers=headers) as response:
total_size = int(response.headers.get("content-length", 0)) + current_size
@ -1522,7 +1266,7 @@ async def deprecated_proxy(
if path == "generate":
data = json.loads(body.decode("utf-8"))
if not ("stream" in data and data["stream"] == False):
if data.get("stream", True):
yield json.dumps({"id": request_id, "done": False}) + "\n"
elif path == "chat":

310
backend/apps/openai/main.py
View File

@ -9,6 +9,7 @@ import json
import logging
from pydantic import BaseModel
from starlette.background import BackgroundTask
from apps.webui.models.models import Models
from apps.webui.models.users import Users
@ -185,7 +186,7 @@ async def fetch_url(url, key):
timeout = aiohttp.ClientTimeout(total=5)
try:
headers = {"Authorization": f"Bearer {key}"}
async with aiohttp.ClientSession(timeout=timeout) as session:
async with aiohttp.ClientSession(timeout=timeout, trust_env=True) as session:
async with session.get(url, headers=headers) as response:
return await response.json()
except Exception as e:
@ -194,6 +195,16 @@ async def fetch_url(url, key):
return None
async def cleanup_response(
response: Optional[aiohttp.ClientResponse],
session: Optional[aiohttp.ClientSession],
):
if response:
response.close()
if session:
await session.close()
def merge_models_lists(model_lists):
log.debug(f"merge_models_lists {model_lists}")
merged_list = []
@ -228,6 +239,27 @@ async def get_all_models(raw: bool = False):
) or not app.state.config.ENABLE_OPENAI_API:
models = {"data": []}
else:
# Check if API KEYS length is same than API URLS length
if len(app.state.config.OPENAI_API_KEYS) != len(
app.state.config.OPENAI_API_BASE_URLS
):
# if there are more keys than urls, remove the extra keys
if len(app.state.config.OPENAI_API_KEYS) > len(
app.state.config.OPENAI_API_BASE_URLS
):
app.state.config.OPENAI_API_KEYS = app.state.config.OPENAI_API_KEYS[
: len(app.state.config.OPENAI_API_BASE_URLS)
]
# if there are more urls than keys, add empty keys
else:
app.state.config.OPENAI_API_KEYS += [
""
for _ in range(
len(app.state.config.OPENAI_API_BASE_URLS)
- len(app.state.config.OPENAI_API_KEYS)
)
]
tasks = [
fetch_url(f"{url}/models", app.state.config.OPENAI_API_KEYS[idx])
for idx, url in enumerate(app.state.config.OPENAI_API_BASE_URLS)
@ -313,113 +345,155 @@ async def get_models(url_idx: Optional[int] = None, user=Depends(get_current_use
)
@app.post("/chat/completions")
@app.post("/chat/completions/{url_idx}")
async def generate_chat_completion(
form_data: dict,
url_idx: Optional[int] = None,
user=Depends(get_verified_user),
):
idx = 0
payload = {**form_data}
model_id = form_data.get("model")
model_info = Models.get_model_by_id(model_id)
if model_info:
if model_info.base_model_id:
payload["model"] = model_info.base_model_id
model_info.params = model_info.params.model_dump()
if model_info.params:
if model_info.params.get("temperature", None) is not None:
payload["temperature"] = float(model_info.params.get("temperature"))
if model_info.params.get("top_p", None):
payload["top_p"] = int(model_info.params.get("top_p", None))
if model_info.params.get("max_tokens", None):
payload["max_tokens"] = int(model_info.params.get("max_tokens", None))
if model_info.params.get("frequency_penalty", None):
payload["frequency_penalty"] = int(
model_info.params.get("frequency_penalty", None)
)
if model_info.params.get("seed", None):
payload["seed"] = model_info.params.get("seed", None)
if model_info.params.get("stop", None):
payload["stop"] = (
[
bytes(stop, "utf-8").decode("unicode_escape")
for stop in model_info.params["stop"]
]
if model_info.params.get("stop", None)
else None
)
if model_info.params.get("system", None):
# Check if the payload already has a system message
# If not, add a system message to the payload
if payload.get("messages"):
for message in payload["messages"]:
if message.get("role") == "system":
message["content"] = (
model_info.params.get("system", None) + message["content"]
)
break
else:
payload["messages"].insert(
0,
{
"role": "system",
"content": model_info.params.get("system", None),
},
)
else:
pass
model = app.state.MODELS[payload.get("model")]
idx = model["urlIdx"]
if "pipeline" in model and model.get("pipeline"):
payload["user"] = {"name": user.name, "id": user.id}
# Check if the model is "gpt-4-vision-preview" and set "max_tokens" to 4000
# This is a workaround until OpenAI fixes the issue with this model
if payload.get("model") == "gpt-4-vision-preview":
if "max_tokens" not in payload:
payload["max_tokens"] = 4000
log.debug("Modified payload:", payload)
# Convert the modified body back to JSON
payload = json.dumps(payload)
print(payload)
url = app.state.config.OPENAI_API_BASE_URLS[idx]
key = app.state.config.OPENAI_API_KEYS[idx]
print(payload)
headers = {}
headers["Authorization"] = f"Bearer {key}"
headers["Content-Type"] = "application/json"
r = None
session = None
streaming = False
try:
session = aiohttp.ClientSession(trust_env=True)
r = await session.request(
method="POST",
url=f"{url}/chat/completions",
data=payload,
headers=headers,
)
r.raise_for_status()
# Check if response is SSE
if "text/event-stream" in r.headers.get("Content-Type", ""):
streaming = True
return StreamingResponse(
r.content,
status_code=r.status,
headers=dict(r.headers),
background=BackgroundTask(
cleanup_response, response=r, session=session
),
)
else:
response_data = await r.json()
return response_data
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = await r.json()
print(res)
if "error" in res:
error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
except:
error_detail = f"External: {e}"
raise HTTPException(status_code=r.status if r else 500, detail=error_detail)
finally:
if not streaming and session:
if r:
r.close()
await session.close()
@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
idx = 0
body = await request.body()
# TODO: Remove below after gpt-4-vision fix from Open AI
# Try to decode the body of the request from bytes to a UTF-8 string (Require add max_token to fix gpt-4-vision)
payload = None
try:
if "chat/completions" in path:
body = body.decode("utf-8")
body = json.loads(body)
payload = {**body}
model_id = body.get("model")
model_info = Models.get_model_by_id(model_id)
if model_info:
print(model_info)
if model_info.base_model_id:
payload["model"] = model_info.base_model_id
model_info.params = model_info.params.model_dump()
if model_info.params:
if model_info.params.get("temperature", None):
payload["temperature"] = int(
model_info.params.get("temperature")
)
if model_info.params.get("top_p", None):
payload["top_p"] = int(model_info.params.get("top_p", None))
if model_info.params.get("max_tokens", None):
payload["max_tokens"] = int(
model_info.params.get("max_tokens", None)
)
if model_info.params.get("frequency_penalty", None):
payload["frequency_penalty"] = int(
model_info.params.get("frequency_penalty", None)
)
if model_info.params.get("seed", None):
payload["seed"] = model_info.params.get("seed", None)
if model_info.params.get("stop", None):
payload["stop"] = (
[
bytes(stop, "utf-8").decode("unicode_escape")
for stop in model_info.params["stop"]
]
if model_info.params.get("stop", None)
else None
)
if model_info.params.get("system", None):
# Check if the payload already has a system message
# If not, add a system message to the payload
if payload.get("messages"):
for message in payload["messages"]:
if message.get("role") == "system":
message["content"] = (
model_info.params.get("system", None)
+ message["content"]
)
break
else:
payload["messages"].insert(
0,
{
"role": "system",
"content": model_info.params.get("system", None),
},
)
else:
pass
model = app.state.MODELS[payload.get("model")]
idx = model["urlIdx"]
if "pipeline" in model and model.get("pipeline"):
payload["user"] = {"name": user.name, "id": user.id}
payload["title"] = (
True
if payload["stream"] == False and payload["max_tokens"] == 50
else False
)
# Check if the model is "gpt-4-vision-preview" and set "max_tokens" to 4000
# This is a workaround until OpenAI fixes the issue with this model
if payload.get("model") == "gpt-4-vision-preview":
if "max_tokens" not in payload:
payload["max_tokens"] = 4000
log.debug("Modified payload:", payload)
# Convert the modified body back to JSON
payload = json.dumps(payload)
except json.JSONDecodeError as e:
log.error("Error loading request body into a dictionary:", e)
print(payload)
url = app.state.config.OPENAI_API_BASE_URLS[idx]
key = app.state.config.OPENAI_API_KEYS[idx]
@ -431,40 +505,48 @@ async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
headers["Content-Type"] = "application/json"
r = None
session = None
streaming = False
try:
r = requests.request(
session = aiohttp.ClientSession(trust_env=True)
r = await session.request(
method=request.method,
url=target_url,
data=payload if payload else body,
data=body,
headers=headers,
stream=True,
)
r.raise_for_status()
# Check if response is SSE
if "text/event-stream" in r.headers.get("Content-Type", ""):
streaming = True
return StreamingResponse(
r.iter_content(chunk_size=8192),
status_code=r.status_code,
r.content,
status_code=r.status,
headers=dict(r.headers),
background=BackgroundTask(
cleanup_response, response=r, session=session
),
)
else:
response_data = r.json()
response_data = await r.json()
return response_data
except Exception as e:
log.exception(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
res = await r.json()
print(res)
if "error" in res:
error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
except:
error_detail = f"External: {e}"
raise HTTPException(
status_code=r.status_code if r else 500, detail=error_detail
)
raise HTTPException(status_code=r.status if r else 500, detail=error_detail)
finally:
if not streaming and session:
if r:
r.close()
await session.close()

341
backend/apps/rag/main.py
View File

@ -8,12 +8,15 @@ from fastapi import (
Form,
)
from fastapi.middleware.cors import CORSMiddleware
import requests
import os, shutil, logging, re
from datetime import datetime
from pathlib import Path
from typing import List, Union, Sequence
from typing import List, Union, Sequence, Iterator, Any
from chromadb.utils.batch_utils import create_batches
from langchain_core.documents import Document
from langchain_community.document_loaders import (
WebBaseLoader,
@ -30,6 +33,7 @@ from langchain_community.document_loaders import (
UnstructuredExcelLoader,
UnstructuredPowerPointLoader,
YoutubeLoader,
OutlookMessageLoader,
)
from langchain.text_splitter import RecursiveCharacterTextSplitter
@ -59,9 +63,17 @@ from apps.rag.utils import (
query_doc_with_hybrid_search,
query_collection,
query_collection_with_hybrid_search,
search_web,
)
from apps.rag.search.brave import search_brave
from apps.rag.search.google_pse import search_google_pse
from apps.rag.search.main import SearchResult
from apps.rag.search.searxng import search_searxng
from apps.rag.search.serper import search_serper
from apps.rag.search.serpstack import search_serpstack
from apps.rag.search.serply import search_serply
from apps.rag.search.duckduckgo import search_duckduckgo
from utils.misc import (
calculate_sha256,
calculate_sha256_string,
@ -71,6 +83,7 @@ from utils.misc import (
from utils.utils import get_current_user, get_admin_user
from config import (
AppConfig,
ENV,
SRC_LOG_LEVELS,
UPLOAD_DIR,
@ -96,8 +109,19 @@ from config import (
RAG_TEMPLATE,
ENABLE_RAG_LOCAL_WEB_FETCH,
YOUTUBE_LOADER_LANGUAGE,
ENABLE_RAG_WEB_SEARCH,
RAG_WEB_SEARCH_ENGINE,
SEARXNG_QUERY_URL,
GOOGLE_PSE_API_KEY,
GOOGLE_PSE_ENGINE_ID,
BRAVE_SEARCH_API_KEY,
SERPSTACK_API_KEY,
SERPSTACK_HTTPS,
SERPER_API_KEY,
SERPLY_API_KEY,
RAG_WEB_SEARCH_RESULT_COUNT,
RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
AppConfig,
RAG_EMBEDDING_OPENAI_BATCH_SIZE,
)
from constants import ERROR_MESSAGES
@ -122,6 +146,7 @@ app.state.config.CHUNK_OVERLAP = CHUNK_OVERLAP
app.state.config.RAG_EMBEDDING_ENGINE = RAG_EMBEDDING_ENGINE
app.state.config.RAG_EMBEDDING_MODEL = RAG_EMBEDDING_MODEL
app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE = RAG_EMBEDDING_OPENAI_BATCH_SIZE
app.state.config.RAG_RERANKING_MODEL = RAG_RERANKING_MODEL
app.state.config.RAG_TEMPLATE = RAG_TEMPLATE
@ -136,6 +161,21 @@ app.state.config.YOUTUBE_LOADER_LANGUAGE = YOUTUBE_LOADER_LANGUAGE
app.state.YOUTUBE_LOADER_TRANSLATION = None
app.state.config.ENABLE_RAG_WEB_SEARCH = ENABLE_RAG_WEB_SEARCH
app.state.config.RAG_WEB_SEARCH_ENGINE = RAG_WEB_SEARCH_ENGINE
app.state.config.SEARXNG_QUERY_URL = SEARXNG_QUERY_URL
app.state.config.GOOGLE_PSE_API_KEY = GOOGLE_PSE_API_KEY
app.state.config.GOOGLE_PSE_ENGINE_ID = GOOGLE_PSE_ENGINE_ID
app.state.config.BRAVE_SEARCH_API_KEY = BRAVE_SEARCH_API_KEY
app.state.config.SERPSTACK_API_KEY = SERPSTACK_API_KEY
app.state.config.SERPSTACK_HTTPS = SERPSTACK_HTTPS
app.state.config.SERPER_API_KEY = SERPER_API_KEY
app.state.config.SERPLY_API_KEY = SERPLY_API_KEY
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = RAG_WEB_SEARCH_RESULT_COUNT
app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = RAG_WEB_SEARCH_CONCURRENT_REQUESTS
def update_embedding_model(
embedding_model: str,
update_model: bool = False,
@ -181,6 +221,7 @@ app.state.EMBEDDING_FUNCTION = get_embedding_function(
app.state.sentence_transformer_ef,
app.state.config.OPENAI_API_KEY,
app.state.config.OPENAI_API_BASE_URL,
app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
)
origins = ["*"]
@ -217,6 +258,7 @@ async def get_status():
"embedding_engine": app.state.config.RAG_EMBEDDING_ENGINE,
"embedding_model": app.state.config.RAG_EMBEDDING_MODEL,
"reranking_model": app.state.config.RAG_RERANKING_MODEL,
"openai_batch_size": app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
}
@ -229,6 +271,7 @@ async def get_embedding_config(user=Depends(get_admin_user)):
"openai_config": {
"url": app.state.config.OPENAI_API_BASE_URL,
"key": app.state.config.OPENAI_API_KEY,
"batch_size": app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
},
}
@ -244,6 +287,7 @@ async def get_reraanking_config(user=Depends(get_admin_user)):
class OpenAIConfigForm(BaseModel):
url: str
key: str
batch_size: Optional[int] = None
class EmbeddingModelUpdateForm(BaseModel):
@ -264,9 +308,14 @@ async def update_embedding_config(
app.state.config.RAG_EMBEDDING_MODEL = form_data.embedding_model
if app.state.config.RAG_EMBEDDING_ENGINE in ["ollama", "openai"]:
if form_data.openai_config != None:
if form_data.openai_config is not None:
app.state.config.OPENAI_API_BASE_URL = form_data.openai_config.url
app.state.config.OPENAI_API_KEY = form_data.openai_config.key
app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE = (
form_data.openai_config.batch_size
if form_data.openai_config.batch_size
else 1
)
update_embedding_model(app.state.config.RAG_EMBEDDING_MODEL)
@ -276,6 +325,7 @@ async def update_embedding_config(
app.state.sentence_transformer_ef,
app.state.config.OPENAI_API_KEY,
app.state.config.OPENAI_API_BASE_URL,
app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
)
return {
@ -285,6 +335,7 @@ async def update_embedding_config(
"openai_config": {
"url": app.state.config.OPENAI_API_BASE_URL,
"key": app.state.config.OPENAI_API_KEY,
"batch_size": app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
},
}
except Exception as e:
@ -332,11 +383,27 @@ async def get_rag_config(user=Depends(get_admin_user)):
"chunk_size": app.state.config.CHUNK_SIZE,
"chunk_overlap": app.state.config.CHUNK_OVERLAP,
},
"web_loader_ssl_verification": app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
"youtube": {
"language": app.state.config.YOUTUBE_LOADER_LANGUAGE,
"translation": app.state.YOUTUBE_LOADER_TRANSLATION,
},
"web": {
"ssl_verification": app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
"search": {
"enabled": app.state.config.ENABLE_RAG_WEB_SEARCH,
"engine": app.state.config.RAG_WEB_SEARCH_ENGINE,
"searxng_query_url": app.state.config.SEARXNG_QUERY_URL,
"google_pse_api_key": app.state.config.GOOGLE_PSE_API_KEY,
"google_pse_engine_id": app.state.config.GOOGLE_PSE_ENGINE_ID,
"brave_search_api_key": app.state.config.BRAVE_SEARCH_API_KEY,
"serpstack_api_key": app.state.config.SERPSTACK_API_KEY,
"serpstack_https": app.state.config.SERPSTACK_HTTPS,
"serper_api_key": app.state.config.SERPER_API_KEY,
"serply_api_key": app.state.config.SERPLY_API_KEY,
"result_count": app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
"concurrent_requests": app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
},
},
}
@ -350,11 +417,31 @@ class YoutubeLoaderConfig(BaseModel):
translation: Optional[str] = None
class WebSearchConfig(BaseModel):
enabled: bool
engine: Optional[str] = None
searxng_query_url: Optional[str] = None
google_pse_api_key: Optional[str] = None
google_pse_engine_id: Optional[str] = None
brave_search_api_key: Optional[str] = None
serpstack_api_key: Optional[str] = None
serpstack_https: Optional[bool] = None
serper_api_key: Optional[str] = None
serply_api_key: Optional[str] = None
result_count: Optional[int] = None
concurrent_requests: Optional[int] = None
class WebConfig(BaseModel):
search: WebSearchConfig
web_loader_ssl_verification: Optional[bool] = None
class ConfigUpdateForm(BaseModel):
pdf_extract_images: Optional[bool] = None
chunk: Optional[ChunkParamUpdateForm] = None
web_loader_ssl_verification: Optional[bool] = None
youtube: Optional[YoutubeLoaderConfig] = None
web: Optional[WebConfig] = None
@app.post("/config/update")
@ -365,35 +452,37 @@ async def update_rag_config(form_data: ConfigUpdateForm, user=Depends(get_admin_
else app.state.config.PDF_EXTRACT_IMAGES
)
app.state.config.CHUNK_SIZE = (
form_data.chunk.chunk_size
if form_data.chunk is not None
else app.state.config.CHUNK_SIZE
)
if form_data.chunk is not None:
app.state.config.CHUNK_SIZE = form_data.chunk.chunk_size
app.state.config.CHUNK_OVERLAP = form_data.chunk.chunk_overlap
app.state.config.CHUNK_OVERLAP = (
form_data.chunk.chunk_overlap
if form_data.chunk is not None
else app.state.config.CHUNK_OVERLAP
)
if form_data.youtube is not None:
app.state.config.YOUTUBE_LOADER_LANGUAGE = form_data.youtube.language
app.state.YOUTUBE_LOADER_TRANSLATION = form_data.youtube.translation
app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION = (
form_data.web_loader_ssl_verification
if form_data.web_loader_ssl_verification != None
else app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION
)
if form_data.web is not None:
app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION = (
form_data.web.web_loader_ssl_verification
)
app.state.config.YOUTUBE_LOADER_LANGUAGE = (
form_data.youtube.language
if form_data.youtube is not None
else app.state.config.YOUTUBE_LOADER_LANGUAGE
)
app.state.YOUTUBE_LOADER_TRANSLATION = (
form_data.youtube.translation
if form_data.youtube is not None
else app.state.YOUTUBE_LOADER_TRANSLATION
)
app.state.config.ENABLE_RAG_WEB_SEARCH = form_data.web.search.enabled
app.state.config.RAG_WEB_SEARCH_ENGINE = form_data.web.search.engine
app.state.config.SEARXNG_QUERY_URL = form_data.web.search.searxng_query_url
app.state.config.GOOGLE_PSE_API_KEY = form_data.web.search.google_pse_api_key
app.state.config.GOOGLE_PSE_ENGINE_ID = (
form_data.web.search.google_pse_engine_id
)
app.state.config.BRAVE_SEARCH_API_KEY = (
form_data.web.search.brave_search_api_key
)
app.state.config.SERPSTACK_API_KEY = form_data.web.search.serpstack_api_key
app.state.config.SERPSTACK_HTTPS = form_data.web.search.serpstack_https
app.state.config.SERPER_API_KEY = form_data.web.search.serper_api_key
app.state.config.SERPLY_API_KEY = form_data.web.search.serply_api_key
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = form_data.web.search.result_count
app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = (
form_data.web.search.concurrent_requests
)
return {
"status": True,
@ -402,11 +491,27 @@ async def update_rag_config(form_data: ConfigUpdateForm, user=Depends(get_admin_
"chunk_size": app.state.config.CHUNK_SIZE,
"chunk_overlap": app.state.config.CHUNK_OVERLAP,
},
"web_loader_ssl_verification": app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
"youtube": {
"language": app.state.config.YOUTUBE_LOADER_LANGUAGE,
"translation": app.state.YOUTUBE_LOADER_TRANSLATION,
},
"web": {
"ssl_verification": app.state.config.ENABLE_RAG_WEB_LOADER_SSL_VERIFICATION,
"search": {
"enabled": app.state.config.ENABLE_RAG_WEB_SEARCH,
"engine": app.state.config.RAG_WEB_SEARCH_ENGINE,
"searxng_query_url": app.state.config.SEARXNG_QUERY_URL,
"google_pse_api_key": app.state.config.GOOGLE_PSE_API_KEY,
"google_pse_engine_id": app.state.config.GOOGLE_PSE_ENGINE_ID,
"brave_search_api_key": app.state.config.BRAVE_SEARCH_API_KEY,
"serpstack_api_key": app.state.config.SERPSTACK_API_KEY,
"serpstack_https": app.state.config.SERPSTACK_HTTPS,
"serper_api_key": app.state.config.SERPER_API_KEY,
"serply_api_key": app.state.config.SERPLY_API_KEY,
"result_count": app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
"concurrent_requests": app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
},
},
}
@ -599,7 +704,7 @@ def get_web_loader(url: Union[str, Sequence[str]], verify_ssl: bool = True):
# Check if the URL is valid
if not validate_url(url):
raise ValueError(ERROR_MESSAGES.INVALID_URL)
return WebBaseLoader(
return SafeWebBaseLoader(
url,
verify_ssl=verify_ssl,
requests_per_second=RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
@ -642,17 +747,107 @@ def resolve_hostname(hostname):
return ipv4_addresses, ipv6_addresses
def search_web(engine: str, query: str) -> list[SearchResult]:
"""Search the web using a search engine and return the results as a list of SearchResult objects.
Will look for a search engine API key in environment variables in the following order:
- SEARXNG_QUERY_URL
- GOOGLE_PSE_API_KEY + GOOGLE_PSE_ENGINE_ID
- BRAVE_SEARCH_API_KEY
- SERPSTACK_API_KEY
- SERPER_API_KEY
- SERPLY_API_KEY
Args:
query (str): The query to search for
"""
# TODO: add playwright to search the web
if engine == "searxng":
if app.state.config.SEARXNG_QUERY_URL:
return search_searxng(
app.state.config.SEARXNG_QUERY_URL,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
)
else:
raise Exception("No SEARXNG_QUERY_URL found in environment variables")
elif engine == "google_pse":
if (
app.state.config.GOOGLE_PSE_API_KEY
and app.state.config.GOOGLE_PSE_ENGINE_ID
):
return search_google_pse(
app.state.config.GOOGLE_PSE_API_KEY,
app.state.config.GOOGLE_PSE_ENGINE_ID,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
)
else:
raise Exception(
"No GOOGLE_PSE_API_KEY or GOOGLE_PSE_ENGINE_ID found in environment variables"
)
elif engine == "brave":
if app.state.config.BRAVE_SEARCH_API_KEY:
return search_brave(
app.state.config.BRAVE_SEARCH_API_KEY,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
)
else:
raise Exception("No BRAVE_SEARCH_API_KEY found in environment variables")
elif engine == "serpstack":
if app.state.config.SERPSTACK_API_KEY:
return search_serpstack(
app.state.config.SERPSTACK_API_KEY,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
https_enabled=app.state.config.SERPSTACK_HTTPS,
)
else:
raise Exception("No SERPSTACK_API_KEY found in environment variables")
elif engine == "serper":
if app.state.config.SERPER_API_KEY:
return search_serper(
app.state.config.SERPER_API_KEY,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
)
else:
raise Exception("No SERPER_API_KEY found in environment variables")
elif engine == "serply":
if app.state.config.SERPLY_API_KEY:
return search_serply(
app.state.config.SERPLY_API_KEY,
query,
app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
)
else:
raise Exception("No SERPLY_API_KEY found in environment variables")
elif engine == "duckduckgo":
return search_duckduckgo(query, app.state.config.RAG_WEB_SEARCH_RESULT_COUNT)
else:
raise Exception("No search engine API key found in environment variables")
@app.post("/web/search")
def store_web_search(form_data: SearchForm, user=Depends(get_current_user)):
try:
try:
web_results = search_web(form_data.query)
except Exception as e:
log.exception(e)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.WEB_SEARCH_ERROR,
)
logging.info(
f"trying to web search with {app.state.config.RAG_WEB_SEARCH_ENGINE, form_data.query}"
)
web_results = search_web(
app.state.config.RAG_WEB_SEARCH_ENGINE, form_data.query
)
except Exception as e:
log.exception(e)
print(e)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.WEB_SEARCH_ERROR(e),
)
try:
urls = [result.link for result in web_results]
loader = get_web_loader(urls)
data = loader.load()
@ -710,6 +905,13 @@ def store_docs_in_vector_db(docs, collection_name, overwrite: bool = False) -> b
texts = [doc.page_content for doc in docs]
metadatas = [doc.metadata for doc in docs]
# ChromaDB does not like datetime formats
# for meta-data so convert them to string.
for metadata in metadatas:
for key, value in metadata.items():
if isinstance(value, datetime):
metadata[key] = str(value)
try:
if overwrite:
for collection in CHROMA_CLIENT.list_collections():
@ -725,6 +927,7 @@ def store_docs_in_vector_db(docs, collection_name, overwrite: bool = False) -> b
app.state.sentence_transformer_ef,
app.state.config.OPENAI_API_KEY,
app.state.config.OPENAI_API_BASE_URL,
app.state.config.RAG_EMBEDDING_OPENAI_BATCH_SIZE,
)
embedding_texts = list(map(lambda x: x.replace("\n", " "), texts))
@ -795,6 +998,7 @@ def get_loader(filename: str, file_content_type: str, file_path: str):
"swift",
"vue",
"svelte",
"msg",
]
if file_ext == "pdf":
@ -829,6 +1033,8 @@ def get_loader(filename: str, file_content_type: str, file_path: str):
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
] or file_ext in ["ppt", "pptx"]:
loader = UnstructuredPowerPointLoader(file_path)
elif file_ext == "msg":
loader = OutlookMessageLoader(file_path)
elif file_ext in known_source_ext or (
file_content_type and file_content_type.find("text/") >= 0
):
@ -994,6 +1200,30 @@ def reset_vector_db(user=Depends(get_admin_user)):
CHROMA_CLIENT.reset()
@app.get("/reset/uploads")
def reset_upload_dir(user=Depends(get_admin_user)) -> bool:
folder = f"{UPLOAD_DIR}"
try:
# Check if the directory exists
if os.path.exists(folder):
# Iterate over all the files and directories in the specified directory
for filename in os.listdir(folder):
file_path = os.path.join(folder, filename)
try:
if os.path.isfile(file_path) or os.path.islink(file_path):
os.unlink(file_path) # Remove the file or link
elif os.path.isdir(file_path):
shutil.rmtree(file_path) # Remove the directory
except Exception as e:
print(f"Failed to delete {file_path}. Reason: {e}")
else:
print(f"The directory {folder} does not exist")
except Exception as e:
print(f"Failed to process the directory {folder}. Reason: {e}")
return True
@app.get("/reset")
def reset(user=Depends(get_admin_user)) -> bool:
folder = f"{UPLOAD_DIR}"
@ -1015,6 +1245,33 @@ def reset(user=Depends(get_admin_user)) -> bool:
return True
class SafeWebBaseLoader(WebBaseLoader):
"""WebBaseLoader with enhanced error handling for URLs."""
def lazy_load(self) -> Iterator[Document]:
"""Lazy load text from the url(s) in web_path with error handling."""
for path in self.web_paths:
try:
soup = self._scrape(path, bs_kwargs=self.bs_kwargs)
text = soup.get_text(**self.bs_get_text_kwargs)
# Build metadata
metadata = {"source": path}
if title := soup.find("title"):
metadata["title"] = title.get_text()
if description := soup.find("meta", attrs={"name": "description"}):
metadata["description"] = description.get(
"content", "No description found."
)
if html := soup.find("html"):
metadata["language"] = html.get("lang", "No language found.")
yield Document(page_content=text, metadata=metadata)
except Exception as e:
# Log the error and continue with the next URL
log.error(f"Error loading {path}: {e}")
if ENV == "dev":
@app.get("/ef")

8
backend/apps/rag/search/brave.py
View File

@ -3,13 +3,13 @@ import logging
import requests
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS, RAG_WEB_SEARCH_RESULT_COUNT
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_brave(api_key: str, query: str) -> list[SearchResult]:
def search_brave(api_key: str, query: str, count: int) -> list[SearchResult]:
"""Search using Brave's Search API and return the results as a list of SearchResult objects.
Args:
@ -22,7 +22,7 @@ def search_brave(api_key: str, query: str) -> list[SearchResult]:
"Accept-Encoding": "gzip",
"X-Subscription-Token": api_key,
}
params = {"q": query, "count": RAG_WEB_SEARCH_RESULT_COUNT}
params = {"q": query, "count": count}
response = requests.get(url, headers=headers, params=params)
response.raise_for_status()
@ -33,5 +33,5 @@ def search_brave(api_key: str, query: str) -> list[SearchResult]:
SearchResult(
link=result["url"], title=result.get("title"), snippet=result.get("snippet")
)
for result in results[:RAG_WEB_SEARCH_RESULT_COUNT]
for result in results[:count]
]

46
backend/apps/rag/search/duckduckgo.py Normal file
View File

@ -0,0 +1,46 @@
import logging
from apps.rag.search.main import SearchResult
from duckduckgo_search import DDGS
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_duckduckgo(query: str, count: int) -> list[SearchResult]:
"""
Search using DuckDuckGo's Search API and return the results as a list of SearchResult objects.
Args:
query (str): The query to search for
count (int): The number of results to return
Returns:
List[SearchResult]: A list of search results
"""
# Use the DDGS context manager to create a DDGS object
with DDGS() as ddgs:
# Use the ddgs.text() method to perform the search
ddgs_gen = ddgs.text(
query, safesearch="moderate", max_results=count, backend="api"
)
# Check if there are search results
if ddgs_gen:
# Convert the search results into a list
search_results = [r for r in ddgs_gen]
# Create an empty list to store the SearchResult objects
results = []
# Iterate over each search result
for result in search_results:
# Create a SearchResult object and append it to the results list
results.append(
SearchResult(
link=result["href"],
title=result.get("title"),
snippet=result.get("body"),
)
)
print(results)
# Return the list of search results
return results

6
backend/apps/rag/search/google_pse.py
View File

@ -4,14 +4,14 @@ import logging
import requests
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS, RAG_WEB_SEARCH_RESULT_COUNT
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_google_pse(
api_key: str, search_engine_id: str, query: str
api_key: str, search_engine_id: str, query: str, count: int
) -> list[SearchResult]:
"""Search using Google's Programmable Search Engine API and return the results as a list of SearchResult objects.
@ -27,7 +27,7 @@ def search_google_pse(
"cx": search_engine_id,
"q": query,
"key": api_key,
"num": RAG_WEB_SEARCH_RESULT_COUNT,
"num": count,
}
response = requests.request("GET", url, headers=headers, params=params)

74
backend/apps/rag/search/searxng.py
View File

@ -1,28 +1,68 @@
import logging
import requests
from typing import List
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS, RAG_WEB_SEARCH_RESULT_COUNT
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_searxng(query_url: str, query: str) -> list[SearchResult]:
"""Search a SearXNG instance for a query and return the results as a list of SearchResult objects.
def search_searxng(
query_url: str, query: str, count: int, **kwargs
) -> List[SearchResult]:
"""
Search a SearXNG instance for a given query and return the results as a list of SearchResult objects.
The function allows passing additional parameters such as language or time_range to tailor the search result.
Args:
query_url (str): The URL of the SearXNG instance to search. Must contain "<query>" as a placeholder
query (str): The query to search for
"""
url = query_url.replace("<query>", query)
if "&format=json" not in url:
url += "&format=json"
log.debug(f"searching {url}")
query_url (str): The base URL of the SearXNG server.
query (str): The search term or question to find in the SearXNG database.
count (int): The maximum number of results to retrieve from the search.
r = requests.get(
url,
Keyword Args:
language (str): Language filter for the search results; e.g., "en-US". Defaults to an empty string.
safesearch (int): Safe search filter for safer web results; 0 = off, 1 = moderate, 2 = strict. Defaults to 1 (moderate).
time_range (str): Time range for filtering results by date; e.g., "2023-04-05..today" or "all-time". Defaults to ''.
categories: (Optional[List[str]]): Specific categories within which the search should be performed, defaulting to an empty string if not provided.
Returns:
List[SearchResult]: A list of SearchResults sorted by relevance score in descending order.
Raise:
requests.exceptions.RequestException: If a request error occurs during the search process.
"""
# Default values for optional parameters are provided as empty strings or None when not specified.
language = kwargs.get("language", "en-US")
safesearch = kwargs.get("safesearch", "1")
time_range = kwargs.get("time_range", "")
categories = "".join(kwargs.get("categories", []))
params = {
"q": query,
"format": "json",
"pageno": 1,
"safesearch": safesearch,
"language": language,
"time_range": time_range,
"categories": categories,
"theme": "simple",
"image_proxy": 0,
}
# Legacy query format
if "<query>" in query_url:
# Strip all query parameters from the URL
query_url = query_url.split("?")[0]
log.debug(f"searching {query_url}")
response = requests.get(
query_url,
headers={
"User-Agent": "Open WebUI (https://github.com/open-webui/open-webui) RAG Bot",
"Accept": "text/html",
@ -30,15 +70,17 @@ def search_searxng(query_url: str, query: str) -> list[SearchResult]:
"Accept-Language": "en-US,en;q=0.5",
"Connection": "keep-alive",
},
params=params,
)
r.raise_for_status()
json_response = r.json()
response.raise_for_status() # Raise an exception for HTTP errors.
json_response = response.json()
results = json_response.get("results", [])
sorted_results = sorted(results, key=lambda x: x.get("score", 0), reverse=True)
return [
SearchResult(
link=result["url"], title=result.get("title"), snippet=result.get("content")
)
for result in sorted_results[:RAG_WEB_SEARCH_RESULT_COUNT]
for result in sorted_results[:count]
]

6
backend/apps/rag/search/serper.py
View File

@ -4,13 +4,13 @@ import logging
import requests
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS, RAG_WEB_SEARCH_RESULT_COUNT
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_serper(api_key: str, query: str) -> list[SearchResult]:
def search_serper(api_key: str, query: str, count: int) -> list[SearchResult]:
"""Search using serper.dev's API and return the results as a list of SearchResult objects.
Args:
@ -35,5 +35,5 @@ def search_serper(api_key: str, query: str) -> list[SearchResult]:
title=result.get("title"),
snippet=result.get("description"),
)
for result in results[:RAG_WEB_SEARCH_RESULT_COUNT]
for result in results[:count]
]

68
backend/apps/rag/search/serply.py Normal file
View File

@ -0,0 +1,68 @@
import json
import logging
import requests
from urllib.parse import urlencode
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_serply(
api_key: str,
query: str,
count: int,
hl: str = "us",
limit: int = 10,
device_type: str = "desktop",
proxy_location: str = "US",
) -> list[SearchResult]:
"""Search using serper.dev's API and return the results as a list of SearchResult objects.
Args:
api_key (str): A serply.io API key
query (str): The query to search for
hl (str): Host Language code to display results in (reference https://developers.google.com/custom-search/docs/xml_results?hl=en#wsInterfaceLanguages)
limit (int): The maximum number of results to return [10-100, defaults to 10]
"""
log.info("Searching with Serply")
url = "https://api.serply.io/v1/search/"
query_payload = {
"q": query,
"language": "en",
"num": limit,
"gl": proxy_location.upper(),
"hl": hl.lower(),
}
url = f"{url}{urlencode(query_payload)}"
headers = {
"X-API-KEY": api_key,
"X-User-Agent": device_type,
"User-Agent": "open-webui",
"X-Proxy-Location": proxy_location,
}
response = requests.request("GET", url, headers=headers)
response.raise_for_status()
json_response = response.json()
log.info(f"results from serply search: {json_response}")
results = sorted(
json_response.get("results", []), key=lambda x: x.get("realPosition", 0)
)
return [
SearchResult(
link=result["link"],
title=result.get("title"),
snippet=result.get("description"),
)
for result in results[:count]
]

6
backend/apps/rag/search/serpstack.py
View File

@ -4,14 +4,14 @@ import logging
import requests
from apps.rag.search.main import SearchResult
from config import SRC_LOG_LEVELS, RAG_WEB_SEARCH_RESULT_COUNT
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
def search_serpstack(
api_key: str, query: str, https_enabled: bool = True
api_key: str, query: str, count: int, https_enabled: bool = True
) -> list[SearchResult]:
"""Search using serpstack.com's and return the results as a list of SearchResult objects.
@ -39,5 +39,5 @@ def search_serpstack(
SearchResult(
link=result["url"], title=result.get("title"), snippet=result.get("snippet")
)
for result in results[:RAG_WEB_SEARCH_RESULT_COUNT]
for result in results[:count]
]

206
backend/apps/rag/search/testdata/serply.json vendored Normal file
View File

@ -0,0 +1,206 @@
{
"ads": [],
"ads_count": 0,
"answers": [],
"results": [
{
"title": "Apple",
"link": "https://www.apple.com/",
"description": "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, ...",
"additional_links": [
{
"text": "AppleApplehttps://www.apple.com",
"href": "https://www.apple.com/"
}
],
"cite": {},
"subdomains": [
{
"title": "Support",
"link": "https://support.apple.com/",
"description": "SupportContact - iPhone Support - Billing and Subscriptions - Apple Repair"
},
{
"title": "Store",
"link": "https://www.apple.com/store",
"description": "StoreShop iPhone - Shop iPad - App Store - Shop Mac - ..."
},
{
"title": "Mac",
"link": "https://www.apple.com/mac/",
"description": "MacMacBook Air - MacBook Pro - iMac - Compare Mac models - Mac mini"
},
{
"title": "iPad",
"link": "https://www.apple.com/ipad/",
"description": "iPadShop iPad - iPad Pro - iPad Air - Compare iPad models - ..."
},
{
"title": "Watch",
"link": "https://www.apple.com/watch/",
"description": "WatchShop Apple Watch - Series 9 - SE - Ultra 2 - Nike - Hermès - ..."
}
],
"realPosition": 1
},
{
"title": "Apple",
"link": "https://www.apple.com/",
"description": "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, ...",
"additional_links": [
{
"text": "AppleApplehttps://www.apple.com",
"href": "https://www.apple.com/"
}
],
"cite": {},
"realPosition": 2
},
{
"title": "Apple Inc.",
"link": "https://en.wikipedia.org/wiki/Apple_Inc.",
"description": "Apple Inc. (formerly Apple Computer, Inc.) is an American multinational corporation and technology company headquartered in Cupertino, California, ...",
"additional_links": [
{
"text": "Apple Inc.Wikipediahttps://en.wikipedia.org › wiki › Apple_Inc",
"href": "https://en.wikipedia.org/wiki/Apple_Inc."
},
{
"text": "",
"href": "https://en.wikipedia.org/wiki/Apple_Inc."
},
{
"text": "History",
"href": "https://en.wikipedia.org/wiki/History_of_Apple_Inc."
},
{
"text": "List of Apple products",
"href": "https://en.wikipedia.org/wiki/List_of_Apple_products"
},
{
"text": "Litigation involving Apple Inc.",
"href": "https://en.wikipedia.org/wiki/Litigation_involving_Apple_Inc."
},
{
"text": "Apple Park",
"href": "https://en.wikipedia.org/wiki/Apple_Park"
}
],
"cite": {
"domain": "https://en.wikipedia.org › wiki › Apple_Inc",
"span": " › wiki › Apple_Inc"
},
"realPosition": 3
},
{
"title": "Apple Inc. (AAPL) Company Profile & Facts",
"link": "https://finance.yahoo.com/quote/AAPL/profile/",
"description": "Apple Inc. designs, manufactures, and markets smartphones, personal computers, tablets, wearables, and accessories worldwide. The company offers iPhone, a line ...",
"additional_links": [
{
"text": "Apple Inc. (AAPL) Company Profile & FactsYahoo Financehttps://finance.yahoo.com › quote › AAPL › profile",
"href": "https://finance.yahoo.com/quote/AAPL/profile/"
}
],
"cite": {
"domain": "https://finance.yahoo.com › quote › AAPL › profile",
"span": " › quote › AAPL › profile"
},
"realPosition": 4
},
{
"title": "Apple Inc - Company Profile and News",
"link": "https://www.bloomberg.com/profile/company/AAPL:US",
"description": "Apple Inc. Apple Inc. designs, manufactures, and markets smartphones, personal computers, tablets, wearables and accessories, and sells a variety of related ...",
"additional_links": [
{
"text": "Apple Inc - Company Profile and NewsBloomberghttps://www.bloomberg.com › company › AAPL:US",
"href": "https://www.bloomberg.com/profile/company/AAPL:US"
},
{
"text": "",
"href": "https://www.bloomberg.com/profile/company/AAPL:US"
}
],
"cite": {
"domain": "https://www.bloomberg.com › company › AAPL:US",
"span": " › company › AAPL:US"
},
"realPosition": 5
},
{
"title": "Apple Inc. | History, Products, Headquarters, & Facts",
"link": "https://www.britannica.com/money/Apple-Inc",
"description": "May 22, 2024 — Apple Inc. is an American multinational technology company that revolutionized the technology sector through its innovation of computer ...",
"additional_links": [
{
"text": "Apple Inc. | History, Products, Headquarters, & FactsBritannicahttps://www.britannica.com › money › Apple-Inc",
"href": "https://www.britannica.com/money/Apple-Inc"
},
{
"text": "",
"href": "https://www.britannica.com/money/Apple-Inc"
}
],
"cite": {
"domain": "https://www.britannica.com › money › Apple-Inc",
"span": " › money › Apple-Inc"
},
"realPosition": 6
}
],
"shopping_ads": [],
"places": [
{
"title": "Apple Inc."
},
{
"title": "Apple Inc"
},
{
"title": "Apple Inc"
}
],
"related_searches": {
"images": [],
"text": [
{
"title": "apple inc full form",
"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+Inc+full+form&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhPEAE"
},
{
"title": "apple company history",
"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+company+history&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhOEAE"
},
{
"title": "apple store",
"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+Store&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhQEAE"
},
{
"title": "apple id",
"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+id&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhSEAE"
},
{
"title": "apple inc industry",
"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+Inc+industry&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhREAE"
},
{
"title": "apple login",
"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+login&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhTEAE"
}
]
},
"image_results": [],
"carousel": [],
"total": 2450000000,
"knowledge_graph": "",
"related_questions": [
"What does the Apple Inc do?",
"Why did Apple change to Apple Inc?",
"Who owns Apple Inc.?",
"What is Apple Inc best known for?"
],
"carousel_count": 0,
"ts": 2.491065263748169,
"device_type": null
}

135
backend/apps/rag/utils.py
View File

@ -2,7 +2,7 @@ import os
import logging
import requests
from typing import List
from typing import List, Union
from apps.ollama.main import (
generate_ollama_embeddings,
@ -20,23 +20,8 @@ from langchain.retrievers import (
from typing import Optional
from apps.rag.search.brave import search_brave
from apps.rag.search.google_pse import search_google_pse
from apps.rag.search.main import SearchResult
from apps.rag.search.searxng import search_searxng
from apps.rag.search.serper import search_serper
from apps.rag.search.serpstack import search_serpstack
from config import (
SRC_LOG_LEVELS,
CHROMA_CLIENT,
SEARXNG_QUERY_URL,
GOOGLE_PSE_API_KEY,
GOOGLE_PSE_ENGINE_ID,
BRAVE_SEARCH_API_KEY,
SERPSTACK_API_KEY,
SERPSTACK_HTTPS,
SERPER_API_KEY,
)
from utils.misc import get_last_user_message, add_or_update_system_message
from config import SRC_LOG_LEVELS, CHROMA_CLIENT
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["RAG"])
@ -214,6 +199,7 @@ def get_embedding_function(
embedding_function,
openai_key,
openai_url,
batch_size,
):
if embedding_engine == "":
return lambda query: embedding_function.encode(query).tolist()
@ -237,17 +223,22 @@ def get_embedding_function(
def generate_multiple(query, f):
if isinstance(query, list):
return [f(q) for q in query]
if embedding_engine == "openai":
embeddings = []
for i in range(0, len(query), batch_size):
embeddings.extend(f(query[i : i + batch_size]))
return embeddings
else:
return [f(q) for q in query]
else:
return f(query)
return lambda query: generate_multiple(query, func)
def rag_messages(
def get_rag_context(
docs,
messages,
template,
embedding_function,
k,
reranking_function,
@ -255,31 +246,7 @@ def rag_messages(
hybrid_search,
):
log.debug(f"docs: {docs} {messages} {embedding_function} {reranking_function}")
last_user_message_idx = None
for i in range(len(messages) - 1, -1, -1):
if messages[i]["role"] == "user":
last_user_message_idx = i
break
user_message = messages[last_user_message_idx]
if isinstance(user_message["content"], list):
# Handle list content input
content_type = "list"
query = ""
for content_item in user_message["content"]:
if content_item["type"] == "text":
query = content_item["text"]
break
elif isinstance(user_message["content"], str):
# Handle text content input
content_type = "text"
query = user_message["content"]
else:
# Fallback in case the input does not match expected types
content_type = None
query = ""
query = get_last_user_message(messages)
extracted_collections = []
relevant_contexts = []
@ -350,33 +317,7 @@ def rag_messages(
context_string = context_string.strip()
ra_content = rag_template(
template=template,
context=context_string,
query=query,
)
log.debug(f"ra_content: {ra_content}")
if content_type == "list":
new_content = []
for content_item in user_message["content"]:
if content_item["type"] == "text":
# Update the text item's content with ra_content
new_content.append({"type": "text", "text": ra_content})
else:
# Keep other types of content as they are
new_content.append(content_item)
new_user_message = {**user_message, "content": new_content}
else:
new_user_message = {
**user_message,
"content": ra_content,
}
messages[last_user_message_idx] = new_user_message
return messages, citations
return context_string, citations
def get_model_path(model: str, update_model: bool = False):
@ -418,8 +359,22 @@ def get_model_path(model: str, update_model: bool = False):
def generate_openai_embeddings(
model: str, text: str, key: str, url: str = "https://api.openai.com/v1"
model: str,
text: Union[str, list[str]],
key: str,
url: str = "https://api.openai.com/v1",
):
if isinstance(text, list):
embeddings = generate_openai_batch_embeddings(model, text, key, url)
else:
embeddings = generate_openai_batch_embeddings(model, [text], key, url)
return embeddings[0] if isinstance(text, str) else embeddings
def generate_openai_batch_embeddings(
model: str, texts: list[str], key: str, url: str = "https://api.openai.com/v1"
) -> Optional[list[list[float]]]:
try:
r = requests.post(
f"{url}/embeddings",
@ -427,12 +382,12 @@ def generate_openai_embeddings(
"Content-Type": "application/json",
"Authorization": f"Bearer {key}",
},
json={"input": text, "model": model},
json={"input": texts, "model": model},
)
r.raise_for_status()
data = r.json()
if "data" in data:
return data["data"][0]["embedding"]
return [elem["embedding"] for elem in data["data"]]
else:
raise "Something went wrong :/"
except Exception as e:
@ -536,31 +491,3 @@ class RerankCompressor(BaseDocumentCompressor):
)
final_results.append(doc)
return final_results
def search_web(query: str) -> list[SearchResult]:
"""Search the web using a search engine and return the results as a list of SearchResult objects.
Will look for a search engine API key in environment variables in the following order:
- SEARXNG_QUERY_URL
- GOOGLE_PSE_API_KEY + GOOGLE_PSE_ENGINE_ID
- BRAVE_SEARCH_API_KEY
- SERPSTACK_API_KEY
- SERPER_API_KEY
Args:
query (str): The query to search for
"""
# TODO: add playwright to search the web
if SEARXNG_QUERY_URL:
return search_searxng(SEARXNG_QUERY_URL, query)
elif GOOGLE_PSE_API_KEY and GOOGLE_PSE_ENGINE_ID:
return search_google_pse(GOOGLE_PSE_API_KEY, GOOGLE_PSE_ENGINE_ID, query)
elif BRAVE_SEARCH_API_KEY:
return search_brave(BRAVE_SEARCH_API_KEY, query)
elif SERPSTACK_API_KEY:
return search_serpstack(SERPSTACK_API_KEY, query, https_enabled=SERPSTACK_HTTPS)
elif SERPER_API_KEY:
return search_serper(SERPER_API_KEY, query)
else:
raise Exception("No search engine API key found in environment variables")

139
backend/apps/socket/main.py Normal file
View File

@ -0,0 +1,139 @@
import socketio
import asyncio
from apps.webui.models.users import Users
from utils.utils import decode_token
sio = socketio.AsyncServer(cors_allowed_origins=[], async_mode="asgi")
app = socketio.ASGIApp(sio, socketio_path="/ws/socket.io")
# Dictionary to maintain the user pool
SESSION_POOL = {}
USER_POOL = {}
USAGE_POOL = {}
# Timeout duration in seconds
TIMEOUT_DURATION = 3
@sio.event
async def connect(sid, environ, auth):
user = None
if auth and "token" in auth:
data = decode_token(auth["token"])
if data is not None and "id" in data:
user = Users.get_user_by_id(data["id"])
if user:
SESSION_POOL[sid] = user.id
if user.id in USER_POOL:
USER_POOL[user.id].append(sid)
else:
USER_POOL[user.id] = [sid]
print(f"user {user.name}({user.id}) connected with session ID {sid}")
await sio.emit("user-count", {"count": len(set(USER_POOL))})
await sio.emit("usage", {"models": get_models_in_use()})
@sio.on("user-join")
async def user_join(sid, data):
print("user-join", sid, data)
auth = data["auth"] if "auth" in data else None
if auth and "token" in auth:
data = decode_token(auth["token"])
if data is not None and "id" in data:
user = Users.get_user_by_id(data["id"])
if user:
SESSION_POOL[sid] = user.id
if user.id in USER_POOL:
USER_POOL[user.id].append(sid)
else:
USER_POOL[user.id] = [sid]
print(f"user {user.name}({user.id}) connected with session ID {sid}")
await sio.emit("user-count", {"count": len(set(USER_POOL))})
@sio.on("user-count")
async def user_count(sid):
await sio.emit("user-count", {"count": len(set(USER_POOL))})
def get_models_in_use():
# Aggregate all models in use
models_in_use = []
for model_id, data in USAGE_POOL.items():
models_in_use.append(model_id)
return models_in_use
@sio.on("usage")
async def usage(sid, data):
model_id = data["model"]
# Cancel previous callback if there is one
if model_id in USAGE_POOL:
USAGE_POOL[model_id]["callback"].cancel()
# Store the new usage data and task
if model_id in USAGE_POOL:
USAGE_POOL[model_id]["sids"].append(sid)
USAGE_POOL[model_id]["sids"] = list(set(USAGE_POOL[model_id]["sids"]))
else:
USAGE_POOL[model_id] = {"sids": [sid]}
# Schedule a task to remove the usage data after TIMEOUT_DURATION
USAGE_POOL[model_id]["callback"] = asyncio.create_task(
remove_after_timeout(sid, model_id)
)
# Broadcast the usage data to all clients
await sio.emit("usage", {"models": get_models_in_use()})
async def remove_after_timeout(sid, model_id):
try:
await asyncio.sleep(TIMEOUT_DURATION)
if model_id in USAGE_POOL:
print(USAGE_POOL[model_id]["sids"])
USAGE_POOL[model_id]["sids"].remove(sid)
USAGE_POOL[model_id]["sids"] = list(set(USAGE_POOL[model_id]["sids"]))
if len(USAGE_POOL[model_id]["sids"]) == 0:
del USAGE_POOL[model_id]
# Broadcast the usage data to all clients
await sio.emit("usage", {"models": get_models_in_use()})
except asyncio.CancelledError:
# Task was cancelled due to new 'usage' event
pass
@sio.event
async def disconnect(sid):
if sid in SESSION_POOL:
user_id = SESSION_POOL[sid]
del SESSION_POOL[sid]
USER_POOL[user_id].remove(sid)
if len(USER_POOL[user_id]) == 0:
del USER_POOL[user_id]
await sio.emit("user-count", {"count": len(USER_POOL)})
else:
print(f"Unknown session ID {sid} disconnected")

61
backend/apps/webui/internal/migrations/012_add_tools.py Normal file
View File

@ -0,0 +1,61 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
@migrator.create_model
class Tool(pw.Model):
id = pw.TextField(unique=True)
user_id = pw.TextField()
name = pw.TextField()
content = pw.TextField()
specs = pw.TextField()
meta = pw.TextField()
created_at = pw.BigIntegerField(null=False)
updated_at = pw.BigIntegerField(null=False)
class Meta:
table_name = "tool"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("tool")

14
backend/apps/webui/main.py
View File

@ -6,6 +6,7 @@ from apps.webui.routers import (
users,
chats,
documents,
tools,
models,
prompts,
configs,
@ -14,6 +15,8 @@ from apps.webui.routers import (
)
from config import (
WEBUI_BUILD_HASH,
SHOW_ADMIN_DETAILS,
ADMIN_EMAIL,
WEBUI_AUTH,
DEFAULT_MODELS,
DEFAULT_PROMPT_SUGGESTIONS,
@ -24,8 +27,8 @@ from config import (
WEBUI_AUTH_TRUSTED_EMAIL_HEADER,
JWT_EXPIRES_IN,
WEBUI_BANNERS,
AppConfig,
ENABLE_COMMUNITY_SHARING,
AppConfig,
)
app = FastAPI()
@ -36,6 +39,12 @@ app.state.config = AppConfig()
app.state.config.ENABLE_SIGNUP = ENABLE_SIGNUP
app.state.config.JWT_EXPIRES_IN = JWT_EXPIRES_IN
app.state.AUTH_TRUSTED_EMAIL_HEADER = WEBUI_AUTH_TRUSTED_EMAIL_HEADER
app.state.config.SHOW_ADMIN_DETAILS = SHOW_ADMIN_DETAILS
app.state.config.ADMIN_EMAIL = ADMIN_EMAIL
app.state.config.DEFAULT_MODELS = DEFAULT_MODELS
app.state.config.DEFAULT_PROMPT_SUGGESTIONS = DEFAULT_PROMPT_SUGGESTIONS
@ -47,7 +56,7 @@ app.state.config.BANNERS = WEBUI_BANNERS
app.state.config.ENABLE_COMMUNITY_SHARING = ENABLE_COMMUNITY_SHARING
app.state.MODELS = {}
app.state.AUTH_TRUSTED_EMAIL_HEADER = WEBUI_AUTH_TRUSTED_EMAIL_HEADER
app.state.TOOLS = {}
app.add_middleware(
@ -63,6 +72,7 @@ app.include_router(users.router, prefix="/users", tags=["users"])
app.include_router(chats.router, prefix="/chats", tags=["chats"])
app.include_router(documents.router, prefix="/documents", tags=["documents"])
app.include_router(tools.router, prefix="/tools", tags=["tools"])
app.include_router(models.router, prefix="/models", tags=["models"])
app.include_router(prompts.router, prefix="/prompts", tags=["prompts"])
app.include_router(memories.router, prefix="/memories", tags=["memories"])

9
backend/apps/webui/models/chats.py
View File

@ -298,6 +298,15 @@ class ChatTable:
# .limit(limit).offset(skip)
]
def get_archived_chats_by_user_id(self, user_id: str) -> List[ChatModel]:
return [
ChatModel(**model_to_dict(chat))
for chat in Chat.select()
.where(Chat.archived == True)
.where(Chat.user_id == user_id)
.order_by(Chat.updated_at.desc())
]
def delete_chat_by_id(self, id: str) -> bool:
try:
query = Chat.delete().where((Chat.id == id))

132
backend/apps/webui/models/tools.py Normal file
View File

@ -0,0 +1,132 @@
from pydantic import BaseModel
from peewee import *
from playhouse.shortcuts import model_to_dict
from typing import List, Union, Optional
import time
import logging
from apps.webui.internal.db import DB, JSONField
import json
from config import SRC_LOG_LEVELS
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
####################
# Tools DB Schema
####################
class Tool(Model):
id = CharField(unique=True)
user_id = CharField()
name = TextField()
content = TextField()
specs = JSONField()
meta = JSONField()
updated_at = BigIntegerField()
created_at = BigIntegerField()
class Meta:
database = DB
class ToolMeta(BaseModel):
description: Optional[str] = None
class ToolModel(BaseModel):
id: str
user_id: str
name: str
content: str
specs: List[dict]
meta: ToolMeta
updated_at: int # timestamp in epoch
created_at: int # timestamp in epoch
####################
# Forms
####################
class ToolResponse(BaseModel):
id: str
user_id: str
name: str
meta: ToolMeta
updated_at: int # timestamp in epoch
created_at: int # timestamp in epoch
class ToolForm(BaseModel):
id: str
name: str
content: str
meta: ToolMeta
class ToolsTable:
def __init__(self, db):
self.db = db
self.db.create_tables([Tool])
def insert_new_tool(
self, user_id: str, form_data: ToolForm, specs: List[dict]
) -> Optional[ToolModel]:
tool = ToolModel(
**{
**form_data.model_dump(),
"specs": specs,
"user_id": user_id,
"updated_at": int(time.time()),
"created_at": int(time.time()),
}
)
try:
result = Tool.create(**tool.model_dump())
if result:
return tool
else:
return None
except Exception as e:
print(f"Error creating tool: {e}")
return None
def get_tool_by_id(self, id: str) -> Optional[ToolModel]:
try:
tool = Tool.get(Tool.id == id)
return ToolModel(**model_to_dict(tool))
except:
return None
def get_tools(self) -> List[ToolModel]:
return [ToolModel(**model_to_dict(tool)) for tool in Tool.select()]
def update_tool_by_id(self, id: str, updated: dict) -> Optional[ToolModel]:
try:
query = Tool.update(
**updated,
updated_at=int(time.time()),
).where(Tool.id == id)
query.execute()
tool = Tool.get(Tool.id == id)
return ToolModel(**model_to_dict(tool))
except:
return None
def delete_tool_by_id(self, id: str) -> bool:
try:
query = Tool.delete().where((Tool.id == id))
query.execute() # Remove the rows, return number of rows removed.
return True
except:
return False
Tools = ToolsTable(DB)

119
backend/apps/webui/routers/auths.py
View File

@ -269,73 +269,88 @@ async def add_user(form_data: AddUserForm, user=Depends(get_admin_user)):
raise HTTPException(500, detail=ERROR_MESSAGES.DEFAULT(err))
############################
# GetAdminDetails
############################
@router.get("/admin/details")
async def get_admin_details(request: Request, user=Depends(get_current_user)):
if request.app.state.config.SHOW_ADMIN_DETAILS:
admin_email = request.app.state.config.ADMIN_EMAIL
admin_name = None
print(admin_email, admin_name)
if admin_email:
admin = Users.get_user_by_email(admin_email)
if admin:
admin_name = admin.name
else:
admin = Users.get_first_user()
if admin:
admin_email = admin.email
admin_name = admin.name
return {
"name": admin_name,
"email": admin_email,
}
else:
raise HTTPException(400, detail=ERROR_MESSAGES.ACTION_PROHIBITED)
############################
# ToggleSignUp
############################
@router.get("/signup/enabled", response_model=bool)
async def get_sign_up_status(request: Request, user=Depends(get_admin_user)):
return request.app.state.config.ENABLE_SIGNUP
@router.get("/admin/config")
async def get_admin_config(request: Request, user=Depends(get_admin_user)):
return {
"SHOW_ADMIN_DETAILS": request.app.state.config.SHOW_ADMIN_DETAILS,
"ENABLE_SIGNUP": request.app.state.config.ENABLE_SIGNUP,
"DEFAULT_USER_ROLE": request.app.state.config.DEFAULT_USER_ROLE,
"JWT_EXPIRES_IN": request.app.state.config.JWT_EXPIRES_IN,
"ENABLE_COMMUNITY_SHARING": request.app.state.config.ENABLE_COMMUNITY_SHARING,
}
@router.get("/signup/enabled/toggle", response_model=bool)
async def toggle_sign_up(request: Request, user=Depends(get_admin_user)):
request.app.state.config.ENABLE_SIGNUP = not request.app.state.config.ENABLE_SIGNUP
return request.app.state.config.ENABLE_SIGNUP
class AdminConfig(BaseModel):
SHOW_ADMIN_DETAILS: bool
ENABLE_SIGNUP: bool
DEFAULT_USER_ROLE: str
JWT_EXPIRES_IN: str
ENABLE_COMMUNITY_SHARING: bool
############################
# Default User Role
############################
@router.get("/signup/user/role")
async def get_default_user_role(request: Request, user=Depends(get_admin_user)):
return request.app.state.config.DEFAULT_USER_ROLE
class UpdateRoleForm(BaseModel):
role: str
@router.post("/signup/user/role")
async def update_default_user_role(
request: Request, form_data: UpdateRoleForm, user=Depends(get_admin_user)
@router.post("/admin/config")
async def update_admin_config(
request: Request, form_data: AdminConfig, user=Depends(get_admin_user)
):
if form_data.role in ["pending", "user", "admin"]:
request.app.state.config.DEFAULT_USER_ROLE = form_data.role
return request.app.state.config.DEFAULT_USER_ROLE
request.app.state.config.SHOW_ADMIN_DETAILS = form_data.SHOW_ADMIN_DETAILS
request.app.state.config.ENABLE_SIGNUP = form_data.ENABLE_SIGNUP
if form_data.DEFAULT_USER_ROLE in ["pending", "user", "admin"]:
request.app.state.config.DEFAULT_USER_ROLE = form_data.DEFAULT_USER_ROLE
############################
# JWT Expiration
############################
@router.get("/token/expires")
async def get_token_expires_duration(request: Request, user=Depends(get_admin_user)):
return request.app.state.config.JWT_EXPIRES_IN
class UpdateJWTExpiresDurationForm(BaseModel):
duration: str
@router.post("/token/expires/update")
async def update_token_expires_duration(
request: Request,
form_data: UpdateJWTExpiresDurationForm,
user=Depends(get_admin_user),
):
pattern = r"^(-1|0|(-?\d+(\.\d+)?)(ms|s|m|h|d|w))$"
# Check if the input string matches the pattern
if re.match(pattern, form_data.duration):
request.app.state.config.JWT_EXPIRES_IN = form_data.duration
return request.app.state.config.JWT_EXPIRES_IN
else:
return request.app.state.config.JWT_EXPIRES_IN
if re.match(pattern, form_data.JWT_EXPIRES_IN):
request.app.state.config.JWT_EXPIRES_IN = form_data.JWT_EXPIRES_IN
request.app.state.config.ENABLE_COMMUNITY_SHARING = (
form_data.ENABLE_COMMUNITY_SHARING
)
return {
"SHOW_ADMIN_DETAILS": request.app.state.config.SHOW_ADMIN_DETAILS,
"ENABLE_SIGNUP": request.app.state.config.ENABLE_SIGNUP,
"DEFAULT_USER_ROLE": request.app.state.config.DEFAULT_USER_ROLE,
"JWT_EXPIRES_IN": request.app.state.config.JWT_EXPIRES_IN,
"ENABLE_COMMUNITY_SHARING": request.app.state.config.ENABLE_COMMUNITY_SHARING,
}
############################

41
backend/apps/webui/routers/chats.py
View File

@ -113,6 +113,19 @@ async def get_user_chats(user=Depends(get_current_user)):
]
############################
# GetArchivedChats
############################
@router.get("/all/archived", response_model=List[ChatResponse])
async def get_user_chats(user=Depends(get_current_user)):
return [
ChatResponse(**{**chat.model_dump(), "chat": json.loads(chat.chat)})
for chat in Chats.get_archived_chats_by_user_id(user.id)
]
############################
# GetAllChatsInDB
############################
@ -148,7 +161,7 @@ async def get_archived_session_user_chat_list(
############################
@router.post("/archive/all", response_model=List[ChatTitleIdResponse])
@router.post("/archive/all", response_model=bool)
async def archive_all_chats(user=Depends(get_current_user)):
return Chats.archive_all_chats_by_user_id(user.id)
@ -288,6 +301,32 @@ async def delete_chat_by_id(request: Request, id: str, user=Depends(get_current_
return result
############################
# CloneChat
############################
@router.get("/{id}/clone", response_model=Optional[ChatResponse])
async def clone_chat_by_id(id: str, user=Depends(get_current_user)):
chat = Chats.get_chat_by_id_and_user_id(id, user.id)
if chat:
chat_body = json.loads(chat.chat)
updated_chat = {
**chat_body,
"originalChatId": chat.id,
"branchPointMessageId": chat_body["history"]["currentId"],
"title": f"Clone of {chat.title}",
}
chat = Chats.insert_new_chat(user.id, ChatForm(**{"chat": updated_chat}))
return ChatResponse(**{**chat.model_dump(), "chat": json.loads(chat.chat)})
else:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, detail=ERROR_MESSAGES.DEFAULT()
)
############################
# ArchiveChat
############################

8
backend/apps/webui/routers/documents.py
View File

@ -73,7 +73,7 @@ async def create_new_doc(form_data: DocumentForm, user=Depends(get_admin_user)):
############################
@router.get("/name/{name}", response_model=Optional[DocumentResponse])
@router.get("/doc", response_model=Optional[DocumentResponse])
async def get_doc_by_name(name: str, user=Depends(get_current_user)):
doc = Documents.get_doc_by_name(name)
@ -105,7 +105,7 @@ class TagDocumentForm(BaseModel):
tags: List[dict]
@router.post("/name/{name}/tags", response_model=Optional[DocumentResponse])
@router.post("/doc/tags", response_model=Optional[DocumentResponse])
async def tag_doc_by_name(form_data: TagDocumentForm, user=Depends(get_current_user)):
doc = Documents.update_doc_content_by_name(form_data.name, {"tags": form_data.tags})
@ -128,7 +128,7 @@ async def tag_doc_by_name(form_data: TagDocumentForm, user=Depends(get_current_u
############################
@router.post("/name/{name}/update", response_model=Optional[DocumentResponse])
@router.post("/doc/update", response_model=Optional[DocumentResponse])
async def update_doc_by_name(
name: str, form_data: DocumentUpdateForm, user=Depends(get_admin_user)
):
@ -152,7 +152,7 @@ async def update_doc_by_name(
############################
@router.delete("/name/{name}/delete", response_model=bool)
@router.delete("/doc/delete", response_model=bool)
async def delete_doc_by_name(name: str, user=Depends(get_admin_user)):
result = Documents.delete_doc_by_name(name)
return result

183
backend/apps/webui/routers/tools.py Normal file
View File

@ -0,0 +1,183 @@
from fastapi import Depends, FastAPI, HTTPException, status, Request
from datetime import datetime, timedelta
from typing import List, Union, Optional
from fastapi import APIRouter
from pydantic import BaseModel
import json
from apps.webui.models.tools import Tools, ToolForm, ToolModel, ToolResponse
from apps.webui.utils import load_toolkit_module_by_id
from utils.utils import get_current_user, get_admin_user
from utils.tools import get_tools_specs
from constants import ERROR_MESSAGES
from importlib import util
import os
from config import DATA_DIR
TOOLS_DIR = f"{DATA_DIR}/tools"
os.makedirs(TOOLS_DIR, exist_ok=True)
router = APIRouter()
############################
# GetToolkits
############################
@router.get("/", response_model=List[ToolResponse])
async def get_toolkits(user=Depends(get_current_user)):
toolkits = [toolkit for toolkit in Tools.get_tools()]
return toolkits
############################
# ExportToolKits
############################
@router.get("/export", response_model=List[ToolModel])
async def get_toolkits(user=Depends(get_admin_user)):
toolkits = [toolkit for toolkit in Tools.get_tools()]
return toolkits
############################
# CreateNewToolKit
############################
@router.post("/create", response_model=Optional[ToolResponse])
async def create_new_toolkit(
request: Request, form_data: ToolForm, user=Depends(get_admin_user)
):
if not form_data.id.isidentifier():
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Only alphanumeric characters and underscores are allowed in the id",
)
form_data.id = form_data.id.lower()
toolkit = Tools.get_tool_by_id(form_data.id)
if toolkit == None:
toolkit_path = os.path.join(TOOLS_DIR, f"{form_data.id}.py")
try:
with open(toolkit_path, "w") as tool_file:
tool_file.write(form_data.content)
toolkit_module = load_toolkit_module_by_id(form_data.id)
TOOLS = request.app.state.TOOLS
TOOLS[form_data.id] = toolkit_module
specs = get_tools_specs(TOOLS[form_data.id])
toolkit = Tools.insert_new_tool(user.id, form_data, specs)
if toolkit:
return toolkit
else:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.DEFAULT("Error creating toolkit"),
)
except Exception as e:
print(e)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.DEFAULT(e),
)
else:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.ID_TAKEN,
)
############################
# GetToolkitById
############################
@router.get("/id/{id}", response_model=Optional[ToolModel])
async def get_toolkit_by_id(id: str, user=Depends(get_admin_user)):
toolkit = Tools.get_tool_by_id(id)
if toolkit:
return toolkit
else:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail=ERROR_MESSAGES.NOT_FOUND,
)
############################
# UpdateToolkitById
############################
@router.post("/id/{id}/update", response_model=Optional[ToolModel])
async def update_toolkit_by_id(
request: Request, id: str, form_data: ToolForm, user=Depends(get_admin_user)
):
toolkit_path = os.path.join(TOOLS_DIR, f"{id}.py")
try:
with open(toolkit_path, "w") as tool_file:
tool_file.write(form_data.content)
toolkit_module = load_toolkit_module_by_id(id)
TOOLS = request.app.state.TOOLS
TOOLS[id] = toolkit_module
specs = get_tools_specs(TOOLS[id])
updated = {
**form_data.model_dump(exclude={"id"}),
"specs": specs,
}
print(updated)
toolkit = Tools.update_tool_by_id(id, updated)
if toolkit:
return toolkit
else:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.DEFAULT("Error updating toolkit"),
)
except Exception as e:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=ERROR_MESSAGES.DEFAULT(e),
)
############################
# DeleteToolkitById
############################
@router.delete("/id/{id}/delete", response_model=bool)
async def delete_toolkit_by_id(request: Request, id: str, user=Depends(get_admin_user)):
result = Tools.delete_tool_by_id(id)
if result:
TOOLS = request.app.state.TOOLS
if id in TOOLS:
del TOOLS[id]
# delete the toolkit file
toolkit_path = os.path.join(TOOLS_DIR, f"{id}.py")
os.remove(toolkit_path)
return result

7
backend/apps/webui/routers/users.py
View File

@ -19,7 +19,12 @@ from apps.webui.models.users import (
from apps.webui.models.auths import Auths
from apps.webui.models.chats import Chats
from utils.utils import get_verified_user, get_password_hash, get_admin_user
from utils.utils import (
get_verified_user,
get_password_hash,
get_current_user,
get_admin_user,
)
from constants import ERROR_MESSAGES
from config import SRC_LOG_LEVELS

26
backend/apps/webui/routers/utils.py
View File

@ -7,6 +7,8 @@ from pydantic import BaseModel
from fpdf import FPDF
import markdown
import black
from apps.webui.internal.db import DB
from utils.utils import get_admin_user
@ -26,6 +28,21 @@ async def get_gravatar(
return get_gravatar_url(email)
class CodeFormatRequest(BaseModel):
code: str
@router.post("/code/format")
async def format_code(request: CodeFormatRequest):
try:
formatted_code = black.format_str(request.code, mode=black.Mode())
return {"code": formatted_code}
except black.NothingChanged:
return {"code": request.code}
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))
class MarkdownForm(BaseModel):
md: str
@ -107,3 +124,12 @@ async def download_db(user=Depends(get_admin_user)):
media_type="application/octet-stream",
filename="webui.db",
)
@router.get("/litellm/config")
async def download_litellm_config_yaml(user=Depends(get_admin_user)):
return FileResponse(
f"{DATA_DIR}/litellm/config.yaml",
media_type="application/octet-stream",
filename="config.yaml",
)

23
backend/apps/webui/utils.py Normal file
View File

@ -0,0 +1,23 @@
from importlib import util
import os
from config import TOOLS_DIR
def load_toolkit_module_by_id(toolkit_id):
toolkit_path = os.path.join(TOOLS_DIR, f"{toolkit_id}.py")
spec = util.spec_from_file_location(toolkit_id, toolkit_path)
module = util.module_from_spec(spec)
try:
spec.loader.exec_module(module)
print(f"Loaded module: {module.__name__}")
if hasattr(module, "Tools"):
return module.Tools()
else:
raise Exception("No Tools class found")
except Exception as e:
print(f"Error loading module: {toolkit_id}")
# Move the file to the error folder
os.rename(toolkit_path, f"{toolkit_path}.error")
raise e

270
backend/config.py
View File

@ -180,6 +180,17 @@ WEBUI_BUILD_HASH = os.environ.get("WEBUI_BUILD_HASH", "dev-build")
DATA_DIR = Path(os.getenv("DATA_DIR", BACKEND_DIR / "data")).resolve()
FRONTEND_BUILD_DIR = Path(os.getenv("FRONTEND_BUILD_DIR", BASE_DIR / "build")).resolve()
RESET_CONFIG_ON_START = (
os.environ.get("RESET_CONFIG_ON_START", "False").lower() == "true"
)
if RESET_CONFIG_ON_START:
try:
os.remove(f"{DATA_DIR}/config.json")
with open(f"{DATA_DIR}/config.json", "w") as f:
f.write("{}")
except:
pass
try:
CONFIG_DATA = json.loads((DATA_DIR / "config.json").read_text())
except:
@ -295,7 +306,11 @@ STATIC_DIR = Path(os.getenv("STATIC_DIR", BACKEND_DIR / "static")).resolve()
frontend_favicon = FRONTEND_BUILD_DIR / "favicon.png"
if frontend_favicon.exists():
shutil.copyfile(frontend_favicon, STATIC_DIR / "favicon.png")
try:
shutil.copyfile(frontend_favicon, STATIC_DIR / "favicon.png")
except Exception as e:
logging.error(f"An error occurred: {e}")
else:
logging.warning(f"Frontend favicon not found at {frontend_favicon}")
@ -353,6 +368,14 @@ DOCS_DIR = os.getenv("DOCS_DIR", f"{DATA_DIR}/docs")
Path(DOCS_DIR).mkdir(parents=True, exist_ok=True)
####################################
# Tools DIR
####################################
TOOLS_DIR = os.getenv("TOOLS_DIR", f"{DATA_DIR}/tools")
Path(TOOLS_DIR).mkdir(parents=True, exist_ok=True)
####################################
# LITELLM_CONFIG
####################################
@ -590,6 +613,92 @@ WEBUI_BANNERS = PersistentConfig(
[BannerModel(**banner) for banner in json.loads("[]")],
)
SHOW_ADMIN_DETAILS = PersistentConfig(
"SHOW_ADMIN_DETAILS",
"auth.admin.show",
os.environ.get("SHOW_ADMIN_DETAILS", "true").lower() == "true",
)
ADMIN_EMAIL = PersistentConfig(
"ADMIN_EMAIL",
"auth.admin.email",
os.environ.get("ADMIN_EMAIL", None),
)
####################################
# TASKS
####################################
TASK_MODEL = PersistentConfig(
"TASK_MODEL",
"task.model.default",
os.environ.get("TASK_MODEL", ""),
)
TASK_MODEL_EXTERNAL = PersistentConfig(
"TASK_MODEL_EXTERNAL",
"task.model.external",
os.environ.get("TASK_MODEL_EXTERNAL", ""),
)
TITLE_GENERATION_PROMPT_TEMPLATE = PersistentConfig(
"TITLE_GENERATION_PROMPT_TEMPLATE",
"task.title.prompt_template",
os.environ.get(
"TITLE_GENERATION_PROMPT_TEMPLATE",
"""Here is the query:
{{prompt:middletruncate:8000}}
Create a concise, 3-5 word phrase with an emoji as a title for the previous query. Suitable Emojis for the summary can be used to enhance understanding but avoid quotation marks or special formatting. RESPOND ONLY WITH THE TITLE TEXT.
Examples of titles:
📉 Stock Market Trends
🍪 Perfect Chocolate Chip Recipe
Evolution of Music Streaming
Remote Work Productivity Tips
Artificial Intelligence in Healthcare
🎮 Video Game Development Insights""",
),
)
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE = PersistentConfig(
"SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE",
"task.search.prompt_template",
os.environ.get(
"SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE",
"""You are tasked with generating web search queries. Give me an appropriate query to answer my question for google search. Answer with only the query. Today is {{CURRENT_DATE}}.
Question:
{{prompt:end:4000}}""",
),
)
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD = PersistentConfig(
"SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD",
"task.search.prompt_length_threshold",
int(
os.environ.get(
"SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD",
100,
)
),
)
TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE = PersistentConfig(
"TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE",
"task.tools.prompt_template",
os.environ.get(
"TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE",
"""Tools: {{TOOLS}}
If a function tool doesn't match the query, return an empty string. Else, pick a function tool, fill in the parameters from the function tool's schema, and return it in the format { "name": \"functionName\", "parameters": { "key": "value" } }. Only pick a function if the user asks. Only return the object. Do not return any other text.""",
),
)
####################################
# WEBUI_SECRET_KEY
####################################
@ -672,6 +781,12 @@ RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE = (
os.environ.get("RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE", "").lower() == "true"
)
RAG_EMBEDDING_OPENAI_BATCH_SIZE = PersistentConfig(
"RAG_EMBEDDING_OPENAI_BATCH_SIZE",
"rag.embedding_openai_batch_size",
os.environ.get("RAG_EMBEDDING_OPENAI_BATCH_SIZE", 1),
)
RAG_RERANKING_MODEL = PersistentConfig(
"RAG_RERANKING_MODEL",
"rag.reranking_model",
@ -766,28 +881,81 @@ YOUTUBE_LOADER_LANGUAGE = PersistentConfig(
os.getenv("YOUTUBE_LOADER_LANGUAGE", "en").split(","),
)
SEARXNG_QUERY_URL = os.getenv("SEARXNG_QUERY_URL", "")
GOOGLE_PSE_API_KEY = os.getenv("GOOGLE_PSE_API_KEY", "")
GOOGLE_PSE_ENGINE_ID = os.getenv("GOOGLE_PSE_ENGINE_ID", "")
BRAVE_SEARCH_API_KEY = os.getenv("BRAVE_SEARCH_API_KEY", "")
SERPSTACK_API_KEY = os.getenv("SERPSTACK_API_KEY", "")
SERPSTACK_HTTPS = os.getenv("SERPSTACK_HTTPS", "True").lower() == "true"
SERPER_API_KEY = os.getenv("SERPER_API_KEY", "")
RAG_WEB_SEARCH_ENABLED = (
SEARXNG_QUERY_URL != ""
or (GOOGLE_PSE_API_KEY != "" and GOOGLE_PSE_ENGINE_ID != "")
or BRAVE_SEARCH_API_KEY != ""
or SERPSTACK_API_KEY != ""
or SERPER_API_KEY != ""
ENABLE_RAG_WEB_SEARCH = PersistentConfig(
"ENABLE_RAG_WEB_SEARCH",
"rag.web.search.enable",
os.getenv("ENABLE_RAG_WEB_SEARCH", "False").lower() == "true",
)
RAG_WEB_SEARCH_RESULT_COUNT = int(os.getenv("RAG_WEB_SEARCH_RESULT_COUNT", "3"))
RAG_WEB_SEARCH_CONCURRENT_REQUESTS = int(
os.getenv("RAG_WEB_SEARCH_CONCURRENT_REQUESTS", "10")
RAG_WEB_SEARCH_ENGINE = PersistentConfig(
"RAG_WEB_SEARCH_ENGINE",
"rag.web.search.engine",
os.getenv("RAG_WEB_SEARCH_ENGINE", ""),
)
SEARXNG_QUERY_URL = PersistentConfig(
"SEARXNG_QUERY_URL",
"rag.web.search.searxng_query_url",
os.getenv("SEARXNG_QUERY_URL", ""),
)
GOOGLE_PSE_API_KEY = PersistentConfig(
"GOOGLE_PSE_API_KEY",
"rag.web.search.google_pse_api_key",
os.getenv("GOOGLE_PSE_API_KEY", ""),
)
GOOGLE_PSE_ENGINE_ID = PersistentConfig(
"GOOGLE_PSE_ENGINE_ID",
"rag.web.search.google_pse_engine_id",
os.getenv("GOOGLE_PSE_ENGINE_ID", ""),
)
BRAVE_SEARCH_API_KEY = PersistentConfig(
"BRAVE_SEARCH_API_KEY",
"rag.web.search.brave_search_api_key",
os.getenv("BRAVE_SEARCH_API_KEY", ""),
)
SERPSTACK_API_KEY = PersistentConfig(
"SERPSTACK_API_KEY",
"rag.web.search.serpstack_api_key",
os.getenv("SERPSTACK_API_KEY", ""),
)
SERPSTACK_HTTPS = PersistentConfig(
"SERPSTACK_HTTPS",
"rag.web.search.serpstack_https",
os.getenv("SERPSTACK_HTTPS", "True").lower() == "true",
)
SERPER_API_KEY = PersistentConfig(
"SERPER_API_KEY",
"rag.web.search.serper_api_key",
os.getenv("SERPER_API_KEY", ""),
)
SERPLY_API_KEY = PersistentConfig(
"SERPLY_API_KEY",
"rag.web.search.serply_api_key",
os.getenv("SERPLY_API_KEY", ""),
)
RAG_WEB_SEARCH_RESULT_COUNT = PersistentConfig(
"RAG_WEB_SEARCH_RESULT_COUNT",
"rag.web.search.result_count",
int(os.getenv("RAG_WEB_SEARCH_RESULT_COUNT", "3")),
)
RAG_WEB_SEARCH_CONCURRENT_REQUESTS = PersistentConfig(
"RAG_WEB_SEARCH_CONCURRENT_REQUESTS",
"rag.web.search.concurrent_requests",
int(os.getenv("RAG_WEB_SEARCH_CONCURRENT_REQUESTS", "10")),
)
####################################
# Transcribe
####################################
@ -855,25 +1023,59 @@ IMAGE_GENERATION_MODEL = PersistentConfig(
# Audio
####################################
AUDIO_OPENAI_API_BASE_URL = PersistentConfig(
"AUDIO_OPENAI_API_BASE_URL",
"audio.openai.api_base_url",
os.getenv("AUDIO_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL),
AUDIO_STT_OPENAI_API_BASE_URL = PersistentConfig(
"AUDIO_STT_OPENAI_API_BASE_URL",
"audio.stt.openai.api_base_url",
os.getenv("AUDIO_STT_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL),
)
AUDIO_OPENAI_API_KEY = PersistentConfig(
"AUDIO_OPENAI_API_KEY",
"audio.openai.api_key",
os.getenv("AUDIO_OPENAI_API_KEY", OPENAI_API_KEY),
AUDIO_STT_OPENAI_API_KEY = PersistentConfig(
"AUDIO_STT_OPENAI_API_KEY",
"audio.stt.openai.api_key",
os.getenv("AUDIO_STT_OPENAI_API_KEY", OPENAI_API_KEY),
)
AUDIO_OPENAI_API_MODEL = PersistentConfig(
"AUDIO_OPENAI_API_MODEL",
"audio.openai.api_model",
os.getenv("AUDIO_OPENAI_API_MODEL", "tts-1"),
AUDIO_STT_ENGINE = PersistentConfig(
"AUDIO_STT_ENGINE",
"audio.stt.engine",
os.getenv("AUDIO_STT_ENGINE", ""),
)
AUDIO_OPENAI_API_VOICE = PersistentConfig(
"AUDIO_OPENAI_API_VOICE",
"audio.openai.api_voice",
os.getenv("AUDIO_OPENAI_API_VOICE", "alloy"),
AUDIO_STT_MODEL = PersistentConfig(
"AUDIO_STT_MODEL",
"audio.stt.model",
os.getenv("AUDIO_STT_MODEL", "whisper-1"),
)
AUDIO_TTS_OPENAI_API_BASE_URL = PersistentConfig(
"AUDIO_TTS_OPENAI_API_BASE_URL",
"audio.tts.openai.api_base_url",
os.getenv("AUDIO_TTS_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL),
)
AUDIO_TTS_OPENAI_API_KEY = PersistentConfig(
"AUDIO_TTS_OPENAI_API_KEY",
"audio.tts.openai.api_key",
os.getenv("AUDIO_TTS_OPENAI_API_KEY", OPENAI_API_KEY),
)
AUDIO_TTS_ENGINE = PersistentConfig(
"AUDIO_TTS_ENGINE",
"audio.tts.engine",
os.getenv("AUDIO_TTS_ENGINE", ""),
)
AUDIO_TTS_MODEL = PersistentConfig(
"AUDIO_TTS_MODEL",
"audio.tts.model",
os.getenv("AUDIO_TTS_MODEL", "tts-1"),
)
AUDIO_TTS_VOICE = PersistentConfig(
"AUDIO_TTS_VOICE",
"audio.tts.voice",
os.getenv("AUDIO_TTS_VOICE", "alloy"),
)

7
backend/constants.py
View File

@ -32,6 +32,7 @@ class ERROR_MESSAGES(str, Enum):
COMMAND_TAKEN = "Uh-oh! This command is already registered. Please choose another command string."
FILE_EXISTS = "Uh-oh! This file is already registered. Please choose another file."
ID_TAKEN = "Uh-oh! This id is already registered. Please choose another id string."
MODEL_ID_TAKEN = "Uh-oh! This model id is already registered. Please choose another model id string."
NAME_TAG_TAKEN = "Uh-oh! This name tag is already registered. Please choose another name tag string."
@ -82,5 +83,9 @@ class ERROR_MESSAGES(str, Enum):
)
WEB_SEARCH_ERROR = (
"Oops! Something went wrong while searching the web. Please try again later."
lambda err="": f"{err if err else 'Oops! Something went wrong while searching the web.'}"
)
OLLAMA_API_DISABLED = (
"The Ollama API is disabled. Please enable it to use this feature."
)

738
backend/main.py
View File

@ -9,8 +9,12 @@ import logging
import aiohttp
import requests
import mimetypes
import shutil
import os
import inspect
import asyncio
from fastapi import FastAPI, Request, Depends, status
from fastapi import FastAPI, Request, Depends, status, UploadFile, File, Form
from fastapi.staticfiles import StaticFiles
from fastapi.responses import JSONResponse
from fastapi import HTTPException
@ -20,26 +24,48 @@ from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import StreamingResponse, Response
from apps.ollama.main import app as ollama_app, get_all_models as get_ollama_models
from apps.openai.main import app as openai_app, get_all_models as get_openai_models
from apps.socket.main import app as socket_app
from apps.ollama.main import (
app as ollama_app,
OpenAIChatCompletionForm,
get_all_models as get_ollama_models,
generate_openai_chat_completion as generate_ollama_chat_completion,
)
from apps.openai.main import (
app as openai_app,
get_all_models as get_openai_models,
generate_chat_completion as generate_openai_chat_completion,
)
from apps.audio.main import app as audio_app
from apps.images.main import app as images_app
from apps.rag.main import app as rag_app
from apps.webui.main import app as webui_app
import asyncio
from pydantic import BaseModel
from typing import List, Optional
from apps.webui.models.models import Models, ModelModel
from apps.webui.models.tools import Tools
from apps.webui.utils import load_toolkit_module_by_id
from utils.utils import (
get_admin_user,
get_verified_user,
get_current_user,
get_http_authorization_cred,
)
from apps.rag.utils import rag_messages
from utils.task import (
title_generation_template,
search_query_generation_template,
tools_function_calling_generation_template,
)
from utils.misc import get_last_user_message, add_or_update_system_message
from apps.rag.utils import get_rag_context, rag_template
from config import (
CONFIG_DATA,
@ -60,9 +86,14 @@ from config import (
SRC_LOG_LEVELS,
WEBHOOK_URL,
ENABLE_ADMIN_EXPORT,
RAG_WEB_SEARCH_ENABLED,
AppConfig,
WEBUI_BUILD_HASH,
TASK_MODEL,
TASK_MODEL_EXTERNAL,
TITLE_GENERATION_PROMPT_TEMPLATE,
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
AppConfig,
)
from constants import ERROR_MESSAGES
@ -116,27 +147,133 @@ app.state.config.ENABLE_OLLAMA_API = ENABLE_OLLAMA_API
app.state.config.ENABLE_MODEL_FILTER = ENABLE_MODEL_FILTER
app.state.config.MODEL_FILTER_LIST = MODEL_FILTER_LIST
app.state.config.WEBHOOK_URL = WEBHOOK_URL
app.state.config.TASK_MODEL = TASK_MODEL
app.state.config.TASK_MODEL_EXTERNAL = TASK_MODEL_EXTERNAL
app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE = TITLE_GENERATION_PROMPT_TEMPLATE
app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE = (
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE
)
app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD = (
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD
)
app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE = (
TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE
)
app.state.MODELS = {}
origins = ["*"]
# Custom middleware to add security headers
# class SecurityHeadersMiddleware(BaseHTTPMiddleware):
# async def dispatch(self, request: Request, call_next):
# response: Response = await call_next(request)
# response.headers["Cross-Origin-Opener-Policy"] = "same-origin"
# response.headers["Cross-Origin-Embedder-Policy"] = "require-corp"
# return response
async def get_function_call_response(messages, tool_id, template, task_model_id, user):
tool = Tools.get_tool_by_id(tool_id)
tools_specs = json.dumps(tool.specs, indent=2)
content = tools_function_calling_generation_template(template, tools_specs)
user_message = get_last_user_message(messages)
prompt = (
"History:\n"
+ "\n".join(
[
f"{message['role'].upper()}: \"\"\"{message['content']}\"\"\""
for message in messages[::-1][:4]
]
)
+ f"\nQuery: {user_message}"
)
print(prompt)
payload = {
"model": task_model_id,
"messages": [
{"role": "system", "content": content},
{"role": "user", "content": f"Query: {prompt}"},
],
"stream": False,
}
try:
payload = filter_pipeline(payload, user)
except Exception as e:
raise e
model = app.state.MODELS[task_model_id]
response = None
try:
if model["owned_by"] == "ollama":
response = await generate_ollama_chat_completion(
OpenAIChatCompletionForm(**payload), user=user
)
else:
response = await generate_openai_chat_completion(payload, user=user)
content = None
if hasattr(response, "body_iterator"):
async for chunk in response.body_iterator:
data = json.loads(chunk.decode("utf-8"))
content = data["choices"][0]["message"]["content"]
# Cleanup any remaining background tasks if necessary
if response.background is not None:
await response.background()
else:
content = response["choices"][0]["message"]["content"]
# Parse the function response
if content is not None:
print(f"content: {content}")
result = json.loads(content)
print(result)
# Call the function
if "name" in result:
if tool_id in webui_app.state.TOOLS:
toolkit_module = webui_app.state.TOOLS[tool_id]
else:
toolkit_module = load_toolkit_module_by_id(tool_id)
webui_app.state.TOOLS[tool_id] = toolkit_module
function = getattr(toolkit_module, result["name"])
function_result = None
try:
# Get the signature of the function
sig = inspect.signature(function)
# Check if '__user__' is a parameter of the function
if "__user__" in sig.parameters:
# Call the function with the '__user__' parameter included
function_result = function(
**{
**result["parameters"],
"__user__": {
"id": user.id,
"email": user.email,
"name": user.name,
"role": user.role,
},
}
)
else:
# Call the function without modifying the parameters
function_result = function(**result["parameters"])
except Exception as e:
print(e)
# Add the function result to the system prompt
if function_result:
return function_result
except Exception as e:
print(f"Error: {e}")
return None
# app.add_middleware(SecurityHeadersMiddleware)
class RAGMiddleware(BaseHTTPMiddleware):
class ChatCompletionMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
return_citations = False
@ -153,35 +290,98 @@ class RAGMiddleware(BaseHTTPMiddleware):
# Parse string to JSON
data = json.loads(body_str) if body_str else {}
user = get_current_user(
get_http_authorization_cred(request.headers.get("Authorization"))
)
# Remove the citations from the body
return_citations = data.get("citations", False)
if "citations" in data:
del data["citations"]
# Example: Add a new key-value pair or modify existing ones
# data["modified"] = True # Example modification
# Set the task model
task_model_id = data["model"]
if task_model_id not in app.state.MODELS:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Model not found",
)
# Check if the user has a custom task model
# If the user has a custom task model, use that model
if app.state.MODELS[task_model_id]["owned_by"] == "ollama":
if (
app.state.config.TASK_MODEL
and app.state.config.TASK_MODEL in app.state.MODELS
):
task_model_id = app.state.config.TASK_MODEL
else:
if (
app.state.config.TASK_MODEL_EXTERNAL
and app.state.config.TASK_MODEL_EXTERNAL in app.state.MODELS
):
task_model_id = app.state.config.TASK_MODEL_EXTERNAL
prompt = get_last_user_message(data["messages"])
context = ""
# If tool_ids field is present, call the functions
if "tool_ids" in data:
print(data["tool_ids"])
for tool_id in data["tool_ids"]:
print(tool_id)
try:
response = await get_function_call_response(
messages=data["messages"],
tool_id=tool_id,
template=app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
task_model_id=task_model_id,
user=user,
)
if response:
context += ("\n" if context != "" else "") + response
except Exception as e:
print(f"Error: {e}")
del data["tool_ids"]
print(f"tool_context: {context}")
# If docs field is present, generate RAG completions
if "docs" in data:
data = {**data}
data["messages"], citations = rag_messages(
rag_context, citations = get_rag_context(
docs=data["docs"],
messages=data["messages"],
template=rag_app.state.config.RAG_TEMPLATE,
embedding_function=rag_app.state.EMBEDDING_FUNCTION,
k=rag_app.state.config.TOP_K,
reranking_function=rag_app.state.sentence_transformer_rf,
r=rag_app.state.config.RELEVANCE_THRESHOLD,
hybrid_search=rag_app.state.config.ENABLE_RAG_HYBRID_SEARCH,
)
if rag_context:
context += ("\n" if context != "" else "") + rag_context
del data["docs"]
log.debug(
f"data['messages']: {data['messages']}, citations: {citations}"
log.debug(f"rag_context: {rag_context}, citations: {citations}")
if context != "":
system_prompt = rag_template(
rag_app.state.config.RAG_TEMPLATE, context, prompt
)
print(system_prompt)
data["messages"] = add_or_update_system_message(
f"\n{system_prompt}", data["messages"]
)
modified_body_bytes = json.dumps(data).encode("utf-8")
# Replace the request body with the modified one
request._body = modified_body_bytes
# Set custom header to ensure content-length matches new body length
request.headers.__dict__["_list"] = [
(b"content-length", str(len(modified_body_bytes)).encode("utf-8")),
@ -224,7 +424,77 @@ class RAGMiddleware(BaseHTTPMiddleware):
yield data
app.add_middleware(RAGMiddleware)
app.add_middleware(ChatCompletionMiddleware)
def filter_pipeline(payload, user):
user = {"id": user.id, "name": user.name, "role": user.role}
model_id = payload["model"]
filters = [
model
for model in app.state.MODELS.values()
if "pipeline" in model
and "type" in model["pipeline"]
and model["pipeline"]["type"] == "filter"
and (
model["pipeline"]["pipelines"] == ["*"]
or any(
model_id == target_model_id
for target_model_id in model["pipeline"]["pipelines"]
)
)
]
sorted_filters = sorted(filters, key=lambda x: x["pipeline"]["priority"])
model = app.state.MODELS[model_id]
if "pipeline" in model:
sorted_filters.append(model)
for filter in sorted_filters:
r = None
try:
urlIdx = filter["urlIdx"]
url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
key = openai_app.state.config.OPENAI_API_KEYS[urlIdx]
if key != "":
headers = {"Authorization": f"Bearer {key}"}
r = requests.post(
f"{url}/{filter['id']}/filter/inlet",
headers=headers,
json={
"user": user,
"body": payload,
},
)
r.raise_for_status()
payload = r.json()
except Exception as e:
# Handle connection error here
print(f"Connection error: {e}")
if r is not None:
try:
res = r.json()
except:
pass
if "detail" in res:
raise Exception(r.status_code, res["detail"])
else:
pass
if "pipeline" not in app.state.MODELS[model_id]:
if "chat_id" in payload:
del payload["chat_id"]
if "title" in payload:
del payload["title"]
return payload
class PipelineMiddleware(BaseHTTPMiddleware):
@ -242,76 +512,17 @@ class PipelineMiddleware(BaseHTTPMiddleware):
# Parse string to JSON
data = json.loads(body_str) if body_str else {}
model_id = data["model"]
filters = [
model
for model in app.state.MODELS.values()
if "pipeline" in model
and "type" in model["pipeline"]
and model["pipeline"]["type"] == "filter"
and (
model["pipeline"]["pipelines"] == ["*"]
or any(
model_id == target_model_id
for target_model_id in model["pipeline"]["pipelines"]
)
user = get_current_user(
get_http_authorization_cred(request.headers.get("Authorization"))
)
try:
data = filter_pipeline(data, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
]
sorted_filters = sorted(filters, key=lambda x: x["pipeline"]["priority"])
user = None
if len(sorted_filters) > 0:
try:
user = get_current_user(
get_http_authorization_cred(
request.headers.get("Authorization")
)
)
user = {"id": user.id, "name": user.name, "role": user.role}
except:
pass
for filter in sorted_filters:
r = None
try:
urlIdx = filter["urlIdx"]
url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
key = openai_app.state.config.OPENAI_API_KEYS[urlIdx]
if key != "":
headers = {"Authorization": f"Bearer {key}"}
r = requests.post(
f"{url}/{filter['id']}/filter/inlet",
headers=headers,
json={
"user": user,
"body": data,
},
)
r.raise_for_status()
data = r.json()
except Exception as e:
# Handle connection error here
print(f"Connection error: {e}")
if r is not None:
try:
res = r.json()
if "detail" in res:
return JSONResponse(
status_code=r.status_code,
content=res,
)
except:
pass
else:
pass
if "chat_id" in data:
del data["chat_id"]
modified_body_bytes = json.dumps(data).encode("utf-8")
# Replace the request body with the modified one
@ -368,6 +579,9 @@ async def update_embedding_function(request: Request, call_next):
return response
app.mount("/ws", socket_app)
app.mount("/ollama", ollama_app)
app.mount("/openai", openai_app)
@ -469,6 +683,237 @@ async def get_models(user=Depends(get_verified_user)):
return {"data": models}
@app.get("/api/task/config")
async def get_task_config(user=Depends(get_verified_user)):
return {
"TASK_MODEL": app.state.config.TASK_MODEL,
"TASK_MODEL_EXTERNAL": app.state.config.TASK_MODEL_EXTERNAL,
"TITLE_GENERATION_PROMPT_TEMPLATE": app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE,
"SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE": app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
"SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD": app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
"TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE": app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
}
class TaskConfigForm(BaseModel):
TASK_MODEL: Optional[str]
TASK_MODEL_EXTERNAL: Optional[str]
TITLE_GENERATION_PROMPT_TEMPLATE: str
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE: str
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD: int
TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE: str
@app.post("/api/task/config/update")
async def update_task_config(form_data: TaskConfigForm, user=Depends(get_admin_user)):
app.state.config.TASK_MODEL = form_data.TASK_MODEL
app.state.config.TASK_MODEL_EXTERNAL = form_data.TASK_MODEL_EXTERNAL
app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE = (
form_data.TITLE_GENERATION_PROMPT_TEMPLATE
)
app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE = (
form_data.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE
)
app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD = (
form_data.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD
)
app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE = (
form_data.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE
)
return {
"TASK_MODEL": app.state.config.TASK_MODEL,
"TASK_MODEL_EXTERNAL": app.state.config.TASK_MODEL_EXTERNAL,
"TITLE_GENERATION_PROMPT_TEMPLATE": app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE,
"SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE": app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
"SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD": app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
"TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE": app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
}
@app.post("/api/task/title/completions")
async def generate_title(form_data: dict, user=Depends(get_verified_user)):
print("generate_title")
model_id = form_data["model"]
if model_id not in app.state.MODELS:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Model not found",
)
# Check if the user has a custom task model
# If the user has a custom task model, use that model
if app.state.MODELS[model_id]["owned_by"] == "ollama":
if app.state.config.TASK_MODEL:
task_model_id = app.state.config.TASK_MODEL
if task_model_id in app.state.MODELS:
model_id = task_model_id
else:
if app.state.config.TASK_MODEL_EXTERNAL:
task_model_id = app.state.config.TASK_MODEL_EXTERNAL
if task_model_id in app.state.MODELS:
model_id = task_model_id
print(model_id)
model = app.state.MODELS[model_id]
template = app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE
content = title_generation_template(
template, form_data["prompt"], user.model_dump()
)
payload = {
"model": model_id,
"messages": [{"role": "user", "content": content}],
"stream": False,
"max_tokens": 50,
"chat_id": form_data.get("chat_id", None),
"title": True,
}
print(payload)
try:
payload = filter_pipeline(payload, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if model["owned_by"] == "ollama":
return await generate_ollama_chat_completion(
OpenAIChatCompletionForm(**payload), user=user
)
else:
return await generate_openai_chat_completion(payload, user=user)
@app.post("/api/task/query/completions")
async def generate_search_query(form_data: dict, user=Depends(get_verified_user)):
print("generate_search_query")
if len(form_data["prompt"]) < app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Skip search query generation for short prompts (< {app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD} characters)",
)
model_id = form_data["model"]
if model_id not in app.state.MODELS:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Model not found",
)
# Check if the user has a custom task model
# If the user has a custom task model, use that model
if app.state.MODELS[model_id]["owned_by"] == "ollama":
if app.state.config.TASK_MODEL:
task_model_id = app.state.config.TASK_MODEL
if task_model_id in app.state.MODELS:
model_id = task_model_id
else:
if app.state.config.TASK_MODEL_EXTERNAL:
task_model_id = app.state.config.TASK_MODEL_EXTERNAL
if task_model_id in app.state.MODELS:
model_id = task_model_id
print(model_id)
model = app.state.MODELS[model_id]
template = app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE
content = search_query_generation_template(
template, form_data["prompt"], user.model_dump()
)
payload = {
"model": model_id,
"messages": [{"role": "user", "content": content}],
"stream": False,
"max_tokens": 30,
}
print(payload)
try:
payload = filter_pipeline(payload, user)
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
if model["owned_by"] == "ollama":
return await generate_ollama_chat_completion(
OpenAIChatCompletionForm(**payload), user=user
)
else:
return await generate_openai_chat_completion(payload, user=user)
@app.post("/api/task/tools/completions")
async def get_tools_function_calling(form_data: dict, user=Depends(get_verified_user)):
print("get_tools_function_calling")
model_id = form_data["model"]
if model_id not in app.state.MODELS:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Model not found",
)
# Check if the user has a custom task model
# If the user has a custom task model, use that model
if app.state.MODELS[model_id]["owned_by"] == "ollama":
if app.state.config.TASK_MODEL:
task_model_id = app.state.config.TASK_MODEL
if task_model_id in app.state.MODELS:
model_id = task_model_id
else:
if app.state.config.TASK_MODEL_EXTERNAL:
task_model_id = app.state.config.TASK_MODEL_EXTERNAL
if task_model_id in app.state.MODELS:
model_id = task_model_id
print(model_id)
template = app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE
try:
context = await get_function_call_response(
form_data["messages"], form_data["tool_id"], template, model_id, user
)
return context
except Exception as e:
return JSONResponse(
status_code=e.args[0],
content={"detail": e.args[1]},
)
@app.post("/api/chat/completions")
async def generate_chat_completions(form_data: dict, user=Depends(get_verified_user)):
model_id = form_data["model"]
if model_id not in app.state.MODELS:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Model not found",
)
model = app.state.MODELS[model_id]
print(model)
if model["owned_by"] == "ollama":
return await generate_ollama_chat_completion(
OpenAIChatCompletionForm(**form_data), user=user
)
else:
return await generate_openai_chat_completion(form_data, user=user)
@app.post("/api/chat/completed")
async def chat_completed(form_data: dict, user=Depends(get_verified_user)):
data = form_data
@ -490,6 +935,13 @@ async def chat_completed(form_data: dict, user=Depends(get_verified_user)):
]
sorted_filters = sorted(filters, key=lambda x: x["pipeline"]["priority"])
print(model_id)
if model_id in app.state.MODELS:
model = app.state.MODELS[model_id]
if "pipeline" in model:
sorted_filters = [model] + sorted_filters
for filter in sorted_filters:
r = None
try:
@ -537,7 +989,11 @@ async def get_pipelines_list(user=Depends(get_admin_user)):
responses = await get_openai_models(raw=True)
print(responses)
urlIdxs = [idx for idx, response in enumerate(responses) if "pipelines" in response]
urlIdxs = [
idx
for idx, response in enumerate(responses)
if response != None and "pipelines" in response
]
return {
"data": [
@ -550,6 +1006,63 @@ async def get_pipelines_list(user=Depends(get_admin_user)):
}
@app.post("/api/pipelines/upload")
async def upload_pipeline(
urlIdx: int = Form(...), file: UploadFile = File(...), user=Depends(get_admin_user)
):
print("upload_pipeline", urlIdx, file.filename)
# Check if the uploaded file is a python file
if not file.filename.endswith(".py"):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Only Python (.py) files are allowed.",
)
upload_folder = f"{CACHE_DIR}/pipelines"
os.makedirs(upload_folder, exist_ok=True)
file_path = os.path.join(upload_folder, file.filename)
try:
# Save the uploaded file
with open(file_path, "wb") as buffer:
shutil.copyfileobj(file.file, buffer)
url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
key = openai_app.state.config.OPENAI_API_KEYS[urlIdx]
headers = {"Authorization": f"Bearer {key}"}
with open(file_path, "rb") as f:
files = {"file": f}
r = requests.post(f"{url}/pipelines/upload", headers=headers, files=files)
r.raise_for_status()
data = r.json()
return {**data}
except Exception as e:
# Handle connection error here
print(f"Connection error: {e}")
detail = "Pipeline not found"
if r is not None:
try:
res = r.json()
if "detail" in res:
detail = res["detail"]
except:
pass
raise HTTPException(
status_code=(r.status_code if r is not None else status.HTTP_404_NOT_FOUND),
detail=detail,
)
finally:
# Ensure the file is deleted after the upload is completed or on failure
if os.path.exists(file_path):
os.remove(file_path)
class AddPipelineForm(BaseModel):
url: str
urlIdx: int
@ -811,11 +1324,20 @@ async def get_app_config():
"auth": WEBUI_AUTH,
"auth_trusted_header": bool(webui_app.state.AUTH_TRUSTED_EMAIL_HEADER),
"enable_signup": webui_app.state.config.ENABLE_SIGNUP,
"enable_web_search": RAG_WEB_SEARCH_ENABLED,
"enable_web_search": rag_app.state.config.ENABLE_RAG_WEB_SEARCH,
"enable_image_generation": images_app.state.config.ENABLED,
"enable_community_sharing": webui_app.state.config.ENABLE_COMMUNITY_SHARING,
"enable_admin_export": ENABLE_ADMIN_EXPORT,
},
"audio": {
"tts": {
"engine": audio_app.state.config.TTS_ENGINE,
"voice": audio_app.state.config.TTS_VOICE,
},
"stt": {
"engine": audio_app.state.config.STT_ENGINE,
},
},
}
@ -860,23 +1382,7 @@ class UrlForm(BaseModel):
async def update_webhook_url(form_data: UrlForm, user=Depends(get_admin_user)):
app.state.config.WEBHOOK_URL = form_data.url
webui_app.state.WEBHOOK_URL = app.state.config.WEBHOOK_URL
return {
"url": app.state.config.WEBHOOK_URL,
}
@app.get("/api/community_sharing", response_model=bool)
async def get_community_sharing_status(request: Request, user=Depends(get_admin_user)):
return webui_app.state.config.ENABLE_COMMUNITY_SHARING
@app.get("/api/community_sharing/toggle", response_model=bool)
async def toggle_community_sharing(request: Request, user=Depends(get_admin_user)):
webui_app.state.config.ENABLE_COMMUNITY_SHARING = (
not webui_app.state.config.ENABLE_COMMUNITY_SHARING
)
return webui_app.state.config.ENABLE_COMMUNITY_SHARING
return {"url": app.state.config.WEBHOOK_URL}
@app.get("/api/version")
@ -894,7 +1400,7 @@ async def get_app_changelog():
@app.get("/api/version/updates")
async def get_app_latest_release_version():
try:
async with aiohttp.ClientSession() as session:
async with aiohttp.ClientSession(trust_env=True) as session:
async with session.get(
"https://api.github.com/repos/open-webui/open-webui/releases/latest"
) as response:

6
backend/requirements.txt
View File

@ -56,4 +56,8 @@ PyJWT[crypto]==2.8.0
black==24.4.2
langfuse==2.33.0
youtube-transcript-api==0.6.2
pytube==15.0.0
pytube==15.0.0
extract_msg
pydub
duckduckgo-search~=6.1.5

4
backend/start.sh
View File

@ -20,12 +20,12 @@ if test "$WEBUI_SECRET_KEY $WEBUI_JWT_SECRET_KEY" = " "; then
WEBUI_SECRET_KEY=$(cat "$KEY_FILE")
fi
if [ "$USE_OLLAMA_DOCKER" = "true" ]; then
if [[ "${USE_OLLAMA_DOCKER,,}" == "true" ]]; then
echo "USE_OLLAMA is set to true, starting ollama serve."
ollama serve &
fi
if [ "$USE_CUDA_DOCKER" = "true" ]; then
if [[ "${USE_CUDA_DOCKER,,}" == "true" ]]; then
echo "CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries."
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib/python3.11/site-packages/torch/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib"
fi

3
backend/start_windows.bat
View File

@ -8,6 +8,7 @@ cd /d "%SCRIPT_DIR%" || exit /b
SET "KEY_FILE=.webui_secret_key"
IF "%PORT%"=="" SET PORT=8080
IF "%HOST%"=="" SET HOST=0.0.0.0
SET "WEBUI_SECRET_KEY=%WEBUI_SECRET_KEY%"
SET "WEBUI_JWT_SECRET_KEY=%WEBUI_JWT_SECRET_KEY%"
@ -29,4 +30,4 @@ IF "%WEBUI_SECRET_KEY%%WEBUI_JWT_SECRET_KEY%" == " " (
:: Execute uvicorn
SET "WEBUI_SECRET_KEY=%WEBUI_SECRET_KEY%"
uvicorn main:app --host 0.0.0.0 --port "%PORT%" --forwarded-allow-ips '*'
uvicorn main:app --host "%HOST%" --port "%PORT%" --forwarded-allow-ips '*'

81
backend/utils/misc.py
View File

@ -3,7 +3,48 @@ import hashlib
import json
import re
from datetime import timedelta
from typing import Optional
from typing import Optional, List
def get_last_user_message(messages: List[dict]) -> str:
for message in reversed(messages):
if message["role"] == "user":
if isinstance(message["content"], list):
for item in message["content"]:
if item["type"] == "text":
return item["text"]
return message["content"]
return None
def get_last_assistant_message(messages: List[dict]) -> str:
for message in reversed(messages):
if message["role"] == "assistant":
if isinstance(message["content"], list):
for item in message["content"]:
if item["type"] == "text":
return item["text"]
return message["content"]
return None
def add_or_update_system_message(content: str, messages: List[dict]):
"""
Adds a new system message at the beginning of the messages list
or updates the existing system message at the beginning.
:param msg: The message to be added or appended.
:param messages: The list of message dictionaries.
:return: The updated list of message dictionaries.
"""
if messages and messages[0].get("role") == "system":
messages[0]["content"] += f"{content}\n{messages[0]['content']}"
else:
# Insert at the beginning
messages.insert(0, {"role": "system", "content": content})
return messages
def get_gravatar_url(email):
@ -123,11 +164,25 @@ def parse_ollama_modelfile(model_text):
"repeat_penalty": float,
"temperature": float,
"seed": int,
"stop": str,
"tfs_z": float,
"num_predict": int,
"top_k": int,
"top_p": float,
"num_keep": int,
"typical_p": float,
"presence_penalty": float,
"frequency_penalty": float,
"penalize_newline": bool,
"numa": bool,
"num_batch": int,
"num_gpu": int,
"main_gpu": int,
"low_vram": bool,
"f16_kv": bool,
"vocab_only": bool,
"use_mmap": bool,
"use_mlock": bool,
"num_thread": int,
}
data = {"base_model_id": None, "params": {}}
@ -156,10 +211,18 @@ def parse_ollama_modelfile(model_text):
param_match = re.search(rf"PARAMETER {param} (.+)", model_text, re.IGNORECASE)
if param_match:
value = param_match.group(1)
if param_type == int:
value = int(value)
elif param_type == float:
value = float(value)
try:
if param_type == int:
value = int(value)
elif param_type == float:
value = float(value)
elif param_type == bool:
value = value.lower() == "true"
except Exception as e:
print(e)
continue
data["params"][param] = value
# Parse adapter
@ -171,8 +234,14 @@ def parse_ollama_modelfile(model_text):
system_desc_match = re.search(
r'SYSTEM\s+"""(.+?)"""', model_text, re.DOTALL | re.IGNORECASE
)
system_desc_match_single = re.search(
r"SYSTEM\s+([^\n]+)", model_text, re.IGNORECASE
)
if system_desc_match:
data["params"]["system"] = system_desc_match.group(1).strip()
elif system_desc_match_single:
data["params"]["system"] = system_desc_match_single.group(1).strip()
# Parse messages
messages = []

10
backend/utils/models.py
View File

@ -1,10 +0,0 @@
from apps.webui.models.models import Models, ModelModel, ModelForm, ModelResponse
def get_model_id_from_custom_model_id(id: str):
model = Models.get_model_by_id(id)
if model:
return model.id
else:
return id

117
backend/utils/task.py Normal file
View File

@ -0,0 +1,117 @@
import re
import math
from datetime import datetime
from typing import Optional
def prompt_template(
template: str, user_name: str = None, current_location: str = None
) -> str:
# Get the current date
current_date = datetime.now()
# Format the date to YYYY-MM-DD
formatted_date = current_date.strftime("%Y-%m-%d")
# Replace {{CURRENT_DATE}} in the template with the formatted date
template = template.replace("{{CURRENT_DATE}}", formatted_date)
if user_name:
# Replace {{USER_NAME}} in the template with the user's name
template = template.replace("{{USER_NAME}}", user_name)
if current_location:
# Replace {{CURRENT_LOCATION}} in the template with the current location
template = template.replace("{{CURRENT_LOCATION}}", current_location)
return template
def title_generation_template(
template: str, prompt: str, user: Optional[dict] = None
) -> str:
def replacement_function(match):
full_match = match.group(0)
start_length = match.group(1)
end_length = match.group(2)
middle_length = match.group(3)
if full_match == "{{prompt}}":
return prompt
elif start_length is not None:
return prompt[: int(start_length)]
elif end_length is not None:
return prompt[-int(end_length) :]
elif middle_length is not None:
middle_length = int(middle_length)
if len(prompt) <= middle_length:
return prompt
start = prompt[: math.ceil(middle_length / 2)]
end = prompt[-math.floor(middle_length / 2) :]
return f"{start}...{end}"
return ""
template = re.sub(
r"{{prompt}}|{{prompt:start:(\d+)}}|{{prompt:end:(\d+)}}|{{prompt:middletruncate:(\d+)}}",
replacement_function,
template,
)
template = prompt_template(
template,
**(
{"user_name": user.get("name"), "current_location": user.get("location")}
if user
else {}
),
)
return template
def search_query_generation_template(
template: str, prompt: str, user: Optional[dict] = None
) -> str:
def replacement_function(match):
full_match = match.group(0)
start_length = match.group(1)
end_length = match.group(2)
middle_length = match.group(3)
if full_match == "{{prompt}}":
return prompt
elif start_length is not None:
return prompt[: int(start_length)]
elif end_length is not None:
return prompt[-int(end_length) :]
elif middle_length is not None:
middle_length = int(middle_length)
if len(prompt) <= middle_length:
return prompt
start = prompt[: math.ceil(middle_length / 2)]
end = prompt[-math.floor(middle_length / 2) :]
return f"{start}...{end}"
return ""
template = re.sub(
r"{{prompt}}|{{prompt:start:(\d+)}}|{{prompt:end:(\d+)}}|{{prompt:middletruncate:(\d+)}}",
replacement_function,
template,
)
template = prompt_template(
template,
**(
{"user_name": user.get("name"), "current_location": user.get("location")}
if user
else {}
),
)
return template
def tools_function_calling_generation_template(template: str, tools_specs: str) -> str:
template = template.replace("{{TOOLS}}", tools_specs)
return template

73
backend/utils/tools.py Normal file
View File

@ -0,0 +1,73 @@
import inspect
from typing import get_type_hints, List, Dict, Any
def doc_to_dict(docstring):
lines = docstring.split("\n")
description = lines[1].strip()
param_dict = {}
for line in lines:
if ":param" in line:
line = line.replace(":param", "").strip()
param, desc = line.split(":", 1)
param_dict[param.strip()] = desc.strip()
ret_dict = {"description": description, "params": param_dict}
return ret_dict
def get_tools_specs(tools) -> List[dict]:
function_list = [
{"name": func, "function": getattr(tools, func)}
for func in dir(tools)
if callable(getattr(tools, func)) and not func.startswith("__")
]
specs = []
for function_item in function_list:
function_name = function_item["name"]
function = function_item["function"]
function_doc = doc_to_dict(function.__doc__ or function_name)
specs.append(
{
"name": function_name,
# TODO: multi-line desc?
"description": function_doc.get("description", function_name),
"parameters": {
"type": "object",
"properties": {
param_name: {
"type": param_annotation.__name__.lower(),
**(
{
"enum": (
str(param_annotation.__args__)
if hasattr(param_annotation, "__args__")
else None
)
}
if hasattr(param_annotation, "__args__")
else {}
),
"description": function_doc.get("params", {}).get(
param_name, param_name
),
}
for param_name, param_annotation in get_type_hints(
function
).items()
if param_name != "return" and param_name != "__user__"
},
"required": [
name
for name, param in inspect.signature(
function
).parameters.items()
if param.default is param.empty
],
},
}
)
return specs

21
cypress/e2e/settings.cy.ts
View File

@ -28,19 +28,6 @@ describe('Settings', () => {
});
});
context('Connections', () => {
it('user can open the Connections modal and hit save', () => {
cy.get('button').contains('Connections').click();
cy.get('button').contains('Save').click();
});
});
context('Models', () => {
it('user can open the Models modal', () => {
cy.get('button').contains('Models').click();
});
});
context('Interface', () => {
it('user can open the Interface modal and hit save', () => {
cy.get('button').contains('Interface').click();
@ -55,14 +42,6 @@ describe('Settings', () => {
});
});
context('Images', () => {
it('user can open the Images modal and hit save', () => {
cy.get('button').contains('Images').click();
// Currently fails because the backend requires a valid URL
// cy.get('button').contains('Save').click();
});
});
context('Chats', () => {
it('user can open the Chats modal', () => {
cy.get('button').contains('Chats').click();

BIN
demo.gif
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 5.0 MiB

After

Width:  |  Height:  |  Size: 4.1 MiB

3
docs/CONTRIBUTING.md
View File

@ -41,10 +41,11 @@ Looking to contribute? Great! Here's how you can help:
We welcome pull requests. Before submitting one, please:
1. Discuss your idea or issue in the [issues section](https://github.com/open-webui/open-webui/issues).
1. Open a discussion regarding your ideas [here](https://github.com/open-webui/open-webui/discussions/new/choose).
2. Follow the project's coding standards and include tests for new features.
3. Update documentation as necessary.
4. Write clear, descriptive commit messages.
5. It's essential to complete your pull request in a timely manner. We move fast, and having PRs hang around too long is not feasible. If you can't get it done within a reasonable time frame, we may have to close it to keep the project moving forward.
### 📚 Documentation & Tutorials

1409
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

11
package.json
View File

@ -1,6 +1,6 @@
{
"name": "open-webui",
"version": "0.2.0.dev3",
"version": "0.3.4",
"private": true,
"scripts": {
"dev": "npm run pyodide:fetch && vite dev --host",
@ -16,7 +16,7 @@
"format:backend": "black . --exclude \".venv/|/venv/\"",
"i18n:parse": "i18next --config i18next-parser.config.ts && prettier --write \"src/lib/i18n/**/*.{js,json}\"",
"cy:open": "cypress open",
"test:frontend": "vitest",
"test:frontend": "vitest --passWithNoTests",
"pyodide:fetch": "node scripts/prepare-pyodide.js"
},
"devDependencies": {
@ -48,10 +48,14 @@
},
"type": "module",
"dependencies": {
"@codemirror/lang-javascript": "^6.2.2",
"@codemirror/lang-python": "^6.1.6",
"@codemirror/theme-one-dark": "^6.1.2",
"@pyscript/core": "^0.4.32",
"@sveltejs/adapter-node": "^1.3.1",
"async": "^3.2.5",
"bits-ui": "^0.19.7",
"codemirror": "^6.0.1",
"dayjs": "^1.11.10",
"eventsource-parser": "^1.1.2",
"file-saver": "^2.0.5",
@ -63,7 +67,10 @@
"js-sha256": "^0.10.1",
"katex": "^0.16.9",
"marked": "^9.1.0",
"mermaid": "^10.9.1",
"pyodide": "^0.26.0-alpha.4",
"socket.io-client": "^4.7.5",
"sortablejs": "^1.15.2",
"svelte-sonner": "^0.3.19",
"tippy.js": "^6.3.7",
"uuid": "^9.0.1"

6
pyproject.toml
View File

@ -26,8 +26,6 @@ dependencies = [
"PyMySQL==1.1.0",
"bcrypt==4.1.3",
"litellm[proxy]==1.37.20",
"boto3==1.34.110",
"argon2-cffi==23.1.0",
@ -66,6 +64,10 @@ dependencies = [
"langfuse==2.33.0",
"youtube-transcript-api==0.6.2",
"pytube==15.0.0",
"extract_msg",
"pydub",
"duckduckgo-search~=6.1.5"
]
readme = "README.md"
requires-python = ">= 3.11, < 3.12.0a1"

88
requirements-dev.lock
View File

@ -12,7 +12,6 @@
aiohttp==3.9.5
# via langchain
# via langchain-community
# via litellm
# via open-webui
aiosignal==1.3.1
# via aiohttp
@ -20,11 +19,9 @@ annotated-types==0.6.0
# via pydantic
anyio==4.3.0
# via httpx
# via openai
# via starlette
# via watchfiles
apscheduler==3.10.4
# via litellm
# via open-webui
argon2-cffi==23.1.0
# via open-webui
@ -38,7 +35,6 @@ av==11.0.0
# via faster-whisper
backoff==2.2.1
# via langfuse
# via litellm
# via posthog
# via unstructured
bcrypt==4.1.3
@ -46,6 +42,7 @@ bcrypt==4.1.3
# via open-webui
# via passlib
beautifulsoup4==4.12.3
# via extract-msg
# via unstructured
bidict==0.23.1
# via python-socketio
@ -83,17 +80,20 @@ chromadb==0.5.0
# via open-webui
click==8.1.7
# via black
# via duckduckgo-search
# via flask
# via litellm
# via nltk
# via peewee-migrate
# via rq
# via typer
# via uvicorn
colorclass==2.2.2
# via oletools
coloredlogs==15.0.1
# via onnxruntime
compressed-rtf==1.0.6
# via extract-msg
cryptography==42.0.7
# via litellm
# via msoffcrypto-tool
# via pyjwt
ctranslate2==4.2.1
# via faster-whisper
@ -109,33 +109,34 @@ defusedxml==0.7.1
deprecated==1.2.14
# via opentelemetry-api
# via opentelemetry-exporter-otlp-proto-grpc
distro==1.9.0
# via openai
dnspython==2.6.1
# via email-validator
docx2txt==0.8
# via open-webui
duckduckgo-search==6.1.5
# via open-webui
easygui==0.98.3
# via oletools
ebcdic==1.1.1
# via extract-msg
ecdsa==0.19.0
# via python-jose
email-validator==2.1.1
# via fastapi
# via pydantic
emoji==2.11.1
# via unstructured
et-xmlfile==1.1.0
# via openpyxl
extract-msg==0.48.5
# via open-webui
fake-useragent==1.5.1
# via open-webui
fastapi==0.111.0
# via chromadb
# via fastapi-sso
# via langchain-chroma
# via litellm
# via open-webui
fastapi-cli==0.0.4
# via fastapi
fastapi-sso==0.10.0
# via litellm
faster-whisper==1.0.2
# via open-webui
filelock==3.14.0
@ -191,8 +192,6 @@ grpcio==1.63.0
# via opentelemetry-exporter-otlp-proto-grpc
grpcio-status==1.62.2
# via google-api-core
gunicorn==22.0.0
# via litellm
h11==0.14.0
# via httpcore
# via uvicorn
@ -206,9 +205,7 @@ httptools==0.6.1
# via uvicorn
httpx==0.27.0
# via fastapi
# via fastapi-sso
# via langfuse
# via openai
huggingface-hub==0.23.0
# via faster-whisper
# via sentence-transformers
@ -225,7 +222,6 @@ idna==3.7
# via unstructured-client
# via yarl
importlib-metadata==7.0.0
# via litellm
# via opentelemetry-api
importlib-resources==6.4.0
# via chromadb
@ -234,7 +230,6 @@ itsdangerous==2.2.0
jinja2==3.1.4
# via fastapi
# via flask
# via litellm
# via torch
jmespath==1.0.1
# via boto3
@ -272,8 +267,8 @@ langsmith==0.1.57
# via langchain
# via langchain-community
# via langchain-core
litellm==1.37.20
# via open-webui
lark==1.1.8
# via rtfde
lxml==5.2.2
# via unstructured
markdown==3.6
@ -294,6 +289,8 @@ monotonic==1.6
# via posthog
mpmath==1.3.0
# via sympy
msoffcrypto-tool==5.4.1
# via oletools
multidict==6.0.5
# via aiohttp
# via yarl
@ -325,15 +322,19 @@ numpy==1.26.4
# via transformers
# via unstructured
oauthlib==3.2.2
# via fastapi-sso
# via kubernetes
# via requests-oauthlib
olefile==0.47
# via extract-msg
# via msoffcrypto-tool
# via oletools
oletools==0.60.1
# via pcodedmp
# via rtfde
onnxruntime==1.17.3
# via chromadb
# via faster-whisper
# via rapidocr-onnxruntime
openai==1.28.1
# via litellm
opencv-python==4.9.0.80
# via rapidocr-onnxruntime
opencv-python-headless==4.9.0.80
@ -375,15 +376,14 @@ ordered-set==4.1.0
# via deepdiff
orjson==3.10.3
# via chromadb
# via duckduckgo-search
# via fastapi
# via langsmith
# via litellm
overrides==7.7.0
# via chromadb
packaging==23.2
# via black
# via build
# via gunicorn
# via huggingface-hub
# via langchain-core
# via langfuse
@ -397,6 +397,8 @@ passlib==1.7.4
# via open-webui
pathspec==0.12.1
# via black
pcodedmp==1.2.6
# via oletools
peewee==3.17.5
# via open-webui
# via peewee-migrate
@ -437,27 +439,27 @@ pycparser==2.22
pydantic==2.7.1
# via chromadb
# via fastapi
# via fastapi-sso
# via google-generativeai
# via langchain
# via langchain-core
# via langfuse
# via langsmith
# via open-webui
# via openai
pydantic-core==2.18.2
# via pydantic
pydub==0.25.1
# via open-webui
pygments==2.18.0
# via rich
pyjwt==2.8.0
# via litellm
# via open-webui
pymysql==1.1.0
# via open-webui
pypandoc==1.13
# via open-webui
pyparsing==3.1.2
pyparsing==2.4.7
# via httplib2
# via oletools
pypdf==4.2.0
# via open-webui
# via unstructured-client
@ -465,6 +467,8 @@ pypika==0.48.9
# via chromadb
pyproject-hooks==1.1.0
# via build
pyreqwest-impersonate==0.4.7
# via duckduckgo-search
python-dateutil==2.9.0.post0
# via botocore
# via kubernetes
@ -472,7 +476,6 @@ python-dateutil==2.9.0.post0
# via posthog
# via unstructured-client
python-dotenv==1.0.1
# via litellm
# via uvicorn
python-engineio==4.9.0
# via python-socketio
@ -484,7 +487,6 @@ python-magic==0.4.27
# via unstructured
python-multipart==0.0.9
# via fastapi
# via litellm
# via open-webui
python-socketio==5.11.2
# via open-webui
@ -503,7 +505,6 @@ pyyaml==6.0.1
# via langchain
# via langchain-community
# via langchain-core
# via litellm
# via rapidocr-onnxruntime
# via transformers
# via uvicorn
@ -513,11 +514,10 @@ rapidfuzz==3.9.0
# via unstructured
rapidocr-onnxruntime==1.3.22
# via open-webui
redis==5.0.4
# via rq
red-black-tree-mod==1.20
# via extract-msg
regex==2024.5.10
# via nltk
# via tiktoken
# via transformers
requests==2.32.2
# via chromadb
@ -527,11 +527,9 @@ requests==2.32.2
# via langchain
# via langchain-community
# via langsmith
# via litellm
# via open-webui
# via posthog
# via requests-oauthlib
# via tiktoken
# via transformers
# via unstructured
# via unstructured-client
@ -540,11 +538,11 @@ requests-oauthlib==2.0.0
# via kubernetes
rich==13.7.1
# via typer
rq==1.16.2
# via litellm
rsa==4.9
# via google-auth
# via python-jose
rtfde==0.1.1
# via extract-msg
s3transfer==0.10.1
# via boto3
safetensors==0.4.3
@ -577,7 +575,6 @@ six==1.16.0
sniffio==1.3.1
# via anyio
# via httpx
# via openai
soupsieve==2.5
# via beautifulsoup4
sqlalchemy==2.0.30
@ -597,12 +594,9 @@ tenacity==8.3.0
# via langchain-core
threadpoolctl==3.5.0
# via scikit-learn
tiktoken==0.6.0
# via litellm
tokenizers==0.15.2
# via chromadb
# via faster-whisper
# via litellm
# via transformers
torch==2.3.0
# via sentence-transformers
@ -611,7 +605,6 @@ tqdm==4.66.4
# via google-generativeai
# via huggingface-hub
# via nltk
# via openai
# via sentence-transformers
# via transformers
transformers==4.39.3
@ -624,7 +617,6 @@ typing-extensions==4.11.0
# via fastapi
# via google-generativeai
# via huggingface-hub
# via openai
# via opentelemetry-sdk
# via pydantic
# via pydantic-core
@ -641,6 +633,7 @@ tzdata==2024.1
# via pandas
tzlocal==5.2
# via apscheduler
# via extract-msg
ujson==5.10.0
# via fastapi
unstructured==0.14.0
@ -657,7 +650,6 @@ urllib3==2.2.1
uvicorn==0.22.0
# via chromadb
# via fastapi
# via litellm
# via open-webui
uvloop==0.19.0
# via uvicorn

88
requirements.lock
View File

@ -12,7 +12,6 @@
aiohttp==3.9.5
# via langchain
# via langchain-community
# via litellm
# via open-webui
aiosignal==1.3.1
# via aiohttp
@ -20,11 +19,9 @@ annotated-types==0.6.0
# via pydantic
anyio==4.3.0
# via httpx
# via openai
# via starlette
# via watchfiles
apscheduler==3.10.4
# via litellm
# via open-webui
argon2-cffi==23.1.0
# via open-webui
@ -38,7 +35,6 @@ av==11.0.0
# via faster-whisper
backoff==2.2.1
# via langfuse
# via litellm
# via posthog
# via unstructured
bcrypt==4.1.3
@ -46,6 +42,7 @@ bcrypt==4.1.3
# via open-webui
# via passlib
beautifulsoup4==4.12.3
# via extract-msg
# via unstructured
bidict==0.23.1
# via python-socketio
@ -83,17 +80,20 @@ chromadb==0.5.0
# via open-webui
click==8.1.7
# via black
# via duckduckgo-search
# via flask
# via litellm
# via nltk
# via peewee-migrate
# via rq
# via typer
# via uvicorn
colorclass==2.2.2
# via oletools
coloredlogs==15.0.1
# via onnxruntime
compressed-rtf==1.0.6
# via extract-msg
cryptography==42.0.7
# via litellm
# via msoffcrypto-tool
# via pyjwt
ctranslate2==4.2.1
# via faster-whisper
@ -109,33 +109,34 @@ defusedxml==0.7.1
deprecated==1.2.14
# via opentelemetry-api
# via opentelemetry-exporter-otlp-proto-grpc
distro==1.9.0
# via openai
dnspython==2.6.1
# via email-validator
docx2txt==0.8
# via open-webui
duckduckgo-search==6.1.5
# via open-webui
easygui==0.98.3
# via oletools
ebcdic==1.1.1
# via extract-msg
ecdsa==0.19.0
# via python-jose
email-validator==2.1.1
# via fastapi
# via pydantic
emoji==2.11.1
# via unstructured
et-xmlfile==1.1.0
# via openpyxl
extract-msg==0.48.5
# via open-webui
fake-useragent==1.5.1
# via open-webui
fastapi==0.111.0
# via chromadb
# via fastapi-sso
# via langchain-chroma
# via litellm
# via open-webui
fastapi-cli==0.0.4
# via fastapi
fastapi-sso==0.10.0
# via litellm
faster-whisper==1.0.2
# via open-webui
filelock==3.14.0
@ -191,8 +192,6 @@ grpcio==1.63.0
# via opentelemetry-exporter-otlp-proto-grpc
grpcio-status==1.62.2
# via google-api-core
gunicorn==22.0.0
# via litellm
h11==0.14.0
# via httpcore
# via uvicorn
@ -206,9 +205,7 @@ httptools==0.6.1
# via uvicorn
httpx==0.27.0
# via fastapi
# via fastapi-sso
# via langfuse
# via openai
huggingface-hub==0.23.0
# via faster-whisper
# via sentence-transformers
@ -225,7 +222,6 @@ idna==3.7
# via unstructured-client
# via yarl
importlib-metadata==7.0.0
# via litellm
# via opentelemetry-api
importlib-resources==6.4.0
# via chromadb
@ -234,7 +230,6 @@ itsdangerous==2.2.0
jinja2==3.1.4
# via fastapi
# via flask
# via litellm
# via torch
jmespath==1.0.1
# via boto3
@ -272,8 +267,8 @@ langsmith==0.1.57
# via langchain
# via langchain-community
# via langchain-core
litellm==1.37.20
# via open-webui
lark==1.1.8
# via rtfde
lxml==5.2.2
# via unstructured
markdown==3.6
@ -294,6 +289,8 @@ monotonic==1.6
# via posthog
mpmath==1.3.0
# via sympy
msoffcrypto-tool==5.4.1
# via oletools
multidict==6.0.5
# via aiohttp
# via yarl
@ -325,15 +322,19 @@ numpy==1.26.4
# via transformers
# via unstructured
oauthlib==3.2.2
# via fastapi-sso
# via kubernetes
# via requests-oauthlib
olefile==0.47
# via extract-msg
# via msoffcrypto-tool
# via oletools
oletools==0.60.1
# via pcodedmp
# via rtfde
onnxruntime==1.17.3
# via chromadb
# via faster-whisper
# via rapidocr-onnxruntime
openai==1.28.1
# via litellm
opencv-python==4.9.0.80
# via rapidocr-onnxruntime
opencv-python-headless==4.9.0.80
@ -375,15 +376,14 @@ ordered-set==4.1.0
# via deepdiff
orjson==3.10.3
# via chromadb
# via duckduckgo-search
# via fastapi
# via langsmith
# via litellm
overrides==7.7.0
# via chromadb
packaging==23.2
# via black
# via build
# via gunicorn
# via huggingface-hub
# via langchain-core
# via langfuse
@ -397,6 +397,8 @@ passlib==1.7.4
# via open-webui
pathspec==0.12.1
# via black
pcodedmp==1.2.6
# via oletools
peewee==3.17.5
# via open-webui
# via peewee-migrate
@ -437,27 +439,27 @@ pycparser==2.22
pydantic==2.7.1
# via chromadb
# via fastapi
# via fastapi-sso
# via google-generativeai
# via langchain
# via langchain-core
# via langfuse
# via langsmith
# via open-webui
# via openai
pydantic-core==2.18.2
# via pydantic
pydub==0.25.1
# via open-webui
pygments==2.18.0
# via rich
pyjwt==2.8.0
# via litellm
# via open-webui
pymysql==1.1.0
# via open-webui
pypandoc==1.13
# via open-webui
pyparsing==3.1.2
pyparsing==2.4.7
# via httplib2
# via oletools
pypdf==4.2.0
# via open-webui
# via unstructured-client
@ -465,6 +467,8 @@ pypika==0.48.9
# via chromadb
pyproject-hooks==1.1.0
# via build
pyreqwest-impersonate==0.4.7
# via duckduckgo-search
python-dateutil==2.9.0.post0
# via botocore
# via kubernetes
@ -472,7 +476,6 @@ python-dateutil==2.9.0.post0
# via posthog
# via unstructured-client
python-dotenv==1.0.1
# via litellm
# via uvicorn
python-engineio==4.9.0
# via python-socketio
@ -484,7 +487,6 @@ python-magic==0.4.27
# via unstructured
python-multipart==0.0.9
# via fastapi
# via litellm
# via open-webui
python-socketio==5.11.2
# via open-webui
@ -503,7 +505,6 @@ pyyaml==6.0.1
# via langchain
# via langchain-community
# via langchain-core
# via litellm
# via rapidocr-onnxruntime
# via transformers
# via uvicorn
@ -513,11 +514,10 @@ rapidfuzz==3.9.0
# via unstructured
rapidocr-onnxruntime==1.3.22
# via open-webui
redis==5.0.4
# via rq
red-black-tree-mod==1.20
# via extract-msg
regex==2024.5.10
# via nltk
# via tiktoken
# via transformers
requests==2.32.2
# via chromadb
@ -527,11 +527,9 @@ requests==2.32.2
# via langchain
# via langchain-community
# via langsmith
# via litellm
# via open-webui
# via posthog
# via requests-oauthlib
# via tiktoken
# via transformers
# via unstructured
# via unstructured-client
@ -540,11 +538,11 @@ requests-oauthlib==2.0.0
# via kubernetes
rich==13.7.1
# via typer
rq==1.16.2
# via litellm
rsa==4.9
# via google-auth
# via python-jose
rtfde==0.1.1
# via extract-msg
s3transfer==0.10.1
# via boto3
safetensors==0.4.3
@ -577,7 +575,6 @@ six==1.16.0
sniffio==1.3.1
# via anyio
# via httpx
# via openai
soupsieve==2.5
# via beautifulsoup4
sqlalchemy==2.0.30
@ -597,12 +594,9 @@ tenacity==8.3.0
# via langchain-core
threadpoolctl==3.5.0
# via scikit-learn
tiktoken==0.6.0
# via litellm
tokenizers==0.15.2
# via chromadb
# via faster-whisper
# via litellm
# via transformers
torch==2.3.0
# via sentence-transformers
@ -611,7 +605,6 @@ tqdm==4.66.4
# via google-generativeai
# via huggingface-hub
# via nltk
# via openai
# via sentence-transformers
# via transformers
transformers==4.39.3
@ -624,7 +617,6 @@ typing-extensions==4.11.0
# via fastapi
# via google-generativeai
# via huggingface-hub
# via openai
# via opentelemetry-sdk
# via pydantic
# via pydantic-core
@ -641,6 +633,7 @@ tzdata==2024.1
# via pandas
tzlocal==5.2
# via apscheduler
# via extract-msg
ujson==5.10.0
# via fastapi
unstructured==0.14.0
@ -657,7 +650,6 @@ urllib3==2.2.1
uvicorn==0.22.0
# via chromadb
# via fastapi
# via litellm
# via open-webui
uvloop==0.19.0
# via uvicorn

21
src/app.css
View File

@ -92,10 +92,18 @@ select {
visibility: hidden;
}
.scrollbar-hidden::-webkit-scrollbar-corner {
display: none;
}
.scrollbar-none::-webkit-scrollbar {
display: none; /* for Chrome, Safari and Opera */
}
.scrollbar-none::-webkit-scrollbar-corner {
display: none;
}
.scrollbar-none {
-ms-overflow-style: none; /* IE and Edge */
scrollbar-width: none; /* Firefox */
@ -111,3 +119,16 @@ input::-webkit-inner-spin-button {
input[type='number'] {
-moz-appearance: textfield; /* Firefox */
}
.cm-editor {
height: 100%;
width: 100%;
}
.cm-scroller {
@apply scrollbar-hidden;
}
.cm-editor.cm-focused {
outline: none;
}

135
src/app.html
View File

@ -32,6 +32,9 @@
} else if (localStorage.theme && localStorage.theme === 'system') {
systemTheme = window.matchMedia('(prefers-color-scheme: dark)').matches;
document.documentElement.classList.add(systemTheme ? 'dark' : 'light');
} else if (localStorage.theme && localStorage.theme === 'her') {
document.documentElement.classList.add('dark');
document.documentElement.classList.add('her');
} else {
document.documentElement.classList.add('dark');
}
@ -59,15 +62,7 @@
<div
id="splash-screen"
style="
position: fixed;
z-index: 100;
background: #fff;
top: 0;
left: 0;
width: 100%;
height: 100%;
"
style="position: fixed; z-index: 100; top: 0; left: 0; width: 100%; height: 100%"
>
<style type="text/css" nonce="">
html {
@ -76,20 +71,138 @@
</style>
<img
id="logo"
style="
position: absolute;
width: 6rem;
height: 6rem;
top: 46%;
top: 41%;
left: 50%;
margin: -40px 0 0 -40px;
margin-left: -3rem;
"
src="/logo.svg"
/>
<div
style="
position: absolute;
top: 33%;
left: 50%;
width: 24rem;
margin-left: -12rem;
display: flex;
flex-direction: column;
align-items: center;
"
>
<img
id="logo-her"
style="width: 13rem; height: 13rem"
src="/logo.svg"
class="animate-pulse-fast"
/>
<div style="position: relative; width: 24rem; margin-top: 0.5rem">
<div
id="progress-background"
style="
position: absolute;
width: 100%;
height: 0.75rem;
border-radius: 9999px;
background-color: #fafafa9a;
"
></div>
<div
id="progress-bar"
style="
position: absolute;
width: 0%;
height: 0.75rem;
border-radius: 9999px;
background-color: #fff;
"
class="bg-white"
></div>
</div>
</div>
<!-- <span style="position: absolute; bottom: 32px; left: 50%; margin: -36px 0 0 -36px">
Footer content
</span> -->
</div>
</body>
</html>
<style type="text/css" nonce="">
html {
overflow-y: hidden !important;
}
#splash-screen {
background: #fff;
}
html.dark #splash-screen {
background: #000;
}
html.dark #splash-screen img {
filter: invert(1);
}
html.her #splash-screen {
background: #983724;
}
#logo-her {
display: none;
}
#progress-background {
display: none;
}
#progress-bar {
display: none;
}
html.her #logo {
display: none;
}
html.her #logo-her {
display: block;
filter: invert(1);
}
html.her #progress-background {
display: block;
}
html.her #progress-bar {
display: block;
}
@media (max-width: 24rem) {
html.her #progress-background {
display: none;
}
html.her #progress-bar {
display: none;
}
}
@keyframes pulse {
50% {
opacity: 0.65;
}
}
.animate-pulse-fast {
animation: pulse 1.5s cubic-bezier(0.4, 0, 0.6, 1) infinite;
}
</style>

6
src/lib/apis/audio/index.ts
View File

@ -98,7 +98,7 @@ export const synthesizeOpenAISpeech = async (
token: string = '',
speaker: string = 'alloy',
text: string = '',
model: string = 'tts-1'
model?: string
) => {
let error = null;
@ -109,9 +109,9 @@ export const synthesizeOpenAISpeech = async (
'Content-Type': 'application/json'
},
body: JSON.stringify({
model: model,
input: text,
voice: speaker
voice: speaker,
...(model && { model })
})
})
.then(async (res) => {

82
src/lib/apis/auths/index.ts
View File

@ -1,5 +1,87 @@
import { WEBUI_API_BASE_URL } from '$lib/constants';
export const getAdminDetails = async (token: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/auths/admin/details`, {
method: 'GET',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
error = err.detail;
return null;
});
if (error) {
throw error;
}
return res;
};
export const getAdminConfig = async (token: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/auths/admin/config`, {
method: 'GET',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
error = err.detail;
return null;
});
if (error) {
throw error;
}
return res;
};
export const updateAdminConfig = async (token: string, body: object) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/auths/admin/config`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify(body)
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
error = err.detail;
return null;
});
if (error) {
throw error;
}
return res;
};
export const getSessionUser = async (token: string) => {
let error = null;

69
src/lib/apis/chats/index.ts
View File

@ -162,6 +162,37 @@ export const getAllChats = async (token: string) => {
return res;
};
export const getAllArchivedChats = async (token: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/chats/all/archived`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...(token && { authorization: `Bearer ${token}` })
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const getAllUserChats = async (token: string) => {
let error = null;
@ -325,6 +356,44 @@ export const getChatByShareId = async (token: string, share_id: string) => {
return res;
};
export const cloneChatById = async (token: string, id: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/chats/${id}/clone`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...(token && { authorization: `Bearer ${token}` })
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err;
if ('detail' in err) {
error = err.detail;
} else {
error = err;
}
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const shareChatById = async (token: string, id: string) => {
let error = null;

20
src/lib/apis/documents/index.ts
View File

@ -76,7 +76,10 @@ export const getDocs = async (token: string = '') => {
export const getDocByName = async (token: string, name: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}`, {
const searchParams = new URLSearchParams();
searchParams.append('name', name);
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/docs?${searchParams.toString()}`, {
method: 'GET',
headers: {
Accept: 'application/json',
@ -113,7 +116,10 @@ type DocUpdateForm = {
export const updateDocByName = async (token: string, name: string, form: DocUpdateForm) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}/update`, {
const searchParams = new URLSearchParams();
searchParams.append('name', name);
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/doc/update?${searchParams.toString()}`, {
method: 'POST',
headers: {
Accept: 'application/json',
@ -154,7 +160,10 @@ type TagDocForm = {
export const tagDocByName = async (token: string, name: string, form: TagDocForm) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}/tags`, {
const searchParams = new URLSearchParams();
searchParams.append('name', name);
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/doc/tags?${searchParams.toString()}`, {
method: 'POST',
headers: {
Accept: 'application/json',
@ -190,7 +199,10 @@ export const tagDocByName = async (token: string, name: string, form: TagDocForm
export const deleteDocByName = async (token: string, name: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}/delete`, {
const searchParams = new URLSearchParams();
searchParams.append('name', name);
const res = await fetch(`${WEBUI_API_BASE_URL}/documents/doc/delete?${searchParams.toString()}`, {
method: 'DELETE',
headers: {
Accept: 'application/json',

200
src/lib/apis/index.ts
View File

@ -29,8 +29,24 @@ export const getModels = async (token: string = '') => {
models = models
.filter((models) => models)
// Sort the models
.sort((a, b) => {
// Compare case-insensitively
// Check if models have position property
const aHasPosition = a.info?.meta?.position !== undefined;
const bHasPosition = b.info?.meta?.position !== undefined;
// If both a and b have the position property
if (aHasPosition && bHasPosition) {
return a.info.meta.position - b.info.meta.position;
}
// If only a has the position property, it should come first
if (aHasPosition) return -1;
// If only b has the position property, it should come first
if (bHasPosition) return 1;
// Compare case-insensitively by name for models without position property
const lowerA = a.name.toLowerCase();
const lowerB = b.name.toLowerCase();
@ -39,8 +55,8 @@ export const getModels = async (token: string = '') => {
// If same case-insensitively, sort by original strings,
// lowercase will come before uppercase due to ASCII values
if (a < b) return -1;
if (a > b) return 1;
if (a.name < b.name) return -1;
if (a.name > b.name) return 1;
return 0; // They are equal
});
@ -88,6 +104,147 @@ export const chatCompleted = async (token: string, body: ChatCompletedForm) => {
return res;
};
export const getTaskConfig = async (token: string = '') => {
let error = null;
const res = await fetch(`${WEBUI_BASE_URL}/api/task/config`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...(token && { authorization: `Bearer ${token}` })
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
error = err;
return null;
});
if (error) {
throw error;
}
return res;
};
export const updateTaskConfig = async (token: string, config: object) => {
let error = null;
const res = await fetch(`${WEBUI_BASE_URL}/api/task/config/update`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...(token && { authorization: `Bearer ${token}` })
},
body: JSON.stringify(config)
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
if ('detail' in err) {
error = err.detail;
} else {
error = err;
}
return null;
});
if (error) {
throw error;
}
return res;
};
export const generateTitle = async (
token: string = '',
model: string,
prompt: string,
chat_id?: string
) => {
let error = null;
const res = await fetch(`${WEBUI_BASE_URL}/api/task/title/completions`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
model: model,
prompt: prompt,
...(chat_id && { chat_id: chat_id })
})
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
if ('detail' in err) {
error = err.detail;
}
return null;
});
if (error) {
throw error;
}
return res?.choices[0]?.message?.content.replace(/["']/g, '') ?? 'New Chat';
};
export const generateSearchQuery = async (
token: string = '',
model: string,
messages: object[],
prompt: string
) => {
let error = null;
const res = await fetch(`${WEBUI_BASE_URL}/api/task/query/completions`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
model: model,
messages: messages,
prompt: prompt
})
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
if ('detail' in err) {
error = err.detail;
}
return null;
});
if (error) {
throw error;
}
return res?.choices[0]?.message?.content.replace(/["']/g, '') ?? prompt;
};
export const getPipelinesList = async (token: string = '') => {
let error = null;
@ -117,6 +274,43 @@ export const getPipelinesList = async (token: string = '') => {
return pipelines;
};
export const uploadPipeline = async (token: string, file: File, urlIdx: string) => {
let error = null;
// Create a new FormData object to handle the file upload
const formData = new FormData();
formData.append('file', file);
formData.append('urlIdx', urlIdx);
const res = await fetch(`${WEBUI_BASE_URL}/api/pipelines/upload`, {
method: 'POST',
headers: {
...(token && { authorization: `Bearer ${token}` })
// 'Content-Type': 'multipart/form-data' is not needed as Fetch API will set it automatically
},
body: formData
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
if ('detail' in err) {
error = err.detail;
} else {
error = err;
}
return null;
});
if (error) {
throw error;
}
return res;
};
export const downloadPipeline = async (token: string, url: string, urlIdx: string) => {
let error = null;

2
src/lib/apis/memories/index.ts
View File

@ -3,7 +3,7 @@ import { WEBUI_API_BASE_URL } from '$lib/constants';
export const getMemories = async (token: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/memories`, {
const res = await fetch(`${WEBUI_API_BASE_URL}/memories/`, {
method: 'GET',
headers: {
Accept: 'application/json',

61
src/lib/apis/ollama/index.ts
View File

@ -1,5 +1,5 @@
import { OLLAMA_API_BASE_URL } from '$lib/constants';
import { promptTemplate } from '$lib/utils';
import { titleGenerationTemplate } from '$lib/utils';
export const getOllamaConfig = async (token: string = '') => {
let error = null;
@ -212,7 +212,7 @@ export const generateTitle = async (
) => {
let error = null;
template = promptTemplate(template, prompt);
template = titleGenerationTemplate(template, prompt);
console.log(template);
@ -369,42 +369,29 @@ export const generateChatCompletion = async (token: string = '', body: object) =
return [res, controller];
};
export const cancelOllamaRequest = async (token: string = '', requestId: string) => {
export const createModel = async (
token: string,
tagName: string,
content: string,
urlIdx: string | null = null
) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/cancel/${requestId}`, {
method: 'GET',
headers: {
'Content-Type': 'text/event-stream',
Authorization: `Bearer ${token}`
const res = await fetch(
`${OLLAMA_API_BASE_URL}/api/create${urlIdx !== null ? `/${urlIdx}` : ''}`,
{
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
name: tagName,
modelfile: content
})
}
}).catch((err) => {
error = err;
return null;
});
if (error) {
throw error;
}
return res;
};
export const createModel = async (token: string, tagName: string, content: string) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/create`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
name: tagName,
modelfile: content
})
}).catch((err) => {
).catch((err) => {
error = err;
return null;
});
@ -461,8 +448,10 @@ export const deleteModel = async (token: string, tagName: string, urlIdx: string
export const pullModel = async (token: string, tagName: string, urlIdx: string | null = null) => {
let error = null;
const controller = new AbortController();
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/pull${urlIdx !== null ? `/${urlIdx}` : ''}`, {
signal: controller.signal,
method: 'POST',
headers: {
Accept: 'application/json',
@ -485,7 +474,7 @@ export const pullModel = async (token: string, tagName: string, urlIdx: string |
if (error) {
throw error;
}
return res;
return [res, controller];
};
export const downloadModel = async (

9
src/lib/apis/openai/index.ts
View File

@ -1,5 +1,5 @@
import { OPENAI_API_BASE_URL } from '$lib/constants';
import { promptTemplate } from '$lib/utils';
import { titleGenerationTemplate } from '$lib/utils';
import { type Model, models, settings } from '$lib/stores';
export const getOpenAIConfig = async (token: string = '') => {
@ -336,11 +336,12 @@ export const generateTitle = async (
template: string,
model: string,
prompt: string,
chat_id?: string,
url: string = OPENAI_API_BASE_URL
) => {
let error = null;
template = promptTemplate(template, prompt);
template = titleGenerationTemplate(template, prompt);
console.log(template);
@ -361,7 +362,9 @@ export const generateTitle = async (
],
stream: false,
// Restricting the max tokens to 50 to avoid long titles
max_tokens: 50
max_tokens: 50,
...(chat_id && { chat_id: chat_id }),
title: true
})
})
.then(async (res) => {

42
src/lib/apis/rag/index.ts
View File

@ -359,6 +359,32 @@ export const scanDocs = async (token: string) => {
return res;
};
export const resetUploadDir = async (token: string) => {
let error = null;
const res = await fetch(`${RAG_API_BASE_URL}/reset/uploads`, {
method: 'GET',
headers: {
Accept: 'application/json',
authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
error = err.detail;
return null;
});
if (error) {
throw error;
}
return res;
};
export const resetVectorDB = async (token: string) => {
let error = null;
@ -415,6 +441,7 @@ export const getEmbeddingConfig = async (token: string) => {
type OpenAIConfigForm = {
key: string;
url: string;
batch_size: number;
};
type EmbeddingModelUpdateForm = {
@ -518,8 +545,10 @@ export const runWebSearch = async (
token: string,
query: string,
collection_name?: string
): Promise<SearchDocument | undefined> => {
return await fetch(`${RAG_API_BASE_URL}/web/search`, {
): Promise<SearchDocument | null> => {
let error = null;
const res = await fetch(`${RAG_API_BASE_URL}/web/search`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
@ -536,8 +565,15 @@ export const runWebSearch = async (
})
.catch((err) => {
console.log(err);
return undefined;
error = err.detail;
return null;
});
if (error) {
throw error;
}
return res;
};
export interface SearchDocument {

193
src/lib/apis/tools/index.ts Normal file
View File

@ -0,0 +1,193 @@
import { WEBUI_API_BASE_URL } from '$lib/constants';
export const createNewTool = async (token: string, tool: object) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/tools/create`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
authorization: `Bearer ${token}`
},
body: JSON.stringify({
...tool
})
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
error = err.detail;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const getTools = async (token: string = '') => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/tools/`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err.detail;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const exportTools = async (token: string = '') => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/tools/export`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err.detail;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const getToolById = async (token: string, id: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/tools/id/${id}`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err.detail;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const updateToolById = async (token: string, id: string, tool: object) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/tools/id/${id}/update`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
authorization: `Bearer ${token}`
},
body: JSON.stringify({
...tool
})
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err.detail;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};
export const deleteToolById = async (token: string, id: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/tools/id/${id}/delete`, {
method: 'DELETE',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
authorization: `Bearer ${token}`
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.then((json) => {
return json;
})
.catch((err) => {
error = err.detail;
console.log(err);
return null;
});
if (error) {
throw error;
}
return res;
};

69
src/lib/apis/utils/index.ts
View File

@ -22,6 +22,39 @@ export const getGravatarUrl = async (email: string) => {
return res;
};
export const formatPythonCode = async (code: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/utils/code/format`, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
code: code
})
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
error = err;
if (err.detail) {
error = err.detail;
}
return null;
});
if (error) {
throw error;
}
return res;
};
export const downloadChatAsPDF = async (chat: object) => {
let error = null;
@ -108,3 +141,39 @@ export const downloadDatabase = async (token: string) => {
throw error;
}
};
export const downloadLiteLLMConfig = async (token: string) => {
let error = null;
const res = await fetch(`${WEBUI_API_BASE_URL}/utils/litellm/config`, {
method: 'GET',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
}
})
.then(async (response) => {
if (!response.ok) {
throw await response.json();
}
return response.blob();
})
.then((blob) => {
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'config.yaml';
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
})
.catch((err) => {
console.log(err);
error = err.detail;
return null;
});
if (error) {
throw error;
}
};

390
src/lib/components/admin/Settings.svelte Normal file
View File

@ -0,0 +1,390 @@
<script>
import { getContext, tick } from 'svelte';
import { toast } from 'svelte-sonner';
import Database from './Settings/Database.svelte';
import General from './Settings/General.svelte';
import Users from './Settings/Users.svelte';
import Pipelines from './Settings/Pipelines.svelte';
import Audio from './Settings/Audio.svelte';
import Images from './Settings/Images.svelte';
import Interface from './Settings/Interface.svelte';
import Models from './Settings/Models.svelte';
import Connections from './Settings/Connections.svelte';
import Documents from './Settings/Documents.svelte';
import WebSearch from './Settings/WebSearch.svelte';
import { config } from '$lib/stores';
import { getBackendConfig } from '$lib/apis';
const i18n = getContext('i18n');
let selectedTab = 'general';
</script>
<div class="flex flex-col lg:flex-row w-full h-full py-2 lg:space-x-4">
<div
class="tabs flex flex-row overflow-x-auto space-x-1 max-w-full lg:space-x-0 lg:space-y-1 lg:flex-col lg:flex-none lg:w-44 dark:text-gray-200 text-xs text-left scrollbar-none"
>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 lg:flex-none flex text-right transition {selectedTab ===
'general'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'general';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M6.955 1.45A.5.5 0 0 1 7.452 1h1.096a.5.5 0 0 1 .497.45l.17 1.699c.484.12.94.312 1.356.562l1.321-1.081a.5.5 0 0 1 .67.033l.774.775a.5.5 0 0 1 .034.67l-1.08 1.32c.25.417.44.873.561 1.357l1.699.17a.5.5 0 0 1 .45.497v1.096a.5.5 0 0 1-.45.497l-1.699.17c-.12.484-.312.94-.562 1.356l1.082 1.322a.5.5 0 0 1-.034.67l-.774.774a.5.5 0 0 1-.67.033l-1.322-1.08c-.416.25-.872.44-1.356.561l-.17 1.699a.5.5 0 0 1-.497.45H7.452a.5.5 0 0 1-.497-.45l-.17-1.699a4.973 4.973 0 0 1-1.356-.562L4.108 13.37a.5.5 0 0 1-.67-.033l-.774-.775a.5.5 0 0 1-.034-.67l1.08-1.32a4.971 4.971 0 0 1-.561-1.357l-1.699-.17A.5.5 0 0 1 1 8.548V7.452a.5.5 0 0 1 .45-.497l1.699-.17c.12-.484.312-.94.562-1.356L2.629 4.107a.5.5 0 0 1 .034-.67l.774-.774a.5.5 0 0 1 .67-.033L5.43 3.71a4.97 4.97 0 0 1 1.356-.561l.17-1.699ZM6 8c0 .538.212 1.026.558 1.385l.057.057a2 2 0 0 0 2.828-2.828l-.058-.056A2 2 0 0 0 6 8Z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('General')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'users'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'users';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M8 8a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5ZM3.156 11.763c.16-.629.44-1.21.813-1.72a2.5 2.5 0 0 0-2.725 1.377c-.136.287.102.58.418.58h1.449c.01-.077.025-.156.045-.237ZM12.847 11.763c.02.08.036.16.046.237h1.446c.316 0 .554-.293.417-.579a2.5 2.5 0 0 0-2.722-1.378c.374.51.653 1.09.813 1.72ZM14 7.5a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0ZM3.5 9a1.5 1.5 0 1 0 0-3 1.5 1.5 0 0 0 0 3ZM5 13c-.552 0-1.013-.455-.876-.99a4.002 4.002 0 0 1 7.753 0c.136.535-.324.99-.877.99H5Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Users')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'connections'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'connections';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M1 9.5A3.5 3.5 0 0 0 4.5 13H12a3 3 0 0 0 .917-5.857 2.503 2.503 0 0 0-3.198-3.019 3.5 3.5 0 0 0-6.628 2.171A3.5 3.5 0 0 0 1 9.5Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Connections')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'models'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'models';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M10 1c3.866 0 7 1.79 7 4s-3.134 4-7 4-7-1.79-7-4 3.134-4 7-4zm5.694 8.13c.464-.264.91-.583 1.306-.952V10c0 2.21-3.134 4-7 4s-7-1.79-7-4V8.178c.396.37.842.688 1.306.953C5.838 10.006 7.854 10.5 10 10.5s4.162-.494 5.694-1.37zM3 13.179V15c0 2.21 3.134 4 7 4s7-1.79 7-4v-1.822c-.396.37-.842.688-1.306.953-1.532.875-3.548 1.369-5.694 1.369s-4.162-.494-5.694-1.37A7.009 7.009 0 013 13.179z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Models')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'documents'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'documents';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="w-4 h-4"
>
<path d="M11.625 16.5a1.875 1.875 0 1 0 0-3.75 1.875 1.875 0 0 0 0 3.75Z" />
<path
fill-rule="evenodd"
d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875Zm6 16.5c.66 0 1.277-.19 1.797-.518l1.048 1.048a.75.75 0 0 0 1.06-1.06l-1.047-1.048A3.375 3.375 0 1 0 11.625 18Z"
clip-rule="evenodd"
/>
<path
d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Documents')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'web'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'web';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M21.721 12.752a9.711 9.711 0 0 0-.945-5.003 12.754 12.754 0 0 1-4.339 2.708 18.991 18.991 0 0 1-.214 4.772 17.165 17.165 0 0 0 5.498-2.477ZM14.634 15.55a17.324 17.324 0 0 0 .332-4.647c-.952.227-1.945.347-2.966.347-1.021 0-2.014-.12-2.966-.347a17.515 17.515 0 0 0 .332 4.647 17.385 17.385 0 0 0 5.268 0ZM9.772 17.119a18.963 18.963 0 0 0 4.456 0A17.182 17.182 0 0 1 12 21.724a17.18 17.18 0 0 1-2.228-4.605ZM7.777 15.23a18.87 18.87 0 0 1-.214-4.774 12.753 12.753 0 0 1-4.34-2.708 9.711 9.711 0 0 0-.944 5.004 17.165 17.165 0 0 0 5.498 2.477ZM21.356 14.752a9.765 9.765 0 0 1-7.478 6.817 18.64 18.64 0 0 0 1.988-4.718 18.627 18.627 0 0 0 5.49-2.098ZM2.644 14.752c1.682.971 3.53 1.688 5.49 2.099a18.64 18.64 0 0 0 1.988 4.718 9.765 9.765 0 0 1-7.478-6.816ZM13.878 2.43a9.755 9.755 0 0 1 6.116 3.986 11.267 11.267 0 0 1-3.746 2.504 18.63 18.63 0 0 0-2.37-6.49ZM12 2.276a17.152 17.152 0 0 1 2.805 7.121c-.897.23-1.837.353-2.805.353-.968 0-1.908-.122-2.805-.353A17.151 17.151 0 0 1 12 2.276ZM10.122 2.43a18.629 18.629 0 0 0-2.37 6.49 11.266 11.266 0 0 1-3.746-2.504 9.754 9.754 0 0 1 6.116-3.985Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Web Search')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'interface'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'interface';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M2 4.25A2.25 2.25 0 0 1 4.25 2h7.5A2.25 2.25 0 0 1 14 4.25v5.5A2.25 2.25 0 0 1 11.75 12h-1.312c.1.128.21.248.328.36a.75.75 0 0 1 .234.545v.345a.75.75 0 0 1-.75.75h-4.5a.75.75 0 0 1-.75-.75v-.345a.75.75 0 0 1 .234-.545c.118-.111.228-.232.328-.36H4.25A2.25 2.25 0 0 1 2 9.75v-5.5Zm2.25-.75a.75.75 0 0 0-.75.75v4.5c0 .414.336.75.75.75h7.5a.75.75 0 0 0 .75-.75v-4.5a.75.75 0 0 0-.75-.75h-7.5Z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Interface')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'audio'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'audio';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M7.557 2.066A.75.75 0 0 1 8 2.75v10.5a.75.75 0 0 1-1.248.56L3.59 11H2a1 1 0 0 1-1-1V6a1 1 0 0 1 1-1h1.59l3.162-2.81a.75.75 0 0 1 .805-.124ZM12.95 3.05a.75.75 0 1 0-1.06 1.06 5.5 5.5 0 0 1 0 7.78.75.75 0 1 0 1.06 1.06 7 7 0 0 0 0-9.9Z"
/>
<path
d="M10.828 5.172a.75.75 0 1 0-1.06 1.06 2.5 2.5 0 0 1 0 3.536.75.75 0 1 0 1.06 1.06 4 4 0 0 0 0-5.656Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Audio')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'images'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'images';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M2 4a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V4Zm10.5 5.707a.5.5 0 0 0-.146-.353l-1-1a.5.5 0 0 0-.708 0L9.354 9.646a.5.5 0 0 1-.708 0L6.354 7.354a.5.5 0 0 0-.708 0l-2 2a.5.5 0 0 0-.146.353V12a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5V9.707ZM12 5a1 1 0 1 1-2 0 1 1 0 0 1 2 0Z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Images')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'pipelines'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'pipelines';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="size-4"
>
<path
d="M11.644 1.59a.75.75 0 0 1 .712 0l9.75 5.25a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.712 0l-9.75-5.25a.75.75 0 0 1 0-1.32l9.75-5.25Z"
/>
<path
d="m3.265 10.602 7.668 4.129a2.25 2.25 0 0 0 2.134 0l7.668-4.13 1.37.739a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.71 0l-9.75-5.25a.75.75 0 0 1 0-1.32l1.37-.738Z"
/>
<path
d="m10.933 19.231-7.668-4.13-1.37.739a.75.75 0 0 0 0 1.32l9.75 5.25c.221.12.489.12.71 0l9.75-5.25a.75.75 0 0 0 0-1.32l-1.37-.738-7.668 4.13a2.25 2.25 0 0 1-2.134-.001Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Pipelines')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'db'
? 'bg-gray-200 dark:bg-gray-800'
: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
on:click={() => {
selectedTab = 'db';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path d="M8 7c3.314 0 6-1.343 6-3s-2.686-3-6-3-6 1.343-6 3 2.686 3 6 3Z" />
<path
d="M8 8.5c1.84 0 3.579-.37 4.914-1.037A6.33 6.33 0 0 0 14 6.78V8c0 1.657-2.686 3-6 3S2 9.657 2 8V6.78c.346.273.72.5 1.087.683C4.42 8.131 6.16 8.5 8 8.5Z"
/>
<path
d="M8 12.5c1.84 0 3.579-.37 4.914-1.037.366-.183.74-.41 1.086-.684V12c0 1.657-2.686 3-6 3s-6-1.343-6-3v-1.22c.346.273.72.5 1.087.683C4.42 12.131 6.16 12.5 8 12.5Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Database')}</div>
</button>
</div>
<div class="flex-1 mt-3 lg:mt-0 overflow-y-scroll">
{#if selectedTab === 'general'}
<General
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'users'}
<Users
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'connections'}
<Connections
on:save={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'models'}
<Models />
{:else if selectedTab === 'documents'}
<Documents
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'web'}
<WebSearch
saveHandler={async () => {
toast.success($i18n.t('Settings saved successfully!'));
await tick();
await config.set(await getBackendConfig());
}}
/>
{:else if selectedTab === 'interface'}
<Interface
on:save={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'audio'}
<Audio
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'images'}
<Images
on:save={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'db'}
<Database
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'pipelines'}
<Pipelines
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{/if}
</div>
</div>

302
src/lib/components/admin/Settings/Audio.svelte Normal file
View File

@ -0,0 +1,302 @@
<script lang="ts">
import { getAudioConfig, updateAudioConfig } from '$lib/apis/audio';
import { user, settings, config } from '$lib/stores';
import { createEventDispatcher, onMount, getContext } from 'svelte';
import { toast } from 'svelte-sonner';
import Switch from '$lib/components/common/Switch.svelte';
import { getBackendConfig } from '$lib/apis';
const dispatch = createEventDispatcher();
const i18n = getContext('i18n');
export let saveHandler: Function;
// Audio
let TTS_OPENAI_API_BASE_URL = '';
let TTS_OPENAI_API_KEY = '';
let TTS_ENGINE = '';
let TTS_MODEL = '';
let TTS_VOICE = '';
let STT_OPENAI_API_BASE_URL = '';
let STT_OPENAI_API_KEY = '';
let STT_ENGINE = '';
let STT_MODEL = '';
let voices = [];
let models = [];
let nonLocalVoices = false;
const getOpenAIVoices = () => {
voices = [
{ name: 'alloy' },
{ name: 'echo' },
{ name: 'fable' },
{ name: 'onyx' },
{ name: 'nova' },
{ name: 'shimmer' }
];
};
const getOpenAIModels = () => {
models = [{ name: 'tts-1' }, { name: 'tts-1-hd' }];
};
const getWebAPIVoices = () => {
const getVoicesLoop = setInterval(async () => {
voices = await speechSynthesis.getVoices();
// do your loop
if (voices.length > 0) {
clearInterval(getVoicesLoop);
}
}, 100);
};
const updateConfigHandler = async () => {
const res = await updateAudioConfig(localStorage.token, {
tts: {
OPENAI_API_BASE_URL: TTS_OPENAI_API_BASE_URL,
OPENAI_API_KEY: TTS_OPENAI_API_KEY,
ENGINE: TTS_ENGINE,
MODEL: TTS_MODEL,
VOICE: TTS_VOICE
},
stt: {
OPENAI_API_BASE_URL: STT_OPENAI_API_BASE_URL,
OPENAI_API_KEY: STT_OPENAI_API_KEY,
ENGINE: STT_ENGINE,
MODEL: STT_MODEL
}
});
if (res) {
toast.success('Audio settings updated successfully');
config.set(await getBackendConfig());
}
};
onMount(async () => {
const res = await getAudioConfig(localStorage.token);
if (res) {
console.log(res);
TTS_OPENAI_API_BASE_URL = res.tts.OPENAI_API_BASE_URL;
TTS_OPENAI_API_KEY = res.tts.OPENAI_API_KEY;
TTS_ENGINE = res.tts.ENGINE;
TTS_MODEL = res.tts.MODEL;
TTS_VOICE = res.tts.VOICE;
STT_OPENAI_API_BASE_URL = res.stt.OPENAI_API_BASE_URL;
STT_OPENAI_API_KEY = res.stt.OPENAI_API_KEY;
STT_ENGINE = res.stt.ENGINE;
STT_MODEL = res.stt.MODEL;
}
if (TTS_ENGINE === 'openai') {
getOpenAIVoices();
getOpenAIModels();
} else {
getWebAPIVoices();
}
});
</script>
<form
class="flex flex-col h-full justify-between space-y-3 text-sm"
on:submit|preventDefault={async () => {
await updateConfigHandler();
dispatch('save');
}}
>
<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
<div class="flex flex-col gap-3">
<div>
<div class=" mb-1 text-sm font-medium">{$i18n.t('STT Settings')}</div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Speech-to-Text Engine')}</div>
<div class="flex items-center relative">
<select
class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
bind:value={STT_ENGINE}
placeholder="Select an engine"
>
<option value="">{$i18n.t('Whisper (Local)')}</option>
<option value="openai">OpenAI</option>
<option value="web">{$i18n.t('Web API')}</option>
</select>
</div>
</div>
{#if STT_ENGINE === 'openai'}
<div>
<div class="mt-1 flex gap-2 mb-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('API Base URL')}
bind:value={STT_OPENAI_API_BASE_URL}
required
/>
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('API Key')}
bind:value={STT_OPENAI_API_KEY}
required
/>
</div>
</div>
<hr class=" dark:border-gray-850 my-2" />
<div>
<div class=" mb-1.5 text-sm font-medium">{$i18n.t('STT Model')}</div>
<div class="flex w-full">
<div class="flex-1">
<input
list="model-list"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={STT_MODEL}
placeholder="Select a model"
/>
<datalist id="model-list">
<option value="whisper-1" />
</datalist>
</div>
</div>
</div>
{/if}
</div>
<hr class=" dark:border-gray-800" />
<div>
<div class=" mb-1 text-sm font-medium">{$i18n.t('TTS Settings')}</div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Text-to-Speech Engine')}</div>
<div class="flex items-center relative">
<select
class=" dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
bind:value={TTS_ENGINE}
placeholder="Select a mode"
on:change={(e) => {
if (e.target.value === 'openai') {
getOpenAIVoices();
TTS_VOICE = 'alloy';
TTS_MODEL = 'tts-1';
} else {
getWebAPIVoices();
TTS_VOICE = '';
}
}}
>
<option value="">{$i18n.t('Web API')}</option>
<option value="openai">{$i18n.t('Open AI')}</option>
</select>
</div>
</div>
{#if TTS_ENGINE === 'openai'}
<div>
<div class="mt-1 flex gap-2 mb-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('API Base URL')}
bind:value={TTS_OPENAI_API_BASE_URL}
required
/>
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('API Key')}
bind:value={TTS_OPENAI_API_KEY}
required
/>
</div>
</div>
{/if}
<hr class=" dark:border-gray-850 my-2" />
{#if TTS_ENGINE === ''}
<div>
<div class=" mb-1.5 text-sm font-medium">{$i18n.t('TTS Voice')}</div>
<div class="flex w-full">
<div class="flex-1">
<select
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={TTS_VOICE}
>
<option value="" selected={TTS_VOICE !== ''}>{$i18n.t('Default')}</option>
{#each voices as voice}
<option
value={voice.voiceURI}
class="bg-gray-100 dark:bg-gray-700"
selected={TTS_VOICE === voice.voiceURI}>{voice.name}</option
>
{/each}
</select>
</div>
</div>
</div>
{:else if TTS_ENGINE === 'openai'}
<div class=" flex gap-2">
<div class="w-full">
<div class=" mb-1.5 text-sm font-medium">{$i18n.t('TTS Voice')}</div>
<div class="flex w-full">
<div class="flex-1">
<input
list="voice-list"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={TTS_VOICE}
placeholder="Select a voice"
/>
<datalist id="voice-list">
{#each voices as voice}
<option value={voice.name} />
{/each}
</datalist>
</div>
</div>
</div>
<div class="w-full">
<div class=" mb-1.5 text-sm font-medium">{$i18n.t('TTS Model')}</div>
<div class="flex w-full">
<div class="flex-1">
<input
list="model-list"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={TTS_MODEL}
placeholder="Select a model"
/>
<datalist id="model-list">
{#each models as model}
<option value={model.name} />
{/each}
</datalist>
</div>
</div>
</div>
</div>
{/if}
</div>
</div>
</div>
<div class="flex justify-end text-sm font-medium">
<button
class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
type="submit"
>
{$i18n.t('Save')}
</button>
</div>
</form>

137
src/lib/components/admin/Settings/Banners.svelte
View File

@ -1,137 +0,0 @@
<script lang="ts">
import { v4 as uuidv4 } from 'uuid';
import { getContext, onMount } from 'svelte';
import { banners as _banners } from '$lib/stores';
import type { Banner } from '$lib/types';
import { getBanners, setBanners } from '$lib/apis/configs';
import type { Writable } from 'svelte/store';
import type { i18n as i18nType } from 'i18next';
import Tooltip from '$lib/components/common/Tooltip.svelte';
import Switch from '$lib/components/common/Switch.svelte';
const i18n: Writable<i18nType> = getContext('i18n');
export let saveHandler: Function;
let banners: Banner[] = [];
onMount(async () => {
banners = await getBanners(localStorage.token);
});
const updateBanners = async () => {
_banners.set(await setBanners(localStorage.token, banners));
};
</script>
<form
class="flex flex-col h-full justify-between space-y-3 text-sm"
on:submit|preventDefault={async () => {
updateBanners();
saveHandler();
}}
>
<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80 h-full">
<div class=" space-y-3 pr-1.5">
<div class="flex w-full justify-between mb-2">
<div class=" self-center text-sm font-semibold">
{$i18n.t('Banners')}
</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
if (banners.length === 0 || banners.at(-1).content !== '') {
banners = [
...banners,
{
id: uuidv4(),
type: '',
title: '',
content: '',
dismissible: true,
timestamp: Math.floor(Date.now() / 1000)
}
];
}
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
/>
</svg>
</button>
</div>
<div class="flex flex-col space-y-1">
{#each banners as banner, bannerIdx}
<div class=" flex justify-between">
<div class="flex flex-row flex-1 border rounded-xl dark:border-gray-800">
<select
class="w-fit capitalize rounded-xl py-2 px-4 text-xs bg-transparent outline-none"
bind:value={banner.type}
>
{#if banner.type == ''}
<option value="" selected disabled class="text-gray-900">{$i18n.t('Type')}</option
>
{/if}
<option value="info" class="text-gray-900">{$i18n.t('Info')}</option>
<option value="warning" class="text-gray-900">{$i18n.t('Warning')}</option>
<option value="error" class="text-gray-900">{$i18n.t('Error')}</option>
<option value="success" class="text-gray-900">{$i18n.t('Success')}</option>
</select>
<input
class="pr-5 py-1.5 text-xs w-full bg-transparent outline-none"
placeholder={$i18n.t('Content')}
bind:value={banner.content}
/>
<div class="relative top-1.5 -left-2">
<Tooltip content="Dismissible" className="flex h-fit items-center">
<Switch bind:state={banner.dismissible} />
</Tooltip>
</div>
</div>
<button
class="px-2"
type="button"
on:click={() => {
banners.splice(bannerIdx, 1);
banners = banners;
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
/>
</svg>
</button>
</div>
{/each}
</div>
</div>
</div>
<div class="flex justify-end pt-3 text-sm font-medium">
<button
class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
type="submit"
>
Save
</button>
</div>
</form>

71
src/lib/components/chat/Settings/Connections.svelte → src/lib/components/admin/Settings/Connections.svelte
View File

@ -1,6 +1,6 @@
<script lang="ts">
import { models, user } from '$lib/stores';
import { createEventDispatcher, onMount, getContext } from 'svelte';
import { createEventDispatcher, onMount, getContext, tick } from 'svelte';
const dispatch = createEventDispatcher();
import {
@ -23,10 +23,14 @@
import Switch from '$lib/components/common/Switch.svelte';
import Spinner from '$lib/components/common/Spinner.svelte';
import Tooltip from '$lib/components/common/Tooltip.svelte';
import { getModels as _getModels } from '$lib/apis';
const i18n = getContext('i18n');
export let getModels: Function;
const getModels = async () => {
const models = await _getModels(localStorage.token);
return models;
};
// External
let OLLAMA_BASE_URLS = [''];
@ -54,6 +58,8 @@
pipelineUrls[OPENAI_API_BASE_URLS[idx]] = true;
}
}
await models.set(await getModels());
};
const verifyOllamaHandler = async (idx) => {
@ -67,26 +73,53 @@
if (res) {
toast.success($i18n.t('Server connection verified'));
}
};
const updateOpenAIHandler = async () => {
OPENAI_API_BASE_URLS = await updateOpenAIUrls(localStorage.token, OPENAI_API_BASE_URLS);
OPENAI_API_KEYS = await updateOpenAIKeys(localStorage.token, OPENAI_API_KEYS);
await models.set(await getModels());
};
const updateOpenAIHandler = async () => {
// Check if API KEYS length is same than API URLS length
if (OPENAI_API_KEYS.length !== OPENAI_API_BASE_URLS.length) {
// if there are more keys than urls, remove the extra keys
if (OPENAI_API_KEYS.length > OPENAI_API_BASE_URLS.length) {
OPENAI_API_KEYS = OPENAI_API_KEYS.slice(0, OPENAI_API_BASE_URLS.length);
}
// if there are more urls than keys, add empty keys
if (OPENAI_API_KEYS.length < OPENAI_API_BASE_URLS.length) {
const diff = OPENAI_API_BASE_URLS.length - OPENAI_API_KEYS.length;
for (let i = 0; i < diff; i++) {
OPENAI_API_KEYS.push('');
}
}
}
OPENAI_API_BASE_URLS = await updateOpenAIUrls(localStorage.token, OPENAI_API_BASE_URLS);
OPENAI_API_KEYS = await updateOpenAIKeys(localStorage.token, OPENAI_API_KEYS);
await models.set(await getModels());
};
const updateOllamaUrlsHandler = async () => {
OLLAMA_BASE_URLS = await updateOllamaUrls(localStorage.token, OLLAMA_BASE_URLS);
OLLAMA_BASE_URLS = OLLAMA_BASE_URLS.filter((url) => url !== '');
console.log(OLLAMA_BASE_URLS);
const ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => {
toast.error(error);
return null;
});
if (OLLAMA_BASE_URLS.length === 0) {
ENABLE_OLLAMA_API = false;
await updateOllamaConfig(localStorage.token, ENABLE_OLLAMA_API);
if (ollamaVersion) {
toast.success($i18n.t('Server connection verified'));
await models.set(await getModels());
toast.info($i18n.t('Ollama API disabled'));
} else {
OLLAMA_BASE_URLS = await updateOllamaUrls(localStorage.token, OLLAMA_BASE_URLS);
const ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => {
toast.error(error);
return null;
});
if (ollamaVersion) {
toast.success($i18n.t('Server connection verified'));
await models.set(await getModels());
}
}
};
@ -129,7 +162,7 @@
dispatch('save');
}}
>
<div class="space-y-3 pr-1.5 overflow-y-scroll h-[24rem] max-h-[25rem]">
<div class="space-y-3 overflow-y-scroll scrollbar-hidden h-full">
{#if ENABLE_OPENAI_API !== null && ENABLE_OLLAMA_API !== null}
<div class=" space-y-3">
<div class="mt-2 space-y-2 pr-1.5">
@ -271,7 +304,7 @@
</div>
</div>
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
<div class="pr-1.5 space-y-2">
<div class="flex justify-between items-center text-sm">
@ -282,6 +315,10 @@
bind:state={ENABLE_OLLAMA_API}
on:change={async () => {
updateOllamaConfig(localStorage.token, ENABLE_OLLAMA_API);
if (OLLAMA_BASE_URLS.length === 0) {
OLLAMA_BASE_URLS = [''];
}
}}
/>
</div>

43
src/lib/components/admin/Settings/Database.svelte
View File

@ -2,7 +2,7 @@
import fileSaver from 'file-saver';
const { saveAs } = fileSaver;
import { downloadDatabase } from '$lib/apis/utils';
import { downloadDatabase, downloadLiteLLMConfig } from '$lib/apis/utils';
import { onMount, getContext } from 'svelte';
import { config, user } from '$lib/stores';
import { toast } from 'svelte-sonner';
@ -30,7 +30,7 @@
saveHandler();
}}
>
<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
<div>
<div class=" mb-2 text-sm font-medium">{$i18n.t('Database')}</div>
@ -68,10 +68,8 @@
</button>
</div>
<hr class=" dark:border-gray-700 my-1" />
<button
class=" flex rounded-md py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
class=" flex rounded-md py-2 px-3 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
on:click={() => {
exportAllUserChats();
}}
@ -96,6 +94,41 @@
</div>
</button>
{/if}
<hr class=" dark:border-gray-850 my-1" />
<div class=" flex w-full justify-between">
<!-- <div class=" self-center text-xs font-medium">{$i18n.t('Allow Chat Deletion')}</div> -->
<button
class=" flex rounded-md py-1.5 px-3 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
type="button"
on:click={() => {
downloadLiteLLMConfig(localStorage.token).catch((error) => {
toast.error(error);
});
}}
>
<div class=" self-center mr-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="size-4"
>
<path
fill-rule="evenodd"
d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875Zm5.845 17.03a.75.75 0 0 0 1.06 0l3-3a.75.75 0 1 0-1.06-1.06l-1.72 1.72V12a.75.75 0 0 0-1.5 0v4.19l-1.72-1.72a.75.75 0 0 0-1.06 1.06l3 3Z"
clip-rule="evenodd"
/>
<path
d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
/>
</svg>
</div>
<div class=" self-center text-sm font-medium">Export LiteLLM config.yaml</div>
</button>
</div>
</div>
</div>

424
src/lib/components/documents/Settings/General.svelte → src/lib/components/admin/Settings/Documents.svelte
View File

@ -8,7 +8,10 @@
getEmbeddingConfig,
updateEmbeddingConfig,
getRerankingConfig,
updateRerankingConfig
updateRerankingConfig,
resetUploadDir,
getRAGConfig,
updateRAGConfig
} from '$lib/apis/rag';
import { documents, models } from '$lib/stores';
@ -24,13 +27,19 @@
let updateRerankingModelLoading = false;
let showResetConfirm = false;
let showResetUploadDirConfirm = false;
let embeddingEngine = '';
let embeddingModel = '';
let rerankingModel = '';
let chunkSize = 0;
let chunkOverlap = 0;
let pdfExtractImages = true;
let OpenAIKey = '';
let OpenAIUrl = '';
let OpenAIBatchSize = 1;
let querySettings = {
template: '',
@ -92,7 +101,8 @@
? {
openai_config: {
key: OpenAIKey,
url: OpenAIUrl
url: OpenAIUrl,
batch_size: OpenAIBatchSize
}
}
: {})
@ -148,6 +158,16 @@
if (querySettings.hybrid) {
rerankingModelUpdateHandler();
}
const res = await updateRAGConfig(localStorage.token, {
pdf_extract_images: pdfExtractImages,
chunk: {
chunk_overlap: chunkOverlap,
chunk_size: chunkSize
}
});
await updateQuerySettings(localStorage.token, querySettings);
};
const setEmbeddingConfig = async () => {
@ -159,6 +179,7 @@
OpenAIKey = embeddingConfig.openai_config.key;
OpenAIUrl = embeddingConfig.openai_config.url;
OpenAIBatchSize = embeddingConfig.openai_config.batch_size ?? 1;
}
};
@ -180,6 +201,15 @@
await setRerankingConfig();
querySettings = await getQuerySettings(localStorage.token);
const res = await getRAGConfig(localStorage.token);
if (res) {
pdfExtractImages = res.pdf_extract_images;
chunkSize = res.chunk.chunk_size;
chunkOverlap = res.chunk.chunk_overlap;
}
});
</script>
@ -190,7 +220,7 @@
saveHandler();
}}
>
<div class=" space-y-2.5 pr-1.5 overflow-y-scroll max-h-[28rem]">
<div class=" space-y-2.5 overflow-y-scroll scrollbar-hidden h-full">
<div class="flex flex-col gap-0.5">
<div class=" mb-0.5 text-sm font-medium">{$i18n.t('General Settings')}</div>
@ -282,6 +312,30 @@
required
/>
</div>
<div class="flex mt-0.5 space-x-2">
<div class=" self-center text-xs font-medium">{$i18n.t('Embedding Batch Size')}</div>
<div class=" flex-1">
<input
id="steps-range"
type="range"
min="1"
max="2048"
step="1"
bind:value={OpenAIBatchSize}
class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
/>
</div>
<div class="">
<input
bind:value={OpenAIBatchSize}
type="number"
class=" bg-transparent text-center w-14"
min="-2"
max="16000"
step="1"
/>
</div>
</div>
{/if}
<div class=" flex w-full justify-between">
@ -303,7 +357,7 @@
</div>
</div>
<hr class=" dark:border-gray-700 my-1" />
<hr class=" dark:border-gray-850 my-1" />
<div class="space-y-2" />
<div>
@ -321,10 +375,8 @@
{#if !embeddingModel}
<option value="" disabled selected>{$i18n.t('Select a model')}</option>
{/if}
{#each $models.filter((m) => m.id && !m.external) as model}
<option value={model.name} class="bg-gray-100 dark:bg-gray-700"
>{model.name + ' (' + (model.size / 1024 ** 3).toFixed(1) + ' GB)'}</option
>
{#each $models.filter((m) => m.id && m.ollama && !(m?.preset ?? false)) as model}
<option value={model.id} class="bg-gray-100 dark:bg-gray-700">{model.name}</option>
{/each}
</select>
</div>
@ -469,99 +521,319 @@
{/if}
</div>
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
{#if showResetConfirm}
<div class="flex justify-between rounded-md items-center py-2 px-3.5 w-full transition">
<div class="flex items-center space-x-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path d="M2 3a1 1 0 0 1 1-1h10a1 1 0 0 1 1 1v1a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3Z" />
<path
fill-rule="evenodd"
d="M13 6H3v6a2 2 0 0 0 2 2h6a2 2 0 0 0 2-2V6ZM5.72 7.47a.75.75 0 0 1 1.06 0L8 8.69l1.22-1.22a.75.75 0 1 1 1.06 1.06L9.06 9.75l1.22 1.22a.75.75 0 1 1-1.06 1.06L8 10.81l-1.22 1.22a.75.75 0 0 1-1.06-1.06l1.22-1.22-1.22-1.22a.75.75 0 0 1 0-1.06Z"
clip-rule="evenodd"
<div class=" ">
<div class=" text-sm font-medium">{$i18n.t('Query Params')}</div>
<div class=" flex">
<div class=" flex w-full justify-between">
<div class="self-center text-xs font-medium min-w-fit">{$i18n.t('Top K')}</div>
<div class="self-center p-3">
<input
class=" w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="number"
placeholder={$i18n.t('Enter Top K')}
bind:value={querySettings.k}
autocomplete="off"
min="0"
/>
</svg>
<span>{$i18n.t('Are you sure?')}</span>
</div>
</div>
<div class="flex space-x-1.5 items-center">
{#if querySettings.hybrid === true}
<div class="flex w-full">
<div class=" self-center text-xs font-medium min-w-fit">
{$i18n.t('Minimum Score')}
</div>
<div class="self-center p-3">
<input
class=" w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="number"
step="0.01"
placeholder={$i18n.t('Enter Score')}
bind:value={querySettings.r}
autocomplete="off"
min="0.0"
title={$i18n.t('The score should be a value between 0.0 (0%) and 1.0 (100%).')}
/>
</div>
</div>
{/if}
</div>
{#if querySettings.hybrid === true}
<div class="mt-2 mb-1 text-xs text-gray-400 dark:text-gray-500">
{$i18n.t(
'Note: If you set a minimum score, the search will only return documents with a score greater than or equal to the minimum score.'
)}
</div>
<hr class=" dark:border-gray-850 my-3" />
{/if}
<div>
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('RAG Template')}</div>
<textarea
bind:value={querySettings.template}
class="w-full rounded-lg px-4 py-3 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
rows="4"
/>
</div>
</div>
<hr class=" dark:border-gray-850" />
<div class=" ">
<div class=" text-sm font-medium">{$i18n.t('Chunk Params')}</div>
<div class=" my-2 flex gap-1.5">
<div class=" w-full justify-between">
<div class="self-center text-xs font-medium min-w-fit mb-1">{$i18n.t('Chunk Size')}</div>
<div class="self-center">
<input
class=" w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="number"
placeholder={$i18n.t('Enter Chunk Size')}
bind:value={chunkSize}
autocomplete="off"
min="0"
/>
</div>
</div>
<div class="w-full">
<div class=" self-center text-xs font-medium min-w-fit mb-1">
{$i18n.t('Chunk Overlap')}
</div>
<div class="self-center">
<input
class="w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="number"
placeholder={$i18n.t('Enter Chunk Overlap')}
bind:value={chunkOverlap}
autocomplete="off"
min="0"
/>
</div>
</div>
</div>
<div class="my-3">
<div class="flex justify-between items-center text-xs">
<div class=" text-xs font-medium">{$i18n.t('PDF Extract Images (OCR)')}</div>
<button
class="hover:text-white transition"
class=" text-xs font-medium text-gray-500"
type="button"
on:click={() => {
const res = resetVectorDB(localStorage.token).catch((error) => {
toast.error(error);
return null;
});
if (res) {
toast.success($i18n.t('Success'));
}
showResetConfirm = false;
}}
pdfExtractImages = !pdfExtractImages;
}}>{pdfExtractImages ? $i18n.t('On') : $i18n.t('Off')}</button
>
</div>
</div>
</div>
<hr class=" dark:border-gray-850" />
<div>
{#if showResetUploadDirConfirm}
<div class="flex justify-between rounded-md items-center py-2 px-3.5 w-full transition">
<div class="flex items-center space-x-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
viewBox="0 0 24 24"
fill="currentColor"
class="size-4"
>
<path
fill-rule="evenodd"
d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875ZM9.75 14.25a.75.75 0 0 0 0 1.5H15a.75.75 0 0 0 0-1.5H9.75Z"
clip-rule="evenodd"
/>
<path
d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
/>
</svg>
<span>{$i18n.t('Are you sure?')}</span>
</div>
<div class="flex space-x-1.5 items-center">
<button
class="hover:text-white transition"
on:click={() => {
const res = resetUploadDir(localStorage.token).catch((error) => {
toast.error(error);
return null;
});
if (res) {
toast.success($i18n.t('Success'));
}
showResetUploadDirConfirm = false;
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M16.704 4.153a.75.75 0 01.143 1.052l-8 10.5a.75.75 0 01-1.127.075l-4.5-4.5a.75.75 0 011.06-1.06l3.894 3.893 7.48-9.817a.75.75 0 011.05-.143z"
clip-rule="evenodd"
/>
</svg>
</button>
<button
class="hover:text-white transition"
type="button"
on:click={() => {
showResetUploadDirConfirm = false;
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
/>
</svg>
</button>
</div>
</div>
{:else}
<button
class=" flex rounded-xl py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
on:click={() => {
showResetUploadDirConfirm = true;
}}
type="button"
>
<div class=" self-center mr-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="size-4"
>
<path
fill-rule="evenodd"
d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875ZM9.75 14.25a.75.75 0 0 0 0 1.5H15a.75.75 0 0 0 0-1.5H9.75Z"
clip-rule="evenodd"
/>
<path
d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
/>
</svg>
</div>
<div class=" self-center text-sm font-medium">{$i18n.t('Reset Upload Directory')}</div>
</button>
{/if}
{#if showResetConfirm}
<div class="flex justify-between rounded-md items-center py-2 px-3.5 w-full transition">
<div class="flex items-center space-x-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M16.704 4.153a.75.75 0 01.143 1.052l-8 10.5a.75.75 0 01-1.127.075l-4.5-4.5a.75.75 0 011.06-1.06l3.894 3.893 7.48-9.817a.75.75 0 011.05-.143z"
d="M3.5 2A1.5 1.5 0 0 0 2 3.5v9A1.5 1.5 0 0 0 3.5 14h9a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 12.5 4H9.621a1.5 1.5 0 0 1-1.06-.44L7.439 2.44A1.5 1.5 0 0 0 6.38 2H3.5Zm6.75 7.75a.75.75 0 0 0 0-1.5h-4.5a.75.75 0 0 0 0 1.5h4.5Z"
clip-rule="evenodd"
/>
</svg>
</button>
<button
class="hover:text-white transition"
on:click={() => {
showResetConfirm = false;
}}
>
<span>{$i18n.t('Are you sure?')}</span>
</div>
<div class="flex space-x-1.5 items-center">
<button
class="hover:text-white transition"
on:click={() => {
const res = resetVectorDB(localStorage.token).catch((error) => {
toast.error(error);
return null;
});
if (res) {
toast.success($i18n.t('Success'));
}
showResetConfirm = false;
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M16.704 4.153a.75.75 0 01.143 1.052l-8 10.5a.75.75 0 01-1.127.075l-4.5-4.5a.75.75 0 011.06-1.06l3.894 3.893 7.48-9.817a.75.75 0 011.05-.143z"
clip-rule="evenodd"
/>
</svg>
</button>
<button
class="hover:text-white transition"
on:click={() => {
showResetConfirm = false;
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
/>
</svg>
</button>
</div>
</div>
{:else}
<button
class=" flex rounded-xl py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
on:click={() => {
showResetConfirm = true;
}}
type="button"
>
<div class=" self-center mr-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
fill-rule="evenodd"
d="M3.5 2A1.5 1.5 0 0 0 2 3.5v9A1.5 1.5 0 0 0 3.5 14h9a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 12.5 4H9.621a1.5 1.5 0 0 1-1.06-.44L7.439 2.44A1.5 1.5 0 0 0 6.38 2H3.5Zm6.75 7.75a.75.75 0 0 0 0-1.5h-4.5a.75.75 0 0 0 0 1.5h4.5Z"
clip-rule="evenodd"
/>
</svg>
</button>
</div>
</div>
{:else}
<button
class=" flex rounded-md py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
on:click={() => {
showResetConfirm = true;
}}
>
<div class=" self-center mr-3">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M3.5 2A1.5 1.5 0 0 0 2 3.5v9A1.5 1.5 0 0 0 3.5 14h9a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 12.5 4H9.621a1.5 1.5 0 0 1-1.06-.44L7.439 2.44A1.5 1.5 0 0 0 6.38 2H3.5Zm6.75 7.75a.75.75 0 0 0 0-1.5h-4.5a.75.75 0 0 0 0 1.5h4.5Z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center text-sm font-medium">{$i18n.t('Reset Vector Storage')}</div>
</button>
{/if}
</div>
<div class=" self-center text-sm font-medium">{$i18n.t('Reset Vector Storage')}</div>
</button>
{/if}
</div>
</div>
<div class="flex justify-end pt-3 text-sm font-medium">
<button

259
src/lib/components/admin/Settings/General.svelte
View File

@ -6,61 +6,44 @@
updateWebhookUrl
} from '$lib/apis';
import {
getAdminConfig,
getDefaultUserRole,
getJWTExpiresDuration,
getSignUpEnabledStatus,
toggleSignUpEnabledStatus,
updateAdminConfig,
updateDefaultUserRole,
updateJWTExpiresDuration
} from '$lib/apis/auths';
import Switch from '$lib/components/common/Switch.svelte';
import { onMount, getContext } from 'svelte';
const i18n = getContext('i18n');
export let saveHandler: Function;
let signUpEnabled = true;
let defaultUserRole = 'pending';
let JWTExpiresIn = '';
let adminConfig = null;
let webhookUrl = '';
let communitySharingEnabled = true;
const toggleSignUpEnabled = async () => {
signUpEnabled = await toggleSignUpEnabledStatus(localStorage.token);
};
const updateDefaultUserRoleHandler = async (role) => {
defaultUserRole = await updateDefaultUserRole(localStorage.token, role);
};
const updateJWTExpiresDurationHandler = async (duration) => {
JWTExpiresIn = await updateJWTExpiresDuration(localStorage.token, duration);
};
const updateWebhookUrlHandler = async () => {
const updateHandler = async () => {
webhookUrl = await updateWebhookUrl(localStorage.token, webhookUrl);
};
const res = await updateAdminConfig(localStorage.token, adminConfig);
const toggleCommunitySharingEnabled = async () => {
communitySharingEnabled = await toggleCommunitySharingEnabledStatus(localStorage.token);
if (res) {
toast.success(i18n.t('Settings updated successfully'));
} else {
toast.error(i18n.t('Failed to update settings'));
}
};
onMount(async () => {
await Promise.all([
(async () => {
signUpEnabled = await getSignUpEnabledStatus(localStorage.token);
})(),
(async () => {
defaultUserRole = await getDefaultUserRole(localStorage.token);
})(),
(async () => {
JWTExpiresIn = await getJWTExpiresDuration(localStorage.token);
adminConfig = await getAdminConfig(localStorage.token);
})(),
(async () => {
webhookUrl = await getWebhookUrl(localStorage.token);
})(),
(async () => {
communitySharingEnabled = await getCommunitySharingEnabledStatus(localStorage.token);
})()
]);
});
@ -69,156 +52,94 @@
<form
class="flex flex-col h-full justify-between space-y-3 text-sm"
on:submit|preventDefault={() => {
updateJWTExpiresDurationHandler(JWTExpiresIn);
updateWebhookUrlHandler();
updateHandler();
saveHandler();
}}
>
<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
<div>
<div class=" mb-2 text-sm font-medium">{$i18n.t('General Settings')}</div>
<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
{#if adminConfig !== null}
<div>
<div class=" mb-3 text-sm font-medium">{$i18n.t('General Settings')}</div>
<div class=" flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Enable New Sign Ups')}</div>
<div class=" flex w-full justify-between pr-2">
<div class=" self-center text-xs font-medium">{$i18n.t('Enable New Sign Ups')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
toggleSignUpEnabled();
}}
type="button"
>
{#if signUpEnabled}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
<Switch bind:state={adminConfig.ENABLE_SIGNUP} />
</div>
<div class=" my-3 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Default User Role')}</div>
<div class="flex items-center relative">
<select
class="dark:bg-gray-900 w-fit pr-8 rounded px-2 text-xs bg-transparent outline-none text-right"
bind:value={adminConfig.DEFAULT_USER_ROLE}
placeholder="Select a role"
>
<path
d="M11.5 1A3.5 3.5 0 0 0 8 4.5V7H2.5A1.5 1.5 0 0 0 1 8.5v5A1.5 1.5 0 0 0 2.5 15h7a1.5 1.5 0 0 0 1.5-1.5v-5A1.5 1.5 0 0 0 9.5 7V4.5a2 2 0 1 1 4 0v1.75a.75.75 0 0 0 1.5 0V4.5A3.5 3.5 0 0 0 11.5 1Z"
/>
</svg>
<span class="ml-2 self-center">{$i18n.t('Enabled')}</span>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
<option value="pending">{$i18n.t('pending')}</option>
<option value="user">{$i18n.t('user')}</option>
<option value="admin">{$i18n.t('admin')}</option>
</select>
</div>
</div>
<hr class=" dark:border-gray-850 my-2" />
<div class="my-3 flex w-full items-center justify-between pr-2">
<div class=" self-center text-xs font-medium">
{$i18n.t('Show Admin Details in Account Pending Overlay')}
</div>
<Switch bind:state={adminConfig.SHOW_ADMIN_DETAILS} />
</div>
<div class="my-3 flex w-full items-center justify-between pr-2">
<div class=" self-center text-xs font-medium">{$i18n.t('Enable Community Sharing')}</div>
<Switch bind:state={adminConfig.ENABLE_COMMUNITY_SHARING} />
</div>
<hr class=" dark:border-gray-850 my-2" />
<div class=" w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('JWT Expiration')}</div>
</div>
<div class="flex mt-2 space-x-2">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={`e.g.) "30m","1h", "10d". `}
bind:value={adminConfig.JWT_EXPIRES_IN}
/>
</div>
<div class="mt-2 text-xs text-gray-400 dark:text-gray-500">
{$i18n.t('Valid time units:')}
<span class=" text-gray-300 font-medium"
>{$i18n.t("'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.")}</span
>
<path
fill-rule="evenodd"
d="M8 1a3.5 3.5 0 0 0-3.5 3.5V7A1.5 1.5 0 0 0 3 8.5v5A1.5 1.5 0 0 0 4.5 15h7a1.5 1.5 0 0 0 1.5-1.5v-5A1.5 1.5 0 0 0 11.5 7V4.5A3.5 3.5 0 0 0 8 1Zm2 6V4.5a2 2 0 1 0-4 0V7h4Z"
clip-rule="evenodd"
/>
</svg>
</div>
</div>
<span class="ml-2 self-center">{$i18n.t('Disabled')}</span>
{/if}
</button>
</div>
<hr class=" dark:border-gray-850 my-2" />
<div class=" flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Default User Role')}</div>
<div class="flex items-center relative">
<select
class="dark:bg-gray-900 w-fit pr-8 rounded py-2 px-2 text-xs bg-transparent outline-none text-right"
bind:value={defaultUserRole}
placeholder="Select a theme"
on:change={(e) => {
updateDefaultUserRoleHandler(e.target.value);
}}
>
<option value="pending">{$i18n.t('pending')}</option>
<option value="user">{$i18n.t('user')}</option>
<option value="admin">{$i18n.t('admin')}</option>
</select>
<div class=" w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Webhook URL')}</div>
</div>
<div class="flex mt-2 space-x-2">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={`https://example.com/webhook`}
bind:value={webhookUrl}
/>
</div>
</div>
</div>
<div class=" flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Enable Community Sharing')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
toggleCommunitySharingEnabled();
}}
type="button"
>
{#if communitySharingEnabled}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M11.5 1A3.5 3.5 0 0 0 8 4.5V7H2.5A1.5 1.5 0 0 0 1 8.5v5A1.5 1.5 0 0 0 2.5 15h7a1.5 1.5 0 0 0 1.5-1.5v-5A1.5 1.5 0 0 0 9.5 7V4.5a2 2 0 1 1 4 0v1.75a.75.75 0 0 0 1.5 0V4.5A3.5 3.5 0 0 0 11.5 1Z"
/>
</svg>
<span class="ml-2 self-center">{$i18n.t('Enabled')}</span>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M8 1a3.5 3.5 0 0 0-3.5 3.5V7A1.5 1.5 0 0 0 3 8.5v5A1.5 1.5 0 0 0 4.5 15h7a1.5 1.5 0 0 0 1.5-1.5v-5A1.5 1.5 0 0 0 11.5 7V4.5A3.5 3.5 0 0 0 8 1Zm2 6V4.5a2 2 0 1 0-4 0V7h4Z"
clip-rule="evenodd"
/>
</svg>
<span class="ml-2 self-center">{$i18n.t('Disabled')}</span>
{/if}
</button>
</div>
<hr class=" dark:border-gray-700 my-3" />
<div class=" w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Webhook URL')}</div>
</div>
<div class="flex mt-2 space-x-2">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={`https://example.com/webhook`}
bind:value={webhookUrl}
/>
</div>
</div>
<hr class=" dark:border-gray-700 my-3" />
<div class=" w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('JWT Expiration')}</div>
</div>
<div class="flex mt-2 space-x-2">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={`e.g.) "30m","1h", "10d". `}
bind:value={JWTExpiresIn}
/>
</div>
<div class="mt-2 text-xs text-gray-400 dark:text-gray-500">
{$i18n.t('Valid time units:')}
<span class=" text-gray-300 font-medium"
>{$i18n.t("'s', 'm', 'h', 'd', 'w' or '-1' for no expiration.")}</span
>
</div>
</div>
</div>
{/if}
</div>
<div class="flex justify-end pt-3 text-sm font-medium">

8
src/lib/components/chat/Settings/Images.svelte → src/lib/components/admin/Settings/Images.svelte
View File

@ -23,8 +23,6 @@
const i18n = getContext('i18n');
export let saveSettings: Function;
let loading = false;
let imageGenerationEngine = '';
@ -171,7 +169,7 @@
loading = false;
}}
>
<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-[24rem]">
<div class=" space-y-3 overflow-y-scroll scrollbar-hidden">
<div>
<div class=" mb-1 text-sm font-medium">{$i18n.t('Image Settings')}</div>
@ -228,7 +226,7 @@
</div>
</div>
</div>
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
{#if imageGenerationEngine === ''}
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('AUTOMATIC1111 Base URL')}</div>
@ -326,7 +324,7 @@
{/if}
{#if enableImageGeneration}
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
<div>
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Default Model')}</div>

339
src/lib/components/admin/Settings/Interface.svelte Normal file
View File

@ -0,0 +1,339 @@
<script lang="ts">
import { v4 as uuidv4 } from 'uuid';
import { toast } from 'svelte-sonner';
import { getBackendConfig, getTaskConfig, updateTaskConfig } from '$lib/apis';
import { setDefaultPromptSuggestions } from '$lib/apis/configs';
import { config, models, settings, user } from '$lib/stores';
import { createEventDispatcher, onMount, getContext } from 'svelte';
import { banners as _banners } from '$lib/stores';
import type { Banner } from '$lib/types';
import { getBanners, setBanners } from '$lib/apis/configs';
import Tooltip from '$lib/components/common/Tooltip.svelte';
import Switch from '$lib/components/common/Switch.svelte';
const dispatch = createEventDispatcher();
const i18n = getContext('i18n');
let taskConfig = {
TASK_MODEL: '',
TASK_MODEL_EXTERNAL: '',
TITLE_GENERATION_PROMPT_TEMPLATE: '',
SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE: '',
SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD: 0
};
let promptSuggestions = [];
let banners: Banner[] = [];
const updateInterfaceHandler = async () => {
taskConfig = await updateTaskConfig(localStorage.token, taskConfig);
promptSuggestions = await setDefaultPromptSuggestions(localStorage.token, promptSuggestions);
await updateBanners();
await config.set(await getBackendConfig());
};
onMount(async () => {
taskConfig = await getTaskConfig(localStorage.token);
promptSuggestions = $config?.default_prompt_suggestions;
banners = await getBanners(localStorage.token);
});
const updateBanners = async () => {
_banners.set(await setBanners(localStorage.token, banners));
};
</script>
<form
class="flex flex-col h-full justify-between space-y-3 text-sm"
on:submit|preventDefault={() => {
updateInterfaceHandler();
dispatch('save');
}}
>
<div class=" overflow-y-scroll scrollbar-hidden h-full pr-1.5">
<div>
<div class=" mb-2.5 text-sm font-medium flex">
<div class=" mr-1">{$i18n.t('Set Task Model')}</div>
<Tooltip
content={$i18n.t(
'A task model is used when performing tasks such as generating titles for chats and web search queries'
)}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-5 h-5"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z"
/>
</svg>
</Tooltip>
</div>
<div class="flex w-full gap-2">
<div class="flex-1">
<div class=" text-xs mb-1">{$i18n.t('Local Models')}</div>
<select
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={taskConfig.TASK_MODEL}
placeholder={$i18n.t('Select a model')}
>
<option value="" selected>{$i18n.t('Current Model')}</option>
{#each $models.filter((m) => m.owned_by === 'ollama') as model}
<option value={model.id} class="bg-gray-100 dark:bg-gray-700">
{model.name}
</option>
{/each}
</select>
</div>
<div class="flex-1">
<div class=" text-xs mb-1">{$i18n.t('External Models')}</div>
<select
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={taskConfig.TASK_MODEL_EXTERNAL}
placeholder={$i18n.t('Select a model')}
>
<option value="" selected>{$i18n.t('Current Model')}</option>
{#each $models as model}
<option value={model.id} class="bg-gray-100 dark:bg-gray-700">
{model.name}
</option>
{/each}
</select>
</div>
</div>
<div class="mt-3">
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Title Generation Prompt')}</div>
<textarea
bind:value={taskConfig.TITLE_GENERATION_PROMPT_TEMPLATE}
class="w-full rounded-lg py-3 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
rows="6"
/>
</div>
<div class="mt-3">
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Search Query Generation Prompt')}</div>
<textarea
bind:value={taskConfig.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE}
class="w-full rounded-lg py-3 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
rows="6"
/>
</div>
<div class="mt-3">
<div class=" mb-2.5 text-sm font-medium">
{$i18n.t('Search Query Generation Prompt Length Threshold')}
</div>
<input
bind:value={taskConfig.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD}
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
type="number"
/>
</div>
</div>
<hr class=" dark:border-gray-850 my-3" />
<div class=" space-y-3 {banners.length > 0 ? ' mb-3' : ''}">
<div class="flex w-full justify-between">
<div class=" self-center text-sm font-semibold">
{$i18n.t('Banners')}
</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
if (banners.length === 0 || banners.at(-1).content !== '') {
banners = [
...banners,
{
id: uuidv4(),
type: '',
title: '',
content: '',
dismissible: true,
timestamp: Math.floor(Date.now() / 1000)
}
];
}
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
/>
</svg>
</button>
</div>
<div class="flex flex-col space-y-1">
{#each banners as banner, bannerIdx}
<div class=" flex justify-between">
<div class="flex flex-row flex-1 border rounded-xl dark:border-gray-800">
<select
class="w-fit capitalize rounded-xl py-2 px-4 text-xs bg-transparent outline-none"
bind:value={banner.type}
required
>
{#if banner.type == ''}
<option value="" selected disabled class="text-gray-900">{$i18n.t('Type')}</option
>
{/if}
<option value="info" class="text-gray-900">{$i18n.t('Info')}</option>
<option value="warning" class="text-gray-900">{$i18n.t('Warning')}</option>
<option value="error" class="text-gray-900">{$i18n.t('Error')}</option>
<option value="success" class="text-gray-900">{$i18n.t('Success')}</option>
</select>
<input
class="pr-5 py-1.5 text-xs w-full bg-transparent outline-none"
placeholder={$i18n.t('Content')}
bind:value={banner.content}
/>
<div class="relative top-1.5 -left-2">
<Tooltip content={$i18n.t('Dismissible')} className="flex h-fit items-center">
<Switch bind:state={banner.dismissible} />
</Tooltip>
</div>
</div>
<button
class="px-2"
type="button"
on:click={() => {
banners.splice(bannerIdx, 1);
banners = banners;
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
/>
</svg>
</button>
</div>
{/each}
</div>
</div>
{#if $user.role === 'admin'}
<div class=" space-y-3">
<div class="flex w-full justify-between mb-2">
<div class=" self-center text-sm font-semibold">
{$i18n.t('Default Prompt Suggestions')}
</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
if (promptSuggestions.length === 0 || promptSuggestions.at(-1).content !== '') {
promptSuggestions = [...promptSuggestions, { content: '', title: ['', ''] }];
}
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
/>
</svg>
</button>
</div>
<div class="grid lg:grid-cols-2 flex-col gap-1.5">
{#each promptSuggestions as prompt, promptIdx}
<div class=" flex dark:bg-gray-850 rounded-xl py-1.5">
<div class="flex flex-col flex-1 pl-1">
<div class="flex border-b dark:border-gray-800 w-full">
<input
class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-800"
placeholder={$i18n.t('Title (e.g. Tell me a fun fact)')}
bind:value={prompt.title[0]}
/>
<input
class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-800"
placeholder={$i18n.t('Subtitle (e.g. about the Roman Empire)')}
bind:value={prompt.title[1]}
/>
</div>
<input
class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-800"
placeholder={$i18n.t('Prompt (e.g. Tell me a fun fact about the Roman Empire)')}
bind:value={prompt.content}
/>
</div>
<button
class="px-3"
type="button"
on:click={() => {
promptSuggestions.splice(promptIdx, 1);
promptSuggestions = promptSuggestions;
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
/>
</svg>
</button>
</div>
{/each}
</div>
{#if promptSuggestions.length > 0}
<div class="text-xs text-left w-full mt-2">
{$i18n.t('Adjusting these settings will apply changes universally to all users.')}
</div>
{/if}
</div>
{/if}
</div>
<div class="flex justify-end text-sm font-medium">
<button
class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
type="submit"
>
{$i18n.t('Save')}
</button>
</div>
</form>

1078
src/lib/components/admin/Settings/Models.svelte Normal file
View File

File diff suppressed because it is too large Load Diff

150
src/lib/components/admin/Settings/Pipelines.svelte
View File

@ -14,7 +14,8 @@
getModels,
getPipelinesList,
downloadPipeline,
deletePipeline
deletePipeline,
uploadPipeline
} from '$lib/apis';
import Spinner from '$lib/components/common/Spinner.svelte';
@ -24,6 +25,9 @@
export let saveHandler: Function;
let downloading = false;
let uploading = false;
let pipelineFiles;
let PIPELINES_LIST = null;
let selectedPipelinesUrlIdx = '';
@ -40,8 +44,10 @@
const pipeline = pipelines[selectedPipelineIdx];
if (pipeline && (pipeline?.valves ?? false)) {
if (valves?.pipelines ?? false) {
valves.pipelines = valves.pipelines.split(',').map((v) => v.trim());
for (const property in valves_spec.properties) {
if (valves_spec.properties[property]?.type === 'array') {
valves[property] = valves[property].split(',').map((v) => v.trim());
}
}
const res = await updatePipelineValves(
@ -79,8 +85,10 @@
selectedPipelinesUrlIdx
);
if (valves?.pipelines ?? false) {
valves.pipelines = valves.pipelines.join(',');
for (const property in valves_spec.properties) {
if (valves_spec.properties[property]?.type === 'array') {
valves[property] = valves[property].join(',');
}
}
};
@ -122,6 +130,41 @@
downloading = false;
};
const uploadPipelineHandler = async () => {
uploading = true;
if (pipelineFiles && pipelineFiles.length !== 0) {
const file = pipelineFiles[0];
console.log(file);
const res = await uploadPipeline(localStorage.token, file, selectedPipelinesUrlIdx).catch(
(error) => {
console.log(error);
toast.error('Something went wrong :/');
return null;
}
);
if (res) {
toast.success('Pipeline downloaded successfully');
setPipelines();
models.set(await getModels(localStorage.token));
}
} else {
toast.error('No file selected');
}
pipelineFiles = null;
const pipelineUploadInputElement = document.getElementById('pipeline-upload-input');
if (pipelineUploadInputElement) {
pipelineUploadInputElement.value = null;
}
uploading = false;
};
const deletePipelineHandler = async () => {
const res = await deletePipeline(
localStorage.token,
@ -157,7 +200,7 @@
updateHandler();
}}
>
<div class=" pr-1.5 overflow-y-scroll max-h-80 h-full">
<div class="overflow-y-scroll scrollbar-hidden h-full">
{#if PIPELINES_LIST !== null}
<div class="flex w-full justify-between mb-2">
<div class=" self-center text-sm font-semibold">
@ -192,6 +235,91 @@
</div>
</div>
<div class=" my-2">
<div class=" mb-2 text-sm font-medium">
{$i18n.t('Upload Pipeline')}
</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<input
id="pipelines-upload-input"
bind:files={pipelineFiles}
type="file"
accept=".py"
hidden
/>
<button
class="w-full text-sm font-medium py-2 bg-transparent hover:bg-gray-100 border border-dashed dark:border-gray-800 dark:hover:bg-gray-850 text-center rounded-xl"
type="button"
on:click={() => {
document.getElementById('pipelines-upload-input')?.click();
}}
>
{#if pipelineFiles}
{pipelineFiles.length > 0 ? `${pipelineFiles.length}` : ''} pipeline(s) selected.
{:else}
{$i18n.t('Click here to select a py file.')}
{/if}
</button>
</div>
<button
class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
on:click={() => {
uploadPipelineHandler();
}}
disabled={uploading}
type="button"
>
{#if uploading}
<div class="self-center">
<svg
class=" w-4 h-4"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
>
<style>
.spinner_ajPY {
transform-origin: center;
animation: spinner_AtaB 0.75s infinite linear;
}
@keyframes spinner_AtaB {
100% {
transform: rotate(360deg);
}
}
</style>
<path
d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
opacity=".25"
/>
<path
d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
class="spinner_ajPY"
/>
</svg>
</div>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="size-4"
>
<path
d="M7.25 10.25a.75.75 0 0 0 1.5 0V4.56l2.22 2.22a.75.75 0 1 0 1.06-1.06l-3.5-3.5a.75.75 0 0 0-1.06 0l-3.5 3.5a.75.75 0 0 0 1.06 1.06l2.22-2.22v5.69Z"
/>
<path
d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
/>
</svg>
{/if}
</button>
</div>
</div>
<div class=" my-2">
<div class=" mb-2 text-sm font-medium">
{$i18n.t('Install from Github URL')}
@ -259,6 +387,14 @@
{/if}
</button>
</div>
<div class="mt-2 text-xs text-gray-500">
<span class=" font-semibold dark:text-gray-200">Warning:</span> Pipelines are a plugin
system with arbitrary code execution —
<span class=" font-medium dark:text-gray-400"
>don't fetch random pipelines from sources you don't trust.</span
>
</div>
</div>
<hr class=" dark:border-gray-800 my-3 w-full" />
@ -372,6 +508,8 @@
</div>
</div>
{/if}
{:else}
<div>Pipelines Not Detected</div>
{/if}
{:else}
<div class="flex justify-center h-full">

8
src/lib/components/admin/Settings/Users.svelte
View File

@ -48,7 +48,7 @@
await config.set(await getBackendConfig());
}}
>
<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
<div class=" space-y-3 overflow-y-scroll max-h-full">
<div>
<div class=" mb-2 text-sm font-medium">{$i18n.t('User Permissions')}</div>
@ -58,11 +58,11 @@
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
permissions.chat.deletion = !permissions.chat.deletion;
permissions.chat.deletion = !(permissions?.chat?.deletion ?? true);
}}
type="button"
>
{#if permissions.chat.deletion}
{#if permissions?.chat?.deletion ?? true}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
@ -94,7 +94,7 @@
</div>
</div>
<hr class=" dark:border-gray-700 my-2" />
<hr class=" dark:border-gray-850 my-2" />
<div class="mt-2 space-y-3">
<div>

311
src/lib/components/admin/Settings/WebSearch.svelte Normal file
View File

@ -0,0 +1,311 @@
<script lang="ts">
import { getRAGConfig, updateRAGConfig } from '$lib/apis/rag';
import Switch from '$lib/components/common/Switch.svelte';
import { documents, models } from '$lib/stores';
import { onMount, getContext } from 'svelte';
import { toast } from 'svelte-sonner';
const i18n = getContext('i18n');
export let saveHandler: Function;
let webConfig = null;
let webSearchEngines = [
'searxng',
'google_pse',
'brave',
'serpstack',
'serper',
'serply',
'duckduckgo'
];
let youtubeLanguage = 'en';
let youtubeTranslation = null;
const submitHandler = async () => {
const res = await updateRAGConfig(localStorage.token, {
web: webConfig,
youtube: {
language: youtubeLanguage.split(',').map((lang) => lang.trim()),
translation: youtubeTranslation
}
});
};
onMount(async () => {
const res = await getRAGConfig(localStorage.token);
if (res) {
webConfig = res.web;
youtubeLanguage = res.youtube.language.join(',');
youtubeTranslation = res.youtube.translation;
}
});
</script>
<form
class="flex flex-col h-full justify-between space-y-3 text-sm"
on:submit|preventDefault={async () => {
await submitHandler();
saveHandler();
}}
>
<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
{#if webConfig}
<div>
<div class=" mb-1 text-sm font-medium">
{$i18n.t('Web Search')}
</div>
<div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">
{$i18n.t('Enable Web Search')}
</div>
<Switch bind:state={webConfig.search.enabled} />
</div>
</div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Web Search Engine')}</div>
<div class="flex items-center relative">
<select
class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
bind:value={webConfig.search.engine}
placeholder={$i18n.t('Select a engine')}
required
>
<option disabled selected value="">{$i18n.t('Select a engine')}</option>
{#each webSearchEngines as engine}
<option value={engine}>{engine}</option>
{/each}
</select>
</div>
</div>
{#if webConfig.search.engine !== ''}
<div class="mt-1.5">
{#if webConfig.search.engine === 'searxng'}
<div>
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Searxng Query URL')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Searxng Query URL')}
bind:value={webConfig.search.searxng_query_url}
autocomplete="off"
/>
</div>
</div>
</div>
{:else if webConfig.search.engine === 'google_pse'}
<div>
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Google PSE API Key')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Google PSE API Key')}
bind:value={webConfig.search.google_pse_api_key}
autocomplete="off"
/>
</div>
</div>
</div>
<div class="mt-1.5">
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Google PSE Engine Id')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Google PSE Engine Id')}
bind:value={webConfig.search.google_pse_engine_id}
autocomplete="off"
/>
</div>
</div>
</div>
{:else if webConfig.search.engine === 'brave'}
<div>
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Brave Search API Key')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Brave Search API Key')}
bind:value={webConfig.search.brave_search_api_key}
autocomplete="off"
/>
</div>
</div>
</div>
{:else if webConfig.search.engine === 'serpstack'}
<div>
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Serpstack API Key')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Serpstack API Key')}
bind:value={webConfig.search.serpstack_api_key}
autocomplete="off"
/>
</div>
</div>
</div>
{:else if webConfig.search.engine === 'serper'}
<div>
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Serper API Key')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Serper API Key')}
bind:value={webConfig.search.serper_api_key}
autocomplete="off"
/>
</div>
</div>
</div>
{:else if webConfig.search.engine === 'serply'}
<div>
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Serply API Key')}
</div>
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter Serply API Key')}
bind:value={webConfig.search.serply_api_key}
autocomplete="off"
/>
</div>
</div>
</div>
{/if}
</div>
{/if}
{#if webConfig.search.enabled}
<div class="mt-2 flex gap-2 mb-1">
<div class="w-full">
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Search Result Count')}
</div>
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('Search Result Count')}
bind:value={webConfig.search.result_count}
required
/>
</div>
<div class="w-full">
<div class=" self-center text-xs font-medium mb-1">
{$i18n.t('Concurrent Requests')}
</div>
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('Concurrent Requests')}
bind:value={webConfig.search.concurrent_requests}
required
/>
</div>
</div>
{/if}
</div>
<hr class=" dark:border-gray-850 my-2" />
<div>
<div class=" mb-1 text-sm font-medium">
{$i18n.t('Web Loader Settings')}
</div>
<div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">
{$i18n.t('Bypass SSL verification for Websites')}
</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
webConfig.ssl_verification = !webConfig.ssl_verification;
submitHandler();
}}
type="button"
>
{#if webConfig.ssl_verification === true}
<span class="ml-2 self-center">{$i18n.t('On')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Off')}</span>
{/if}
</button>
</div>
</div>
<div class=" mt-2 mb-1 text-sm font-medium">
{$i18n.t('Youtube Loader Settings')}
</div>
<div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" w-20 text-xs font-medium self-center">{$i18n.t('Language')}</div>
<div class=" flex-1 self-center">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
type="text"
placeholder={$i18n.t('Enter language codes')}
bind:value={youtubeLanguage}
autocomplete="off"
/>
</div>
</div>
</div>
</div>
{/if}
</div>
<div class="flex justify-end pt-3 text-sm font-medium">
<button
class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
type="submit"
>
{$i18n.t('Save')}
</button>
</div>
</form>

176
src/lib/components/admin/SettingsModal.svelte
View File

@ -39,181 +39,5 @@
</svg>
</button>
</div>
<div class="flex flex-col md:flex-row w-full p-4 md:space-x-4">
<div
class="tabs flex flex-row overflow-x-auto space-x-1 md:space-x-0 md:space-y-1 md:flex-col flex-1 md:flex-none md:w-40 dark:text-gray-200 text-xs text-left mb-3 md:mb-0"
>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'general'
? 'bg-gray-200 dark:bg-gray-700'
: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
on:click={() => {
selectedTab = 'general';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M6.955 1.45A.5.5 0 0 1 7.452 1h1.096a.5.5 0 0 1 .497.45l.17 1.699c.484.12.94.312 1.356.562l1.321-1.081a.5.5 0 0 1 .67.033l.774.775a.5.5 0 0 1 .034.67l-1.08 1.32c.25.417.44.873.561 1.357l1.699.17a.5.5 0 0 1 .45.497v1.096a.5.5 0 0 1-.45.497l-1.699.17c-.12.484-.312.94-.562 1.356l1.082 1.322a.5.5 0 0 1-.034.67l-.774.774a.5.5 0 0 1-.67.033l-1.322-1.08c-.416.25-.872.44-1.356.561l-.17 1.699a.5.5 0 0 1-.497.45H7.452a.5.5 0 0 1-.497-.45l-.17-1.699a4.973 4.973 0 0 1-1.356-.562L4.108 13.37a.5.5 0 0 1-.67-.033l-.774-.775a.5.5 0 0 1-.034-.67l1.08-1.32a4.971 4.971 0 0 1-.561-1.357l-1.699-.17A.5.5 0 0 1 1 8.548V7.452a.5.5 0 0 1 .45-.497l1.699-.17c.12-.484.312-.94.562-1.356L2.629 4.107a.5.5 0 0 1 .034-.67l.774-.774a.5.5 0 0 1 .67-.033L5.43 3.71a4.97 4.97 0 0 1 1.356-.561l.17-1.699ZM6 8c0 .538.212 1.026.558 1.385l.057.057a2 2 0 0 0 2.828-2.828l-.058-.056A2 2 0 0 0 6 8Z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('General')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'users'
? 'bg-gray-200 dark:bg-gray-700'
: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
on:click={() => {
selectedTab = 'users';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M8 8a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5ZM3.156 11.763c.16-.629.44-1.21.813-1.72a2.5 2.5 0 0 0-2.725 1.377c-.136.287.102.58.418.58h1.449c.01-.077.025-.156.045-.237ZM12.847 11.763c.02.08.036.16.046.237h1.446c.316 0 .554-.293.417-.579a2.5 2.5 0 0 0-2.722-1.378c.374.51.653 1.09.813 1.72ZM14 7.5a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0ZM3.5 9a1.5 1.5 0 1 0 0-3 1.5 1.5 0 0 0 0 3ZM5 13c-.552 0-1.013-.455-.876-.99a4.002 4.002 0 0 1 7.753 0c.136.535-.324.99-.877.99H5Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Users')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'db'
? 'bg-gray-200 dark:bg-gray-700'
: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
on:click={() => {
selectedTab = 'db';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path d="M8 7c3.314 0 6-1.343 6-3s-2.686-3-6-3-6 1.343-6 3 2.686 3 6 3Z" />
<path
d="M8 8.5c1.84 0 3.579-.37 4.914-1.037A6.33 6.33 0 0 0 14 6.78V8c0 1.657-2.686 3-6 3S2 9.657 2 8V6.78c.346.273.72.5 1.087.683C4.42 8.131 6.16 8.5 8 8.5Z"
/>
<path
d="M8 12.5c1.84 0 3.579-.37 4.914-1.037.366-.183.74-.41 1.086-.684V12c0 1.657-2.686 3-6 3s-6-1.343-6-3v-1.22c.346.273.72.5 1.087.683C4.42 12.131 6.16 12.5 8 12.5Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Database')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'banners'
? 'bg-gray-200 dark:bg-gray-700'
: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
on:click={() => {
selectedTab = 'banners';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="size-4"
>
<path
d="M5.85 3.5a.75.75 0 0 0-1.117-1 9.719 9.719 0 0 0-2.348 4.876.75.75 0 0 0 1.479.248A8.219 8.219 0 0 1 5.85 3.5ZM19.267 2.5a.75.75 0 1 0-1.118 1 8.22 8.22 0 0 1 1.987 4.124.75.75 0 0 0 1.48-.248A9.72 9.72 0 0 0 19.266 2.5Z"
/>
<path
fill-rule="evenodd"
d="M12 2.25A6.75 6.75 0 0 0 5.25 9v.75a8.217 8.217 0 0 1-2.119 5.52.75.75 0 0 0 .298 1.206c1.544.57 3.16.99 4.831 1.243a3.75 3.75 0 1 0 7.48 0 24.583 24.583 0 0 0 4.83-1.244.75.75 0 0 0 .298-1.205 8.217 8.217 0 0 1-2.118-5.52V9A6.75 6.75 0 0 0 12 2.25ZM9.75 18c0-.034 0-.067.002-.1a25.05 25.05 0 0 0 4.496 0l.002.1a2.25 2.25 0 1 1-4.5 0Z"
clip-rule="evenodd"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Banners')}</div>
</button>
<button
class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
'pipelines'
? 'bg-gray-200 dark:bg-gray-700'
: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
on:click={() => {
selectedTab = 'pipelines';
}}
>
<div class=" self-center mr-2">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
class="size-4"
>
<path
d="M11.644 1.59a.75.75 0 0 1 .712 0l9.75 5.25a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.712 0l-9.75-5.25a.75.75 0 0 1 0-1.32l9.75-5.25Z"
/>
<path
d="m3.265 10.602 7.668 4.129a2.25 2.25 0 0 0 2.134 0l7.668-4.13 1.37.739a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.71 0l-9.75-5.25a.75.75 0 0 1 0-1.32l1.37-.738Z"
/>
<path
d="m10.933 19.231-7.668-4.13-1.37.739a.75.75 0 0 0 0 1.32l9.75 5.25c.221.12.489.12.71 0l9.75-5.25a.75.75 0 0 0 0-1.32l-1.37-.738-7.668 4.13a2.25 2.25 0 0 1-2.134-.001Z"
/>
</svg>
</div>
<div class=" self-center">{$i18n.t('Pipelines')}</div>
</button>
</div>
<div class="flex-1 md:min-h-[380px]">
{#if selectedTab === 'general'}
<General
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'users'}
<Users
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'db'}
<Database
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'banners'}
<Banners
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{:else if selectedTab === 'pipelines'}
<Pipelines
saveHandler={() => {
toast.success($i18n.t('Settings saved successfully!'));
}}
/>
{/if}
</div>
</div>
</div>
</Modal>

767
src/lib/components/chat/Chat.svelte
View File

File diff suppressed because it is too large Load Diff

1390
src/lib/components/chat/MessageInput.svelte
View File

File diff suppressed because it is too large Load Diff

694
src/lib/components/chat/MessageInput/CallOverlay.svelte Normal file
View File

@ -0,0 +1,694 @@
<script lang="ts">
import { config, settings, showCallOverlay } from '$lib/stores';
import { onMount, tick, getContext } from 'svelte';
import { blobToFile, calculateSHA256, extractSentences, findWordIndices } from '$lib/utils';
import { synthesizeOpenAISpeech, transcribeAudio } from '$lib/apis/audio';
import { toast } from 'svelte-sonner';
import Tooltip from '$lib/components/common/Tooltip.svelte';
import VideoInputMenu from './CallOverlay/VideoInputMenu.svelte';
import { get } from 'svelte/store';
const i18n = getContext('i18n');
export let submitPrompt: Function;
export let files;
let loading = false;
let confirmed = false;
let camera = false;
let cameraStream = null;
let assistantSpeaking = false;
let assistantAudio = {};
let assistantAudioIdx = null;
let rmsLevel = 0;
let hasStartedSpeaking = false;
let currentUtterance = null;
let mediaRecorder;
let audioChunks = [];
const MIN_DECIBELS = -45;
const VISUALIZER_BUFFER_LENGTH = 300;
// Function to calculate the RMS level from time domain data
const calculateRMS = (data: Uint8Array) => {
let sumSquares = 0;
for (let i = 0; i < data.length; i++) {
const normalizedValue = (data[i] - 128) / 128; // Normalize the data
sumSquares += normalizedValue * normalizedValue;
}
return Math.sqrt(sumSquares / data.length);
};
const normalizeRMS = (rms) => {
rms = rms * 10;
const exp = 1.5; // Adjust exponent value; values greater than 1 expand larger numbers more and compress smaller numbers more
const scaledRMS = Math.pow(rms, exp);
// Scale between 0.01 (1%) and 1.0 (100%)
return Math.min(1.0, Math.max(0.01, scaledRMS));
};
const analyseAudio = (stream) => {
const audioContext = new AudioContext();
const audioStreamSource = audioContext.createMediaStreamSource(stream);
const analyser = audioContext.createAnalyser();
analyser.minDecibels = MIN_DECIBELS;
audioStreamSource.connect(analyser);
const bufferLength = analyser.frequencyBinCount;
const domainData = new Uint8Array(bufferLength);
const timeDomainData = new Uint8Array(analyser.fftSize);
let lastSoundTime = Date.now();
hasStartedSpeaking = false;
const detectSound = () => {
const processFrame = () => {
if (!mediaRecorder || !$showCallOverlay) {
if (mediaRecorder) {
mediaRecorder.stop();
}
return;
}
analyser.getByteTimeDomainData(timeDomainData);
analyser.getByteFrequencyData(domainData);
// Calculate RMS level from time domain data
rmsLevel = calculateRMS(timeDomainData);
// Check if initial speech/noise has started
const hasSound = domainData.some((value) => value > 0);
if (hasSound) {
stopAllAudio();
hasStartedSpeaking = true;
lastSoundTime = Date.now();
}
// Start silence detection only after initial speech/noise has been detected
if (hasStartedSpeaking) {
if (Date.now() - lastSoundTime > 2000) {
confirmed = true;
if (mediaRecorder) {
mediaRecorder.stop();
}
}
}
window.requestAnimationFrame(processFrame);
};
window.requestAnimationFrame(processFrame);
};
detectSound();
};
const stopAllAudio = () => {
if (currentUtterance) {
speechSynthesis.cancel();
currentUtterance = null;
}
if (assistantAudio[assistantAudioIdx]) {
assistantAudio[assistantAudioIdx].pause();
assistantAudio[assistantAudioIdx].currentTime = 0;
}
const audioElement = document.getElementById('audioElement');
audioElement.pause();
audioElement.currentTime = 0;
assistantSpeaking = false;
};
const playAudio = (idx) => {
if ($showCallOverlay) {
return new Promise((res) => {
assistantAudioIdx = idx;
const audioElement = document.getElementById('audioElement');
const audio = assistantAudio[idx];
audioElement.src = audio.src; // Assume `assistantAudio` has objects with a `src` property
audioElement.muted = true;
audioElement
.play()
.then(() => {
audioElement.muted = false;
})
.catch((error) => {
toast.error(error);
});
audioElement.onended = async (e) => {
await new Promise((r) => setTimeout(r, 300));
if (Object.keys(assistantAudio).length - 1 === idx) {
assistantSpeaking = false;
}
res(e);
};
});
} else {
return Promise.resolve();
}
};
const getOpenAISpeech = async (text) => {
const res = await synthesizeOpenAISpeech(
localStorage.token,
$settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice,
text
).catch((error) => {
toast.error(error);
assistantSpeaking = false;
return null;
});
if (res) {
const blob = await res.blob();
const blobUrl = URL.createObjectURL(blob);
const audio = new Audio(blobUrl);
assistantAudio = audio;
}
};
const transcribeHandler = async (audioBlob) => {
// Create a blob from the audio chunks
await tick();
const file = blobToFile(audioBlob, 'recording.wav');
const res = await transcribeAudio(localStorage.token, file).catch((error) => {
toast.error(error);
return null;
});
if (res) {
console.log(res.text);
if (res.text !== '') {
const _responses = await submitPrompt(res.text);
console.log(_responses);
if (_responses.at(0)) {
const content = _responses[0];
if ((content ?? '').trim() !== '') {
assistantSpeakingHandler(content);
}
}
}
}
};
const assistantSpeakingHandler = async (content) => {
assistantSpeaking = true;
if (($config.audio.tts.engine ?? '') == '') {
let voices = [];
const getVoicesLoop = setInterval(async () => {
voices = await speechSynthesis.getVoices();
if (voices.length > 0) {
clearInterval(getVoicesLoop);
const voice =
voices
?.filter(
(v) => v.voiceURI === ($settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice)
)
?.at(0) ?? undefined;
currentUtterance = new SpeechSynthesisUtterance(content);
if (voice) {
currentUtterance.voice = voice;
}
speechSynthesis.speak(currentUtterance);
}
}, 100);
} else if ($config.audio.tts.engine === 'openai') {
console.log('openai');
const sentences = extractSentences(content).reduce((mergedTexts, currentText) => {
const lastIndex = mergedTexts.length - 1;
if (lastIndex >= 0) {
const previousText = mergedTexts[lastIndex];
const wordCount = previousText.split(/\s+/).length;
if (wordCount < 2) {
mergedTexts[lastIndex] = previousText + ' ' + currentText;
} else {
mergedTexts.push(currentText);
}
} else {
mergedTexts.push(currentText);
}
return mergedTexts;
}, []);
console.log(sentences);
let lastPlayedAudioPromise = Promise.resolve(); // Initialize a promise that resolves immediately
for (const [idx, sentence] of sentences.entries()) {
const res = await synthesizeOpenAISpeech(
localStorage.token,
$settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice,
sentence
).catch((error) => {
toast.error(error);
assistantSpeaking = false;
return null;
});
if (res) {
const blob = await res.blob();
const blobUrl = URL.createObjectURL(blob);
const audio = new Audio(blobUrl);
assistantAudio[idx] = audio;
lastPlayedAudioPromise = lastPlayedAudioPromise.then(() => playAudio(idx));
}
}
}
};
const stopRecordingCallback = async () => {
if ($showCallOverlay) {
if (confirmed) {
loading = true;
if (cameraStream) {
const imageUrl = takeScreenshot();
files = [
{
type: 'image',
url: imageUrl
}
];
}
const audioBlob = new Blob(audioChunks, { type: 'audio/wav' });
await transcribeHandler(audioBlob);
confirmed = false;
loading = false;
}
audioChunks = [];
mediaRecorder = false;
startRecording();
} else {
audioChunks = [];
mediaRecorder = false;
}
};
const startRecording = async () => {
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
mediaRecorder = new MediaRecorder(stream);
mediaRecorder.onstart = () => {
console.log('Recording started');
audioChunks = [];
analyseAudio(stream);
};
mediaRecorder.ondataavailable = (event) => {
if (hasStartedSpeaking) {
audioChunks.push(event.data);
}
};
mediaRecorder.onstop = async () => {
console.log('Recording stopped');
await stopRecordingCallback();
};
mediaRecorder.start();
};
let videoInputDevices = [];
let selectedVideoInputDeviceId = null;
const getVideoInputDevices = async () => {
const devices = await navigator.mediaDevices.enumerateDevices();
videoInputDevices = devices.filter((device) => device.kind === 'videoinput');
if (!!navigator.mediaDevices.getDisplayMedia) {
videoInputDevices = [
...videoInputDevices,
{
deviceId: 'screen',
label: 'Screen Share'
}
];
}
console.log(videoInputDevices);
if (selectedVideoInputDeviceId === null && videoInputDevices.length > 0) {
selectedVideoInputDeviceId = videoInputDevices[0].deviceId;
}
};
const startCamera = async () => {
await getVideoInputDevices();
if (cameraStream === null) {
camera = true;
await tick();
try {
await startVideoStream();
} catch (err) {
console.error('Error accessing webcam: ', err);
}
}
};
const startVideoStream = async () => {
const video = document.getElementById('camera-feed');
if (video) {
if (selectedVideoInputDeviceId === 'screen') {
cameraStream = await navigator.mediaDevices.getDisplayMedia({
video: {
cursor: 'always'
},
audio: false
});
} else {
cameraStream = await navigator.mediaDevices.getUserMedia({
video: {
deviceId: selectedVideoInputDeviceId ? { exact: selectedVideoInputDeviceId } : undefined
}
});
}
if (cameraStream) {
await getVideoInputDevices();
video.srcObject = cameraStream;
await video.play();
}
}
};
const stopVideoStream = async () => {
if (cameraStream) {
const tracks = cameraStream.getTracks();
tracks.forEach((track) => track.stop());
}
cameraStream = null;
};
const takeScreenshot = () => {
const video = document.getElementById('camera-feed');
const canvas = document.getElementById('camera-canvas');
if (!canvas) {
return;
}
const context = canvas.getContext('2d');
// Make the canvas match the video dimensions
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
// Draw the image from the video onto the canvas
context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
// Convert the canvas to a data base64 URL and console log it
const dataURL = canvas.toDataURL('image/png');
console.log(dataURL);
return dataURL;
};
const stopCamera = async () => {
await stopVideoStream();
camera = false;
};
$: if ($showCallOverlay) {
startRecording();
} else {
stopCamera();
}
</script>
{#if $showCallOverlay}
<audio id="audioElement" src="" style="display: none;" />
<div class=" absolute w-full h-screen max-h-[100dvh] flex z-[999] overflow-hidden">
<div
class="absolute w-full h-screen max-h-[100dvh] bg-white text-gray-700 dark:bg-black dark:text-gray-300 flex justify-center"
>
<div class="max-w-lg w-full h-screen max-h-[100dvh] flex flex-col justify-between p-3 md:p-6">
{#if camera}
<div class="flex justify-center items-center w-full min-h-20">
{#if loading}
<svg
class="size-12 text-gray-900 dark:text-gray-400"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_qM83 {
animation: spinner_8HQG 1.05s infinite;
}
.spinner_oXPr {
animation-delay: 0.1s;
}
.spinner_ZTLf {
animation-delay: 0.2s;
}
@keyframes spinner_8HQG {
0%,
57.14% {
animation-timing-function: cubic-bezier(0.33, 0.66, 0.66, 1);
transform: translate(0);
}
28.57% {
animation-timing-function: cubic-bezier(0.33, 0, 0.66, 0.33);
transform: translateY(-6px);
}
100% {
transform: translate(0);
}
}
</style><circle class="spinner_qM83" cx="4" cy="12" r="3" /><circle
class="spinner_qM83 spinner_oXPr"
cx="12"
cy="12"
r="3"
/><circle class="spinner_qM83 spinner_ZTLf" cx="20" cy="12" r="3" /></svg
>
{:else}
<div
class=" {rmsLevel * 100 > 4
? ' size-[4.5rem]'
: rmsLevel * 100 > 2
? ' size-16'
: rmsLevel * 100 > 1
? 'size-14'
: 'size-12'} transition-all bg-black dark:bg-white rounded-full"
/>
{/if}
<!-- navbar -->
</div>
{/if}
<div class="flex justify-center items-center flex-1 h-full w-full max-h-full">
{#if !camera}
{#if loading}
<svg
class="size-44 text-gray-900 dark:text-gray-400"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_qM83 {
animation: spinner_8HQG 1.05s infinite;
}
.spinner_oXPr {
animation-delay: 0.1s;
}
.spinner_ZTLf {
animation-delay: 0.2s;
}
@keyframes spinner_8HQG {
0%,
57.14% {
animation-timing-function: cubic-bezier(0.33, 0.66, 0.66, 1);
transform: translate(0);
}
28.57% {
animation-timing-function: cubic-bezier(0.33, 0, 0.66, 0.33);
transform: translateY(-6px);
}
100% {
transform: translate(0);
}
}
</style><circle class="spinner_qM83" cx="4" cy="12" r="3" /><circle
class="spinner_qM83 spinner_oXPr"
cx="12"
cy="12"
r="3"
/><circle class="spinner_qM83 spinner_ZTLf" cx="20" cy="12" r="3" /></svg
>
{:else}
<div
class=" {rmsLevel * 100 > 4
? ' size-52'
: rmsLevel * 100 > 2
? 'size-48'
: rmsLevel * 100 > 1
? 'size-[11.5rem]'
: 'size-44'} transition-all bg-black dark:bg-white rounded-full"
/>
{/if}
{:else}
<div
class="relative flex video-container w-full max-h-full pt-2 pb-4 md:py-6 px-2 h-full"
>
<video
id="camera-feed"
autoplay
class="rounded-2xl h-full min-w-full object-cover object-center"
playsinline
/>
<canvas id="camera-canvas" style="display:none;" />
<div class=" absolute top-4 md:top-8 left-4">
<button
type="button"
class="p-1.5 text-white cursor-pointer backdrop-blur-xl bg-black/10 rounded-full"
on:click={() => {
stopCamera();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="size-6"
>
<path
d="M5.28 4.22a.75.75 0 0 0-1.06 1.06L6.94 8l-2.72 2.72a.75.75 0 1 0 1.06 1.06L8 9.06l2.72 2.72a.75.75 0 1 0 1.06-1.06L9.06 8l2.72-2.72a.75.75 0 0 0-1.06-1.06L8 6.94 5.28 4.22Z"
/>
</svg>
</button>
</div>
</div>
{/if}
</div>
<div class="flex justify-between items-center pb-2 w-full">
<div>
{#if camera}
<VideoInputMenu
devices={videoInputDevices}
on:change={async (e) => {
console.log(e.detail);
selectedVideoInputDeviceId = e.detail;
await stopVideoStream();
await startVideoStream();
}}
>
<button class=" p-3 rounded-full bg-gray-50 dark:bg-gray-900" type="button">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="size-5"
>
<path
fill-rule="evenodd"
d="M15.312 11.424a5.5 5.5 0 0 1-9.201 2.466l-.312-.311h2.433a.75.75 0 0 0 0-1.5H3.989a.75.75 0 0 0-.75.75v4.242a.75.75 0 0 0 1.5 0v-2.43l.31.31a7 7 0 0 0 11.712-3.138.75.75 0 0 0-1.449-.39Zm1.23-3.723a.75.75 0 0 0 .219-.53V2.929a.75.75 0 0 0-1.5 0V5.36l-.31-.31A7 7 0 0 0 3.239 8.188a.75.75 0 1 0 1.448.389A5.5 5.5 0 0 1 13.89 6.11l.311.31h-2.432a.75.75 0 0 0 0 1.5h4.243a.75.75 0 0 0 .53-.219Z"
clip-rule="evenodd"
/>
</svg>
</button>
</VideoInputMenu>
{:else}
<Tooltip content={$i18n.t('Camera')}>
<button
class=" p-3 rounded-full bg-gray-50 dark:bg-gray-900"
type="button"
on:click={async () => {
await navigator.mediaDevices.getUserMedia({ video: true });
startCamera();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="size-5"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M6.827 6.175A2.31 2.31 0 0 1 5.186 7.23c-.38.054-.757.112-1.134.175C2.999 7.58 2.25 8.507 2.25 9.574V18a2.25 2.25 0 0 0 2.25 2.25h15A2.25 2.25 0 0 0 21.75 18V9.574c0-1.067-.75-1.994-1.802-2.169a47.865 47.865 0 0 0-1.134-.175 2.31 2.31 0 0 1-1.64-1.055l-.822-1.316a2.192 2.192 0 0 0-1.736-1.039 48.774 48.774 0 0 0-5.232 0 2.192 2.192 0 0 0-1.736 1.039l-.821 1.316Z"
/>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M16.5 12.75a4.5 4.5 0 1 1-9 0 4.5 4.5 0 0 1 9 0ZM18.75 10.5h.008v.008h-.008V10.5Z"
/>
</svg>
</button>
</Tooltip>
{/if}
</div>
<div>
<button type="button">
<div class=" line-clamp-1 text-sm font-medium">
{#if loading}
{$i18n.t('Thinking...')}
{:else}
{$i18n.t('Listening...')}
{/if}
</div>
</button>
</div>
<div>
<button
class=" p-3 rounded-full bg-gray-50 dark:bg-gray-900"
on:click={async () => {
showCallOverlay.set(false);
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="size-5"
>
<path
d="M6.28 5.22a.75.75 0 0 0-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 1 0 1.06 1.06L10 11.06l3.72 3.72a.75.75 0 1 0 1.06-1.06L11.06 10l3.72-3.72a.75.75 0 0 0-1.06-1.06L10 8.94 6.28 5.22Z"
/>
</svg>
</button>
</div>
</div>
</div>
</div>
</div>
{/if}

51
src/lib/components/chat/MessageInput/CallOverlay/VideoInputMenu.svelte Normal file
View File

@ -0,0 +1,51 @@
<script lang="ts">
import { DropdownMenu } from 'bits-ui';
import { flyAndScale } from '$lib/utils/transitions';
import { getContext, createEventDispatcher } from 'svelte';
const i18n = getContext('i18n');
const dispatch = createEventDispatcher();
import Dropdown from '$lib/components/common/Dropdown.svelte';
export let onClose: Function = () => {};
export let devices: any;
let show = false;
</script>
<Dropdown
bind:show
on:change={(e) => {
if (e.detail === false) {
onClose();
}
}}
>
<slot />
<div slot="content">
<DropdownMenu.Content
class="w-full max-w-[180px] rounded-lg px-1 py-1.5 border border-gray-300/30 dark:border-gray-700/50 z-[9999] bg-white dark:bg-gray-900 dark:text-white shadow-sm"
sideOffset={6}
side="top"
align="start"
transition={flyAndScale}
>
{#each devices as device}
<DropdownMenu.Item
class="flex gap-2 items-center px-3 py-2 text-sm cursor-pointer hover:bg-gray-50 dark:hover:bg-gray-800 rounded-md"
on:click={() => {
dispatch('change', device.deviceId);
}}
>
<div class="flex items-center">
<div class=" line-clamp-1">
{device?.label ?? 'Camera'}
</div>
</div>
</DropdownMenu.Item>
{/each}
</DropdownMenu.Content>
</div>
</Dropdown>

2
src/lib/components/chat/MessageInput/Documents.svelte
View File

@ -101,7 +101,7 @@
</script>
{#if filteredItems.length > 0 || prompt.split(' ')?.at(0)?.substring(1).startsWith('http')}
<div class="md:px-2 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
<div class="pl-1 pr-12 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
<div class="flex w-full px-2">
<div class=" bg-gray-100 dark:bg-gray-700 w-10 rounded-l-xl text-center">
<div class=" text-lg font-semibold mt-2">#</div>

53
src/lib/components/chat/MessageInput/InputMenu.svelte
View File

@ -4,24 +4,33 @@
import { getContext } from 'svelte';
import Dropdown from '$lib/components/common/Dropdown.svelte';
import GarbageBin from '$lib/components/icons/GarbageBin.svelte';
import Pencil from '$lib/components/icons/Pencil.svelte';
import Tooltip from '$lib/components/common/Tooltip.svelte';
import Tags from '$lib/components/chat/Tags.svelte';
import Share from '$lib/components/icons/Share.svelte';
import ArchiveBox from '$lib/components/icons/ArchiveBox.svelte';
import DocumentArrowUpSolid from '$lib/components/icons/DocumentArrowUpSolid.svelte';
import Switch from '$lib/components/common/Switch.svelte';
import GlobeAltSolid from '$lib/components/icons/GlobeAltSolid.svelte';
import { config } from '$lib/stores';
import WrenchSolid from '$lib/components/icons/WrenchSolid.svelte';
const i18n = getContext('i18n');
export let uploadFilesHandler: Function;
export let selectedToolIds: string[] = [];
export let webSearchEnabled: boolean;
export let tools = {};
export let onClose: Function;
$: tools = Object.fromEntries(
Object.keys(tools).map((toolId) => [
toolId,
{
...tools[toolId],
enabled: selectedToolIds.includes(toolId)
}
])
);
let show = false;
</script>
@ -39,20 +48,48 @@
<div slot="content">
<DropdownMenu.Content
class="w-full max-w-[190px] rounded-xl px-1 py-1 border-gray-300/30 dark:border-gray-700/50 z-50 bg-white dark:bg-gray-850 dark:text-white shadow"
class="w-full max-w-[200px] rounded-xl px-1 py-1 border-gray-300/30 dark:border-gray-700/50 z-50 bg-white dark:bg-gray-850 dark:text-white shadow"
sideOffset={15}
alignOffset={-8}
side="top"
align="start"
transition={flyAndScale}
>
{#if Object.keys(tools).length > 0}
<div class=" max-h-28 overflow-y-auto scrollbar-hidden">
{#each Object.keys(tools) as toolId}
<div
class="flex gap-2 items-center px-3 py-2 text-sm font-medium cursor-pointer rounded-xl"
>
<div class="flex-1 flex items-center gap-2">
<WrenchSolid />
<Tooltip content={tools[toolId]?.description ?? ''} className="flex-1">
<div class=" line-clamp-1">{tools[toolId].name}</div>
</Tooltip>
</div>
<Switch
bind:state={tools[toolId].enabled}
on:change={(e) => {
selectedToolIds = e.detail
? [...selectedToolIds, toolId]
: selectedToolIds.filter((id) => id !== toolId);
}}
/>
</div>
{/each}
</div>
<hr class="border-gray-100 dark:border-gray-800 my-1" />
{/if}
{#if $config?.features?.enable_web_search}
<div
class="flex gap-2 items-center px-3 py-2 text-sm font-medium cursor-pointer rounded-xl"
>
<div class="flex-1 flex items-center gap-2">
<GlobeAltSolid />
<div class="flex items-center">{$i18n.t('Web Search')}</div>
<div class=" line-clamp-1">{$i18n.t('Web Search')}</div>
</div>
<Switch bind:state={webSearchEnabled} />
@ -68,7 +105,7 @@
}}
>
<DocumentArrowUpSolid />
<div class="flex items-center">{$i18n.t('Upload Files')}</div>
<div class=" line-clamp-1">{$i18n.t('Upload Files')}</div>
</DropdownMenu.Item>
</DropdownMenu.Content>
</div>

2
src/lib/components/chat/MessageInput/Models.svelte
View File

@ -133,7 +133,7 @@
{#if prompt.charAt(0) === '@'}
{#if filteredModels.length > 0}
<div class="md:px-2 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
<div class="pl-1 pr-12 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
<div class="flex w-full px-2">
<div class=" bg-gray-100 dark:bg-gray-700 w-10 rounded-l-xl text-center">
<div class=" text-lg font-semibold mt-2">@</div>

29
src/lib/components/chat/MessageInput/PromptCommands.svelte
View File

@ -6,6 +6,7 @@
const i18n = getContext('i18n');
export let files;
export let prompt = '';
let selectedCommandIdx = 0;
let filteredPromptCommands = [];
@ -35,6 +36,32 @@
return '{{CLIPBOARD}}';
});
console.log(clipboardText);
const clipboardItems = await navigator.clipboard.read();
let imageUrl = null;
for (const item of clipboardItems) {
// Check for known image types
for (const type of item.types) {
if (type.startsWith('image/')) {
const blob = await item.getType(type);
imageUrl = URL.createObjectURL(blob);
console.log(`Image URL (${type}): ${imageUrl}`);
}
}
}
if (imageUrl) {
files = [
...files,
{
type: 'image',
url: imageUrl
}
];
}
text = command.content.replaceAll('{{CLIPBOARD}}', clipboardText);
}
@ -61,7 +88,7 @@
</script>
{#if filteredPromptCommands.length > 0}
<div class="md:px-2 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
<div class="pl-1 pr-12 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
<div class="flex w-full px-2">
<div class=" bg-gray-100 dark:bg-gray-700 w-10 rounded-l-xl text-center">
<div class=" text-lg font-semibold mt-2">/</div>

458
src/lib/components/chat/MessageInput/VoiceRecording.svelte Normal file
View File

@ -0,0 +1,458 @@
<script lang="ts">
import { toast } from 'svelte-sonner';
import { createEventDispatcher, tick, getContext } from 'svelte';
import { config, settings } from '$lib/stores';
import { blobToFile, calculateSHA256, findWordIndices } from '$lib/utils';
import { transcribeAudio } from '$lib/apis/audio';
const i18n = getContext('i18n');
const dispatch = createEventDispatcher();
export let recording = false;
let loading = false;
let confirmed = false;
let durationSeconds = 0;
let durationCounter = null;
let transcription = '';
const startDurationCounter = () => {
durationCounter = setInterval(() => {
durationSeconds++;
}, 1000);
};
const stopDurationCounter = () => {
clearInterval(durationCounter);
durationSeconds = 0;
};
$: if (recording) {
startRecording();
} else {
stopRecording();
}
const formatSeconds = (seconds) => {
const minutes = Math.floor(seconds / 60);
const remainingSeconds = seconds % 60;
const formattedSeconds = remainingSeconds < 10 ? `0${remainingSeconds}` : remainingSeconds;
return `${minutes}:${formattedSeconds}`;
};
let speechRecognition;
let mediaRecorder;
let audioChunks = [];
const MIN_DECIBELS = -45;
const VISUALIZER_BUFFER_LENGTH = 300;
let visualizerData = Array(VISUALIZER_BUFFER_LENGTH).fill(0);
// Function to calculate the RMS level from time domain data
const calculateRMS = (data: Uint8Array) => {
let sumSquares = 0;
for (let i = 0; i < data.length; i++) {
const normalizedValue = (data[i] - 128) / 128; // Normalize the data
sumSquares += normalizedValue * normalizedValue;
}
return Math.sqrt(sumSquares / data.length);
};
const normalizeRMS = (rms) => {
rms = rms * 10;
const exp = 1.5; // Adjust exponent value; values greater than 1 expand larger numbers more and compress smaller numbers more
const scaledRMS = Math.pow(rms, exp);
// Scale between 0.01 (1%) and 1.0 (100%)
return Math.min(1.0, Math.max(0.01, scaledRMS));
};
const analyseAudio = (stream) => {
const audioContext = new AudioContext();
const audioStreamSource = audioContext.createMediaStreamSource(stream);
const analyser = audioContext.createAnalyser();
analyser.minDecibels = MIN_DECIBELS;
audioStreamSource.connect(analyser);
const bufferLength = analyser.frequencyBinCount;
const domainData = new Uint8Array(bufferLength);
const timeDomainData = new Uint8Array(analyser.fftSize);
let lastSoundTime = Date.now();
const detectSound = () => {
const processFrame = () => {
if (!recording || loading) return;
if (recording && !loading) {
analyser.getByteTimeDomainData(timeDomainData);
analyser.getByteFrequencyData(domainData);
// Calculate RMS level from time domain data
const rmsLevel = calculateRMS(timeDomainData);
// Push the calculated decibel level to visualizerData
visualizerData.push(normalizeRMS(rmsLevel));
// Ensure visualizerData array stays within the buffer length
if (visualizerData.length >= VISUALIZER_BUFFER_LENGTH) {
visualizerData.shift();
}
visualizerData = visualizerData;
// if (domainData.some((value) => value > 0)) {
// lastSoundTime = Date.now();
// }
// if (recording && Date.now() - lastSoundTime > 3000) {
// if ($settings?.speechAutoSend ?? false) {
// confirmRecording();
// }
// }
}
window.requestAnimationFrame(processFrame);
};
window.requestAnimationFrame(processFrame);
};
detectSound();
};
const transcribeHandler = async (audioBlob) => {
// Create a blob from the audio chunks
await tick();
const file = blobToFile(audioBlob, 'recording.wav');
const res = await transcribeAudio(localStorage.token, file).catch((error) => {
toast.error(error);
return null;
});
if (res) {
console.log(res.text);
dispatch('confirm', res.text);
}
};
const saveRecording = (blob) => {
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
document.body.appendChild(a);
a.style = 'display: none';
a.href = url;
a.download = 'recording.wav';
a.click();
window.URL.revokeObjectURL(url);
};
const startRecording = async () => {
startDurationCounter();
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
mediaRecorder = new MediaRecorder(stream);
mediaRecorder.onstart = () => {
console.log('Recording started');
audioChunks = [];
analyseAudio(stream);
};
mediaRecorder.ondataavailable = (event) => audioChunks.push(event.data);
mediaRecorder.onstop = async () => {
console.log('Recording stopped');
if (($settings?.audio?.stt?.engine ?? '') === 'web') {
audioChunks = [];
} else {
if (confirmed) {
const audioBlob = new Blob(audioChunks, { type: 'audio/wav' });
await transcribeHandler(audioBlob);
confirmed = false;
loading = false;
}
audioChunks = [];
recording = false;
}
};
mediaRecorder.start();
if ($config.audio.stt.engine === 'web' || ($settings?.audio?.stt?.engine ?? '') === 'web') {
if ('SpeechRecognition' in window || 'webkitSpeechRecognition' in window) {
// Create a SpeechRecognition object
speechRecognition = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
// Set continuous to true for continuous recognition
speechRecognition.continuous = true;
// Set the timeout for turning off the recognition after inactivity (in milliseconds)
const inactivityTimeout = 2000; // 3 seconds
let timeoutId;
// Start recognition
speechRecognition.start();
// Event triggered when speech is recognized
speechRecognition.onresult = async (event) => {
// Clear the inactivity timeout
clearTimeout(timeoutId);
// Handle recognized speech
console.log(event);
const transcript = event.results[Object.keys(event.results).length - 1][0].transcript;
transcription = `${transcription}${transcript}`;
await tick();
document.getElementById('chat-textarea')?.focus();
// Restart the inactivity timeout
timeoutId = setTimeout(() => {
console.log('Speech recognition turned off due to inactivity.');
speechRecognition.stop();
}, inactivityTimeout);
};
// Event triggered when recognition is ended
speechRecognition.onend = function () {
// Restart recognition after it ends
console.log('recognition ended');
confirmRecording();
dispatch('confirm', transcription);
confirmed = false;
loading = false;
};
// Event triggered when an error occurs
speechRecognition.onerror = function (event) {
console.log(event);
toast.error($i18n.t(`Speech recognition error: {{error}}`, { error: event.error }));
dispatch('cancel');
stopRecording();
};
}
}
};
const stopRecording = async () => {
if (recording && mediaRecorder) {
await mediaRecorder.stop();
}
stopDurationCounter();
audioChunks = [];
};
const confirmRecording = async () => {
loading = true;
confirmed = true;
if (recording && mediaRecorder) {
await mediaRecorder.stop();
}
clearInterval(durationCounter);
};
</script>
<div
class="{loading
? ' bg-gray-100/50 dark:bg-gray-850/50'
: 'bg-indigo-300/10 dark:bg-indigo-500/10 '} rounded-full flex p-2.5"
>
<div class="flex items-center mr-1">
<button
type="button"
class="p-1.5
{loading
? ' bg-gray-200 dark:bg-gray-700/50'
: 'bg-indigo-400/20 text-indigo-600 dark:text-indigo-300 '}
rounded-full"
on:click={async () => {
dispatch('cancel');
stopRecording();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="3"
stroke="currentColor"
class="size-4"
>
<path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12" />
</svg>
</button>
</div>
<div
class="flex flex-1 self-center items-center justify-between ml-2 mx-1 overflow-hidden h-6"
dir="rtl"
>
<div class="flex-1 flex items-center gap-0.5 h-6">
{#each visualizerData.slice().reverse() as rms}
<div
class="w-[2px]
{loading
? ' bg-gray-500 dark:bg-gray-400 '
: 'bg-indigo-500 dark:bg-indigo-400 '}
inline-block h-full"
style="height: {Math.min(100, Math.max(14, rms * 100))}%;"
/>
{/each}
</div>
</div>
<div class=" mx-1.5 pr-1 flex justify-center items-center">
<div
class="text-sm
{loading ? ' text-gray-500 dark:text-gray-400 ' : ' text-indigo-400 '}
font-medium flex-1 mx-auto text-center"
>
{formatSeconds(durationSeconds)}
</div>
</div>
<div class="flex items-center mr-1">
{#if loading}
<div class=" text-gray-500 rounded-full cursor-not-allowed">
<svg
width="24"
height="24"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
fill="currentColor"
><style>
.spinner_OSmW {
transform-origin: center;
animation: spinner_T6mA 0.75s step-end infinite;
}
@keyframes spinner_T6mA {
8.3% {
transform: rotate(30deg);
}
16.6% {
transform: rotate(60deg);
}
25% {
transform: rotate(90deg);
}
33.3% {
transform: rotate(120deg);
}
41.6% {
transform: rotate(150deg);
}
50% {
transform: rotate(180deg);
}
58.3% {
transform: rotate(210deg);
}
66.6% {
transform: rotate(240deg);
}
75% {
transform: rotate(270deg);
}
83.3% {
transform: rotate(300deg);
}
91.6% {
transform: rotate(330deg);
}
100% {
transform: rotate(360deg);
}
}
</style><g class="spinner_OSmW"
><rect x="11" y="1" width="2" height="5" opacity=".14" /><rect
x="11"
y="1"
width="2"
height="5"
transform="rotate(30 12 12)"
opacity=".29"
/><rect
x="11"
y="1"
width="2"
height="5"
transform="rotate(60 12 12)"
opacity=".43"
/><rect
x="11"
y="1"
width="2"
height="5"
transform="rotate(90 12 12)"
opacity=".57"
/><rect
x="11"
y="1"
width="2"
height="5"
transform="rotate(120 12 12)"
opacity=".71"
/><rect
x="11"
y="1"
width="2"
height="5"
transform="rotate(150 12 12)"
opacity=".86"
/><rect x="11" y="1" width="2" height="5" transform="rotate(180 12 12)" /></g
></svg
>
</div>
{:else}
<button
type="button"
class="p-1.5 bg-indigo-500 text-white dark:bg-indigo-500 dark:text-blue-950 rounded-full"
on:click={async () => {
await confirmRecording();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="2.5"
stroke="currentColor"
class="size-4"
>
<path stroke-linecap="round" stroke-linejoin="round" d="m4.5 12.75 6 6 9-13.5" />
</svg>
</button>
{/if}
</div>
</div>
<style>
.visualizer {
display: flex;
height: 100%;
}
.visualizer-bar {
width: 2px;
background-color: #4a5aba; /* or whatever color you need */
}
</style>

12
src/lib/components/chat/Messages.svelte
View File

@ -1,8 +1,7 @@
<script lang="ts">
import { v4 as uuidv4 } from 'uuid';
import { chats, config, settings, user as _user, mobile } from '$lib/stores';
import { tick, getContext } from 'svelte';
import { tick, getContext, onMount } from 'svelte';
import { toast } from 'svelte-sonner';
import { getChatList, updateChatById } from '$lib/apis/chats';
@ -80,7 +79,7 @@
history.currentId = userMessageId;
await tick();
await sendPrompt(userPrompt, userMessageId);
await sendPrompt(userPrompt, userMessageId, undefined, false);
};
const updateChatMessages = async () => {
@ -242,7 +241,7 @@
};
</script>
<div class="h-full flex mb-16">
<div class="h-full flex">
{#if messages.length == 0}
<Placeholder
modelIds={selectedModels}
@ -285,9 +284,9 @@
<div class="w-full pt-2">
{#key chatId}
{#each messages as message, messageIdx}
<div class=" w-full {messageIdx === messages.length - 1 ? 'pb-28' : ''}">
<div class=" w-full {messageIdx === messages.length - 1 ? ' pb-12' : ''}">
<div
class="flex flex-col justify-between px-5 mb-3 {$settings?.fullScreenMode ?? null
class="flex flex-col justify-between px-5 mb-3 {$settings?.widescreenMode ?? null
? 'max-w-full'
: 'max-w-5xl'} mx-auto rounded-lg group"
>
@ -340,6 +339,7 @@
<CompareMessages
bind:history
{messages}
{readOnly}
{chatId}
parentMessage={history.messages[message.parentId]}
{messageIdx}

5
src/lib/components/chat/Messages/CompareMessages.svelte
View File

@ -13,6 +13,8 @@
export let parentMessage;
export let readOnly = false;
export let updateChatMessages: Function;
export let confirmEditResponseMessage: Function;
export let rateMessage: Function;
@ -107,7 +109,7 @@
class=" snap-center min-w-80 w-full max-w-full m-1 border {history.messages[
currentMessageId
].model === model
? 'border-gray-100 dark:border-gray-700 border-[1.5px]'
? 'border-gray-100 dark:border-gray-850 border-[1.5px]'
: 'border-gray-50 dark:border-gray-850 '} transition p-5 rounded-3xl"
on:click={() => {
currentMessageId = groupedMessages[model].messages[groupedMessagesIdx[model]].id;
@ -134,6 +136,7 @@
{confirmEditResponseMessage}
showPreviousMessage={() => showPreviousMessage(model)}
showNextMessage={() => showNextMessage(model)}
{readOnly}
{rateMessage}
{copyToClipboard}
{continueGeneration}

6
src/lib/components/chat/Messages/Placeholder.svelte
View File

@ -29,7 +29,7 @@
</script>
{#key mounted}
<div class="m-auto w-full max-w-6xl px-8 lg:px-24 pb-16">
<div class="m-auto w-full max-w-6xl px-8 lg:px-24 pb-10">
<div class="flex justify-start">
<div class="flex -space-x-4 mb-1" in:fade={{ duration: 200 }}>
{#each models as model, modelIdx}
@ -64,7 +64,7 @@
</div>
<div in:fade={{ duration: 200, delay: 200 }}>
{#if models[selectedModelIdx]?.info}
{#if models[selectedModelIdx]?.info?.meta?.description ?? null}
<div class="mt-0.5 text-base font-normal text-gray-500 dark:text-gray-400 line-clamp-3">
{models[selectedModelIdx]?.info?.meta?.description}
</div>
@ -85,7 +85,7 @@
</div>
{/if}
{:else}
<div class=" font-medium text-gray-400 dark:text-gray-500">
<div class=" font-medium text-gray-400 dark:text-gray-500 line-clamp-1">
{$i18n.t('How can I help you today?')}
</div>
{/if}

4
src/lib/components/chat/Messages/RateComment.svelte
View File

@ -46,8 +46,8 @@
}
onMount(() => {
selectedReason = message.annotation.reason;
comment = message.annotation.comment;
selectedReason = message?.annotation?.reason ?? '';
comment = message?.annotation?.comment ?? '';
loadReasons();
});

230
src/lib/components/chat/Messages/ResponseMessage.svelte
View File

@ -5,6 +5,7 @@
import tippy from 'tippy.js';
import auto_render from 'katex/dist/contrib/auto-render.mjs';
import 'katex/dist/katex.min.css';
import mermaid from 'mermaid';
import { fade } from 'svelte/transition';
import { createEventDispatcher } from 'svelte';
@ -210,82 +211,98 @@
speaking = null;
speakingIdx = null;
} else {
speaking = true;
if ((message?.content ?? '').trim() !== '') {
speaking = true;
if ($settings?.audio?.TTSEngine === 'openai') {
loadingSpeech = true;
if ($config.audio.tts.engine === 'openai') {
loadingSpeech = true;
const sentences = extractSentences(message.content).reduce((mergedTexts, currentText) => {
const lastIndex = mergedTexts.length - 1;
if (lastIndex >= 0) {
const previousText = mergedTexts[lastIndex];
const wordCount = previousText.split(/\s+/).length;
if (wordCount < 2) {
mergedTexts[lastIndex] = previousText + ' ' + currentText;
const sentences = extractSentences(message.content).reduce((mergedTexts, currentText) => {
const lastIndex = mergedTexts.length - 1;
if (lastIndex >= 0) {
const previousText = mergedTexts[lastIndex];
const wordCount = previousText.split(/\s+/).length;
if (wordCount < 2) {
mergedTexts[lastIndex] = previousText + ' ' + currentText;
} else {
mergedTexts.push(currentText);
}
} else {
mergedTexts.push(currentText);
}
} else {
mergedTexts.push(currentText);
return mergedTexts;
}, []);
console.log(sentences);
sentencesAudio = sentences.reduce((a, e, i, arr) => {
a[i] = null;
return a;
}, {});
let lastPlayedAudioPromise = Promise.resolve(); // Initialize a promise that resolves immediately
for (const [idx, sentence] of sentences.entries()) {
const res = await synthesizeOpenAISpeech(
localStorage.token,
$settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice,
sentence
).catch((error) => {
toast.error(error);
speaking = null;
loadingSpeech = false;
return null;
});
if (res) {
const blob = await res.blob();
const blobUrl = URL.createObjectURL(blob);
const audio = new Audio(blobUrl);
sentencesAudio[idx] = audio;
loadingSpeech = false;
lastPlayedAudioPromise = lastPlayedAudioPromise.then(() => playAudio(idx));
}
}
return mergedTexts;
}, []);
} else {
let voices = [];
const getVoicesLoop = setInterval(async () => {
voices = await speechSynthesis.getVoices();
if (voices.length > 0) {
clearInterval(getVoicesLoop);
console.log(sentences);
const voice =
voices
?.filter(
(v) =>
v.voiceURI === ($settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice)
)
?.at(0) ?? undefined;
sentencesAudio = sentences.reduce((a, e, i, arr) => {
a[i] = null;
return a;
}, {});
console.log(voice);
let lastPlayedAudioPromise = Promise.resolve(); // Initialize a promise that resolves immediately
const speak = new SpeechSynthesisUtterance(message.content);
for (const [idx, sentence] of sentences.entries()) {
const res = await synthesizeOpenAISpeech(
localStorage.token,
$settings?.audio?.speaker,
sentence,
$settings?.audio?.model
).catch((error) => {
toast.error(error);
console.log(speak);
speaking = null;
loadingSpeech = false;
speak.onend = () => {
speaking = null;
if ($settings.conversationMode) {
document.getElementById('voice-input-button')?.click();
}
};
return null;
});
if (voice) {
speak.voice = voice;
}
if (res) {
const blob = await res.blob();
const blobUrl = URL.createObjectURL(blob);
const audio = new Audio(blobUrl);
sentencesAudio[idx] = audio;
loadingSpeech = false;
lastPlayedAudioPromise = lastPlayedAudioPromise.then(() => playAudio(idx));
}
speechSynthesis.speak(speak);
}
}, 100);
}
} else {
let voices = [];
const getVoicesLoop = setInterval(async () => {
voices = await speechSynthesis.getVoices();
if (voices.length > 0) {
clearInterval(getVoicesLoop);
const voice =
voices?.filter((v) => v.name === $settings?.audio?.speaker)?.at(0) ?? undefined;
const speak = new SpeechSynthesisUtterance(message.content);
speak.onend = () => {
speaking = null;
if ($settings.conversationMode) {
document.getElementById('voice-input-button')?.click();
}
};
speak.voice = voice;
speechSynthesis.speak(speak);
}
}, 100);
toast.error('No content to speak');
}
}
};
@ -340,9 +357,24 @@
generatingImage = false;
};
$: if (!edit) {
(async () => {
await tick();
renderStyling();
await mermaid.run({
querySelector: '.mermaid'
});
})();
}
onMount(async () => {
await tick();
renderStyling();
await mermaid.run({
querySelector: '.mermaid'
});
});
</script>
@ -388,26 +420,29 @@
class="prose chat-{message.role} w-full max-w-full dark:prose-invert prose-headings:my-0 prose-headings:-mb-4 prose-p:m-0 prose-p:-mb-6 prose-pre:my-0 prose-table:my-0 prose-blockquote:my-0 prose-img:my-0 prose-ul:-my-4 prose-ol:-my-4 prose-li:-my-3 prose-ul:-mb-6 prose-ol:-mb-8 prose-ol:p-0 prose-li:-mb-4 whitespace-pre-line"
>
<div>
{#if message?.status}
{#if (message?.statusHistory ?? [...(message?.status ? [message?.status] : [])]).length > 0}
{@const status = (
message?.statusHistory ?? [...(message?.status ? [message?.status] : [])]
).at(-1)}
<div class="flex items-center gap-2 pt-1 pb-1">
{#if message?.status?.done === false}
{#if status.done === false}
<div class="">
<Spinner className="size-4" />
</div>
{/if}
{#if message?.status?.action === 'web_search' && message?.status?.urls}
<WebSearchResults urls={message?.status?.urls}>
{#if status?.action === 'web_search' && status?.urls}
<WebSearchResults {status}>
<div class="flex flex-col justify-center -space-y-0.5">
<div class="text-base line-clamp-1 text-wrap">
{message.status.description}
{status?.description}
</div>
</div>
</WebSearchResults>
{:else}
<div class="flex flex-col justify-center -space-y-0.5">
<div class=" text-gray-500 dark:text-gray-500 text-base line-clamp-1 text-wrap">
{message.status.description}
{status?.description}
</div>
</div>
{/if}
@ -451,7 +486,34 @@
</div>
{:else}
<div class="w-full">
{#if message?.error === true}
{#if message.content === '' && !message.error}
<Skeleton />
{:else if message.content && message.error !== true}
<!-- always show message contents even if there's an error -->
<!-- unless message.error === true which is legacy error handling, where the error message is stored in message.content -->
{#each tokens as token, tokenIdx}
{#if token.type === 'code'}
{#if token.lang === 'mermaid'}
<pre class="mermaid">{revertSanitizedResponseContent(token.text)}</pre>
{:else}
<CodeBlock
id={`${message.id}-${tokenIdx}`}
lang={token?.lang ?? ''}
code={revertSanitizedResponseContent(token?.text ?? '')}
/>
{/if}
{:else}
{@html marked.parse(token.raw, {
...defaults,
gfm: true,
breaks: true,
renderer
})}
{/if}
{/each}
{/if}
{#if message.error}
<div
class="flex mt-2 mb-4 space-x-2 border px-4 py-3 border-red-800 bg-red-800/30 font-medium rounded-lg"
>
@ -471,28 +533,9 @@
</svg>
<div class=" self-center">
{message.content}
{message?.error?.content ?? message.content}
</div>
</div>
{:else if message.content === ''}
<Skeleton />
{:else}
{#each tokens as token, tokenIdx}
{#if token.type === 'code'}
<CodeBlock
id={`${message.id}-${tokenIdx}`}
lang={token?.lang ?? ''}
code={revertSanitizedResponseContent(token?.text ?? '')}
/>
{:else}
{@html marked.parse(token.raw, {
...defaults,
gfm: true,
breaks: true,
renderer
})}
{/if}
{/each}
{/if}
{#if message.citations}
@ -733,7 +776,7 @@
</Tooltip>
{#if $config?.features.enable_image_generation && !readOnly}
<Tooltip content="Generate Image" placement="bottom">
<Tooltip content={$i18n.t('Generate Image')} placement="bottom">
<button
class="{isLastMessage
? 'visible'
@ -833,8 +876,8 @@
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {message
?.annotation?.rating === 1
: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {(message
?.annotation?.rating ?? null) === 1
? 'bg-gray-100 dark:bg-gray-800'
: ''} dark:hover:text-white hover:text-black transition"
on:click={() => {
@ -868,8 +911,8 @@
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {message
?.annotation?.rating === -1
: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {(message
?.annotation?.rating ?? null) === -1
? 'bg-gray-100 dark:bg-gray-800'
: ''} dark:hover:text-white hover:text-black transition"
on:click={() => {
@ -939,6 +982,7 @@
? 'visible'
: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg dark:hover:text-white hover:text-black transition regenerate-response-button"
on:click={() => {
showRateComment = false;
regenerateResponse(message);
}}
>

45
src/lib/components/chat/Messages/ResponseMessage/WebSearchResults.svelte
View File

@ -1,10 +1,11 @@
<script lang="ts">
import ChevronDown from '$lib/components/icons/ChevronDown.svelte';
import ChevronUp from '$lib/components/icons/ChevronUp.svelte';
import MagnifyingGlass from '$lib/components/icons/MagnifyingGlass.svelte';
import { Collapsible } from 'bits-ui';
import { slide } from 'svelte/transition';
export let urls = [];
export let status = { urls: [], query: '' };
let state = false;
</script>
@ -27,15 +28,51 @@
class=" text-sm border border-gray-300/30 dark:border-gray-700/50 rounded-xl"
transition={slide}
>
{#each urls as url, urlIdx}
{#if status?.query}
<a
href="https://www.google.com/search?q={status.query}"
target="_blank"
class="flex w-full items-center p-3 px-4 border-b border-gray-300/30 dark:border-gray-700/50 group/item justify-between font-normal text-gray-800 dark:text-gray-300 no-underline"
>
<div class="flex gap-2 items-center">
<MagnifyingGlass />
<div class=" line-clamp-1">
{status.query}
</div>
</div>
<div
class=" ml-1 text-white dark:text-gray-900 group-hover/item:text-gray-600 dark:group-hover/item:text-white transition"
>
<!-- -->
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="size-4"
>
<path
fill-rule="evenodd"
d="M4.22 11.78a.75.75 0 0 1 0-1.06L9.44 5.5H5.75a.75.75 0 0 1 0-1.5h5.5a.75.75 0 0 1 .75.75v5.5a.75.75 0 0 1-1.5 0V6.56l-5.22 5.22a.75.75 0 0 1-1.06 0Z"
clip-rule="evenodd"
/>
</svg>
</div>
</a>
{/if}
{#each status.urls as url, urlIdx}
<a
href={url}
target="_blank"
class="flex w-full items-center p-3 px-4 {urlIdx === urls.length - 1
class="flex w-full items-center p-3 px-4 {urlIdx === status.urls.length - 1
? ''
: 'border-b border-gray-300/30 dark:border-gray-700/50'} group/item justify-between font-normal text-gray-800 dark:text-gray-300"
>
{url}
<div class=" line-clamp-1">
{url}
</div>
<div
class=" ml-1 text-white dark:text-gray-900 group-hover/item:text-gray-600 dark:group-hover/item:text-white transition"

4
src/lib/components/chat/Messages/UserMessage.svelte
View File

@ -196,7 +196,7 @@
<div class=" mt-2 mb-1 flex justify-end space-x-1.5 text-sm font-medium">
<button
id="close-edit-message-button"
class="px-4 py-2 bg-white hover:bg-gray-100 text-gray-800 transition rounded-3xl"
class="px-4 py-2 bg-white dark:bg-gray-900 hover:bg-gray-100 text-gray-800 dark:text-gray-100 transition rounded-3xl"
on:click={() => {
cancelEditMessage();
}}
@ -206,7 +206,7 @@
<button
id="save-edit-message-button"
class=" px-4 py-2 bg-gray-900 hover:bg-gray-850 text-gray-100 transition rounded-3xl"
class=" px-4 py-2 bg-gray-900 dark:bg-white hover:bg-gray-850 text-gray-100 dark:text-gray-800 transition rounded-3xl"
on:click={() => {
editMessageConfirmHandler();
}}

230
src/lib/components/chat/ModelSelector/Selector.svelte
View File

@ -8,7 +8,7 @@
import Check from '$lib/components/icons/Check.svelte';
import Search from '$lib/components/icons/Search.svelte';
import { cancelOllamaRequest, deleteModel, getOllamaVersion, pullModel } from '$lib/apis/ollama';
import { deleteModel, getOllamaVersion, pullModel } from '$lib/apis/ollama';
import { user, MODEL_DOWNLOAD_POOL, models, mobile } from '$lib/stores';
import { toast } from 'svelte-sonner';
@ -42,9 +42,16 @@
let searchValue = '';
let ollamaVersion = null;
$: filteredItems = searchValue
? items.filter((item) => item.value.toLowerCase().includes(searchValue.toLowerCase()))
: items;
$: filteredItems = items.filter(
(item) =>
(searchValue
? item.value.toLowerCase().includes(searchValue.toLowerCase()) ||
item.label.toLowerCase().includes(searchValue.toLowerCase()) ||
(item.model?.info?.meta?.tags ?? []).some((tag) =>
tag.name.toLowerCase().includes(searchValue.toLowerCase())
)
: true) && !(item.model?.info?.meta?.hidden ?? false)
);
const pullModelHandler = async () => {
const sanitizedModelTag = searchValue.trim().replace(/^ollama\s+(run|pull)\s+/, '');
@ -65,10 +72,12 @@
return;
}
const res = await pullModel(localStorage.token, sanitizedModelTag, '0').catch((error) => {
toast.error(error);
return null;
});
const [res, controller] = await pullModel(localStorage.token, sanitizedModelTag, '0').catch(
(error) => {
toast.error(error);
return null;
}
);
if (res) {
const reader = res.body
@ -76,6 +85,16 @@
.pipeThrough(splitStream('\n'))
.getReader();
MODEL_DOWNLOAD_POOL.set({
...$MODEL_DOWNLOAD_POOL,
[sanitizedModelTag]: {
...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
abortController: controller,
reader,
done: false
}
});
while (true) {
try {
const { value, done } = await reader.read();
@ -94,19 +113,6 @@
throw data.detail;
}
if (data.id) {
MODEL_DOWNLOAD_POOL.set({
...$MODEL_DOWNLOAD_POOL,
[sanitizedModelTag]: {
...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
requestId: data.id,
reader,
done: false
}
});
console.log(data);
}
if (data.status) {
if (data.digest) {
let downloadProgress = 0;
@ -146,6 +152,7 @@
toast.error(error);
// opts.callback({ success: false, error, modelName: opts.modelName });
break;
}
}
@ -174,11 +181,12 @@
});
const cancelModelPullHandler = async (model: string) => {
const { reader, requestId } = $MODEL_DOWNLOAD_POOL[model];
const { reader, abortController } = $MODEL_DOWNLOAD_POOL[model];
if (abortController) {
abortController.abort();
}
if (reader) {
await reader.cancel();
await cancelOllamaRequest(localStorage.token, requestId);
delete $MODEL_DOWNLOAD_POOL[model];
MODEL_DOWNLOAD_POOL.set({
...$MODEL_DOWNLOAD_POOL
@ -212,7 +220,7 @@
<DropdownMenu.Content
class=" z-40 {$mobile
? `w-full`
: `${className}`} max-w-[calc(100vw-1rem)] justify-start rounded-xl bg-white dark:bg-gray-850 dark:text-white shadow-lg border border-gray-300/30 dark:border-gray-700/50 outline-none "
: `${className}`} max-w-[calc(100vw-1rem)] justify-start rounded-xl bg-white dark:bg-gray-850 dark:text-white shadow-lg border border-gray-300/30 dark:border-gray-850/50 outline-none "
transition={flyAndScale}
side={$mobile ? 'bottom' : 'bottom-start'}
sideOffset={4}
@ -245,87 +253,113 @@
show = false;
}}
>
<div class="flex items-center gap-2">
<div class="flex items-center">
<div class="line-clamp-1">
{item.label}
</div>
{#if item.model.owned_by === 'ollama' && (item.model.ollama?.details?.parameter_size ?? '') !== ''}
<div class="flex ml-1 items-center">
<Tooltip
content={`${
item.model.ollama?.details?.quantization_level
? item.model.ollama?.details?.quantization_level + ' '
: ''
}${
item.model.ollama?.size
? `(${(item.model.ollama?.size / 1024 ** 3).toFixed(1)}GB)`
: ''
}`}
className="self-end"
<div class="flex flex-col">
{#if $mobile && (item?.model?.info?.meta?.tags ?? []).length > 0}
<div class="flex gap-0.5 self-start h-full mb-0.5 -translate-x-1">
{#each item.model?.info?.meta.tags as tag}
<div
class=" text-xs font-black px-1 rounded uppercase line-clamp-1 bg-gray-500/20 text-gray-700 dark:text-gray-200"
>
<span class=" text-xs font-medium text-gray-600 dark:text-gray-400"
>{item.model.ollama?.details?.parameter_size ?? ''}</span
{tag.name}
</div>
{/each}
</div>
{/if}
<div class="flex items-center gap-2">
<div class="flex items-center min-w-fit">
<div class="line-clamp-1">
{item.label}
</div>
{#if item.model.owned_by === 'ollama' && (item.model.ollama?.details?.parameter_size ?? '') !== ''}
<div class="flex ml-1 items-center translate-y-[0.5px]">
<Tooltip
content={`${
item.model.ollama?.details?.quantization_level
? item.model.ollama?.details?.quantization_level + ' '
: ''
}${
item.model.ollama?.size
? `(${(item.model.ollama?.size / 1024 ** 3).toFixed(1)}GB)`
: ''
}`}
className="self-end"
>
</Tooltip>
<span
class=" text-xs font-medium text-gray-600 dark:text-gray-400 line-clamp-1"
>{item.model.ollama?.details?.parameter_size ?? ''}</span
>
</Tooltip>
</div>
{/if}
</div>
{#if !$mobile && (item?.model?.info?.meta?.tags ?? []).length > 0}
<div class="flex gap-0.5 self-center items-center h-full translate-y-[0.5px]">
{#each item.model?.info?.meta.tags as tag}
<div
class=" text-xs font-black px-1 rounded uppercase line-clamp-1 bg-gray-500/20 text-gray-700 dark:text-gray-200"
>
{tag.name}
</div>
{/each}
</div>
{/if}
<!-- {JSON.stringify(item.info)} -->
{#if item.model.owned_by === 'openai'}
<Tooltip content={`${'External'}`}>
<div class="">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="size-3"
>
<path
fill-rule="evenodd"
d="M8.914 6.025a.75.75 0 0 1 1.06 0 3.5 3.5 0 0 1 0 4.95l-2 2a3.5 3.5 0 0 1-5.396-4.402.75.75 0 0 1 1.251.827 2 2 0 0 0 3.085 2.514l2-2a2 2 0 0 0 0-2.828.75.75 0 0 1 0-1.06Z"
clip-rule="evenodd"
/>
<path
fill-rule="evenodd"
d="M7.086 9.975a.75.75 0 0 1-1.06 0 3.5 3.5 0 0 1 0-4.95l2-2a3.5 3.5 0 0 1 5.396 4.402.75.75 0 0 1-1.251-.827 2 2 0 0 0-3.085-2.514l-2 2a2 2 0 0 0 0 2.828.75.75 0 0 1 0 1.06Z"
clip-rule="evenodd"
/>
</svg>
</div>
</Tooltip>
{/if}
{#if item.model?.info?.meta?.description}
<Tooltip
content={`${sanitizeResponseContent(
item.model?.info?.meta?.description
).replaceAll('\n', '<br>')}`}
>
<div class="">
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z"
/>
</svg>
</div>
</Tooltip>
{/if}
</div>
<!-- {JSON.stringify(item.info)} -->
{#if item.model.owned_by === 'openai'}
<Tooltip content={`${'External'}`}>
<div class="">
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="size-3"
>
<path
fill-rule="evenodd"
d="M8.914 6.025a.75.75 0 0 1 1.06 0 3.5 3.5 0 0 1 0 4.95l-2 2a3.5 3.5 0 0 1-5.396-4.402.75.75 0 0 1 1.251.827 2 2 0 0 0 3.085 2.514l2-2a2 2 0 0 0 0-2.828.75.75 0 0 1 0-1.06Z"
clip-rule="evenodd"
/>
<path
fill-rule="evenodd"
d="M7.086 9.975a.75.75 0 0 1-1.06 0 3.5 3.5 0 0 1 0-4.95l2-2a3.5 3.5 0 0 1 5.396 4.402.75.75 0 0 1-1.251-.827 2 2 0 0 0-3.085-2.514l-2 2a2 2 0 0 0 0 2.828.75.75 0 0 1 0 1.06Z"
clip-rule="evenodd"
/>
</svg>
</div>
</Tooltip>
{/if}
{#if item.model?.info?.meta?.description}
<Tooltip
content={`${sanitizeResponseContent(
item.model?.info?.meta?.description
).replaceAll('\n', '<br>')}`}
>
<div class="">
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z"
/>
</svg>
</div>
</Tooltip>
{/if}
</div>
{#if value === item.value}
<div class="ml-auto">
<div class="ml-auto pl-2">
<Check />
</div>
{/if}

4
src/lib/components/chat/Settings/About.svelte
View File

@ -92,7 +92,7 @@
</div>
{#if ollamaVersion}
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
<div>
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Ollama Version')}</div>
@ -104,7 +104,7 @@
</div>
{/if}
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
<div class="flex space-x-1">
<a href="https://discord.gg/5rJgQTnV4s" target="_blank">

2
src/lib/components/chat/Settings/Account.svelte
View File

@ -234,7 +234,7 @@
<UpdatePassword />
</div>
<hr class=" dark:border-gray-700 my-4" />
<hr class=" dark:border-gray-850 my-4" />
<div class="flex justify-between items-center text-sm">
<div class=" font-medium">{$i18n.t('API keys')}</div>

259
src/lib/components/chat/Settings/Advanced/AdvancedParams.svelte
View File

@ -5,21 +5,26 @@
const i18n = getContext('i18n');
export let admin = false;
export let params = {
// Advanced
seed: 0,
seed: null,
stop: null,
temperature: '',
frequency_penalty: '',
repeat_last_n: '',
mirostat: '',
mirostat_eta: '',
mirostat_tau: '',
top_k: '',
top_p: '',
tfs_z: '',
num_ctx: '',
max_tokens: '',
temperature: null,
frequency_penalty: null,
repeat_last_n: null,
mirostat: null,
mirostat_eta: null,
mirostat_tau: null,
top_k: null,
top_p: null,
tfs_z: null,
num_ctx: null,
max_tokens: null,
use_mmap: null,
use_mlock: null,
num_thread: null,
template: null
};
@ -109,10 +114,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.temperature = (params?.temperature ?? '') === '' ? 0.8 : '';
params.temperature = (params?.temperature ?? null) === null ? 0.8 : null;
}}
>
{#if (params?.temperature ?? '') === ''}
{#if (params?.temperature ?? null) === null}
<span class="ml-2 self-center"> {$i18n.t('Default')} </span>
{:else}
<span class="ml-2 self-center"> {$i18n.t('Custom')} </span>
@ -120,7 +125,7 @@
</button>
</div>
{#if (params?.temperature ?? '') !== ''}
{#if (params?.temperature ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -140,7 +145,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="1"
step="0.05"
step="any"
/>
</div>
</div>
@ -155,10 +160,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.mirostat = (params?.mirostat ?? '') === '' ? 0 : '';
params.mirostat = (params?.mirostat ?? null) === null ? 0 : null;
}}
>
{#if (params?.mirostat ?? '') === ''}
{#if (params?.mirostat ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -166,7 +171,7 @@
</button>
</div>
{#if (params?.mirostat ?? '') !== ''}
{#if (params?.mirostat ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -201,10 +206,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.mirostat_eta = (params?.mirostat_eta ?? '') === '' ? 0.1 : '';
params.mirostat_eta = (params?.mirostat_eta ?? null) === null ? 0.1 : null;
}}
>
{#if (params?.mirostat_eta ?? '') === ''}
{#if (params?.mirostat_eta ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -212,7 +217,7 @@
</button>
</div>
{#if (params?.mirostat_eta ?? '') !== ''}
{#if (params?.mirostat_eta ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -232,7 +237,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="1"
step="0.05"
step="any"
/>
</div>
</div>
@ -247,10 +252,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.mirostat_tau = (params?.mirostat_tau ?? '') === '' ? 5.0 : '';
params.mirostat_tau = (params?.mirostat_tau ?? null) === null ? 5.0 : null;
}}
>
{#if (params?.mirostat_tau ?? '') === ''}
{#if (params?.mirostat_tau ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -258,7 +263,7 @@
</button>
</div>
{#if (params?.mirostat_tau ?? '') !== ''}
{#if (params?.mirostat_tau ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -278,7 +283,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="10"
step="0.5"
step="any"
/>
</div>
</div>
@ -293,10 +298,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.top_k = (params?.top_k ?? '') === '' ? 40 : '';
params.top_k = (params?.top_k ?? null) === null ? 40 : null;
}}
>
{#if (params?.top_k ?? '') === ''}
{#if (params?.top_k ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -304,7 +309,7 @@
</button>
</div>
{#if (params?.top_k ?? '') !== ''}
{#if (params?.top_k ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -324,7 +329,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="100"
step="0.5"
step="any"
/>
</div>
</div>
@ -339,10 +344,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.top_p = (params?.top_p ?? '') === '' ? 0.9 : '';
params.top_p = (params?.top_p ?? null) === null ? 0.9 : null;
}}
>
{#if (params?.top_p ?? '') === ''}
{#if (params?.top_p ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -350,7 +355,7 @@
</button>
</div>
{#if (params?.top_p ?? '') !== ''}
{#if (params?.top_p ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -370,7 +375,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="1"
step="0.05"
step="any"
/>
</div>
</div>
@ -379,16 +384,16 @@
<div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Frequencey Penalty')}</div>
<div class=" self-center text-xs font-medium">{$i18n.t('Frequency Penalty')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.frequency_penalty = (params?.frequency_penalty ?? '') === '' ? 1.1 : '';
params.frequency_penalty = (params?.frequency_penalty ?? null) === null ? 1.1 : null;
}}
>
{#if (params?.frequency_penalty ?? '') === ''}
{#if (params?.frequency_penalty ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -396,7 +401,7 @@
</button>
</div>
{#if (params?.frequency_penalty ?? '') !== ''}
{#if (params?.frequency_penalty ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -416,7 +421,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="2"
step="0.05"
step="any"
/>
</div>
</div>
@ -431,10 +436,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.repeat_last_n = (params?.repeat_last_n ?? '') === '' ? 64 : '';
params.repeat_last_n = (params?.repeat_last_n ?? null) === null ? 64 : null;
}}
>
{#if (params?.repeat_last_n ?? '') === ''}
{#if (params?.repeat_last_n ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -442,7 +447,7 @@
</button>
</div>
{#if (params?.repeat_last_n ?? '') !== ''}
{#if (params?.repeat_last_n ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -477,10 +482,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.tfs_z = (params?.tfs_z ?? '') === '' ? 1 : '';
params.tfs_z = (params?.tfs_z ?? null) === null ? 1 : null;
}}
>
{#if (params?.tfs_z ?? '') === ''}
{#if (params?.tfs_z ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -488,7 +493,7 @@
</button>
</div>
{#if (params?.tfs_z ?? '') !== ''}
{#if (params?.tfs_z ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -508,7 +513,7 @@
class=" bg-transparent text-center w-14"
min="0"
max="2"
step="0.05"
step="any"
/>
</div>
</div>
@ -523,10 +528,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.num_ctx = (params?.num_ctx ?? '') === '' ? 2048 : '';
params.num_ctx = (params?.num_ctx ?? null) === null ? 2048 : null;
}}
>
{#if (params?.num_ctx ?? '') === ''}
{#if (params?.num_ctx ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -534,7 +539,7 @@
</button>
</div>
{#if (params?.num_ctx ?? '') !== ''}
{#if (params?.num_ctx ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -553,12 +558,13 @@
type="number"
class=" bg-transparent text-center w-14"
min="-1"
step="10"
step="1"
/>
</div>
</div>
{/if}
</div>
<div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Max Tokens (num_predict)')}</div>
@ -567,10 +573,10 @@
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.max_tokens = (params?.max_tokens ?? '') === '' ? 128 : '';
params.max_tokens = (params?.max_tokens ?? null) === null ? 128 : null;
}}
>
{#if (params?.max_tokens ?? '') === ''}
{#if (params?.max_tokens ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@ -578,7 +584,7 @@
</button>
</div>
{#if (params?.max_tokens ?? '') !== ''}
{#if (params?.max_tokens ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
@ -604,36 +610,125 @@
</div>
{/if}
</div>
<div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Template')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.template = (params?.template ?? null) === null ? '' : null;
}}
>
{#if (params?.template ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
{/if}
</button>
{#if admin}
<div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('use_mmap (Ollama)')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.use_mmap = (params?.use_mmap ?? null) === null ? true : null;
}}
>
{#if (params?.use_mmap ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('On')}</span>
{/if}
</button>
</div>
</div>
{#if (params?.template ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<textarea
class="px-3 py-1.5 text-sm w-full bg-transparent border dark:border-gray-600 outline-none rounded-lg -mb-1"
placeholder="Write your model template content here"
rows="4"
bind:value={params.template}
/>
</div>
<div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('use_mlock (Ollama)')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.use_mlock = (params?.use_mlock ?? null) === null ? true : null;
}}
>
{#if (params?.use_mlock ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('On')}</span>
{/if}
</button>
</div>
{/if}
</div>
</div>
<div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('num_thread (Ollama)')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.num_thread = (params?.num_thread ?? null) === null ? 2 : null;
}}
>
{#if (params?.num_thread ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
{/if}
</button>
</div>
{#if (params?.num_thread ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<input
id="steps-range"
type="range"
min="1"
max="256"
step="1"
bind:value={params.num_thread}
class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
/>
</div>
<div class="">
<input
bind:value={params.num_thread}
type="number"
class=" bg-transparent text-center w-14"
min="1"
max="256"
step="1"
/>
</div>
</div>
{/if}
</div>
<!-- <div class=" py-0.5 w-full justify-between">
<div class="flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Template')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
type="button"
on:click={() => {
params.template = (params?.template ?? null) === null ? '' : null;
}}
>
{#if (params?.template ?? null) === null}
<span class="ml-2 self-center">{$i18n.t('Default')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
{/if}
</button>
</div>
{#if (params?.template ?? null) !== null}
<div class="flex mt-0.5 space-x-2">
<div class=" flex-1">
<textarea
class="px-3 py-1.5 text-sm w-full bg-transparent border dark:border-gray-600 outline-none rounded-lg -mb-1"
placeholder="Write your model template content here"
rows="4"
bind:value={params.template}
/>
</div>
</div>
{/if}
</div> -->
{/if}
</div>

241
src/lib/components/chat/Settings/Audio.svelte
View File

@ -1,8 +1,8 @@
<script lang="ts">
import { getAudioConfig, updateAudioConfig } from '$lib/apis/audio';
import { user, settings } from '$lib/stores';
import { user, settings, config } from '$lib/stores';
import { createEventDispatcher, onMount, getContext } from 'svelte';
import { toast } from 'svelte-sonner';
import Switch from '$lib/components/common/Switch.svelte';
const dispatch = createEventDispatcher();
const i18n = getContext('i18n');
@ -10,24 +10,15 @@
export let saveSettings: Function;
// Audio
let OpenAIUrl = '';
let OpenAIKey = '';
let STTEngines = ['', 'openai'];
let STTEngine = '';
let conversationMode = false;
let speechAutoSend = false;
let responseAutoPlayback = false;
let nonLocalVoices = false;
let TTSEngines = ['', 'openai'];
let TTSEngine = '';
let STTEngine = '';
let voices = [];
let speaker = '';
let models = [];
let model = '';
let voice = '';
const getOpenAIVoices = () => {
voices = [
@ -40,10 +31,6 @@
];
};
const getOpenAIVoicesModel = () => {
models = [{ name: 'tts-1' }, { name: 'tts-1-hd' }];
};
const getWebAPIVoices = () => {
const getVoicesLoop = setInterval(async () => {
voices = await speechSynthesis.getVoices();
@ -55,21 +42,6 @@
}, 100);
};
const toggleConversationMode = async () => {
conversationMode = !conversationMode;
if (conversationMode) {
responseAutoPlayback = true;
speechAutoSend = true;
}
saveSettings({
conversationMode: conversationMode,
responseAutoPlayback: responseAutoPlayback,
speechAutoSend: speechAutoSend
});
};
const toggleResponseAutoPlayback = async () => {
responseAutoPlayback = !responseAutoPlayback;
saveSettings({ responseAutoPlayback: responseAutoPlayback });
@ -80,66 +52,35 @@
saveSettings({ speechAutoSend: speechAutoSend });
};
const updateConfigHandler = async () => {
if (TTSEngine === 'openai') {
const res = await updateAudioConfig(localStorage.token, {
url: OpenAIUrl,
key: OpenAIKey,
model: model,
speaker: speaker
});
if (res) {
OpenAIUrl = res.OPENAI_API_BASE_URL;
OpenAIKey = res.OPENAI_API_KEY;
model = res.OPENAI_API_MODEL;
speaker = res.OPENAI_API_VOICE;
}
}
};
onMount(async () => {
conversationMode = $settings.conversationMode ?? false;
speechAutoSend = $settings.speechAutoSend ?? false;
responseAutoPlayback = $settings.responseAutoPlayback ?? false;
STTEngine = $settings?.audio?.STTEngine ?? '';
TTSEngine = $settings?.audio?.TTSEngine ?? '';
speaker = $settings?.audio?.speaker ?? '';
model = $settings?.audio?.model ?? '';
STTEngine = $settings?.audio?.stt?.engine ?? '';
voice = $settings?.audio?.tts?.voice ?? $config.audio.tts.voice ?? '';
nonLocalVoices = $settings.audio?.tts?.nonLocalVoices ?? false;
if (TTSEngine === 'openai') {
if ($config.audio.tts.engine === 'openai') {
getOpenAIVoices();
getOpenAIVoicesModel();
} else {
getWebAPIVoices();
}
if ($user.role === 'admin') {
const res = await getAudioConfig(localStorage.token);
if (res) {
OpenAIUrl = res.OPENAI_API_BASE_URL;
OpenAIKey = res.OPENAI_API_KEY;
model = res.OPENAI_API_MODEL;
speaker = res.OPENAI_API_VOICE;
}
}
});
</script>
<form
class="flex flex-col h-full justify-between space-y-3 text-sm"
on:submit|preventDefault={async () => {
if ($user.role === 'admin') {
await updateConfigHandler();
}
saveSettings({
audio: {
STTEngine: STTEngine !== '' ? STTEngine : undefined,
TTSEngine: TTSEngine !== '' ? TTSEngine : undefined,
speaker: speaker !== '' ? speaker : undefined,
model: model !== '' ? model : undefined
stt: {
engine: STTEngine !== '' ? STTEngine : undefined
},
tts: {
voice: voice !== '' ? voice : undefined,
nonLocalVoices: $config.audio.tts.engine === '' ? nonLocalVoices : undefined
}
}
});
dispatch('save');
@ -149,53 +90,25 @@
<div>
<div class=" mb-1 text-sm font-medium">{$i18n.t('STT Settings')}</div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Speech-to-Text Engine')}</div>
<div class="flex items-center relative">
<select
class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
bind:value={STTEngine}
placeholder="Select a mode"
on:change={(e) => {
if (e.target.value !== '') {
navigator.mediaDevices.getUserMedia({ audio: true }).catch(function (err) {
toast.error(
$i18n.t(`Permission denied when accessing microphone: {{error}}`, {
error: err
})
);
STTEngine = '';
});
}
}}
>
<option value="">{$i18n.t('Default (Web API)')}</option>
<option value="whisper-local">{$i18n.t('Whisper (Local)')}</option>
</select>
{#if $config.audio.stt.engine !== 'web'}
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Speech-to-Text Engine')}</div>
<div class="flex items-center relative">
<select
class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
bind:value={STTEngine}
placeholder="Select an engine"
>
<option value="">{$i18n.t('Default')}</option>
<option value="web">{$i18n.t('Web API')}</option>
</select>
</div>
</div>
</div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Conversation Mode')}</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
toggleConversationMode();
}}
type="button"
>
{#if conversationMode === true}
<span class="ml-2 self-center">{$i18n.t('On')}</span>
{:else}
<span class="ml-2 self-center">{$i18n.t('Off')}</span>
{/if}
</button>
</div>
{/if}
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">
{$i18n.t('Auto-send input after 3 sec.')}
{$i18n.t('Instant Auto-Send After Voice Transcription')}
</div>
<button
@ -217,50 +130,6 @@
<div>
<div class=" mb-1 text-sm font-medium">{$i18n.t('TTS Settings')}</div>
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Text-to-Speech Engine')}</div>
<div class="flex items-center relative">
<select
class=" dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
bind:value={TTSEngine}
placeholder="Select a mode"
on:change={(e) => {
if (e.target.value === 'openai') {
getOpenAIVoices();
speaker = 'alloy';
model = 'tts-1';
} else {
getWebAPIVoices();
speaker = '';
}
}}
>
<option value="">{$i18n.t('Default (Web API)')}</option>
<option value="openai">{$i18n.t('Open AI')}</option>
</select>
</div>
</div>
{#if $user.role === 'admin'}
{#if TTSEngine === 'openai'}
<div class="mt-1 flex gap-2 mb-1">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('API Base URL')}
bind:value={OpenAIUrl}
required
/>
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder={$i18n.t('API Key')}
bind:value={OpenAIKey}
required
/>
</div>
{/if}
{/if}
<div class=" py-0.5 flex w-full justify-between">
<div class=" self-center text-xs font-medium">{$i18n.t('Auto-playback response')}</div>
@ -280,28 +149,39 @@
</div>
</div>
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
{#if TTSEngine === ''}
{#if $config.audio.tts.engine === ''}
<div>
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Voice')}</div>
<div class="flex w-full">
<div class="flex-1">
<select
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={speaker}
placeholder="Select a voice"
bind:value={voice}
>
<option value="" selected>{$i18n.t('Default')}</option>
{#each voices.filter((v) => v.localService === true) as voice}
<option value={voice.name} class="bg-gray-100 dark:bg-gray-700">{voice.name}</option
<option value="" selected={voice !== ''}>{$i18n.t('Default')}</option>
{#each voices.filter((v) => nonLocalVoices || v.localService === true) as _voice}
<option
value={_voice.name}
class="bg-gray-100 dark:bg-gray-700"
selected={voice === _voice.name}>{_voice.name}</option
>
{/each}
</select>
</div>
</div>
<div class="flex items-center justify-between my-1.5">
<div class="text-xs">
{$i18n.t('Allow non-local voices')}
</div>
<div class="mt-1">
<Switch bind:state={nonLocalVoices} />
</div>
</div>
</div>
{:else if TTSEngine === 'openai'}
{:else if $config.audio.tts.engine === 'openai'}
<div>
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Voice')}</div>
<div class="flex w-full">
@ -309,7 +189,7 @@
<input
list="voice-list"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={speaker}
bind:value={voice}
placeholder="Select a voice"
/>
@ -321,25 +201,6 @@
</div>
</div>
</div>
<div>
<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Model')}</div>
<div class="flex w-full">
<div class="flex-1">
<input
list="model-list"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={model}
placeholder="Select a model"
/>
<datalist id="model-list">
{#each models as model}
<option value={model.name} />
{/each}
</datalist>
</div>
</div>
</div>
{/if}
</div>

4
src/lib/components/chat/Settings/Chats.svelte
View File

@ -161,7 +161,7 @@
</div>
</div>
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
<div class="flex flex-col">
<input
@ -218,7 +218,7 @@
</button>
</div>
<hr class=" dark:border-gray-700" />
<hr class=" dark:border-gray-850" />
<div class="flex flex-col">
{#if showArchiveConfirm}

66
src/lib/components/chat/Settings/General.svelte
View File

@ -4,7 +4,7 @@
import { getLanguages } from '$lib/i18n';
const dispatch = createEventDispatcher();
import { models, settings, theme } from '$lib/stores';
import { models, settings, theme, user } from '$lib/stores';
const i18n = getContext('i18n');
@ -43,19 +43,19 @@
let params = {
// Advanced
seed: 0,
temperature: '',
frequency_penalty: '',
repeat_last_n: '',
mirostat: '',
mirostat_eta: '',
mirostat_tau: '',
top_k: '',
top_p: '',
seed: null,
temperature: null,
frequency_penalty: null,
repeat_last_n: null,
mirostat: null,
mirostat_eta: null,
mirostat_tau: null,
top_k: null,
top_p: null,
stop: null,
tfs_z: '',
num_ctx: '',
max_tokens: ''
tfs_z: null,
num_ctx: null,
max_tokens: null
};
const toggleRequestFormat = async () => {
@ -79,12 +79,6 @@
requestFormat = $settings.requestFormat ?? '';
keepAlive = $settings.keepAlive ?? null;
params.seed = $settings.seed ?? 0;
params.temperature = $settings.temperature ?? '';
params.frequency_penalty = $settings.frequency_penalty ?? '';
params.top_k = $settings.top_k ?? '';
params.top_p = $settings.top_p ?? '';
params.num_ctx = $settings.num_ctx ?? '';
params = { ...params, ...$settings.params };
params.stop = $settings?.params?.stop ? ($settings?.params?.stop ?? []).join(',') : null;
});
@ -146,6 +140,7 @@
<option value="dark">🌑 {$i18n.t('Dark')}</option>
<option value="oled-dark">🌃 {$i18n.t('OLED Dark')}</option>
<option value="light">☀️ {$i18n.t('Light')}</option>
<option value="her">🌷 Her</option>
<!-- <option value="rose-pine dark">🪻 {$i18n.t('Rosé Pine')}</option>
<option value="rose-pine-dawn light">🌷 {$i18n.t('Rosé Pine Dawn')}</option> -->
</select>
@ -203,7 +198,7 @@
</div>
</div>
<hr class=" dark:border-gray-700 my-3" />
<hr class=" dark:border-gray-850 my-3" />
<div>
<div class=" my-2.5 text-sm font-medium">{$i18n.t('System Prompt')}</div>
@ -227,8 +222,8 @@
</div>
{#if showAdvanced}
<AdvancedParams bind:params />
<hr class=" dark:border-gray-700" />
<AdvancedParams admin={$user?.role === 'admin'} bind:params />
<hr class=" dark:border-gray-850" />
<div class=" py-1 w-full justify-between">
<div class="flex w-full justify-between">
@ -300,20 +295,23 @@
saveSettings({
system: system !== '' ? system : undefined,
params: {
seed: (params.seed !== 0 ? params.seed : undefined) ?? undefined,
seed: (params.seed !== null ? params.seed : undefined) ?? undefined,
stop: params.stop ? params.stop.split(',').filter((e) => e) : undefined,
temperature: params.temperature !== '' ? params.temperature : undefined,
temperature: params.temperature !== null ? params.temperature : undefined,
frequency_penalty:
params.frequency_penalty !== '' ? params.frequency_penalty : undefined,
repeat_last_n: params.repeat_last_n !== '' ? params.repeat_last_n : undefined,
mirostat: params.mirostat !== '' ? params.mirostat : undefined,
mirostat_eta: params.mirostat_eta !== '' ? params.mirostat_eta : undefined,
mirostat_tau: params.mirostat_tau !== '' ? params.mirostat_tau : undefined,
top_k: params.top_k !== '' ? params.top_k : undefined,
top_p: params.top_p !== '' ? params.top_p : undefined,
tfs_z: params.tfs_z !== '' ? params.tfs_z : undefined,
num_ctx: params.num_ctx !== '' ? params.num_ctx : undefined,
max_tokens: params.max_tokens !== '' ? params.max_tokens : undefined
params.frequency_penalty !== null ? params.frequency_penalty : undefined,
repeat_last_n: params.repeat_last_n !== null ? params.repeat_last_n : undefined,
mirostat: params.mirostat !== null ? params.mirostat : undefined,
mirostat_eta: params.mirostat_eta !== null ? params.mirostat_eta : undefined,
mirostat_tau: params.mirostat_tau !== null ? params.mirostat_tau : undefined,
top_k: params.top_k !== null ? params.top_k : undefined,
top_p: params.top_p !== null ? params.top_p : undefined,
tfs_z: params.tfs_z !== null ? params.tfs_z : undefined,
num_ctx: params.num_ctx !== null ? params.num_ctx : undefined,
max_tokens: params.max_tokens !== null ? params.max_tokens : undefined,
use_mmap: params.use_mmap !== null ? params.use_mmap : undefined,
use_mlock: params.use_mlock !== null ? params.use_mlock : undefined,
num_thread: params.num_thread !== null ? params.num_thread : undefined
},
keepAlive: keepAlive ? (isNaN(keepAlive) ? keepAlive : parseInt(keepAlive)) : undefined
});

Some files were not shown because too many files have changed in this diff Show More