dnrops
/
vectordb-recipes
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
438 Commits 42 Branches 0 Tags 95 MiB
Jupyter Notebook 96.9%
Python 2.4%
TypeScript 0.4%
HTML 0.2%
JavaScript 0.1%
Go to file
Michael Becker d62902aecc
Update README.md (#203) 		2024-06-07 19:09:31 +05:30
.github/workflows node test ci fix (#163) 2024-03-27 23:38:00 +05:30
applications 🩹 broken link fix (#186) 2024-05-13 16:39:46 +05:30
assets README update (#198) 2024-06-03 10:31:22 +05:30
examples Update README.md (#203) 2024-06-07 19:09:31 +05:30
tutorials restructured README (#202) 2024-06-07 18:43:21 +05:30
.gitignore Imagebind demo update (#150) 2024-03-02 23:48:57 +05:30
LICENSE Initial commit 2023-06-25 11:40:35 +05:30
README.md restructured README (#202) 2024-06-07 18:43:21 +05:30
compile_testing.js Fix (#155) 2024-03-14 03:04:38 +05:30
package.json test 2023-07-28 13:11:26 -07:00
requirements.txt pin openai 2023-11-14 14:43:42 +05:30

README.md

VectorDB-recipes


Dive into building GenAI applications! This repository contains examples, applications, starter code, & tutorials to help you kickstart your GenAI projects.
  • These are built using LanceDB, a free, open-source, serverless vectorDB that requires no setup.
  • It integrates into python data ecosystem so you can simply start using these in your existing data pipelines in pandas, arrow, pydantic etc.
  • LanceDB has native Typescript SDK using which you can run vector search in serverless functions!

Join our community for support - Discord • Twitter

This repository is divided into 2 sections:

  • Examples - Get right into the code with minimal introduction, aimed at getting you from an idea to PoC within minutes!
  • Applications - Ready to use Python and web apps using applied LLMs, VectorDB and GenAI tools

Examples

Applied examples that get right into the code with minimal introduction, aimed at getting you from an idea to PoC within minutes! Examples are available as:

  • Colab notebooks - that builds the application is stages allowing you to investigate results at every intermediate stage.
  • Python scripts - for cases where you'd like directly to use the file or snippets to integrate in your application
  • JS/TS scripts - Some examples are written using lancedb's native js library! These script/snippets can also be directly integrated in your web applications.

The following examples are organized into different tables to make similar types of examples easily accessible.

Build from Scratch     Interactive Notebook & Scripts  
Build RAG from Scratch Open In Colab LLM beginner
Local RAG from Scratch with Llama3 Python local LLM beginner
Multimodal     Interactive Notebook & Scripts   Blog
Multimodal CLIP: DiffusionDB Open In Colab Python LLM beginner Ghost
Multimodal CLIP: Youtube videos Open In Colab Python LLM beginner Ghost
Multimodal Image + Text Search Open In Colab Python LLM intermediate Ghost
RAG     Interactive Notebook & Scripts   Blog
Improve RAG with Re-ranking Open In Colab LLM beginner Ghost
Instruct-Multitask Open In Colab Python LLM beginner Ghost
Improve RAG with HyDE Open In Colab LLM intermediate Ghost
Improve RAG with LOTR Open In Colab LLM intermediate Ghost
Advanced RAG: Parent Document Retriever Open In Colab LLM intermediate Ghost
Corrective RAG with Langgraph Open In Colab LLM intermediate Ghost
Contextual-Compression-with-RAG Open In Colab local LLM intermediate Ghost
Improve RAG with FLARE Open In Colab local LLM LLM advanced Ghost
Query Expansion and Reranker Open In Colab LLM advanced Ghost
RAG Fusion Open In Colab LLM advanced
Vector Search     Interactive Notebook & Scripts   Blog
Inbuilt Hybrid Search Open In Colab LLM beginner
Hybrid search BM25 & lancedb Open In Colab LLM beginner Ghost
NER powered Semantic Search Open In Colab local LLM beginner Ghost
Audio Search Open In Colab Python LLM beginner
Multi-lingual search Open In Colab Python LLM beginner
Facial Recognition Open In Colab beginner
Sentiment Analysis : Analysing Hotel Reviews Open In Colab local LLM beginner Ghost
Imagebind demo app hf spaces intermediate
Search Within Images Open In Colab local LLM intermediate Ghost
Vector Search with TransformersJS JS LLM advanced
Accelerate Vector Search Applications Using OpenVINO Open In Colab local LLM advanced Ghost
Chatbot     Interactive Notebook & Scripts   Blog  
Databricks DBRX Website Bot Python Databricks LLM beginner
CLI-based SDK Manual Chatbot with Phidata Python local LLM beginner
Youtube transcript search bot Open In Colab Python JS LLM intermediate
Langchain: Code Docs QA bot Open In Colab Python JS LLM intermediate
Context-Aware Chatbot using Llama 2 & LanceDB Open In Colab local LLM advanced Ghost
Evaluation     Interactive Notebook & Scripts   Blog
Evaluating Prompts with Prompttools Open In Colab LLM local LLM advanced
Evaluating RAG with RAGAs Open In Colab LLM intermediate
AI Agents     Interactive Notebook & Scripts   Blog
AI Agents: Reducing Hallucination Open In Colab Python JS LLM advanced Ghost
AI Trends Searcher with CrewAI Open In Colab LLM beginner Ghost
SuperAgent Autogen Open In Colab LLM intermediate
Recommender Systems     Interactive Notebook & Scripts   Blog
Movie Recommender Open In Colab Python beginner
Movie Recommender with Genre Open In Colab beginner Ghost
Product Recommender Open In Colab Pythonintermediate
Arxiv paper recommender Open In Colab Python LLM beginner
Concepts Interactive Notebook Blog Link
A Primer on Text Chunking and its Types Open In Colab beginner Ghost
Langchain LlamaIndex Chunking Open In Colab beginner Ghost
Comparing Cohere Rerankers with LanceDB beginner Ghost
Product Quantization: Compress High Dimensional Vectors intermediate Ghost
LLMs, RAG, & the missing storage layer for AI intermediate Ghost
Fine-Tuning LLM using PEFT & QLoRA Open In Colab local LLM advanced Ghost
Extracting Complex tables-text from PDFs using LlamaParse Open In Colab LLM LlamaCloud beginner

Projects & Applications

These are ready to use applications built using LanceDB serverless vector database. You can explore these open source projects, use parts of them in your projects or build your applications on top of these.

Project Name Description Screenshot
YOLOExplorer Iterate on your YOLO / CV datasets using SQL, Vector semantic search, and more within seconds YOLOExplorer
Website Chatbot (Deployable Vercel Template) Create a chatbot from the sitemap of any website/docs of your choice. Built using vectorDB serverless native javascript package. Chatbot
Chat with multiple URL/website Conversational AI for Any Website with Mistral,Bge Embedding & LanceDB webui_aa
Talk with Youtube Video using GPT4 Vision API Talk with Youtube Video using GPT4 Vision API and Langchain demo
Talk with Podcast Talk with Youtube Podcast using Ollama and insanely-fast-whisper demo
Talk with Wikipedia Talk with Wikipedia Pages demo
Talk with Github Talk with Github Codespaces using Qwen1.5 demo
Document Chat with Langroid Talk with your Documents using Langroid demo
Hr chatbot Hr chatbot - ask your personal query using zero-shot React agent & tools image
Advanced Chatbot with Parler TTS This Chatbot app uses Lancedb Hybrid search, FTS & reranker method with Parlers TTS library. image
Multi-Modal Search Engine Create a Multi-modal search engine app, to search images using both images or text Search
Multimodal Myntra Fashion Search Engine This app uses OpenAI's CLIP to make a search engine that can understand and deal with both written words and pictures. image
Multilingual-RAG Multilingual RAG with cohere embedding & support 100+ languages image
Fastapi RAG template FastAPI based RAG template with Websocket support image
GTE MLX RAG mlx based RAG model using lancedb api support image
Healthcare Chatbot Healthcare chatbot using domain specific LLM & Embedding model image

🌟 New! 🌟 Applied GenAI and VectorDB course on Udacity Learn about GenAI and vectorDBs using LanceDB in the recently launched Udacity Course

Contributing Examples

If you're working on some cool applications that you'd like to add to this repo, please open a PR!