commit 8b9fcd1b308eac923e0be1d511d1875946a89469 Author: Hamza Date: Thu May 21 10:59:10 2026 +0200 Add README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..8f434be --- /dev/null +++ b/README.md @@ -0,0 +1,586 @@ +# Flowise Chatbot Setup Guide — Linux/Ubuntu + +> Build a fully local AI-powered chatbot using Flowise, Ollama, and a Conversational Retrieval QA Chain with RAG (Retrieval-Augmented Generation). + +--- + +## Table of Contents + +1. [Prerequisites](#prerequisites) +2. [Install Ollama](#install-ollama) +3. [Pull Required Models](#pull-required-models) +4. [Install Flowise](#install-flowise) +5. [Start Flowise](#start-flowise) +6. [Build the Chatbot Flow](#build-the-chatbot-flow) + - [Step 1: Recursive Character Text Splitter](#step-1-recursive-character-text-splitter) + - [Step 2: Document Store (RAG)](#step-2-document-store-rag) + - [Step 3: Ollama Embeddings](#step-3-ollama-embeddings) + - [Step 4: Faiss Vector Store](#step-4-faiss-vector-store) + - [Step 5: Ollama Chat Model](#step-5-ollama-chat-model) + - [Step 6: Buffer Memory](#step-6-buffer-memory) + - [Step 7: Conversational Retrieval QA Chain](#step-7-conversational-retrieval-qa-chain) +7. [Upsert Documents](#upsert-documents) +8. [Using Cloud LLM APIs (Optional)](#using-cloud-llm-apis-optional) +9. [Deploy Chatbot to Users](#deploy-chatbot-to-users) + - [Direct Link](#direct-link) + - [Embed in Website](#embed-in-website) + - [Session Management](#session-management) + - [Rate Limiting](#rate-limiting) +10. [Troubleshooting](#troubleshooting) + +--- + +## Prerequisites + +- Ubuntu 20.04 or later +- At least **8GB RAM** (16GB recommended) +- **Node.js 18+** +- Internet access for first-time model downloads + +### Install Node.js 18+ + +```bash +curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash - +sudo apt-get install -y nodejs +node -v # should print v18.x.x or higher +``` + +--- + +## Install Ollama + +Ollama runs LLMs locally on your machine. + +```bash +curl -fsSL https://ollama.com/install.sh | sh +``` + +### Make Ollama Accessible on All Interfaces + +By default, Ollama only listens on `localhost`. To allow Flowise (and other services) to reach it: + +```bash +sudo systemctl edit ollama.service +``` + +Add the following inside the file: + +```ini +[Service] +Environment="OLLAMA_HOST=0.0.0.0" +``` + +Save and restart: + +```bash +sudo systemctl daemon-reload +sudo systemctl restart ollama +``` + +Verify it's running: + +```bash +curl http://localhost:11434/api/tags +``` + +You should see a JSON response listing available models. + +--- + +## Pull Required Models + +### Chat Model (LLM) + +```bash +ollama pull qwen2.5:3b # lightweight, good for most use cases +# OR +ollama pull phi3:mini # alternative lightweight model +# OR +ollama pull llama3:8b # better quality, needs more RAM +``` + +### Embedding Model + +```bash +ollama pull nomic-embed-text # required for RAG / vector search +``` + +### (Optional) Vision Model — for image understanding + +```bash +ollama pull llava # allows users to upload screenshots +``` + +Verify all models are available: + +```bash +ollama list +``` + +--- + +## Install Flowise + +```bash +npm install -g flowise +``` + +--- + +## Start Flowise + +```bash +npx flowise start +``` + +Flowise will start on **http://localhost:3000** + +### Run as a Background Service (Recommended for Production) + +```bash +# Install PM2 process manager +npm install -g pm2 + +# Start Flowise with PM2 +pm2 start "npx flowise start" --name flowise + +# Auto-start on system reboot +pm2 startup +pm2 save +``` + +### Set Username and Password (Recommended) + +```bash +npx flowise start --FLOWISE_USERNAME=admin --FLOWISE_PASSWORD=yourpassword +``` + +Or with PM2: + +```bash +pm2 start "npx flowise start --FLOWISE_USERNAME=admin --FLOWISE_PASSWORD=yourpassword" --name flowise +``` + +--- + +## Build the Chatbot Flow + +Open Flowise at `http://localhost:3000` → click **Add New** to create a new chatflow. + +The final flow looks like this: + +``` +Document Store + ↓ +Recursive Character Text Splitter + ↓ +Ollama Embeddings → Faiss Vector Store → Conversational Retrieval QA Chain + ↑ + Ollama (Chat) + ↑ + Buffer Memory +``` + +--- + +### Step 1: Recursive Character Text Splitter + +Search for **"Recursive Character Text Splitter"** in the nodes panel and drag it onto the canvas. + +| Setting | Recommended Value | +|---|---| +| Chunk Size | `1000` | +| Chunk Overlap | `200` | + +--- + +### Step 2: Document Store (RAG) + +Search for **"Document Store"** and drag it onto the canvas. + +1. Click **Select Store** → **Create New Store** +2. Name it (e.g., `Product Documentation`) +3. Assign the **Recursive Character Text Splitter** to the Document Store +4. Add your documents: + - Click the store → **Add Document Loader** + - Choose a loader: **Text File**, **PDF File**, **Docx File**, etc. + - Upload your knowledge base files +5. Connect the **Document** output of the Document Store to the Faiss node + +> **What to put in your documents:** +> Write plain text files or PDFs describing your software — features, FAQs, installation steps, error explanations, etc. The more detailed your docs, the better the chatbot answers. + +--- + +### Step 3: Ollama Embeddings + +Search for **"Ollama Embeddings"** and drag it onto the canvas. + +| Setting | Value | +|---|---| +| Base URL | `http://localhost:11434` (or your server IP) | +| Model Name | `nomic-embed-text:latest` | + +Connect the **OllamaEmbeddings** output to the **Embeddings** input of the Faiss node. + +--- + +### Step 4: Faiss Vector Store + +Search for **"Faiss"** and drag it onto the canvas. + +| Setting | Value | +|---|---| +| Base Path to load | `/root/.flowise/vectorstore` (must be an existing directory) | + +Create the directory if it doesn't exist: + +```bash +mkdir -p /root/.flowise/vectorstore +``` + +Connect: +- **Document** output from Document Store → **Document** input of Faiss +- **OllamaEmbeddings** → **Embeddings** input of Faiss + +The **Faiss Retriever** output connects to the **Vector Store Retriever** input of the QA Chain. + +--- + +### Step 5: Ollama Chat Model + +Search for **"ChatOllama"** or **"Ollama"** and drag it onto the canvas. + +| Setting | Value | +|---|---| +| Base URL | `http://localhost:11434` (or your server IP) | +| Model Name | `qwen2.5:3b` | +| Temperature | `0.7` (lower = more factual, higher = more creative) | + +> **Important:** Use the exact model name with a colon, e.g. `qwen2.5:3b` not `qwen2.5-3b`. + +Connect the **ChatOllama** output to the **Chat Model** input of the QA Chain. + +--- + +### Step 6: Buffer Memory + +Search for **"Buffer Memory"** and drag it onto the canvas. + +| Setting | Value | +|---|---| +| Session ID | *(leave empty — auto-uses the session ID passed by the user)* | +| Memory Key | `chat_history` | + +Connect the **BufferMemory** output to the **Memory** input of the QA Chain. + +> Buffer Memory keeps conversation history so users can ask follow-up questions naturally. +> Note: Buffer Memory resets if Flowise restarts. For persistent memory across restarts, use **Redis-Backed Memory** or **MongoDB Memory** instead. + +--- + +### Step 7: Conversational Retrieval QA Chain + +Search for **"Conversational Retrieval QA Chain"** and drag it onto the canvas. + +Connect: +- **ChatOllama** → **Chat Model** +- **Faiss Retriever** → **Vector Store Retriever** +- **BufferMemory** → **Memory** + +Leave **Input Moderation** and **Return Source Documents** as optional based on your needs. + +**Save the flow** using the save button (top right). + +--- + +## Upsert Documents + +Before the chatbot can answer questions, you must index your documents into the vector store. + +1. Click the **Upsert** button (top right of the canvas) +2. Wait for it to complete — you should see a success message +3. Verify the index files were created: + +```bash +ls /root/.flowise/vectorstore/ +# Expected output: faiss.index faiss.json +``` + +> **Re-upsert every time** you update or add documents to the Document Store. + +--- + +## Using Cloud LLM APIs (Optional) + +If you prefer using cloud providers instead of (or in addition to) Ollama: + +### OpenAI + +1. Get your API key at https://platform.openai.com/api-keys +2. In Flowise: **Settings** → **API Keys** → Add key +3. Replace the **ChatOllama** node with a **ChatOpenAI** node +4. Enter your API key and choose a model (e.g., `gpt-4o`) + +### Anthropic (Claude) + +1. Get your API key at https://console.anthropic.com +2. Replace the **ChatOllama** node with a **ChatAnthropic** node +3. Enter your API key and choose a model (e.g., `claude-sonnet-4-5`) + +### Google Gemini + +1. Get your API key at https://aistudio.google.com +2. Use the **ChatGoogleGenerativeAI** node +3. Enter your API key and model name (e.g., `gemini-1.5-pro`) + +> **For Embeddings with cloud APIs:** Replace Ollama Embeddings with **OpenAI Embeddings** or **Google Generative AI Embeddings** nodes, using the same API key setup. + +--- + +## Deploy Chatbot to Users + +### Direct Link + +Share the chatbot link directly — no account needed for users: + +``` +http://your-server-ip:3000/chatbot/YOUR-FLOW-ID +``` + +Find your flow ID in the URL when editing the flow. + +Users just open the link and start chatting immediately. No signup required. + +--- + +### Embed in Website + +Paste this into any HTML page to show a chat bubble: + +```html + +``` + +Get your embed code from Flowise: open the flow → click the **`<>` (Embed)** button (top right). + +--- + +### Session Management + +Each user needs an isolated session to prevent seeing each other's conversation history. + +| Scenario | Behavior | +|---|---| +| No `sessionId` set | All users share memory — **dangerous** | +| Unique `sessionId` per user | Fully isolated conversations | +| Using your own auth system | Pass your user's ID as the `sessionId` | + +```javascript +// If users are logged into your system, use their user ID +sessionId: currentUser.id // e.g. "user_12345" + +// For anonymous users, use localStorage (persists across page refreshes) +sessionId: localStorage.getItem("chatSessionId") || (() => { + const id = crypto.randomUUID(); + localStorage.setItem("chatSessionId", id); + return id; +})() +``` + +--- + +### Rate Limiting + +Limit how many messages a user can send to protect your server. + +1. Open your flow in Flowise +2. Click **⚙️ Configuration** (top right) +3. Go to the **Rate Limiting** tab +4. Configure: + +| Setting | Example Value | +|---|---| +| Message Limit | `20` | +| Duration (seconds) | `60` | +| Limit Message | `"Too many messages. Please wait a moment."` | + +This limits each IP address to 20 messages per 60 seconds. + +--- + +## Troubleshooting + +### ❌ `fetch failed` when chatting + +**Cause:** Flowise cannot reach Ollama. + +**Fix:** +```bash +# Verify Ollama is reachable +curl http://localhost:11434/api/tags + +# Check Ollama is running +systemctl status ollama + +# Check model name is correct (must use colon, not dash) +# ✅ qwen2.5:3b +# ❌ qwen2.5-3b +ollama list +``` + +--- + +### ❌ `Cannot read properties of undefined (reading 'startsWith')` + +**Cause:** The **Base Path to load** field in the Faiss node is empty. + +**Fix:** +1. Click the Faiss node +2. Fill in Base Path to load: `/root/.flowise/vectorstore` +3. Create the directory: `mkdir -p /root/.flowise/vectorstore` +4. Re-upsert + +--- + +### ❌ `could not open faiss.index for reading` + +**Cause:** Upsert was never completed successfully — the index file doesn't exist. + +**Fix:** +```bash +# Check if the file exists +ls /root/.flowise/vectorstore/ + +# If empty, fix the Base Path issue above, then Upsert again +``` + +--- + +### ❌ Ollama model not found + +**Cause:** Model name is wrong or model was never pulled. + +**Fix:** +```bash +# List available models +ollama list + +# Pull the missing model +ollama pull qwen2.5:3b +ollama pull nomic-embed-text:latest + +# Always use colon format in Flowise: qwen2.5:3b not qwen2.5-3b +``` + +--- + +### ❌ Flowise crashes or won't start + +**Fix:** +```bash +# Check Node.js version (must be 18+) +node -v + +# Clear Flowise cache +rm -rf ~/.flowise/cache + +# Check port 3000 is not already in use +sudo lsof -i :3000 + +# Restart with PM2 +pm2 restart flowise +pm2 logs flowise +``` + +--- + +### ❌ Out of memory / model too slow + +**Fix:** + +```bash +# Check available RAM +free -h + +# Use a smaller model +ollama pull phi3:mini # 3.8B — lighter than qwen2.5:3b + +# Check what's loaded in Ollama +ollama ps +``` + +If RAM is under 8GB, use `phi3:mini` or `qwen2.5:3b` (both under 4GB). + +--- + +### ❌ Bot answers incorrectly or doesn't use documents + +**Cause:** Documents are not properly indexed, or chunks are too small/large. + +**Fix:** +1. Make sure you clicked **Upsert** after adding documents +2. Verify `faiss.index` and `faiss.json` exist in the Base Path +3. Try increasing **Chunk Size** to `1500` and re-upsert +4. Make sure your documents are in plain text, not scanned images + +--- + +### ❌ Users share each other's conversation history + +**Cause:** No `sessionId` is being passed, or all users get the same ID. + +**Fix:** Use the embed code with a unique `sessionId` per user (see [Session Management](#session-management)). + +--- + +## Quick Reference + +| Component | Purpose | +|---|---| +| **Ollama** | Runs LLMs locally | +| **nomic-embed-text** | Converts text to vectors for search | +| **Faiss** | Stores and searches vectors | +| **Document Store** | Manages your knowledge base files | +| **Recursive Text Splitter** | Breaks documents into searchable chunks | +| **Buffer Memory** | Remembers conversation history per user | +| **Conversational Retrieval QA Chain** | Ties everything together | + +| Port | Service | +|---|---| +| `3000` | Flowise UI and API | +| `11434` | Ollama API | + +--- + +*Built with Flowise + Ollama. All models run locally — no data leaves your server.* \ No newline at end of file