Chatbot/README.md

# Flowise Chatbot Setup Guide — Linux/Ubuntu

> Build a fully local AI-powered chatbot using Flowise, Ollama, and a Conversational Retrieval QA Chain with RAG (Retrieval-Augmented Generation).

---

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Install Ollama](#install-ollama)
3. [Pull Required Models](#pull-required-models)
4. [Install Flowise](#install-flowise)
5. [Start Flowise](#start-flowise)
6. [Import an Existing Flow](#import-an-existing-flow)
7. [Build the Chatbot Flow](#build-the-chatbot-flow)
   - [Step 1: Recursive Character Text Splitter](#step-1-recursive-character-text-splitter)
   - [Step 2: Document Store (RAG)](#step-2-document-store-rag)
   - [Step 3: Ollama Embeddings](#step-3-ollama-embeddings)
   - [Step 4: Faiss Vector Store](#step-4-faiss-vector-store)
   - [Step 5: Ollama Chat Model](#step-5-ollama-chat-model)
   - [Step 6: Buffer Memory](#step-6-buffer-memory)
8. [Upsert Documents](#upsert-documents)
10. [Using Firecrawl (Scrape Websites into RAG)](#using-firecrawl-scrape-websites-into-rag)
11. [Using Cloud LLM APIs (Optional)](#using-cloud-llm-apis-optional)
12. [Multilingual Support & Embeddings](#multilingual-support--embeddings)
    - [How Language Works in the Pipeline](#how-language-works-in-the-pipeline)
    - [The Golden Rule of Embeddings](#the-golden-rule-of-embeddings)
    - [Choosing an Embedding Model](#choosing-an-embedding-model)
    - [North African & Middle Eastern Users](#north-african--middle-eastern-users)
13. [Deploy Chatbot to Users](#deploy-chatbot-to-users)
   - [Direct Link](#direct-link)
   - [Embed in Website](#embed-in-website)
   - [Session Management](#session-management)
   - [Rate Limiting](#rate-limiting)
14. [Troubleshooting](#troubleshooting)

---

## Prerequisites

- Ubuntu 20.04 or later
- At least **8GB RAM** (16GB recommended)
- **Node.js 18+**
- Internet access for first-time model downloads

### Install Node.js 18+

```bash
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs
node -v   # should print v18.x.x or higher
```

---

## Install Ollama

Ollama runs LLMs locally on your machine.

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

### Make Ollama Accessible on All Interfaces

By default, Ollama only listens on `localhost`. To allow Flowise (and other services) to reach it:

```bash
sudo systemctl edit ollama.service
```

Add the following inside the file:

```ini
[Service]
Environment="OLLAMA_HOST=0.0.0.0"
```

Save and restart:

```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```

Verify it's running:

```bash
curl http://localhost:11434/api/tags
```

You should see a JSON response listing available models.

---

## Pull Required Models

### Chat Model (LLM)

```bash
ollama pull qwen2.5:3b        # lightweight, good for most use cases
# OR
ollama pull phi3:mini         # alternative lightweight model
# OR
ollama pull llama3:8b         # better quality, needs more RAM
```

### Embedding Model

```bash
ollama pull nomic-embed-text  # required for RAG / vector search
```

### (Optional) Vision Model — for image understanding

```bash
ollama pull llava             # allows users to upload screenshots
```

Verify all models are available:

```bash
ollama list
```

---

## Install Flowise

```bash
npm install -g flowise
```

---

## Start Flowise

```bash
npx flowise start
# OR
npx flowise start --PORT=3030 # set port
```

Flowise will start on **http://localhost:3000**

### Run as a Background Service (Recommended for Production)

```bash
# Install PM2 process manager
npm install -g pm2

# Start Flowise with PM2
pm2 start "npx flowise start" --name flowise

# Auto-start on system reboot
pm2 startup
pm2 save
```

### Set Username and Password (Recommended)

```bash
npx flowise start --FLOWISE_USERNAME=admin --FLOWISE_PASSWORD=yourpassword
```

Or with PM2:

```bash
pm2 start "npx flowise start --FLOWISE_USERNAME=admin --FLOWISE_PASSWORD=yourpassword" --name flowise
```

---
 
## Import an Existing Flow
 
If you already have a chatflow exported as a `.json` file (e.g. `Product Detection Q&A Chatflow.json`), you can import it directly into Flowise instead of building from scratch.
 
### Download the Flow File
 
Get the `.json` file from your repository:
 
```bash
# Using wget
wget -O "Product Detection Q&A Chatflow.json" \
  "http://145.239.66.197:3000/Hamza/Chatbot/raw/branch/main/Product%20Detection%20Q%26A%20Chatflow.json"
 
# OR using curl
curl -L -o "Product Detection Q&A Chatflow.json" \
  "http://145.239.66.197:3000/Hamza/Chatbot/raw/branch/main/Product%20Detection%20Q%26A%20Chatflow.json"
```
 
> **Note:** The URL uses `/raw/branch/` to get the raw file content, not the Gitea preview page.
 
### Import into Flowise
 
1. Open Flowise at `http://localhost:3000`
2. On the **Chatflows** home page, click the **Add New** button (top right)
3. Instead of building from scratch, click the **Load** button (upload icon, top right of the canvas)
4. Select your `Product Detection Q&A Chatflow.json` file
5. The full flow will appear on the canvas automatically
Alternatively, from the **Chatflows** home page:
1. Click the **⋮ (three dots)** menu on any existing chatflow card
2. Select **Duplicate** — or use **Import** if available in your Flowise version
### After Importing
 
The flow is ready but you need to **reconfigure credentials** since API keys and local paths don't transfer between machines:
 
| Node | What to Reconfigure |
|---|---|
| **Ollama / ChatOllama** | Set Base URL to `http://localhost:11434` |
| **Ollama Embeddings** | Set Base URL to `http://localhost:11434` |
| **Faiss** | Set Base Path to `/root/.flowise/vectorstore` |
| **Document Store** | Re-upload or re-link your documents |
| **Any API node** | Re-enter API keys (OpenAI, Anthropic, etc.) |
 
### Create the Vector Store Directory
 
```bash
mkdir -p /root/.flowise/vectorstore
```
 
### Upsert After Import
 
After reconfiguring, always run Upsert before chatting:
 
1. Click the **Upsert** button (top right of canvas)
2. Wait for success confirmation
3. Verify the index was created:
```bash
ls /root/.flowise/vectorstore/
# Expected: faiss.index  faiss.json
```
 
### Export Your Flow (for sharing or backup)
 
To export your current flow as a `.json` file:
 
1. Open the chatflow in Flowise
2. Click the **⋮ (three dots)** menu → **Export**
3. Save the `.json` file — commit it to your repository for teammates to import

---

## Build the Chatbot Flow

Open Flowise at `http://localhost:3000` → click **Add New** to create a new chatflow.

The final flow looks like this:

<img height=700 src="assets/Chatbot%20Flowchart.PNG" />

---

### Step 1: Recursive Character Text Splitter

Search for **"Recursive Character Text Splitter"** in the nodes panel and drag it onto the canvas.

| Setting | Recommended Value |
|---|---|
| Chunk Size | `1000` |
| Chunk Overlap | `200` |

---

### Step 2: Document Store (RAG)

Search for **"Document Store"** and drag it onto the canvas.

1. Click **Select Store** → **Create New Store**
2. Name it (e.g., `Product Documentation`)
3. Assign the **Recursive Character Text Splitter** to the Document Store
4. Add your documents:
   - Click the store → **Add Document Loader**
   - Choose a loader: **Text File**, **PDF File**, **Docx File**, etc.
   - Upload your knowledge base files
5. Connect the **Document** output of the Document Store to the Faiss node

> **What to put in your documents:**
> Write plain text files or PDFs describing your software — features, FAQs, installation steps, error explanations, etc. The more detailed your docs, the better the chatbot answers.

---

### Step 3: Ollama Embeddings

Search for **"Ollama Embeddings"** and drag it onto the canvas.

| Setting | Value |
|---|---|
| Base URL | `http://localhost:11434` (or your server IP) |
| Model Name | `nomic-embed-text:latest` |

Connect the **OllamaEmbeddings** output to the **Embeddings** input of the Faiss node.

---

### Step 4: Faiss Vector Store

Search for **"Faiss"** and drag it onto the canvas.

| Setting | Value |
|---|---|
| Base Path to load | `/root/.flowise/vectorstore` (must be an existing directory) |

Create the directory if it doesn't exist:

```bash
mkdir -p /root/.flowise/vectorstore
```

Connect:
- **Document** output from Document Store → **Document** input of Faiss
- **OllamaEmbeddings** → **Embeddings** input of Faiss

The **Faiss Retriever** output connects to the **Vector Store Retriever** input of the QA Chain.

---

### Step 5: Ollama Chat Model

Search for **"ChatOllama"** or **"Ollama"** and drag it onto the canvas.

| Setting | Value |
|---|---|
| Base URL | `http://localhost:11434` (or your server IP) |
| Model Name | `qwen2.5:3b` |
| Temperature | `0.7` (lower = more factual, higher = more creative) |

> **Important:** Use the exact model name with a colon, e.g. `qwen2.5:3b` not `qwen2.5-3b`.

Connect the **ChatOllama** output to the **Chat Model** input of the QA Chain.

---

### Step 6: Buffer Memory

Search for **"Buffer Memory"** and drag it onto the canvas.

| Setting | Value |
|---|---|
| Session ID | *(leave empty — auto-uses the session ID passed by the user)* |
| Memory Key | `chat_history` |

Connect the **BufferMemory** output to the **Memory** input of the QA Chain.

> Buffer Memory keeps conversation history so users can ask follow-up questions naturally.
> Note: Buffer Memory resets if Flowise restarts. For persistent memory across restarts, use **Redis-Backed Memory** or **MongoDB Memory** instead.

---

## Upsert Documents

Before the chatbot can answer questions, you must index your documents into the vector store.

1. Click the **Upsert** button (top right of the canvas)
2. Wait for it to complete — you should see a success message
3. Verify the index files were created:

```bash
ls /root/.flowise/vectorstore/
# Expected output: faiss.index  faiss.json
```

> **Re-upsert every time** you update or add documents to the Document Store.

---

## Using Cloud LLM APIs (Optional)

If you prefer using cloud providers instead of (or in addition to) Ollama:

### OpenAI

1. Get your API key at https://platform.openai.com/api-keys
2. In Flowise: **Settings** → **API Keys** → Add key
3. Replace the **ChatOllama** node with a **ChatOpenAI** node
4. Enter your API key and choose a model (e.g., `gpt-4o`)

### Anthropic (Claude)

1. Get your API key at https://console.anthropic.com
2. Replace the **ChatOllama** node with a **ChatAnthropic** node
3. Enter your API key and choose a model (e.g., `claude-sonnet-4-5`)

### Google Gemini

1. Get your API key at https://aistudio.google.com
2. Use the **ChatGoogleGenerativeAI** node
3. Enter your API key and model name (e.g., `gemini-1.5-pro`)

> **For Embeddings with cloud APIs:** Replace Ollama Embeddings with **OpenAI Embeddings** or **Google Generative AI Embeddings** nodes, using the same API key setup.

---

## Deploy Chatbot to Users

### Direct Link

Share the chatbot link directly — no account needed for users:

```
http://your-server-ip:3000/chatbot/YOUR-FLOW-ID
```

Find your flow ID in the URL when editing the flow.

Users just open the link and start chatting immediately. No signup required.

---

### Embed in Website

Paste this into any HTML page to show a chat bubble:

```html
<script type="module">
  import Chatbot from "https://cdn.jsdelivr.net/npm/flowise-embed/dist/web.js"

  // Generate or reuse a session ID per user
  let sessionId = localStorage.getItem("chatSessionId");
  if (!sessionId) {
    sessionId = crypto.randomUUID();
    localStorage.setItem("chatSessionId", sessionId);
  }

  Chatbot.init({
    chatflowid: "YOUR-FLOW-ID",
    apiHost: "http://your-server-ip:3000",
    chatflowConfig: {
      sessionId: sessionId
    },
    theme: {
      button: {
        backgroundColor: "#your-brand-color",
        right: 20,
        bottom: 20,
      },
      chatWindow: {
        title: "Software Support",
        welcomeMessage: "Hello! How can I help you today?",
        height: 600,
        width: 400,
      }
    }
  })
</script>
```

Get your embed code from Flowise: open the flow → click the **`<>` (Embed)** button (top right).

---

### Session Management

Each user needs an isolated session to prevent seeing each other's conversation history.

| Scenario | Behavior |
|---|---|
| No `sessionId` set | All users share memory — **dangerous** |
| Unique `sessionId` per user | Fully isolated conversations |
| Using your own auth system | Pass your user's ID as the `sessionId` |

```javascript
// If users are logged into your system, use their user ID
sessionId: currentUser.id  // e.g. "user_12345"

// For anonymous users, use localStorage (persists across page refreshes)
sessionId: localStorage.getItem("chatSessionId") || (() => {
  const id = crypto.randomUUID();
  localStorage.setItem("chatSessionId", id);
  return id;
})()
```

---

### Rate Limiting

Limit how many messages a user can send to protect your server.

1. Open your flow in Flowise
2. Click **⚙️ Configuration** (top right)
3. Go to the **Rate Limiting** tab
4. Configure:

| Setting | Example Value |
|---|---|
| Message Limit | `20` |
| Duration (seconds) | `60` |
| Limit Message | `"Too many messages. Please wait a moment."` |

This limits each IP address to 20 messages per 60 seconds.

---

## Using Firecrawl (Scrape Websites into RAG)
 
Firecrawl lets you scrape entire websites and feed the content directly into your Document Store as a knowledge base. This is useful if your software documentation lives on a website or wiki.
 
---
 
### What Firecrawl Does
 
```
Your website / docs URL
        ↓
  Firecrawl crawls all pages
        ↓
  Returns clean Markdown text
        ↓
  Loaded into Document Store
        ↓
  Indexed into Faiss for RAG
```
 
---
 
### Option A: Firecrawl Cloud (Easiest)
 
1. Sign up at [https://firecrawl.dev](https://firecrawl.dev)
2. Get your API key from the dashboard
3. Add the API key to Flowise:
   - Go to **Settings** → **API Keys** → **Add Credential**
   - Choose **Firecrawl API** and paste your key
---
 
### Option B: Self-Host Firecrawl (No API Cost)
 
If you want to keep everything local and free:
 
#### Requirements
 
```bash
# Install Docker and Docker Compose
sudo apt-get install -y docker.io docker-compose
```
 
#### Setup
 
```bash
# Clone Firecrawl repository
git clone https://github.com/mendableai/firecrawl.git
cd firecrawl
 
# Copy environment file
cp .env.example .env
```
 
Edit the `.env` file:
 
```bash
nano .env
```
 
Set these values at minimum:
 
```env
NUM_WORKERS_PER_QUEUE=8
PORT=3002
HOST=0.0.0.0
REDIS_URL=redis://redis:6379
PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/scrape
```
 
Start Firecrawl:
 
```bash
docker-compose up -d
```
 
Verify it's running:
 
```bash
curl http://localhost:3002/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'
```
 
Your self-hosted Firecrawl API is now at `http://localhost:3002`
 
---
 
## Multilingual Support & Embeddings
 
### How Language Works in the Pipeline
 
Your chatbot has two separate language-handling stages that behave differently:
 
| Stage | Component | What it Does |
|---|---|---|
| **Retrieval** | Embedding model + Faiss | Finds relevant doc chunks |
| **Generation** | LLM (Gemini, Ollama, etc.) | Writes the answer |
 
The LLM (Gemini 2.5 Flash, qwen2.5, etc.) is naturally multilingual — it will respond in whatever language the user writes in. The weak point is always **retrieval**: if the embedding model can't match a French or Arabic query to English docs, the LLM receives empty context and responds with "I am not sure."
 
```
User asks in Arabic
      ↓
Embedding model converts query to vector
      ↓
Faiss searches English docs → poor match → returns nothing
      ↓
LLM receives empty context → "Hmm, I am not sure" ❌
```
 
---
 
### The Golden Rule of Embeddings
 
> **The model used during Upsert MUST be the same model used during Chat.**
 
Each embedding model has its own internal "language" for converting text to vectors. They are not compatible with each other.
 
```
Upsert with nomic-embed-text:
"product warranty" → [0.23, 0.87, 0.12, ...]
 
Chat query with text-embedding-004:
"product warranty" → [0.91, 0.04, 0.67, ...]
 
Faiss compares these → completely different → no match ❌
```
 
If you switch embedding models you **must delete the old index and re-upsert**:
 
```bash
rm /root/.flowise/vectorstore/faiss.index
rm /root/.flowise/vectorstore/faiss.json
# Then hit Upsert again in Flowise
```
 
---
 
### Choosing an Embedding Model
 
| Model                    | Where      | Arabic      | French      | Darija | Code-switching |
| ------------------------ | ---------- | ----------- | ----------- | ------ | -------------- |
| `nomic-embed-text`       | Ollama     | ❌           | ❌           | ❌      | ❌              |
| `mxbai-embed-large`      | Ollama     | ❌           | ❌           | ❌      | ❌              |
| `bge-m3`                 | Ollama     | ✅ Excellent | ✅ Excellent | ❌ Poor | ⚠️ Partial     |
| `text-embedding-004`     | Google API | ✅ Good      | ✅ Good      | ❌ Poor | ❌ Poor         |
| `text-embedding-3-small` | OpenAI API | ✅ Good      | ✅ Good      | ❌ Poor | ❌ Poor         |
| `multilingual-e5-large`  | Ollama     | ✅ Good      | ✅ Good      | ❌ Poor | ⚠️ Partial     |

 
**Recommendation:** Switch to `bge-m3` for the best multilingual retrieval without any API cost:
 
```bash
ollama pull bge-m3
```
 
Then update the **Ollama Embeddings** node model name to `bge-m3:latest`, delete the old index, and re-upsert.
 
---
 
### North African & Middle Eastern Users
 
North African users (Algeria, Morocco, Tunisia) often write in **code-switched** messages mixing Arabic dialect (Darija) with French in the same sentence:
 
```
"كيفاش نdir la configuration?"
"le produit مايخدمش properly"
"comment تاع l'installation?"
```
 
This is called **code-switching** and no embedding model handles it perfectly — Darija is underrepresented in all training datasets. Use a combination of strategies:
 
#### Strategy 1: System Prompt (Do This First)
 
In **Conversational Retrieval QA Chain** → **Additional Parameters** → **System Message**:
 
```
You are a helpful product support assistant for North African users.
Users may write in Arabic (Modern Standard or Darija dialect),
French, or a mix of both languages in the same message.
 
When you receive a message:
1. Understand it regardless of the language mix
2. Answer in the same language(s) the user wrote in
3. If the retrieved context is insufficient, use your general
   knowledge to help the user as best as possible
4. Never respond with "I am not sure" without first attempting
   to answer based on your knowledge
```
 
Your LLM (especially Gemini) understands Darija + French mixing very well — this prompt prevents it from giving up when retrieval returns weak results.
 
#### Strategy 2: Add Mixed-Language Content to Your Docs
 
Write a FAQ section in your Document Store using the way your users actually type:
 
```
كيفاش نinstalli le produit؟ / Comment installer le produit?
→ Go to Settings → Install → follow the steps...
 
le produit مايخدمش / Le produit ne fonctionne pas
→ First check that your internet connection is active...
 
واش فيه version جديدة؟ / Y a-t-il une nouvelle version?
→ Check the Updates section in your dashboard...
```
 
This ensures even a weaker embedding model can match queries because the vocabulary overlaps directly with the docs.
 
#### Strategy 3: Switch to bge-m3 (Best Local Option)
 
```bash
ollama pull bge-m3
```
 
`bge-m3` is trained on 100+ languages with strong cross-lingual alignment. It handles mixed-language sentences better than any other locally available model.
 
#### Combined Recommended Setup for North Africa
 
```
1. Pull bge-m3:        ollama pull bge-m3
2. Update Ollama Embeddings node → model: bge-m3:latest
3. Add the system prompt above to your QA Chain
4. Add Darija/French mixed FAQ to your Document Store
5. Delete old index:   rm /root/.flowise/vectorstore/faiss.*
6. Re-upsert
```
 
---
 
### ❌ Bot Says "I am not sure" in Other Languages
 
This is a retrieval failure, not an LLM failure. Debug it in order:
 
**1. Check your system prompt is set** — the LLM needs permission to answer from its own knowledge when context is thin.
 
**2. Check your embedding model** — if you're using `nomic-embed-text` with non-English queries, switch to `bge-m3`.
 
**3. Verify the index exists after re-upserting:**
```bash
ls /root/.flowise/vectorstore/
# Must show: faiss.index  faiss.json
```
 
**4. Add multilingual content to your docs** — even a few translated FAQ entries dramatically improve retrieval for those languages.

---

## Troubleshooting

### ❌ `fetch failed` when chatting

**Cause:** Flowise cannot reach Ollama.

**Fix:**
```bash
# Verify Ollama is reachable
curl http://localhost:11434/api/tags

# Check Ollama is running
systemctl status ollama

# Check model name is correct (must use colon, not dash)
# ✅ qwen2.5:3b
# ❌ qwen2.5-3b
ollama list
```

---

### ❌ `Cannot read properties of undefined (reading 'startsWith')`

**Cause:** The **Base Path to load** field in the Faiss node is empty.

**Fix:**
1. Click the Faiss node
2. Fill in Base Path to load: `/root/.flowise/vectorstore`
3. Create the directory: `mkdir -p /root/.flowise/vectorstore`
4. Re-upsert

---

### ❌ `could not open faiss.index for reading`

**Cause:** Upsert was never completed successfully — the index file doesn't exist.

**Fix:**
```bash
# Check if the file exists
ls /root/.flowise/vectorstore/

# If empty, fix the Base Path issue above, then Upsert again
```

---

### ❌ Ollama model not found

**Cause:** Model name is wrong or model was never pulled.

**Fix:**
```bash
# List available models
ollama list

# Pull the missing model
ollama pull qwen2.5:3b
ollama pull nomic-embed-text:latest

# Always use colon format in Flowise: qwen2.5:3b not qwen2.5-3b
```

---

### ❌ Flowise crashes or won't start

**Fix:**
```bash
# Check Node.js version (must be 18+)
node -v

# Clear Flowise cache
rm -rf ~/.flowise/cache

# Check port 3000 is not already in use
sudo lsof -i :3000

# Restart with PM2
pm2 restart flowise
pm2 logs flowise
```

---

### ❌ Out of memory / model too slow

**Fix:**

```bash
# Check available RAM
free -h

# Use a smaller model
ollama pull phi3:mini   # 3.8B — lighter than qwen2.5:3b

# Check what's loaded in Ollama
ollama ps
```

If RAM is under 8GB, use `phi3:mini` or `qwen2.5:3b` (both under 4GB).

---

### ❌ Bot answers incorrectly or doesn't use documents

**Cause:** Documents are not properly indexed, or chunks are too small/large.

**Fix:**
1. Make sure you clicked **Upsert** after adding documents
2. Verify `faiss.index` and `faiss.json` exist in the Base Path
3. Try increasing **Chunk Size** to `1500` and re-upsert
4. Make sure your documents are in plain text, not scanned images

---

### ❌ Users share each other's conversation history

**Cause:** No `sessionId` is being passed, or all users get the same ID.

**Fix:** Use the embed code with a unique `sessionId` per user (see [Session Management](#session-management)).

---

## Quick Reference

| Component | Purpose |
|---|---|
| **Ollama** | Runs LLMs locally |
| **nomic-embed-text** | Converts text to vectors for search |
| **Faiss** | Stores and searches vectors |
| **Document Store** | Manages your knowledge base files |
| **Recursive Text Splitter** | Breaks documents into searchable chunks |
| **Buffer Memory** | Remembers conversation history per user |
| **Conversational Retrieval QA Chain** | Ties everything together |

| Port | Service |
|---|---|
| `3000` | Flowise UI and API |
| `11434` | Ollama API |

---

*Built with Flowise + Ollama. All models run locally — no data leaves your server.*