diff --git a/README.md b/README.md index a39d2e3..bb848ae 100644 --- a/README.md +++ b/README.md @@ -22,12 +22,17 @@ 8. [Upsert Documents](#upsert-documents) 10. [Using Firecrawl (Scrape Websites into RAG)](#using-firecrawl-scrape-websites-into-rag) 11. [Using Cloud LLM APIs (Optional)](#using-cloud-llm-apis-optional) -12. [Deploy Chatbot to Users](#deploy-chatbot-to-users) +12. [Multilingual Support & Embeddings](#multilingual-support--embeddings) + - [How Language Works in the Pipeline](#how-language-works-in-the-pipeline) + - [The Golden Rule of Embeddings](#the-golden-rule-of-embeddings) + - [Choosing an Embedding Model](#choosing-an-embedding-model) + - [North African & Middle Eastern Users](#north-african--middle-eastern-users) +13. [Deploy Chatbot to Users](#deploy-chatbot-to-users) - [Direct Link](#direct-link) - [Embed in Website](#embed-in-website) - [Session Management](#session-management) - [Rate Limiting](#rate-limiting) -13. [Troubleshooting](#troubleshooting) +14. [Troubleshooting](#troubleshooting) --- @@ -572,6 +577,166 @@ curl http://localhost:3002/v1/scrape \ Your self-hosted Firecrawl API is now at `http://localhost:3002` +--- + +## Multilingual Support & Embeddings + +### How Language Works in the Pipeline + +Your chatbot has two separate language-handling stages that behave differently: + +| Stage | Component | What it Does | +|---|---|---| +| **Retrieval** | Embedding model + Faiss | Finds relevant doc chunks | +| **Generation** | LLM (Gemini, Ollama, etc.) | Writes the answer | + +The LLM (Gemini 2.5 Flash, qwen2.5, etc.) is naturally multilingual — it will respond in whatever language the user writes in. The weak point is always **retrieval**: if the embedding model can't match a French or Arabic query to English docs, the LLM receives empty context and responds with "I am not sure." + +``` +User asks in Arabic + ↓ +Embedding model converts query to vector + ↓ +Faiss searches English docs → poor match → returns nothing + ↓ +LLM receives empty context → "Hmm, I am not sure" ❌ +``` + +--- + +### The Golden Rule of Embeddings + +> **The model used during Upsert MUST be the same model used during Chat.** + +Each embedding model has its own internal "language" for converting text to vectors. They are not compatible with each other. + +``` +Upsert with nomic-embed-text: +"product warranty" → [0.23, 0.87, 0.12, ...] + +Chat query with text-embedding-004: +"product warranty" → [0.91, 0.04, 0.67, ...] + +Faiss compares these → completely different → no match ❌ +``` + +If you switch embedding models you **must delete the old index and re-upsert**: + +```bash +rm /root/.flowise/vectorstore/faiss.index +rm /root/.flowise/vectorstore/faiss.json +# Then hit Upsert again in Flowise +``` + +--- + +### Choosing an Embedding Model + +| Model | Where | Arabic | French | Darija | Code-switching | +|---|---|---|---|---|---| +| `nomic-embed-text` | Ollama | ⚠️ Partial | ✅ Good | ❌ Poor | ❌ Poor | +| `mxbai-embed-large` | Ollama | ✅ Good | ✅ Good | ⚠️ Partial | ⚠️ Partial | +| `bge-m3` | Ollama | ✅ Excellent | ✅ Excellent | ⚠️ Partial | ✅ Best local | +| `text-embedding-004` | Google API | ✅ Good | ✅ Good | ⚠️ Partial | ⚠️ Partial | +| `text-embedding-3-small` | OpenAI API | ✅ Good | ✅ Good | ⚠️ Partial | ⚠️ Partial | +| `multilingual-e5-large` | Ollama | ✅ Excellent | ✅ Excellent | ⚠️ Partial | ✅ Best local | + +**Recommendation:** Switch to `bge-m3` for the best multilingual retrieval without any API cost: + +```bash +ollama pull bge-m3 +``` + +Then update the **Ollama Embeddings** node model name to `bge-m3:latest`, delete the old index, and re-upsert. + +--- + +### North African & Middle Eastern Users + +North African users (Algeria, Morocco, Tunisia) often write in **code-switched** messages mixing Arabic dialect (Darija) with French in the same sentence: + +``` +"كيفاش نdir la configuration?" +"le produit مايخدمش properly" +"comment تاع l'installation?" +``` + +This is called **code-switching** and no embedding model handles it perfectly — Darija is underrepresented in all training datasets. Use a combination of strategies: + +#### Strategy 1: System Prompt (Do This First) + +In **Conversational Retrieval QA Chain** → **Additional Parameters** → **System Message**: + +``` +You are a helpful product support assistant for North African users. +Users may write in Arabic (Modern Standard or Darija dialect), +French, or a mix of both languages in the same message. + +When you receive a message: +1. Understand it regardless of the language mix +2. Answer in the same language(s) the user wrote in +3. If the retrieved context is insufficient, use your general + knowledge to help the user as best as possible +4. Never respond with "I am not sure" without first attempting + to answer based on your knowledge +``` + +Your LLM (especially Gemini) understands Darija + French mixing very well — this prompt prevents it from giving up when retrieval returns weak results. + +#### Strategy 2: Add Mixed-Language Content to Your Docs + +Write a FAQ section in your Document Store using the way your users actually type: + +``` +كيفاش نinstalli le produit؟ / Comment installer le produit? +→ Go to Settings → Install → follow the steps... + +le produit مايخدمش / Le produit ne fonctionne pas +→ First check that your internet connection is active... + +واش فيه version جديدة؟ / Y a-t-il une nouvelle version? +→ Check the Updates section in your dashboard... +``` + +This ensures even a weaker embedding model can match queries because the vocabulary overlaps directly with the docs. + +#### Strategy 3: Switch to bge-m3 (Best Local Option) + +```bash +ollama pull bge-m3 +``` + +`bge-m3` is trained on 100+ languages with strong cross-lingual alignment. It handles mixed-language sentences better than any other locally available model. + +#### Combined Recommended Setup for North Africa + +``` +1. Pull bge-m3: ollama pull bge-m3 +2. Update Ollama Embeddings node → model: bge-m3:latest +3. Add the system prompt above to your QA Chain +4. Add Darija/French mixed FAQ to your Document Store +5. Delete old index: rm /root/.flowise/vectorstore/faiss.* +6. Re-upsert +``` + +--- + +### ❌ Bot Says "I am not sure" in Other Languages + +This is a retrieval failure, not an LLM failure. Debug it in order: + +**1. Check your system prompt is set** — the LLM needs permission to answer from its own knowledge when context is thin. + +**2. Check your embedding model** — if you're using `nomic-embed-text` with non-English queries, switch to `bge-m3`. + +**3. Verify the index exists after re-upserting:** +```bash +ls /root/.flowise/vectorstore/ +# Must show: faiss.index faiss.json +``` + +**4. Add multilingual content to your docs** — even a few translated FAQ entries dramatically improve retrieval for those languages. + --- ## Troubleshooting