diff --git a/README.md b/README.md index 51450ef..cb79db8 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,20 @@ # Product Detection & Classification Pipeline - + + + + + + +
+
+ Scene + 		+
+ Product Detection - YOLOv8n + 		+
+ Product Classification - EfficientNet B0 +
A two-stage computer vision pipeline that combines a fine-tuned **YOLOv8n** detector with a **EfficientNet** classifier to detect and identify products on retail shelves. ``` @@ -196,12 +211,22 @@ data/ Use any file manager, or a tool like [Label Studio](https://labelstud.io/) for faster annotation. +**Class Labeller** +
+ +
+ + --- ### Step 2 — Auto-label & Review > Use this step once you have a trained classifier. It replaces manual labelling almost entirely. +
+ +
+ Runs the full detect → crop → classify pipeline in one command, then opens a browser UI where you review, confirm, fix, or reject each prediction. ```bash @@ -427,6 +452,13 @@ python 2_train_classifier.py \ - Go to [kaggle.com/datasets](https://www.kaggle.com/datasets) → **New Dataset** - Upload the zip, give it a name (e.g. `my-product-crops`) - Wait for processing to complete + + *Note: Don't need to create New Dataset you just need to update it from [here](https://www.kaggle.com/datasets/triztechdata/product-dataset)* + +
+ +
+ 2. Create a new Kaggle Notebook: - Go to [kaggle.com/code](https://www.kaggle.com/code) → **New Notebook** @@ -438,7 +470,7 @@ python 2_train_classifier.py \ - It will appear at `/kaggle/input/my-product-crops/` 4. Enable the GPU: - - Right panel → **Session options → Accelerator → GPU T4 x1** + - Right panel → **Session options → Accelerator → GPU T4 x2** 5. Edit **Cell 2 — Configuration** only: ```python @@ -470,6 +502,10 @@ After training, download from `/kaggle/working/runs/classify/`: These two files are all you need to run `3_inference.py` and `api_server.py`. +
+ +
+ #### Option C — Fully automated via `kaggle_train.py` Run the entire Kaggle pipeline — upload, train, download — from a single command with no browser needed: @@ -780,3 +816,28 @@ balance_and_augment.py (equalise class counts, generate augmented copies) ``` Each iteration your classifier gets better, which means the auto-labelling in the next iteration needs less manual correction. + +--- + + + + + + + + + + +
+
+ Per-Class Accuracy + 		+
+ Class Distribution +
+
+ Confusion Matrix + 		+
+ Training Curves +
diff --git a/assets/Scene - Product Detection.jpeg b/assets/Scene - Product Detection.jpeg new file mode 100644 index 0000000..12c0d6f Binary files /dev/null and b/assets/Scene - Product Detection.jpeg differ diff --git a/assets/Scene.jpeg b/assets/Scene.jpeg new file mode 100644 index 0000000..edb3b58 Binary files /dev/null and b/assets/Scene.jpeg differ diff --git a/assets/Scene_frame000000.jpg b/assets/Scene_frame000000.jpg new file mode 100644 index 0000000..a80478a Binary files /dev/null and b/assets/Scene_frame000000.jpg differ diff --git a/assets/video/Auto Label.mp4 b/assets/video/Auto Label.mp4 new file mode 100644 index 0000000..eaa19c2 Binary files /dev/null and b/assets/video/Auto Label.mp4 differ diff --git a/assets/video/Labeling.mp4 b/assets/video/Labeling.mp4 new file mode 100644 index 0000000..19940c7 Binary files /dev/null and b/assets/video/Labeling.mp4 differ diff --git a/assets/video/Training Model.mp4 b/assets/video/Training Model.mp4 new file mode 100644 index 0000000..529a229 Binary files /dev/null and b/assets/video/Training Model.mp4 differ diff --git a/assets/video/Update Dataset.mp4 b/assets/video/Update Dataset.mp4 new file mode 100644 index 0000000..d0a6a37 Binary files /dev/null and b/assets/video/Update Dataset.mp4 differ diff --git a/detector/README.md b/detector/README.md new file mode 100644 index 0000000..278c589 --- /dev/null +++ b/detector/README.md @@ -0,0 +1,316 @@ +# Phase 1 — Shelf Product Detection with YOLOv8n +### Pipeline Documentation · SKU110K Dataset + +--- + +## Overview + +This document covers Phase 1 of a two-phase retail shelf analysis pipeline. The goal of Phase 1 is to **detect all products** visible on retail shelves using a fine-tuned YOLOv8n object detection model trained on the SKU110K dataset. The output (bounding boxes around each product) feeds directly into Phase 2, which classifies each detected product. + +``` +Shelf Image → [Phase 1: YOLOv8n Detection] → Bounding Boxes → [Phase 2: Classifier] → Product Labels +``` + +--- + +## Environment + +| Item | Details | +|---|---| +| Platform | Kaggle Notebooks (GPU) | +| GPU | Required (auto-selected with `batch=-1`) | +| Framework | Ultralytics YOLOv8 | +| Language | Python 3 | + +--- + +## Prerequisites + +### 1. Install Dependencies + +```bash +pip install -q ultralytics +``` + +This installs the Ultralytics library, which includes YOLOv8 and all required dependencies (PyTorch, OpenCV, etc.). + +--- + +## Dataset — SKU110K + +### What It Is + +SKU110K is the primary dataset used for training. It contains: + +- **11,762 shelf images** capturing dense retail environments +- **~1.2 million annotated products** (bounding boxes) +- Designed for dense object detection in challenging, cluttered shelf scenarios + +| Resource | Link | +|---|---| +| GitHub (paper + info) | https://github.com/eg4000/SKU110K_CVPR19 | +| Kaggle (download) | https://www.kaggle.com/datasets/thedatasith/sku110k-annotations | + +### Alternative Datasets (for future reference) + +If you need to augment or swap the dataset, consider: + +- **Grocery Store Dataset (Grozi-120)** — 120 grocery product classes +- **WebMarket** — web-scraped retail product images +- **RPC (Retail Product Checkout)** — checkout-counter product recognition + +--- + +## Step-by-Step Reproduction Guide + +### Step 1 — Locate the Dataset Folder + +After adding the SKU110K dataset to your Kaggle notebook, run the following to find and verify the dataset structure: + +```python +import os + +BASE_PATH = "/kaggle/input/datasets/thedatasith/sku110k-annotations" +DATASET_FOLDER = None + +# Find SKU110K_fixed folder dynamically +for item in os.listdir(BASE_PATH): + if "SKU110K" in item: + DATASET_FOLDER = os.path.join(BASE_PATH, item) + break + +print("📁 Dataset folder:", DATASET_FOLDER) + +# Print directory tree (2 levels deep) +for root, dirs, files in os.walk(DATASET_FOLDER): + level = root.replace(DATASET_FOLDER, '').count(os.sep) + indent = ' ' * 2 * level + print(f"{indent}📁 {os.path.basename(root)}/") + for f in files[:5]: + print(f"{indent} 📄 {f}") + if level >= 2: + break +``` + +**Why:** The folder name inside the dataset archive contains `SKU110K` but may have a suffix (e.g., `SKU110K_fixed`), so we detect it dynamically rather than hardcoding it. + +--- + +### Step 2 — Inspect the YAML Configuration + +The dataset comes with a `data_kaggle.yaml` file that defines paths and class names. Read it to verify its contents before making any changes: + +```python +import yaml + +YAML_PATH = BASE_PATH + "/data_kaggle.yaml" + +with open(YAML_PATH, "r") as f: + data = yaml.safe_load(f) + +print("Classes:", data.get("names")) +print("Number of classes:", len(data.get("names", []))) +print("Train path:", data.get("train")) +print("Val path:", data.get("val")) +``` + +--- + +### Step 3 — Fix the YAML Paths + +The paths in the original YAML file point to locations that don't match the Kaggle input filesystem. You must override them with the correct absolute paths before training: + +```python +import yaml + +YAML_PATH = BASE_PATH + "/data_kaggle.yaml" + +with open(YAML_PATH, "r") as f: + data = yaml.safe_load(f) + +# Override with correct Kaggle paths +data["train"] = DATASET_FOLDER + "/images/train" +data["val"] = DATASET_FOLDER + "/images/val" + +# Save the corrected YAML to the working directory +FIXED_YAML_PATH = "/kaggle/working/fixed_data.yaml" + +with open(FIXED_YAML_PATH, "w") as f: + yaml.dump(data, f) + +print("✅ Fixed YAML saved at:", FIXED_YAML_PATH) +print(data) +``` + +> **Important:** Always write the fixed YAML to `/kaggle/working/`, not back to the input directory (which is read-only on Kaggle). + +--- + +### Step 4 — Train YOLOv8n + +Verify GPU availability and launch training: + +```python +import torch +import numpy +from ultralytics import YOLO + +# Verify GPU +print("GPU:", torch.cuda.get_device_name(0) if torch.cuda.is_available() else "None") +print("CUDA Capability:", torch.cuda.get_device_capability(0)) + +# Load pretrained YOLOv8 nano model +model = YOLO('yolov8n.pt') + +# Train +model.train( + device='0', # GPU index + data=FIXED_YAML_PATH, # Path to the fixed dataset config + epochs=30, # Total training epochs + imgsz=800, # Input image size (px) + batch=-1, # Auto-select largest batch that fits in GPU memory + workers=4, # Dataloader worker threads + optimizer="AdamW", # Optimizer + lr0=0.002, # Initial learning rate + patience=10, # Early stopping: stop if no improvement for 10 epochs + save_period=5, # Save a checkpoint every 5 epochs + project="kaggle/working/runs", # Output directory + name="exp", # Experiment name (auto-increments: exp, exp2, ...) + save=True # Save final weights +) +``` + +#### Training Hyperparameters Explained + +| Parameter | Value | Rationale | +|---|---|---| +| `imgsz` | 800 | Higher resolution improves detection of small/dense products | +| `batch` | -1 | Auto-batch maximizes GPU utilization without OOM errors | +| `optimizer` | AdamW | Better generalization than SGD for this type of task | +| `lr0` | 0.002 | Starting learning rate | +| `patience` | 10 | Stops training early if validation loss plateaus | +| `save_period` | 5 | Checkpoint every 5 epochs for resumability | +| `epochs` | 30 | Total epochs (may stop earlier due to `patience`) | + +--- + +### Step 5 — Download Trained Weights + +After training completes, package the experiment output folder as a ZIP file and create a download link: + +```python +import shutil +from IPython.display import FileLink + +# Update this path to match your actual experiment folder name +folder_path = '/kaggle/working/runs/detect/kaggle/working/runs/exp-6' +zip_path = '/kaggle/working/exp-6.zip' + +shutil.make_archive('/kaggle/working/exp-6', 'zip', folder_path) +print("Weights saved to:", zip_path) + +FileLink(zip_path) +``` + +> **Note:** The experiment folder name auto-increments (e.g., `exp`, `exp-2`, `exp-6`). Check `/kaggle/working/runs/detect/` to find your folder name before running this cell. + +The ZIP will contain: +- `weights/best.pt` — best checkpoint (highest validation mAP) +- `weights/last.pt` — final checkpoint (used for resuming) +- Training curves, confusion matrix, and metrics logs + +--- + +### Step 6 — Resume an Interrupted Training Run + +If training is interrupted (e.g., Kaggle session timeout), you can resume from the last saved checkpoint. Upload `last.pt` as a Kaggle dataset input first, then: + +```python +from ultralytics import YOLO + +model = YOLO("/kaggle/input/weights/last.pt") +model.train(resume=True) +``` + +> `resume=True` automatically picks up the training configuration, optimizer state, and epoch counter from the checkpoint. + +--- + +## Output Files + +After a successful training run, the experiment folder contains: + +``` +exp/ +├── weights/ +│ ├── best.pt ← Use this for inference and Phase 2 input +│ └── last.pt ← Use this to resume training +├── results.csv ← Per-epoch metrics (loss, mAP, precision, recall) +├── results.png ← Training curves plot +├── confusion_matrix.png +├── val_batch*.jpg ← Validation batch visualizations +└── args.yaml ← Full training config (for reproducibility) +``` + +--- + +## Using the Trained Model for Inference (Phase 2 Input) + +After training, use `best.pt` to detect products in new shelf images: + +```python +from ultralytics import YOLO + +model = YOLO("best.pt") +results = model.predict(source="shelf_image.jpg", conf=0.25, iou=0.45) + +for result in results: + boxes = result.boxes.xyxy # Bounding boxes [x1, y1, x2, y2] + confs = result.boxes.conf # Confidence scores + # Pass boxes to Phase 2 classifier +``` + +--- + +## Troubleshooting + +| Problem | Likely Cause | Fix | +|---|---|---| +| `DATASET_FOLDER` is `None` | Folder name doesn't contain "SKU110K" | Print `os.listdir(BASE_PATH)` and update the `if` condition | +| YAML path errors during training | Paths still point to original locations | Re-run Step 3 and confirm `FIXED_YAML_PATH` exists | +| CUDA out of memory | Batch size too large | Set `batch` to a fixed value (e.g., `8` or `16`) instead of `-1` | +| Training not resuming | Wrong path to `last.pt` | Upload `last.pt` as a Kaggle dataset, verify the input path | +| Experiment folder not found | Auto-increment changed the name | List `/kaggle/working/runs/detect/` to find the correct name | + +--- + +## Pipeline Context + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ Full Pipeline │ +│ │ +│ Input Image │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────┐ │ +│ │ PHASE 1 — Detection │ ← This notebook │ +│ │ YOLOv8n trained on SKU110K │ │ +│ │ Output: bounding boxes │ │ +│ └─────────────────────────────────┘ │ +│ │ │ +│ │ Cropped product regions │ +│ ▼ │ +│ ┌─────────────────────────────────┐ │ +│ │ PHASE 2 — Classification │ ← Next phase │ +│ │ Classify each detected product │ │ +│ │ Output: product labels │ │ +│ └─────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +Phase 1 produces bounding boxes for every product on the shelf. Each cropped region is then passed to the Phase 2 classifier for product identification. + +--- + +*Documentation generated from `shelf-products-detection-train-yolov8n.ipynb`* \ No newline at end of file