Update Pipline

Hamza 2026-03-29 09:06:22 +02:00
parent 166b47a8db
commit 31e968669a

@ -18,21 +18,21 @@ M --> N[Render in Leaflet]
```
# Why C/C++
What we're building here is a **GPS Trajectory Reconstruction and Stop-Detection Engine**, engines demand high performance code, it's a critical thing. We could've choose a different approach we could've decided to use Python and call Skitlearn library to use DBSCAN, then use OSRM as REST API service, or a different approach integrate DBSCAN into TrizDistribution using Java then use API calls for OSRM.
What we're building here is a **GPS Trajectory Reconstruction and Stop-Detection Engine**, engines demand high performance code, it's a critical thing. We could've choose a different approach we could've decided to use Python and call Sklearn library to use **DBSCAN**, then use **OSRM** as **REST API** service, or a different approach integrate **DBSCAN** into TrizDistribution using **Java** then use API calls for **OSRM**.
The Python approach one seems good, we don't need to implement DBSCAN ourselves, well imagine install with your current project an entire additional python interpreter to call an entire library like Skitlearn just for a sake of one algorithm, it's too much resources for a simple thing. Also because we didn't implement the algorithm ourselves we lose control over the implementation, so if you want to implement T-DBSCAN, Adaptive eps or any variants, you will either need to use to ask Sklearn to implement it for us or we will need to understand how Sklearn is implemented and do it ourselves.
The Python approach one seems good, we don't need to implement **DBSCAN** ourselves, well imagine install with your current project an entire additional python interpreter to call an entire library like Skitlearn just for a sake of one algorithm, it's too much resources for a simple thing. Also because we didn't implement the algorithm ourselves we lose control over the implementation, so if you want to implement **T-DBSCAN**, Adaptive eps or any variants, you will either need to use to ask **Sklearn** to implement it for us or we will need to understand how Sklearn is implemented and do it ourselves.
The Java approach seems good, but Java is not fast of language also due to everything is an object in Java it will cause the engine to run in slower speeds and the existence of Garbage Collector will make us loose performance, also if we want further optimizations like SIMD Java is not going to help, also we want from our engine to be separated from the project so we use the engine in other projects.
Using C/C++ will help us to ingrate OSRM easily as of it's built in C++, also our engine will take the form of an executable, which makes it portable. Using language like Java/Python will require the system to have JDK or JRE in case of Java, or Interpreter in case of Python, which unnecessary bloat.
Using **C/C++** will help us to ingrate **OSRM** easily as of it's built in **C++**, also our engine will take the form of an executable, which makes it portable. Using language like Java/Python will require the system to have JDK or JRE in case of Java, or Interpreter in case of Python, which unnecessary bloat.
# DBSCAN Clustering (Stop Detection)
Due to GPS drifts stops form clusters, this behavior can be used to detect when the truck stopped, and how long did it stop.
To extract this behavior a data clustering algorithm was used, unlike k-means DBSCAN does not require one to specify the number of clusters in the data a priori, also DBSCAN can find arbitrarily-shaped clusters and robust to outliers. Therefor DBSCAN was picked instead of other algorithms.
Due to **GPS** drifts stops form clusters, this behavior can be used to detect when the truck stopped, and how long did it stop.
To extract this behavior a data clustering algorithm was used, unlike **k-means** **DBSCAN** does not require one to specify the number of clusters in the data a priori, also DBSCAN can find arbitrarily-shaped clusters and robust to outliers. Therefor DBSCAN was picked instead of other algorithms.
### T-DBSCAN (Spatiotemporal Density Clustering)
Our implementation of DBSCAN is a bit different, what we implemented is T-DBSCAN where it clusters not just using x and y coordinates, but also adds the aspect of time.
Our implementation of **DBSCAN** is a bit different, what we implemented is **T-DBSCAN** where it clusters not just using **x** and **y** coordinates, but also adds the aspect of time.
Time is needed to separate trips, let me give you an example:
let's say we passed by a place **A** at **08:00AM**, and then we passed by the same place at **01:00PM**, now we passed by the same place but in different time, if we don't include time this may form a cluster and indicate a stop even though we didn't stop at that location.
@ -41,16 +41,21 @@ By adding the time aspect the algorithm becomes spatiotemporal, and will fix the
### Intuition (all together)
Two points belong to the same cluster only if:
- they are close in space (ε)
- close in time (Δt)
- and part of a dense neighborhood (MinPts)
- they are close in space **ε**
- close in time **Δt**
- and part of a dense neighborhood **MinPts**
### Parameter
- MinPts: Minimum number of points required to form a dense region (core point).
- ε: Maximum spatial distance between two points to be considered neighbors.
- Δt: Maximum time difference allowed between points for them to be considered neighbors in time.
- **MinPts:** Minimum number of points required to form a dense region (core point).
- **ε:** Maximum spatial distance between two points to be considered neighbors.
- **Δt:** Maximum time difference allowed between points for them to be considered neighbors in time.
### Optimization
The T-DBSCAN algorithm is optimized with KD-Tree, this will help find the nearest points faster, if further optimizations are needed then we should look into improving the **Cash Hits** and maybe change the algorithm to use **SIMD** and **Multi-Threading**.
The T-DBSCAN algorithm is optimized with **KD-Tree**, this will help find the nearest points faster, if further optimizations are needed then we should look into improving the **Cash Hits** and maybe change the algorithm to use **SIMD** and **Multi-Threading**.
# Douglas-Peucker Simplification
Ramer-Douglas-Peucker is an algorithm that decimates a curve composed of line segments to a similar curve with fewer points.
![RamerDouglasPeucker_algorithm](https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm#/media/File:RDP,_varying_epsilon.gif)
# Map Matching