Preprocessing
| API | Description |
|---|---|
filter |
Trajectory filtering. |
compress |
Trajectory compression. |
stay_locations |
Stops detection. |
cluster |
Cluster stop locations. |
fastmob.preprocessing.filter(traj, max_speed_kmh=500.0, include_loops=False, speed_kmh=5.0, max_loop=6, ratio_max=0.25, *, datetime_col=None, lat_col=None, lng_col=None, uid_col=None, is_sorted=False)
Filter trajectory noise by removing high-speed outlier points.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
traj
|
Any
|
Trajectory dataframe; any Narwhals-compatible eager backend. |
required |
max_speed_kmh
|
float
|
Remove points where speed from the previous point exceeds this threshold. |
500.0
|
include_loops
|
bool
|
If True, also remove points forming short fast return loops. |
False
|
speed_kmh
|
float
|
Minimum loop speed threshold (km/h); only used when include_loops=True. |
5.0
|
max_loop
|
int
|
Maximum number of points to look ahead for loop detection. |
6
|
ratio_max
|
float
|
Distance ratio threshold for loop detection. |
0.25
|
datetime_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
lat_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
lng_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
uid_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
is_sorted
|
|
False
|
|
Setting
|
|
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Filtered trajectory in the same backend as input. |
Examples:
>>> import pandas as pd
>>> import fastmob
>>> url = fastmob.utils.constants.BRIGHTKITE_SAMPLE
>>> df = pd.read_csv(
... url,
... sep="\t",
... header=0,
... nrows=5000,
... names=["uid", "datetime", "lat", "lng", "location id"],
... )
>>> df["datetime"] = pd.to_datetime(df["datetime"], errors="coerce")
>>> df["location_id"] = df["location id"].astype("string")
>>> df = df.dropna(subset=["uid", "datetime", "lat", "lng"])[
... ["uid", "datetime", "lat", "lng", "location_id"]
... ]
>>> print(df.head().to_string(index=False))
uid datetime lat lng location_id
0 2010-10-16 06:02:04+00:00 39.891383 -105.070814 7a0f88982aa015062b95e3b4843f9ca2
0 2010-10-16 03:48:54+00:00 39.891077 -105.068532 dd7cd3d264c2d063832db506fba8bf79
0 2010-10-14 18:25:51+00:00 39.750469 -104.999073 9848afcc62e500a01cf6fbf24b797732f8963683
0 2010-10-14 00:21:47+00:00 39.752713 -104.996337 2ef143e12038c870038df53e0478cefc
0 2010-10-13 23:31:51+00:00 39.752508 -104.996637 424eb3dd143292f9e013efa00486c907
>>> from fastmob.preprocessing import filter
>>> filtered = filter(df, max_speed_kmh=500.0)
>>> print(len(filtered))
4898
>>> print(filtered.head().to_string(index=False))
uid datetime lat lng location_id
0 2009-05-25 20:56:10+00:00 37.774929 -122.419415 ee81ef22a22411ddb5e97f082c799f59
0 2009-05-25 21:35:28+00:00 37.600747 -122.382376 248b82709e6c11ddbf68003048c0801e
0 2009-05-25 21:37:44+00:00 37.600747 -122.382376 248b82709e6c11ddbf68003048c0801e
0 2009-05-25 21:42:47+00:00 37.600747 -122.382376 248b82709e6c11ddbf68003048c0801e
0 2009-05-25 22:13:23+00:00 37.615223 -122.389979 be2f1e669cc111dd9a50003048c0801e
References
- [Z2015] Zheng, Y. (2015) Trajectory data mining: an overview. ACM Transactions on Intelligent Systems and Technology 6(3), https://dl.acm.org/citation.cfm?id=2743025
fastmob.preprocessing.compress(traj, spatial_radius_km=0.2, *, datetime_col=None, lat_col=None, lng_col=None, uid_col=None, presorted=False)
Compress trajectory by collapsing nearby points into single representative points.
All points within spatial_radius_km from an initial point are collapsed into one point with median lat/lng and the initial point's timestamp.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
traj
|
Any
|
Trajectory dataframe; any Narwhals-compatible eager backend. |
required |
spatial_radius_km
|
float
|
Minimum distance (km) between consecutive output points. |
0.2
|
datetime_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
lat_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
lng_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
uid_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
presorted
|
Whether the trajectory is already sorted by user and time. When True, assumes the data is pre-cleaned (no null lat/lng/datetime rows). |
False
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
Compressed trajectory in the same backend as input. |
Examples:
>>> import pandas as pd
>>> import fastmob
>>> url = fastmob.utils.constants.BRIGHTKITE_SAMPLE
>>> df = pd.read_csv(
... url,
... sep="\t",
... header=0,
... nrows=5000,
... names=["uid", "datetime", "lat", "lng", "location id"],
... )
>>> df["datetime"] = pd.to_datetime(df["datetime"], errors="coerce")
>>> df["location_id"] = df["location id"].astype("string")
>>> df = df.dropna(subset=["uid", "datetime", "lat", "lng"])[
... ["uid", "datetime", "lat", "lng", "location_id"]
... ]
>>> print(df.head().to_string(index=False))
uid datetime lat lng location_id
0 2010-10-16 06:02:04+00:00 39.891383 -105.070814 7a0f88982aa015062b95e3b4843f9ca2
0 2010-10-16 03:48:54+00:00 39.891077 -105.068532 dd7cd3d264c2d063832db506fba8bf79
0 2010-10-14 18:25:51+00:00 39.750469 -104.999073 9848afcc62e500a01cf6fbf24b797732f8963683
0 2010-10-14 00:21:47+00:00 39.752713 -104.996337 2ef143e12038c870038df53e0478cefc
0 2010-10-13 23:31:51+00:00 39.752508 -104.996637 424eb3dd143292f9e013efa00486c907
>>> from fastmob.preprocessing import compress
>>> compressed = compress(df, spatial_radius_km=0.2)
>>> print(len(compressed))
3173
>>> print(compressed.head().to_string(index=False))
uid datetime lat lng location_id
0 2009-05-25 20:56:10+00:00 37.774929 -122.419415 ee81ef22a22411ddb5e97f082c799f59
0 2009-05-25 21:35:28+00:00 37.600747 -122.382376 248b82709e6c11ddbf68003048c0801e
0 2009-05-25 22:13:23+00:00 37.615223 -122.389979 be2f1e669cc111dd9a50003048c0801e
0 2009-05-26 02:21:12+00:00 39.878664 -104.682105 e12721ce84e911dd8019003048c0801e
0 2009-05-26 04:59:44+00:00 39.739154 -104.984703 ee8b1d0ea22411ddb074dbd65f1665cf
References
- [Z2015] Zheng, Y. (2015) Trajectory data mining: an overview. ACM Transactions on Intelligent Systems and Technology 6(3), https://dl.acm.org/citation.cfm?id=2743025
fastmob.preprocessing.stay_locations(traj, minutes_for_a_stop=20.0, spatial_radius_km=0.2, leaving_time=True, no_data_for_minutes=1000000000000.0, min_speed_kmh=None, *, datetime_col=None, lat_col=None, lng_col=None, uid_col=None, presorted=False)
Detect stay locations (stops) in trajectory data.
A stop is detected when the individual stays within spatial_radius_km for at least minutes_for_a_stop minutes. Stop coordinates are median lat/lng.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
traj
|
Any
|
Trajectory dataframe; any Narwhals-compatible eager backend. |
required |
minutes_for_a_stop
|
float
|
Minimum duration (minutes) to qualify as a stop. |
20.0
|
spatial_radius_km
|
float
|
Radius (km) within which points are grouped into a stop. |
0.2
|
leaving_time
|
bool
|
If True, add a 'leaving_datetime' column with the departure time. |
True
|
no_data_for_minutes
|
float
|
Gap threshold (minutes) above which data is treated as missing. |
1000000000000.0
|
min_speed_kmh
|
float | None
|
If set, trim trailing high-speed points from the end of each stop. |
None
|
datetime_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
lat_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
lng_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
uid_col
|
str | None
|
Explicit column name overrides; auto-detected when None. |
None
|
presorted
|
Whether the trajectory is already sorted by user and time. |
False
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
Stop locations in the same backend as input. Schema: [uid_col, lat_col, lng_col, datetime_col, (leaving_datetime)] |
Examples:
>>> import pandas as pd
>>> import fastmob
>>> url = fastmob.utils.constants.BRIGHTKITE_SAMPLE
>>> df = pd.read_csv(
... url,
... sep="\t",
... header=0,
... nrows=5000,
... names=["uid", "datetime", "lat", "lng", "location id"],
... )
>>> df["datetime"] = pd.to_datetime(df["datetime"], errors="coerce")
>>> df["location_id"] = df["location id"].astype("string")
>>> df = df.dropna(subset=["uid", "datetime", "lat", "lng"])[
... ["uid", "datetime", "lat", "lng", "location_id"]
... ]
>>> print(df.head().to_string(index=False))
uid datetime lat lng location_id
0 2010-10-16 06:02:04+00:00 39.891383 -105.070814 7a0f88982aa015062b95e3b4843f9ca2
0 2010-10-16 03:48:54+00:00 39.891077 -105.068532 dd7cd3d264c2d063832db506fba8bf79
0 2010-10-14 18:25:51+00:00 39.750469 -104.999073 9848afcc62e500a01cf6fbf24b797732f8963683
0 2010-10-14 00:21:47+00:00 39.752713 -104.996337 2ef143e12038c870038df53e0478cefc
0 2010-10-13 23:31:51+00:00 39.752508 -104.996637 424eb3dd143292f9e013efa00486c907
>>> from fastmob.preprocessing import stay_locations
>>> stops = stay_locations(df, spatial_radius_km=0.2, minutes_for_a_stop=20.0)
>>> print(len(stops))
3029
>>> print(stops.head().to_string(index=False))
lat lng datetime uid leaving_datetime
37.774929 -122.419415 2009-05-25 20:56:10 0 2009-05-25 21:35:28
37.600747 -122.382376 2009-05-25 21:35:28 0 2009-05-25 22:13:23
37.615223 -122.389979 2009-05-25 22:13:23 0 2009-05-26 02:21:12
39.878664 -104.682105 2009-05-26 02:21:12 0 2009-05-26 04:59:44
39.739154 -104.984703 2009-05-26 04:59:44 0 2009-05-26 16:43:59
References
- [RT2004] Ramaswamy, H. & Toyama, K. (2004) Project Lachesis: parsing and modeling location histories. In International Conference on Geographic Information Science, 106-124, http://kentarotoyama.com/papers/Hariharan_2004_Project_Lachesis.pdf
- [Z2015] Zheng, Y. (2015) Trajectory data mining: an overview. ACM Transactions on Intelligent Systems and Technology 6(3), https://dl.acm.org/citation.cfm?id=2743025
fastmob.preprocessing.cluster(traj, cluster_radius_km=0.1, min_samples=1, *, datetime_col=None, lat_col=None, lng_col=None, uid_col=None, presorted=False, n_jobs=None)
Cluster stop locations using DBSCAN with Haversine metric.