Evaluation Measures

Compare two Trips or two FlowDataFrames through sparse Rust CPC.

Compare distance_km distributions in two Trips using Rust CPCD.

Compare active sparse OD links in two Trips or FlowDataFrames.

Compare value_col between two dataframes, optionally grouped by group_col.

metric defaults to :func:wasserstein_distance and accepts any (array, array) -> float callable, e.g. :func:jensen_shannon_divergence. When group_col is given, the comparison runs once per label present in both dataframes (sorted for determinism) and overall is the mean across groups.

Return the information gain (KL divergence variant) of true over pred.

where \(N = \sum_i y_i\).

Parameters:

Returns:

Examples:

>>> from fastmob import information_gain
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(information_gain(observed, predicted), 3))
0.005

Return Jensen-Shannon divergence between two distributions.

Inputs are normalised internally (raw counts and probabilities both work) and must have the same length -- unlike :func:wasserstein_distance, the two arrays are paired by index (one entry per category/bin), not independent samples.

Return the Kullback–Leibler divergence \(D_\mathrm{KL}(\mathrm{true}\,\|\,\mathrm{pred})\).

Inputs are normalized to probability distributions before evaluation, matching the historical scipy.stats.entropy(true, pred) behavior.

Parameters:

Returns:

Examples:

>>> from fastmob import kullback_leibler_divergence
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(kullback_leibler_divergence(observed, predicted), 3))
0.005

Return the maximum signed error max(true_i - pred_i).

Parameters:

Returns:

Examples:

>>> from fastmob import max_error
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(max_error(observed, predicted), 3))
3.0

Return the mean squared error between true and predicted values.

Parameters:

Returns:

Examples:

>>> from fastmob import mse
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(mse(observed, predicted), 3))
6.5

Return the normalized root mean squared error (RMSE / sum(true)).

Parameters:

Returns:

Examples:

>>> from fastmob import nrmse
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(nrmse(observed, predicted), 3))
0.025

Return the coefficient of determination R².

where

Parameters:

Returns:

Examples:

>>> from fastmob import r_squared
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(r_squared(observed, predicted), 3))
0.948

Return the root mean squared error between true and predicted values.

Parameters:

Returns:

Examples:

>>> from fastmob import rmse
>>> observed = [10, 20, 30, 40]
>>> predicted = [12, 18, 33, 37]
>>> print(round(rmse(observed, predicted), 3))
2.55

Compute the spatio-temporal Wasserstein distance between two distributions.

Each input is a DataFrame representing a spatial-temporal distribution with columns for a time bin (HH:MM string), a weight/volume value, and a WGS84 latitude/longitude centroid — the exact shape produced by :func:fastmob.measures.collective.build_stvd.

Distances are computed as an entropy-regularised (Sinkhorn) Earth Mover's Distance over an explicit cost matrix combining the great-circle (Haversine) distance between centroids with a cyclical-time term: two points at times t1/t2 contribute 2 * r * sin(d_theta / 2) metres, where r = alpha * cyclical_period / (2*pi), theta = 2*pi*(t / cyclical_period), and d_theta is the shortest angular difference between the two times.

Parameters:

Returns:

Examples:

>>> import pandas as pd
>>> from fastmob.measures.evaluation import stvd_emd
>>> dist_a = pd.DataFrame(
...     {
...         "time_bin": ["08:00", "08:10"],
...         "mean_volume": [2.0, 1.0],
...         "center_lat": [0.0, 0.0],
...         "center_lng": [0.0, 0.001],
...     }
... )
>>> dist_b = pd.DataFrame(
...     {
...         "time_bin": ["08:00", "08:10"],
...         "mean_volume": [1.0, 2.0],
...         "center_lat": [0.0, 0.0],
...         "center_lng": [0.0, 0.002],
...     }
... )
>>> print(round(stvd_emd(dist_a, dist_b), 3))
74.156

Return Rust-backed 1D Wasserstein distance between empirical samples.