Generate mobility with models

Question. How can I generate realistic synthetic trajectories and flows, for testing methods, for privacy-safe sharing, or as a baseline to compare real data against?

You will get synthetic trajectories from six generators (three exploration-and-return variants, a diary-based one and two with social ties), a fitted and sampled gravity model, and a radiation model, all on one small synthetic city.

Prerequisites. Population measures. This notebook uses only synthetic data, so nothing is downloaded.

Data. None downloaded. The city is a synthetic 6 by 6 grid of 5 km cells with random relevance values (think of it as the number of points of interest per cell). Every result is synthetic; nothing here describes a real place.

%pip install -q 'fastmob>=0.3.4' pandas geopandas shapely statsmodels

A synthetic city

Generation models need a set of places with a relevance (how attractive each place is). Locations.from_tessellation turns a table of polygons (a tessellation) into the Locations catalogue the models take. We build a 6 by 6 grid with random relevance.

import warnings

import fastmob
import geopandas as gpd
import numpy as np
import pandas as pd
from fastmob import models
from shapely.geometry import box
from statsmodels.tools.sm_exceptions import SingularMatrixWarning

rng = np.random.default_rng(0)
n = 6
tessellation = gpd.GeoDataFrame(
    {
        "tile_id": [str(k) for k in range(n * n)],
        "relevance": rng.integers(10, 1000, n * n),
    },
    geometry=[
        box(
            116 + j * 0.05, 39.8 + i * 0.05, 116 + (j + 1) * 0.05, 39.8 + (i + 1) * 0.05
        )
        for i in range(n)
        for j in range(n)
    ],
    crs="EPSG:4326",
)
city = fastmob.Locations.from_tessellation(tessellation)
start, end = pd.Timestamp("2024-01-01"), pd.Timestamp("2024-01-15")
city.to_pandas().drop(columns="geometry").head(3)

The catalogue has 36 places with their relevance and centre coordinates. The relevance values come from a seeded random generator, so the example is reproducible.

Exploration and preferential return

The EPR family models people who explore new places and, with a probability that falls as they explore more, return to places they know. EPR and DensityEPR run the same exploration: the choice of a new place is weighted by a gravity score that combines the relevance of the places with the distance between them (you can pass your own fitted model as gravity_singly). DensityEPR is the name used in the d-EPR papers. SpatialEPR chooses by distance only and ignores relevance. rho, gamma and beta set the exploration and waiting behaviour. All return a TrajDataFrame. Fix random_state to reproduce a run.

epr = models.EPR(rho=0.6, gamma=0.21).generate(
    start, end, city, n_agents=50, random_state=1
)
spatial = models.SpatialEPR().generate(start, end, city, n_agents=50, random_state=1)
density = models.DensityEPR().generate(start, end, city, n_agents=50, random_state=1)

for name, traj in (("EPR", epr), ("SpatialEPR", spatial), ("DensityEPR", density)):
    d = traj.to_pandas()
    places = d.groupby("uid")[["lat", "lng"]].apply(lambda g: len(g.drop_duplicates()))
    print(
        f"{name:11s} {len(d):6d} points, {places.mean():.1f} distinct places per agent"
    )
epr.to_pandas().head()

EPR          10876 points, 24.9 distinct places per agent
SpatialEPR   10876 points, 28.3 distinct places per agent
DensityEPR   10876 points, 24.9 distinct places per agent

Fifty agents over two weeks make 10,876 points in all three models: the waiting-time mechanism and the seed are shared. SpatialEPR agents visit slightly more distinct places (28.3 against 24.9 on average). EPR and DensityEPR gave identical trajectories because they run the same algorithm; passing a fitted gravity_singly model changes both.

Realistic daily rhythm: diaries

MarkovDiaryGenerator learns when people are home or away from example trajectories: a 48-state Markov chain over the hour of the day (24 hours, each at home or away) and whether the person is at their most visited place. fit needs trajectories with a location id column. We train it on the EPR trajectories above, which is a stand-in for real data. generate produces a diary of abstract places. Ditras combines a fitted diary generator with the spatial model to produce trajectories with a realistic daily rhythm. diary_length=48 asks for 48 time slots, hourly here, so two days; consecutive slots at the same place are collapsed into one entry, so the diary has fewer entries than slots.

d = epr.to_pandas()
d["location"] = d.groupby(["lat", "lng"]).ngroup()

diary_model = models.MarkovDiaryGenerator(granularity_minutes=60)
diary_model.fit(d, n_individuals=50, lid="location")
diary = diary_model.generate(diary_length=48, start_date=pd.Timestamp("2024-03-01"))
ditras = models.Ditras(diary_model).generate(
    start, end, city, n_agents=50, random_state=2
)

print(len(diary), "diary entries;", len(ditras), "trajectory points from Ditras")
diary.to_pandas().head()

27 diary entries; 7856 trajectory points from Ditras

The diary is a sequence of timestamped abstract places: 0 stands for home and other numbers for other places in order of first appearance, so a diary says when a person is home or away but not where. Ditras fills in the places with the spatial model and produced 7,856 points for 50 agents over two weeks.

Social ties

GeoSim and STS_epr let agents influence each other through a social graph, so that friends share places. GeoSim is spatial and social; STS_epr adds the diary rhythm (it needs a fitted diary generator). With social_graph="random" the graph is generated randomly.

geosim = models.GeoSim().generate(
    start, pd.Timestamp("2024-01-08"), city, n_agents=30, random_state=3
)
sts = models.STS_epr().generate(
    start,
    pd.Timestamp("2024-01-08"),
    city,
    diary_generator=diary_model,
    n_agents=30,
    random_state=3,
)

print(len(geosim), "points from GeoSim,", len(sts), "from STS_epr")

1593 points from GeoSim, 2302 from STS_epr

Thirty agents over one week make 1,593 points with GeoSim and 2,302 with STS_epr.

Flows: gravity and radiation models

Instead of individual trajectories, the gravity and radiation models generate the flow between places directly from their relevance and the distances. The gravity model's trip counts fall with distance; the radiation model uses only populations and the intervening opportunities. Both need each place's total outflow (tot_outflow), here set proportional to relevance. Gravity.fit estimates the distance-decay exponent from observed flows; we use flows aggregated from a larger EPR run as the 'observations', and generate samples flows.

city_flows = tessellation.assign(tot_outflow=tessellation["relevance"] * 10)
city_out = fastmob.Locations.from_tessellation(city_flows)

observed = fastmob.TrajDataFrame(
    models.EPR().generate(start, end, city, n_agents=200, random_state=5).to_pandas()
).to_flowdataframe(tessellation, self_loops=False)
observed = fastmob.FlowDataFrame(observed.to_pandas(), locations=city)
gravity = models.Gravity(gravity_type="singly constrained")
with warnings.catch_warnings():
    # The origin indicator columns are collinear by construction (see below).
    warnings.simplefilter("ignore", SingularMatrixWarning)
    gravity.fit(observed, relevance_column="relevance")
gravity_flows = gravity.generate(city_out, out_format="flows", random_state=1)
radiation_flows = models.Radiation().generate(
    city_out, out_format="flows", random_state=1
)

print("fitted distance exponent:", np.round(gravity.deterrence_func_args, 2))
print(len(gravity_flows), "gravity flows;", len(radiation_flows), "radiation flows")
gravity_flows.to_pandas().nlargest(3, "flow")

fitted distance exponent: [-1.14]
1260 gravity flows; 1028 radiation flows

The fitted gravity model has a distance exponent of about -1.14: flows fall roughly with the distance to the power 1.14. statsmodels warns that the design matrix is rank-deficient; the cell above hides that warning. It is structural, not a sign of too little data: the singly constrained fit has an intercept and one indicator column per origin place, and these columns add up to the intercept, so the intercept and the origin effects are not separately identified. The distance exponent is still estimated; the warning appears whatever the data. The gravity model generated 1,260 flows and the radiation model 1,028; the largest gravity flow, about 1,360 trips, links two neighbouring cells.

Interpretation and limits

• These are models: they reproduce features they were built to reproduce (heavy-tailed jumps, return to favourite places, daily rhythm) and nothing else. Always compare the output with real data on the measures you care about, using the metrics in Fit and compare distributions.
• The city is random. The relevance values, the distances and the 'observed' flows used to fit the gravity model are synthetic, so the fitted exponent describes the toy city, not a real one.
• The diary model was trained on trajectories that came from another model, which is circular. Train it on real trajectories with at least a few weeks per person.
• GeoSim and STS_epr with a random social graph do not reproduce real social structure; pass your own graph to study it.

Next steps

• Compare generated and observed trajectories or flows: Fit and compare distributions.
• Check how identifiable generated trajectories are: Privacy risk.

Related API

EPR · SpatialEPR · DensityEPR · MarkovDiaryGenerator · Ditras · GeoSim · STS_epr · Gravity · Radiation · from_tessellation