Population measures

Question. Where do people in a population live and gather, how far do they spread over time, and how many different people use each place?

You will get counts of homes and visits per place, check-in activity by day, a measure of how far people spread over time, and the diversity of visitors per place.

Prerequisites. Measure individual mobility.

Data. Brightkite check-ins (Stanford SNAP, 2008 to 2010): the first 300,000 rows (about 4 MB of the 60 MB file) with fastmob.data.load_brightkite; users with fewer than 100 check-ins are dropped, which leaves about 306 users. Check-ins are voluntary and sparse. The notebook stops if the download fails.

%pip install -q 'fastmob>=0.3.4' pandas

Load and clean

The raw file contains placeholder and impossible coordinates. We drop check-ins at exactly latitude 0 and longitude 0 (a placeholder for "unknown") and the few rows with a latitude whose absolute value is 90 degrees or more, which is not a valid position. Always check coordinate ranges before a population analysis.

import fastmob
from fastmob.data import load_brightkite
from fastmob.measures import collective

raw = load_brightkite(max_rows=300_000).to_pandas()

bad = ((raw["lat"] == 0) & (raw["lng"] == 0)) | (raw["lat"].abs() >= 90)
print(f"dropping {bad.sum()} check-ins with placeholder or invalid coordinates")
checkins = raw[~bad]
counts = checkins.groupby("uid").size()
checkins = (
    checkins[checkins["uid"].isin(counts[counts >= 100].index)]
    .sort_values(["uid", "datetime"])
    .reset_index(drop=True)
)
print(len(checkins), "check-ins by", checkins["uid"].nunique(), "users")

dropping 8327 check-ins with placeholder or invalid coordinates
283543 check-ins by 306 users

Homes and visits per place

homes_per_location counts the users whose inferred home is at each coordinate (home is the most visited place at night, in UTC here). visits_per_location counts visits per place id; we use H3 cells about 0.9 km across (resolution 8, edge about 0.46 km) as place ids. visits_per_time_unit counts check-ins per day.

homes = collective.homes_per_location(checkins)
cells = fastmob.latlng_to_h3(checkins, resolution=8).rename(
    columns={"h3_cell": "location_id"}
)
visits = collective.visits_per_location(
    cells, location_id_col="location_id", uid_col="uid", datetime_col="datetime"
)
per_day = collective.visits_per_time_unit(checkins, freq="1D")

print("most shared homes:")
print(homes.nlargest(3, "n_homes").to_string(index=False))
print("busiest places:")
print(visits.nlargest(3, "n_visits").to_string(index=False))
per_day.nlargest(3, "n_visits")

most shared homes:
      lat         lng  n_homes
39.739154 -104.984703       13
37.774929 -122.419415        8
37.580304 -122.343679        4
busiest places:
       location_id  n_visits
613167731774062591      5174
613196574496915455      3583
613166961024565247      2684

The most shared home coordinate is in central Denver (13 users), then central San Francisco (8). The busiest H3 cell has 5,174 check-ins. Check-in activity peaks on single days: 891 on 17 March 2009, the busiest day, which suggests an event or a surge in sign-ups rather than a daily rhythm.

How far do people spread over time?

mean_square_displacement is the average squared distance (km²) people have moved from their starting point after the chosen time (here 1 day), averaged over users. A value that grows linearly with the time lag is the signature of ordinary diffusion; faster or slower growth indicates other regimes.

msd = {
    d: collective.mean_square_displacement(checkins, days=d, hours=0) for d in (1, 7)
}
msd

{1: 915757.6763045556, 7: 658942.7669518194}

After one day the mean squared displacement is about 916,000 km² (a root-mean-square distance near 960 km), and after seven days about 659,000 km²: it does not grow with the time lag. Check-ins are sparse and a few users travel between continents, so the average rests on a small number of long moves. Treat it as a description of this sample, not as evidence of a diffusion regime.

How many different people use a place?

random_location_entropy is the entropy of the number of distinct visitors of each place (log2 of that number), and uncorrelated_location_entropy weights visitors by how often they visit. Both are 0 for places visited by one person and grow with the diversity of visitors.

random_h = collective.random_location_entropy(checkins)
uncorrelated_h = collective.uncorrelated_location_entropy(checkins)

print(
    len(random_h),
    "distinct coordinates;",
    f"{(random_h['random_entropy'] == 0).mean():.0%} have a single visitor",
)
random_h.nlargest(3, "random_entropy").merge(uncorrelated_h, on=["lat", "lng"])

43520 distinct coordinates; 90% have a single visitor

There are 43,520 distinct coordinates and 90% of them have a single visitor. The top coordinate (central Denver) has a random entropy of 6.4 bits, which corresponds to about 82 distinct visitors, and an uncorrelated entropy of only 4.5 bits: a few visitors account for most of the check-ins there.

Interpretation and limits

• Check-in data are not mobility data. People check in where and when they choose, so homes, entropies and spread describe check-in behaviour. The first 300,000 rows are not a random sample of users.
• Place ids come from a grid. Visit counts and entropy change with the grid size; state it, and test a second size.
• mean_square_displacement rests on a few long moves in this sample; it describes the sample, not a diffusion regime.

Next steps

• Count the people who move between places: Flows between places.
• Study when places are busy and use phone records: Occupancy and phone records.

Related API

homes_per_location · visits_per_location · visits_per_time_unit · mean_square_displacement · random_location_entropy · uncorrelated_location_entropy