Skip to content

Data

API Description Example
list_datasets List datasets. Which datasets does the library list?
load_dataset Load dataset.
get_dataset_info Get dataset info. Which datasets does the library list?
load_geolife Load GPS trajectories of selected GeoLife users, downloading only what is needed. Load the data
load_brightkite Load Brightkite location check-ins, downloading only the rows requested. Load and clean
DatasetBuilder Base class for dataset-specific builders.
DatasetBuilder.prepare Turn the downloaded files (a list of paths) into the dataset object; implemented by each dataset builder.

Datasets

Name Type Description
foursquare_nyc trajectory Foursquare checkins of individuals in New York City.
flow_foursquare_nyc flow Movements between 4 km tiles in NYC derived from Foursquare data.
taxi_san_francisco trajectory GPS traces of ~500 taxis over 30 days in the San Francisco Bay Area. Requires auth.
parking_san_francisco auxiliar Parking meters in San Francisco.
nyc_boundaries shape NYC Borough Boundaries shapefile (water areas included).

API Reference

list_datasets(details=False, data_types=None)

List datasets.

List all the names of the datasets available in the data module of scikit-mobility.

Parameters:

Name Type Description Default
details bool

Whether to return full metadata for each dataset instead of the name only. The default is False.

 False
data_types str or list of str

Specify which dataset types to show. Accepted values: "trajectory", "flow", "shape", "auxiliar". The default is None (all types).

 None

Returns:

Type Description
list of str or dict

A list of dataset names, or a dict mapping name to metadata when details=True.

Examples:

>>> from fastmob.data import list_datasets
>>> list_datasets()
['flow_foursquare_nyc', 'foursquare_nyc', 'nyc_boundaries', 'parking_san_francisco', 'taxi_san_francisco']

Worked example: Which datasets does the library list?

load_dataset(name, drop_columns=False, auth=None, show_progress=False)

Load dataset.

Load one of the datasets that are present in the repository of scikit-mobility.

Parameters:

Name Type Description Default
name str

The name of the dataset to load (e.g., foursquare_nyc).

 required
drop_columns bool

Whether to keep additional columns when returning a TrajDataFrame. The default is False.

 False
auth tuple of str

Pair of strings (user, password) used when the dataset requires authentication. The default is None.

 None
show_progress bool

If True, show a progress bar during download. The default is False.

 False

Returns:

Type Description
TrajDataFrame or FlowDataFrame or GeoDataFrame or DataFrame

An object containing the downloaded dataset.

Examples:

>>> from fastmob.data import load_dataset, list_datasets
>>> tdf_nyc = load_dataset("foursquare_nyc", drop_columns=True)
>>> print(tdf_nyc.head())
   uid        lat        lng                  datetime
0  470  40.719810 -74.002581 2012-04-03 18:00:09+00:00
1  979  40.606800 -74.044170 2012-04-03 18:00:25+00:00
2   69  40.716162 -73.883070 2012-04-03 18:02:24+00:00
3  395  40.745164 -73.982519 2012-04-03 18:02:41+00:00
4   87  40.740104 -73.989658 2012-04-03 18:03:00+00:00

get_dataset_info(name)

Get dataset info.

Return the metadata stored in the JSON file associated with the dataset.

Parameters:

Name Type Description Default
name str

The name of the dataset to query (e.g., foursquare_nyc).

 required

Returns:

Type Description
dict

A dictionary containing the dataset metadata.

Examples:

>>> from fastmob.data import get_dataset_info
>>> get_dataset_info("foursquare_nyc")
{'name': 'Foursquare_NYC',
 'description': 'Dataset containing the Foursquare checkins of individuals moving in New York City',
 'url': 'http://www-public.it-sudparis.eu/~zhang_da/pub/dataset_tsmc2014.zip',
 'hash': 'cbe3fdab373d24b09b5fc53509c8958c77ff72b6c1a68589ce337d4f9a80235b',
 'auth': 'no',
 'data_type': 'trajectory',
 'download_format': 'zip',
 'sep': '\t',
 'encoding': 'ISO-8859-1'}

Worked example: Which datasets does the library list?

load_geolife(users=('000', '001', '002'), max_trips_per_user=25, cache=None)

Load GPS trajectories of selected GeoLife users, downloading only what is needed.

GeoLife is a public GPS dataset collected by Microsoft Research Asia (182 users, about 25 million fixes, mostly in Beijing, sampled every 1 to 5 seconds). The full archive is about 313 MB; this loader reads the requested users' .plt files straight from the archive with HTTP range requests, so a few users cost a few megabytes. Files are cached per combination of arguments, so repeated calls with the same arguments do not download again and a call never returns files cached for a different selection.

Parameters:

Name Type Description Default
users iterable of str or int

GeoLife user ids (three-digit codes such as "000"). None loads every user, which downloads the whole archive. The default is the first three users.

 ('000', '001', '002')
max_trips_per_user int

Keep only each user's first N trip files, in file-name order. None keeps all.

 25
cache str or path

Directory to cache downloads in. The default is a fastmob_data folder inside the system temporary directory.

 None

Returns:

Type Description
TrajDataFrame

Columns uid (string), lat/lng (degrees, WGS 84) and datetime (UTC, as recorded by GeoLife).

Raises:

Type Description
OSError

If the archive cannot be downloaded or does not support partial downloads. No substitute data is generated.

Examples:

>>> from fastmob.data import load_geolife
>>> tdf = load_geolife(users=["000"], max_trips_per_user=5)

Worked example: Load the data

load_brightkite(max_rows=300000, cache=None)

Load Brightkite location check-ins, downloading only the rows requested.

Brightkite (Stanford SNAP) holds about 4.7 million check-ins by 58 thousand users of a location-based social network, 2008 to 2010, worldwide. The compressed file is about 60 MB; the file is streamed and the download stops after max_rows rows, so the default 300,000 rows cost about 4 MB. Rows without coordinates are dropped. The first rows are not a random sample of users: they hold the earliest user ids' complete histories only up to the cut-off.

Parameters:

Name Type Description Default
max_rows int

Number of raw rows to read before dropping rows without coordinates. None reads the whole file (about 60 MB).

 300000
cache str or path

Directory to cache the rows in. The default is a fastmob_data folder inside the system temporary directory.

 None

Returns:

Type Description
TrajDataFrame

Columns uid (string), datetime (UTC), lat and lng (degrees).

Raises:

Type Description
OSError

If the file cannot be downloaded. No substitute data is generated.

Examples:

>>> from fastmob.data import load_brightkite
>>> tdf = load_brightkite(max_rows=50_000)

Worked example: Load and clean

DatasetBuilder

Bases: ABC

Base class for dataset-specific builders.

prepare(full_path_files) abstractmethod

Turn the downloaded files (a list of paths) into the dataset object; implemented by each dataset builder.