Using Probe-Based Speed Data and Interactive Maps for Long-Term and COVID-Era Congestion Monitoring in San Francisco

Bhargava Sana, Xu Zhang, Joe Castiglione, Mei Chen, Gregory D. Erhardt

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review


Probe data that provide roadway speeds and travel times are increasingly being used for a variety of purposes in the transportation domain. A key use of these datasets has been roadway performance monitoring by state and local transportation agencies that are mandated to measure and report performance of their transportation networks. The San Francisco County Transportation Authority (SFCTA) monitors roadway performance as a part of the biennial Congestion Management Program (CMP) and primarily uses probe-based speed data for that purpose. Despite considerable savings in time and effort for data collection, integrating and processing the probe data still required a significant amount of manual work. This study highlights these challenges and proposes a data processing pipeline which includes an automated network conflation process, an efficient large data processing framework, and an interactive web-based visualization. In addition, all the scripts and code developed were made open source and are readily accessible from a public repository on GitHub. The value of the pipeline is demonstrated through the development of web-based interactive maps to monitor both long-term and short-term congestion in San Francisco. The short-term congestion monitoring application is timely given the spread of the COVID-19 pandemic and the region’s rapidly changing traffic conditions. Several valuable lessons learned from use of probe data for roadway performance monitoring are shared. Developing tools to ensure consistency of the data product and to reduce reliance on any one data vendor is of key importance.

Original languageEnglish
Title of host publicationTransportation Research Record
PublisherSAGE Publications Ltd
Number of pages13
StatePublished - Jun 2022

Bibliographical note

Publisher Copyright:
© National Academy of Sciences: Transportation Research Board 2022.


  • Analytic data visualization
  • Congestion
  • Data analytics
  • Data and data science
  • Data visualization
  • Geospatial data
  • Geospatial data visualization
  • Including big data
  • Information systems and technology
  • Interactive visualization
  • National and state transportation data and information systems
  • Speed data
  • Visualization in transportation

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Mechanical Engineering


Dive into the research topics of 'Using Probe-Based Speed Data and Interactive Maps for Long-Term and COVID-Era Congestion Monitoring in San Francisco'. Together they form a unique fingerprint.

Cite this