Probe data that provide roadway speeds and travel times are increasingly being used for a variety of purposes in the transportation domain. A key use of these datasets has been roadway performance monitoring by state and local transportation agencies that are mandated to measure and report performance of their transportation networks. The San Francisco County Transportation Authority (SFCTA) monitors roadway performance as a part of the biennial Congestion Management Program (CMP) and primarily uses probe-based speed data for that purpose. Despite considerable savings in time and effort for data collection, integrating and processing the probe data still required a significant amount of manual work. This study highlights these challenges and proposes a data processing pipeline which includes an automated network conflation process, an efficient large data processing framework, and an interactive web-based visualization. In addition, all the scripts and code developed were made open source and are readily accessible from a public repository on GitHub. The value of the pipeline is demonstrated through the development of web-based interactive maps to monitor both long-term and short-term congestion in San Francisco. The short-term congestion monitoring application is timely given the spread of the COVID-19 pandemic and the region’s rapidly changing traffic conditions. Several valuable lessons learned from use of probe data for roadway performance monitoring are shared. Developing tools to ensure consistency of the data product and to reduce reliance on any one data vendor is of key importance.
|Title of host publication||Transportation Research Record|
|Publisher||SAGE Publications Ltd|
|Number of pages||13|
|State||Published - Jun 2022|
|Name||Transportation Research Record|
Bibliographical noteFunding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: San Francisco’s Congestion Management Program (CMP) is funded by STP (Surface Transportation Program, Federal) and Prop K grants.
© National Academy of Sciences: Transportation Research Board 2022.
- Analytic data visualization
- Data analytics
- Data and data science
- Data visualization
- Geospatial data
- Geospatial data visualization
- Including big data
- Information systems and technology
- Interactive visualization
- National and state transportation data and information systems
- Speed data
- Visualization in transportation
ASJC Scopus subject areas
- Civil and Structural Engineering
- Mechanical Engineering