Recommendations for big data programs at transportation agencies

Big data that record mobility patterns have the potential to provide important insight to transportation agencies about how to better plan and operate the transportation system. Those agencies often already have access to valuable data assets, but faced with limited resources and a lack of experience working with emerging data sources and methods, those data may go under-utilized. In the case of real-time data feeds, the data may be lost altogether if they are not archived. This paper provides recommendations for planning and establishing big data programs at transportation agencies. These recommendations are derived from the lessons learned during the development of a big data fusion tool for measuring transportation system changes in San Francisco. It focuses on broad data management issues that staff at a public agency may face, such as identifying and prioritizing data sources and uses, managing privacy considerations and data sharing policies, and addressing data issues that may arise in contracting situations.