How smart is your smart card? Evaluating transit smart card data with privacy restrictions and limited penetration rates

Transit smart card data can be analyzed for a number of planning applications, but not all smart card systems produce data of similarly high quality. The primary objective of this research is to evaluate the usefulness and validity of smart card data that are constrained by strong privacy protections and a limited penetration rate. In addition, a method is proposed to mitigate the biases inherent in the data. This analysis, done for the Clipper Card system in the San Francisco Bay Area of California, provides evidence for other agencies seeking to understand the value and limitations of their own data. The evaluation finds that the major limitation of the data is that a combination of the card technology, data coding, and privacy restrictions prevents the transaction location from being identified when the tag-on occurs on a vehicle instead of at a station. Several biases are identified in the users of Clipper Cards in terms of when the data are compared with external data sources, including automated passenger counters and onboard survey data. The onboard survey data are used to estimate a discrete choice model of Clipper Card use. The reciprocal of the modeled probability of using a Clipper Card is proposed as a correction factor. The proposed correction factor is found to mitigate, but not to eliminate fully, the biases in Clipper use. In spite of these limitations, the data are found to be valuable for certain applications, such as identifying transfers. Recommendations are provided for how the data can be improved.