Persistence and tracking: Putting vehicles and trajectories in context

City-scale tracking of all objects visible in a camera network or aerial video surveillance is an important tool in surveillance and traffic monitoring. We propose a framework for human guided tracking based on explicitly considering the context surrounding the urban multi-vehicle tracking problem. This framework is based on a standard (but state of the art) probabilistic tracking model. Our contribution is to explicitly detail where human annotation of the scene (e.g. "this is a lane"), a track (e.g. "this track is bad"), or a pair of tracks (e.g. "these two tracks are confused") can be naturally integrated within the probabilistic tracking framework. For an early prototype system, we offer results and examples from a dense urban traffic camera network tracking, querying data with thousands of vehicles over 30 minutes.