How To Use Big Data to Inform Planning and Operations.

Transport for London (TfL) has the challenging task of carrying millions of passengers, every day, via its buses, underground tubes, over ground trains and tram links.

The organisation operates one of the world’s largest metropolitan telecoms networks so coping with the vast swathes of data generated by its 270 stations during peak hours is a mammoth task.

Speaking at Whitehall Media’s 4th biannual Big Data Analytics conference, the head of TfL’s Analytics division, Lauren Sager Weinstein, shared her insights about what data is gathered about customers, journey times, repairs and breakdowns as well as how all of it can be put to good use to inform key planning and operations decisions.

The Oyster card, which most travellers now use to pay for their regular journeys, is an instrumental tool to gather and extract essential information about TfL’s operations. With 3.5 million cards in use every day, the Oyster card provides a rich source of data. Not only does it allow oversight over where people enter and exit the TfL system but, as Weinstein, explains:

“We can [also] look at the journeys over a typical week. You can look at peak times, days of the week. You can understand which days are busier than other days and you can look at how people look at how people travel on the underground versus how they travel on the bus.”

The data is linked back to individuals for a period of eight weeks, before it is aggregated and held within summary tables where it is no longer traceable to any individual.

To harness the insights such big data sets can offer to a large organisation like the TfL, Weinstein maintains that data has to be collected for more than just data’s sake.

“We have to think about why we’re answering some of these questions. What are the business benefits we are going to get at TfL by looking at these data sets?”

In one example of a typical journey, Weinstein demonstrated to the conference how a typical journey’s median time can allow TfL to cross check that data with its service provisions, to monitor and plan for defective trains as well as signal and track failures.

Oyster based metrics therefore go beyond daily monitoring – and mere statistics – to allow for greater inferences to be made about effective and efficient planning and performance. Looking to the future, Weinstein adds that TfL could harness even better insights by using social media and weather data sets to gain a better understanding of customers and TfL operations.

Whitehall Media’s 5th bi-annual Big Data Analytics conference will take place at Hotel Russell, London on 19 June 2014. Visit for the latest conference agenda.