Understanding big data requirements to improve data analytics program for transport engineering

New ways of collecting, managing and analysing voluminous data have converted transport engineering from a data hungry discipline to a data rich one. The data is coming from different sources such as smart cards, loop detectors and smartphones. These volumes of data, known also as "big data", have generated a strong interest among the transport research community. However, challenges regarding the management of big data have prevented researchers from harvesting the data to its full potential. The Transport Research Group of UQ has access to a huge volume of data, collected from sensors across South East Queensland on traffic counts, Bluetooth detectors, smart card transactions, bus vehicle location data, and transport incident reports. However, the characteristics of this data, in terms of its volume, velocity and variety make traditional data management approaches based on relational data bases inefficient and costly. As a result, the current practice in our research group is to slice the data and perform the analysis over a restricted time period. Moreover, each researcher repeats similar lengthy data preparation processes due to the complexities of exchanging data and results. To harness the available big data, we need to overcome these challenges by first understanding the specific needs of data stakeholders toward the management of the big data. To achieve this, we need to explore the requirements across all the stages of the big data lifecycle. These requirements will be the basis for selection and adoption of the appropriate big data technologies to be implemented. In this paper, we have performed a broad study on big data, the existing big data technologies, and the transport business requirements of big data technologies. This study can serve as a first-step guide for other research centres that face the challenges of managing their large scale data sets.


  • English

Media Info

  • Pagination: 12p
  • Monograph Title: 38th Australasian Transport Research Forum (ATRF 2016), Melbourne, 16th - 18th November 2016

Subject/Index Terms

Filing Info

  • Accession Number: 01627484
  • Record Type: Publication
  • Source Agency: ARRB
  • Files: ITRD, ATRI
  • Created Date: Feb 27 2017 10:14AM