Automatic Quality Control of Transportation Reports Using Statistical Language Processing
The processes of developing, monitoring, and maintaining transportation systems produce large volumes of information. Human fieldworkers are often responsible for gathering this information, and despite their best efforts, they will inevitably introduce errors into the collected data. This is a critical problem since: 1) the collected data are used to justify key infrastructure maintenance and development decisions; and 2) the volume of unstructured information (e.g., plain text) makes manual quality control prohibitively expensive. The authors introduce a solution to this problem in the example domain of vehicle accident reports. First, a sample of accident reports were analyzed and the existence of many data entry errors was confirmed. Second, a statistical language processing approach that automatically identifies reports containing data entry errors was developed and evaluated. The authors tested a variety of system configurations on real-world data and compared their performance with multiple baseline methods. The best configuration achieved a performance score of 84%, far outperforming the baseline methods. The results and analyses have quality control implications for any data source that pairs structured text (e.g., coded fields) with unstructured text.
- Record URL:
-
Availability:
- Find a library where document is available. Order URL: http://worldcat.org/oclc/41297384
-
Supplemental Notes:
- Abstract reprinted with permission of IEEE.
-
Authors:
- Gerber, Matthew S
- Tang, Lu
- Publication Date: 2013-12
Language
- English
Media Info
- Media Type: Digital/other
- Features: Figures; References; Tables;
- Pagination: pp 1681-1689
-
Serial:
- IEEE Transactions on Intelligent Transportation Systems
- Volume: 14
- Issue Number: 4
- Publisher: Institute of Electrical and Electronics Engineers (IEEE)
- ISSN: 1524-9050
- Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979
Subject/Index Terms
- TRT Terms: Data collection; Data mining; Methodology; Quality control; Reports; Statistical analysis
- Uncontrolled Terms: Natural language processing (Computer science)
- Subject Areas: Data and Information Technology; Highways; I72: Traffic and Transport Planning;
Filing Info
- Accession Number: 01527808
- Record Type: Publication
- Files: TLIB, TRIS
- Created Date: Jun 5 2014 9:08AM