Evaluation of the agents' behaviour quality in simulation: application to a driving simulator in virtual environment

This research work is conducted in the context of Multi-Agents Simulation. It focuses on the evaluatiion of the agents' ability to reproduce human behaviors. This research question has many application domains such as virtual reality, embodied conversational agents, immersive simulation, games... The dominant approach for the evaluation of the agents' behaviors is based on human judgement, through Social Science questionnaires. Few approaches are based on Artificial Intelligence and automatic data analysis at the microscopic scale. This is due to the semantic gap between low-level raw data used by such algorithms and high-level behavior, which requires complex processing. However, these two approaches can obtain different information on high-level behavior. We show in this thesis that these two types of information complete each other and that the evaluation can benefit from both approaches equally. We first present a semi-automatic method for evaluating the agents' behavior quality. It relies on the automatic extraction of behavior clusters. The proposed method combines the Artificial Intelligence approach and the Social Science approach. The first one consists in an observation of human's simulation logs in order to build clusters used as high-level behavior abstractions through indicators given by the experts. The second one evaluates users categories by an annotation of the exhibited behaviors. We then present an algorithm based on the aggregation of the agents to the humans clusters, considered as baseline behaviors. This algorithm allows us to compare agents to humans by studying the clusters composition in order to assess the capacities, the lacks, and the errors in the agent model. We provide metrics based on these cluster types and confidence rates for each cluster. We then make these behaviors explicite based on user categories.

• Record URL:
  • http://madis-externe.ifsttar.fr/exl-php/DOC00021696
• Authors:
  • DARTY, Kevin
• Publication Date: 2015

Language

• French

Media Info

• Media Type: Digital/other
• Pagination: 179 p

Subject/Index Terms

• TRT Terms: Behavior; Driving simulators; Evaluation; Simulation
• ITRD Terms: 9001: Comportement; 9020: Evaluation; 1680: Simulateur (conduite); 9103: Simulation

Filing Info

• Accession Number: 01688987
• Record Type: Publication
• Source Agency: Institut Francais des Sciences et Technologies des Transports, de l'Amenagement et des Reseaux (IFSTTAR)
• Files: ITRD
• Created Date: Dec 18 2018 10:20AM