Towards Standardized Experiments in Human Robot Interactions
International workshop held in conjunction with IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’15)
Official website of the conference
LATEST NEWS: The workshop proceedings and the presentations (under the “speakers” tab) are now available!
While the R&D community produces great amounts of scientific outputs on HRI, the results are scattered in a myriad of different approaches and ways of performing and testing the interaction; metrics which have been used include efficacy, effectiveness, users satisfaction, emotional impact and social components. The main consequence is that results are not comparable and benchmarking of the various approaches proposed is not possible. The community is still missing consensus tools to benchmark robot products (robot producer/industrial perspective) and robot applications (research/academic perspective). Modes are required for the standardized assessment of robot products and applications in use in terms of safety, performance, user experience, and ergonomics. The benefit of agreed approaches and methods to the assessment of HRI is the production of results, so called “normative” data in the standardization community, meaning that they have been formulated via wide consultation in an open and transparent manner.
In this way, the results become widely acceptable, and can be exploited for the creation of international quality norms and standards which in turn would mean measurable robot performances in terms of HRI. We would like to draw from a wide set of experts from the industry, academy and standardization to focus on the key areas of industrial, personal care and medical robots. Together, we will work on establishing benchmarking scenarios and identifying suitable metrics common to HRI in these central and related robotics domains. As a result we aim for providing metrics and scenarios for robot producers and HRI researchers to evaluate their robots and robot systems and setups on a comparable level. Reproducible and comparable results and interoperable systems should be a long-term goal will be a valuable contribution to our community. Within the WS, there will be paper presentations, posters and hands-on panel discussion sessions in the WS and we invite interested WS participants to present their inputs as appropriate.
Topics of interest include, but are not limited to:
- Defining metrics for benchmarking HRI
- Safety and physical interactions
- Performance indicators for experiments
- User-experience
- Ergonomics
- Defining standardized scenarios for HRI
- Industrial, medical and personal care robots
- Publicly available datasets for evaluation of HRI
- Robot autonomy levels
Important Dates
Submission: 15th July, 2015 24th August, 2015 (EXTENDED)
Notification: 29th July, 2015 7th September, 2015
Camera-ready: 26th August, 2015 21st September, 2015
Workshop: 28th September, 2015 We invite workshop participants to submit extended abstracts (up to 4 pages) in PDF and IEEE format. For templates and examples follow the link
http://ras.papercept.net/conferences/support/support.php.
The abstracts will be presented as posters in a dedicated slot during our workshop.
Alternatively, workshop participants can submit a poster only. Those posters will be presented in the same poster session as extended abstract posters.
To enable us a structured planning of the workshop day, please submit your poster together with a short abstract (less than one page).
Please submit your abstract to IROSstandardHRI@kontor46.eu cc:nicole.mirnig@sbg.ac.at All submission will be refereed by the organizing committee of the workshop.
- Paolo Barattini, Kontor 46, Italy – Update from previous workshops from the TG Standardization
- Gurvinder Virk, University of Gävle, Sweden – Update on international standardization projects and relevance in Europe [presentation in pdf]
- Sven Wachsmuth, University of Bielefeld, Germany – Standardization in Robocup@home [presentation in pdf]
- Björn Matthias, ABB – The Role of Collision Experiments in Safety Standardization and in the Characterization of Collaborative Robots, Systems and Applications [presentation in pdf]
- Claudia Pagliari, University of Edinburgh, UK – Taxonomies and operational definitions of interactive robot applications
- Andrea Bonarini, Politecnico di Milano – Benchmarking medical HRI, robot therapy for the disabled [presentation in pdf]
- Reinhard Lafrenz, TUM, Germany – Standards and Experiments in Echord++
- Stefan Profanter, Fortiss, Germany – Industrial HRI Experiments in the SMErobotics project [presentation in pdf]
- Hooman Samani, National Taipei University, Taiwan – Possible standard evaluation method for HRI [presentation in pdf]
- Laurence Devillers, Paris-Sorbonne University, France – Affective and social spoken dialog in robotics: evaluation of user engagement
- Agnieszka Wykowska – Technische Universität München (TUM), Germany – The method and objective measures of social cognitive neuroscience for reliability of results in HRI research [presentation in pdf]
08:30 – 09:00 | Opening |
09:00 – 10:00 | Talks 1 |
10:00 – 10:30 | Coffee Break |
10:30 – 12:00 | Hands-on, panel experiment drafting session |
12:00 – 12:30 | Poster Session |
12:30 – 14:00 | Lunch Break |
14:00 – 15:30 | Talks 2 |
15:30 – 16:00 | Coffee Break |
16:00 – 17:30 | Hands-on, panel benchmarking session |
17:30 – 18:00 | Wrap-up |
18:00 | End of Workshop |
Center for Human-Computer Interaction
University of Salzburg, Austria
Sigmund-Haffner-Gasse 18
5020 Salzburg, Austria