Workshop @ IROS’15

Towards Standardized Experiments in Human Robot Interactions

International workshop held in conjunction with IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’15)

Official website of the conference

LATEST NEWS: The workshop proceedings and the presentations (under the “speakers” tab) are now available!

This workshop aims to advance the topic of standardization of robot experiments in Human-Robot Interaction (HRI) scenarios. The workshop follows up on previous workshops focusing on international robot standardization and benchmarking in the areas of industrial, medical, and personal care robots (ICRA 2013, HRI 2014, IROS 2014 and ERF 2015).

While the R&D community produces great amounts of scientific outputs on HRI, the results are scattered in a myriad of different approaches and ways of performing and testing the interaction; metrics which have been used include efficacy, effectiveness, users satisfaction, emotional impact and social components. The main consequence is that results are not comparable and benchmarking of the various approaches proposed is not possible. The community is still missing consensus tools to benchmark robot products (robot producer/industrial perspective) and robot applications (research/academic perspective). Modes are required for the standardized assessment of robot products and applications in use in terms of safety, performance, user experience, and ergonomics. The benefit of agreed approaches and methods to the assessment of HRI is the production of results, so called “normative” data in the standardization community, meaning that they have been formulated via wide consultation in an open and transparent manner.

In this way, the results become widely acceptable, and can be exploited for the creation of international quality norms and standards which in turn would mean measurable robot performances in terms of HRI. We would like to draw from a wide set of experts from the industry, academy and standardization to focus on the key areas of industrial, personal care and medical robots. Together, we will work on establishing benchmarking scenarios and identifying suitable metrics common to HRI  in these central and related robotics domains. As a result we aim for providing metrics and scenarios for robot producers and HRI researchers to evaluate their robots and robot systems and setups on a comparable level. Reproducible and comparable results and interoperable systems should be a long-term goal will be a valuable contribution to our community. Within the WS, there will be paper presentations, posters and hands-on panel discussion sessions in the WS and we invite interested WS participants to present their inputs as appropriate.

Topics of interest include, but are not limited to:

  • Defining metrics for benchmarking HRI
    • Safety and physical interactions
    • Performance indicators for experiments
    • User-experience
    • Ergonomics
  • Defining standardized scenarios for HRI
    • Industrial, medical and personal care robots
    • Publicly available datasets for evaluation of HRI
    • Robot autonomy levels

Important Dates

Submission: 15th July, 2015 24th August, 2015 (EXTENDED)

Notification: 29th July, 2015 7th September, 2015

Camera-ready: 26th August, 2015 21st September, 2015

Workshop: 28th September, 2015 We invite workshop participants to submit extended abstracts (up to 4 pages) in PDF and IEEE format. For templates and examples follow the link

http://ras.papercept.net/conferences/support/support.php.

The abstracts will be presented as posters in a dedicated slot during our workshop.

Alternatively, workshop participants can submit a poster only. Those posters will be presented in the same poster session as extended abstract posters.

To enable us a structured planning of the workshop day, please submit your poster together with a short abstract (less than one page).

Please submit your abstract to IROSstandardHRI@kontor46.eu  cc:nicole.mirnig@sbg.ac.at All submission will be refereed by the organizing committee of the workshop.

  1. Paolo Barattini, Kontor 46, Italy – Update from previous workshops from the TG Standardization
  2. Gurvinder Virk, University of Gävle, Sweden – Update on international standardization projects and relevance in Europe [presentation in pdf]
  3. Sven Wachsmuth, University of Bielefeld, Germany – Standardization in Robocup@home [presentation in pdf]
  4. Björn Matthias, ABB – The Role of Collision Experiments in Safety Standardization and in the Characterization of Collaborative Robots, Systems and Applications [presentation in pdf]
  5. Claudia Pagliari, University of Edinburgh, UK – Taxonomies and operational definitions of interactive robot applications
  6. Andrea Bonarini, Politecnico di Milano – Benchmarking medical HRI, robot therapy for the disabled [presentation in pdf]
  7. Reinhard Lafrenz, TUM, Germany – Standards and Experiments in Echord++
  8. Stefan Profanter, Fortiss, Germany – Industrial HRI Experiments in the SMErobotics project [presentation in pdf]
  9. Hooman Samani, National Taipei University, Taiwan – Possible standard evaluation method for HRI [presentation in pdf]
  10. Laurence Devillers, Paris-Sorbonne University, France – Affective and social spoken dialog in robotics: evaluation of user engagement
  11. Agnieszka Wykowska – Technische Universität München (TUM), Germany – The method and objective measures of social cognitive neuroscience for reliability of results in HRI research [presentation in pdf]
08:30 – 09:00 Opening
09:00 – 10:00 Talks 1
10:00 – 10:30 Coffee Break
10:30 – 12:00 Hands-on, panel experiment drafting session
12:00 – 12:30 Poster Session
12:30 – 14:00 Lunch Break
14:00 – 15:30 Talks 2
15:30 – 16:00 Coffee Break
16:00 – 17:30 Hands-on, panel benchmarking session
17:30 – 18:00 Wrap-up
18:00 End of Workshop
Nicole Mirnig – University of Salzburg, Austria Paolo Barattini – Kontor 46, Torino, Italy Dimitris Chrysostomou – Aalborg University, Denmark Lars Dalgaard – Danish Technological Institute, Denmark Maria Elena Giannaccini – University of Salford, Manchester, United Kingdom Manuel Giuliani, Center for Human-Computer Interaction, University of Salzburg, Austria Tamas Haidegger – Obuda University, Hungary Adriana Tapus, Robotics and Computer Vision Lab, ENSTA-ParisTech, Paris, France Gurvinder Singh Virk – University of Gävle, Sweden
Nicole Mirnig

Center for Human-Computer Interaction

University of Salzburg, Austria

Sigmund-Haffner-Gasse 18

5020 Salzburg, Austria

nicole DOT mirnig AT sbg DOT ac DOT at