**** Artificial Social Agent Questionnaire Instrument ****
Author: Siska Fitrianie, Merijn Bruijnes, Amal Abdulrahman, Willem-Paul Brinkman
Corresponding author: Siska Fitrianie, s.fitrianie@tudelft.nl
https://doi.org/10.4121/19650846

Interactive Intelligence Group
Department of Intelligent Systems
P.O. Box 5031,
2600 GA Delft,
The Netherlands

Description of the dataset
------------------------------------------------------------------------------------------------------------------------------------
The ASA_questionnaire_instrument.zip contains the output of the research into Siska Fitrianie, Merijn Bruijnes, Amal Abdulrahman, and Willem-Paul Brinkman. 2025. The Artificial Social Agent Questionnaire (ASAQ) - Development and evaluation of a validated instrument for capturing human interaction experiences with artificial social agents. International Journal of Human-Computer Studies (2025), 103482. https://doi.org/10.1016/j.ijhcs.2025.103482


The ASA Questionnaire is an instrument for evaluating human interaction with an artificial social agent (ASA), resulted from multi-year efforts involving more than 100 Intelligent Virtual Agent (IVA) researchers worldwide within the OSF work-group of Artificial Social Agent Evaluation Instrument (https://osf.io/6duf7/). It has 19 measurement constructs constituted by 90 items, which capture more than 80% of the constructs identified in empirical studies published in the IVA conference 2013-2018. In addition, a representative item of each construct or dimension was selected to create a 24-item short version of the ASA questionnaire. Whereas the long version is suitable for a comprehensive evaluation of human-ASA interaction, the short version allows quick analysis and description of the interaction with the ASA. To support reporting ASA questionnaire results, we also put forward ASAQ charts. The charts provide a quick overview of agent profile.


Abstract of the article Fitrianie et al. (2025)
------------------------------------------------------------------------------------------------------------------------------------
Validating claims and replicating findings on the impact of artificial social agents (ASA), such as virtual agents, conversational agents, and social robots, requires a standardised measurement instrument that researchers can employ in different settings and for various agents. Such an instrument would allow researchers to evaluate their agents and establish insights beyond their specific study context. Therefore, we present the long and short versions of the ASA questionnaire (ASAQ) for evaluating human-ASA interaction on 19 constructs, such as the agent’s believability, sociability, and coherence. It has been developed by an international workgroup with more than 100 ASA-researchers over multiple years who identified community-relevant constructs and associated questionnaire items and examined the questionnaire’s reliability, validity, and interpretability. The result is a questionnaire that can capture more than 80% of the constructs that studies in the intelligent virtual agent community investigate, with acceptable levels of reliability, content validity, construct validity, and cross-validity. We suggest that ASA-researchers use the ASAQ short version to report their agent’s psychographic information and the ASAQ long version to analyse any constructs in-depth that are specifically relevant to their agent or study. Finally, this paper gives instructions for practical use, such as sample size estimations, and how to interpret and present results.

Requirements
------------------------------------------------------------------------------------------------------------------------------------
1. Pdf reader
2. Internet browser
3. Excel 

Quick Instruction
------------------------------------------------------------------------------------------------------------------------------------
1. Unzip the ASAQ-instrument.zip
2. Use ASA-instrument.pdf to access the long and short versions of the ASA Questionnaire, also an introduction to the ASAQ Charts
3. Use the .html files in the folder 'ASAQ Chart Generator' to generate ASAQ Charts. The README.txt in this folder explains how to use the files.

Revision
------------------------------------------------------------------------------------------------------------------------------------
01/09/2023
- Fix typeO on ASA Questionnaire HLB2
- Fix typeO on ASA Questionnaire HLB3

01/09/2023
- Fix typeO on ASA Questionnaire UEP1
- Fix typeO on ASA Questionnaire HLB2

10/10/2025
- Change reference to cite and the authorships
- Replace ASA_questionnaire.pdf to the update version ASAQ-instrument.pdf, which includes an introduction to the ASAQ Percentile Chart
- Update the ASAQ chart generator codes.
