TY - DATA
T1 - Enhancing Proof Assistant Error Messages with Hints: A User Study
PY - 2025/06/18
AU - Maria Khakimova
AU - Jesper Cockx
AU - Sára Juhošová
AU - Jaro Reinders
UR - 
DO - 10.4121/79e7c4eb-81dc-492a-9ac4-69f33166de8e.v1
KW - agda
KW - error messages
KW - user study
KW - proof assistants
KW - human-computer interaction
KW - dependent types
KW - computer science education
KW - novice programmers
KW - hints in error messages
N2 - <p>This repository contains the user study data accompanying the master thesis by Maria Khakimova titled "Enhancing Proof Assistant Error Messages with Hints: A User Study". The goal of the research was to investigate the impacts of hint-based error message enhancements in Agda on novice programmers. To do this, we enhanced three error messages with hints, and conducted a user study.</p><p><br></p><p>In the user study, we asked participants to resolve errors in pre-written Agda code, and rate the helpfulness of the error message. We collected the following data:</p><ul><li>code compilation status (success/fail),</li><li>compilation timestamps, and</li><li>responses to the "Did you find the error message helpful?" question (on a Likert scale).</li></ul><p><br></p><p>This repository contains the programming questions created for the user study, with the accompanying error messages (both original and enhanced) in <code>programming_exercises.zip</code>. We also provide the (anonymised) collected data in <code>JSON</code> format in <code>response-data.json</code>.</p><p><br></p><p>For more details, please read the provided <code>README</code>.</p>
ER -