The influence of combination of response confidence scale with adaptive methods in threshold estimation experiments

Abstract

In a threshold estimation experiment, the threshold is commonly estimated by adaptive methods. These experiments are based on binary responses (Yes/No). However, there are some limitations of binary responses that could affect the results of experiments. To solve this problem, scholars have tried to include the role of response confidence in the experiment by extending the number of category of responses (Hsu & Chin, 2014; Kaernbach, 2001; Klein, 2001). Following this idea, in this thesis I used simulation to survey the influence of introducing a continuous response confidence scale to different adaptive methods (including the Weighted Up-Down method, Biased Coin Design, and Derman'’s Up-Down method) in two types of threshold experiments (Yes/No tasks and Same/Difference tasks). Moreover, I systematically manipulated the initial value, step size, trial number and other experiment variables to discuss their influences on adaptive methods. The simulation results show that the performance of the Weighted Up-Down method is better than the other two adaptive methods. Note that the placement of cut-offs on the response confidence scale might affect the performance of adaptive methods, and in some cases leads to the emergence of biases. We show that these biases can be reduced or even eliminated by adjusting some experiment variables. To summarize, the introduction of the response confidence scale can provide researchers with more freedom on experimental design, and there exists reasonable solutions to reduce the potential biases generated by this scale. 在一個閾值實驗裡閾值通常以適測方法估計；這些實驗是基於二選項回答（「是」/「否」），但二選項回答所帶來的一些限制會影響實驗結果。為解決此問題，學者試圖通過擴展回答選項個數的方式將回答信心包含進實驗裡（Hsu & Chin, 2014; Kaernbach, 2001; Klein, 2001）。跟隨該研究方向，在這篇論文裡，我採用電腦模擬探討在兩種實驗（「是」/「否」作業與「同」/「異」作業）情境中引入一個連續回答信心量尺對不同適測方法（適測方法包含帶權重上下法、偏差硬幣投擲法與德爾曼上下法）的影響。更進一步地，我經由系統性地操作初始值、步長、試驗數等實驗變量探討它們對適測方法的影響。模擬結果表明帶權重上下法整體比另兩個適測方法表現好。值得注意的是，回答信心量尺的截點位置會影響適測方法的表現，且在一些條件下會引發偏誤的出現。我們研究表明通過調整實驗變量可以減小甚至消除這些偏誤。綜上，在閾值實驗裡引入回答信心量尺可給予研究者在實驗設計階段更多的自由，其附帶偏誤亦存在合理解決方式以減小其影響。