Langenbucher, J., Labouvie, E., Morgenstern, J. (1996). Methodological evolution: measurement of the diagnostic agreement. Journal of Consulting and Clinical Psychology, 64, 1285-1289. It is clear that the approach to partial agreement at regular intervals is stricter than the overall census as a measure of the agreement between two observers. However, the most conservative approach to the IOA would be to overlook any discordance as a total disagreement during these intervals and to regard any disparity as null and void. A specific agreement is such an approach. Using this ratio, only specific agreements over an interval lead to this interval being estimated at 100% (or 1.0). The example of our race example would allow access to specific agreements for intervals of 5 to 14 or 10 of the 15 intervals. The division of 10 by the total number of intervals (15) gives an IOA of 66.7% – a slightly lower approval rate than the approach of the partial agreement at regular intervals. Test s.i.A. IOA. Savvy readers will find that IOA algorithms based on the above events are adapted to free-operator responses, responses that can occur at any time and are not anchored in events, but these measures do not explicitly take into account the experience-based reaction, which measures binary results (e.g.
B presence/non-presence, yes/no, on-task/task). Thus, the experimental IOA measures the number of trials with consent divided by the total number of trials. This metric is as strict as the exact approach to the agreement. Among all event-based IOA algorithms, analysis of the match between frequency counts and event records is usual. These measures consist of (a) the global census, (b) partial agreement at regular intervals, (c) a precise agreement and (d) IOA trial test algorithms. After a brief overview of the different event-based algorithms, Table 1 summarizes the strengths of the four event-based algorithms for behavioral reliability analysis considerations. Suppose a research team collects frequency data to respond to 15-1 m observations (see Figure 1). Average duration pro-deposits IOA. If the number of calendars is high, it is important to limit data aggregation in order to identify possible variations in the permanent data of two observers. The average duration IOA algorithm per deposit achieves this by determining an IOA score for each timing, and then by deifing them by the total number of timings in which the two observers collected data. Note that this approach is similar to the approach described above of partial agreement at regular intervals. In the example of Figure 3, there were 99.7, 2.3, 69.2 and 92.7% approval levels for intervals 1 to 4, respectively.
The average of these four levels of the agreement results in an average of 66% per event agreement – a much more conservative estimate than that of the statistics of the total duration of the IOA. Given that much of the articles published in the Journal of Applied Behavior Analysis depend on a single number/duration or intervals to measure reliability (see Mudford et al., 2009), some readers may be surprised to see that there are many formulas for the behavioural analyst to perform an IOA analysis. Because of the exceptional status that the field has put on Cooper,, Heron and Heward`s manual “Applied Behavior Analysis 2007”, as well as our own professional experience, as this book contains the most comprehensive discussion of the various IOA algorithms used in our field, our discussion of the IOA procedures below is mainly based on Chapter 5 of this text.