26.3.12
An Empirical Investigation of the Influence of a Type of Side Effects on Program Comprehension
I've bemoaned before the lack of empirical research on programming language design. Here is one modest result, showing what you might expect, that side effects in expressions harm program comprehension. The sign of a good psychological experiment is surprise. While the result in this case is unsurprising, perhaps the observation of similar results for both inexperienced and experienced programmers is a bit more surprising.
This paper reports the results of a study on the impact of a type of side effect (SE) upon program comprehension. We applied a crossover design on different tests involving fragments of C code that include increment and decrement operators. Each test had an SE version and a side-effect-free (SEF) counterpart. The variables measured in the treatments were the number of correct answers and the time spent in answering. The results show that the side-effect operators considered significantly reduce performance in comprehension-related tasks, providing empirical justification for the belief that side effects are harmful.
Comments:
<< Home
It's good to see some empirical research on the topic, but it's a bit misleading to state that experienced programmers had difficulty similar to that of the inexperienced programmers. While the data in the paper clearly show that this was the case, it is probable that the experienced programmers did not have a lot of experience with code snippets like
"if (++i!=0) x=i++; else x=--i;"
. Such code fragments are generally considered terrible form and so aren't as common in the wild. This is arguably due to the fact that such fragments are side-effecting and hard to understand, but indirectly so; by and large, we recognized this problem decades ago and started stigmatizing that kind of code, which means that we don't write (or read) it much today.
But three cheers for some rigorous results. More please! ;)
"if (++i!=0) x=i++; else x=--i;"
. Such code fragments are generally considered terrible form and so aren't as common in the wild. This is arguably due to the fact that such fragments are side-effecting and hard to understand, but indirectly so; by and large, we recognized this problem decades ago and started stigmatizing that kind of code, which means that we don't write (or read) it much today.
But three cheers for some rigorous results. More please! ;)
# posted by 
: 26/3/12 7:17 PM
: 26/3/12 7:17 PM
Phil, as much as I agree that we need such research, I will also say that this kind of research is questionable. For every paper that tells us about the difficulties of comprehending i++ and i = i +1, I am sure you will be able to find a dozen papers on the difficulties of designing and comprehending recursive functions. Or higher-order functions. Or combinators and other forms of point-free programming. Let's think how we can do better -- Matthias [I am not reading your blog, someone pointed me there]
Matthias, You and I agree that a paper of this sort does little to resolve the perennial dispute between functional and imperative programming. Please send references to empirical studies on recursive, higher-order, and point-free programming. I'd love to see them.
Post a Comment
<< Home
