I returned, and saw under the sun, that the race
is not to the swift, nor the battle to the strong, neither yet bread to the
wise, nor yet riches to men of understanding, nor yet favour to men of skill;
but time and chance happeneth to them all.
Ecclesiastes 9:11
Yes, they have
more readers
Eye-opening results of the Great prose or not? quiz
By Mikhail Simkin
Let me tell you about the very rich.
They are different from you and me.
F. Scott Fitzgerald
Yes, they have more money.
Ernest Hemingway
Are the very famous writers different
from the obscure ones?
Edward Bulwer-Lytton is the worst writer in history of letters. An
annual wretched writing contest was
established in his honor. In contrast, Charles Dickens is one of the best
writers ever. Can one tell the difference between their prose? To check this I
wrote the Great prose or not? quiz. It consists of a dozen
of representative literary passages, written either by Bulwer-Lytton or by
Dickens, and the takers are to choose the author of each quote.
The
distribution of the scores received by over three thousands quiz-takers is
shown in Figure 1.The average score is 5.74 or 48% correct. Due to the large
number of quiz-takers the standard error of this average is small: 0.035 or
0.3%.
Figure 1. The histogram of the scores earned by 3,643 people on
the “Great prose or not?” quiz. The average score is 5.74 or 48% correct. The
standard error of this average is 0.035 or 0.3%.
There are two possible answers to
each test question. If one is completely clueless and resorts to random
guessing, he will on average get 50% of the questions right. With the average
score of 48% our quiz-takers lost to a monkey.
On average, a quote from
Bulwer-Litton was selected as Dickens (or great prose) by 52% of quiz-takers,
while a quote from Dickens was selected as Dickens by only 48%. Does this mean
that Bulwer-Lytton is a better writer than Dickens? Probably not. Table 1 shows
for every quote the fraction of people who attributed it to Dickens. This fraction varies between the quotes with
the lowest being 36% (No. 9) and the highest 74% (No. 12). This suggests that a
different selection of quotes could lead to a different average score. For
example, if we remove the most Dickensian Bulwer (No. 12) and the most
Bulwerian Dickens (No. 10), and recalculate the scores based on 10 remaining
questions, - the average score becomes 51%.
An interesting thing is that out of
3,643 people 10 got every question wrong and 11 got everything right. The
approximate equality of these numbers is consistent with random guessing, but
their magnitude is not. It is more than ten times bigger than what random
guessing would give. The explanation is that some of quiz takers can sniff
stylistic similarities between certain literary passages and attribute them to
the same writer. This would help them to get a higher score, provided that they
can determine if this writer is good or bad, otherwise they are equally likely
to get a very high or a very low score.
Table 1. Fraction of people who attributed each quote to Dickens and
Bulwer-Lytton, along with the true author.
|
Question
number
|
The real author, and the book the quote is taken from
|
Fraction of quiz-takers who selected the quote as
|
|
Dickens
|
Bulwer-Lytton
|
|
1
|
Charles Dickens, Great
Expectations
|
40%
|
60%
|
|
2
|
Edward Bulwer-Lytton, Eugene Aram
|
50%
|
50%
|
|
3
|
Charles Dickens, Great
Expectations
|
54%
|
46%
|
|
4
|
Edward Bulwer-Lytton,
Eugene Aram
|
50%
|
50%
|
|
5
|
Charles Dickens, David
Copperfield
|
52%
|
48%
|
|
6
|
Edward Bulwer-Lytton, Eugene Aram
|
51%
|
49%
|
|
7
|
Charles Dickens, Great
Expectations
|
59%
|
41%
|
|
8
|
Edward Bulwer-Lytton, Paul Clifford
|
50%
|
50%
|
|
9
|
Edward Bulwer-Lytton, Eugene Aram
|
36%
|
64%
|
|
10
|
Charles Dickens, Great
Expectations
|
39%
|
61%
|
|
11
|
Charles Dickens, David
Copperfield
|
41%
|
59%
|
|
12
|
Edward Bulwer-Lytton, Paul Clifford
|
74%
|
26%
|
The performance of our quiz-takers is
bad. But could this be because they don’t know English? The testing was done
over the Internet and people from all countries could participate. Fortunately, the quizzing script records
taker’s IP address. From it one can infer where their computers were located. I
selected a subset of scores, which were received by people coming from
American, British, Australian, and New Zealandian universities. The histogram
of the scores received by 277 of such people is shown in Figure 2. The average
score is 5.62 or 47% correct. The standard error of this average is 0.14 or 1%.
Educated English-speaking folks lost to the general public, whose average score
is 48%. The difference between the scores is, however, statistically
insignificant, because it does not exceed the standard error.
Figure 2. The histogram of the test scores earned by 277 people,
coming from American, British, Australian, and New Zealandian universities. The
average score is 5.62 or 47% correct. The standard error of this average is
0.14 or 1%.
But, perhaps, just knowing English is
not enough? May be the beauty of Dickens’ prose is beyond the apprehensions of
the vulgar and only the most cultured people can appreciate it? To check this I selected a subset of scores,
earned by people coming from the elite universities. Table 2 contains the statistics of the scores
received by 35 of such folks, coming from 10 elite schools. The average score
is 6 or 50% correct. The elite won over crowd by the whole 2%. The difference
between the elite and general scores is, however, statistically insignificant.
Due to the small size of the elite sample the standard error of the average
elite score is 4%.
Table 2. Statistics of the elite scores on “Dickens or
Bulwer-Lytton?” quiz. The average elite score is 6 or 50% correct. The standard
error of this average is 0.5 or 4%.
The results of the quiz show that
people can’t appreciate great prose when the name of a great writer is detached
from it. The answer to the question, we started with, is: Yes, they have more readers.
But how did they get those
readers? That I shall explain in one of
the future articles in the Ecclesiastes
9:11 series. To be posted of our future releases subscribe
to our newsletter
January 16, 2006
Discuss this
article in our Book Forum
Technical issues
When I looked at the quiz results I
noticed hundreds of cases when two or more scores came from the same IP address
within few minutes. In many of such cases the later score was 100%. This suggests that many people took several
shots at the quiz. To eliminate this cheating I cleaned the data by selecting
only the first score from each IP address. Afterwards I cleaned the data from
the results, where one or more questions were skipped.
American, British, Australian, and
New Zealandian universities were identified by Internet domains: edu, ac.uk, edu.au, and ac.nz.