They Don't Read Well: The Reading Comprehension Skills of English Majors

A Profile of Participants and Preliminary Data-Gathering

Of the 85 undergraduate English majors in our study, 58 came from one Kansas regional university (KRU1) and 27 from another (and neighboring) one (KRU2). Both universities are similar in size and student population, and in 2015, incoming freshmen from both universities had an average ACT Reading score of 22.4 out of a possible 36 points, above the national ACT Reading score of 21.4 for that same year (ACT Profile 2015 9). All our subjects signed a consent form, which stated that their identities would remain anonymous and that their names would not be used in any presentation or publication. At KRU1, we gathered volunteers for our study from seven English classes; at KRU2, we set up our study outside the English Department and asked individual students to participate.

Prior to our test, each subject filled out a survey that asked for personal data and took the Degrees of Reading Power Test 10A, a national exam to establish a base tenth-grade literacy level. Almost all the student participants were Caucasian, two-thirds were female, and almost all had graduated from Kansas public high schools. All except three self-reported “A’s” and “B’s” in their English courses. The number of African-American and Latino subjects was too small a group to be statistically representative. [End Page 3] 35 percent of our study’s subjects were seniors, 34 percent were juniors, 19 percent were sophomores, and four percent were freshman, with the remaining eight percent of subjects unknown for this category. 41 percent of our subjects were English Education majors, and the rest were English majors with a traditional emphasis like Literature or Creative Writing

As for the results of the Degrees of Reading test, most of our subjects scored in the 85-to-100 percent range on the test, with a few scoring from 41 to 69 percent. The following is a breakdown of the reading test scores of all 85 subjects:

After the tests from Bleak House were analyzed, the test results were divided into three reading categories of reading abilities: problematic, competent, and proficient. We then compared the 10th-grade reading scores against the reading test results. All our subjects who scored 79 percent or less on the 10th-grade literacy test were also evaluated as problematic readers of Bleak House. However, 59 percent of the problematic readers scored a 90 or above on the literacy test, suggesting that the ability to read on a 10th-grade level does not ensure that students have the proficient-prose literacy skills to read complex texts.

Each taped reading test began with a brief questionnaire in which subjects were asked to give authors and titles of specific nineteenth-century American and British literary works and to explain briefly what they knew about nineteenth-century American and British history and culture. The purpose of these questions was to see how much literary and/or cultural knowledge the subjects possessed. According to Wolfgang Iser in The Act of Reading, one’s ability to read complex literature is partly dependent on one’s knowledge of what he calls the “repertoire” of the text, “the form of references to earlier works, or to social and historical norms, or to the whole culture from which the text has emerged” (69). With Bleak House, this knowledge is crucial.

The results from the questionnaire revealed that most of these subjects could not rely on previous knowledge to help them with Bleak House; in fact, they could not remember much of what they had studied in previous or current English classes. When we asked our subjects to name British and American authors and/or works of the nineteenth-century, 48 percent of those from KRU2 and 52 percent of those from KRU1 could recall at most only one author or title on their own. The majority also could not [End Page 4] access any detail on the information they recalled; they could mention the Industrial Revolution, for example, but could not define what it was. These results suggest that the majority of the subjects in our study were not transferring the literary texts or information from previous classes into their long-term memories.

Individual Think-Aloud Study

Our next step was the individual think-aloud study. Each subject was tested in a private, one-on-one taped 20-minute session with a facilitator. We made sure that the facilitators were not familiar with the specific subjects whom they were testing. For the KRU1 study, the facilitators were English graduate students; for the KRU2 study, the facilitators were English undergraduate students from KRU1. During the sessions, subjects were asked to read out loud and then translate each sentence of the passage from Bleak House. Subjects were encouraged to go at their own pace and were not required to finish the entire passage. Those who were uncomfortable reading out loud had the option to read silently.

Because we wanted to see how well students could read a complex text on their own, we told the facilitators not to help the subjects interpret the text. Instead, facilitators were there to record how subjects were understanding the material and to stop them every few sentences to request an interpretation. The taped recordings show that facilitators followed this training and politely refused any request for help from subjects. Facilitators also provided subjects with access to online resources and dictionaries and told them that they could also use their own cell phones as a resource. If subjects did go to Google or an outside website for help, the facilitator recorded that fact. At the end of each reading study, the facilitator asked each subject a brief series of questions on what the subject thought would happen next and their comfort level in reading the rest of the novel. These questions were designed to see how well our subjects understood the passage and how they perceived their own success with reading the text. All responded that they believed that they could read the rest of Bleak House with no problem.

After all the reading tests were completed, the taped read-aloud sessions were transcribed, coded, and organized into two categories: (1) narration (with 18 codes for activities like mispronunciation, skipping words, etc.) and (2) reading comprehension (with 62 codes of actions, including misunderstanding a metaphor or defining a figure of speech correctly). The data were then transferred to Excel sheets, where Dr. Ananda Jayawardhana, a mathematics professor at our university verified the final results. [End Page 5]

Major Findings and Reading Categories

Given the results, we placed the 85 subjects from both universities into three categories of readers: problematic, competent, and proficient. A summary of our major conclusions gives some basic data for our ensuing discussion:

* 58 percent (49 of 85 subjects) understood so little of the introduction to Bleak House that they would not be able to read the novel on their own. However, these same subjects (defined in the study as problematic readers) also believed they would have no problem reading the rest of the 900-page novel.

* Problematic readers often described their reading process as skimming and/or relying on SparkNotes.

* The majority of the 85 subjects used vague generalizations to summarize compound-complex sentences.

* 38 percent (or 32 of the 85 subjects) could understand more vocabulary and figures of speech than the problematic readers. These competent readers, however, could interpret only about half of the literal prose in the passage.

* Only 5 percent (4 of the 85 subjects) had a detailed, literal understanding of the first paragraphs of Bleak House.

The following is the breakdown by class year:

Overall, we found that problematic readers had no successful reading tactics to help them understand Bleak House, so they became quickly lost and floundered throughout the reading test. Competent readers were not so lost because they came to the reading test with a larger vocabulary and could thereby understand more of Dickens’ language. They were, however, as likely as the problematic readers to avoid translating more difficult language or extended figures of speech. The only active readers were in the proficient group (five percent of the test subjects). These readers immediately recognized when they were lost and were able to turn back and use successful tactics to understand the meaning of each clause and phrase in an extended sentence. Because they were able to understand much more of the concrete detail in each sentence, they could follow Dickens’ narrative [End Page 6] in Bleak House. They could translate some of the figurative language as well and understood some of the major symbolic meanings of the passage.

Problematic Readers

Because problematic readers constituted 58 percent (or 49 of the 85) of our subjects in our study, we’ll focus our discussion on this category. The majority of these subjects could understand very little of Bleak House and did not have effective reading tactics. All had so much trouble comprehending concrete detail in consecutive clauses and phrases that they could not link the meaning of one sentence to the next. Although it was clear that these subjects did try to use various tactics while they read the passage, they were not able to use those tactics successfully. For example, 43 percent of the problematic readers tried to look up words they did not understand, but only five percent were able to look up the meaning of a word and place it back correctly into a sentence. The subjects frequently looked up a word they did not know, realized that they did not understand the sentence the word had come from, and skipped translating the sentence altogether.

Dickens’ rhetorical style is, to say the least, unfamiliar, so entering his world entails making imaginative leaps and consistently thinking on a higher level. None of the subjects in the problematic category had the reading skills to meet this challenge. Although many could vaguely understand the focus of each of Dickens’ descriptions (they saw the mud and fog, for example), they could not interpret the literal, concrete details that composed each description. A special challenge for problematic readers was legal language and information. Often, these readers were also too confused to recognize that Bleak House begins by focusing on a law court: 71 percent of the problematic readers (or 35 of the 49) had no idea that Dickens was focusing on a court of law, a judge, and lawyers. Their misunderstanding happened even though “Chancery” (a specific type of English court where the head Judge [the “Lord Chancellor”] and other judges would make decisions on legal trusts, divisions of property, the property of “lunatics,” and the guardians for infants and orphans [“Chancery Division”]) and “Lincoln’s Inn Hall” (the building that held the Court of Chancery [“Illustration”]) are mentioned in the first sentence as well as in the passage’s ending with its talk of solicitors and a long list of law documents:

On such an afternoon, the various solicitors in the cause, some two or three of whom have inherited it from their fathers, who made a fortune by it, ought to be—as are they not?—ranged in a line, in a long matted well (but you might look in vain for Truth at the bottom of it), between the registrar’s red table and the silk gowns, with bills, cross-bills, answers, rejoinders, injunctions, affidavits, issues, references to masters, masters’ reports, mountains of costly nonsense, piled before them.

(50) [End Page 7]

It is understandable that problematic readers skipped over “In Chancery” and “Lincoln’s Inn Hall” in the first two sentences because they could assume they would understand this language later in the reading. However, based on the reading tests, none of the subjects subsequently looked up any of the legal language, and the 39 percent who guessed correctly that the setting was “a court” or the Lord Chancellor was “a judge” never went further to understand the Court of Chancery in any more depth. The subjects’ understanding of the setting thus remained vague, and they were not able to interpret most of the concrete legal details in the text. A typical case will stand for the rest:

The subject was far from alone in her reading process, a situation we explored from a wider perspective.

The three main reading tactics that problematic readers used were oversimplifying, guessing, and commenting. The most common was oversimplifying—that is, reducing the details of a complex sentence to a generic statement. In the first paragraphs of Bleak House, Dickens follows the fog on the river Thames as it moves from the center of London and winds down to the Essex marshes, 51 miles away from the city. He then centers on the shipyard in the Holborn District of London, the same district that included Lincoln’s Inn, the home of the Court of Chancery. Again, one subject’s response will stand for the others:

By reducing all these details in the passage to vague, generic language, the subject does not read closely enough to follow the fog as it moves throughout the shipyards. And, as she continues to skip over almost all the concrete details in the following sentences, she never recognizes that this literal fog, as it expands throughout London, becomes a symbol for the confusion, disarray, and blindness of the Court of Chancery.

96 percent of the problematic readers used oversimplified phrases at least once to summarize a sentence in the test passage while 61 percent used this method for five or more sentences. Often, subjects used this tactic as a shortcut when they became overwhelmed by a sentence with multiple clauses. One subject disclosed that oversimplifying was her normal tactic, explaining, “I normally don’t try to analyze individual sentences as I’m reading something. I try to look at the overall bigger picture of what’s going on.” Another subject said that she separated reading from thinking: “I’m just reading it [the text]; I’m not thinking about it yet.” Those subjects, however, who relied on oversimplification became more and more lost as they continued to read the test passage. In fact, 82 percent of the problematic readers told the facilitators that they were confused at least once during the test, and 26 percent said they were lost five or more times.

The second most commonly used tactic by problematic readers was guessing. Instead of looking up the definition of a word or phrase, these subjects often just guessed at the meaning. 75 percent of the readers did so incorrectly and then misread that section of the sentence. This problem, however, did not occur all the time: 41 percent of these same readers guessed the right meaning of a word, and 20 percent who defined a word incorrectly did not continue to misinterpret the language that followed. Many problematic readers seemed to rely on guessing as a last resort because most did not, or perhaps could not, connect the meaning of one sentence to the next. The subjects’ inability to understand the literal meaning of Bleak House might be one reason they continued to rely on guessing, however arbitrary or irrational the results. In a typical example, the result is that a subject ventures far from the text’s meaning:

Note that the subject, who is not accessing any of the concrete details in the passage, finds a subject (the Lord Chancellor) and one recognizable word, [End Page 9] “whiskers,” and concludes that the character is in a room with a cat. At this point, she does not seem to understand what she is reading, and so she links a few words together to form some kind of response.

Finally, our subjects with the most challenges relied on commenting— that is, giving personal reactions to the text instead of trying to interpret it. Most of the time, they commented on the mood or difficulty of the passage, and sometimes they generalized so vaguely in a comment that it would be hard to know if they really understood what they had read:

This subject’s comments would be fine were they to have moved from remarking on the scene to translating the specific language in the passage. The subject, however, along with other problematic readers, consistently turned to commentary to avoid translating Dickens’ language altogether. 92 percent of the problematic readers chose this tactic instead of translating at least one sentence in the passage, and 45 percent replaced interpretation with commentary in five or more sentences. Some subjects gave general comments on five or six sentences in succession and then struggled when they were asked to explain the literal meaning of that same text. Two of the 49 subjects in the problematic category used commentary to explain fifteen or more sentences on the reading test.

Beyond their reading tactics, problematic readers were continually challenged by the figures of speech that are woven into the novel’s descriptions. 57 percent of the subjects would ignore a figure of speech altogether and try to translate the literal meanings around it while 41 percent would interpret at least one figure of speech literally, even if it made no sense in the context of the sentence. One subject even imagined dinosaurs lumbering around London:

The subject cannot make the leap to figurative language. She first guesses that the dinosaur is just “bones” and then is stuck stating that the bones are “waddling, um, all up the hill” because she can see that Dickens has the dinosaur moving. Because she cannot logically tie the ideas together, she just leaves her interpretation as is and goes on to the next sentence. Like [End Page 10] this subject, most of the problematic readers were not concerned if their literal translations of Bleak House were not coherent, so obvious logical errors never seemed to affect them. In fact, none of the readers in this category ever questioned their own interpretations of figures of speech, no matter how irrational the results. Worse, their inability to understand figurative language was constant, even though most of the subjects had spent at least two years in literature classes that discussed figures of speech. Some could correctly identify a figure of speech, and even explain its use in a sentence, but correct responses were inconsistent and haphazard. None of the problematic readers showed any evidence that they could read recursively or fix previous errors in comprehension. They would stick to their reading tactics even if they were unhappy with the results.

Why would all the problematic readers be so sure they could read the rest of Bleak House on their own when they had such trouble understanding the first seven paragraphs of the novel? According to their responses at the end of the reading tests, many subjects in the category defined reading Bleak House as skimming the text and relying on SparkNotes (which give plot summaries, characterizations, and analyses) to understand what they had just read. 35 percent (17 of the 49) would rely heavily on SparkNotes to understand the basic meaning of the text. 12 percent (six of the 49) relied on other outside sources like Google or Wikipedia. These subjects seemed to use outside sources as the main way to understand what they could not comprehend on their own. As one subject said, “If I was to read this [Bleak House] by itself and didn’t use anything like that [SparkNotes], I don’t think I would actually understand what’s going on 100% of the time.” Even those subjects who did not mention relying on outside sources usually equated active reading with some type of skimming. For example, one subject said she would read Bleak House by “skim[ming] through most of the novel and read[ing] only certain passages in detail.” Several of the problematic readers in this category admitted that they had successfully used skimming and SparkNotes to read Jane Austen’s novels and Shakespeare’s plays in other English classes.

Competent Readers

The competent readers in our study, who constituted 38 percent (or 32 of the 85) of our subjects, were better readers than the problematic group because they knew more vocabulary and could interpret some figurative language. 88 percent could accurately guess the meanings of some words. Some looked up definitions, and 35 percent were able to look up a definition and then use that word correctly in a translation of the sentence. As a result, most of the competent readers understood about half of the literal prose in the passage. They also could recognize more figures of speech: some were able, for example, to interpret the meaning of Dickens’ similes, [End Page 11] and some could just guess that certain words, like “fog,” might be symbolic. Because these readers had a grasp of the generic meaning of most sentences, they could keep moving through the first few paragraphs of Bleak House without feeling completely lost.

The results were not all good. The competent readers, like the problematic group, were not active in their practice: 96 percent would define words incorrectly and 46 percent would skip words they did not understand. Essentially, they were comfortable with their confusion. If they became lost translating a sentence or a figure of speech, they would often just make an arbitrary guess or skip that section and move on. In the following example, the subject seems to want to breeze through the translation quickly and has to be reminded to translate by the facilitator:

The subject does not really understand what is happening in the passage and makes guesses that do not help her comprehension. For example, she decides that “Michaelmas Term” is a person instead of a set term (September or late October to December for law courts and universities [“Michaelmas term”]). She also does not look up “Megalosaurus” (though she says she will) and assumes the word means “some kind of an animal,” which ensures that she will not notice the figurative image of a dinosaur walking down a London street.

Because the majority of subjects in the competent category were passive readers, they would probably give up their attempts to read Bleak [End Page 12] House after a few chapters. In the reading tests, most of the competent readers began to move to vague summaries of the sentences halfway through the passage and did not look up definitions of words, even after they were confused by the language. None of the subjects in this group was actively trying to link the ideas of one section to the next or build a “big picture” meaning of the narrative. Like the problematic readers, most would interpret specific details in each sentence without linking ideas together. Without recursive tactics for comprehension, it is probable that their reliance on generic or partial translation would run out of steam, and they would eventually become too lost to understand what they were reading. For example, 59 percent of competent readers did not look up legal words like “Chancery” or “advocate,” and by the end of their reading tests, 55 percent had no idea that the passage was focused on lawyers and a courtroom. However, six of the 32 competent readers were much better readers who understood more of the literal prose in Bleak House. They actively looked up some terms and were more skilled than other competent readers in interpreting figures of speech. Though they did not understand as much of the text as the proficient readers, and read very slowly, they could probably comprehend more of Bleak House than the other competent readers.

Proficient Readers

Only five percent (or four of the 85) of the subjects in our study were proficient readers who could translate most of the literal prose in the passage and had the reading tactics to understand most of Bleak House on their own. They stood out because they continually looked up words they did not know. They clearly had a better basic vocabulary than the other student readers: they could correctly guess, for example, the meanings of words like “implacable” and “pensioners.” They could also recognize figurative language and avoided the trap of translating metaphors and similes literally. With these abilities, proficient readers comprehended many of the details in each sentence and could interpret successive phrases and clauses of a sentence to grasp its full meaning. One subject in this category demonstrated this higher level of comprehension:

The difference in this reading test is immediately clear: instead of making a generalized statement to summarize the entire sentence, the subject carefully attempts to interpret each successive clause. He is interested in the details of the setting, stating that the setting is in London and then trying to find a reason why so many people would be “slipping and sliding” on the road. (Perhaps, he thinks, it is because the street is constructed of cobblestones instead of pavement.) He then finds a good way to explain Dickens’ clause, “where tens of thousands of other foot passengers have been slipping and sliding since the day broke,” by stating that the people in this sentence are just the “latest generation” to be walking in the mud that day. This subject is also able to define what “compound interest” is and shows how this financial term is being used by Dickens to describe the layers of mud in the street. The subject’s ability to comprehend details and create meaning makes him a proficient reader.

In terms of the passage’s meaning to the novel’s plot, all the proficient readers knew that Dickens was discussing a judge, lawyers, and a court of law. They could link major ideas together and find patterns in Dickens’ imagery that helped them build a better understanding of the content. For example, one subject was able to recognize a recurring pattern by linking the “mud” reference in the fourth paragraph (“the muddy streets are muddiest, near that leaden-headed old obstruction . . . Temple Bar”) to Dickens’ first reference of mud in the fourth sentence of Bleak House. Finally, the proficient readers’ knowledge of historical details, whether nineteenth-century pollution from coal, child labor, or the British custom of judges wearing wigs, allowed these readers to fill in some context and understand Dickens’ specific references to life in the 1820s.

Dropping Literacy Skills in High-School Students and College Graduates

Recent studies indicate the students in this study who have trouble reading are not alone. Nationally, many American college graduates are not [End Page 14] reaching proficient levels of prose literacy. According to the findings of a report by the Educational Testing Service Center for Research on Human Capital and Education in 2019, “One of every five bachelor’s degree holders among employed college graduates ages 21 to 65 lacks some important skills in literacy” (Fogg 1). The last ACT report in 2022, two years after the COVID-19 pandemic, reported the lowest national ACT scores in thirty years (“Average”), with the average Reading Comprehension score at 20.4 (ACT Profile 2022 9). The newest test scores of middle- and high-school students also show a drop in skills. Sarah Mervosh observes in an article from the New York Times, “Nationwide, two in three eighth graders are not reading with proficiency, according to the National Assessment of Educational Progress, a rigorous exam overseen by the U.S. Education Department. Nearly one in three falls ‘below basic,’ meaning they have not demonstrated even partial mastery of the comprehension and analysis skills expected for their age” (1A). University faculty will probably see even more students who have trouble reading because today’s eighth-graders will soon enough be college students.

At this point, we must ask two obvious questions: Why is proficient prose literacy so important? Why does the ability to read complex texts matter if college students can still graduate with good grades, find employment, and earn entry into graduate schools?

Although there are many graduates who probably succeed without advanced reading skills, the evidence from long-term studies reveal that lower literacy skills affect college graduates throughout their professional careers. The PIACC (Programme for the Assessment of Adult Competencies) has been continually testing since 2012 what it formulates as “the literacy, numeracy, and problem-solving” skills of the working-age population (ages 16–65) in over 40 countries, including the U.S., (“Program” 1). The research has shown that high literacy rates in reading and math are a main indicator of professional and financial success. From the PIACC results in 2014, only 12.9 percent of the U.S. population had a 4 or 5 literacy level, which is the level of reading proficiency needed to understand Dickens’ novels and most other texts in the literary canon (U.S. Department of Education). According to an ETS Center report, which analyzed the PIACC data, those with the highest literacy skills will make 38 percent more a month than other college graduates (Fogg 30). The report observed that “College graduates with higher levels of literacy and numeracy proficiency also had a higher likelihood of working in a CLM (college labor market) occupation” (Fogg 21). It does not stop there. A student who graduates college with below-level literacy (a “3” on a 5-point scale) risks becoming what the ETS calls the “mal-employed”—that is, “an employed college graduate who works in a job that does not require the proficiencies associated with a college degree to obtain employment in the occupation” (Fogg 13). The “mal-employed” were earning wages that were only 14 percent higher than their colleagues with high school degrees while those [End Page 15] employed in the CLM were making 125 percent more a month than high school graduates (Fogg 32–33).

Our study has proven valuable to us, but more work needs to be done. There has been a great deal of research on the reading comprehension of K-12 students, but less attention has been paid to the literacy skills of those college students who are not defined as developmental readers. This literacy issue may also prove more challenging as students from the COVID-19 Generation enter college. In the end, the lesson is clear: if we teachers in the university ignore our students’ actual reading levels, we run the risk of passing out diplomas to students who have not mastered reading complex texts and who, as a result, might find that their literacy skills prevent them from achieving their professional goals and personal dreams.