"If you forget that you're reading a student's paper, you know that you're dealing with an A." So said one of my colleagues, and it got me thinking once again about the relationship between the syntactic component of students' writing and the grades that they receive on papers. I remembered sitting in my Chairman's Office, many years ago at Shenandoah College, discussing the senior essay of a bright non-traditional student with the three other members of the English Department. Much was said about the style of her writing. As I look back now, I think we liked it so much because it sounded so much like our own.
      Although a lot remains to be learned about the statistical analysis of syntax, we do know that professionals tend to average 20 words per main clause, to vary the length of main clauses, to vary sentence openers, to use a liberal sprinkling of subordinate clauses, gerundives and appositives, and to make relatively few errors. Theoretically, in other words, we could design a research project that would measure the correlation between syntactic aspects of style and the grades that students get on papers. I probably will not live long enough to do so, however, because the task is not that simple. It has been tried, and this document will review some of those attempts to explore what worked and what didn't.

     Perhaps the best way to summarize this fascinating study is to quote the abstract:

     "A study was conducted to determine whether a significant correlation existed between an essay's letter grade and five important factors of syntactic maturity (clause length, t-unit length, sentence length, clauses per t-unit, and t-units per sentence). A total of 45 essays -- 15 A, 15 B, and 15 C papers as graded by five junior college English teachers -- were assigned a point value and analyzed for a correlation between the grade and each of the five syntactic maturity factors. The results indicated a slight correlation between the grades and the number of clauses per t-unit and the number of t-units per sentence, suggesting that the teachers were influenced somewhat in their grading by the writer's ability to build compound sentences. The second portion of the study was an analysis of the difference between each of the three grade groups. Here, the number of words per sentence was found to be the most significant factor. The significant difference between the A and the C papers was possibly due to the fact that the A students connected independent clauses with semicolons more frequently than did the other students and thus produced much longer sentences and more t-units per sentence. While the results infer that sentence combining exercises may help increase the quality of a student's writing, factors of syntactic maturity overall had only a low correlation with the quality of the writing."

Question # 1: Who is doing the grading?

     One of the interesting things that Roos did was to ask each of the five graders "to estimate roughly what percentage of an essay grade is determined by a student's proficiency in each of four areas." He then presents a table of the results:
 

According to Roos, two of these teachers taught lower track Freshmen, two taught upper track, and one (C) taught Sophomore literature. Each of these five teachers was asked to submit nine papers, randomly chosen from the same assignment -- 3 A's, 3 B's, and 3 C's. Although Roos was most interested in style, he notes that "no teacher rated it higher than 10 percent, and two of the three [sic] teachers insisted it had no effect on a paper's grade. One of them commented on the survey, 'I don't think we have students who are ready to work on style.'" (7) Asking the teachers to indicate what they saw as the relative weight of each of the four areas was an interesting idea, but there is no indication that the five teachers meant the same thing by "Style." Perhaps it would have been better to include more categories, such as sentence length, variety, etc.
     Another problem is implied by Roos when he discusses combining sentences with semicolons. He specifically notes that although teachers had "dabbled" with sentence combining, he states that "none of the teachers used sentence combining extensively in class." (6) But this raises the question of what did they do in class? Suppose, for example, and it is not a far-out assumption, that the teachers taught students how to use semicolons. [See above.] These were, after all, mainly English teachers teaching comp to college Freshmen, Freshmen whom the teachers believed were unable to deal with "style." Thus the significant results that Roos finds could be simply the result of the teachers recognizing in the writing exactly what the teachers had been teaching. In itself, this is not bad, but it certainly would mess up the research study, and without further background from Roos, we have no way of knowing whether or not this was the case. Roos also does not discuss the errors in the students' writing -- fragments, comma-splices, etc. that could have affected the teachers grading. The teachers may have considered these as matters of "mechanics," but they certainly affect t-unit and word/sentence counts.
     Still another problem is that Roos looks for results in the total averages, i.e., he averages all the A papers, B papers, etc. In effect, this may obscure some very significant results. Suppose, for example, that teachers A and C do give much better grades to students who write longer t-units with more subordinate clauses. When their eighteen  papers were added to the 27 papers of the other teachers -- and averaged -- the significance would have been lost. Roos, in other words, should have looked for statistical correlations between the grading of individual teachers and the syntactic structures in the papers that each graded. To say that, on average, syntactic style does not affect grading, is not the same as saying that, for example, for one of every five teachers, syntactic style may make the difference of a letter grade on a paper. A typical college degree entails forty courses. If the grades in eight of those courses can be shown to be affected by the students' syntactic style, that is a pretty good reason for looking more closely at the teaching of syntax.
     Indeed, the final problem with Roos' choice of graders is that they are all English teachers -- four of them English teachers of writing. As my critique of NCTE's attitude toward teaching grammar indicates, it might not be difficult to show that English teachers are more lenient and less influenced by students' "style" than are teachers in history, psychology, biology, business, math, etc. Any study that claims to make a valid correlation (or lack of) between syntactic maturity and grades should include graded papers from a wide range of disciplines. This is particularly true because papers in history, psychology, etc. may require advanced syntactic constructions -- subordinate clauses, appositives, gerundives -- to convey their content clearly.

Question # 2: Definitions of Syntactic Constructions

     We need to remember that Roos was presenting a paper at a conference and thus was constrained for both time and space. But for his study to be valid, we need to know exactly what he counted, for example, as main and subordinate clauses. As I have noted elsewhere, the researchers do not agree on what counts as what, but such definitions can have a major impact on the statistical results. [Mellon, for example, counts some subordinate clauses as separate main clauses.]

Question # 3: The Definition of Syntactic Maturity

      For the time in which he was working, Roos includes a fair summary of previous research, but he missed some important work by Kellogg Hunt on "Late Blooming" constructions. Hunt not only argues that the appositive and gerundive are "late blooming," but he also points out that they reduce both clause count and clause length. Consider, for example, the following sets of sentences:

Appositive:
1. She was a biologist at Cornell: she studied marine life.
2. When she was a biologist at Cornell, she studied marine life.
3. A biologist at Cornell, she studied marine life.

Gerundive:
1. He was working in Omaha: he was an engineer.
2. When he was working in Omaha, he was an engineer.
3. Working in Omaha, he was an engineer.

Question # 3: Narrowness of Scope

     I have already noted the problem of Roos limiting the study to papers graded by English teachers, but he makes another assumption that he himself questioned:

"The study was limited to the top three grades because it was assumed that any significant results for these grades would hold true as well for D's and F's. This assumption may not be true, however, and future studies should explore these grades also." (7)

Syntax affects readability.
Whatever affects readability affects grades.
Therefore syntax affects grades.

Conclusion

     Although Roos' conclusions are not valid, his study certainly points in an important direction and suggests some of the features that future studies should incorporate. Ultimately, it would be nice to have some large-scale studies of the correlations between syntactic maturity and grades at different grade levels and in different subject areas. Such studies, however, are extremely time-consuming and expensive. In the meantime, numerous studies on a smaller scale would be helpful. A graduate student or a teacher could, for example, collect a set of graded essays from another teacher (at any grade level, and preferably not from an English teacher), analyze the syntax, and see if there are correlations. Ideally, such a study would include permission to reproduce the original writing on the internet. That would make the originals available not only for review, but also for inclusion in larger studies.