In this presentation, Annie Agard presents a whole range of evidence gathered from her ESL classes. In this PowerPoint presentation, Agard shares many different findings from her classroom research on her ESL. class. The evidence is not only useful for ESL teachers but as a model of going public with classroom research.

The evidence covers many different kinds of methods for gathering student performance data, such as through pre/post data.

Some of her evidence is based on student attitudes and perspectives.

One of the consequences of her classroom research is the ways that she discovers “complicating evidence” from her students. One example of this is her analysis of perforamance and atittudes related to group work, where she discovered in part that some students who performed better through group work actually prefered working alone.

from The Power of the Pursuit, Suzanne Crawford, Lydia Alvarez, and Lynn Serwin (Cerritos College)

———-

Two Versions of a Student Self-Assessment Survey, Suzanne Crawford

Surveys mapping student progress in using more effective and detailed language in writing:

2006 survey:

Hello English 20 Students!

As you all know, we have been working this semester to, among other things, improve your word choice when it comes to using more concrete and specific language (the “green” words). Please give me some of your thoughts about your progress with this skill by answering the following questions. Thank you very much.

S. Crawford

1. Do you feel that, as a result of our various efforts, you are more likely to use such better words?

For example, do you see the value in writing like the first of these (A) and not the second (B)?

A. The boy sat down on a chair.

B. Tall, slim Joe plopped himself down on the old wooden rocking chair.

Do you think the various handouts we used were helpful?

Do you think using the markers helped you understand this concept better?

2007 survey:

ENGLISH 52: Self-assessment of your writing progress

Describe what progress you have made this semester in your ability to craft effective essays. Please use specific evidence from your papers to describe your writing improvement. Consider comparing the skill levels of your earlier papers with those of your later papers. In particular, please note any improvement in language use. Do you now, for example, use more vivid or concrete words? Be sure to support your points with direct references from your papers. For example, if you claim that you now use more specific words, give before and after examples.

Adapted from Beverly DiSalvo and Shirley Kahlert (Merced College), “An Integrated Model: A Cluster of Pedagogies.”

We gain understanding by asking students what they are thinking

It is impossible to predict what a student might be thinking or how someone might be working to understand a concept. We have learned to watch for “moments of difficulty” and ask students what they are thinking at the moment they “understand” a concept

In one video, Ysabel Solorio is writing seven cards which contain sentences using coordinating conjunctions. Follow the link above to view the video. Pay attention to her “moment of difficulty,” and what she asks to try to make sense of what she is doing. Her instructor expected that because of the nature of the exercise, she would understand that all the conjunctions function in the same way even though she might have trouble understanding the meanings. Ysabel’s confusion was not with the meanings. It was that each of the conjunctions were in the same class and joined the same kind of structures.

Read what students say about their moments of difficulty.

Like many teachers of developmental English, we have been looking for better ways to strengthen our students’ literacy skills. We hypothesized that a more integrated approach, combining reading, writing, and grammar, would be a powerful context for improvement.

From Lydia Alvarez, Suzanne Crawford, and Lynn Serwin, “The Solution: The Power of the Pursuit”

Most instructors want to make changes to their classses in order to see their students improve, but many do not know what to do to guarantee that they succeed. We too had this dilemma. We often wondered if the changes we were making in our classes were even working. The process of inquiry and research was the solution. We began with reflection. What were we doing? What was happening? What would our classes look like if they were different? We used concepts learned through the scholarship of teaching and learning to begin an inquiry process.

We began with a reserach question. “What would happen if students recognized vivid and percise language in their writing?” “What would happen if students used a process to add more engaging details to their paragraphs?” “What would happen if I stopped writing on student papers?” We developed tools to try as interventions to see what would happen. We looked at student work, we wrote our own reflections, we analyzed results. What emerged? More intentional teaching. Our process of developing ways to address our research questions led to more questions. The process transformed our teaching. We see the power of the pursuit as the most effective way to approach any problem in any classroom setting.

Below are links to our individual projects.

Suzanne’s pursuit began with a question that considered the connection between students’ growing “awareness, understanding, and appreciation of specificity/concreteness in word choice” and their improved writing. As Suzanne’s teaching became more intentionally focused on this question, student awareness of the power of their word and language choices grew.

Lynn’s pursuit began when she observed that better student writers employed steps of a writing process, while struggling student writers did not. She wondered if requiring each step of the writing process from all of her students would lead to more improvement. Lynn’s teaching became focused on guiding students through required steps, creating repeated experiences where the process led to improvement. Would these repeated successes eventually lead to students employing the steps intentionally and using specific details intentionally?

Lydia’s pursuit began with a concern that student writers could be more active in the decisions required to improve their writing. She reconsidered her role in the classroom, and intentionally reduced her presence as the person who prescribed change. More room for students to own their writing and learning was created.

From Lydia Alvarez, Cerritos College: “The Power of the Pursuit”

The Before and After is a reflective tool I have designed based on a handout I have been creating for my students since the beginning of my teaching career. I have always created handouts called “Celebrating Revision” where I have copied short excerpts from student papers before and after they were revised. Students respond enthusiastically to these dramatic “makeovers.” Even when I haven’t had the time to use examples from a class I am currently working with and have used a handout previously created with another class, there is very little detectable difference in the students’ enthusiastic response. Just like any “makeover” the value of this tool is in the “wow” factor.

Now, in my writer centered classroom, the Before and After tool has evolved into a two part activity requiring students to produce the before and after examples themselves. Any time student writers have more than one draft, they evaluate one of their own earlier drafts, looking for a moment that has dramatically changed in a later draft. They “take a picture” of both the before and after; meaning, they copy both excerpts exactly as they appear. The second part of this reflective tool asks students to describe the changes, explain why they made them and what effect they were hoping the change will have on their audience.

Choosing the revised moment and reflecting on its improvement encourages students with varying levels of confidence. This reflection also sets students up to be more actively involved in their writer’s groups. Writers are now better prepared to participate in response groups and to interact with their audience, to see whether or not the changes they made will have the effect they wanted.

See Lydia’s Tools Page here.

From Pat Wagener, Can Problem Solving Become a Habit of Mind?

Pat says:

Students’ own perceptions of their math skills and abilities are often considered as anecdotal evidence, rather than factual information that has been quantified. In a meeting at the start of the Fall 2006 semester, this subject came up, and it occurred to me that this need not be the case. So I asked myself:

“Can students’ perception of their math skills and abilities be quantified, and measured with an acceptable level of statistical significance?”

And then,

“What is the research question that would enable me to answer this question?”

At first I thought to just have students assess their math skills in general. But then as I continued to think about this project, I realized it was important to know as much as possible about their self-assessment in connection with all the elements of the course: Multiple representation of mathematics, group work, solving word problems, computer aided instruction, and students making presentations of their solutions. And so I ended up with eight (8) areas for the students to assess.

A means to address this study was an attempt to quantify and measure each student’s self-assessed confidence learn in eight (8) specific areas or aspects of math and how they learn math. A survey would be given to the students early in the semester (pre-) and late in the semester (post-) to determine the change. These areas are as follows:

With this method, I found that the anecdotal can become “factual” with a high degree confidence. Here are two key insights:

(1) Students’ self-perception changed very significantly in all questions (p-value<<0.01).

(2) An area of the greatest change (and most positive comments) was in the role of student presentations and self-confidence.

Click here to get a copy of the confidence survey and read more details about the method and results.

]]> http://specctoolkit.carnegiefoundation.org/2008/11/14/measuring-student-perceptions-of-confidence/feed/ 0 http://specctoolkit.carnegiefoundation.org/2008/11/14/think-alouds-in-an-english-course/ http://specctoolkit.carnegiefoundation.org/2008/11/14/think-alouds-in-an-english-course/#comments Fri, 14 Nov 2008 11:43:54 +0000 Molly Breen https://digitalcommons.georgetown.edu/blogs/fitoolkit/?p=307

A focus on students and their learning is central to the faculty inquiry process. Consider Katalina Wethington, who participated in a scholarship of teaching and learning seminar at Los Medanos College in fall 2006. Wethington was puzzled by a problem in a developmental English course one level below freshman English. In Wethington’s case, the problem was a specific roadblock to learning that she had observed again and again:

The impetus for this investigation is my observation that the majority of students in developmental classes have difficulty properly applying evidence to the argument they are building or the point they are trying to support. I have especially noted the frequent misapplication of quotations from primary and secondary sources….Does this difficulty reflect a lack of understanding of the role of quotes in academic discourse, difficulty in simply choosing the right quote(s), or a larger weakness as far as creating logical connections between pieces of an argument? (Wethington, 2007, Explanation of Project)

Using a trio of techniques to investigate these questions, Wethington assessed students’ initial understanding of quote incorporation by asking them to write an ungraded paragraph-and took that opportunity to solicit her students’ help as co-inquirers. “This task is anonymous and will not be graded. So don’t feel pressured, but do give it your best shot! This will help me see how well you as a class have understood the concept of incorporating quotes…” (Wethington, 2007, Methods of Investigation). Wethington also analyzed student work throughout the semester by means of a specially-designed rubric to categorize quotation expertise; and conducted and videotaped “think alouds” with selected students earlier and later in the semester.

In these sessions, she asked students to write a short essay and then discuss with the investigator a set of questions like: “Can you explain to me why you chose the quote in the paragraph?” and, if they find it difficult to explain or feel they may have chosen the wrong quote, “What kind of quote might have worked better?” (Wethington, 2007, Short and Long Assignment Think-Aloud Questions).
Wethington found out that early in the semester, while students appeared to understand the technique of quotation incorporation, “it hadn’t yet sunk in that it wasn’t a ‘I did it everything is fine I don’t have to think about it anymore’ equation.

In essence, the idea of a thoughtful, recursive process wasn’t clear, nor was the idea [that] using [the three-step process they had been taught] didn’t [bring] immediate success. The inner logic of what they were writing and why still eluded them and made it impossible for them to self-edit.” For Wethington, this meant emphasizing that good writing was about clear thinking, for which there is no “trick” (2007, Think-Aloud Results).

Later in the semester, Wethington found that one of these same students who earlier felt that “nothing is wrong” was now aware that something was not quite right with her use of quotation and could say exactly what it was. If rewriting now, the student said: “I think…what I would have done is find a quote that would have described more that [the character] was obsessive because the quote I use doesn’t describe that he was obsessive; it just describes the part where [another character] kills him” (2007, Think-Aloud Results). Her student’s willingness to pause and doubt her earlier choice pleased Wethington. “This represents, to me,” she said, “the best I could ask for in self-reflection and growth as a student learns a new skill” (2007, Think-Aloud Results).

Wethington realized that it is wrong to assume that students can easily transfer a concept (like using quotes effectively) from one genre to another, and that it would be better for instructors to discuss the effective use of quotes with every kind of text they teach.

Adapted from Mary Taylor Huber, The Promise of Faculty Inquiry for Teaching and Learning Basic Skills. Carnegie Foundation for the Advancement of Teaching. 2008. Strengthening Pre-Collegiate Education in the Community Colleges.

SPECC campuses explored many models for bringing people together in faculty inquiry groups, which varied depending on the circumstances of the college, the history of faculty collaboration in developmental education, the creativity of the coordinators, and the purpose at hand. Some groups emphasized individual projects; others focused on a theme of common concern. Some involved colleagues from across disciplines; some were specific to teachers of sections of the same course. At one campus, staff from the Teaching and Learning Center facilitated faculty inquiry groups; at another, faculty with special responsibility for coordinating basic skills instruction did the job. Laney College used a special model of “reflective inquiry” to conduct its faculty inquiry groups; others developed highly structured activities (like CCSF’s student focus group exchange); and many ended up with a productive mix. As Katie Hern noted of Chabot College:

We’ve done three kinds of faculty inquiry in the SPECC grant-from primarily solo inquiries like my sustainability research (which was then shared and discussed in broader faculty forums), to our developmental English Faculty Inquiry Group focused around Student Learning Outcomes in one class, to the multi-disciplinary Faculty Inquiry Group with social science that connects individual inquiries by each faculty member with a central question around “How can we all be basic skills teachers while still addressing content coverage. (2008)

With all of these possibilities in play, it is clear that the potential of faculty inquiry to improve basic skills instruction goes beyond informing classroom teaching and extends to the design of the courses and programs that particular classrooms serve. (13)

Adapted from Mary Taylor Huber, The Promise of Faculty Inquiry for Teaching and Learning Basic Skills. Carnegie Foundation for the Advancement of Teaching. 2008. Strengthening Pre-Collegiate Education in the Community Colleges.

Faculty Inquiry Groups (FIGs) treat professional development as a collaborative enterprise. One of the most persistent impediments to educational improvement is that teachers have-because institutions provide-so few purposeful, constructive occasions for sharing what they know and do with one another. Thus, one of the most important moves a campus can make is to create occasions for educators to talk, to find colleagues, to be part of a community of practice.

Of course talk is not enough, and not all talk is created equal. Skeptics worry that FIGs may produce energetic conversation but no real advance in knowledge or improvement in practice. One external reviewer of SPECC wondered if the open exchange encouraged in FIGs might reinforce misguided notions about, say, the capacity of certain groups of students to succeed.

Thus, it is important to stress that collaboration is not “just talk.” Indeed, many of the campuses have worked their way toward FIGs with carefully structured routines and protocols for collaboration. The English group at Los Medanos, for instance, operates as a kind of graduate seminar, with clear tasks and homework for each meeting and an emphasis on developing new understandings and products-course assignments, for instance, and assessment instruments. At Glendale Community College, FIGs employed by the math department are dedicated to the design and analysis of common final exams, and at Cerritos College one focus has been on identifying explicit student learning outcomes. At City College of San Francisco, several FIGs now organize themselves around a carefully structured process of classroom observation, which is then grist for discussion during their meetings. One might in fact observe that FIGs benefit from the same principles that operate in effective developmental classrooms: high structure, high expectations, intense engagement, intentionality, and inquiry. Teachers are developmental learners as well.

Adapted from Basic Skills for Complex Lives: Designing for Learning in the Community College. Carnegie Foundation for the Advancement of Teaching. 2008. Strengthening Pre-Collegiate Education in the Community Colleges.

Glendale Community College’s mathematics program, in 2000, instituted a common final examination for all sections of pre-collegiate algebra. The department produces tabularized information after each examination in order to show, among other things, the dropout rate and mean GPA for each class, as well as the performance of each class (properly coded to ensure anonymity) on the overall test and on subtopics.

The faculty as a whole discuss topical areas in which students appear to be learning well and those they are still struggling with. Individual instructors examine the performance on the test of their own students in various ways that reveal important aspects of their teaching practice and grading standards. For example, instructors whose A and B students do relatively poorly on the final examination must ask themselves whether their standards are too lax. Instructors whose C students perform well on the test must ask themselves if their standards are unrealistically high. The entire project stimulates faculty discussion and reflection in ways that did not occur before.

Additionally, as participants in this process testify, the process of developing and coming to consensus on an assessment framework, along with the development of exercises and a scoring rubric, all tend to get faculty on the same page about what is important for students to know and be able to do. Instructors who entertain idiosyncratic notions about grading or essential content must defend their ideas to their colleagues in an open forum where departmental objectives and disciplinary considerations are the reference standards. Glendale’s experience with the common examination nicely illustrates its power to encourage honest discussion about the appropriate weight to be given to effort over outcome, to growth over absolute level of achievement, to test performance over class participation-crucial considerations in a commitment, like SPECC’s, to documenting improvement over time.

Yet Glendale enjoys an additional benefit that in its long-term effects may prove to be more important than all the rest. It is exemplified in how the math faculty use test results in professional development. Noting that some instructors’ students repeatedly performed well above average on the examinations or on particular topical areas, the department began a program in which faculty observe these highly effective instructors in action. In this way, the Glendale experience points to another important lesson about impact: while improvements in student learning are the bottom line, they are often wrapped up in other kinds of impact that are hard to untangle. Indeed, the experience at Glendale and many other SPECC campuses suggests that faculty learning may be the single most important variable in improving student learning. In today’s accountability culture, this is a point that can get left behind, and it is worth hammering home. Student learning matters; of course one wants to see an upward trajectory in student success. But faculty learning matters as well. And on a healthy campus, the two work together. (41)

Adapted from Basic Skills for Complex Lives: Designing for Learning in the Community College. Carnegie Foundation for the Advancement of Teaching. 2008. Strengthening Pre-Collegiate Education in the Community Colleges.