National School Reform Faculty

Research That Supports Our Work
Date: July 21-24, 2003
Listserv: Coaches

Monday, July 21, 2003 9:05 AM
I got this from the assistant superintendent from a district near
Boston I have been working with - has anybody out there written this paragraph already? Any ideas for research sources?
Many thanks,

In order to use Title II money to support CFGs for next year, I need a paragraph response to the following:

"Describe the process by which scientifically-based research was reviewed to appropriately select the proposed activity (CFG Coaches Training)." They want to know what "hard" evidence supports going with this program.
Perhaps this has been done already? Wishful thinking? Many thanks. I need something by July 25...Alec Wyeth, Asst. Superintendent, Waltham Public Schools
Gene, MA

Monday, July 21, 2003 9:19 AM
Wouldn't the research by Faith/Bill/Ann have something??

Link below
http://www.pdkintl.org/edres/resbul28.htm
Alan, NY

Monday, July 21, 2003 10:26 AM
There is a really impressive study that was done at Stanford by Milbrey McLaughlin and Joan Talbert. The book about it is published by University of Chicago Press and (I believe) it is about $16.00 in paperback. Anyway, here is the publisher's paragraph about the book:

"American high schools have never been under more pressure to reform: student populations are more diverse than ever, resources are limited, and teachers are expected to teach to high standards for all students. While many reformers look for change at the state or district level, the authors here argue that the most local contexts--schools, departments, and communities--matter the most to how well teachers perform in the classroom and how satisfied they are professionally. Their findings--based on one of the most extensive research projects ever done on secondary teaching--show that departmental cultures play a crucial role in classroom settings and expectations. In the same school, for example, social studies teachers described their students as "apathetic and unwilling to work," while English teachers described the same students as "bright, interesting, and energetic." With wide-ranging implications for educational practice and policy, this unprecedented look into teacher communities is essential reading for educators, administrators, and all those concerned with U. S. High Schools."

The website for ordering the book is
http://www.press.uchicago.edu/cgi-bin/hfs.cgi/00/14326.ctl

I think this is one of the most respected studies in educational reform movements. I think you might be able to Google more info about Milbrey or this one study and get what you need. Otherwise, I suggest buying the book. ;-)
Angela, CA

Wednesday, July 23, 2003 10:25 AM
Hi Gene,
What about the work that Michael Fullan describes in his latest book making the direct connection between collaborative practice and student achievement? It's in The New Meaning of Educational Change 2001, Third Edition, Teacher's College Press, pp. 123-136 in Chapter 7. It's what we have used to make the link based upon his research. Hope it helps.
Mary, ME

Wednesday, July 23, 2003 5:08 PM
Gene: the challenge Alec (et al) are facing is the scientifically based research was reviewed component of this request, especially with the government's definition of 'scientifically-based' research (quantitative, statistical with control groups...and typically based on student-achievement data as represented in standardized test scores). Depending on who reviews the application, they may also be looking for the 'reviewed' element, as in having looked at a variety of similar studies from which CFGs were selected.

We've not had to address this particular question, but you might direct him to the National Staff Development Council's website (www.nscd.org) as a number their recommended professional development practices could be framed as CFGs. Also the Wisconsin Center for Educational Research (www.wcer.wisc.edu) has a wealth of professional development studies that may be helpful.

And perhaps, NSRF could begin doing it's own research/documentation. While these agencies prefer 'outside' research, most of the documentation of this type is done as part of internal program evaluations.
Hope that helps.
Hannah, CA

Wednesday, July 23, 2003 6:27 PM
On the issue of "research" that supports CFG work.

I believe that there is quite a bit of research that supports CFG work. The problem is that except for the Dunne, Naves, Lewis piece (Dunne, F., Nave, B. & Lewis, A. (2000) "Critical Friends Groups: Helping Teachers Improve Student Learning." Phi Delta Kappa Center for Evaluation, Development and Research. December 2000, No. 28) not much that I know of talks specifically about CFGs. However, there is a lot of work that relates to professional communities, learning communities, and collaborative practices and culture that makes a very powerful case for CFGs, and can fulfill the requirements for "research" that grant funders, especially of Title II, are looking for.

The most important research is Newmann and Whelage (Newmann, F. & Wehlage, G. (1995). Successful school restructuring. Madison, WI: Center on Organization and Restructuring of Schools, 1995.) The relevant finding of this important study - based on standardized tests - is "Schoolwide teacher professional community affected the level of classroom authentic pedagogy which in turn affected student performance.' Much of this work is based on research by Louis and Kruse in their study on professional community and urban high schools (Louis, K. & Kruse, S. (1998). Professionalism and community: Perspectives on reforming urban schools. Thousand Oaks, CA: Corwin Press).

Another series of studies, reported by Millbury McLaughlin, focuses on the relationship of collaboration, school change and improvements in practice and student performance. A few references are: (1) McLaughlin, M.W. (1990). "Enabling professional development: What have we learned?" In Staff Development for Education in the 1990's: New demands, new realities, new perspectives, edited by A. Lieberman and L. Miller, 61-82. New York: Teachers College Press. (2) McLaughlin, M. & Oberman eds. (1996). Teacher learning: New policies, new practices. New York: Teachers College Press. (3) McLaughlin, M. & Talbert, J. (2001). Professional communities and the work of high school teaching. Chicago: University of Chicago Press.
An earlier study that was very influential is Rosenholtz, S. (1989). Teachers' workplace: The social organization of schools. New York: Teachers College Press. In this study Rosenholtz studies schools that have collaborative professional cultures and those that do not with respect to improvements in teacher practice and school change.

Richard Elmore, as well, reports a lot of research - mostly his own - around these topics. Two good references are: (1) Elmore, R. F., & D. Burney. (1999). "Investing in Teacher Learning." In L. Darling Hammond & G. Sykes (eds.), Teaching as the learning profession. San Francisco: Jossey-Bass. And (2) Elmore, R. F., P. L. Peterson, & S. J. McCarthy. (1996). Restructuring in the classroom: teaching, learning and school organization. 1st ed. San Francisco: Jossey-Bass Publishers.

There's more. Another important source of support is the field of organizational learning. The value and power of collaborative practice is well documented in the organizational learning literature. A good place to begin: Watkins, K. and Marsick, V. (1993). Sculpting the learning organization: Lessons in the art and science of systemic change. San Francisco: Jossey-Bass. There is a lot more out there in this field.

I write all of this not only to help with the important question that has been raised on the listserv, but also to encourage people to start to take a more research-based approach to evaluating their CFG work. The problem with evaluating CFG work - as supposed to something like Success for All or other more scripted programs - is that these programs forget about the teacher, the school, the culture, the kids etc. Therefore it is much easier to associate Success for All with increased test scores because everyone (according to them) is doing the same thing. CFG work is much more messy (as it should be). To connect CFG work with increased student performance, you would have to show some change in the culture of the school or group, changes in teacher practice and, ultimately, increases in student learning. At the same time, you also have to be able to argue that whatever changes happen in any of these steps are not the result of some other factor. It just is not easy.

One strategy that a colleague of mine and I have used is to give a statistically validated instrument, the Team Learning Survey (Dechant, K. and Marsick, V. (1993). Team learning survey. King of Prussia, PA: Organization and Development, Inc.) to different CFGs we have started. My CFG was with a school system leadership team and hers with a group of teachers in her school. At the end of a year, we gave the survey again to find out what changes had happened on the team in terms of collaborative practice. The results were very positive and "research based." We are trying to start to build a statically based case that confirms that something important happens in a CFG. We are both starting new CFGs this fall with another school system leadership team and other school groups, and intend to use this strategy once more. It is an easy instrument to use and score, gives the group some good feedback about itself and seems to help with policy makers and funders. The point of all of the above is that not only is there good research to support CFGs, but also, with all of the folks that are working in CFGs all over the countryside, we should be able to devise some thoughtful and valid research based strategy to evaluate the work.
I hope some of this is helpful,
Kevin, MA

Thursday, July 24, 2003 8:51 AM
Hi Kevin,
Wow, this is really terrific material for those of us hearing the old "research-based" question every time we recommend CFG work. Thanks so much.
Mary, ME