-
The Museum Learning Collaborative: Phase 2 November 1998 Gaea Leinhardt & Kevin Crowley
Use these buttons to view the sections that comprise the full version of the research plan:
We stated in our original proposal that we view museums as among our preeminent institutions for learning. They are places in which our society gathers and preserves visible records of social, scientific, and artistic accomplishments; in which it supports scholarship that deepens and extends knowledge; and to which people of all ages turn to extend their understanding of history and society, to expand their cultural horizons, or to explore scientific phenomena. Yet we know less than we should about how learning actually occurs in museums.
After a year and a half of serious work on the issue of learning in museums we have refined our understanding of the issues that must be addressed if we are to advance research on exactly how learning is affected by museum experiences. As with all scientific endeavors, clarifying the problems and sharpening the questions are the principal ways in which we can make progress. We noted the central problem of the lack of theoretical coherence in extant museum learning research and we proposed that using sociocultural theory would be advantageous. Our reading of the literature, our participation at museum conferences, and our own museum learning research suggests that there are four specific problems in the field that arise from this lack of theoretical coherence: 1) the learning-definition problem, 2) the univariate problem, 3) the museum-diversity problem, and 4) the quantitative/qualitative problem. Our design for research will address these problems in a manner that is informed by sociocultural theory.
First, while there are numerous studies (we have over 2,000 in our literature base) in which learning is examined in the museum context, these studies tend to have unique, idiosyncratic, and often implicit definitions of learning. These definitions range from lists of facts retained or recognized (Anderson & Lucas, 1997: Balling & Falk, 1980; Boggs, 1977; Gottfried, 1980; Koran, Lehman, Shafer, & Koran, 1983; Stronck, 1983) to experiencing a great rush of "ah ha," and from emotional reminiscences (Falk & Dierking, 1990, 1992, 1997; McManus, 1993a; Stevenson, 1991) to time-on-task indices (Alt, 1979; Bitgood & Patterson, 1993b; Cone & Kendall, 1978; Falk, 1982, 1983b; Sandifer, 1997; Serrell, 1995, 1997). It is natural that in an emerging field such as museum learning, there should be a rich variety of different meanings assigned to the notion of learning. However, it is critical that, regardless of the meaning of learning assumed, meaning should be public and theoretically supported rather than implicit and ad hoc. This public accountability in defining learning opens the field to a healthy dialogue. The opening encourages the use of dynamic rather than static definitions. Static definitions of learning have tended to come from fields other than museum learning research without sufficient consideration for how the unique social contexts and institutional affordances of museums might transform meanings of learning. We want to be particularly clear here--we are not advocating major debates over what constitutes learning. Rather, we are arguing for accountability and for attention to whatever definition is used in any particular study.
A second problem is one we call the univariate problem. What we mean by this is the tendency to associate a particular indicator of learning such as recognition of ideas, terms, or items with a single factor such as age, presence of signs, or time spent at the exhibit. In truth, the museum experience is multivariate; that is, for each museum experience there are multiple influences and multiple outcomes. Most museum researchers would agree with this statement at the conceptual level, but at the pragmatic and analytic level, there have been few studies that could directly describe museum learning beyond univariate terms. This is not a necessary state of affairs. There exist analytic procedures for capturing at least some of the richness of the multiple levels of experience and relating them systematically to rich theoretically defensible definitions of learning. We do not wish to be critical of those studies whose purpose was to ask single, well focussed, pointed questions in the service of evaluation; rather we wish to point to the limitations of such approaches when the purpose is to understand the meaning and nature of learning.
A third problem is one of museum diversity and the corresponding difficulty of generalizing findings from one kind of museum to another. Our research has indicated that the overwhelming body of work has been conducted in science and children's museums. While there is a rich tradition of research in natural history and art museums, it is a much smaller corpus. There is very little research that spans museum types. To some extent this is natural and appropriate since it can be argued that the differences between museum types are epistemologically grounded and so profound that they preclude attempting to work across them. In fact, to do so would violate the museum's cultural intention. That is, while many very different organizations are technically labeled "museums," this category is formed arbitrarily based more on economic and political reasons than on reasons of inherent cultural meaning. However, we believe that we can conduct parallel studies in different museum settings in ways that honor the distinctions among different museum types while generating some level of comparability across them.
Fourth, there is the problem of a tension between rich, conceptually grounded qualitative studies that inform us about a single institution, a single population, or a single experience and more generalizable quantitative studies that inform us about more than one situation yet tend to be more superficial. As is true in all the social sciences, it is important to make use of both quantitative and qualitative methodologies so that we avoid the extremes of having convincing and rich knowledge about a completely unique situation or having convincing but trivial information about many situations. Unfortunately, many seem to feel this is an either/or choice--it is not. We believe we have a system of designs that will take good advantage of the more detailed and valid information that can be obtained from qualitative studies and then use that information to inform the dimensions and measures in a comprehensive quantitative study.
Theory highlights the questions and issues worthy of exploration, spotlights what is central in the research findings, and provides an integrating frame for a series of independent studies. In Phase we go beyond that initial set of assumptions to argue for a single, large, integrative study. The single integrative study that we are proposing will make use of the three theoretical themes from our previous work by transforming them into measurable dimensions in a model of learning. Of course, one problem with conducting a large comprehensive study is that unless one measures dimensions that are theoretically grounded and meaningful with respect to previous museum research, there is the risk of obtaining technically correct but conceptually uninteresting findings. To insure that the measures we use for each dimension in the model capture theoretically essential elements of sociocultural theory, we propose a series of validation studies. Thus, the large integrative study will build upon a systematic set of validation studies through which ideas, systems of measurement, and techniques can be tried out and revised. These studies, conducted in the first 18 months of the three-year Phase cooperative agreement, will incorporate prior visitor studies findings and culminate in the creation of accurate and general measures. At the same time, these validation studies will contribute to the larger picture of learning in museums. Thus, our pragmatic strategy is to build towards a single unified study to be conducted during last 18 month of the agreement.
Defining Learning as Conversational Elaboration. The first step in constructing a unified coherent study of learning is to define learning in a way that is consistent both with the values and intentions of museums and with the understandings we can gain from sociocultural theory. In order to study learning, how it occurs, and what actually develops, we need an operational definition. In the field of education, for example, learning can be considered as knowledge acquisition, conceptual change, or meaning-making; but until those concepts are each operationalized by defined and measurable ideas and terms, we have no idea how to interpret a particular study or finding. For our pragmatic and operational definition of learning in museums we have chosen conversational elaboration.
We focus on conversational elaboration because it is a naturally occurring and meaningful process and product of the museum experience. By conversation we mean a particular kind of talk that occurs within a group (and also in the sense of within an individual) both during and surrounding a museum visit. This kind of talk focuses on the meaning and experiential nature of the museum but excludes planning and management discussions. Conversation is important because, as Harrison White has indicated, it is a reflection of the "inter-twining of the social with cultural processes." (White, 1995 p.1). Sociocultural theorists such as Rogoff (1990) and Wertsch (1997) have emphasized that this intertwining is the primary activity through which knowledge is co-constructed and through which new knowledge is appropriated, both across and within generational boundaries. In addition, although the museum experience can be formal, it is most often used as a time for recreation and informal learning. Conversation is a natural process and consequence of an enjoyable, shared experience between people visiting as a group. Finally, just as cultural history can be intertwined with social activity, so can personal history; thus conversations are one of the primary means through which past experience is incorporated into current activity, and current experiences are carried forward to shape future activity.
Our interest in conversational elaboration stems from the expectations that the kinds of conversations about a particular phenomenon (e.g., waves, glass and steel, impressionism, puppets, medicinal herbs, nocturnal animals) that occur before a museum visit would be more impoverished and less detailed than after such a visit. Specifically, we assume that during and after viewing or participating in an exhibit, a coherent conversational group (CCG) would expand upon the particular elements about which they conversed (i.e., they would refer to more items); would include greater detail, in an analytic sense, about their observations and experiences; would connect or synthesize one element more extensively to other elements both in and outside the exhibit; and would increase the level of explanation of phenomena that they share amongst themselves. Conversation, of course, is much more than these four components of listing, analyzing, synthesizing, and explaining. Conversation includes emotive sharing, group solidification moves, rehearsal for group development, and the development of social roles. However, we believe that the conversational elements that we have identified will be sufficiently powerful that they can encompass other features while at the same time allowing us to scale the conversations accurately enough to permit comparisons across museum settings.
There is a premise in the literature that conversations contribute to learning, with position pieces that call for efforts to promote conversation in museums (e.g., Leichter, Hensel, & Larsen, 1989; Sakofs, 1984) and research that targets certain aspects of conversation. Museum research that has focused on conversations has tended to emphasize the amount of talk as a simple indicator of learning or has attempted to classify types of talk in fairly broad terms (e.g., content-based versus management, questioning versus explanatory, technical versus interpretive) without building a solid link between the kind of talk and the kind of learning process that is supports (e.g., Diamond, Smith, & Bond, 1988; Flanders & Flanders, 1976; Rosenfeld & Terkel, 1982). Some studies have traced conversations in non-museum settings (e.g., Callanan & Oaks, 1992) while others have considered only a single, narrow category of museum conversation, such as "text-echo" in label reading (McManus, 1989). A notable exception is the recent work conducted by the Philadelphia/Camden Informal Science Education Collaborative (PISEC), where researchers identified three levels of learning exhibited by visitor groups and found these learning levels to be associated with specific categories of verbal behavior that significantly discriminated between levels (Borun, Chambers, & Cleghorn, 1996). One category of conversation (Comment/Explain), however, proved to be so large both in its scope and in its actual number of instances that it seemed to beg for further examination into the kinds of variation that surely must reside within it. More detailed analysis of items assigned to this category should reveal differences in a host of features (such as depth, complexity, elaboration, specificity, connections to prior knowledge, and links to other exhibits in the museum) that would tell us more about how explanatory conversation functions in museum settings to support learning.
Outside the museum world, researchers in education and psychology have concerned themselves with analyzing discourse in order to understand how such discourse supports learning (e.g., Burton, 1981; Cazden, 1986; Cobb & Yackel, 1996; Forman, 1992) and psychologists have begun to investigate the structure of reasoning in dyadic or group conversations as opposed to traditional analyses of reasoning in individuals (e.g., Grice, 1989; Salmon & Zeitz, 1995; van Eemeren, Grootendorst, & Kruiger, 1987; Walton, 1992). We use the research on discourse and conversation in and out of museums as a springboard from which to launch our own series of investigations.
The research that we are proposing considers conversational elaboration to be one type of social learning that occurs in museums and thus our question becomes:
How does conversation as a socially mediating activity act both as a process and as an outcome of museum learning experiences?
For the purposes of designing a large integrative study of museum learning, we need to establish a theoretical model that will withstand rigorous empirical assessment while encompassing the essential qualitative elements that deeper research indicates are critical to learning in museums. The model we propose considers conversational elaboration as the simultaneous consequence of the following three dimensions: the sense of identity shared by a cohesive conversational group (CCG) visiting the museum, the explanatory engagement of the group while visiting, and the structure of the museum learning environment in terms of the mediating explanatory support that it offers to the group. We can express this unified model of museum learning as a general regression equation:
The comprehensive study we propose will allow a systematic investigation of how the three dimensions simultaneously influence conversational elaboration during and after museum visits. Each dimension in this model is in turn made up of a system of measures of the dimension. The study will investigate the ways in which the relations among dimensions vary across six types of museums and among different kinds CCGs.
While the linear regression model proposed above assumes that all of the specified variables are independent and additive, other analytic models permit us to specify more complex relations between the dimensions. For example, as shown in the figure below, we can express the dimensions in terms of a structural equation model (Goldberger & Duncan, 1973).
In this structural equation model, museum learning is seen as a direct consequence of the explanatory engagement of the visiting group. Explanatory engagement, in turn, is seen as a consequence of the other two dimensions: the identity of the visiting group and the learning environment. Further, museum learning is seen as being affected by the identity of the group, independent of what the group does or says while visiting the museum. The dotted line represents a possible direct relation between the learning environment and museum learning, independent of what the group does or says while visiting the museum. We will explore the implications of both linear regression and structural equation models further in the methodology section.
We propose to conduct such a comprehensive study. In order to develop theoretically grounded and valid measures for the comprehensive study we will conduct a series of validation studies. Each of these studies will be publishable in their own right but together they will form the building blocks for measures, sampling procedures, and logistical support systems for the comprehensive study. This comprehensive study will allow a systematic investigation of how all the dimensions simultaneously influence conversational elaboration during and after museum visits. Each dimension will itself be made up of multiple measures of the domain. The comprehensive study will also investigate the ways in which the relations among dimensions vary across six types of museums and among different kinds CCGs.
Identity. The first construct in our model is identity. Identity is reflective of motivation and interest as well as prior knowledge relevant to the content of museum exhibitions. Identity is the filter through which museum experiences are interpreted and it influences future museum participation. The identity dimension emphasizes inter- and intra- personal histories and relationships. This dimension of the model concerns the ways that people see themselves as learners of history, art, or science and the ways in which their interests and knowledge about those areas are expressed. How and what cohesive groups learn in museums will depend in part on their motivations (why, in their own minds, they have come there), their expectations, and their interests or enduring propensities to engage with a topic (Hood, 1989; Salmi, 1993).
Identity is made up of the specific and general experiences and knowledge of the group. For example, if a family were to visit the Heinz Pittsburgh Regional History Center's glass exhibit ("Glass: Shattering Notions"), it would be significant to their learning if one of their relatives had worked in the Pittsburgh Plate Glass factories, if a family member blew glass as a hobby, or if someone in the family cared a lot about the history of stained glass. We would consider these kinds of personal knowledge and family history to be local and specific as contrasted to the general knowledge people have about industrialization in Pittsburgh at the beginning of the century or the general experience of frequent visits to museums. Family identity, then, is made up of a collection of specific life events and experiences that are shared and molded into a sense of shared family consciousness. This sense of family identity is bounded by what is unique about that group in contrast to other groups and what is common within the group (Levine, 1998). This sense of identity also forms the groups' local expectations, purposes, and intentions (Hayward & Brydon-Miller, 1984; Jensen, 1994; Leichter, Hensel, & Larsen, 1989).
Previous visitor studies research has explored different aspects of the identity dimension. Draper (1984) found that visitors who came to the museum as part of a recognizable social group enjoyed a richer experience and had a greater potential for learning than did solitary visitors. Korn (1988, 1995) and Tilden (1977) note the importance of visitor expectations in framing experiences in particular museum settings. Likewise, Doering and Pekarik (1996), Falk and Dierking (1992), and Falk, Moussouri, and Coulson (1998) have shown that how people see themselves and what they expect to experience affects the experience and perhaps their learning. Similarly, Hein (1998b), Roschell (1995), and Allen (1997), echoing longstanding findings from education, have emphasized the role of prior content knowledge in museum learning. Visitors' interest in a given topic will also affect the depth and richness of their experience (Morrissey, 1991). These variables of identity, motivation, interest, expectation, and prior knowledge are not unrelated, even though the research has sometimes treated them as though they were. For example, in the hypothetical visiting family described above, prior knowledge is a consequence of having worked in a glass factory or of having collected different glass objects, but those experiences are also the source of the family's sense of itself and of its interest in going to see a glass exhibit. Our task, then is to develop a system of measures of identity that reflect the group's prior knowledge and values.
Explanatory Engagement. In the initial MLC proposal, we put forward the theme of interpretation, meaning, and explanation as processes and products of social interaction in museums. The theme reflected the issue of dialectics between curator, institution, viewer, and viewers, acknowledging that meaning is inherently social. In the comprehensive model proposed here, we transform this theme into the dimension of explanatory engagement among members of the CCG. The dimension is revealed through the parts of conversations where CCGs describe exhibits in terms of detailed content, build connections across exhibits, and explore the relationship of the exhibit to previous experiences or anticipated future experiences. This dimension is a place where we intend to tap into ongoing conversation and where we expect conversation to reveal how the process of museum learning unfolds or transpires.
Explanatory Engagement refers to conversation and actions where CCGs connect directly to the exhibit content with or without reference to any mediating devices that may be provided by the museum. Visitors can make meaning from the exhibits themselves directly. For example, the signage in the Mister Roger's Neighborhood traveling exhibit (developed by the Pittsburgh Children's Museum) is frequently directed toward parents, encouraging them to notice and appreciate the skills and talents of their developing children. However, it is altogether reasonable for a family to interact with the objects at each site, talk about the content in deep and meaningful ways, yet not attend to the signs or the explicit agenda. The dimension of explanatory engagement reflects the activity that the CCG brings to each exhibit in terms of language and actions.
Explanatory engagement is a dimension of the model that directly captures what people do during museum visits. Visitor studies and evaluation research have tended to treat this aspect in terms of time spent in various behaviors (Cone & Kendall, 1978; Falk, 1991; Serrell, 1995, 1997). It is logical that if a group or individual spends time doing something they will learn more than if they do not spend time doing it--but what constitutes time well spent and how do groups actively appropriate meanings in the museum? In the comprehensive model we will analyze what people say and do in ecologically valid and connected ways. That is, we will build a dimension that reflects the content of behavior not just the structure of these behaviors.
Learning Environment. The Learning Environment dimension articulates the interrelations between groups and the deliberate mediating devices employed by the museum to support visitor learning about exhibits. This dimension addresses the ways in which the explicit forms of texts, images, models, and activities serve as mediators and affordances for learning (Harvey, 1995; Schauble & Bartlett, 1997). These affordances would include, for example, the use of different signage strategies such as question-posing (Bitgood & Patterson, 1993; Litwak, 1996), differing roles for docents (Birney, 1988; Flanders & Flanders, 1976; Gilman, 1923; Martinello, Cook & Wiskemann, 1983; Stronck, 1983), the utilization of actors (Alderson & Low, 1996; Alexander, 1979), or the contrast of audio-tours to static audio presentation (Hitzemann, Mellish, & Oberlander, 1997; Screven, 1975), and other devices that explicitly pose questions, provide explanations, or portray curatorial premises to the visitors. This dimension would also include the degree to which the museum makes explicitly available information from "behind the wall" (Silverman, 1993), thus opening up the debates and controversies within a field (such as the debates aired in the Museum of the American Indian). How visitors make use of this support and connect with controversies as they appropriate the meanings into their ongoing conversations defines this learning environment dimension.
The distinction between explanatory engagement and learning environments is to some extent rooted in debates about how constructivism and didactics play out in museum learning (Bitgood, 1997; Hein, 1998a). A traditional constructivist approach suggests that the important aspects of meaning-making arise from the interaction of visitors' prior knowledge with the content of a particular exhibit (Gelman, Massey, & McManus, 1991; Griffin & Symington, 1997; Hein, 1995, 1998b; Hilke, 1988; Jensen, 1994; Silverman, 1995). An alternative view is that the process of negotiating and appropriating meaning is a result of an explicit or implicit dialogue with the artifacts and the curatorial premises (Bitgood, 1990, 1993; Bitgood & Patterson, 1993; Bitgood, Patterson, & Benefield, 1988; Bitgood, Pierce, Nichols, & Patterson, 1987; Screven, 1974, 1975, 1976; Shettel, 1973). We believe that both approaches have elements of truth. Thus, by creating these two dimensions, we are able to advance these conversations within the museum community.
Phase 2 culminates in the conducting of a comprehensive study of museum learning. The study will permit a test of the simultaneous impact of the three dimensions on museum learning as estimated by conversational elaboration. As we describe here, the study includes multiple kinds of visiting groups at six kinds of museums.
Sampling and Population. We will maximize flexibility of analysis by constructing a sample that systematically draws on three populations within each museum type. The population types are visiting family groups, organized peer groups, and a random sample of visitors that are reflective of the normal demographics for each particular museum. Sampling in this way will permit us to answer questions about how families, in particular, learn in the different types of museums; how organized groups of children or adults learn in the different types of museums; or, finally, how the "typical" visiting group to each museum learns.
At a minimum we will draw a sample of 30 CCGs for each museum type, for a minimum sample size in the comprehensive study of 180 CCG units. This sample will be large enough so that we can both answer questions about how particular museum types affect learning. For example, museum types can be collapsed so that data for all interactive museums are combined providing a sample size of between 60 CCGs (Children's museums and Science museums) and 90 CCGs (including other interactive museums such as Natural History). This will require field data collection of at least 15-20 days per museum, for a total of at least 90-120 person days of field data collection. We will collect data in a balanced fashion across days of the week and times of the day.
The procedure for constructing the three CCG populations for each museum is as follows. For each museum type we will randomly sample family groupings (using a table of random numbers) until we have 10 that have completed all critical events. As described below, the critical events involve orienting conversations with researchers, unobtrusive videotaped data collection during the visit, and interviews that support conversational elaboration at the conclusion of the visit. Because we expect that some CCGs who initially agree to participate may, during their visit, decide not to go to the relevant exhibits or not to participate in the concluding discussion, we will need to oversample.
Within each museum, the second sample will consist of organized visiting groups. We will sample 10 units of organized visiting groups (where the specific group targeted is no larger than 5 to 8--although the entire organized group may be much larger). By organized visiting groups we are referring to school groups, student teacher groups, elder groups, and common interest groups such as garden clubs. Because such groups often make advanced arrangements with the museum, we will recruit by randomly sampling from the reservation list and obtaining appropriate advanced permission where necessary. In all other respects, the procedures will be the same.
Since social configurations of people in the normal visiting population differ from museum type to museum type, it is important to also randomly sample from the total visitor population. For example, in an art museum single visitors, groups of adults, and organized elder groups are more common than groups of families with young children, while the opposite may be true of a children's museum. In order to be valid for the purposes of the museum type itself, we will also construct a 10 CCG representative sample that reflects the base visiting rates of different groups. Thus, the representative sample could include individuals, family groups, or peer groups. We will employ appropriate methods to ensure that the sample accurately reflects typical demographics for each museum type.
Procedures for selecting exhibits and developing research protocols. In each museum we will identify a particular set of target exhibits around which we will conduct our study. In order to obtain appropriate information with respect to the learning environments dimension, we will need to select exhibits about which there is a record of the curatorial and design decisions and intentions. We will also work with each museum to select exhibits that they think are important and most directly embody their museum philosophies. Developing the specific set of target exhibits is one of the primary functions of the validation studies.
There will be at least three moments when researchers collect videotaped data from each CCG in the comprehensive study. First, we will conduct an initial conversation in which we obtain permission, explain the procedure, obtain identity information which might include prior knowledge and purposes for the visit, and engage the CCG in a supported pre-visit conversation focusing on content relevant to the particular target exhibits. Second, we will monitor the spontaneous conversations and actions of the CCGs while engaged with the target exhibits. Third, we will support the final conversation in which we tap the extent of their conversational elaboration. For some groups, such as families, we may also conduct a fourth follow-up telephone interview.
Because we are sensitive to issues of priming and of interfering with the natural activities of the CCGs, we will need to explore the consequence of different levels of explicitness in the researchers' interaction with the CCGs. The particular procedure we adopt will reflect the findings of a series of validation studies where different procedures are piloted with different populations of visitors in different locations. For example, one technique for the final conversation might be to invite visiting groups to the museum cafe and simply ask them to engage in an undirected discussion of the visit. Another technique might be to interrupt the visit as soon as the CCG has disengaged from the last target exhibit in order to conduct a semi-structured prompted conversation, something closer to a semi-structured interview. The validation studies are necessary because, although these and other techniques may have been employed in previous visitor studies research, the complexity and size of the comprehensive study requires that we systematically compare the advantages and disadvantages of various approaches with respect to the depth and completeness of conversational elaboration.
Each moment of data collection will be videotaped. Videotapes will be segmented, combined, copied, transcribed, and coded. We expect that direct processing of the data for each CCG unit will take approximately 20 hours, excluding data collection time, coding scheme development time, and analysis time. This represents, for the comprehensive study alone, about 4000 person hours of direct processing for the 180 CCGs.
Analyses. Based on the visitor studies literature, the evaluation literature, and learning theory in general, we expect that each of the three dimensions in the model will correlate positively with increased learning in the form of conversational elaboration. Thus, if we had a perfect measure of each of the three dimensions, we would see a positive regression coefficient for each dimension with museum learning. However, as we pointed out previously, this may not reflect the complex nature of learning in museums.
The regression logic (both linear and structural equation) will allow us an overarching structure to simultaneously identify the direct, indirect, and interactive effects of a number of important factors on museum learning. The logic will allow us to directly test for interactions between factors that have often been considered independently. We will be able to look for interactions between some subset of factors that will directly test the conditions under which particular types of museum intervention support particular types of learning.
After the raw data has been coded and used to construct values for each CCG unit on each dimension, we will carry out two sets of analyses: simple linear regressions and structural equation analyses. For the linear regression we will run an analysis of all CCG's (180-200) across all museum types (6). This will indicate which dimensions in the model contribute significantly to learning. Second, we will run separate analyses for each of the three population types (families, organized groups, and random visitors). This set of analyses will help us to understand whether the pattern of "what matters" is different for different populations. For example, identity issues may be especially important for organized groups but not for families. Third, we will run separate analyses for each of three museum types ( interactive, collection, and living). This will help us understand whether different aspects of the model matter more than others depending on the museum type. For example, it may be that interactivity and group accessibility are very important in science and children's museums but of less importance in art museums. We will follow a parallel procedure for the structural equation models. The structural equation model, remember, allows us to ask how the dimensions relate to each other, not just to the outcomes.
In building the systems of measures for each dimension we will face many technical and conceptual challenges. For example, should the separate measures within a dimension be weighted or summed with unit values? If weighted, should that be determined using factor analytic techniques or by arbitrary assignment? We will make decisions about these and other issues by planning to address them in our validation studies and by employing Exploratory Data Analysis techniques (Tukey, 1977; Leinhardt & Leinhardt, 1980) to carefully inspect the data records for each CCG Unit.
This represents a prodigious amount of work but the final corpus of data will be an enormous resource in and of itself. We will be able to conduct many secondary analyses on it. We will be able to re-analyze video taped behaviors, interviews, and the impact of background histories on the efficacy of targeted exhibits. In addition, if it is of interest, we will be able to make available the entire transcribed, coded, and indexed corpus as a publicly usable data set that other museum researchers could use.
The validation studies should be seen as a set of problem solving, goal directed, qualitative studies that will support the comprehensive study. The underlying strategy is to use the validation studies to gather an extremely rich set of information on a small set of groups so that we can understand what we are specifically losing when we use a more restricted type of measurement or of coding. Each construct in the model poses particular problems that we need to address before we start the major study. For each construct we will conduct several studies and each study is likely to inform more than one construct, however, we give the flavor of the activity below. In addition we will prepare two or three separate stand alone literature reviews. Each review will be targeted to a particular audience and answer a particular question.
We envision conducting validation studies that will inform us about conversational elaboration and about each dimension in the model. A central question that we need to answer about conversational elaboration is how well spontaneous conversation in a museum and conversation in response to a variety of prompts reflect what a family or other group actually thinks, knows, and feels. We need to find a way to get "underneath" the limited verbal behavior in public places so that we can connect observed behavior to general characteristics of the particular group. To do that, we will conduct an ethnographic study of 2-4 families over the next 12 months. Each family will be "followed" as they visit several different museums. We will also "follow" the family into their homes and observe their behaviors as they interact in other conversational and recreational settings. This will allow us to understand their conversational history, the specific goals and agendas they have in multiple situations, and to see them interacting with a variety of museum situations. We will also "measure" them in these settings. That is, we will try out a variety of more intrusive and impersonal measures. By comparing what we know to be the real and more likely situation for the family with the one we "see" when we interview or video tape them we can understand our own measurement shortcomings and correct them. In a measurement sense, we will have two quite different estimates of the family's "true" score (Holland & Rubin, 1982).
Another question that we would need to answer is exactly how well a group-based conversation connects to the individual understandings of each member. Theory suggests that the conversations can potentially go well beyond any one member's understanding (Matusov & Rogoff, 1995; Vygotsky, 1978), but this has not been verified in the museum context. For example, one way to investigate this would be to use an organized group of teachers and obtain estimates of the individual knowledge of the teachers about a topic before and after a visit to a museum such as the Birmingham Civil Rights Institute. We can obtain individual knowledge by having teachers draft lesson plans, construct web diagrams of their own design that reflect important topics of the civil rights era both before and after the visit, and answer a series of question about what they expect to see in such a museum. We can also prompt them to have particular kinds of conversations before and after their visit. The analysis of the recording of these conversations can then be compared to the estimates obtained at the individual level. Such study will allow us to consider both the effectiveness of prompted conversations and the ways in which conversations capture, expand upon, or distort personal knowledge.
Our collection of validation studies will not only help us to understand the nuances of each construct more deeply, it will also allow us to address technical issues. For example, we need to know exactly how disruptive a pre-interview is to normal behavior in the museum. Having a pre-interview is very helpful because we can clearly delineate what has been learned. However, if the pre-interview drastically changes behaviors as CCGs go through the museum, it will produce invalid results with respect to "typical" museum activity. Our validation studies will provide control conditions that will help us understand the extent of such problems. Similarly, the validation studies will provide information about baseline rates of verbal behavior. This is essential because, although prior museum research has provided estimates of time spent in museum activity, it has not provided us with similar baseline information about conversation and conversational content.
As a result of this work we will understand museum learning in the following way. We will have developed conversational elaboration as one meaningful estimate of museum learning: one that exists as a phenomenon before visits, one that exists as a process during visits, and one that is enriched by the visit and continues on after the visit is finished. We will have obtained general quantitative findings that directly address the question of how conversational elaboration is influenced by the three dimensions, the six museum types, and multiple types of visiting groups. We will have constructed qualitative research and literature reviews that provide a strong conceptual basis for relating specific findings to the extant research literatures in sociocultural theory, visitor studies, and the learning sciences. In addition, we will have contributed to the ongoing development of novel, stable, and disseminatable methodologies for conceptualizing, collecting, and analyzing conversations as a process and outcome of learning in the context of museum visits.
We believe that our approach successfully addresses the four problems that we posed at the beginning of the proposal. We have addressed the learning definition problem by suggesting that the social act of conversation be placed at the center of our investigation of learning in a museum. We have addressed the univariate problem by proposing a multivariate system in which the three sociocultural themes are developed into measured dimensions of learning. We have addressed the museum diversity problem by choosing a cross cutting process of learning, conversation and by designing a system of analyses that will permit us to directly ask the question of how similar or dissimilar the dimensions of learning are in different museums. We have addressed the quantitative/qualitative problem by using qualitative studies to ensure the validity and interpretability of our quantitative research.
Abbey, D.S. (1986). The exhibit as educator. Museum, 38(3), 172-174.
Alderson, W. T., & Low, S. P. (1996). Interpretation of historic sites. Walnut Creek, CA: Alta Mira Press.
Alexander, E. P. (1979). Museums in motion: An introduction to the history and function of museums. Nashville, TN: American Association for State and Local History.
Allen, S. (1997). Using scientific inquiry activities in exhibit explanations. Science Education (Informal Science Education - Special Issue), 81(6), 715-734.
Alt, M. (1979). Improving audio-visual presentations. Curator, 22(2), 85-95.
Anderson, D., & Lucas, K.B. (1997). The effectiveness of orienting students to the physical features of a science museum prior to visitation. Research in Science Education, 27(4), 485-495.
Arth, M., & Claremon, L. (1977). The Discovery Room. Curator, 20(3), 169-180.
Balling, J. D., & Falk, J. H. (1980). A perspective on field trips: Environmental effects on learning. Curator, 23(4), 229-240.
Barnard, W., & Loomis, R. (1994). The museum exhibit as a visual learning museum. Visitor Behavior, 9(2), 14-17.
Beer, V. (1987). Great expectations: Do museums know what visitors are doing? Curator, 30(3), 206-215.
Birney, B. (1988). Criteria for successful museum and zoo visits: Children offer guidance. Curator, 31(4), 292-316.
Bitgood, S. (1990). Toward an objective description of the visitor immersion experience. Visitor Behavior, 5(2), 11-14.
Bitgood, S. (1993). What do we know about school field trips? In R. J. Hannapel (Ed.), What research says about learning in science museums (Vol. 2, pp. 12-16). Washington, DC: Association of Science Technology Centers.
Bitgood, S. (1997). The Hein-Miles debate: An introduction, explanation, and commentary. Visitor Behavior, 12(3&4), 3-7.
Bitgood, S., & Cleghorn, A. (1994). Memory of objects, labels and other sensory lessons from a museum visit. Visitor Behavior, 9(2), 11-12.
Bitgood, S. C., & Patterson, D. D. (1993). The effects of gallery changes on visitor reading and object viewing time.. Environment and Behavior (Special Issue), 25(6), 761-781.
Bitgood, S., Patterson, D., & Benefield, A. (1988). Exhibit design and visitor behavior: Empirical relationships. Environment and Behavior, 20(4), 474-491.
Bitgood, S., Pierce, M., Nichols, M., & Patterson, D. (1987). Formative evaluation of a cave exhibit. Curator, 30(1), 31-39.
Boggs, D. L. (1977). Visitor learning at the Ohio Historical Center. Curator, 20(3), 205-214.
Borun, M., Chambers, M., & Cleghorn, A. (1996). Families are learning in science museums. Curator, 39(2), 124-138.
Borun, M., Chambers, M. B., Dritsas, J., & Johnson, J. I. (1997). Enhancing family learning through exhibits. Curator, 40(4), 279-295.
Borun, M., & Dritsas, J. (1997). Developing family-friendly exhibits. Curator, 40(3), 178-196.
Burton, D. (1981). The sociolinguistic analysis of spoken discourse. In P. French & M. MacLure (Eds.), Adult-child conversation. New York: St. Martins Press (pp. 21-46).
Callanan, M.A., & Oakes, L.M. (1992). Pre-schoolers' questions and parents' explanations: Causal thinking in everyday activity. Cognitive Development, 7, 213-233.
Cazden, C. B. (1986). Classroom discourse. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd Ed., pp. 432-463). New York: Macmillan.
Cobb, P. & Yackel, E. (1996). Constructivist, emergent, and sociocultural perspectives in the context of development research. Educational Psychologist, 31(3/4), 175-190.
Cone, C. A., & Kendall, K. (1978). Space, time and family interactions: Visitor behavior at the Science Museum of Minnesota. Curator, 21(3), 245-258.
Diamond, J., Smith, A., & Bond, A. (1988). California Academy of Sciences discovery room. Curator, 31(3), 157-166.
Doering, Z. D., & Pekarik, A. J. (1996). Questioning the entrance narrative. Journal of Museum Education, 21(3), 20-22.
Draper, L. (1984). Friendship and the museum experience: The interrelationship of social ties and learning. Unpublished doctoral dissertation, University of California, Berkeley.
Falk, J. H. (1982). The use of time as a measure of visitor behavior and exhibit effectiveness. Journal of Museum Education: Roundtable Reports, 7(4), 10-13.
Falk, J. H. (1983). Time and behavior as predictors of learning. Science Education, 67(2), 267-276.
Falk, J. H. (1991). Analysis of the behavior of family visitors in history museums: The National Museum of Natural History. Curator, 34(1), 44-50.
Falk, J. H., & Dierking, L. D. (1990). The effect of visitation frequency on long-term recollection. In S. Bitgood (Ed.), Visitor studies: Proceedings of the 3rd annual visitor studies conference (pp. 94-103). Jacksonville, AL: Center for Social Design.
Falk, J. H., & Dierking, L. D. (1992). The museum experience. Washington, DC: Whalesback Books.
Falk, J. H., & Dierking, L. D. (1997). School field trips: Assessing their long-term impact. Curator, 40(3), 211-218.
Falk, J. H., Moussouri, T., & Coulson, D. (1998). The effect of visitors' agendas on museum learning. Curator, 41(2), 106-120.
Flanders, M. P., & Flanders, N. A. (1976). Evaluating docent performance. Curator, 19(3), 198-225.
Forman, E. A. (1992). Discourse, intersubjectivity, and the development of peer collaboration: A Vygotskian approach. In L. T. Winegar & J. Valsiner (Eds.), Children's development within social context (Vol. 1, pp. 143-159). Hillsdale, NJ: Lawrence Erlbaum Associates.
Friedman, A. J. (1995). Creating an academic home for informal science education. Curator, 38(4), 214-220.
Gelman, R., Massey, C. M., & McManus, M. (1991). Characterizing supporting environments for cognitive development: Lessons from children in a museum. In L. Resnick & J. Levine (Eds.), Perspectives on socially shared cognition (pp. 226-256). Washington, DC: American Psychology Society.
Gilman, B. I. (1923). Museum ideals of purpose and method. Cambridge, MA: Harvard University Press.
Goldberger, A. S., & Duncan, O. D. (1973). Structural equation models in the social sciences. New York: Seminar Press, Harcourt Brace Jovanovich.
Goode, G. B. (1891). The museums of the future. In G. B. Goode (Ed.), Smithsonian Institution Report of the National Museum, 1888-1889. Washington, DC: Government Printing Office.
Gottfried, J. L. (1980). Do children learn on field trips? Curator, 23(3), 165-174.
Grice, P. (1989). Studies in the way of words. Cambridge, MA: Harvard University Press.
Griffin, J., & Symington, D. (1997). Moving from task-oriented learning strategies on school excursions to museums. Science Education (Informal Science Education - Special Issue), 81(6), 763-780.
Hackney, S. (1995). Who owns history? Humanities, 16(1), 6-11,50-53.
Harvey, M. L. (1995). The influence of exhibit space design features on visitor attention. Unpublished doctoral dissertation, Colorado State University, Fort Collins.
Hayward, D. G., & Brydon-Miller, M. L. (1984). Spatial and conceptual aspects of orientation: Visitor experiences at an outdoor history museum. Journal of Environmental Education, 13(4), 317-332.
Hein, G. E. (1995). The constructivist museum. Journal of Education in Museums (16), 21-23.
Hein, G. E. (1998a). Faulty logic: A reply to S. Bitgood. Posting on the Museum-ed listserv (museum-ed@freedom.mtn.org), March 31, 1998.
Hein, G.E. (1998b). Learning in the museum. London: Routledge.
Hilke, D. D. (1988). Strategies for family learning in museums. In S. Bitgood, J. Roper, & A. Benefield (Eds.), Visitor Studies -- 1988: Theory, research and practice (Vol. 1, pp. 120-134). Jacksonville, AL: Center for Social Design.
Hitzemann, J., Mellish, C., & Oberlander, J. (1997). Dynamic generation of museum web pages: The Intelligent Labelling Explorer. Archives and Museum Information, 11, 107-115.
Holland, P. W., & Rubin, D. B. (1982). Test equating. New York: Academic Press.
Hood, M. G. (1989). Leisure criteria of family participation and nonparticipation in museums. Marriage and Family Review, 13(4), 151-169.
Jensen, N. (1994). Children's perceptions of their museum experiences: A contextual perspective. Children's Environments, 11(4), 300-324.
Koran, J. J., & Baker, S. D. (1979). Evaluating the effectiveness of field experiences. In M. B. Rowe (Ed.), What research says to the science teacher (pp. 50-76). Washington, DC: National Science Teachers Association.
Koran, J. J., Koran, M. L., Foster, J., & Dierking, L. (1988). Using modeling to direct attention. Curator, 31(1), 36-42.]
Koran, J. J., Lehman, J. R., Shafer, L. D., & Koran, M. L. (1983). The relative effects of pre- and post attention directing devices on learning from a 'walk-through' museum exhibit. Journal of Research in Science Teaching, 20(4), 341-346.
Korn, R. (1988). Self-guiding brochures: An evaluation. Curator, 31(1), 9-19.
Korn, R. (1995). An analysis of differences between visitors at natural history museums and science centers. Curator, 38(3), 150-160.
Leichter, H. J., Hensel, K., & Larsen, E. (1989). Families and museums: Issues and perspectives. Marriage and Family Review, 13(4), 15-50.
Leinhardt, G., & Leinhardt, S. (1980). Exploratory data analysis: New tools for the analysis of empirical data. In D. Berliner (Ed.), Review of Research in Education (Vol. 8, pp. 85-137). Washington, DC: American Educational Research Association.
Levine, L. (1998, April). The search for an American identity. Invited address presented at the annual meeting of the American Educational Research Association, San Diego, CA.
Litwak, J. M. (1996). Visitors learn more from labels that ask questions." Current Trends in Audience Research and Evaluation, 10, 40-51.]
Martin, W. W., Falk, J. H., & Balling, J. D. (1981). Environmental effects on learning: The outdoor field trip. Science Education, 65(3), 301-309.
Matusov, E. & Rogoff, B. (1995). Evidence of development from people's participation in communities of learners. In J. H. Falk & L. D. Dierking (Eds.), Public institutions for personal learning: Establishing a research agenda (pp. 97-104). Washington, DC: American Association of Museums.
Martinello, M. L., Cook, G. E., & Wiskemann, S. (1983). Preparing community volunteers for museum education. Curator, 26(1), 37-57.
McManus, P. M. (1989). Oh yes they do! How visitors read labels and interact with exhibit text. Curator, 32(3), 174-189.
McManus, P. M. (1993). Memories as indicators of the impact of museum visits. Museum management and curatorship, 12, 367-380.
Morrisey, K. (1991). Visitor behavior and interactive video. Curator, 34(2), 109-118.
Perry, D. L. (1993). Beyond cognition and affect: The anatomy of a museum exhibit. In D. Thompson, S. Bitgood, A. Benefeld, H. Shettel, & R. Williams (Eds.), Visitor studies: Theory, research and practice (Vol. 6, pp. 43-47). Jacksonville, AL: Center for Social Design.
Price, S., & Hein, G. E. (1991). More than a field trip: Science programmes for elementary school groups at museums. International Journal of Science Education, 13(5), 505-519.
Reynolds, S. S. (1984). How to unstuff a museum: A preschool teacher's guide. Curator, 27(1), 59-64.
Rogoff, B. (1990). Apprenticeship in thinking. New York: Oxford University Press.
Roschell, J. (1995). Learning in interactive environments: Prior knowledge and new experience. In Falk, J. H., & Dierking, L. D. (Eds.) Public institutions for personal learning: Establishing a research agenda (pp. 37-51). Washington, DC: American Association of Museums.
Rosenfeld, S., & Terkel, A. (1982). A naturalistic study of visitors at an interactive mini-zoo. Curator, 25(3), 187-212.
Sakofs, M. S. (1984). Optimizing the educational impact of a museum tour. Curator, 27(2), 135-140.
Salmi, H. (1993). Science centre education. Motivation and learning in informal education. (Research report 119). Unpublished doctoral dissertation, University of Helsinki, Helsinki.
Salmon, M. H., & Zeitz, C. M. (1995). A method for analyzing conversational reasoning. Informal Logic, 19(1), 1-24.
Sandifer, C. (1997). An examination of time-based behaviors at an interactive science museum: How much learning is really going on? Science Education (Informal Science Education - Special Issue), 81(6), 689-702.
Schauble, L., & Bartlett, K. (1997). Constructing a science gallery for children and families: The role of research in an innovative design process. Science Education (Informal Science Education - Special Issue), 81(6), 781-793.
Schauble, L., Beane, D. B., Coates, G. D., Martin, L., & Sterling, P. (1996). Outside the classroom walls: Learning in informal environments. In L. Schauble & R. Glaser (Eds.), Innovations in learning: New environments for education (pp. 5-24). Hillsdale, NJ: Erlbaum.
Screven, C. G. (1974). The measurement and facilitation of learning in the museum environment: An experimental analysis. Washington, DC: The Smithsonian Press.
Screven, C. G. (1975). The effectiveness of guidance devices on visitor learning. Curator, 18(3), 219-243.
Screven, C. (1976). Exhibit evaluation: A goal-referenced approach. Curator, 19(4), 271-289.
Serrell, B. (1995). The 51% solution research project: A meta-analysis of visitor time/use in museum exhibitions. Visitor Behavior(3).
Serrell, B. (1997). Paying attention: The duration and allocation of visitor's time in museum exhibitions. Curator 40(2), 108-125.
Shettel, H. H. (1973). Exhibits: Art form or educational medium? Museum News, 52(1), 32-41.
Silverman, L. (1993). Making meaning together: Lessons from the field of American history. Journal of Museum Education,18(3), 7-11.
Silverman, L. H. (1995). Visitor meaning-making in museums for a new age. Curator, 38, 161-170.
Stevenson, J. (1991). The long-term impact of interactive exhibits. International Journal of Science Education, 13(5), 521-531.
Stronck, D., R. (1983). The comparative effects of different museum tours on children's attitudes and learning. Journal of Research in Science Teaching, 20(4), 283-290.
Taylor, S. M. (1986). Understanding processes of informal education: A naturalistic study of visitors to a public aquarium. Unpublished doctoral dissertation, University of California, Berkeley.
Tieken, N. (1991). Take a long look. Museum News, 70(3), 70-72.
Tilden, F. (1977). Interpreting our heritage. Chapel Hill: University of North Carolina Press.
Tukey, J. W. (1977). Exploratory data analysis. Reading, MA: Addison-Wesley.
Van Eemeren, F., Grootendorst, R., & Kruiger, T. (1987). Handbook of argumentation theory. Dordrecht, Netherlands: Foris Publications.
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
Walton, D. (1992). Plausible argument in everyday conversation. Buffalo: State University of New York Press.
Wendling, L. M. (1991). The critical thinking level and teaching behaviors of museum docents: Assessment, interrelationships and implications for the development of children's thinking. Unpublished doctoral dissertation, University of Washington, Seattle.
Wertsch, J. V. (1991). Voices of the mind: A sociocultural approach to mediated action. Cambridge, MA: Harvard University Press.
Wertsch, J. V. (1997). Narrative tools of history and identity. Culture and Psychology, 3(1), 5-20.
White, H.C. (1995). Where do languages come from? - Switching talk. Unpublished manuscript, Columbia University, New York.