Review of the Efficacy of the Picture Exchange Communication

J Autism Dev Disord (2009) 39:1471–1486 DOI 10. 1007/s10803-009-0763-y ORIGINAL PAPER A Review of the Ef? cacy of the Picture Exchange Communication System Intervention Deborah Preston ? Mark Carter Published online: 3 June 2009 O Springer Science+Business Media, LLC 2009 Abstract The Picture Exchange Communication System (PECS) is a communication program that has become widely used, especially with children with autism. This paper reports the results of a review of the empirical literature on PECS. A descriptive review is provided of the 27 studies identi? d, which included randomized controlled trials (RCTs), other group designs and single subject studies. For 10 appropriate single subject designs the percentage of nonoverlapping data (PND) and percentage exceeding median (PEM) metrics were examined. While there are few RCTs, on balance, available research provides preliminary evidence that PECS is readily learned by most participants and provides a means of communication fo r individuals with little or no functional speech. Very limited data suggest some positive effect on both socialcommunicative and challenging behaviors, while effects on speech development remain unclear.Directions for future research are discussed including the priority need for further well-conducted RCTs. Keywords Picture exchange communication system A Augmentative and alternative communication A Autism Introduction Serious de? cits in communication form part of the primary diagnostic criteria for autism (American Psychiatric Association 2000). It has been estimated that around D. Preston A M. Carter (&) Macquarie University Special Education Centre, Macquarie University, Sydney, NSW 2109, Australia e-mail: mark. [email  protected] edu. au one-third to one-half of children and adults with autism do not have unctional speech (Mirenda 2003). Such individuals may show only pre-intentional communication, such as reaching for a desired item, or communication may demonstrate intent through behaviors such as alternating eye gaze, and conventional gestures such as pointing (Yoder et al. 2001). Communication may also take the form of challenging behaviors (Mirenda 1997). When speech does develop it may be limited mainly to unusual or echolalic verbalizations (Paul 2005). Individuals with serious developmental disabilities other than autism may also fail to develop speech and language skills (Westling and Fox 2004).In order to help develop communication skills, various forms of augmentative and alternative communication (AAC) have been developed. These include the use of manual signs (e. g. , Layton 1988; Yoder and Layton 1988), voice output communication devices (VOCAs) (reviewed by Lancioni et al. 2001), and various picture-based systems (Keen et al. 2001; Sigafoos et al. 1996). The Picture Exchange Communication System (PECS) is a picturebased system developed by Bondy and Frost (1993, 1994) to help young children with autism acquire functional communication sk ills. PECS appears promising for several reasons.First, it avoids dif? culties inherent in other systems by requiring very few prerequisites; in fact the only prerequisite is that the individual can clearly indicate (e. g. , by reaching for an item) what he or she wants, in a way that can be shaped into exchanging a physical symbol such as a picture (Bondy and Frost 2002). Other skills such as eye contact, motor or verbal imitation skills, the ability to sit quietly in a chair, match-to-sample skills, picture discrimination, or the ability to follow verbal prompts are not necessary (Bondy and Frost 1994, 2002), at least at the earliest program stage. 23 1472 J Autism Dev Disord (2009) 39:1471–1486 Second, the ? rst skill taught in PECS is requesting. Requesting has often been targeted in early instruction of individuals with developmental disabilities due to motivational considerations (Reichle and Sigafoos 1991). In relation to PECS, it is argued that individuals with autism in particular are less likely to be motivated by the social consequences of labeling or commenting (Bondy and Frost 1995) and more likely to be motivated by requesting and immediately obtaining a speci? c, typically concrete, desired item (Bondy and Frost 1994).Third, PECS systematically addresses the issue of spontaneity, which has often been reported as problematic in individuals with autism spectrum disorders (ASD) (Chiang and Carter 2008; Koegel 2000). Rather than being dependent on a partner to establish a communicative exchange, or requiring a partner to watch for the learner to point to a picture board or generate a manual sign, which could easily be missed, PECS speci? cally teaches the individual to approach the partner and gain their attention by putting a picture symbol into their hand.Fourth, picture symbols can be highly iconic, closely resembling their referents (Ganz and Simpson 2004; Mirenda 2003). Consequently, they may be easily recognized by the learner (Ganz and Simpson 2004) and are more recognizable by communicative partners than some other systems, such as manual signs (Lancioni et al. 2007). The PECS protocol begins with a reinforcer assessment through which the trainer determines an ordered list of reinforcers for the individual (Bondy and Frost 1998). This is followed by six phases, which are brie? overviewed in Table 1. For each phase, the criterion for successful completion is 80% or more trials successful without prompting in a 10-trial block (Charlop-Christy and Jones 2006). The randomized control trial (RCT) is recognized as providing the gold standard for evaluating clinical interventions in areas such as medicine and education (Evidence-Based Medicine Working Group 1992; What Works Clearinghouse 2006) and ideally evaluations would be limited to such evidence (Carter and Wheldall 2008).Such designs, however, are relatively rare in educationrelated areas (Carter and Wheldall 2008) for a variety of reasons and clinicians must oft en look to a second line of evidence to inform decision-making. Single subject quasiexperimental designs employ repeated measures of the dependent variable over time with a single or small number of participants. Although not offering the standard of evidence of RCTs, the best of these designs are considered capable of effectively controlling major threats to internal validity and strong conclusions about causal inference can be drawn (Campbell and Stanley 1963; Horner et al. 005). Similarly, high quality quasi-experimental group designs, speci? cally those involving non-equivalent groups with pre-test matching, are generally considered to be interpretable (Flay et al. 2004). The weakest level of evidence is provided by pre-experimental designs, such as single group pre-test—post-test studies, where few threats to internal validity are controlled and causal inference cannot be inferred with any degree of con? dence (Campbell and Stanley 1963). Early papers on PECS were largel y descriptive with limited data (e. . , Bondy and Frost 1993, 1994, 1995, 1998), or presented outcome data without control (e. g. , Schwartz et al. 1998). In recent years, however, a number of interpretable group studies (e. g. , Yoder and Stone 2006b) and single subject studies (e. g. , Charlop-Christy et al. 2002; Tincani 2004) have been published. Lancioni et al. (2007) reviewed the use of PECS and VOCA (voice output communication aids) for request making in individuals with developmental disabilities. In addition to studies following the PECS protocols of Frost and BondyTable 1 Summary PECS stages Phase Teaching target I II Make requests through picture exchange Persistence in initiating communication Discrimination between symbols Introduction of sentence structure Answering question with a request Commenting Description Second person acts as a prompter from behind learner; when learner reaches for the desired item, physically prompts to exchange picture; prompts faded as quick ly as possible Communicative partner moves gradually further away; picture is also gradually moved further away; number of communicative partners increased; ‘‘Communication book’’ is introduced; range of items requested is increased, but only one picture and item is available at a time Initially, two pictures are presented (highly preferred and neutral or disliked); more pictures are added; later, more than one preferred item is offered at a time; periodic correspondence checks are carried out to check accuracy at discrimination Taught to use a sentence strip, placing an ‘‘I want’’ as well as the symbol; requests can also be expanded with attributes such as color or size, e. g. ‘‘I want red playdough’’ Taught to answer question ‘‘What do you want? ’’ Taught to respond to other simple questions such as ‘‘What do you see? ’’; gradually, more spontaneous com menting is developed III IV V VI 123 J Autism Dev Disord (2009) 39:1471–1486 1473 (1994, 2002), they also included studies employing ‘‘any conventional use of pictorial material as a way of making requests for preferred items’’ (p. 4).Thus, studies that did not follow PECS protocols, and in which the participants pointed to a picture rather than actually exchanging it, were included (e. g. , Dyches et al. 2002; Keen et al. 2001). In addition, no attempt was made to quantify the data obtained from the studies to evaluate either the overall ef? cacy or effectiveness of the approach or the effect of relevant variables on outcomes. To date, no comprehensive review of empirical literature speci? cally examining the PECS approach of Frost and Bondy (1994, 2002) appears to have been published. The present paper looks speci? cally at studies of PECS intervention as described by Frost and Bondy (1994, 2002).In the absence of a substantial number of gold-stand ard RCT studies that would allow a conventional meta-analysis, a broader approach to evaluation of the research was undertaken. This review is intended to examine the extant empirical research on PECS, with speci? c consideration of the research designs employed and, consequently, the strength of conclusions that can be drawn. excluded (e. g. , Son et al. 2006). One article in which previously taught communication using the PECS program was compared to facilitated communication (Simon et al. 1996) was excluded because there was no PECS intervention during the study. The study of Rosales and Rehfeldt (2007), in which the ? rst three phases of PECS was taught prior to the experiment, was lso excluded because no data on the results of the PECS training was provided. Analysis A summary of each study was prepared including participants, research design, treatment and duration of study, outcomes investigated, setting, PECS version and phases implemented, and a summary of the outcomes, as well as inter-observer and procedural reliability, social validity, maintenance and generalization data. Ages of participants were coded into 5, 5–8, 9–17, or over 18 years. Four categories of dependent variable were identi? ed: PECS exchanges (number or percentage of independent exchanges), speech or vocalization, social-communicative behaviors, and undesirable behaviors.As the majority of studies employed single subject designs, they were coded for quality using an adaptation of the guidelines for single subject research presented by Horner et al. (2005). These indicators addressed several areas: adequacy of participant and setting descriptions; dependent variables; independent variables; baseline; experimental control/internal validity; external validity; and social validity. A total of 10 points were allocated to each area with the exception of external validity, which was allocated 5 points in recognition of the inherent limitations of single subject designs in th is regard. Thus, studies were rated on a scale from 0 to 65, with higher scores indicating greater quality.Details of the criteria are included in the Appendix. Traditionally, single subject studies have been interpreted by visual inspection of graphed data (Reynhout and Carter 2006). More recently, attempts have been made to quantify results of these studies objectively, and to provide reliable data summaries for evaluating evidence-based interventions (Parker et al. 2007). The most commonly used of the resulting statistical indices is the percentage of non-overlapping data (PND) (Scruggs et al. 1987). The PND is the percentage of treatment data points that are above (or below when behavior decrease is targeted) the highest (or lowest) baseline data point.Scruggs and Mastropieri (1998) provided guidelines for the interpretation of PND: scores between 91 and 100 indicate highly effective interventions, between 71 and 90 effective interventions, between 51 and 70 questionable interve ntions, and 50 or below ineffective interventions. Whilst PND has been criticized on a number of grounds (e. g. , Allison and Method Search Strategy Empirical studies using PECS were identi? ed through computerized searches of A? Education, British Education Index, ERIC, Expanded Academic ASAP, Linguistic and Language Behavior Abstracts, PsycINFO, PubMed and ScienceDirect, using the descriptors ‘‘PECS’’ or ‘‘Picture Exchange Communication System’’. In addition, manual searches of the reference lists of articles identi? ed were carried out to locate further studies.Studies were included if they: (1) (2) were journal articles in English from 1992 to July 2007; used PECS (Bondy and Frost 1994; Frost and Bondy 1994, 2002) as whole or part of an intervention strategy as indicated by reference to program documentation and description of implementation; presented group or individual data on the results of the intervention. (3) Articles th at referred to PECS but did not follow Frost and Bondy’s protocol were excluded. For example, Dooley et al. (2001) used a ‘‘PECS-based schedule board’’ (p. 58) but no actual picture exchange. In addition, articles that used a picture exchange system but did not speci? cally stated that the PECS protocols (Bondy and Frost 1994; Frost and Bondy 1994, 2002) were employed were 123 1474 J Autism Dev Disord (2009) 39:1471–1486 Gorman 1993; Salzberg et al. 1987; White 1987), it is nevertheless the most widely used statistic for quantifying data from single subject studies (e. g. Bellini and Akullian 2007; Erion 2006; Lee et al. 2007; Reynhout and Carter 2006; Xin et al. 2005). A particular disadvantage of PND is that if any baseline data point has reached the ceiling or ? oor level of the measurement scale, the calculated PND is 0%, even if visual inspection indicates a treatment effect (Ma 2006). Ma (2006) has suggested an alternative, the percentag e of data points exceeding the median (PEM). The PEM is the percentage of treatment data points that are above (or below when behavior decrease is targeted) the median baseline data point. There is also evidence that PEM may correlate better with author judgments of program ef? cacy than PND (Ma 2006).Nevertheless, PND is by far the most widely used metric for summarizing single subject studies and comparative data are available on a range of interventions. The application of the PEM statistic is very limited to date but, given its potential advantages, it was decided to calculate both PEM and PND values in the current review. It has also been argued that con? dence in ? ndings from analysis of single subject studies may be strengthened if multiple approaches to synthesis converge on similar conclusions (Smoot et al. 1990). PND and PEM statistics were calculated for all single subject studies with graphed data including a baseline and intervention phase. Changing conditions (i. e. , PECS phase changes), were coded as part of the ‘‘intervention’’ phase.Metrics were initially calculated for treatment data only and then for all intervention data, including treatment, maintenance and generalization phases. The PND statistic was calculated for each study using the pooled number of non-overlapping data points across all subjects and categories of dependent variable (PECS exchanges, speech/ vocalization, social-communicative behaviors, undesirable behaviors). In addition, a PND statistic was calculated for each participant and for each category of dependent variable within relevant studies. Similarly, PEM statistics were calculated using the pooled number of data points exceeding, or below when appropriate, (i. e. , for undesirable behavior) the baseline median.In cases where the exact value of data points on a graph was dif? cult to determine, a copy of the graph was obtained from a Portable Document Format copy of the article or a good qua lity digital scan. Subsequently, numeric data were extracted using the Digitizelt (Bormann 2003) computer software. Inter-Rater Reliability PND and PEM values were independently calculated by the ? rst and second authors for ? ve randomly selected single subject studies (50% of studies for which calculation was possible). Values were calculated for each graph that included a baseline and time-series data. Where more than one panel was shown on the same graph (e. g. multiple baseline, alternating treatments), a value was calculated for each panel. For each panel, reliability was calculated by dividing the lower percentage value by the higher percentage value and multiplying by 100 (i. e. , if both raters agreed on the percentage value, the reliability was 100%). The same ? ve studies were independently rated for quality indicators by the ? rst and second authors. Inter-rater reliability was calculated by dividing the number of agreements by the total of agreements and disagreements, and multiplying by 100. Inter-rater reliability for both PND and PEM was 100% for 54 panels and over 90% for the remaining three panels with an overall mean agreement of 99. 8%.There were discrepancies in only three graphs; the majority of these related to determining how many data points were present in very small ? gures. Inter-rater reliability for quality indicators was 97. 5%. Results A summary of the participants, research design, inclusion of maintenance or generalization data, PECS phases taught, and outcomes examined is presented in Table 2. Research Design The early studies (Bondy and Frost 1993, 1994, 1998; Schwartz et al. 1998) were all reports or program evaluation data without adequate experimental control, as were two later studies (Liddle 2001; Webb 2000). Malandraki and Okalidou (2007) used a case study.Magiati and Howlin (2003), in their pilot study, used a pre-PECS treatment measure plus three measures over time, with data mainly from teacher ratings. All of these studies can be considered pre-experimental. Of the 14 single subject studies, 4 used alternating treatments. Adkins and Axelrod (2001), Chambers and Rehfeldt (2003) and Tincani (2004) compared PECS and manual signing, while Bock et al. (2005) compared PECS and VOCA (voice output communication aid). Four studies (Charlop-Christy et al. 2002; Rehfeldt and Root 2005; Tincani et al. 2006, Study 1; Yokoyama et al. 2006) used a multiple baseline across participants, while two (Frea et al. 2001; Kravits et al. 2002) used a multiple baseline across settings, one (Marckel et al. 006) used a multiple baseline across descriptors taught, and one (Cummings and Williams 2000) used a multiple baseline across activities. Two studies (Stoner et al. 2006; Tincani et al. 2006, Study 2) used an ABAB design, while one (Ganz and Simpson 123 Table 2 Summary of Studies Ages Dependent variable Research design Maintenance (M) PECS or generalization (G) Phases data I–III I–III I–III Iâ⠂¬â€œIV I–III I–III I–III I–VI III Picture exchange, sign Picture exchange, VOCA Picture exchange Picture exchange, speech Picture exchange, speech Speech Social/communicative Picture exchange, sign language Speech, social, behavior (variation) Authors Participants DiagnosisAdkins and Axelrod (2001) – 35 Autism Autism Autism Autism Autism 1 ‘‘autistic characteristics’’ Autism 2 autism, 3 PDD 3–5 years Single-subject (multiple baseline) – 3–12 years Single-subject (multiple baseline) M,G 19–40 years Single-subject (alternating treatment) G 3–7 years 3–7 years Non-equivalent control group Non-equivalent control group – G 32 months Program evaluation – 6 years Program evaluation – 6 years– adult Program evaluation – 4 years Single-subject (alternating treatment) G 1 PDD 7 years Single-subject (alternating treatment) G Bock et al. (2005) 6 Bondy and F rost (1993) 74 Bondy and Frost (1994) 85 Bondy and Frost (1998) 1 J Autism Dev Disord (2009) 39:1471–1486 Carr and Felce (2007a) Carr and Felce (2007b) 10 41 Chambers and Rehfeldt (2003) 4 Charlop-Christy et al. (2002) 3Cummings and Williams (2000) Autism Autism Autism Autism 14 ASD Autism or ASD Autism Autism – 16 autism/PDD-NOS – 3–6 years 22–31 years 20–34 years 4–5 years 10 years Case study Single-subject (multiple baseline) Single-subject (multiple probe) Program evaluation Single-subject (ABAB) Single-subject (alternating treatment) 9–11 years Single-subject (multiple baseline, ABAB) 5–12 years Single group School Program evaluation 6 years Single-subject (multiple baseline) 3–7 years 4–11 years Single-subject (changing criterion RCT G M – – G M G – G G G G 4 years Single-subject (multiple baseline) – 5 Picture exchange, other I–III I–IV I–VI I–I II I–VI I–VI I–VI Extension I–III I–IV I–IV I–III I–IV Picture exchange, behavior Picture exchange, speech Picture exchange, speech, ADOS-G Picture exchange, speech, social Picture exchange Picture exchange, speech, other Picture exchange Picture exchange (improvised requests) Picture exchange, other Picture exchange, speech, other Picture exchange Picture exchange Picture exchange, sign, speech PECS, speech 1475 Frea et al. (2001) 1 Ganz and Simpson (2004) Howlin et al. (2007) 3 84 Kravits et al. (2002) 1 Liddle (2001) 21 Magiati and Howlin (2003) 34Malandraki and Okalidou (2007) 1 Marckel et al. (2006) 2 Rehfeldt and Root (2005) 3 Schwartz et al. (1998) 31 Stoner et al. (2006) 5 Tincani (2004) Autism 2 1 autism, 1 PDD-NOS 5–6 years 123 Tincani et al. (2006) 3 1476 J Autism Dev Disord (2009) 39:1471–1486 Maintenance (M) or generalization (G) data M,G 2004) used a within subjects changing criterion design. In seve ral studies, a changing criterion was included, re? ecting the PECS phase changes but it was secondary to the main design (Bock et al. 2005; Chambers and Rehfeldt 2003; Cummings and Williams 2000; Rehfeldt and Root 2005; Stoner et al. 2006; Tincani 2004; Tincani et al. 2006; Yokoyama et al. 2006).Comparative group designs were employed in ? ve papers. Yoder and Stone (2006a, b) used random assignment to PECS or Responsive Education and Prelinguistic Milieu Teaching (RPMT) intervention groups, while Howlin et al. (2007) used random assignment of classes to immediate treatment, delayed treatment or no treatment with PECS groups. Carr and Felce (2007a, b) used a quasiexperimental group design whereby PECS intervention and control groups were chosen by geographical location, and included both within subjects and between group measures. Pre-test equivalence of the groups was established. Participants Picture exchange, speech PECS Phases I–VI I–VI G I–VI G –Sin gle-subject (multiple baseline)) I–IV Picture exchange, speech Dependent variable Speech Social In total, there were 456 participants in the 27 studies; of these, 394 (86%) received PECS intervention and 62 (14%) were in non- or alternative-intervention groups. Of the total, 377 (83%) were described as having ASD. Ages of participants ranged from 20 months to 40 years and there were 198 males (43%) and 38 (8%) females with the gender of 220 (48%) of participants unstated. Where the same or a subgroup of participants were reported in multiple studies (Carr and Felce 2007a, b; Yoder and Stone 2006a, b), they were counted only once. The group experimental (Howlin et al. 007; Yoder and Stone 2006a, b) or quasi-experimental (Carr and Felce 2007a, b) studies involved a total of 161 participants (35% of the total sample): 98 in PECS intervention groups and 92 in control or other treatment groups. The Delayed Treatment Group in the Howlin et al. (2007) study was used as both control and intervention at different times. All these children were described as having autism or PDD-NOS and little or no speech. They ranged in age from 20 months to 11 years at study commencement. These studies all provided information on the initial abilities of the participants based on standardized tests. The single subject studies involved a total of 42 participants (9% of the total sample) and all provided information on diagnosis, age and gender.Only a minority provided information on the diagnostic instrument or protocol used to identify ASD (Ganz and Simpson 2004; Marckel et al. 2006; Yokoyama et al. 2006), described the degree of autism or provided standardized assessment data or a description of general ability for all participants (Chambers and Rehfeldt 2003; Frea et al. 2001; Kravits et al. 2002; Rehfeldt and Root 2005; Stoner et al. 2006; Research design Program evaluation 55–70 months RCT 21–54 months Autism/PDD 36 Yoder and Stone (2006a) 20–53 months Autism/PDD Yoder and Stone (2006b) 36 RCT Diagnosis Participants 6 Table 2 Summary of Studies 123 Yokoyama et al. (2006) Authors Webb (2000) 3 Autism 5 ASD 5–7 years Ages J Autism Dev Disord (2009) 39:1471–1486 1477 Yokoyama et al. 2006).Most researchers did document initial communication skills, either using standardized test results or a description of functional skills, although some descriptions were minimal. Participants were almost entirely described as non-verbal or having little or no functional speech, or in some cases no functional communication. The participants in the Marckel et al. (2006) study were able to use PECS independently to make requests at the start of the research. Participants in three studies were explicitly identi? ed by researchers as having challenging behavior (Adkins and Axelrod 2001; CharlopChristy et al. 2002; Frea et al. 2001). Interobserver and Procedural Reliability Interobserver reliability was reported for 20 of the 27 papers revie wed. Papers in which interobserver reliability was not reported included ? e earlier program evaluations (Bondy and Frost 1993, 1994, 1998; Liddle 2001; Webb 2000) and one single subject study (Adkins and Axelrod 2001). Reliability ranged from 80. 3 to 100% calculated on between 10 and 100% of data. Three studies (Howlin et al. 2007; Kravits et al. 2002; Malandraki and Okalidou 2007) estimated reliability on less than a minimum standard of 20% of total sessions. In contrast, procedural reliability was reported for only 7 of the 27 studies (Bock et al. 2005; Cummings and Williams 2000; Marckel et al. 2006; Tincani 2004; Tincani et al. 2006; Yoder and Stone 2006a, b) and discussed but not formally calculated in one other (Stoner et al. 2006). Where reported, procedural reliability ranged from 96 to 100%.In two papers (Yoder and Stone 2006a, b) less than 20% of sessions were used for the estimate. Social Validity Formal measures of social validity were reported in only four papers (Mag iati and Howlin 2003; Marckel et al. 2006; Tincani 2004; Yoder and Stone 2006a). Settings Fourteen studies were conducted in a special school, special preschool or special classroom setting. Remaining studies were conducted in a variety of settings including an integrated preschool, regular classroom, homes, clinics, day treatment facilities, and combinations of these settings. Ef? cacy and Effectiveness of PECS Of the total group of 394 individuals who received PECS intervention, only one child was reported as being nsuccessful at mastering at least phase I (Liddle 2001), and one adult had dif? culty with the motor and cognitive demands of the training and failed to progress past phase I (‘‘Mike’’, Stoner et al. 2006). ‘‘Carl’’, from Tincani’s (2004) study, was more successful with manual signs than PECS, but, the great majority successfully mastered at least some phases of PECS. Outcome data will now be considered further, initially focusing on pre-experimental designs, then single subject designs, quasi-experimental group designs and ? nally RCTs. This will be followed by a more detailed consideration of maintenance and generalization. Pre-Experimental Studies Several studies used pre-experimental designs.Bondy and Frost (1993) reported data on the implementation of PECS and found increased communicative initiations and use of pictures. Bondy and Frost (1994, 1998), Schwartz et al. (1998), Webb (2000), and Liddle (2001) also presented data on PECS implementation and reported increases in spoken language following PECS training. Schwartz et al. found that children were able to acquire communication with PECS training and there was evidence of generalization across pragmatic function. These studies, however, lacked adequate experimental control, and especially given the young age of the children involved in at least four studies, it is unknown how communication would have developed without the interve ntion.In their pilot study, Magiati and Howlin (2003) used a pre-treatment measure and three teacher ratings over time. They found signi? cant increases in PECS level (d = 2. 91),1 frequency of spontaneous use (d = 1. 75), and number of symbols used (d = 3. 01) over the 6 months following teacher training in PECS and its subsequent introduction. These are very large effect sizes by educational standards. They also found smaller but still statistically signi? cant increases in the number of signs (d = 0. 31), words (d = 0. 32) and phrases (d = 0. 30) used, and in the overall level of spontaneous communication (d = 0. 83). Outcomes were, however, measured mainly through teacher rating scales.The results must be treated with caution as they are likely to have been in? uenced by expectations and the research design was very weak. 1 For pre-test post-test designs, effect sizes were calculated by subtracting the pretest mean from the post-test mean and dividing by the pooled standard devi ation. For studies involving a comparison group, effect sizes were calculated by subtracting the mean of the control or alternate treatment group from the mean of the PECS intervention group and dividing by the pooled standard deviation. 123 1478 J Autism Dev Disord (2009) 39:1471–1486 Single Subject Studies PND and PEM statistics were calculated for the 10 single subject studies that provided baseline and intervention data.Initially, calculations were conducted on treatment data alone and then on all intervention data, including treatment, maintenance and generalization. When compared, the overall mean differences in favor of the treatment alone data were very small, only 0. 4% in the case of PND and 0. 8% for PEM. It was considered that the inclusion of all intervention data provided the best indicator of the ef? cacy of the overall package and these data were used for the remaining analysis. Results are provided in Table 3. Calculations were not possible for the four addit ional single subject studies (Adkins and Axelrod 2001; Cummings and Williams 2000; Ganz and Simpson 2004; Rehfeldt and Root 2005). These studies either lacked baseline data (e. g. alternating treatment design without baseline) or lacked baseline data that corresponded directly to that collected in intervention. The overall mean PND was 78. 5% (range 50– 100), placing the PECS intervention in the effective range (Scruggs and Mastropieri 1998). The overall mean PEM was 89. 1% (range 72. 3–100). Quality indicator scores are also presented in Table 3, and ranged from 30. 6 to 55. 7 out of a possible 65 points. Correlation between Quality Indicator scores for each study and their associated study PND was not signi? cant (rs = -0. 05, p = 0. 87). For PEM there was a trend toward weaker studies producing higher effect sizes but this did not reach signi? cance (rs = -0. 44, p = 0. 19).Mann–Whitney U tests or Kruskal–Wallis one-way ANOVAs were used to compare PND and PEM values across participant and study characteristics and these data are presented in Tables 4 and 5. No signi? cant difference in PND was found for age, gender, setting, inclusion of maintenance or generalization data, or research design. A signi? cant difference was found for PND scores for outcome variables, with studies addressing picture exchange only having a higher mean PND than those that included other dependent variables (i. e. , speech, social, behavioral, with or without picture exchange). A signi? cant difference was also found between PND values for participant diagnosis.Post hoc comparison showed that PND for children identi? ed with autism (i. e. , autistic disorder) were signi? cantly lower than for the other two groups, but these groups were not signi? cantly different from each other. No signi? cant difference was found between PEM values for any of the study or participant characteristics although participant diagnosis approached signi? cance (p = . 06). Fo ur of the single subject studies included data speci? cally relating to speech development from which PND and PEM values could be calculated (Charlop-Christy et al. 2002; Tincani 2004; Tincani et al. 2006; Yokoyama et al. 2006). The mean calculated PND was 49. % (range 19. 5– 100) and PEM 54. 2% (range 25. 0–100). These values are in the non-effective or at best very mildly effective range but with wide variation. Charlop-Christy et al. (2004) found increases in speech during PECS training. Tincani (2004) examined independent word vocalizations during PECS and sign language training. The addition of a Table 3 Single subject studies: PND and PEM results; study quality results Study PND PEM Study quality (Maximum 65) Picture Speech Social Behavior Overall Picture Speech Social Behavior Overall exchange exchange Adkins and Axelrod (2001) Bock et al. (2005) Chambers and Rehfeldt (2003) Charlop-Christy et al. 2002) Cummings and Williams (2000) Frea et al. (2001) Ganz and Si mpson (2004) Kravits et al. (2002) Marckel et al. (2006) Rehfeldt and Root (2005) Stoner et al. (2006) Tincani (2004) Tincani et al. (2006) Yokoyama et al. (2006) Mean SD – 92. 1 100. 0 – – 100. 0 – 87. 7 97. 3 – 77. 5 90. 6 98. 6 68. 7 90. 0 10. 9 – – – 59. 8 – – – – – – – 100. 0 20. 0 19. 5 49. 8 38. 4 – – – 86. 8 – – – – – – – – – – 86. 8 n/a – – – 26. 0 – 0 – – – – – – – – 13. 0 18. 4 – 92. 1 100. 0 55. 6 – 50. 0 – 87. 7 97. 3 – 77. 5 95. 3 70. 5 58. 6 78. 5 18. 8 – 92. 1 100. 0 – – 100. 0 – 87. 7 100. 0 – 90. 1 90. 6 98. 6 89. 9 94. 3 5. 2 – – – 65. 7 – – – – – – – 100. 0 25. 0 26. 0 54. 2 36. 0 – – – 95. – – – – – – – – – – 95. 6 n/a – – – 85. 0 – 100. 0 – – – – – – – – 92. 5 10. 6 90. 1 95. 3 72. 3 76. 7 89. 1 10. 6 – 92. 1 100. 0 76. 3 – 100. 0 – 87. 7 100. 0 30. 6 55. 7 43. 8 52. 4 32. 9 42. 4 35. 3 50. 4 49. 6 43. 8 50. 3 48. 2 45. 7 50. 3 45. 1 7. 6 123 J Autism Dev Disord (2009) 39:1471–1486 Table 4 Means, standard deviations and Mann–Whitney U test results for PND and PEM scores of study and participant characteristics Variable N PND M (SD) Quality indicators C50 50 PECS only Includes other Yes No 5 74. 3 (16. 6) 5 82. 6 (21. 7) 5 90. 9 (8. 9) 5 66. 0 (18. 0) 3. 0 0. 94 84. 6 (7. 5) 93. 5 (12. 0) 3. 0 1. 98* 94. 0 (5. 7) 84. 1 (12. 6) 7. 0 1. 14 5. 0 1. 6 U z PEM M (SD) U z Research design Multiple baseline Alternating treatments ABAB A ge Under 5 5–8 2. 0 1. 56 76. 5 (0. 3) 92. 2 (9. 4) 4. 0 1. 04 87. 9 (11. 2) 93. 9 (8. 7) 6. 0 1. 27 89. 9 (12. 2) 11. 0 0. 21 88. 5 (10. 5) 88. 5 (12. 5) 52. 0 0. 46 92. 5 (11. 2) 6. 0 0. 52 2. 0 1. 56 9–17 18? Diagnosis Autism PDD-NOS/autistic characteristics Other Setting Special school/ preschool Clinic Integrated preschool Home Combination 10 1 1 9 8 90. 1 (12. 5) 3. 87 79. 3 (n/a) 50. 0 (n/a) 74. 3 (30. 4) 82. 3 (21. 8) 2 57. 1 (2. 1) 8 83. 8 (17. 0) 9 8 3 9 13 85. 0 (17. 0) 2. 58 73. 1 (31. 2) 72. 2 (21. 4) 87. 9 (18. 5) 69. 8 (25. 9) 7. 68* 93. 8 (7. 3) 84. 4 (11. 4) 75. 4 (18. 8) 93. 5 (11. 1) 6 3 1 70. 0 (19. 0) 3. 82 95. 8 (4. 0) 77. 5 (n/a) 479 Table 5 Descriptive statistics and Kruskal–Wallis One-Way ANOVA results of PND and PEM scores of study and participant characteristics Variable N PND M (SD) H PEM M (SD) H 85. 5 (12. 3) 1. 62 95. 8 (4. 0) 90. 1 (n/a) 6. 74 Outcome variables Maintenance data included Generalization data included Yes 8 80. 9 (17 . 8) No Yes No Gender Male Female 2 68. 9 (26. 7) 4 88. 8 (12. 4) 6 71. 6 (20. 0) Procedural reliability data 83. 7 (13. 2) 5. 59 100. 0 (0) 92. 8 (9. 8) 90. 4 (11. 8) 2. 29 79. 3 (n/a) 100. 0 (n/a) 87. 8 (13. 1) 89. 2 (13. 4) 2 100. 0 (0) 14 89. 2 (15. 2) 25 78. 9 (23. 9) 40. 5 1. 12 5 92. 5 (11. 2) Note: * Indicates signi? cant result at 0. 05 level for two-tailed test reinforcer delay in phase IIIb resulted in increased in word vocalizations. Tincani et al. 2006) examined word vocalizations and vocal approximations during PECS training, and found a decrease during phases I-III before dramatic increases in phase IV. In a second experiment, looking at phase IV only, a higher percentage of word vocalizations was found with the reinforcement delay procedure than without. Yokoyama et al. (2006) examined frequency and intelligibility of vocalization during PECS training in phases I-IV; these authors also found an increase with the time delay procedure. Several other studies provided da ta on speech development, which was not suitable for calculation of PND or PEM values. Kravits et al. (2002) found an increase in frequency of intelligible speech but not in range of spoken vocabulary.Ganz and Simpson (2004) found that words per trial increased noticeably during phase IV or phases III and IV of PECS training, in particular, simultaneously with delayed word modeling. Charlop-Christy et al. (2002) provided the only appropriate data for calculation of PND and PEM values for social outcomes. From this very small amount of data, the PND of 86. 8% and PEM of 95. 6% suggest an effective or highly effective intervention. Variables that increased in this study were eye contact, joint attention, cooperative play, and frequency of initiations and requests including but not limited to PECS requests. Initiations and requests Note: * Indicates signi? cant result at 0. 05 level for two-tailed test ncreased the most, and joint attention also increased in all three children. It has been suggested that a direct positive relationship exists between joint attention and communication in children with autism, with improvement in one potentially stimulating an increase in the other (CharlopChristy et al. 2002). Kravits and colleagues (2002) reported some increase in duration of social interaction with peers although these data were not suitable for calculation of PND or PEM as only the mean level in each phase was presented. PND and PEM scores were calculated for data from only two studies for behavioral variables (CharlopChristy et al. 2002; Frea et al. 2001). The mean PND was 13. 0% while the mean PEM was 92. %, but, examination of graphed data showed treatment effects, indicating that decreased problem behaviors occurred in conjunction with increased communication skills through PECS training. Two studies compared sign language to PECS interventions (Chambers and Rehfeldt 2003; Tincani 2004) and one compared a VOCA to PECS (Bock et al. 2005). For each of these st udies PND and PEM were equal, and a higher value was found for PECS than for the alternative intervention. For Tincani (2004) calculated values were 95. 3% for PECS and 92. 3% for sign, for Chambers and 123 1480 J Autism Dev Disord (2009) 39:1471–1486 Rehfeldt (2003) values were 100% for PECS, and 65. 7% for sign, and for Bock et al. 2005) values were 92. 1% for PECS and 79. 7% for VOCA. Quasi-Experimental Group Studies Carr and Felce (2007b) found signi? cant improvement in several aspects of communicative interaction between children and staff following 15 h of PECS training (Phases I-III). Signi? cant increases were found for total child-toadult initiations, linguistic initiations, the percentage of adult response, the percentage of child response, and signi? cant decrease in adult-to-child interactions with no opportunity for child response. These differences were found in comparison to both a pre-intervention measure over time and to a non-intervention and non-equivalent control group.Examining a subset of this group, who used at least one word during observations, Carr and Felce (2007a) reported that over 6 weeks training in PECS phases I-III, 3 of the 24 children in the PECS group increased their spoken words. A further 2 who did not speak at pretesting did so at post-testing, while there was a marginal increase in speech for one child in the control group. RCTs Only three RCT studies were located. Yoder and Stone (2006a) conducted an experimental study of 36 children with autism, aged 21–54 months, who were randomly assigned to PECS or RPMT intervention groups. They found that the PECS group showed a signi? cantly greater increase in frequency of speech (d = 0. 3) and in number of different words used (d = 0. 50) after 6 months of intervention, but by 6 months post-intervention the difference was no longer evident. Interestingly, they also found differing effects according to pretreatment characteristics: children who were low in initial object exploration bene? ted more from the RPMT intervention, while those who were higher bene? ted more from PECS, these effects being evident 6 months post-intervention. Overall, there was a signi? cant increase in non-imitative spoken acts over 1 year. The actual increases were from a mean of 0. 25 nonimitative spoken acts in a 15-min session to a mean of 5. 5, and from a mean of 0. 7 different non-imitative words to a mean of 3. Given the young age of the children, the fact that their initial verbal mental age averaged 11. 9 months (range 7–19 months), just at the stage when verbal language is likely to develop naturally, it seems quite possible that this increase could be attributed to maturation. In a second article, Yoder and Stone (2006b) examined the effect of the interventions on the three major types of intentional communication used prior to speech development, (i. e. initiating joint attention, requesting, and turn-taking). They found that, overall all three comm unicative functions increased signi? cantly, but RPMT increased turn-taking more than PECS.Children who were higher in initiating joint attention before treatment had greater increases in both initiating joint attention and requesting following RPMT intervention, while those who were initially lower in initiating joint attention had greater increases following PECS intervention. Howlin et al. (2007) conducted a group RCT of 84 children with autism, examining the effect of teacher training and consultancy in PECS. It should be stressed that this study examined the effectiveness of a consultancy model to deliver PECS, rather than the ef? cacy of PECS per se. Thus, the study was noteworthy in that it appears to be the only research to examine effectiveness (i. e. , outcomes under clinical rather than optimal conditions). Howlin et al. ound that rates of communicative initiations and PECS usage were signi? cantly increased immediately following intervention, but that these effects were not maintained once the intervention ceased. They found no signi? cant increase in frequency of speech. Howlin et al. also examined ADOS-G (Lord et al. 2000) domain scores for communication and reciprocal social interaction. They found no increase in most ADOS-G ratings, with the exception of a decrease in the severity score for the Reciprocal Social Interaction domain at the 10 month followup. Unfortunately, no data was provided on the ? delity of implementation of the PECS program, or indeed on the ? delity of the teacher training.Maintenance and Generalization Only ? ve studies provided data on maintenance. Two of the RCT studies included long-term follow-up. Yoder and Stone (2006a) found that differences in speech variables were not maintained 6 months post-intervention, while Howlin et al. (2007) found that for the 26 children assessed at a 10-month follow-up, the increased rate of communicative initiations and PECS usage found immediately post-intervention was not maintained. Two single subject studies and one case study measured maintenance of skills 6–10 months postintervention (Charlop-Christy et al. 2002; Malandraki and Okalidou 2007; Yokoyama et al. 2006). Charlop-Christy et al. ound that speech and socio-communicative behaviors had been maintained or continued to increase for one participant followed up 10 months post-training. Yokoyama et al. found maintenance of PECS skills both in the training room and at home, 6–8 months after training for the three participants in their study. Malandraki and Okalidou in their study of one child found maintenance of skills 6 months after the main intervention. While the difference was not signi? cant and the number of studies was low, for the single 123 J Autism Dev Disord (2009) 39:1471–1486 1481 subject studies both PND and PEM were lower for studies that included maintenance data (Table 4).Fifteen of the 27 studies included some data on generalization of PECS skills. The great majority o f these were positive, with skills generalizing to different settings, people and stimuli. For some studies, generalization was an integral part of the way data were collected (CharlopChristy et al. 2002; Yoder and Stone 2006a, b). For others, generalization to untrained situations was speci? cally probed (e. g. , participants in Stoner et al. ’s (2006) study generalized their skills to use in fast food restaurants). Several studies demonstrated generalization to the classroom teacher or to home. In a small number of instances, generalization was unimpressive or absent.For example, in the Adkins and Axelrod (2001) study, tests for ‘‘generalization’’ simply required the child to mand for an object without immediately prior prompted trials. ‘‘Carl’’, from Tincani’s (2004) study, failed to generalize PECS skills to classroom teachers, preferring to use sign language. Discussion The PECS program was originally designed to provide a method of communication for children with autism, particularly those who do not use functional speech. PECS appears to be a popular intervention (Howlin et al. 2007) but, unfortunately, popularity of a given treatment does not necessarily re? ect actual ef? cacy (Green et al. 2006; Reynhout and Carter 2006). Only three RCTs have been reported to date. The studies of Yoder and Stone (2006a, b) compared PECS to RMPT.PECS was superior for some children but the study was designed to compare two treatments and, consequently, did not include a control arm. Thus, no conclusions can be drawn about the relative superiority of either intervention to a non-treatment control. Howlin et al. (2007) provided the only effectiveness study conducted. They found signi? cant effects on communicative initiations but this was not maintained once the intervention ceased. Thus, further examination of approaches to the delivery of PECS in clinical settings is needed. Con? dence in the Howlin et a l. study is somewhat weakened by the lack of any data on treatment ? delity, which is a critical feature in study quality (Gersten et al. 2005).The nature and quantity of data arising from RCTs at this point in time is insuf? cient to draw ? rm conclusions regarding the PECS interventions. Thus, probably the highest priority for research in this area is the conduct of further RCTs examining both ef? cacy and effectiveness in applied settings. In the absence of an adequate body of RCTs, clinicians still need to make informed decisions regarding interventions and may need to look to the second line of evidence. Evidence supporting the PECS intervention was provided by the well-designed quasi-experimental studies of Carr and Felce (2007a, b), which incorporated a non-equivalent control group with demonstration of pre-test equivalence between groups.Arguably, the bulk of interpretable data on PECS comes from single subject studies. For the relevant studies, the overall, mean PND (78. 5% ) and PEM (89. 1%) ? gures support the preliminary conclusion that PECS may be an effective intervention, at least when implemented under research conditions. There was a signi? cant difference between the PND results for studies that only looked at picture exchange outcome variables and those that included other collateral variables, such as speech, social, or challenging behavior. This indicates that, unsurprisingly, PECS training appears to be most effective in providing a successful means of communication through picture exchange.Nevertheless, it should be acknowledged that the number of studies remains relatively low and single subject designs have several limitations, including low external validity. While these studies contribute to our knowledge and give us a preliminary indication of the ef? cacy of PECS, they are not a substitute for well-conducted large scale RCTs. A substantial number of the extant studies were preexperimental in nature, particularly the early research. As such, they are not able to provide convincing demonstrations of experimental control. Hence, these studies offer no interpretable evidence on the ef? cacy of PECS. The effect of PECS training on speech development remains unclear.Research into various forms of AAC suggests they may have the potential to enhance speech development (Cress and Marvin 2003; Millar et al. 2006; Romski and Sevcik 2005) although results have sometimes been inconsistent (Carter 1999; Millar et al. 2006). Several of the studies reviewed in this paper reported increases in speech following PECS training, but others, including Howlin et al. (2007), reported little or no effect. Where speech increased, this has often occurred concurrently with phase III or IV of PECS, and in particular when a time delay was introduced. A related question, for which there is as yet no empirical evidence, is whether PECS training affects comprehension.Brady (2000) found increased comprehension skills with the use of VOCAs and it would be worth investigating whether PECS would have a similar effect. In comparison with other AAC systems, better overall results were obtained with PECS in the studies reviewed here (Adkins and Axelrod 2001; Bock et al. 2005; Chambers and Rehfeldt 2003; Tincani 2004). Nevertheless, there was variability in the results depending on initial imitation skills and, possibly, participant preference. It has been argued that individuals with ASD may bene? t from visually cued instruction (Quill 1997) and further examination of this issue would seem warranted. In addition, existing 123 1482 J Autism Dev Disord (2009) 39:1471–1486 application of PECS appears to have been exclusively limited to graphic symbols.There are distinct advantages to the use of three-dimensional tangible symbols, including decreased cognitive load and high iconicity (Rowland and Schweigert 1989, 1990; Turnell and Carter 1994). The exploration of the use of PECS with tangible symbols, especially with indiv iduals who are low functioning, would seem warranted. Only 5 of the 27 studies provided data on maintenance of PECS skills or other dependent variables. It is worthy of note that maintenance was problematic in both RCTs (Howlin et al. 2007; Yoder and Stone 2006a) that examined the issue. Overall, available evidence is mixed but there is certainly suf? cient doubt to indicate that maintenance should be formally and systematically monitored in the clinical application of PECS programs.Fifteen studies provided data on generalization of skills, the vast majority of these found that generalization did occur, but what was described as ‘‘generalization’’ varied greatly. There were several methodological limitations and issues in the research examined that warrant comment. In general, participant descriptions were poor, making it dif? cult to assess whether the intervention is best suited to individuals with particular characteristics. While nearly all studies prov ided a diagnosis, few speci? ed the diagnostic protocol or criteria. Further, when participants were diagnosed with ASD, few researches attempted to quantify the degree of autism.Noting the range of behaviors and symptom severity possible within individuals presenting with autistic disorder, and even greater variation in the broader autism spectrum, this would seem to be relevant, if not critical, information. Few researchers provided standardized assessment data or a detailed functional description of general ability, but probably re? ecting the aim of the intervention, most did provide some description of initial communication skills. While the number of studies was clearly insuf? cient to reach ? rm conclusions, PND data suggest that individuals with PDD-NOS or showing autistic traits made more progress with PECS than those with autistic disorder.It is unclear whether this is because the PECS protocol is better suited to them, or because they would do better with any treatment. W ithout further clear and consistent quanti? cation of the degree of autistic symptomatology, it is impossible to evaluate further this variable in relation to the ef? cacy of the PECS intervention. In addition, there was insuf? cient data on intellectual functioning to enable analysis of any relationship to PND or PEM. It is recommended that, in future studies, standardized psychometric data, standardized functional assessment of adaptive behavior, and clear information on initial communicative abilities should be provided.In addition, where a diagnosis of autism is provided the level of autistic symptomatology should be quanti? ed. Procedural reliability data were very limited, with data only meeting the conventional minimum standard in 5 of the 27 studies. Because of the absence of this data, it is not possible to determine in many cases whether what was being implemented was in fact the PECS program as designed. PECS is a complex and multi-component intervention making the veri? cation of treatment integrity even more critical. The absence of such information in research studies is somewhat dif? cult to understand given that the PECS manual (Frost and Bondy 2002) gives explicit and speci? criteria for assessing the integrity of training during each phase. Nevertheless, research on PECS is not alone in this regard and lack of procedural reliability data has been reported as a problem in other recent intervention reviews in the area of autism (e. g. , Bellini and Akullian 2007; Reynhout and Carter 2006). The calculated PND (88. 8%) and PEM (89. 9%) ? gures for studies that did meet the standard for reporting procedural reliability, are at the high end of the effective intervention range (Ma 2006; Scruggs and Mastropieri 1998), suggesting that monitoring of procedural integrity should be a key feature in research as well as clinical applications of PECS. A signi? ant component of the present review was the application of PND and PEM metrics to the relevant dat a. PND and PEM values were in most cases very similar, with lower variability for PEM (see Table 4). An exception was found for data relating to behavioral variables (CharlopChristy et al. 2002; Frea et al. 2001), where high baseline variability and ‘‘? oor’’ effects occurred, often causing the calculated PND for affected graphs to be low, while the PEM was high. For example, visual inspection of the graphed data in Frea et al. (2001) shows a clear treatment effect of the PECS intervention on disruptive behavior. Nevertheless, the calculated PND for these data was 0%, while the PEM was 100%.The discrepancy between the clear treatment effect seen in the graphed data and the PND value indicates that PEM may be a more appropriate metric for challenging behavior, where variability is likely to be high. Further, the advantages of using multiple methods of calculating effect sizes for single subject research are highlighted. As previously noted, PECS is a complex multi-component intervention program. Consequently, the question arises as to which of the components are most critical to its ef? cacy. For example, reinforcer assessment is formally and systematically incorporated into PECS and this may well be a salient factor in program ef? cacy. The use of picture exchange with a partner (rather than touching or pointing to a symbol) is a key distinguishing feature of PECS, but it is unclear whether exchange per se is essential to ef? cacy.The issue of developing spontaneity is addressed in an unusually systematic way in the PECS program (Chiang and Carter 2008) but extant research 123 J Autism Dev Disord (2009) 39:1471–1486 1483 provides only limited information on the circumstances under which communication occurs. Thus, there would appear to be considerable scope for examination of how speci? c components contribute to the overall ef? cacy of PECS. In addition, there has been only limited comparison of PECS to alternative interventio ns and this stands as a priority for future research. Several limitations of the current review must be acknowledged. Many of the earlier studies were descriptive and clear experimental control was not established.While later studies were of higher quality, only a limited number of RCTs have been conducted and much of the available interpretable data comes from second line of evidence single subject studies. Analysis of PND and PEM was only possible for a subset of the relevant single subject studies examined and analysis of speci? c study and participant characteristics were based on low numbers. In addition, very few studies provided adequate procedural reliability data so the extent to which PECS was appropriately implemented often remained unknown. Conclusion On balance, the studies reviewed provide preliminary evidence that PECS may be ef? cacious for children and adults with ASD and other developmental disabilities, who have little or no speech. Primary bene? s appear to be ev ident in communication by picture exchange. Identi? cation of the core aspects of the program that are important to its success, the individuals to whom it is best suited, and its relationship to other interventions remain to be substantively investigated. PECS stands as a promising intervention with some empirical support but many questions remain. The conduct of further RCTs into the ef? cacy and effectiveness of PECS stands as a high research priority. Appendix See Table 6. Table 6 Quality criteria for single subject research adapted from Horner et al. (2005) Area Indicator Description of participants Participants are described with suf? ient detail to allow others to select individuals with similar characteristics and settings (e. g. , age, gender, disability, diagnosis). One point awarded for each of the following (maximum of 5): 1. Statement of diagnosis such as autism, ASD, Asperger syndrome, intellectual disability (with or without indicating diagnostic source), age and gend er 2. Diagnostic instrument speci? ed (e. g. , WISC, AAMR diagnostic criteria, DSM-IV criteria, ADOS). Must provide if ASD or 0 awarded 3. If ASD, degree of autism speci? ed either with reference to symptoms (DSM-IV) or instrument like CARS. If not ASD, award point 4. Standardized assessment data (e. g. IQ, developmental scale, adaptive behavior) OR detailed functional description of general ability. Disability range (e. g. , moderate) acceptable for intellectual disability 5. Communication skills documented by means of standardized test results OR description of functional skills The process for selecting participants is described with replicable precision. MUST describe the process used to select participants, not just describe the participants or their needs. This would generally include the criteria the participants must meet (e. g. , 3–5 years, less than 5 spoken words, diagnosis of autistic disorder) and or the process of selecting participants (e. g. , the ? rst 5 chil dren on the waiting list).Essentially, authors must explicitly state HOW/WHY participants were selected Critical features of the physical setting are described with suf? cient precision to allow replication Dependent variables All dependent variables are described with operational precision Each dependent variable is measured with a procedure that generates a quanti? able index The measurement process is described with replicable precision Dependent variables are measured repeatedly over time Data are collected on the reliability or inter-observer agreement (IOA) associated with each dependent variable, and IOA levels meet minimal standards (i. e. , IOA = 80%; Kappa = 0. 60).Must be on minimum of 20% of sessions to be acceptable Independent variables Independent variable is described with replicable precision Independent variable is systematically manipulated and under the control of the experimenter Overt measurement of the ? delity of implementation for the independent variable. M UST be measured on a minimum of 20% of sessions to be acceptable 123 1484 Table 6 continued Area Baseline Indicator J Autism Dev Disord (2009) 39:1471–1486 A baseline phase provides repeated measurement of a dependent variable and establishes a pattern of responding that can be used to predict the pattern of future performance if introduction or manipulation of the independent variable did not occur. Should include a minimum of 3 stable data points.High variability is acceptable if intervention effects are unambiguous The procedural characteristics of the baseline conditions should be described operationally Experimental control/internal The design provides at least three demonstrations of experimental effect at different points in time. Effects of validity alternating treatments may be added, as main comparison is not with baseline. AB designs may not be added as they do not demonstrate intervention at different times when comparing to baseline The design controls for common threats to internal validity (e. g. , permits elimination of rival hypotheses). Acceptable designs include multiple baseline, ABAB and alternating treatment with counterbalancing.Unacceptable designs include: AB, ABA, and changing criterion External validity Social validity Experimental effects are replicated across participants, settings, or materials to establish external validity. At least three participants, settings or materials must be apparent The dependent variable is socially important Implementation of the independent variable is practical and cost effective (must be measured) Social validity is enhanced by implementation of the independent variable over extended time periods, by typical intervention agents, in typical ph

Qualitative research Essay

Abstract This paper is the first of a series of three articles relating to a case study conducted at Fairfield University to assess aspects of the rapid introduction of Information Technology at the institution. This article deals with the nature of the problem faced by Fairfield University, the characteristics of the case methodology, and lays the foundation for the selection of this research technique for the current study. The paper begins with an Introduction section to familiarize the reader with the case organization. The following section on Case Methodology explores the history, and some of the applications of the technique. The section ends with specific research protocols for researchers. Introduction Fairfield University is a private liberal arts institution of about 3,000 full time undergraduate students and about 1,000 graduate school students. The undergraduate students are distributed through the College of Arts and Sciences, the School of Business, and the School of Nursing. The graduate students are in the Graduate School of Education, the School of Business, and the School of Nursing. There are also part time students in the School of Continuing Education and the BEI School of Engineering. As with many other private institutions of higher education, Fairfield University faces many challenges. These challenges come from the declining population of college age students and the growing cost of running the institution. The literature will support the preceding statement (Crossland, 1980), but provide little comfort to the institution. One of the areas of greatest concern to college managers is the continuing cost of information technology. With the constant need to increase staff salaries, it is like salaries, inadvisable to reduce the outlay on information technology. Interviews that were conducted by this researcher with the deans and managers indicated that some of the peer institutions of Fairfield University are in fact doing as much if not more in this area. Hence any interruption in the effort to maintain technological currency would result in a competitive disadvantage for the institution. Therein lies the administrative financial challenge. The expense on information technology must be maintained at a time of declining revenues (Nicklin, 1992). The field of information technology at a university is very broad and could encompass many technologies hitherto not considered within its purview. However, there has been a relentless and indeed accelerating pace of convergence of the technologies in telecommunications, library services, and video services. The current study is concerned only with the aspects of information technology as it relates to academic computing and will focus on instructional and research computing. The goals of this study include an examination of the (a) managerial and (b) economic aspects of the rapid acquisition of information technology. The objectives deriving from those goals are: An assessment of the categories of computer use in higher education. An evaluation of the managerial issues of computing, including the centralization/decentralization of computing, client/server computing and the aspects of the Internet and the World Wide Web (WWW) Establish a basis for understanding the current and future economic aspects of information technology acquisition. The research questions arising from the above objectives were as follows: Objective 1 above is addressed by the question: â€Å"What patterns of acquisition emerge from the current computing environment and the perceived needs for computing? † Objective 2 is addressed by the question: â€Å"What characteristics of the categories of computing use contribute to the patterns of acquisition? † The five categories developed by King and Kraemer (1985) and adapted for use by Levy (1988) in his study at the University of Arizona, are used in this study, to examine the computing use at Fairfield University. Objective 3 is addressed by the question: â€Å"What managerial issues arise from the rapid acquisition of information technology and how important have those technologies become to the organization? † Objective 4 is addressed by the question: â€Å"How will the institution balance the need for technological changes with the need to continue the accomplishment of routine tasks?. † Samuel Levy (1988) conducted a study of instructional and research computing at the University of Arizona. This study replicates and extends the Levy (1988) study, and was conducted at Fairfield University. The current study extends the Levy (1988) study in its examination of aspects of the Internet, the World Wide Web, and Client/Server computing. Levy (1988) established the use of the case study as appropriate for the research project, and this researcher also used the literature to confirm the use of case methodology in the study at Fairfield University. The history and development of case methodology is reviewed, in support of the current case study at Fairfield University. There have been periods of intense use followed by periods of disuse of this technique, as documented by Hamel, Dufour, and Fortin (1993) as well as others. The relevance of that history to this study is important in that it establishes the known advantages and disadvantages of the methodology. The particular technique of a single-case study is reviewed, since that is the specific implementation of a case study at Fairfield University and was also used by Levy (1988). Case Study Methodology The history of case study research is marked by periods of intense use and periods of disuse. The earliest use of this form of research can be traced to Europe, predominantly to France. The methodology in the United States was most closely associated with The University of Chicago Department of Sociology. From the early 1900’s until 1935, The Chicago School was preeminent in the field and the source of a great deal of the literature. There was a wealth of material in Chicago, since it was a period of immigration to the United States and various aspects of immigration of different national groups to the city were studied and reported on (Hamel et al. , 1993). Issues of poverty, unemployment, and other conditions deriving from immigration were ideally suited to the case study methodology. Zonabend (1992) stated that case study is done by giving special attention to completeness in observation, reconstruction, and analysis of the cases under study. Case study is done in a way that incorporates the views of the â€Å"actors† in the case under study. The field of sociology is associated most strongly with case study research, and during the period leading up to 1935, several problems were raised by researchers in other fields. This coincided with a movement within sociology, to make it more scientific. This meant providing some quantitative measurements to the research design and analysis. Since The Chicago School was most identified with this methodology, there were serious attacks on their primacy. This resulted in the denigration of case study as a methodology. In 1935, there was a public dispute between Columbia University professors, who were championing the scientific method, and The Chicago School and its supporters. The outcome was a victory for Columbia University and the consequent decline in the use of case study as a research methodology. Hamel (Hamel et al. , 1993) was careful to reject the criticisms of case study as poorly founded, made in the midst of methodological conflict. He asserted that the drawbacks of case study were not being attacked, rather the immaturity of sociology as a discipline was being displayed. As the use of quantitative methods advanced, the decline of the case study hastened. However, in the 1960s, researchers were becoming concerned about the limitations of quantitative methods. Hence there was a renewed interest in case study. Strauss and Glaser (1967) developed the concept of â€Å"grounded theory. † This along with some well regarded studies accelerated the renewed use of the methodology. A frequent criticism of case study methodology is that its dependence on a single case renders it incapable of providing a generalizing conclusion. Yin (1993) presented Giddens’ view that considered case methodology â€Å"microscopic† because it â€Å"lacked a sufficient number† of cases. Hamel (Hamel et al. , 1993) and Yin (1984, 1989a, 1989b, 1993, 1994) forcefully argued that the relative size of the sample whether 2, 10, or 100 cases are used, does not transform a multiple case into a macroscopic study. The goal of the study should establish the parameters, and then should be applied to all research. In this way, even a single case could be considered acceptable, provided it met the established objective. The literature provides some insight into the acceptance of an experimental prototype to perceive the singularity of the object of study. This ensures the transformation from the local to the global for explanation. Hamel (Hamel et al. , 1993) characterized such singularity as a concentration of the global in the local. Yin (1989a) stated that general applicability results from the set of methodological qualities of the case, and the rigor with which the case is constructed. He detailed the procedures that would satisfy the required methodological rigor. Case study can be seen to satisfy the three tenets of the qualitative method: describing, understanding, and explaining. The literature contains numerous examples of applications of the case study methodology. The earliest and most natural examples are to be found in the fields of Law and Medicine, where â€Å"cases† make up the large body of the student work. However, there are some areas that have used case study techniques extensively, particularly in government and in evaluative situations. The government studies were carried out to determine whether particular programs were efficient or if the goals of a particular program were being met. The evaluative applications were carried out to assess the effectiveness of educational initiatives. In both types of investigations, merely quantitative techniques tended to obscure some of the important information that the researchers needed to uncover. The body of literature in case study research is â€Å"primitive and limited† (Yin, 1994), in comparison to that of experimental or quasi-experimental research. The requirements and inflexibility of the latter forms of research make case studies the only viable alternative in some instances. It is a fact that case studies do not need to have a minimum number of cases, or to randomly â€Å"select† cases. The researcher is called upon to work with the situation that presents itself in each case. Case studies can be single or multiple-case designs, where a multiple design must follow a replication rather than sampling logic. When no other cases are available for replication, the researcher is limited to single-case designs. Yin (1994) pointed out that generalization of results, from either single or multiple designs, is made to theory and not to populations. Multiple cases strengthen the results by replicating the pattern-matching, thus increasing confidence in the robustness of the theory. Applications of case study methodology have been carried out in High-Risk Youth Programs (Yin, 1993) by several researchers. The effects of community-based prevention programs have been widely investigated using case methodology. Where the high risk youth studies assumed a single case evaluation, these studies have typically used a collection of cases as a multiple-case study. This has been true in the various substance abuse prevention programs that are community-based (Holder, 1987; Sabol, 1990; Yin, 1993). Numerous such studies sponsored by the U. S. General Accounting Office are distributed in the literature between Evans (1976) and Gopelrud (1990). These studies have gone beyond the quantitative statistical results and explained the conditions through the perspective of the â€Å"actors. † Thus case study evaluations can cover both process and outcomes, because they can include both quantitative and qualitative data. There are several examples of the use of case methodology in the literature. Yin (1993) listed several examples along with the appropriate research design in each case. There were suggestions for a general approach to designing case studies, and also recommendations for exploratory, explanatory, and descriptive case studies. Each of those three approaches can be either single or multiple-case studies, where multiple-case studies are replicatory, not sampled cases. There were also specific examples in education, and management information systems. Education has embraced the case method for instructional use. Some of the applications are reviewed in this paper. In exploratory case studies, fieldwork, and data collection may be undertaken prior to definition of the research questions and hypotheses. This type of study has been considered as a prelude to some social research. However, the framework of the study must be created ahead of time. Pilot projects are very useful in determining the final protocols that will be used. Survey questions may be dropped or added based on the outcome of the pilot study. Selecting cases is a difficult process, but the literature provides guidance in this area (Yin, 1989a). Stake (1995) recommended that the selection offers the opportunity to maximize what can be learned, knowing that time is limited. Hence the cases that are selected should be easy and willing subjects. A good instrumental case does not have to defend its typicality. Explanatory cases are suitable for doing causal studies. In very complex and multivariate cases, the analysis can make use of pattern-matching techniques. Yin and Moore (1988) conducted a study to examine the reason why some research findings get into practical use. They used a funded research project as the unit of analysis, where the topic was constant but the project varied. The utilization outcomes were explained by three rival theories: a knowledge-driven theory, a problem-solving theory, and a social-interaction theory. Knowledge-driven theory means that ideas and discoveries from basic research eventually become commercial products. Problem-solving theory follows the same path, but originates not with a researcher, but with an external source identifying a problem. The social-interaction theory claims that researchers and users belong to overlapping professional networks and are in frequent communication. Descriptive cases require that the investigator begin with a descriptive theory, or face the possibility that problems will occur during the project. Pyecha (1988) used this methodology to study special education, using a pattern-matching procedure. Several states were studied and the data about each state’s activities were compared to another, with idealized theoretic patterns. Thus what is implied in this type of study is the formation of hypotheses of cause-effect relationships. Hence the descriptive theory must cover the depth and scope of the case under study. The selection of cases and the unit of analysis is developed in the same manner as the other types of case studies. Case studies have been increasingly used in education. While law and medical schools have been using the technique for an extended period, the technique is being applied in a variety of instructional situations. Schools of business have been most aggressive in the implementation of case based learning, or â€Å"active learning† (Boisjoly & DeMichiell, 1994). Harvard University has been a leader in this area, and cases developed by the faculty have been published for use by other institutions. The School of Business at Fairfield University has revised the curriculum so that in place of the individual longitudinal courses in the areas of Management, Marketing, Operations, Finance, and Information Systems, students take one course. That course is designed around cases that encompass those disciplines, but are presented in an integrated manner. The students are therefore made aware of the interrelatedness of the various disciplines and begin to think in terms of wider problems and solutions. Later courses add the international dimension to the overall picture. Case studies have been used to develop critical thinking (Alvarez, et al. , 1990). There are also interactive language courses (Carney, 1995), courses designed to broaden the students’ horizons (Brearley, 1990), and even for technical courses (Greenwald, 1991), and philosophical ones (Garvin, 1991). This investigation is a case study of the aspects of Information Technology that are related to client/server computing, the Internet, and the World Wide Web, at Fairfield University. Thus this paper examines issues that will expand the reader’s knowledge of case study methodology as it relates to the design and execution of such a study. Yin (1994) recommended the use of case-study protocol as part of a carefully designed research project that would include the following sections: Overview of the project (project objectives and case study issues) Field procedures (credentials and access to sites) Questions (specific questions that the investigator must keep in mind during data collection) Guide for the report (outline, format for the narrative) (Yin, 1994, p.64) The quintessential characteristic of case studies is that they strive towards a holistic understanding of cultural systems of action (Feagin, Orum, & Sjoberg, 1990). Cultural systems of action refer to sets of interrelated activities engaged in by the actors in a social situation. The case studies must always have boundaries (Stake, 1995). Case study research is not sampling research, which is a fact asserted by all the major researchers in the field, including Yin, Stake, Feagin and others. However, selecting cases must be done so as to maximize what can be learned, in the period of time available for the study. The unit of analysis is a critical factor in the case study. It is typically a system of action rather than an individual or group of individuals. Case studies tend to be selective, focusing on one or two issues that are fundamental to understanding the system being examined. Case studies are multi-perspectival analyses. This means that the researcher considers not just the voice and perspective of the actors, but also of the relevant groups of actors and the interaction between them. This one aspect is a salient point in the characteristic that case studies possess. They give a voice to the powerless and voiceless. When sociological studies present many studies of the homeless and powerless, they do so from the viewpoint of the â€Å"elite† (Feagin, Orum, & Sjoberg, 1991). Case study is known as a triangulated research strategy. Snow and Anderson (cited in Feagin, Orum, & Sjoberg, 1991) asserted that triangulation can occur with data, investigators, theories, and even methodologies. Stake (1995) stated that the protocols that are used to ensure accuracy and alternative explanations are called triangulation. The need for triangulation arises from the ethical need to confirm the validity of the processes. In case studies, this could be done by using multiple sources of data (Yin, 1984). The problem in case studies is to establish meaning rather than location. Designing Case Studies Yin (1994) identified five components of research design that are important for case studies: A study’s questions Its propositions, if any Its unit(s) of analysis The logic linking the data to the propositions The criteria for interpreting the findings (Yin, 1994, p. 20). The study’s questions are most likely to be â€Å"how† and â€Å"why† questions, and their definition is the first task of the researcher. The study’s propositions sometimes derive from the â€Å"how† and â€Å"why† questions, and are helpful in focusing the study’s goals. Not all studies need to have propositions. An exploratory study, rather than having propositions, would have a stated purpose or criteria on which the success will be judged. The unit of analysis defines what the case is. This could be groups, organizations or countries, but it is the primary unit of analysis. Linking the data to propositions and the criteria for interpreting the findings are the least developed aspects in case studies (Yin, 1994). Campbell (1975) described â€Å"pattern-matching† as a useful technique for linking data to the propositions. Campbell (1975) asserted that pattern-matching is a situation where several pieces of information from the same case may be related to some theoretical proposition. His study showed, through pattern-matching, that the observed drop in the level of traffic fatalities in Connecticut was not related to the lowering of the speed limit. His study also illustrated some of the difficulties in establishing the criteria for interpreting the findings. Construct validity is especially problematic in case study research. It has been a source of criticism because of potential investigator subjectivity. Yin (1994) proposed three remedies to counteract this: using multiple sources of evidence, establishing a chain of evidence, and having a draft case study report reviewed by key informants. Internal validity is a concern only in causal (explanatory) cases. This is usually a problem of â€Å"inferences† in case studies, and can be dealt with using pattern-matching, which has been described above. External validity deals with knowing whether the results are generalizable beyond the immediate case. Some of the criticism against case studies in this area relate to single-case studies. However, that criticism is directed at the statistical and not the analytical generalization that is the basis of case studies. Reliability is achieved in many ways in a case study. One of the most important methods is the development of the case study protocol. Case studies can be either single or multiple-case designs. Single cases are used to confirm or challenge a theory, or to represent a unique or extreme case (Yin, 1994). Single-case studies are also ideal for revelatory cases where an observer may have access to a phenomenon that was previously inaccessible. Single-case designs require careful investigation to avoid misrepresentation and to maximize the investigator’s access to the evidence. These studies can be holistic or embedded, the latter occurring when the same case study involves more than one unit of analysis. Multiple-case studies follow a replication logic. This is not to be confused with sampling logic where a selection is made out of a population, for inclusion in the study. This type of sample selection is improper in a case study. Each individual case study consists of a â€Å"whole† study, in which facts are gathered from various sources and conclusions drawn on those facts. Yin (1994) asserted that a case study investigator must be able to operate as a senior investigator during the course of data collection. There should be a period of training which begins with the examination of the definition of the problem and the development of the case study design. If there is only a single investigator, this might not be necessary. The training would cover aspects that the investigator needs to know, such as: the reason for the study, the type of evidence being sought, and what variations might be expected. This could take the form of discussion rather than formal lectures. A case study protocol contains more than the survey instrument, it should also contain procedures and general rules that should be followed in using the instrument. It is to be created prior to the data collection phase. It is essential in a multiple-case study, and desirable in a single-case study. Yin (1994) presented the protocol as a major component in asserting the reliability of the case study research. A typical protocol should have the following sections: An overview of the case study project (objectives, issues, topics being investigated) Field procedures (credentials and access to sites, sources of information) Case study questions (specific questions that the investigator must keep in mind during data collection) A guide for case study report (outline, format for the narrative) (Yin, 1994, p. 64). The overview should communicate to the reader the general topic of inquiry and the purpose of the case study. The field procedures mostly involve data collection issues and must be properly designed. The investigator does not control the data collection environment (Yin, 1994) as in other research strategies; hence the procedures become all the more important. During interviews, which by nature are open ended, the subject’s schedule must dictate the activity (Stake, 1995). Gaining access to the subject organization, having sufficient resources while in the field, clearly scheduling data collection activities, and providing for unanticipated events, must all be planned for. Case study questions are posed to the investigator, and must serve to remind that person of the data to be collected and its possible sources. The guide for the case study report is often neglected, but case studies do not have the uniform outline, as do other research reports. It is essential to plan this report as the case develops, to avoid problems at the end. Stake (1995), and Yin (1994) identified at least six sources of evidence in case studies. The following is not an ordered list, but reflects the research of both Yin (1994) and Stake (1995): Documents Archival records Interviews Direct observation Participant-observation Physical artifacts Documents could be letters, memoranda, agendas, administrative documents, newspaper articles, or any document that is germane to the investigation. In the interest of triangulation of evidence, the documents serve to corroborate the evidence from other sources. Documents are also useful for making inferences about events. Documents can lead to false leads, in the hands of inexperienced researchers, which has been a criticism of case study research. Documents are communications between parties in the study, the researcher being a vicarious observer; keeping this in mind will help the investigator avoid being misled by such documents. Archival documents can be service records, organizational records, lists of names, survey data, and other such records. The investigator has to be careful in evaluating the accuracy of the records before using them. Even if the records are quantitative, they might still not be accurate. Interviews are one of the most important sources of case study information. There are several forms of interviews that are possible: Open-ended, Focused, and Structured or survey. In an open-ended interview, key respondents are asked to comment about certain events. They may propose solutions or provide insight into events. They may also corroborate evidence obtained from other sources. The researcher must avoid becoming dependent on a single informant, and seek the same data from other sources to verify its authenticity. The focused interview is used in a situation where the respondent is interviewed for a short period of time, usually answering set questions. This technique is often used to confirm data collected from another source. The structured interview is similar to a survey, and is used to gather data in cases such as neighborhood studies. The questions are detailed and developed in advance, much as they are in a survey. Direct observation occurs when a field visit is conducted during the case study. It could be as simple as casual data collection activities, or formal protocols to measure and record behaviors. This technique is useful for providing additional information about the topic being studied. The reliability is enhanced when more than one observer is involved in the task. Glesne and Peshkin (1992) recommended that researchers should be as unobtrusive as the wallpaper. Participant-observation makes the researcher into an active participant in the events being studied. This often occurs in studies of neighborhoods or groups. The technique provides some unusual opportunities for collecting data, but could face some major problems as well. The researcher could well alter the course of events as part of the group, which may not be helpful to the study. Physical artifacts can be tools, instruments, or some other physical evidence that may be collected during the study as part of a field visit. The perspective of the researcher can be broadened as a result of the discovery. It is important to keep in mind that not all sources are relevant for all case studies (Yin, 1994). The investigator should be capable of dealing with all of them, should it be necessary, but each case will present different opportunities for data collection. There are some conditions that arise when a case researcher must start data collection before the study questions have been defined and finalized (Yin, 1994). This is likely to be successful only with an experienced investigator. Another important point to review is the benefit of using rival hypotheses and theories as a means of adding quality control to the case study. This improves the perception of the fairness and serious thinking of the researcher. Analyzing Case Study Evidence This aspect of the case study methodology is the least developed and hence the most difficult. As a result, some researchers have suggested that if the study were made conducive to statistical analysis, the process would be easier and more acceptable. This quantitative approach would be appealing to some of the critics of the case study methodology. However not all case studies lend themselves to this type of analysis. Miles and Huberman (1984) suggested analytic techniques such as rearranging the arrays, placing the evidence in a matrix of categories, creating flowcharts or data displays, tabulating the frequency of different events, using means, variances and cross tabulations to examine the relationships between variables, and other such techniques to facilitate analysis. There must first be an analytic strategy, that will lead to conclusions. Yin (1994) presented two strategies for general use: One is to rely on theoretical propositions of the study, and then to analyze the evidence based on those propositions. The other technique is to develop a case description, which would be a framework for organizing the case study. Lynd conducted a widely cited â€Å"Middletown† study in 1929, and used a formal chapter construct to guide the development of the analysis. In other situations, the original objective of the case study may help to identify some causal links that could be analyzed. Pattern-matching is another major mode of analysis. This type of logic compares an empirical pattern with a predicted one. Internal validity is enhanced when the patterns coincide. If the case study is an explanatory one, the patterns may be related to the dependent or independent variables. If it is a descriptive study, the predicted pattern must be defined prior to data collection. Yin (1994) recommended using rival explanations as pattern-matching when there are independent variables involved. This requires the development of rival theoretical propositions, but the overall concern remains the degree to which a pattern matches the predicted one. Yin (1994) encouraged researchers to make every effort to produce an analysis of the highest quality. In order to accomplish this, he presented four principles that should attract the researcher’s attention: Show that the analysis relied on all the relevant evidence Include all major rival interpretations in the analysis Address the most significant aspect of the case study Use the researcher’s prior, expert knowledge to further the analysis Stake (1995) recommended categorical aggregation as another means of analysis and also suggested developing protocols for this phase of the case study to enhance the quality of the research. He also presented ideas on pattern-matching along the lines that Yin (1994) presented. Runkel (1990) used aggregated measures to obtain relative frequencies in a multiple-case study. Stake (1995) favored coding the data and identifying the issues more clearly at the analysis stage. Eisner and Peshkin (1990) placed a high priority on direct interpretation of events, and lower on interpretation of measurement data, which is another viable alternative to be considered.

Homicide Crime Scene Investigation Essay

Criminal Investigation Introduction   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   On November 3, 2000, the dead body of a female was spotted along Old Annapolis road, Columbia. The body, which was later identified as that of Ashley Smith, was first spotted by two truck drivers, who immediately notified police officers. The first police officers arrived at the scene at 1051 hours, checked the victim, and concluded that she was dead. A crime scene perimeter was then established to allow for further investigations and collection of evidence before the body was moved. According to studies, the first action that police officers need to take once they reach a crime scene is to ensure that the scene is secured (Howell, 1999). This is usually in an effort to prevent any tampering of evidence, as well as to create room for further investigation. According to the case study, D/CPL Case was assigned the role of the primary investigator. Hence, it was his responsibility to ensure that all the persons responsible were brought to boo k. Crime scene investigators are persons in charge of the entire CSI operation. Apart from establishing identities of victims as well as suspects of a crime scene, it is also the responsibility of the CSI investigator to acquire witness testimonies, link suspects to crime scenes, and exonerate the innocent (Byrd, 2000). The case study intends to analyze the manner in which the homicide investigation was conducted in addition to lessons learned from it. Reflection on Case Study   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   In my opinion, the homicide investigation was conducted in a fair manner. This is because all processes were followed sequentially and in the right manner — that is, in accordance with the crime scene investigation procedures. For instance, police officers swung into action the moment they received news about a body spotted along the Old Annapolis road. According to studies, the first action that police officers ought to take once they reach a crime scene is to set up a perimeter to prevent vital evidence from being tampered with. One of the reasons why the investigation was conducted smoothly was due to the proper organization of the police officers in charge. After receiving the report about a dead body that had been located, an investigation panel was established in which D/CPL Case was to act as the primary investigator. Through this organization, no conflicts were witnessed among the police officers since all of the officers we re aware of their roles in the investigation.   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   Throughout the investigation, D/CPL Case was of great importance; this is because all his ideas were successful, and it was through him that the investigation was successful. For instance, during the early days of the investigation, progress was quite slow since all leads were hitting a dead end. D/CPL Case increased the pace of the investigation after he announced a reward of $8,000 to anybody who would come forward with information regarding the killing of Ashley Smith. By announcing an award, D/CPL Case knew that somebody withholding information regarding the homicide would eventually come out, which was true. Although the homicide was announced in the media, not all information was aired; instead, some vital information was withheld. For instance, the media did not mention the sections where the victim was stabbed or the number of wounds inflicted on the victim’s body. Apart from the stabbings, nothing was said about strangulati on of the victim. This was a good move by D/CPL Case since these omissions were used as traps during the interrogation process.   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   In my opinion, the interrogations were conducted in the right manner; this is because D/CPL Case ensured that he interrogated all the people whose names were mentioned during the interviews. In order to ensure that the interrogations went smoothly, D/CPL Case used a number of items. For instance, apart from photos and sketches of the crime scene, D/CPL Case also relied on forensic analysts, as well as pathologists, who aided in reconstructing the crime scene, hence, easing the interrogation operations. According to the case study, it is evident that D/CPL Case remained calm throughout the interrogation process since he was never in a hurry. For instance, whenever he was not satisfied with the information obtained during an interrogation, D/CPL Case would set up another interrogation with the same witness and attempt to acquire more information. It is through this tactic that D/CPL Case managed to acquire more information from Scott Jones. Although most of the interrogations were conducted in an appropriate manner, the same cannot be said when it comes to Fredrick Johnson. Although Scott admitted that he was with Johnson during the night of the murder, D/CPL Case interviewed Johnson only once before accusing him of first-degree murder. In my opinion, D/CPL Case was not just in the manner in which he interrogated Johnson. Although the case was solved successfully, the case study does not talk about what happened to the $8,000 award that had been advertised in the media. In my opinion, since nothing is mentioned about this award, the award was just a set-up that lured possible suspects to come forward. Crime Scene Investigation and Evidence Analysis   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   Investigation of the homicide started on 11/3/00 when two truck drivers spotted the body and reported it to the police. After setting up a perimeter, thorough investigation of the crime scene commenced. Investigation at the crime scene was thorough since several professionals were present. For instance, apart from police officers, there were also photographers, forensic scientists, and pathologists. According to Assi (2010), the reason all investigation specialists need to be present is to replicate the crime scene before launching an investigation. For instance, whereas the role of police officers is to search the entire crime scene and take notes, photographing and sketching the crime scene is also of great importance. Taking photographs and sketching the scene makes it easy to review the crime scene in case something arises during the investigation (Howell, 1999). When taking photographs of a homicide investigation, several factors need to be considered. The face of the victim ought to be photographed for the purpose of identification when official investigations commence. Apart from the face, photographs should be taken of all wounds inflicted on the victim, all unusual marks appearing on the victim’s body, and the particularities of the victim’s clothing (Assi, 2010). In the case study, it is evident that crime scene investigators adhered to these requirements. This is because there are photographs depicting the victim’s face, all wounds inflicted on her, and the manner in which the victim was dressed. According to the photographs taken at the crime scene, it is evident that apart from being stabbed in the neck and abdominal area, the victim was also strangled.   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   While still at the crime scene, a number of evidence materials were collected. For instance, apart from the two neoprene skullcaps located at the scene, the forensic scientists and crime scene technicians collected more evidence for further research in the lab. Other items collected included the victim’s blood sample, the victim’s clothes, and DNA swabbings from the victim’s ankles, mouth, vagina, and anus. After analyzing the evidence collected at the crime scene, the fact emerged that the victim had consumed alcohol prior to her death. Additionally, the victim had also engaged in a sexual activity moments before her death. This was discovered by the samples of sperms detected in her pants, vagina, and anus. According to the notes taken at the crime scene, the wounds inflicted on the victim were caused by knife stabbings. The murder weapon, which was later discovered by Richard Myers, was also forwarded to the laborat ory for further forensic investigation. After the media announcement, the investigators took note of two names, Scott Jones and Fredrick Johnson, which had been mentioned in three of the calls received. After setting up surveillance on the two primary suspects, investigators managed to get the suspects’ fingerprints from the cigarette butts they had discarded. These butts were used by the analysts to identify whether the two suspects had any links to the homicide. By the cigarette butts obtained, the investigators realized that the suspects’ DNA matched with the DNA acquired at the crime scene, as well as the murder weapon found. This critical information prompted police officers to apprehend the two suspects for further interrogations. Investigative Steps and Strategies   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   After successfully identifying the victim as Ashley Smith, the next step was to launch an investigation into the homicide, which would result in the arrest of the perpetrators. For the investigation to be successful, the investigating team used a number of steps and strategies. The first step was to get the full profile of the victim. Since she was still underage, her records were not available in the computer system; hence, it was D/CPL Case’s task to acquire this information from Mrs. Smith. After holding an interview with Mrs. Smith, D/CPL Case learned that, before her demise, Ashley had been a student at Grand Senior high school and had a female friend known as Heather Thomas. According to Mrs. Smith, Heather was the last person to see Ashley. Apart from Heather, Mrs. Smith also mentioned Jason Phelps, who had been Ashley’s boyfriend. The information marked the beginning of the investigation since more names were mentione d by Heather and Jason. Since most of the information acquired was not solid, the next step was to announce the homicide in the media and issue an award for anybody that would come forward with substantial information regarding the crime. According to the case study, this strategy was successful since more concrete evidence was obtained. Apart from the mentioning of names such as Scott and Johnson, this strategy also resulted in the identification of the murder weapon, which was a small knife. Richard Myers, the citizen who spotted the knife, presented the knife to a police officer after seeing the homicide report on television.   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   According to Assi (2010), when an investigation is still in the process, some information needs to be confidential. D/CPL Case applied this strategy by not releasing all information about the homicide to the media. Apart from D/CPL Case and the investigating team present at the crime scene, nobody else was aware of the number of wounds inflicted on the victim. Apart from the number of wounds, the media were also not told about the choking. D/CPL Case received many calls, but one call that caught his attention was the one he received on 11/30/00. On this particular day, a female caller claimed that, before her demise, Ashley had been in the company of three men, one of whom was Fredrick Johnson. The caller went on to state that Ashley had been raped by the three men before being killed by Johnson in a move to prevent her from reporting the incident. What made this call unique from the rest was that the caller claimed Ashley had been stabbed in the head and neck. The caller further said that two of the suspects were scratched by the victim during the process. It was this information that prompted D/CPL Case to establish a surveillance on the two main suspects. Although the investigating officers had pictures of the suspects, they were unable to verify the suspects accurately. Hence, with the help of P.O. Gummo, a traffic police officer, they were able to verify that, indeed, the suspects under surveillance were Scott Jones and Fredrick Johnson. Interview of Witnesses and Interrogation of Suspects   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   According to Howell (1999), interviews and interrogations are part of the investigation process since more information is acquired during the process. Apart from interrogating witnesses, investigators also need interrogate all suspects whose names are mentioned. According to the case study, the first person to be interviewed was Mrs. Smith, the victim’s mother. Through this interview, it was noted that the victim had been fond of sneaking out of the house at night. Apart from sneaking out, the victim had also been in a number of love relationships, and her last boyfriend was known as Jason Phelps. During the interview, Mrs. Smith claimed that Ashley had had a close friend known as Heather Thomas. Apart from Mrs. Smith, D/CPL Case also interrogated Heather Thomas. During the interview, D/CPL Case noted that Heather was not telling the whole truth. D/CPL Case wished to apply a polygraph examination on Heather, but her parents were aga inst this idea. Hence, D/CPL Case only asked random questions, which Heather answered. D/CPL Case took the record of the interview to Det. Shelly Madison, who later claimed that out of the nine answers given by Heather Thomas, four were wrong. This was enough evidence to show that that Heather Thomas had not been truthful. It was not until the fourth interrogation that Heather Thomas confessed to knowing Fredrick Johnson. D/CPL Case also interrogated Scott Jones and Fredrick Johnson, who was the primary suspect of the homicide investigation. According to the case study, Scott was interrogated more thoroughly than Fredrick. This is because Fredrick requested an attorney. Hence, he was never questioned. At first, Scott denied taking part in the homicide. However, after numerous interrogations by D/CPL Case, Scott finally confessed to taking part in the killing. Constitutional Challenges   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   During the investigation, there were a number of motions aimed at suppressing some of the evidences. For instance, there was a motion filed to suppress the murder weapon located in the crime scene. The murder weapon, a knife, was brought in by Richard Myers, an American citizen. In my opinion, this item was legally obtained since it was brought in after an advertisement was placed in the media. Apart from the knife, another motion filed was that of illegal surveillance. According to studies, one is in violation of the fourth Amendment if he or she interferes with the privacy of a person by either watching or listening to the Person (Couenhoven, n.d). In my opinion, D/CPL Case and the investigation team went against the fourth amendment after they illegally placed surveillance on Scott and Fredrick. There was also a motion filed on some of the confessions made. It is against the fourth amendment for a witness to be interrogated for a long p eriod (NCIDS, 2002). Due to this, it is therefore evident that Scott was coerced to make confessions during the interrogation, hence, there is high chance that not all the information provided by the suspect was accurate. In my opinion, the information provided by Scott was not legally obtained. It is also against the law for enforcers to acquire information from witnesses and suspects without a warrant (Find Law, 2014). In the case study, D/CPL Case and the investigation team acquired the DNA of Scott and Fredrick after illegally acquiring the suspects’ cigarette butts that they had discarded without the suspects’ consent. Due to this, it is therefore evident that the law enforcers did not acquire the item legally. Lesson Learnt and the Applications   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   The case study has been quite beneficial since through it, I have been able to comprehend most of the lessons I acquired in class concerning the investigation strategies and procedures of a homicide. First, a crime scene needs to be secured by the first police officers to arrive at the scene. In the case study, PFC. Malcolm was the first police officer to arrive at the scene. After confirming that the victim was dead, he established a perimeter around the crime scene. Secondly, after securing the crime scene, investigators need to take photographs of the scene as well as the victim. Apart from taking photographs, they also need to produce sketches of the entire scene. In the case study, apart from sketches, there are also photographs of the victim as well as the crime scene. These photos showed the wounds inflicted on the victim, the posture in which the victim was found, and the manner in which she was clothed when she was first identifie d. After taking photographs, investigators need to ensure that they collect all the evidence required. In the case study, apart from the victim’s blood samples, other evidence collected included her clothes as well as swabs from her mouth, vagina, and anus. Finally, crime scene investigators need to interview all witnesses as well as suspects whose names are mentioned. These interviews are of great importance to the investigation since they enable investigators in identifying the main suspects. In the case study, it was through numerous investigations that D/CPL Case and his investigating team were able to identify the main suspects of the homicide. Conclusion   Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚  Ã‚   On 11/3/00, a female body was found in Columbia along the Old Annapolis road. The body was later identified as that of Ashley Smith. During the investigation, it was noted that the victim was not only stabbed, but was also choked. According to tests conducted in the laboratory, prior to her death, the victim had engaged in sexual activities. This was due to the detection of sperms in her pants, vagina, and anus. After collection of evidence, an investigation was launched to identify perpetrators of the homicide. Apart from the victim’s mother, interrogations were also conducted on Heather Thomas, Jason Phelps, Ronnie Emden, Scott Jones, Fredrick Johnson, Michael Stewart, and Janet Jones and Crystal Jones. It was through the interrogations that the investigation team was able to identify Scott and Johnson as the main perpetrators of the homicide. In my opinion, the information addressed in the case study has been quite beneficial. Th is is because, through the case study, I have been able to fully comprehend the information learned in the classroom. References Assi, H. (2010). Homicide Crime Scene Investigation. Retrieved on 10 December 2014 from www.justiceacademy.org/iShare/Library-Training/Homicide-Scene-Manual.pdf Byrd, M. (2000). â€Å"Duty Description for the Crime Scene Investigator.† Crime Scene Investigator Network. Retrieved on 9 December 2014 from http://www.crime-scene-investigator.net/dutydescription.html Couenhoven,P. (n.d). Suppressing Confessions: Involuntariness and Miranda. Retrieved on 12 December 2014 from www.sdap.org/downloads/research/criminal/confessions.pdf Find Law. (2014). The Fourth Amendment and the â€Å"Exclusionary Rule†. Retrieved on 12 December 2014 from http://criminal.findlaw.com/criminal-rights/the-fourth-amendment-and-the-exclusionary-rule.html Howell, J. (1999). Homicide Investigation Standard Operating Procedures. Retrieved on 9 December 2014 from www.policeforum.org/assets/docs/Free_Online_Documents/Homicide/homicide%20investigation%20standard%20operating%20procedures%201999.pdf NCIDS. (2002). Chapter 14: Suppression Motions. Retrieved on 12 December 2014 from www.ncids.org/Defender%20Training/Drug%20Case%20Training/Supp_Motions.pdf Source document