Channing Bete Company(R) Types of Products and Services

Teen Effectiveness Study


  • Quantitative data were collected from the remaining 280 teens in the study through completion of the pre- and post-reading survey used in the first group but without the discussion portion.
  • Researchers pre-tested and post-tested each participant to precisely measure an increase in understanding of the information in each booklet, and intent to act as a result of that information. This, as opposed to solely asking participants whether they believed they had learned something.

  • Study participants were recruited from eight public schools, health centers, and community/youth centers in urban, suburban, and rural Massachusetts.
  • Data were collected from 368 in-school teens between the ages of 13 and 17. Represented were African Americans, Asian/Pacific Islanders, Latinos, and Caucasians, with a minimum of 80 participants selected from each group.
  • The study was conducted using a representative sampling of booklets divided equally among three booklet formats: Classic illustration, realistic illustration, and story format.
  • Quantitative and qualitative data were collected from 88 teens participating in structured survey/discussion groups, in which participants completed pre- and post-reading surveys and took part in 45-minute group discussions.

Teen Study Results

Quantitative Results

Transfer of Information

Number of all participants studied said they learned "some" or "a lot" of new information from the booklets they had read


Among those reporting very little prior knowledge of the topic, number of participants who reported learning "some" or "a lot"



Number of participants who reported that the booklets were "interesting" or "very interesting" to read


Number of participants reported as "likely" or "very likely" to recommend the booklets to someone they knew


Intent to Act

Among participants expressing little or no knowledge of a topic prior to reading a booklet, the intent to act after reading the booklet increased


Ease of

Number of participants who found the booklets easy to read


Number of participants who understood all or most of the information contained in the booklets


Qualitative Results

Discussion groups provided confirmation of statistical results and added personal context to the quantitative data.

"I like the fact that these booklets are very direct and interesting to read, with real situations."

"These booklets raise the right issues that people often don't talk about enough."

"After reading the booklets, I would know how to help a friend better."

"I didn't know much about self-esteem or what it was before."

Teen Study Executive Summary

Download PDF pdf icon

Executive Summary

Report on the Product Effectiveness Study with In-School Teenagers

Study conducted by John Snow, Inc., & World Education, Inc.

Channing Bete Company (R)

One Community Place
South Deerfield, MA 01373-7328
Phone: 1-800-477-4776
Fax: 1-800-329-2939

©1998 Channing Bete Company, Inc. All rights reserved.
No part of this publication may be reproduced without written permission of the publisher.

Report on the Product Effectiveness Study with In-School Teenagers


The study analyzed quantitative and qualitative data collected in person from a sample of 368 in-school teenagers (ages 13-17) living in Massachusetts. The study included between 80 and 100 individuals from each of the four following racial subgroups:

  • African American
  • Asian/Pacific Islander
  • Caucasian
  • Hispanic.

Abstract Objectives

This independent study assessed the effectiveness of Channing Bete Company (CBC) educational materials with a teenage audience in terms of:

  • knowledge transfer
  • intention to act/attitude change
  • reading ease
  • personal relevance.


Quantitative and qualitative results showed that CBC materials are effective in terms of transfer of information and influence on intention to act. Study participants also found CBC materials to be easy to read and relevant to their own lives. Results remained consistent across all product formats tested (Classic illustrated, realistic illustrated, and story format).


CBC publications in all tested formats are an effective means of communicating information to teenagers. The publications effectively influence readers' intention to take actions suggested by the publications. The publications are easily read, and readers regard the content as relevant and useful to their lives.


Channing Bete Company, Inc., is the publisher of Scriptographic® booklets, i.e., 8- and 16-page booklets designed to communicate information through a combination of simple text and supporting illustrations. CBC has over 50 years of experience providing health, safety and other information on hundreds of topics. Materials are published in a variety of styles and formats; CBC also has a line of educational materials aimed specifically at readers with very limited reading skills.

Many of CBC's customers in the public and private sectors are increasingly concerned about measuring and understanding the effectiveness of their communication efforts. The reasons for this concern vary. For example, among government customers, it often grows out of the need for support in making grant applications; or, the need may arise out of "performance budgeting" requirements in which the level of funding for a program is tied to measurable results. In the medical field, the need to measure effectiveness is related to efforts to improve patient satisfaction, compliance, and outcomes.

In addition, many programs that might utilize CBC materials must first go through a process of "pretesting" materials prior to distribution, to ensure that the materials are understood by and appropriate for target audiences.

In the spring and summer of 1998, John Snow, Inc., an internationally known, independent public health research firm, and the Literacy Division of World Education, Inc., a nonprofit organization committed to furthering adult literacy, conducted an extensive quantitative and qualitative evaluation of CBC educational booklets. These organizations brought broad experience and outstanding qualifications to this study. For example, JSI has partnered with organizations such as the U.S. Department of Health and Human Services' Office of Population Affairs, and Health Resources and Services Administration; WEI has sponsored a national conference on health and literacy and is in partnership with Harvard University in the National Center for Adult Learning and Literacy. The JSI/WEI team thus approached the study with wide expertise in the fields of public health and literacy.

This study sought to obtain data about the effectiveness of the booklets with teenagers based on the following criteria:

  • knowledge transfer -- how much the reader learns from the booklets
  • intention to act -- how the booklets affect the reader's attitudes or intention to act in a positive (i.e., healthier or safer) way
  • reading ease -- how easy the booklets are to read and understand
  • personal relevance -- how relevant the reader perceives the information to be to his or her life.


Data collection for the study was conducted between April and July 1998. The study gathered information from 368 participants, all of whom were between the ages of 13 and 17 and currently enrolled in school between the 6th and 12th grades. The study participants were recruited from eight public schools, health centers, and community/youth centers in Massachusetts. There was a minimum of 80 participants selected from each of the four major racial groups included in the study (African American, Asian/Pacific Islander, Caucasian, and Hispanic).

Overall, 45% of the study participants were male, 55% female. This distribution was similar for all racial groups. Approximately 25% of the sample were 13 or 14 years of age; 25% were 15; 27% were 16; and 24% were 17. Within each age category, racial distribution was roughly equal.

English was the primary language spoken at home for 63% of the participants. Among those whose native language was other than English, the highest proportion was found in the Asian/Pacific Islander group, where 81% spoke a language other than English. Forty-two percent of Hispanics spoke a language other than English.

English language reading proficiency was evaluated by gathering information on participants' reading habits. Most participants read English-language books, magazines or newspapers regularly (defined as at least a few times a month). Only 8% reported reading English-language materials once a month or less. There was no significant difference in reading habits between racial groups or genders.

Data collection was conducted through two types of groups.

1) Survey/Discussion Groups

Quantitative and qualitative data was gathered from 88 participants using small "survey/discussion groups." These groups consisted of eight to 15 participants, who took part in eight different data collection sessions held in a variety of locations (urban, suburban, and rural) throughout Massachusetts. The 88 participants included a minimum of 20 participants from each of the four racial subgroups included in the study, and each session included a mix of races, ages, and genders.

Each data collection session lasted two hours, during which time the participants evaluated and discussed 15 booklets. The following methods were used:

A. Participants filled out a pre-survey for each booklet, which yielded:

  • demographic data: gender, age, race, current grade in school, language spoken at home
  • knowledge of the topic, including the participants' perceived level of knowledge of the topic covered in the test booklet and their actual level of knowledge as measured by their response to survey questions
  • intention to act/attitude -- the current behaviors and attitudes regarding the topic covered in the test booklet.

B. Participants read the selected booklets.

C. Participants completed a post-survey for each booklet studied in their session, revealing:

  • knowledge of topic after reading the booklet
  • intention to act/attitude -- any change in attitude or behavior regarding the topic of the booklet under study
  • booklet's appeal/relevance, including the appeal of illustrations, colors, layout and writing style; relevance to personal circumstances; personal identification with booklet characters; reading ease; likelihood of picking up the title in a school or health center; and interest in other titles.

Following the quantitative data collection, qualitative data was collected from group participants in 45-minute group discussions. During these sessions group participants were encouraged to share their impressions of each booklet and the booklet's impact on them. This qualitative information was used to support, elaborate on, and provide context for the quantitative data drawn from the surveys.

2) Survey Groups

Data collection in the survey groups was similar to that of the survey/discussion groups described above, except that no discussion groups were held. The 280 teenagers who took part in the survey groups read the booklets and completed the same pre- and post-survey questions in sessions that lasted 1 hour. There were 15 of these survey group sessions held.

TABLE 1. Two Types of Data Collection Groups


Survey/discussion groups

Survey-only groups

Activity of group

Read booklets,
complete surveys,
and hold discussions

Read booklets
and complete surveys

Type of data

Quantitative and qualitative

Only quantitative

Length of

2 hours

1 hour




Number of

8 to 15

As many as could be recruited

Incentive paid
to participants



Total number
of groups held



Total number
of participants



Guiding all data collection was a standardized data collection protocol. This protocol clearly directed how the JSI/WEI facilitators were to administer pre- and post-surveys and to lead group discussions. The data collection tools were pilot-tested in a public school and community health center and revised prior to their use in the data collection groups.

Quantitative information was analyzed using the SPSS (Statistical Package for the Social Sciences) Software. Qualitative data collection was tape-recorded. JSI/WEI team members then used these tapes to develop partial transcripts for each of the eight discussion groups.


Overall, the study proved that Scriptographic booklets are an effective means of communicating information and influencing attitudes and intention to change behavior with teenage readers in all racial groups studied. The study also found that teenage readers find Scriptographic booklets to be easy to read and relevant to their lives.

Key findings of the quantitative data collection and analysis are shown in Table 2. (All figures are averages of survey results generated in the survey/discussion groups and the survey groups.)

TABLE 2. Key Findings of Quantitative Data Collection and Analysis


65% reported learning "some" or "a lot" from the booklets.
Among those reporting very little prior knowledge of the topic, 80% reported learning "some" or "a lot."
Actual knowledge gain of those reporting very little prior knowledge was 33%.

to act

25% more teens reported they were "likely" or "very likely" to take positive action after reading a booklet.

Among those reporting very little prior knowledge, those reporting they were "likely" or "very likely" to take positive action increased by an average of 32 percentage points.

Ease of

93% found the booklets easy to read.
reported understanding all or most of the booklets.
felt the text-and-graphics format helped make the booklets easier to read.
felt the amount of information on booklet pages was "just right."


81% felt the booklets were "interesting" or "very interesting" to read.
felt the booklets were "useful" or "very useful" in explaining health topics.
were "likely" or "very likely" to recommend the booklets to someone they knew.
were "interested" or "very interested" in seeing other Scriptographic booklets.

Overall appeal

64% liked the color of the booklets.
liked the drawings in the booklets.
felt that the stories in story format booklets could be real.

The qualitative portion of the study provided the survey/discussion group participants an opportunity to expand on their responses to the pre- and post-surveys. Results of this qualitative data collection confirm the information gathered during quantitative data collection: Scriptographic booklets are effective at communicating important health information and influencing intention to act. The booklets are also easy to read, interesting, and relevant to readers.

Overall, participants were positive about the educational materials they reviewed. They expressed a belief that the topics in the booklets were relevant to their lives:

"I like the fact that these booklets are very direct and interesting to read, with real situations."

It was common for participants to request permission to take home the booklets they had read. Many participants said that they would share what they'd learned with friends or family members:

"I would give this to my cousin. He should read this."

"I would definitely recommend this to someone I know who needs this information."

"After reading the booklets, I would know how to help a friend better."

Another theme in participants' responses was the idea that the information in the booklets was important to them:

"These booklets raise the right issues that people often don't talk enough about."

"It's good to see a booklet on anger 'cause nobody talks about it that much in school."

"It's important to keep getting information on HIV because not everybody knows all the facts."

One facet of the booklets' appeal was their ability to address topics that otherwise might not be easily discussed among some teenagers:

"I think these booklets would be good for uncomfortable topics like sex and HIV, which is hard to talk to your friends about."

Participants were especially likely to express appreciation for the information in the booklets in cases where the booklet topic was one about which they were generally less knowledgeable. In general, participants who had completed fewer years of school -- and so had received "less" education -- found the booklets more helpful. But even those who had previously received information on the booklet topics found the booklets helpful:

"It's good to remind yourself of these things, because you can forget."

In general, participants in all racial groups and at all ages found the booklets to be easy to read:

"I thought the booklets were easy to follow."

"The information was pretty clear and straightforward."

The drawings in the booklets were reported to be an important aid in improving comprehension:

"If you don't understand something, you can go back and look at the pictures and get the information that way."

"Sometimes you can look at the pictures and get the same information."

"I feel like I learned a lot just by reading along with the pictures."

The qualitative portion of the study also provided an opportunity to evaluate readers' responses to the various formats. While the study did not specifically seek to compare formats, it did yield a variety of opinions on preferred presentation of information. Some readers responded more positively to the fact-based presentation of the Classic and realistic illustrated formats. Others preferred the story approach of the story format:

"It's easier to get information through a story."

"I'd be much more likely to read the booklet if it were a story."

"I can identify with stories better than just lots of information."

"The stories seem real because this often happens in high school."

While there was general agreement that the illustrations in all styles aided in understanding, each of the different styles drew praise from different participants. For example, some participants favored the Classic illustration style (which they characterized as a "cartoon" style) because it was perceived as "race neutral" and because it established a tone that seemed more appropriate for the topic under discussion:

"I thought it was kind of neat to see them talk about anger with the cartoon characters."

"Sometimes it's effective to use cartoons with a serious topic."

Other participants favored the styles used in the realistic illustrated and story format booklets. Reasons given were that the realistic style appeared more modern and more like "real" people they could relate to. Racial representations were another factor:

"I could see race better in this one [a RealStory booklet]."

These findings suggest that CBC's different formats offer important options for more effectively communicating with readers who differ in their preferred styles of acquiring information.


The study conducted by John Snow, Inc., and World Education, Inc., provides an independent, statistical, and qualitative analysis of the overall effectiveness of CBC educational materials. The study was structured to isolate the effects of the booklets apart from other elements of the comprehensive educational programs in which the booklets are typically used: specific environmental factors, larger program objectives and components, and so on.

The study demonstrates that CBC materials are a highly effective means of transferring knowledge and influencing readers' intention to act in positive ways. The study also confirms that the materials are regarded as easy to read and relevant to the lives of teenage readers.

The general findings of this study are consistent across all the booklet formats studied. Statistical data yielded similar findings for all racial groups included in the study and across all other demographic variables that were identified.

The study did reveal some differences in the extent to which participants intended to change their behavior in positive ways. Generally speaking, the amount of increase reported in participants' intention to act was proportional to the amount of knowledge about the topic in question reported in pre-surveys: Those readers who had the least prior knowledge of a given topic reported the greatest increase in their intention to act; those with the most prior knowledge had the smallest increase.

These findings are consistent with the stages-of-change behavioral model, which holds that people pass through several distinct stages on their way toward behavior change. Different types and levels of intervention are required at each of these stages. As people progress along the continuum of change, it may become more difficult to measure the smaller, incremental gains in knowledge and changes in attitudes that accumulate to move individuals into new stages of change. In the later stages, repeated and multifaceted exposure to information and motivation is required to overcome resistance and continue progress toward the ultimate goal of adoption of a new behavior. Thus, the impact of a booklet will be greatest on those who possess the lowest preexisting level of information and motivation.

The Channing Bete Company, Inc., recognizes the importance of providing effective educational materials to its customers and readers. The Product Effectiveness Study with In-School Teenagers is part of CBC's ongoing efforts to test and ensure the effectiveness of its educational materials.