Challenges & Lessons Learned

The biggest challenge for any outreach program is how to measure behavioral impact. Sesame Workshop used qualitative research tools -- interviews and surveys to hear directly from end users -- and with Sesame Beginnings, they learned the following:

Establishing content messages:

• Sesame Workshop usually begins an outreach project with a meeting of outside advisors in an effort to learn as much as possible about the topic and to get a fix on what the educational messages should be. Because that kind of information was already available from Sesame Street Beginnings: Language to Literacy, Sesame Workshop was presented with an opportunity to be flexible and to use other methodologies. Thus, instead of starting with an advisory panel, we began with focus groups of intended target audiences.
• Because some of the findings were contradictory from one group (teachers) to another (parents), it is necessary to probe each focus group to get at the heart of what is really being said.
• To ensure that the educational messages are clear and not daunting, and that each component of the kit is consistent with and complementary to the other components, it is necessary to keep messages few and simple and to repeat them often in every component.

Bilingual execution:

• Because it is inevitable that you will need to go back to your translators several times as materials are being developed and polished, it is advisable to establish an ongoing and respectful relationship with them.
• To ensure that recipients who will use the materials developed in languages other than English feel respected and comfortable with them, it is necessary to make certain all materials are both culturally and linguistically relevant. This means that translations should be developed in conjunction with development of the materials in English so that concepts or phrasing that doesn't resonate with the non-English speaker can be caught and adjusted in both English and the other language(s).

Design:

• Because black-and-white materials don't carry the same weight as full-color ones, try to produce everything in full color.
• Because people may assume that the order in which components are stacked in the kit somehow indicates order of importance, it is advisable to put the most "important" ones on top.
• Because recipients can become confused with too many components, it is wise to keep the number of components to a minimum.

Distribution:

• To ensure that the materials reach their intended recipients in a timely manner, develop a distribution plan well before launch and in conjunction with outside partners representing those you want to reach.
• To simplify and speed up the ordering/fulfillment process, try to use e-mail ordering as much as possible. To reduce costs, have orders go directly to the fulfillment house so that internal staff time isn't consumed by processing and fulfilling orders and keeping records of this activity.

Feedback:

• It is important to funders to learn as much as possible about how the project materials have been used and what the impact of this use might be. The challenge is to collect such information in a timely and cost-effective way. Sesame Workshop found online surveys to be very useful. To provide an incentive for people to fill out and return such online feedback, they offered a free Talk, Read, Write! screen saver.
• There is an art to constructing survey questions so that they will give you the information you need and are easy and inexpensive to scan electronically. Sesame Workshop worked extensively with their research division and with an outside company that specializes in surveys to help in this effort. They also contracted with the outside company to scan the surveys and compile the results.
• In addition to surveys, Sesame Workshop also conducted summative research via observations and interviews.