Long-term evaluation of EFNEP and SNAP-Ed

Long-term evaluation of EFNEP and SNAP-Ed

Mary Kay Wardlaw
Family and Consumer Sciences Cooperative Extension
University of Wyoming

Abstract

Nutrition education programs targeting low-income populations are provided across the United States with varying behavioral assessments. However, few programs have conducted research with graduates more than one year after their participation. A mixed-method research project using quantitative and qualitative techniques assessed the long-term impacts of the Expanded Food and Nutrition Education Program (EFNEP) and the Supplemental Nutrition Assistance Program Education (SNAP-Ed) in Wyoming. Adults who participated in the Wyoming program at least one and up to four years previous to the research, completed an eighteen-item behavior checklist. Each participant filled out the behavior checklist when they enrolled in classes, when they graduated, and at follow-up. These three checklists were matched for each respondent to observe changes in behavior over time. Semi-structured interviews were conducted to learn more about changes related to food and nutrition behaviors, as well as other life changes attributable to their involvement in the program.

Keywords
nutrition education, low-income adults, EFNEP, SNAP-Ed, long-term evaluation, paraprofessional educators

Introduction

This research sought to learn if participants in a nutrition education program for low-income adults maintained, decreased, or increased food- and nutrition-related behaviors after their graduation and how graduates believed their involvement in the program affected their lives. The first component of this mixed-method study was a quantitative examination of the maintenance of food- and nutrition-related behaviors at least one year and up to four years after completion of a series of lessons. The second component was a basic interpretive qualitative evaluation of how participants’ involvement influenced nutritional and non-nutritional aspects of their lives. This article discusses the quantitative results and the nutritional qualitative results. In Wyoming, the nutrition education program for low-income families is called the Cent$ible Nutrition Program (CNP) and is funded through the Expanded Food and Nutrition Education Program (EFNEP) and the Supplemental Nutrition Assistance Program Education (SNAP-Ed).

Statement of problem

The United States government commits more than $400 million annually across states and territories for nutrition education intervention programs targeting low-income families with the long-term goal of reducing diet-related disease and food insecurity (U.S. Department of Agriculture, Food, and Nutrition Service 2012; U.S. Department of Agriculture, National Institute for Food and Agriculture 2012). Research into the long-term effects of this education on families is limited. Little is known regarding whether and how low-income adults maintain improved nutrition and money-saving behaviors. Additionally, few studies have analyzed past participants’ stories of how their involvement in nutrition education programs positively influenced varying aspects of their lives. Understanding more about this audience can aid future nutrition education efforts.

Background

The United States Department of Agriculture (USDA) has responded to nutritional needs of low-income Americans by funding EFNEP and SNAP-Ed. EFNEP is part of the National Institute for Food and Agriculture and was initiated in 1969 as one response to hunger in the United States with a goal to help low-income families improve nutrition (Brink 2000). SNAP-Ed began in 1992 as part of the Food and Nutrition Service to promote healthful diets through partnerships with states (Guthrie, Stommes, and Voichick 2006). EFNEP and some SNAP-Ed programs, such as that in Wyoming, deliver nutrition information through group education classes. EFNEP-type programs have been shown to have an influence on eating behaviors (Cullen et al. 2009).

Wyoming’s Cent$ible Nutrition Program, which includes both EFNEP and SNAP-Ed, uses trained paraprofessionals to teach adults how to feed their families better with fewer resources. As part of the community-based program, participants complete an average of 8.5 lessons over an eight- to ten-week period. Areas of emphasis taught through an established and tested interactive curriculum are as follows:

• Food resource management includes practices related to menu planning, thrifty shopping, and awareness of supermarket persuasion techniques.
• Food safety includes safe handling, preparation, and storage of food.
• Dietary quality includes eating and lifestyle behaviors consistent with the Dietary Guidelines for Americans (U.S. Department of Health and Human Services and U.S. Department of Agriculture 2010).

CNP lessons incorporate a dialogue approach to education relying on the expertise and knowledge of learners to enhance the learning environment. Elements of the lessons are patterned after the methods developed by Norris (2003). This method moves educators from the traditional telling and showing model of teaching toward a doing and knowingparadigm. The goal is to create a supportive, positive, and respectful place for low-income adults to find success in learning. The EFNEP Behavior Checklist, with eight additional questions, and twenty-four-hour dietary recall are used at entry and exit (standard program protocol) with adult participants to evaluate program impact.

Research design        

The data collection methods included administering of a follow-up Behavior Checklist and conducting qualitative semi-structured interviews in addition to the standard entry and exit evaluation protocol. The research proposal was submitted and approved by the Institutional Review Board of the University of Wyoming.

Since 1996, the EFNEP program has used the Evaluation and Reporting System (ERS) with a collection of ten core behavior questions to evaluate key food- and nutrition-related behaviors. In Wyoming, an additional eight questions were added from the national checklist question database to capture program-specific desirable behavior changes (U.S. Department of Agriculture, Cooperative State Research, Education and Extension Service 2007). The quantitative research used a quasi-experimental research design (Ary et al. 2006) and incorporated the eighteen-item EFNEP Behavior Checklist. This was completed by participants when they enrolled and when they graduated from the program and was administered in a group setting by paraprofessional educators. The same tool was administered via mail to research participants one to four years following graduation as a repeated-measures study.

The qualitative data collection was basic interpretive emergent design (Ary et al. 2006). Semi-structured interviews with program graduates were conducted after quantitative data collection and preliminary analysis was complete. Adult respondents of the quantitative assessments were selected at random, contacted, and invited to participate in an interview.

The population for this study was past graduates from CNP. The pre-existing intact group that comprised the study sample was graduates enrolled one to four years previous to this data collection, encompassing three program years from 2005 through 2007. Only adult graduates, 18 years of age or older during their participation in the program, were part of the sample. Eligibility for this study required complete entry and exit survey Behavior Checklists and a valid mailing address. These graduates reported consistent improvements in assessed behaviors from entry to exit.

Instrumentation: Behavior Checklist

The eighteen-item Behavior Checklist has been part of Wyoming’s EFNEP and SNAP-Ed evaluation for twelve years. Each behavior statement is answered with a 1- to 5-point Likert-like scale indicating 1 as never, 2 as seldom, 3 as sometimes, 4 as most times, and 5 as always. The conceptual domains or subscales are food resource management, nutrition practices, and food handling and safety (Anliker 2010). In this checklist there are five statements addressing food resource management, nine statements addressing nutrition practices, and four statements addressing food handling and safety.

This study included three data collection points using the Behavior Checklist. These time points were at entry when participants started the program, at exit when they completed the program eight to ten weeks later, and at follow-up one to four years after completion. The follow-up checklist was sent to 1,062 past graduates. Procedures and development for the mailings and documents were based upon survey methodology developed by Don Dillman (Dillman 1978; Dillman 2000; Salant and Dillman 1994) and past experiences with applied research. The four-part mailing procedure employed over the span of two months included an advance-notice postcard, the instrument packet, follow-up thank you postcard, and replacement instrument packet to non-respondents.

Instrumentation: Interview

Semi-structured interviews with past graduates added basic interpretive qualitative data and enhanced understanding of program effects. Following guidelines from Krueger and Casey (2000), interview questions were conversational and used words familiar to participants. Additionally, the questions were clear, easy to say, open-ended, short, and one-dimensional. The question sequence facilitated flow and increasing depth in responses. An evaluation specialist reviewed the questions. Former program graduates not included in this study were used in pilot testing.

The questions asked relating to food behaviors included the following progression:

• One of the areas taught in the program is food safety. What kinds of food safety things did you learn?
• What food safety behaviors do you still do, as a result of the class?
• How have your hand-washing practices changes since you took the class?

Following the interview, a printed copy of the transcript was mailed to each interviewee for their review. Following analysis, qualitative findings and conclusions were also sent to all interviewees. Checking with participants for validation of accuracy prior to dissemination is also known as member checking (Vaterlaus and Higginbotham 2011). Another analysis check was conducted with CNP paraprofessionals who reviewed the qualitative results and commented on the consistency of the findings with their experiences with program graduates.

Research findings

The Behavior Checklist allowed respondents to indicate the frequency with which they perform selected behaviors. The eighteen behavior statements were as follows:

1. Plan meals ahead of time

2. Compare prices before buying food

3. Run out of food before the end of the month

4. Shop with a grocery list

5. Let meat and dairy foods sit out for more than 2 hours

6. Thaw frozen foods at room temperature

7. Think about healthy food choices when deciding what to feed yourself or your family

8. Prepared foods without adding salt

9. Use the “Nutrition Facts” labels to make food choices

10. Eat/serving something within 2 hours of waking

11. Serve more than one kind of fruit each day

12. Serve more than one kind of vegetable each day

13. Eat whole grain bread

14. Make food from scratch

15. Wash hands with soap and warm running water before preparing food

16. Separate raw meat, poultry, and fish from vegetables, fruits, and prepared foods

17. Physically active for at least 30 minutes per day, on 4 or more days per week

18. Order a super-sized portion of food or beverage when it is an option

Four of the checklist items, numbered 3, 5, 6 and 18, were reverse scaled prior to analysis as a higher score indicates less desirable behavior. For analysis, a significance level of .05 was assumed unless otherwise noted and p values were reported. Cronbach’s alpha was calculated as a measure of homogeneity for internal consistency estimates of reliability. The three administrations had similar moderate to high reliability: at entry α = .78, at exit α = .80, and at follow-up α = .77. Each item fell into one of three subscales, food resource management (FRM), food safety and handling (FS), and nutrition practices (NP). FRM reflected items 1-4 and 14. FS reflected items 5, 6, 15, and 16. NP reflected items 7-13, 17, and 18.

The 493 respondents represented a 46.4 percent response rate. Respondents were similar to the study sample in mean family size and years since participation. The groups differed somewhat as a higher percentage of respondents compared to the study sample were female (88 percent to 77.1 percent), more than 40 years of age (55.7 percent to 31 percent), white (85.8 percent to 75.1 percent), and living in rural locations and towns with fewer than 10,000 people (56.4 percent to 46.4 percent). A lower percentage of respondents compared to the study sample were Hispanic/Latino (4.3 percent to 10.5 percent), Native American (6.1 percent to 10.2 percent), and African American (0.2 percent to 1.1 percent).

The biggest difference between the study population and respondents was in income. CNP targets people in households with incomes at or below 185 percent of the federal poverty level. Of the respondents, 60.5 percent were at or below this income level at follow-up assessment while 91.2 percent of the study population was at or below this level at enrollment. The mean family size remained similar, suggesting that some respondents had increased household income since their involvement in the program.

The nineteen female interviewees ranged in age from 27 to 50 years with a mean age of 37.1 years. The interviews were conducted by the same interviewer, a white female in her late 40s, and varied from 18 to 43 minutes with an average length of 28.5 minutes. The majority, sixteen, lived in the same community as when they enrolled in CNP. According to self-reported data, sixteen interviewees were white, one American Indian, one Hispanic, and one Asian.

Five of the women currently had no children living in their households. Four of these lived with a spouse and one lived alone. Of the fourteen households with children, one included a grandchild. Eighteen of the interviewees participated in a group series of classes while two participated in the program one-on-one with the paraprofessional educator.

Graduates maintain food- and nutrition-related behaviors

CNP graduates maintained many positive food- and nutrition-related behaviors one to four years following their participation in the program, and they performed these behaviors more often than they did before they started the program. The behaviors were analyzed using the three subscales FRM, FS, and NP.

Mean scores for the subscales at the three data collection points were calculated and are depicted in Figure 1. Standard deviations were consistent across the scales and data collection points ranging from 0.49 to 0.67.

Figure 1. Mean scores for subscales at three data collection points.

[Alt tag content for Figure 1: Mean scores for subscales at entry, exit, and follow-up.]

There were no main or interaction effects seen based on respondent age, number of years since participation, where they lived, or program eligibility. To further investigate the effect of means over time, Cohen’s d was calculated (Ary et al. 2006) and displayed in Table 1. The three pairings for this analysis were entry to exit, entry to follow-up, and exit to follow-up. Increases from entry to exit and entry to follow-up were considered a large effect size. The effect size of the decrease from exit to follow-up was small.

Table 1. Effect size (Cohen’s d) for changes over time

Scale	Entry to Exit d	Entry to Follow-up d	Exit to Follow-up d
FRM	.717	.583	.133
FS	.720	.610	.160
NP	.724	.638	.091

Note. FRM = Food Resource Management; FS = Food Safety; NP = Nutrition Practices.

[Table 1 Summary. Effect size (Cohen’s d) for changes over time.]

All scores had a significant increase from entry to exit and from entry to follow-up. Table 2 shows the paired-samples t tests results for entry to follow-up and for exit to follow-up. For all pairs, the change was significant. In all subscales, a positive change or improvement occurred from entry to follow-up and a slight negative, or decrease in improvement from exit to follow-up.

Table 2. Paired samples t tests for subscales

Scale Pairs	M	SD	df	t	p
FRM
Entry to follow-up	.36	.61	492	12.90**	.000
Exit to follow-up	-.08	.54	492	-3.40**	.001
FS
Entry to follow-up	.27	.56	492	10.87**	.000
Exit to follow-up	-.08	.55	492	-3.20**	.001
NP
Entry to follow-up	.38	.58	492	14.41**	.000
Exit to follow-up	-.05	.53	492	-2.01*	.045

Note. Pairings indicate behavior maintenance. FRM = Food Resource Management; FS = Food Safety; NP = Nutrition Practices. *p < .05 and **p < .01, two tailed.

[Table 2 Summary. Paired samples t tests for subscales.]

The finding of positive changes over time was supported through the qualitative data. Within the food resource management area, respondents provided sixteen different thematic behaviors they had learned and eleven of those were maintained since program completion. Results indicated past graduates remembered skills learned and continued to use a grocery list, compared prices, planned menus, bought on sale, and cooked from scratch. The behavioral theme mentioned most often was using a grocery list.

When asked what she did to save money on food, one participant replied, “Making a menu and the basic mixes. I really like that. The Bisquick® copy-cat recipe and the basic Master Meat mix I use a lot. I think just making a menu before you go and the grocery list.”

Interviewees reported that when prices increased, they more often cooked at home, planned and used leftovers to eliminate waste, and focused on needs rather than wants by reducing convenience food items. Further, they relied heavily on sales and considered alternative forms of foods that cost less, such as canned fruit instead of fresh. Two individuals said they had cut back on buying fruits and vegetables, particularly fresh, to save money.

One participant expressed her financial frustrations, saying: “We eat a lot more ramen [noodles], which I know is not the healthy choice. But, if I can feed my entire family for a dollar, then that is what we are looking at.  … It’s a little pathetic. The kids have been sick for a while because we don’t have the money for food.  … We’ve got to get gas in the car, got to pay our rent, got to pay our electricity. So the food area has been suffering.  … You know health-wise, the kids go from being really happy and healthy and even good skin color and eyes looking nice and bright, to we don’t have the money for a week or two and they’ve got stomach problems and they’re lethargic and want to sleep.”

For the food safety area, respondents identified eleven thematic behaviors learned and ten maintained.  Most often mentioned were food safety behaviors related to chilling food and cleaning food preparation areas and hands. The FS subscale had the highest starting mean, suggesting that respondents reported more of the desirable food safety behaviors when they enrolled in the program compared to behaviors in food resource management and nutrition practices. This was consistent with respondents who said food safety lessons refreshed what they already knew.

Four individuals reported that they taught their spouses or children what they learned. One participant said, “I had fifth-, sixth-, and seventh-graders at that time … so they could help out, and I’d remind them why you have to keep dirty meat stuff away from fresh vegetables and fruits and whatever else you’re preparing so they didn’t get sick.”

In analyzing qualitative responses to the nutrition practices area, respondents identified twelve thematic areas learned and ten maintained. Most commonly reported were increasing fruits and vegetables and selecting and cooking food lower in fat, sugar, and salt. When asked specifically about changes in cooking, respondents most often talked about making food from scratch including the master mixes which are taught in the program.

One participant shared her thoughts, saying: “I use it [what I learned] every day actually. Every day with what you put into your body. Kind of like a gas tank, what you fill your body with depends on how well your engine will run. We try and eat healthier each day and get those lessons to my kids.”

The quantitative and qualitative data shared here represent past graduates’ reported behaviors and information learned and maintained over time in the areas of food resource management, food safety, and nutrition practices. Data was also collected regarding other life changes beyond food and nutrition; however, it is not reported in this article.

Discussion

The purpose of the study was to examine and enhance the understanding of sustainability or maintenance of positive behavior changes resulting from nutrition education taught to low-income adults who took part in the Cent$ible Nutrition Program. Based on the literature, to do this effectively changes have to be considered over time, from at least six months to multiple years (Sun, Prochaska, Velicer, and Laforge 2007).  

We expect participants to perform desired behaviors with the greatest frequency at the end of a series of lessons. Studies examining skill-based behaviors taught in adult nutrition programs, such as EFNEP, indicate that measured outcomes may be maintained for at least three months after conclusion of the intervention (McDonald, Kranch, and Hongu 2011). However, research on effective health education programs for adults repeatedly indicates that initial implementation of new skills is not typically sustained over the long-term, and uptake, adoption, and maintenance vary greatly (Blissmer et. al 2010; Oldenburg and Parcel 2002). Behaviors are more likely to be sustained over time when instructors are supportive and when learners feel they have a choice, they have the ability, and they are not pressured from external demands (Deci and Ryan 2002).

Participants in this study exhibited increases in positive behaviors from the beginning to the end of the series of classes. There were only small decreases in these same positive behaviors at follow-up assessment, and these decreases did not increase over time. At one year, two years, and even three years after graduation, participants maintained most of the improved behaviors reported at graduation.

Quantitative results documented frequency of selected behaviors, and qualitative findings showed “how” graduates maintained positive changes. The data offered insights as to how and why graduates made these changes and how they incorporated these behaviors into their lives. Qualitative research, such as this, may help us improve programs locally and influence broader adult education efforts by gaining a deeper understanding of participants (Higginbotham, Henderson, and Adler-Baeder 2007). In this study, the interviews supported the behavior changes indicated in the behavior checklist and confirmed graduates’ understanding of how their lives were changed as a result of their participation in CNP.

Conclusion

Leaders of nutrition programs for low-income families, like EFNEP and SNAP-Ed, have assessed participants’ behaviors pre- and post-series for many years. We have reported short-term improvements in those behaviors and have heard stories anecdotally from participants through paraprofessional educators. This study provides evidence that these behaviors are maintained. This study also systematically examined participants’ stories about the program’s positive impacts on their food- and nutrition-related behaviors. EFNEP and SNAP-Ed delivered in a series of hands-on educational lessons by paraprofessionals are effective in improving food and nutrition practices, and their impacts are maintained for low-income adults in Wyoming.

The lack of long-term research for nutrition education programs signifies a need for further studies. These studies would need to target low-income participants and could examine additional benefits such as life changes beyond food and nutrition behaviors.

 

 

References

Anliker, Jean. 2010. “Behavior checklist: Historical perspective, development and testing.”www.csrees.usda.gov/nea/food/efnep/neers5.html

Ary, Donald, Lucy Jacobs, Asghsar Razavieh, and Christine Sorensen. 2006. Introduction to research in education, 7^th edition. Belmont, Calif.: Thompson Wadsworth.

Brink, Muriel. 2000. Expanded Food and Nutrition Education Program: A precedent-setting program. Cortland, New York: EasyWriter Publications.

Cullen, Karen, Agueda Lara-Smalling, Deborah Thompson, Kathleen Watson, Debra Reed, and Karen  Konzelmann. 2009. Creating healthful home food environments: Results of a study with participants in the Expanded Food and Nutrition Education Program. Journal of Nutrition Education and Behavior 41(6): 380-388.

Deci, Edward, and Richard Ryan (eds.). 2002. Handbook of self-determination research. Rochester, N.Y.: The University of Rochester Press.

Dillman, Don. 1978. Mail and telephone surveys: The total design method. New York: John Wiley and Sons.

Dillman, Don. 2000. Mail and Internet surveys: The tailored design method. New York: John Wiley and Sons.

Blissmer, Bryan, James Prochaska, Wayne Velicer, Colleen Redding, Joseph Rossi, Geoffrey Greene, Andrea Paiva, and Mark Robbins. 2010. Common factors predicting long-term changes in multiple health behaviors. Journal of Health Psychology 15(2). http://hpq.sagepub.com/content/15/2/205

Guthrie, Joanne, Eileen Stommes, and Jane Voichick. 2006. Evaluating Food Stamp Nutrition Education: Issues and opportunities. Journal of Nutrition Education and Behavior 38(1): 6-11.

Higginbotham, Brian, Katie Henderson, and Francesca Adler-Baeder. 2007. Using research in marriage and relationship education programming. Forum for Family and Consumer Issues 12(1). http://ncsu.edu/ffci/publications/2007/v12-n1-2007-spring/higginbotham/fa-4-higginbotham.php

Krueger, Richard and Mary Anne Casey. 2000. Focus groups: A practical guide for applied research, 3^rd edition. Thousand Oaks, Calif.: Sage Publications, Inc.

McDonald, Daniel, Rachel Kranch, and Nobuko Hongu. 2011. Follow-up evaluation to determine short-term effectiveness of a nutrition education program in a primarily Latino population. The Forum for Family and Consumer Issues, 16(1).http://ncsu.edu/ffci/publications/2011/v16-n1-2011-spring/mcdonald-kranch-hongu.php  

Norris, Joye. 2003. From telling to teaching: A dialogue approach to adult learning. Myrtle Beach, S.C.: Learning By Dialogue.

Oldenburg, Brian, and Guy Parcel. 2002. Diffusion of innovations. In Health Behavior and Health Education, by Karen Glanz, Barbara Rimer, and Frances Marcus Lewis. San Francisco: Jossey-Bass.

Salant, Priscilla, and Don Dillman. 1994. How to conduct your own survey. New York: John Wiley and Sons.

Sun, Xiaowa, James Prochaska, Wayne Velicer, and Robert Laforge. 2007. Transtheoretical principles and processes for quitting smoking: A 24-month comparison of a representative sample of quitters, relapsers, and non-quitters. Addictive Behaviors (32): 2707-2726.

U.S. Department of Agriculture, Cooperative State Research, Education and Extension Service. 2007. Supporting documentation for ERS4.http://www.csrees.usda.gov/nea/food/efnep/ers/documentation.html

U.S. Department of Agriculture, Food and Nutrition Service. 2012. Approved Federal funds for supplemental nutrition assistance program education. http://www.nal.usda.gov/foodstamp

U.S. Department of Agriculture, National Institute for Food and Agriculture. 2012. Expanded Food and Nutrition Education Program Formula Grant Opportunity.http://www.nifa.usda.gov/business/awards/formula/smithlever.html

U.S. Department of Health and Human Services and U.S. Department of Agriculture. 2010. Dietary Guidelines for Americans, 2010. http://health.gov/dietaryguidelines/

Vaterlaus, J. Mitch and Brian Higginbotham. 2011. Qualitative program evaluation methods. Forum for Family and Consumer Issues 16(1). http://ncsu.edu/ffci/publications/2011/v16-n1-2011-spring/vaterlaus-higginbotham.php

http://www.ncsu.edu/ffci/publications/2012/v17-n2-2012-summer-fall/index-v17-n2-december-2012.php

 

 

Back to table of contents ->https://www.theforumjournal.org/2017/09/03/fall-2012-vol-17-no-2/