Backup — original pdf

Backup

NEW LEAF AGRICULTURE EVALUATION REPORT 2024 TABLE OF CONTENTS 01 02 03 04 05 06 07 Introduction Methods, Quantitative Data Results, Quantitative Data Methods, Qualitative Data Results, Qualitative Data Conclusion Acknowledgements UTHealth | New Leaf Agriculture Report 2024 INTRODUCTION The Multicultural Refugee Coalition (MRC) is a non-profit organization in Austin, Texas, which creates refugee livelihoods through employment, training, and education. Since its inception in 2009, MRC has supported nearly 1,000 refugees from 20 countries by employing and training refugee farmers ensuring their long-term success as engaged community members. One of MRC’s programs, New Leaf Agriculture, is based on a 20- acre USDA-certified organic farm where refugee farmers learn about Central Texas agriculture methods and practice organic and regenerative farming techniques. The program is also designed to provide supplemental income to farmers and increase farmers’ families’ and their communities’ healthy food access. One component of the New Leaf Agriculture program focuses on primary farmers. Primary farmers are refugee farmers who work on the 20-acre farm and are full-time employees of MRC. They have a scheduled work time, are paid a wage, and receive additional benefits such as free transportation and English as a Second Language (ESL) classes. A second component of the New Leaf Agriculture program focuses on community farmers. Community farmers are refugee farmers who each manage a 750 sq. ft. production-size plot on the New Leaf Agriculture farm and earn supplemental income through produce sales. Before receiving a plot, farmers participate in a six-week training course on regenerative farming practices in Central Texas. New Leaf Agriculture provides supplies, land, education, and culturally specific seeds to the community farmers. Farmers choose which crops to grow, focusing on what they and their community desire, and farm according to their schedule. Produce grown by farmers is bought by MRC and then distributed either by the farmers to their communities or through partnerships with local organizations. UTHealth | New Leaf Agriculture Report 2024 As recipients of the Refugee Agricultural Partnership Program (RAPP) grant, MRC is expanding its New Leaf Agriculture program by providing agricultural land and training for at least 80 refugees. Evaluation of the program will be conducted yearly by The University of Texas Health Science Center School of Public Health (UTHealth) research team over three years. The evaluation is formative in design and fulfills the purpose of tracking the program’s progress toward anticipated short- and long-term outcomes while contributing to continuous program improvement. The evaluation employs multiple methods, including interviews, surveys, and pre-post assessments. This interim report presents the results of the year 1 evaluation via quantitative and qualitative assessment of the New Leaf Agriculture program. UTHealth | New Leaf Agriculture Report 2024 METHODS Overview of Evaluation The evaluation study conducted by UTHealth employed a mixed-methods approach to assess the impact of the New Leaf Agriculture program and to collect process data. Quantitative data collection involved a survey measuring access to healthy food, knowledge of a healthy diet, physical activity, confidence in finding answers to certain questions, and demographics (Farmer Survey). Furthermore, a pre- and post-test was given prior to and after the regenerative farming training course measuring knowledge of soil and seeds, irrigation and planting, fieldwork, harvesting and processing, planning and distribution, and organic and regenerative systems (Module Quizzes). Qualitative data were gathered through semi-structured interviews with primary and community farmers assessing their experience with the New Leaf Agriculture program, knowledge of Central Texas farming techniques, challenges, and ways the program can be improved. Instrument development, recruitment, data collection, and data analysis plans varied based on the methodological approaches and are described below. This research was approved by the UTHealth Institutional Review Board (IRB) before data collection (IRB NUMBER: HSC-SPH-23-0297). UTHealth | New Leaf Agriculture Report 2024 METHODS Quantitative Data The quantitative portion of the study was two-fold. It includes a Farmer Survey, which covers indicators such as access to healthy food, knowledge of a healthy diet, physical activity behaviors, and community engagement. Surveys are offered in both English and the farmers' native languages. Additionally, six module quizzes, aligned with New Leaf Agriculture's educational content, were created and administered, with quiz score analysis conducted to assess the impact of instruction on farmers' knowledge, particularly in the first module where before-and-after instruction comparisons were possible. Farmer Surveys UTHealth developed an 18-question survey measuring health indicators as required by the RAPP grant. Those indicators included: access to healthy food, knowledge of a healthy diet, physical activity behaviors, ease of navigating systems for information on nutrition, health, and gardening, community engagement behaviors, and assessing whether their income is adequate for various expenses. The survey concluded with a set of demographic questions. The surveys were offered in English and in the native tongue of the farmer. Those who wished to answer in their native language had an opportunity from LanguageLine, an on-demand language interpretation services provider. Surveys offered in English took about 20 minutes whereas those with an interpreter took twice as long. interpreter to speak to an UTHealth | New Leaf Agriculture Report 2024 METHODS Farmer Recruitment and Data Collection MRC assisted data collection by providing names and contact information of both primary and community farmers (n=14) to UTHealth evaluators and by gathering a portion of survey data. Whereas MRC asked survey questions in-person at the farm, the UTHealth team conducted the remaining survey over the phone. Data collection spanned 3 months, from July through September 2023. Data Analysis Survey data were entered into Google Sheets and analyzed to derive meaningful insights. Descriptive statistics, including measures such as mean, median, and frequencies, were calculated to summarize the central tendencies and variability within the dataset. Module Quizzes UTHealth also created six module quizzes in collaboration with MRC. Each quiz had questions based on the education materials and topics covered throughout the six modules New Leaf Agriculture teaches. Researchers drafted items for the quizzes and the preferred questions selected by the New Leaf Farm Director. The quantity of questions in every quiz fluctuated depending on the content covered in each module and the specific information that New Leaf Agriculture aimed to highlight for the farmers. Because most farmers spoke languages other than English, questions were designed with ease of translation in mind. They were formatted as either multiple-choice, matching, or fill-in-the-blank. The topics of each module are as follows: Module 1 – Soil and seeds (n=4 items); Module 2 – Irrigation and Planting (n=3 items); Module 3 – Field Work (n=3 items); Module 4 - Harvesting, Processing, Food Safety, and Nutrition (n=10 items); Module 5 - Crop Planning and Distribution (n=3 items); and Module 6 - Organic and Regenerative System (n=8 items). UTHealth | New Leaf Agriculture Report 2024 METHODS Farmer Recruitment and Data Collection for Module Quizzes As farmers completed learning modules, the Farm Director coordinated times with them to take the quizzes. Because farmers enter the New Leaf Agriculture program on a rolling basis, some farmers are further along in their learning modules. Quizzes were scheduled with farmers at a time they were on the farm and were administered in the on-site office with pen and paper. Farmers whose primary language was not English utilized a translator from LanguageLine during quizzes (n=10). All quizzes were scanned and shared electronically with researchers. Quizzes were also taken by some community farmers who did not participate in the Farmer Survey (n=5). Quiz Score Analysis Responses to the quizzes were entered into Google Sheets by the MRC team. Answers for the quizzes before instruction and after were added to separate spreadsheets and graded by a UTHealth researcher who marked incorrect answers. Grades for the quizzes were calculated automatically in the spreadsheet based on the ratio of correct answers to total questions. Two sets of quiz scores were created: one for primary farmers (n=6), and the other for community farmers (n=12). Welch's t- inferential statistical method using a test was employed as an significance level of 0.05 to determine whether differences in quiz score improvements between primary farmers and community farmers were statistically significant. This test was selected due to its suitability for comparing means between two independent groups with unequal sample sizes and potentially unequal variances. At this point in the evaluation, the first module is the only quiz set given before and after instruction to this group of 18 farmers. The module 1 quiz has four questions, three of which have multiple write-in answers. UTHealth | New Leaf Agriculture Report 2024 RESULTS Quantitative Findings Sociodemographic Information Out of the 14 farmers surveyed, seven were community farmers and seven were primary farmers. About half of survey respondents were female (n=8), ranged in age from 28-66 (mean: 45.8 years), had lived in the United States for at least one year (minimum: 1 year; maximum: 21 years; mean: 8.5 years; median: 7 years), and came from the following countries: Bhutan, Burma, Congo, Nepal, Myanmar, Ghana, and the Democratic Republic of Congo. On average, the respondents had one child under the age of 18 living in the household and, including themselves, 4 adults living in the household. Monthly household income reported was about $4,000. Food Insecurity Food insecurity was measured via the 2-item validated USDA food screener: We (I) worried whether our food would run out before we(I) got money to buy more. The food that we (I) bought just didn’t last and we (I) didn’t have money to get more. Participants who answered ‘sometimes true’ and/or ‘often true’ to at least one question were classified as food insecure. Within the sample, 21.4% were categorized as food insecure. UTHealth | New Leaf Agriculture Report 2024 RESULTS Health Mental health was measured via Patient Health Questionnaire-2 (PHQ-2), inquiring about the frequency of depressed mood and anhedonia over the past two weeks: In the past two weeks, how often have you been bothered by feeling down, depressed, or hopeless? In the past two weeks, how often have you been bothered by having little interest or pleasure in doing things? The answers of “not at all,” “several days,” “more than half the days,” and “nearly every day” were scored from 0-3, respectively. The summation of two questions yields a total score between 0-6, with a score of 3 or greater indicating a likelihood of a major depressive disorder. Within the sample, 12 farmers scored a 0, one farmer scored a 2, and one farmer scored a 4. Respondents were also asked if they had seen a doctor in the last 12 months and if they were told of any medical conditions. Two most commonly reported medical conditions were high blood pressure (n=4) and obesity/overweight (n=2). Six farmers had no conditions to report and two stated they have not seen a doctor “in the last 12 months.” RAPP Indicator: Client has access to healthy food Access to healthy food was measured via 4-items (Table 1) and scored 1-4 with “never” assigned a score of “1,” “always” a score of “4,” and “sometimes” and “often” scores of 2 and 3, respectively. The question that scored the highest, where most of the farmers answered “always” was: It is easy for me to get fresh produce. UTHealth | New Leaf Agriculture Report 2024 RESULTS Table 1: Questions measuring client’s access to healthy food RAPP Indicator: Client understands what is a healthy diet 1-item was utilized to assess the understanding of a healthy diet. When presented with a MyPlate image (Figure 1) showing various percentages of food groups, 46.2% did not understand that about 30% of the plate should be vegetables, 20% fruit, 30% grains, and 20% protein. UTHealth | New Leaf Agriculture Report 2024 RESULTS RAPP Indicator: Clients are engaged in physical activity Respondents were asked how many days during the last seven days they spent walking and for how long, and how many days they engaged in moderate physical activity, such as carrying light loads, bicycling at a regular pace, or gardening, and for how long. All respondents stated that they walked every day with the majority (69.2%) stating they walked more than 45 minutes each day. 76.9% of respondents engaged in moderate physical activity between 4 and 5 days in the “last seven days,” usually for more than 45 minutes. RAPP Indicator: Client knows how to find answers for questions about nutrition and health Majority of respondents (84.6%) can “often” or “always” find answers to questions related to nutrition and health. Asked how they would go about obtaining the information, respondents most often selected “internet search” (61.5%) and “ask a doctor” (30.8%) options. RAPP Indicator: Client knows how to find answers for questions about gardening/farming Asked if they could find information about gardening/farming, such as leasing land, zoning laws, pesticides, organic production, etc., 92.3% stated they are sure they could “often” or “always” find information about gardening/farming. Asked how they would find an answer to those questions, 76.9% stated they would ask someone at the New Leaf Agriculture farm. 23.1% mentioned asking a friend or a family member for information. UTHealth | New Leaf Agriculture Report 2024 RESULTS RAPP Indicator: Client has sufficient income to cover needs 1-item was used to assess farmers' ability to cover housing, food, and transportation costs. 57.1% of farmers felt their incomes were “often” or “always” sufficient to cover basic needs. RAPP Indicator: Client is engaged in the community Two-items were used to assess farmers' engagement in the community. Survey questions probed for how often they attend events in the community, such as at church, neighborhood associations, school events, and sporting events, and how often they socialize with community members. 57.1% “sometimes” attend community events and activities and 71.4% “often” or “always” socialize with members of the community. RAPP Indicator: Client can navigate relevant systems without grantee intervention The ability to navigate relevant systems, such as local governments and business development processes, and access benefits without grantee intervention, was one of the last things asked. Seven farmers think they can only sometimes navigate various US systems, and two stated they would not at all be able to do so. Probed for why they would be unable to find answers to general things, eight farmers provided the following answers: I do not speak/read/write English well (50.0%) Systems, like the healthcare system, are complicated and confusing (37.5%) I do not have access to the Internet/computer (25.0%) “Not being able to finance what I want to sign up for” (25%) “I do not have anyone to ask” (12.5%) UTHealth | New Leaf Agriculture Report 2024 RESULTS Module Quizzes Six primary farmers and 12 community farmers completed the module 1 lesson and quizzes. Most farmers significantly improved their quiz scores, and no farmer demonstrated a decrease in knowledge of module 1 topics after the education from New Leaf. The average score before the module 1 instruction was 53.51% and after instruction the average score was 79.24%, a 48% improvement in score from pre- to post-instruction (Table 2). The t-test between the improvement in scores reveals a significant difference in base level knowledge and improvement between the two groups after module 1 lessons. The complete data set of individual scores is available in Table 3 at the end of this section. Table 2. Module 1 Quiz scores before and after Module 1 lessons UTHealth | New Leaf Agriculture Report 2024 RESULTS Module Quizzes: Primary Farmers The average scores pre- and post-instruction quizzes were higher in this group than in the group of 12 community farmers. The average score before instruction was 71.93% which increased by 22% after instruction to an average of 87.72%. Although the post-instruction score is higher than the community farmers, it is not significantly higher with only 12.72% more points. Module Quizzes: Community Farmers The average pre-instruction score was 44.3%, which increased to 75.0% after instruction. The average quiz scores were lower for this group, but the 69% improvement after module 1 instruction is significantly more than that of the primary farmers. UTHealth | New Leaf Agriculture Report 2024 RESULTS Table 3. Module 1 Individual Quiz scores UTHealth | New Leaf Agriculture Report 2024 METHODS Qualitative Data: Interviews UTHealth created two sets of semi-structured interview guides for the two farmer groups. Both sets of questions asked farmers to identify ways they heard of New Leaf Agriculture, when they started working on the farm and what their experience has been, how different the farming practices in Central Texas are compared to their previous experience in their native country, what they like most about being a New Leaf Agriculture farmer and ideas on how their experience on the farm can be improved. The community farmers were also asked if they farm alone or with other family/community members, how much time per week they spend farming, frequency and pounds of produce harvested, ways they share harvested produce with their community, resources provided and those they wish were provided, whether supplemental income is worth the time being on the farm, and if they would like to transition to a primary farmer. The interviews were offered in English and the farmer’s native tongue. Those who wished to answer in their native language had an opportunity to speak to an interpreter from LanguageLine (n=9). Interviews conducted in English lasted about 30 minutes whereas those with an interpreter took twice as long. Farmer Recruitment and Data Collection New Leaf Agriculture provided the evaluation team with phone numbers of farmers who agreed to be interviewed and preferred times for the interview. Interviewers would then call or text farmers to schedule an interview or begin interviewing if farmers were ready. Only one interview was conducted in-person at the farm; the rest (n=13) were conducted over the phone. Data Analysis Two UTHealth staff experienced in qualitative research methods evaluated all interviews independently. Following individual analysis, the team members collaboratively discussed individual coded schemas, resolved discrepancies, and decided on emergent themes. UTHealth | New Leaf Agriculture Report 2024 RESULTS Qualitative Findings A total of 14 farmers, 7 primary and 7 community farmers, were interviewed to express their thoughts and experiences with the New Leaf Agriculture program. Interviews were conducted with refugees from the following seven countries: Nepal, Bhutan, Burma, Congo, Myanmar, Ghana, and the Democratic Republic of the Congo. Of these 14 interviews, 5 were conducted in English, and 9 utilized the LanguageLine translation service. Similar themes emerged from the analysis between primary and community farmers, with major deviations only in the benefits received and time commitment to the farming activities. Table 4 contains four significant quotes encapsulating each theme. Farming experience Interviewees had varying levels of farming experience before joining New Leaf Agriculture. Most refugees had decades of experience starting from childhood, and only two had no experience before starting at New Leaf Agriculture. Several of those with previous experience spoke of farming at New Leaf Agriculture as a reminder of home (see Table 4). Farming practices The majority of farmers had the same observation when asked about the differences in farming practices at New Leaf Agriculture farm compared to their previous experience in their home country. Farming at New Leaf Agriculture is more “scientific” and “systematic” than what they had done in the past. A few farmers mentioned farming at New Leaf Agriculture being more convenient due to automated equipment. Many perceive this formal structure simply as an improvement to productivity while some see it as a necessity with climate and conditions in Central Texas. All of the farmers spoke about their dislike and difficulty of farming in the intense Texas summer heat and a desire for more shade. UTHealth | New Leaf Agriculture Report 2024 RESULTS Benefits of New Leaf Agriculture income that growing There are a variety of benefits to participating as a farmer at New Leaf Agriculture. All farmers have access to community farming plots, which allows them to grow crops they want to produce specifically for their communities. Most interviewees also emphasized the benefit of the additional in the community plot provides. Furthermore, many stated enjoying the additional food and the ability to give the surplus to their communities. Even physical activity, the manual labor of farming, was noted as a benefit and as a fun exercise by some farmers. However, a couple of benefits differ between primary farmers and community farmers. In contrast to community farmers, primary farmers receive ESL classes and shuttle transportation to and from the farm each shift. These farmers expressed their gratitude toward the guaranteed transportation and noted its convenience. In addition to fresh produce, income, and physical activity, all of the resources needed to farm such as water, land, equipment, and seeds are provided by New Leaf Agriculture. Personal reasons for participating In addition to getting paid, farmers indicated throughout the interviews that they enjoy working at New Leaf Agriculture for several personal reasons. For many, farming allowed them to pair physical activity with being in an outdoor environment they find pleasant. Growing food is described as a hobby in several instances and they generally stated that farming makes them happy. Farmers also enjoy the additional food they can give to their communities. With many having previously farmed in their home country, growing food outside reminds them of their roots while performing a skill they already feel comfortable doing. Because farmers can choose what they want to grow, some elect to grow produce they recognize and grew in their country of origin. UTHealth | New Leaf Agriculture Report 2024 RESULTS Time commitment and commuting Primary farmers get picked up and dropped off via a shuttle for their shifts on the farm several times a week. The shifts start in the morning and end in the afternoon. According to one interviewee, community farmers arrange their own transportation to the farm, a 30–35-minute commute one way. The community farmers interviewed spent about an hour and a half at the farm twice a week. These farmers tended to bring friends or family while on the farm to help and experience the farm as a group. Reasons for not wanting to be a primary farmer farmer was When community farmers were asked if they would consider being a primary farmer in the future, many would not consider it. The main reason cited for not participating as a primary inadequate monetary compensation compared to their is needed to current support themselves and their families. The time commitment to be a primary farmer would prevent many from holding their current job or role that provides stability. When a community farmer was asked what that pay level would have to be, they said, “$4500 a month.” income or what Increased pay increases chances of moving from community to primary farmer role. UTHealth | New Leaf Agriculture Report 2024 RESULTS Recommendations Throughout the interviews, farmers suggested several ideas to improve their farm experience. The two most cited recommendations were: 1) increase the amount paid to each farmer and 2) desire to “have some sort of shade structure” during the summer months. Farmers also recommended grasses surrounding the farm plots be trimmed more regularly to prevent snakes and other pests, and a wish to better control insect pests. Some primary farmers suggested expansion to the benefits they already have. One farmer mentioned they enjoyed and saw improvement in their English proficiency due to the first level of ESL classes but they wished they could take the next level of classes. Another primary farmer mentioned their desire to have more comprehensive healthcare along with a 401(k) as part of their benefits package. Although primary farmers have access to the Travis County Medical Access Cards (MAP), they desire healthcare options. Lastly, farmers request that New Leaf Agriculture purchase more equipment so that all farmers have equipment on busy days when many farmers work in the fields. 01 03 Compensation Increase farmer wages Additional benefits such as 401(k) Pests Increase grass trimming to minimize snakes Pest control 01 02 04 Shade Provide heat protection during the summer months Equipment Increase amount of tools available UTHealth | New Leaf Agriculture Report 2024 RESULTS Table 4: Interview quotes by theme UTHealth | New Leaf Agriculture Report 2024 RESULTS Table 4: Interview quotes by theme, continued UTHealth | New Leaf Agriculture Report 2024 CONCLUSION Seven primary farmers and seven community farmers were surveyed on their access to healthy food, knowledge of a healthy diet, physical activity behaviors, ease of navigating systems for information on nutrition, health, and gardening, and whether their income is adequate for various expenses. Overall, farmers thought it was easy to find healthy food, but did not know the different food percentages MyPlate should comprise. All respondents walked every day, majority stated they would be able to find answers related to nutrition and health questions and to farming questions, but majority felt they would be unable to navigate various government systems without assistance. Just over half felt their incomes were sufficient to cover basic needs. Food security was determined with two questions, categorizing one-fifth of surveyed respondents as food insecure. Farmers were also given a pre-test prior to module instruction and a post-test after instruction. The 18 farmers who took both tests revealed clear improvements in knowledge on the topic of seeds and soil after taking the module 1 course. This improvement was larger in the community farmers because primary farmers already had significantly higher baseline quiz scores. Community farmers brought a failing group average score to a passing one, a post-test score not significantly different from primary farmers’, after undertaking module 1 lessons. Overall, module 1 lessons effectively increased knowledge among primary and community farmers. UTHealth | New Leaf Agriculture Report 2024 interviews conducted show that MRC creates positive Qualitative experiences with minimal or zero-cost participation for farmers in the New Leaf Agriculture program. Refugees in the program get a myriad of benefits ranging from improved English proficiency and more financial stability to increased exercise in a beautiful outdoor space. To improve these experiences, providing more compensation and protection from extreme heat and pests would go a long way. More compensation can come in many forms: comprehensive health care that also covers families, more positions at higher wages or compensation rates, and fringe benefits such as bus passes or advanced English learning opportunities. For community farmers to move into the primary farmer role, wages must at least match those they currently receive in their jobs outside of farming. Temporary shade cloth structures erected in summer could keep farmers more comfortable in the higher temperatures and reduce some pest pressure. Although farmers admonished unmowed grass near their farming rows, MRC might maintain it for its ecological advantage. Decisions and reasoning behind pest management strategies and current pest pressure should be clearly messaged regularly. UTHealth | New Leaf Agriculture Report 2024 ACKNOWLEDGEMENTS The research team would like to thank the Multicultural Refugee Coalition for their collaboration and partnership during the entirety of this evaluation. We especially thank Meg Erskine, Matt Simon, Carissa Eckle, and Marissa Trussell. A resounding thank-you to all the farmers who took the time to complete surveys, modules, and interviews! Contact UTHealth Austin 1616 Guadalupe St., Austin, TX 78701 512-482-6160 Aida Nielsen, MPH aida.nielsen@uth.tmc.edu Chris Reyes Christopher.D.Reyes@uth.tmc.edu Alexandra van den Berg, PhD, MPH Alexandra.E.VanDenBerg@uth.tmc.edu Taylor Levy, MS Taylor.M.Levy@uth.tmc.edu Martha Diaz Martha.P.DiazMarin@uth.tmc.edu UTHealth | New Leaf Agriculture Report 2024