TRACKING RESULTS CONTINUOUSLY (TRAC)
We collect quantitative data about our social marketing programs by conducting TRaC studies. TRaC is a multi-round survey-based research approach that we use to gather information from a representative sample of our target populations. The data collected from TRaC studies allow us to understand characteristics of the groups we work with such as sociodemographics, health behaviors, knowledge about health products, and much more. This information is very useful to understand how to best design our social marketing programs as well as to assess if they are effective.
TRAC MONITORING ANALYSIS
To measure program indicators and changes in indicators, we conduct TRaC Monitoring analysis. The types of research questions that TRaC Monitoring studies can answer include: What is the perceived availability of safe water solution among mothers and caregivers to children under 5? Or, What percent of mothers and caregivers to children under 5 know how to prevent diarrheal disease?
Monitoring studies are conducted before a program begins to measure baseline levels of indicators, during the program to measure progress against indicators, and/or at the end of a program to measure change over time in indicator levels.
We usually present monitoring information in terms of percentages and means. Here is a section of a monitoring table from a TRaC study in Liberia on diarrheal disease prevention behaviors:
The table includes key behavioral indicators and what percentage of the target group, in this case, caregivers to children under 5, performs the desired behaviors. For example, 12.9 percent of the 1,995 caregivers to children under 5 surveyed treat their drinking water with chlorine-based chemicals.
The monitoring table also includes behavioral factor indicators. These indicators measure dimensions of opportunity, ability and motivation that determine whether or not behaviors are performed. In this study these indicators were measured on a scale of 1-5. (1-Strongly disagree, 2-Disagree, 3- Neutral, 4- Agree, and 5-Strongly agree) The mean, or average, of all of the responses is presented in the monitoring table. We can interpret from this table that caregivers of children under 5 in Liberia on average report that they agree/strongly agree to the statement ‘Treating my family’s drinking water is a good idea’. However, they may not have adequate social support to treat their drinking water, as indicated by the 1.87 mean.
This information is useful to decide how to design a diarrheal disease prevention intervention among caregivers to children under 5 in Liberia in terms of what behaviors and messages to focus on. Data can also be collected over time on the same indicators to assess trends and changes in behaviors and behavioral factors which is important for tracking program progress.
TRAC SEGMENTATION ANALYSIS
To analyze differences in population characteristics and behavioral factors among those who perform a certain behavior and those who don’t, we conduct TRaC Segmentation analysis. The types of questions that segmentation studies can answer include: Is perceived availability of safe water solution different between those caregivers of children under 5 who treat water with chlorine based products and those caregivers of children under 5 who do not treat water with chlorine based products? Or, do those caregivers of children under 5 who treat water with chlorine based products have higher social support to use safe water solution than those caregivers of children under 5 who do not treat water with chlorine based products?
Segmentation studies are usually conducted before an intervention to understand how to best design a program and address the behavioral factors underlying a certain behavior.
We usually present segmentation data in terms of means, percentages, and Odds Ratios (ORs). Here is a section of a segmentation table from a TRaC study in Liberia on diarrheal disease prevention behaviors:
The segmentation table presents demographic characteristics and the opportunity, ability, and motivation factors – those factors that determine behavior – by those who perform the behavior of interest versus those who do not perform the behavior of interest. These indicators were measured on a scale of 1-5. (1-Strongly disagree, 2-Disagree, 3- Neutral, 4- Agree, and 5-Strongly agree) The mean, or average, of all of the responses is presented in the segmentation table.
Reading the percentages and odds ratios in this table, we can see that among caregivers to children under 5 who treat water with chlorine based products, 30.6% have higher than a secondary education, compared to among caregivers to children under 5 who do not treat water with chlorine based products 18.6% have higher than a secondary education. Also, those caregivers to children under 5 who have higher than a secondary education are 2.04 times as likely to treat their water with chlorine based products that those caregivers to children under 5 who do not have a secondary education.
Examining the behavioral correlates, the segmentation results show that caregivers to children under 5 who treat water with chlorine based products, on average, report a mean score of 2.51 on perceived availability of the product on a scale of 1 – 5, compared to 2.18 for caregivers to children under 5 who do not treat water with chlorine based products. The OR tells us that a one point increase in perceived availability score reported is associated with a 1.23 times greater likelihood, or 23% greater likelihood, of water treatment with chlorine based products. In other words, caregivers to children under 5 who report a 5 on perceived availability of safe water solution are 1.23 times more likely to treat water with chlorine based products than caregivers to children under 5 who report a 4 on perceived ability of safe water solution.
This information is very useful for programmers to design a program to target the specific needs of populations at risk.
TRAC EVALUATION ANALYSIS
To determine the effectiveness of PSI interventions, TRaC evaluation analyses are conducted. Although changes in behavior and behavioral correlates may be observed by conducting monitoring studies, evaluation studies are necessary to assess if those changes are attributable to our interventions. Evaluation studies can answer for example, is exposure to our safe water inter personal communication (IPC) intervention associated with an increase in water treatment? Evaluation studies can also assess the dose-response relationship between an intervention and outcomes of interest. So for example, evaluation can answer, is a higher level of exposure our safe water messaging associated with higher knowledge on diarrheal disease compared to low level or no exposure to our safe water messaging?
Evaluation studies are conducted at the end of a program, when at least two rounds of data are available, to determine if the intervention was effective in changing behavior and/or behavioral correlates.
Our evaluation data are presented in terms of percentages and means. Here is a section of an evaluation table from a TRaC study in Liberia on diarrheal disease prevention behaviors:
The evaluation table presents baseline indicator values, and values for the same indicators at endline for three groups: those who received no exposure to the intervention, those who received low exposure to the intervention, and those who received high exposure to the intervention. The letter superscripts indicate pairwise differences.
The results show that among women with children under 5 who were exposed to the safe water intervention (both low and high levels), significantly higher proportions (33.5% and 28.3% respectively) ever treated their drinking water, compared to women with children under 5 at baseline (12.1%) and women with children under 5 who were not exposed to the intervention (17.8%). The results also show that there is no statistically significant difference between the proportion of women with children under 5 that ever treated drinking water at baseline and the proportion of women with children under 5 that ever treated drinking water in the non-exposed group. Those who were highly exposed to the safe water intervention had a significantly higher knowledge score about diarrheal disease (8.31) than respondents at baseline (6.95), those with no exposure (7.25), and those with low exposure (7.62); those with low exposure had a significantly higher knowledge score than those with no exposure. The significant difference between knowledge scores among women with children under 5 at baseline and those in the non-exposed group suggest that there may have been influences external to our safe water messaging intervention that might have also contributed to increased knowledge about diarrheal disease.
Evaluation data are necessary to assess the impact of our interventions, and to make decisions about replicating and scaling up programs.