Sensitivity of Three Impact Assessment Methodologies in Adjusting Preventive Chemotherapy Treatment Decisions for Schistosomiasis Elimination in Ondo State, Nigeria

INTRODUCTION

Schistosomiasis or bilharziasis is a neglected tropical disease (NTD) with over 200 million people across 78 countries currently affected.^1–3 In sub-Saharan Africa, human infections are commonly caused by trematode parasites of the genus Schistosoma, namely Schistosoma hematobium and Schistosoma mansoni,¹ in marginalized urban, semiurban, and rural populations; people are exposed through surface water contact practices that harbor both the larval stages of the parasites and their intermediate snail hosts of the genera Bulinus and Biomphalaria.^1,4 Schistosomiasis is an NTD targeted for elimination as a public health problem (EPHP) by 2030.³ The strategy called preventive chemotherapy (PC) involves mass administration of praziquantel drugs in schools or communities using the WHO endemicity classification system and guidelines.⁵ This strategy aims to reduce the morbidity of heavy-intensity infections in at-risk populations to below 1% across 78 countries and achieve EPHP by 2030.³

In the last two decades, WHO has coordinated the annual distribution of over 250 million praziquantel tablets to several endemic countries, with approximately 75.3 million people being treated in 2021 alone.^3,6 PC programs were implemented exclusively in endemic regions after baseline epidemiological surveys. These surveys were conducted to assess the prevalence of infections to evaluate eligibility for PC and to establish the necessary frequency of PC.⁵ For instance, biannual PC was implemented when baseline prevalence exceeded 50%, annual PC was implemented when prevalence ranged from 10% to 49.9%, and biennial PC was implemented when the prevalence fell between 1% and 10%.⁵ However, recent revisions have recognized that schistosomiasis transmission is typically more localized to water contact site catchment areas, and this focality requires more refined smaller spatial implementation scales, such as subdistricts/communities.^7,8 The latest WHO guidelines have adjusted the prevalence thresholds for PC to >10%, encompassing individuals across all age groups, including children ages older than 2 years, pregnant women beyond the first trimester, and lactating women.⁹

Following these recommendations, a more precise estimation of the prevalence is needed for better targeting and optimal allocation of resources to the transmission foci. This promises to enhance the efficiency and effectiveness of PC toward the 2030 elimination targets.^7,8 WHO has encouraged the conduct of impact assessment studies in implementation units (IUs) after at least 5 years of effective PC (≥75% treatment coverage)¹⁰ and also, put forth eight distinct methodologies for conducting such assessments, each with its own set of limitations.¹⁰ Two prominent methodologies, the sentinel and cluster assessment approaches, have traditionally been used for evaluating schistosomiasis endemicity. The sentinel method involves a purposeful selection of fewer representative schools, whereas the cluster method entails a random selection of more schools.¹⁰ Both approaches assess prevalence at the ecological zone or district level, guiding treatment decisions based on aggregated prevalence (Table 1). Recently, the schistosomiasis practical and precision assessment (SPPA) methodology was developed.¹¹ Unlike traditional methods, SPPA evaluates endemicity at the district level but makes treatment decisions at the subdistrict level (Table 1). This strategy aims to streamline the process and save time and resources while aligning with the 10% prevalence threshold outlined in the latest treatment guidelines.¹¹

Given the magnitude and backlog of impact assessment surveys that need to be conducted, programs are looking for the most reliable and sensitive method for adjusting PC decisions considering scarce resources. Therefore, this study evaluated the sensitivity of cluster, practical, and sentinel assessment methodologies for adjusting PC treatment decisions 10 years after implementation in Ondo state, Nigeria.

MATERIALS AND METHODS

Study area and settings.

This study was carried out in the Ese-Odo, Irele, and Ile-Oluiji local government areas (LGAs) of Ondo state, southwestern Nigeria (Figure 1). Ondo state is administratively divided into 18 LGAs, which comprise 206 wards or subdistricts. There are over 3 million inhabitants, with the majority being the Yoruba tribe and Christians. The state is characterized by a tropical climate and three vegetation types (freshwater swamp, rainforest, and Guinea savannah), which in part, contribute to the transmission dynamics of schistosomiasis. Some LGAs bordering the Atlantic Ocean in the south have several ponds, streams, and rivers, which provide a conducive environment for farming and fishing as well as sites for other recreational activities, such as bathing, swimming, and washing clothes.

View Full Size

Map of schistosomiasis endemicity in Ondo state: indicating 13 endemic LGAs and selected study locations with Nigeria as an inset. LGA = local government areas.

Citation: The American Journal of Tropical Medicine and Hygiene 112, 5; 10.4269/ajtmh.24-0352

• Download Figure
• Download figure as PowerPoint slide

Baseline epidemiological mapping for schistosomiasis was conducted in 2014, and 13 of the 18 LGAs were found endemic.¹² The majority of the LGAs (n = 9) were categorized as having low endemicity (prevalence between 0.1% and 10%), whereas three LGAs were categorized as having moderate endemicity (prevalence between 10% and 49.9%); only one LGA had a high prevalence (prevalence is greater than 50%).¹² On average, six effective (>75% coverage) annual rounds of PC have been delivered across the endemic LGAs by the Federal and State Ministry of Health with support from the Ending Neglected Diseases Fund and the Mission to Safe the Helpless, the nongovernmental organization supporting the implementation of PC in the state. All IUs met the minimum number of effective PC rounds required to conduct an impact assessment survey.

STATISTICAL ANALYSES

The data collection process was remotely monitored using the KoboCollect platform. Field supervisors also recorded summarized daily entries in a closed-tracking tool, facilitating the triangulation of electronic uploads and enabling prompt resolution of discrepancies. Three distinct tracking tools were used throughout the study: one for each of the school field forms and parasitology and malacology components. All variables were imported into R Studio v. 4.3.2 (R Foundation, Vienna, Austria) for the analysis. Initially, these variables were transformed, categorical responses were converted into factors, and numeric responses were converted into integers. Subsequently, responses were encoded into binary codes using the corresponding questionnaires and data codes.

For the prevalence data, entries were initially treated as numeric values and used to estimate the overall infection status by using the “rowSums” function in R. For instance, the overall schistosomiasis infection status was determined by summing all of the positive responses for S. mansoni and S. hematobium. Participants with counts greater than or equal to one were recorded as positive for overall schistosomiasis infection. Regarding infection intensity, participants with S. hematobium egg counts between 1 and 49 were classified as having a light infection, whereas those with counts ≥50 were considered to have a heavy infection.¹² Conversely, for S. mansoni, infections were categorized as light with egg counts between 1 and 99, moderate with counts between 100 and 399, and heavy with counts ≥400 or more.¹²

For the statistical analysis, we adhered to a systematic approach. Initially, descriptive statistics were used to summarize all of the dependent and response variables as proportions. Subsequently, associations were investigated using χ² statistics, primarily focusing on demographic data, such as age and sex. Following this initial analysis, we conducted three independent analyses based on three assessment methodologies: cluster, sentinel, and practical. The cluster methodology analysis involved all 15 schools and participants ages 5–14 years old. However, for practical assessment, we filtered only participants ages 10–14 years old. In contrast, the sentinel assessment used a stratification approach for all 15 schools within each LGA, considering factors such as the ecological zone, endemicity profile, and therapeutic coverage. This stratification facilitated the random selection of 5 representative schools from the sampled 15, using the “unique” and “sample” base functions in R. Subsequently, the dataset was filtered to include only the selected schools, and the analysis was conducted for all participants ages 5–14 years old.

Finally, we calculated the percentage change in prevalence across the three assessment methodologies to illustrate how the data reduction affected the observed prevalence. The prevalence for each approach was estimated and compared with that of the cluster-based approach as a reference because the cluster method was used in the initial baseline survey. All data analyses were conducted separately for each unit. Associations between variables were assessed at a 95% significance level, and where applicable, Clopper–Pearson CIs were computed for prevalence estimates. Additional files containing the dataset and analysis code are attached at https://zenodo.org/records/11080490.

RESULTS

Programmatic interpretation of prevalence data based on the three assessment methodologies.

Figures 2–4 present the endemicity maps of the study LGAs. In 2011, the Irele LGA was categorized as having a low endemicity status, with an aggregated prevalence below 10%, and only one of the six sampled schools exhibited a prevalence of 10% (Figure 2A). However, the current study (Figure 2B) also revealed an aggregate prevalence below 10% based on cluster sampling (Figure 2B), with three schools reporting a site prevalence >10%. Similarly, findings from the sentinel approach indicated an aggregated prevalence of <10%, with only one of five schools reporting a prevalence of >10% (Figure 2C). The implication of these findings as indicated by the guidelines for the sentinel and cluster approaches suggests that Irele does not require PC because the aggregated prevalence is below 10%. However, with the practical approach, 3 of the 15 sampling sites had prevalence rates above 10%, implying heterogeneous transmission in the LGA with hot spots in Irele 3, Irele 4, and Omi Iyasan and hence, confirming that the LGA requires an annual PC (Figure 2D).

View Full Size

Comparative impact assessment of schistosomiasis endemicity maps in Irele LGA: evaluating the sentinel, cluster, and practical survey methods against baseline data. LGA = local government areas.

Citation: The American Journal of Tropical Medicine and Hygiene 112, 5; 10.4269/ajtmh.24-0352

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Comparative impact assessment of schistosomiasis endemicity maps in Ese-Odo LGA: evaluating the sentinel, cluster, and practical survey methods against baseline data. LGA = local government areas.

Citation: The American Journal of Tropical Medicine and Hygiene 112, 5; 10.4269/ajtmh.24-0352

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Comparative impact assessment of schistosomiasis endemicity maps in Ile-Oluiji LGA: evaluating the sentinel, cluster, and practical survey methods against baseline data. LGA = local government areas.

Citation: The American Journal of Tropical Medicine and Hygiene 112, 5; 10.4269/ajtmh.24-0352

• Download Figure
• Download figure as PowerPoint slide

Figure 3 shows the endemicity map of Ese-Odo LGA. In 2011, LGA was categorized as having a low endemicity status, with an aggregated prevalence below 10%, and only two of the five sampled schools exhibited a prevalence between 1% and 9.9% (Figure 3A). However, the current study (Figure 3B) also revealed an aggregate prevalence below 10% based on cluster and sentinel sampling (Figure 3B and C), with only one school reporting a site prevalence between 1% and 9.9%. The implication of these findings as indicated by the guidelines for the sentinel and cluster approaches suggests that Ese-Odo does not require PC because the aggregated prevalence is below 10%. Also, with the practical approach (Figure 3D), none of the 15 sampling sites had prevalence rates above 10%, implying a homogeneously low prevalence of the LGA and confirming that the LGA does not require PC (Figure 3D).

Figure 4 presents an endemicity map of Ile-Oluiji LGA. In 2011, Ile-Oluiji was categorized as highly endemic with an aggregated prevalence of >50% (Figure 4A). However, the current study (Figure 4B) revealed an aggregate prevalence below 10% based on cluster and sentinel sampling (Figure 4B and C), with only one and four schools reporting site prevalence between 1% and 9.9% for sentinel and cluster surveys, respectively. The implication of these findings as indicated by the guidelines for the sentinel and cluster approaches suggests that Ile-Oluiji does not require PC because the aggregated prevalence is below 10%. Also, with the practical approach (Figure 4D), none of the 15 sampling sites had prevalence rates above 10%, implying a homogeneously low prevalence of the LGA and confirming that the LGA does not require PC (Figure 4D).

DISCUSSION

There is substantial evidence of the burden of schistosomiasis in Nigeria, and efforts to control this disease have been ongoing for over a decade.^18–22 Recently, these efforts have shifted toward schistosomiasis EPHP.³ Country programs now aim to reduce the proportion of individuals with moderate- and heavy-intensity schistosomiasis infections to less than 1% and decrease the number of tablets required for PC by 50%.³ The schistosomiasis PC program in Nigeria has expanded significantly over the years, benefiting from substantial investments from pharmaceutical companies, WHO guidance, and crucial support from governmental and nonprofit organizations.³ Since 2014, approximately one fifth of all endemic IUs in Nigeria have started PC, reaching an average of 4–16 million Nigerians annually.²³ Current projections toward the NTD 2030 Roadmap indicate that approximately 76 million Nigerians across 317 IUs still require PC, whereas approximately 90 IUs might have reached stages of transmission interruption.²³ Continuing PC in IUs where morbidity has been reduced is counterproductive given limited resources. Therefore, WHO recommends conducting impact assessment surveys after 5 years of effective PC^5,11 to redefine endemicity levels, identify hot spots, adjust PC thresholds, determine when to stop or continue PC, and better estimate the resources required.¹⁰

As many programs are currently planning to conduct these surveys, we present evidence of the sensitivity of some commonly used assessment methodologies that are essential for adjusting PC decisions. This study represents the first impact assessment survey of schistosomiasis in Ondo state, in which 15 sampling sites (schools) were surveyed across each IU. Among the three IUs surveyed, two (Ese-Odo and Ile-Oluiji) exhibited over 90% significant reduction in aggregated prevalence estimates compared with their baseline estimates. Specifically, a prevalence of 1.8%, with only 0.1% harboring heavy intensity, was observed in Ile-Oluiji, a previously hyperendemic IU. These results illustrate the impact of PC, which has been the only intervention for LGAs over the past decade. Similar reports have confirmed significant declines in prevalence estimates after years of PC implementation.^18,24,25

More importantly, in Ese-Odo, recent remapping assessments conducted at the community level among children ages 5–14 years old in 2021 (immediately after the coronavirus disease 2019 [COVID-19] pandemic) revealed three hot spots, particularly in Apoi 1, 4, and 5 wards, with prevalence rates ranging from 22% to 58%.¹³ The sampling methodology, which used one community per subdistrict/ward, was more representative than initial baseline mapping efforts. These findings suggest that the LGA might have cryptic hot spots and may have been misclassified at baseline, supporting our argument for more robust mapping and assessment methodologies. However, our assessment 30 months later in the same location indicated nonendemicity. The discrepancies in prevalence estimates may be attributed to the impact of PC implemented after the mapping exercise or other epidemiological parameters, including differences in the spatiotemporal scales of both assessments (community versus school based and 2021 versus 2024), the approach used in recruiting participants (enrollment of both enrolled and nonenrolled SAC versus only enrolled SAC),^26–28 and the participants’ history of involvement in PC programs^29,30 and other associated water contact practices that were previously reported elsewhere to be more frequent during the COVID-19 lockdown.^14,31 This warrants further investigation in subsequent analyses. Nevertheless, these findings add additional credence to the possibility of cryptic hot spots among nonenrolled SAC who are often missed during regular PC programs typically implemented at schools.²⁸ Because community-based studies are more likely to sample children who have never benefited from PC programs and are likely to have higher worm loads, more careful consideration is thus necessary in areas where there are disparities in prevalence estimates between school-based and community-based epidemiological studies.

Furthermore, we observed an increase in prevalence from 3.0% to 5.3% in Irele, with approximately 2.3% of the sampled population harboring high-intensity infections. This LGA was classified as having low endemicity in 2014, and PC was administered biennially until 2021 when outbreaks were reported in some communities.¹³ This prompted the identification of hot spots and led to the reclassification of the LGA for annual PC.¹⁴ In Nigeria, most baseline epidemiological surveys conducted over a decade ago used sampling methodologies that used only a few sampling sites (typically around five or six) within smaller sampling frames, such as wards or subdistricts, to represent the entire district.¹³ It has been established that the limited number of sampled schools reduces the power to accurately delineate hot spots,³² particularly in heterogeneous IUs, where the focal nature of schistosomiasis varies based on ecological and other epidemiological parameters,³³ or in areas where infections are sparse.³⁴

Therefore, we argue that the observed increase in prevalence in Irele, although statistically insignificant, reflects the consequences of underestimating baseline prevalence. This underestimation subsequently influenced the decision to implement PC as the IU was misclassified based on existing evidence. Similar findings have been reported elsewhere, with outbreaks in areas previously considered to have low endemicity.¹⁴ Thus, it is crucial to emphasize the need to prioritize more robust assessment methodologies over convenience and cost during the formative stages of PC programs.^18,32 This includes careful consideration of the scale (district or subdistrict), spread, and number of sampling sites on which the preferred methodologies are built.

WHO has advocated for more robust methodologies to address the limitations of sampling a few schools, such as cluster sampling,¹⁰ and to capture transmission dynamics across various ecological zones within the IU as seen in the SPPA methodology.¹¹ The SPPA methodology introduces a new approach to determining treatment thresholds, such as >10%,¹¹ which align with the newly published WHO guidelines for schistosomiasis elimination.⁹ Our comparative analysis of three impact assessment methodologies revealed that the sentinel approach had lower sensitivity in areas where baseline or previous endemicity reports were either consistently low (Ese-Odo) or high (Ile-Oluiji).

However, in Irele, where outbreaks were recently reported after baseline mapping, the sensitivity of the sentinel approach has increased. Conversely, the practical methodology demonstrated higher sensitivity in areas deemed to have low endemicity. As previously emphasized, adjusting PC thresholds is based solely on prevalence estimates obtained from impact assessment surveys. Historically, the most prominent impact assessment evaluations used aggregated estimates at the district level to make treatment decisions using the WHO thresholds of biannual PC when baseline prevalence exceeds 50%, annual PC when prevalence ranges from 10% to 49.9%, and biennial PC when the prevalence falls between 1% and 10%.^5,34 However, the latest WHO guidelines have adjusted the prevalence thresholds for PC to >10% and further suggest that decisions should be made at the subdistrict level. Thus, impact assessment methodologies can be modified to align with these guidelines and to make treatment decisions.

CONCLUSION

In summary, different decisions would have been made if the methodologies had shifted. For instance, in Irele, sentinel and cluster methodologies suggest that the LGA has an aggregate prevalence below 10%, indicating that PC is not required. However, the SPPA approach suggests otherwise as 3 of 15 sampling sites have >10% prevalence.¹¹ This reflects the heterogeneity in schistosomiasis transmission in the LGA, indicating the need to optimize the frequency of PC from a biennial mode to an annual mode, particularly in Irele 3, Irele 4, and Omi Iyasan subdistricts. Using Irele, we demonstrate how decisions regarding surveys may have been influenced by the methodology used for assessment. Therefore, it is crucial to emphasize that the percentage changes in prevalence across the methodologies were insignificant, possibly because of the limited number of positive cases. However, this should not be taken seriously as significant differences in prevalence estimates are not currently used to adjust treatment decisions. Nevertheless, our study highlights that the practical assessment methodology is inherently more robust owing to its systematic approach to site selection, larger number of sites, and more restricted eligibility group of children ages 10–14 years old.

One major limitation of this study was the relatively small number of children ages 10–14 years old across a few sampled schools. The SPPA methodology specifies that children who have undergone several rounds of PC are the best target group for monitoring impact, recommending the enrollment of about 30 children ages 10–14 years old. Nigeria has the largest number of out-of-school children in the world, with about 10.5 million children ages 5–14 years old out of school because of economic reasons, the need for labor, early marriage, and poor school quality.³⁵ With these high dropout rates, there are likely to be fewer children ages 10–14 years old in school for the SPPA methods. Although an alternative approach would be to sample additional children ages 10–14 years old in the community to meet the target size, this was largely not feasible as the children were often not at home, likely because of work commitments. Therefore, more careful planning for both enrolled and nonenrolled children should be made ahead of mapping or assessment surveys to ensure representative sampling. Additionally, we used a single epidemiological survey to collect data regarding the three different assessment methodologies, and our inferences relied on disaggregated estimations after using data reduction approaches based on the criteria of each assessment methodology. If resources allow, we recommend that independent studies be conducted. Despite these limitations, this study offers valuable insights and provides evidence for adjusting the treatment decisions for LGAs. This study lays the groundwork for further research and exploration of the drivers of transmission in Irele to address the unique challenges that may limit control and elimination efforts.