Ecological Validity

Summary

Approaches that are designed and built by and with the community (authenticity, organic, grassroots, culturally relevant) have higher transfers of beliefs and skills to daily life.  A "program" that is transferred from one environment to another is not a valid Social Network Health approach, and is unlikely to address the cultural norms of the new environment.

Notes and References

Effectiveness Trial of Wingman-Connect Implemented Across Career Phases

Submitted to the National Institute of Mental Health, PAR21-130, 10/13/2022

Group training that builds cohesive, healthy military units is promising for upstream suicide prevention and may be essential for ecological validity

Data-informed implementation preparation.

A “voltage drop” and poor sustainment are common as interventions transition from controlled efficacy trials to regular use.

Conversely, scaled out interventions are more impactful with careful preparation using implementation science frameworks.

“The concept of ecological validity has become familiar to psychologists. It reminds them that the artificial situation created for an experiment may differ from the everyday world in crucial ways. When this is so, the results may be irrelevant to the phenomena that one would really like to explain.”

To bridge the gap between lab and life, many researchers have called for experiments with more ‘ecological validity’ to ensure that experiments more closely resemble and generalize to the ‘real-world.’

Holleman GA, Hooge ITC, Kemner C, Hessels RS. The 'Real-World Approach' and Its Problems: A Critique of the Term Ecological Validity. Front Psychol. 2020 Apr 30;11:721. doi: 10.3389/fpsyg.2020.00721. PMID: 32425850; PMCID: PMC7204431.

The main intention for cultural adaptation frameworks (EVM) is to increase the cultural acceptability and effectiveness of the psychological treatment. This is accomplished by making changes that align with the culture of the beneficiary population, while maintaining the components of the evidence-based research that supports the treatment

The EVM was selected because it is based on the view that individuals must be understood within their cultural, social, and political environment. The EVM framework serves to ‘culturally center’ an intervention through eight dimensions that must be incorporated for an intervention to have ecological validity and be embedded within the cultural context (Bernal, 2003). These dimensions include language, persons, metaphors, content, concepts, goals, methods, and context (Table 1 in online Supplementary material).

Sangraula M, Kohrt BA, Ghimire R, Shrestha P, Luitel NP, Van't Hof E, Dawson K, Jordans MJD. Development of the mental health cultural adaptation and contextualization for implementation (mhCACI) procedure: a systematic framework to prepare evidence-based psychological interventions for scaling. Glob Ment Health (Camb). 2021 Feb 19;8:e6. doi: 10.1017/gmh.2021.5. PMID: 33996110; PMCID: PMC8082944.

NIMH

Data-informed implementation preparation.

A “voltage drop” and poor sustainment are common as interventions transition from controlled efficacy trials to regular use. Conversely, scaled out interventions are more impactful with careful preparation using implementation science frameworks.

JAMA Open Network, October, 2020

group training design ecologically valid and appealing for participants. Wingman-Connect development benefited from strong organizational collaboration, resulting in an ecologically valid program.

Evidence-based Methodology: A commitment to follow the evidence of impact. Networks of Support’s structure and key elements strengthen relationship networks, cohesion, adaptive coping, and group norms. Group-based, interactive training uses research-informed strategies.

JAMA Open Network, October, 2020

Research Outcomes – Evidence Based Key Findings include:

• Increased class cohesion, morale, positive bonds, and acceptability of help seeking
• Reduction in suicide risks scores, depression, and reactivity to anger
• Nearly 50% reduction in occupational impairment (corrective training, negative counseling)

Measurable Impact on both Protective Factors and Risk Factors

Wingman-Connect was tested using the gold-standard research design with Airmen in Training from 2017-2019 (randomized controlled trial).  Training showed statistically significant increases in group and individual level protective factors, as well as reductions in negative outcomes including and depression and suicide risk.  

Class exercises create more cohesive units with skills extended into group culture.  Wingman Connect training is delivered to organizational units using interactive exercises to build key group and individual protective factors.

Wingman-Connect impact promoting more cohesive, healthy training classes was a mechanism that reduced risk for suicidal thoughts and behaviors. More cohesive, healthy classes ‘lifted up’ individuals who were vulnerable to fitness and mental health problems. Study findings suggest that group training for organizational units may be critical for ‘transfer’ of training concepts and skills into the organization’s culture.

Social Science & Medicine                     Social Science & Medicine 296 (2022) 114737

The current attention on the detection and treatment of suicidal military members is necessary, but ultimately insufficient to meet the needs of many suicidal service members. Results from our study suggest that the status quo progression for many service members at elevated suicide risk is one of worsening disconnection, evidenced by the trajectory of declining valued connections found in the active control condition.

Program achieved the objective of strengthening the network structure of unit relationships over time, including socially integrating those currently at elevated risk for suicide. This is an especially important hypothesis since W-CP training has no content encouraging Airmen to form connections with at-risk peers.

NIMH

Effective upstream suicide prevention programs that can be implemented in real-world settings are urgently needed to address the significant and growing problem of suicides.

No RCT-validated universal programs shown to reduce vulnerability to suicide are in wide use. To fill this gap, we developed the Wingman-Connect Program–group-based prevention approach that strengthens protective relationship networks and skills for managing challenge.   This strategy of reducing suicide risk could be highly impactful as a complement to the predominant approach of trying to mitigate risk only after suicidal individuals are identified.

Wyman has led research (NIH, CDC funded) testing that program, which disseminates skills for social health through youth peer networks.  The high-energy, interactive training improves student connectedness and coping norms, and protective effects spread school-wide including adult help for suicidal youth.  A study aggregating three RCTs (N=78 schools; 39,900 students) showed fewer suicide deaths in schools implementing this approach.  

Automatically generated stuff from ChatGPT to look over

In the context of preventative mental health care, "ecological validity" refers to the extent to which the findings and interventions of mental health research or practices are applicable or relevant to real-world settings. This concept emphasizes the importance of ensuring that research studies, assessment tools, and interventions are reflective of the conditions, challenges, and dynamics that individuals encounter in their everyday lives. Ecological validity is crucial for the effective translation of theoretical knowledge and clinical insights into practical strategies that can be implemented in various environments, such as schools, workplaces, communities, and healthcare settings.

For preventative mental health care, ecological validity ensures that:

1. Assessment Tools are relevant and applicable to the diverse situations and contexts in which individuals might experience mental health challenges. This means that tools should accurately reflect the range of symptoms and behaviors as they occur in natural settings, outside of the controlled conditions of a clinical or research environment.
2. Interventions are designed to be feasible and effective in the real world. For example, an intervention that promotes mental well-being through regular physical activity should consider factors like access to safe outdoor spaces, availability of time, and cultural attitudes towards exercise.
3. Research Findings can be generalized or applied to the target population in their everyday environments. Studies that have high ecological validity provide more useful and actionable insights for public health policies, school-based programs, workplace wellness initiatives, and community mental health services.
4. Cultural and Contextual Relevance is maintained, acknowledging that the effectiveness of preventative strategies may vary based on cultural norms, societal structures, and environmental factors. Interventions with high ecological validity respect and incorporate these differences to ensure relevance and effectiveness across diverse populations.

Improving the ecological validity of preventative mental health care initiatives involves ongoing collaboration between researchers, practitioners, community members, and policy-makers. It also requires a commitment to participatory research methods, where interventions are co-developed with those who are meant to benefit from them, and continuous evaluation and adaptation based on real-world feedback and outcomes. By prioritizing ecological validity, preventative mental health care can offer more meaningful, accessible, and sustainable support to individuals and communities.