Importance
of Health Education Research to Health Education
Robert
S. Gold, PhD, DrPH, FAAHB; Nancy L. Atkinson, PhD
Objective:
To focus on the benefits and importance of research to the practice
of health education. Methods: The paper discussed the potential
of quality research as well as the barriers that keep health educators
from using, applying, and sharing their work. Results: The
basic challenges health educators face in translating their research
into practice relate to: becoming well-versed in the science base
and previous lessons learned; collaborating effectively; and passing
on knowledge by mentoring. Conclusions: To move forward,
health educators need to organize their knowledge and make it accessible.
This includes explicit and tacit knowledge, work in progress, and
a coordinated research agenda. Finally, health educators need to
be flexible so they can enable future research needs.
In
presenting this closing keynote address, I'm following in the footsteps
of some true giants in health education research and practice. You'll
remember that Larry Green opened this inaugural conference as its
first award winner. He talked about this "relaxed venue"
and then proceeded to raise the stakes for all other speakers by
taking on the challenges posited by the Ottawa Charter and illustrating
the landmark applications of best practices in health promotion
and public health while explaining both the virtues and the problems
of generating best practices1 but that was just the beginning.
Glover, Leischow, Sussman, Elder, Werch, DiClemente, Hansen, Holder,
Rainey, Jeffery, Black, Peng, Hobbs, and Torabi all showed why there
was a need for, and a benefit to the creation of the American
Academy of Health Behavior (AAHB).
Glover's
vision, and the hard work of the people attending and wishing to
attend this inaugural meeting of AAHB, have made this a wonderful
experience. We would like to express our appreciation for the opportunity
to participate in both The Academy and this first of many very successful
meetings.
The Hedgehog
and the Fox
Sometimes
there are models that are worth following. Today, ours is a series
of speeches delivered in 1998 by Donna Shalala. On October 15 of
that year, she delivered the Millenium Address at Hunter College,
New York. Her address was titled "The Hedgehog and the Fox."2
We thought of that address for 2 reasons: a) because addressing
the topic for today requires some stretching of the imagination
and b) because she borrowed from a great philosopher to make her
points at the leading edge of the new millennium.
Shalala
related the story of Isaiah Berlin, the brilliant social and political
theorist who tried to address some questions in his famous essay
on Tolstoy's view of history: "The Hedgehog and the Fox."
The title comes from a line in an ancient Greek poem: "The
Fox knows many things, but the Hedgehog knows one big thing."
According to Berlin, Tolstoy desperately wanted to be a hedgehog.
He wanted to see and understand the world according to a central
vision, one universal organizing principle that would explain the
direction of history. As luck would have it, he had the soul of
a fox, divining from his deep understanding of human relationships
a mosaic world of unrelated experiences and beliefs that did not
fit into any single precept.
This
led Tolstoy to draw 2 important conclusions. First, no individual
even a Napoleon can control the course of human events.
Second, to paraphrase Berlin's critique of Tolstoy, "the higher
we are in the pyramid of authority, the farther we must be from
its base the ordinary men and women whose lives are the actual
stuff of history." In other words, it is the will of nations
as expressed in the countless actions, thoughts, loves, dreams
and beliefs of people not the will of kings or presidents
or CEOs that ultimately forces change and moves history. Such is
the story with research in health education. Its potential will
never be realized unless we link researchers with populations and
practitioners.
As
a New Yorker, I believe that most people in this country recognize
Yogi Berra for his unintentionally witty philosophy sayings
like "90% of the game is half mental," "It's déjà
vu all over again," and "It ain't over `til the fat lady
sings." If you think about it, some of his best lines offer
important lessons when it comes to explaining the importance of
research to health education. So as we move on, let me offer "Five
Lessons from Yogi Berra."
Potential
of Research
"It
gets late early out there." Yogi once described
how left field in Yankee Stadium is tough to play during the World
Series because the late autumn sun casts deep shadows across the
field. As Yogi explained, "It gets late early out there."
That's Yogi Lesson Number 1: My father used to say, "The days
are long, but the years are short." Notwithstanding the long
days, research projects pass more quickly than we expect. Before
you know it, we are trying to get continuation grants to finish
the original work we proposed. It would be nice if we could begin
with a crystal ball that allows us to ask the important questions
and to figure out ways to answer them.
Why
do research? When it works well, we engage in research to answer
questions of importance. Most important, well-crafted research in
health education puts us in a position to act prospectively rather
than reactively to health needs. It also allows us to act in an
informed manner.
Ordinarily,
the major objective of research is to describe or explain phenomena
for prediction, control, or understanding of
causal mechanisms. We attempt to predict the circumstances under
which health consequences will occur or the likely health outcomes
of choices or prevention programs, to control causal chains of events
to reduce the potential for health risks, or to gain a better understanding
of how these causes operate. Our attempts to study relationships
between presumed risks and their subsequent effects on health, factors
that influence individual decision making, or potential success
of interventions designed to enhance the health of individuals are
all related to these 3 activities. Usually, the research process
in health education will attempt only one of these 3 key objectives
because to do more in one study would be difficult, unmanageable,
or unnecessary. Finally, to do more than one of these may come from
failing to recognize that they build upon one another.
- Specifically,
in order to make predictions, we must be able to describe
the conditions under which an event occurs. As we more clearly
describe these antecedent conditions, our ability to make
predictions improves.
- We
understand that two assumptions must be made to ensure our capacity
to control a sequence of events: (a) There must be a clearly
described causal chain between antecedents (causes)
and consequents (effects; ie, the linkage must not be by
chance); and (b) the causal chain must be amenable to manipulation.
In the absence of these 2 assumptions, control cannot be exercised.
- Understanding
requires being able to explain how the causal chain between antecedents
and consequences operates. Many would suggest that this is the
highest level of explanation, yet it is important to recognize
that such understanding can occur in the absence of our ability
to manipulate or control a chain of events.
Research
as a way of knowing. In order to deliver on this promise,
we need some attention to detail. An American philosopher, Charles
Pierce, suggested that there are 4 ways of "knowing,"
or "fixing belief."3 A way of knowing is sometimes
called a metaphor, and metaphors themselves are not necessarily
clearly understood. This does not prevent our use of them to establish
meaning, and metaphors are widely used despite our lack of understanding
of them.4
The
principal metaphors mentioned by Pierce3 include the
method of tenacity, the method of authority, the a
priori method, and the method of science. It is worth
reviewing these methods of knowing as a backdrop to our intent and
purpose here.
- The
method of tenacity posits that truth comes from firmly held beliefs.
- Authority
is a method of belief based on the weight of the source of information.
- The
a priori method, sometimes called intuition, is able to be used
to identify truths because truths are "self-evident."
- The
last method mentioned by Pierce is the method of science.
Research
as a scientific process provides certain advantages over other means
of "knowing," or of answering pertinent questions. The
scientific process is a logical process with procedural specifications
that provide a framework for gathering information in many forms.
It is reductive, in that it allows us to amass large amounts of
information and extract meaningful generalizations. The scientific
process is also objective. Objectivity means that the process and
outcomes should be independent of the individuals conducting that
process. The scientific process is empirical, in that conclusions
are based on relevant data collected for specific purposes. Note
carefully that the definition of empirical does not describe the
nature of the data itself. Both quantitative and qualitative strategies
can be used to provide empirical evidence.
The
scientific process is benefited by replication. Replication enhances
the confidence in the conclusions we draw from the data collected,
but perhaps most important, the scientific process is a self-correcting
process. More than any other way of knowing, we depend on the continued
process to guide us in our beliefs in theory and process. Further
work in any area provides more support for those theories or raises
more questions that then must be examined by the same process.
Quality
research. Research design is defined as a series of systematic
compromises between what is desirable and what is realistic in any
given research situation. Kish5 suggests:
Statistical
designs always involve compromises between the desirable and the
possible. We face inevitable compromises in the choice of the
very nature and the structure of statistical designs; in their
scope and breadth, and in the size of research projects. . . research
design involves compromises that are the basic philosophical problems
of all empirical sciences: how to make inferences to large populations,
to infinite universes, and to causal systems from limited samples
of observations, which are also subject to diverse errors and
to random fluctuations.
The
importance and potential of research design, then, comes from 2
different elements: (a) improving the power of design by making
appropriate and systematic compromises and (b) enhancing the replicability
of research by documenting the nature of the compromises made and
the reasons for making those choices. Based on these 2 points, the
issue of appropriate compromise is a most critical element in research
design. Our capacity to do these 2 things may, in fact, reduce the
gap between the presumption of hard and soft science.
Participatory
research. This circumstance sets up a classic good-news,
bad-news situation. The good news is that the notions of participatory
research and engagement of audiences in the planning phases of research
are well developed. The bad news is that many of our public institutions,
policies, programs, and our very assumptions may not be ready for
engagement with a larger population of diverse people than ever
before.
Barriers
to Effective Research
"We
were the overwhelming underdogs." Yogi Berra's
fond description of his baseball team one year leads to Yogi Lesson
Number 2. He said, "We were the overwhelming underdogs."
We may feel that just getting our research funded is enough, but
we must not become comfortable with an underdog role, of settling
for the status quo. Here are some threats to escaping this mire.
Not
using what we know. Dr. William Foege has been known to say
that 90% of what we know is not used. There are some researchable
questions that arise out of that assertion, but we can easily think
of several reasons why. One is that we are too busy in the present
to recollect the past. At numerous conferences, I have been struck
by the number of papers I hear that repeat a seminal idea from 20
years ago, yet give no recognition that they are repeating past
work. This shows that we have no institutional memory and, even
worse, that we are wasting resources recreating the wheel. Why are
we wasting our time and efforts like this? Largely, the problem
is that it is not easy to use the knowledge we have because it is
fragmented and disorganized. The closest thing we have to a taxonomic
structure is the Medline thesaurus. We have not taken the time as
a field to organize our knowledge.
Point
2: We need to turn some of our research ability inward and actually
study the structure of our knowledge _ both explicit knowledge
and tacit knowledge. Once that structure is in place, then tools
can be crafted to make it more accessible. At that point, knowledge
becomes more accessible to researchers, practitioners, and consumers
alike.
Not applying
what we know. Besides not using what we know, we would
add a further assertion that less than 50% of practicing health
educators are effectively applying the 10% of what we do know. This
issue is clearly illustrated by the Prevention Research Center (PRC)
Program. This is a network of 23 PRCs and 3 Urban Research Centers
(URCs) focusing on community-based prevention research centered
on research themes that are specific to each Center. The overarching
goal of this network is best characterized by the interaction of
3 specific tasks: (a) conducting innovative prevention research,
(b) generating new knowledge, and (c) translating knowledge into
improved public health practice and policies.
In
1997, the Institute of Medicine (IOM) conducted a formal assessment
of the PRC Program. The IOM documented many strengths of the PRC
program, while pointing out a weakness in documenting the translation
of knowledge gained from research into public health programs and
policy: "PRCs have not regularly and systematically reported
their findings concerning research dissemination and implementation
to CDC, and CDC does not have a mechanism for assembling findings
from the PRCs in order to promote such activities."
What
are the barriers? Weiss6 provided 5 suggestions on the
inherent limitations with utilization issues related to evaluation,
but they may be extended even now to health education research:
(a) unclear direction or answers to research questions due to inconclusive
evidence from studies, (b) conflicting findings from multiple studies,
(c) postponement of action to a more appropriate time, the result
of lack of motivation or opportunity, (d) annulment of research
recommendations due to competing operational and programmatic constraints,
and (e) use of evaluation results to justify predetermined program
changes.
Health
education research involves the translation of established and promising
methods of disease prevention and health promotion to communities.
Despite its importance, relatively little attention has been paid
to systematic approaches to determining the impact of health education
research on health education practice. Evaluation of these effects
is challenging, particularly in light of multifactor causation,
long time periods between the conduct of research and its subsequent
publication, and difficulties in determining the influence of other
factors such as costs. Greater translation of research findings
is needed to accomplish our broad health education goals. These
efforts may be enhanced by academic-practice partnerships.7
Not asking
what others know. Our effective use of the results of research
is dependent upon the effective integration of behavioral science
advances with much other promising genetic, medical, nutritional,
technology, health care, and policy research in order to broaden
our understanding of the underlying mechanisms of behavior that
might be amenable to health education and promotion, but also to
improve quality of life and public health for all persons.8
Although so necessary to a field such as ours, interdisciplinary
study is difficult. The difficulty comes from having to integrate
what we hear from others into our thinking, which can represent
a loss of control. Making the commitment to ask, it should not be
an empty gesture, or it is simply a waste of time (eg, "Don't
ask if you've already made up your mind").
Another
fear in asking others for input or participation is that it sometimes
results in the loss of control over a project, sometimes loss of
a project altogether. For example, our university encourages interdepartmental
research, but each player inevitably counts the number of collaborators
and thinks how working with others will affect his or her own piece
of the pie, both the size and the quality of the slice.
Academic
arrogance is another barrier to asking others what they know. We
must be less concerned about the letters after someone's name (or
lack thereof) than about their experience, insight, and abilities.
Participatory research principles show that we have come a long
way from research on to research with. However, we
may not extend the same courtesy to those with whom we work, whether
they are students, staff, colleagues, or partners. We lose the opportunity
to learn from them or to teach them, and they are less likely to
be fully engaged in the effort.
Not telling
others what we know. Changes are required on the part
of behavioral scientists in how they organize and present their
research and on the part of potential users of this knowledge, including
other health professions, organizations, and funding agencies and
communities. Research is often a competitive process in which we
hold our own knowledge close to us, lest we give away our advantage.
We also may not share what we know when we report on our research
becauseif things go wrongwe may be perceived negatively,
even lose funding. Further, sharing what we know takes time, and
there are only so many days, hours, and minutes. For example, I
was supposed to be at 2 other meetings today!
The Challenge
"You
can observe a lot by watching." We, as researchers,
can take several steps right now to enhance the practice of health
education and to engage other health education researchers as partners
in that endeavor. That leads to the next lesson from Yogi Berra.
He once said, "You can observe a lot by watching." When
it comes to making the millennium a golden age for the collaboration
of research and practice, we can also observe a lot by listening
Listening to the research as well as listening to each other.
Research expertsyes. Practitionersyes. Populations,
service providers, advocates, and otherswhether public, private,
or NGOfrom nations all over the world.
Point
3: The best research processes model how we should practice--in
a multidisciplinary, multiprofessional, collaborative environment,
and based on participatory paradigms.
How
do we prepare for this eventuality? How do we make sure our researchers,
research institutions, and assumptions keep up with practitioners
and the potential recipients of health education? How can we make
sure we all enjoy the benefits of a sound research agenda? How do
we create "best practices" that are alive and able to
evolve over time, across populations and settings?
Organizing
our research. There are different types of research,
and they are not all as useful to health education as they have
the potential to be (eg, the Behavioral Risk Surveillance System
[BRFSS] and other survey research may be very useful; some outcome
studies should be).9 Access and ease of use are primary
tasks in making research useful.
It
is worth distinguishing between the 2 different types of knowledge
we need to move forward. One of these is the explicit knowledge
that can be found in our collective literature resulting from active
research. All of our documented knowledge is explicit and potentially
accessible to all. Access to such knowledge would be improved immensely
by giving it structure so it can be reviewed and searched in a meaningful
way. Access would further be enhanced if all the literature were
made available, not just those with significant results. Not only
is research limited by publication bias, but it is also months or
years old by the time it reaches us. Theses and dissertations reflect
a less biased pool of research knowledge because they escape from
publication bias. However, we also need a process for making research
available, even while in process and when it is published elsewhere
(eg, final reports, progress reports).
The
other type of knowledge is tacit knowledge, which is contained in
only a few placesthe gut, instincts, and intuition that guides
some of our unconscious or subconscious processes as professional
health educators and researchers. This kind of knowledge is not
documented wellif at alland we do not often know how
to communicate it or share it with others because we often use it
without recognition. One of our biggest challenges is to begin documenting
these heuristics so we may pass them on to colleagues and succeeding
generations of health educators. Another challenge is encouraging
those with the tacit knowledge to share it, given their time constraints
and level of willingness.
Organizing
our knowledge base is also a challenge because it is constantly
evolving. Even if well structured, the research base will be worthless
and unused if it is not current. Therefore, we have to commit to
knowledge development and sharing as a way of life.
Learning
how to apply it. Having access to the research is not sufficient
to guide health education researchers. Part of the reason is that
research is studied within a context of a specific population, community,
and situation. Even replicated research may not explain fully how
the research would change, given a new context.
We
recently did a consensus development process that, in part, looked
at the types of information researchers and community practitioners
needed the most.
We
found the following top 10 content areas on prevention research,
in priority order:
1.
Describing what happened.
2.
Describing barriers and their solutions.
3.
Describing what works and how it can be made to work in your community.
4.
Describing what has been tried.
5.
Including interventions aimed at a broad spectrum of health conditions
or at promoting health.
6.
Providing information on the sustainability of interventions and
on strategies for sustaining them after initial grant funding runs
out
7.
Describing unintended consequences of actions.
8.
Providing the outcomes of prevention programs (data) in forms that
are usable within lay community groups
9.
Describing what does/does not work.
10.
Describing how to deal with challenges.
What
is clear from this list is that much of this information is hard
to find because published literature does not focus on these issues.
Tacit knowledge provides at least one strategy to develop and deliver
this information. Researchers simply query each other about what
works in which situations and how to solve problems that present
themselves. The challenge will be developing a mechanism that researchers
will know and trust.
Collaborating.
Despite the real and perceived barriers to collaboration,
we must also be committed to overcoming those barriers in order
to produce valid, useful, and used research. In terms of validity,
collaborating in teams or with colleagues also provides a check
and balance on the research we propose and conduct. This is clearly
a step in the publication of research, but it begins much earlier.
For example, one of the important lessons in grantwriting is the
need to let peers review and comment on the work we propose before
we turn it in. Sidney McNairy, the director of Research Infrastructure
at NIH's National Center for Research Resources, makes a point of
asking young investigators if anyone besides themselves read their
proposal. However, collaboration is not only important for success
in publication and funding, but it is also essential for the validity
of the research to our populations of interest. As we have said
before, multidisciplinary, multiprofessional teams allow us to create
research that looks at all sides of a health issue and is, therefore,
able to reflect it and the needs of target audiences completely
and clearly.
Beyond
validity, collaboration makes research more useful by assuring the
questions we ask are important and the procedures we use are acceptable.
For example, we know that our instruments must be reliable and valid,
but increasing emphasis is on their acceptability to the target
audience. Specifically, the target audience must accept the way
in which questions are stated, or their responses may be influenced.
Including the needs and concerns of the target audience, practitioners,
and other researchers throughout the research process through collaboration
is a hallmark of participatory research.
Active
collaboration between researchers and service agencies also results
in more successful program adoption than distribution of implementation
packages alone.10 Therefore, we should look at ways to
engage researchers actively with the practice community. Larry Green,
with the work done at his Institute for Health Promotion Research
(IHPR) at the University of British Columbia provided a model. Not
only did the IHPR have a network of high quality of researchers
from throughout the university, but that research network also had
effective collaborations with most local, Provincial, and national
agencies in Canada.
Mentoring.
It is our belief that there are many methods of technology
transfer. Training is one of them, but the technology transfer literature
suggests that the best way to transfer technology is to transfer
people. One of the most powerful strategies for professional development
and technology transfer is mentoring, which gives us an important
clue about the importance of research mentorship. The "health
education family tree"the active engagement of students
in the research of their mentorshas a profound effect on the
practice of health education. Students not only learn the methodologies,
but they also learn the application of results, adaptation and evolution
of research questions, and expanded scope of impact on health education
(Matt Kreuter/Vic Strecher; Becky Smith/Ann Nolte).11
To that end, we each have a professional responsibility to be the
best mentors we can, and we would suggest that we offer each of
our mentees a lifetime guarantee.
Recommendations
When
you come to a fork, take it. Now Yogi Lesson Number
4: "When you come to a fork in the road, take it." Right
now, many researchers and practitioners are at a fork in the road
when it comes to their potential for collaboration. Many of us are
staring at the 2 paths: One fork is called the "Status Quo."
On that road, it's research as usual. The other fork the
fork can be called "The Future."
Point
4: High-quality research gives us options. Long ago we began
to realize that "one size does not fit all." To that
end, research provides a road map for customizing, tailoring,
and individualizing opportunities for choice even within the
context of population-based work.
Call
to Action
Researchers
can't ask questions that only benefit them. These conclusions
are exactly why the lessons learned from those like Larry are so
important. Researchers cannot and should not ask questions
of interest only to them, conduct research in such a way that neither
the subjects nor the intended recipients of the results understand
or benefit from the work, nor engage in research on populations
rather than with populations.
Leaders
need to create opportunity for growth. Today, all leaders
whether in politics, education, or business have a
moral obligation to create opportunity for growth. This is the case
for researchers as well. That's a truth we should all hold to be
self-evident, but in addition to moral obligation, what about self-interest?
For those of us in positions of leadership in research and/or practice,
is our ability to direct our discipline's destiny so firmly within
our grasp that we can afford to do anything less than what is right?
As leaders in health education research, we have an obligation to
actively support and participate in efforts to organize research,
collaborate, and mentor. To that end, we make the following recommendations.
Recommendation
-- To deal with the need to manage knowledge, create a system
of knowledge growth, including both explicit and tacit knowledge.
Recommendation
-- Support not only the publication of research literature,
but also the active participation with practitioners on the application
of research results.
Recommendation
-- Create a Web-based look-up system for all MA, MPH, PhD,
DrPH, ScD work in progress. As soon as it is approved for conduct,
submit an abstract and related information.
Recommendation
-- To deal with the kind of technology transfer that comes from
mentoring, create a health education family research and practice
genealogy. If we do this, we have a database of the origin of ideas
in health education. From this, let us examine the origin of conceptual
lines of research to help us understand how to improve the efficiency
and effectiveness of such relationships.
Recommendation
-- Create a national coordinated research agenda. Provide a road
map showing were independent researchers can contribute. In each
of the more than 200 academic departments of health education in
our nation's colleges and universities that offer graduate degrees
in health education and public health, there are many independent
researchers - graduate students doing thesis and dissertation research
and faculty members whose promotion and tenure efforts are tied
to their scholarly activities. For many of these independent researchers,
the identification of a researchable problem and a desire to contribute
to the field are difficult tasks. An effective agenda will provide
guidance to these researchers.
Recommendation
-- Create a consensus process to organize what has been accomplished
to date.
Recommendation
-- Take advantage of research networks already in place. Take a
look at the research network that already exists. Figure 1 contains
a map of the location of CDC's Prevention Research Centers (PRC).
There are 23 PRCs and 3 Urban Research Centers (URCs). If we were
to overlay this with the more than 200 college and university-based
health education training and degree programs, imagine the potential
network that would be. Imagine how telementoring/knowledge management
systems would help this network reach the practitioner in the field
in communities outside this network.

Preparing
for the Future
Stay
flexible to the unforeseen future so we can enable it. Finally,
Yogi Lesson Number 5: "The future ain't what it used to be."
Most people who have dabbled with futures work recognize that there
is no single future. The future keeps changing based on the outcomes
of events in the present. In complex societal and health events,
stochastic processes ensure that randomness often plays a role in
these outcomes. Antoine de Saint-Exupery, in The Wisdom of the
Sands, tells us: "As for the future, your task is not to
foresee, but to enable it." From our perspective, one of the
most important lessons to be learned is that we cannot just let
the future happen to us or let others determine what it will be.
It is time for us to establish and act on a vision of a health education
research and development paradigm that propels health education
into its future of choice.
No
longer will we have to explain ourselves, redefine ourselves, or
provide a rationale for our existence as a profession. Rather, the
soundness of our research agenda will allow us to be better prepared,
able to act with knowledge and timeliness, make more informed choices,
and improve the quality of our work as a discipline. By applying
sound research collaboration and participatory models, we will provide
a role model for our practice outside the research arena.
Summary
We
have identified 5 benefits of research to health education:
- Sound
research gives us the capacity to act prospectively with strength
and reactively with knowledge.
- Once
a structure is in place to organize our knowledge base, knowledge
becomes more accessible to researchers, practitioners, and consumers
alike.
- The
best research processes model how we should practice in
a multidisciplinary, multiprofessional, collaborative environment,
and based on participatory paradigms.
- High-quality
research provides a road map for customizing, tailoring, and individualizing
practice opportunities even within the context of complex population-based
work.
- A
well-crafted and executed research and development agenda will
enable our vision of the future.
References
1.Green
LW. From research to "Best Practices" in other settings
and populations. American Academy of Health Behavior, Research Scholar
Award Address, First National Conference, 2000, Sante Fe, NM. American
Journal of Health Behavior 2001;25(3):165-178.
2.Shalala
D. The Hedgehog and the Fox. in The Millenium Address. 1998. Hunter
College, New York. Can be accessed at http://www.hunter.cuny.edu/prs/prhspeech1.html
3.Buchler
J. Philosophical Writings of Peirce. New York: Dover 1955.
4.Kaiser
C, Gold RS. Perception, psychedelics and social change. J Drug
Educ 1973:3;141-150.
5.Kish
L. Statistical Design for Research. New York: John Wiley & Sons
1987.
6.Weiss
C. Evaluation Research: Methods for Assessing Program Effectiveness.
Englewood Cliffs: Prentice-Hall: 1972.
7.Brownson
RC, Simoes EJ. Measuring the impact of prevention research on public
health practice. Am J Prev Med 1999;16(3 Suppl):72-79.
8.Glasgow
RE. Behavioral science in diabetes. contributions and opportunities.
Diabetes Care 1999;22(5):832-843.
9.Bloom
Y. Data uses, benefits, and barriers for the behavioral risk factor
surveillance system: a qualitative study of users. J Public Health
Manag Pract 2000;6(1):78-86.
10.Kelly
JA. Bridging the gap between the science and service of HIV prevention:
transferring effective research-based HIV prevention interventions
to community AIDS service providers. Am J Public Health
2000;90(7):1082-1088.
11.Fuller
SS. Enabling, empowering, inspiring: research and mentorship through
the years. Bull Med Libr Assoc 2000;88(1):1-10.
Am J Health Behav 2001;25(3):301-310
|