ERIC EJ1142977: Collaboration Patterns as a Function of Article Genre among Mixed Researchers: A Mixed Methods Bibliometric Study

2017·0.2 MB·English

by  ERIC

Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
Collaboration Patterns as a Function of Article Genre
among Mixed Researchers: A Mixed Methods

Bibliometric Study
John Jordan
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Melanie Wachsmann
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Susan Hoisington
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Vanessa Gonzalez
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Rachel Valle
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Jarod Lambert
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

83 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
Majed Aleisa
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Rachael Wilcox
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Cindy L. Benge
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA

Anthony J. Onwuegbuzie (Corresponding author)
Department of Educational Leadership and Counseling, Sam Houston State University
Box 2119, Huntsville, Texas 77341-2119, USA
&
Distinguished Visiting Professor, Faculty of Education, University of Johannesburg
B-Ring 433, Kingsway Campus, Auckland Park, South Africa
E-mail: [email protected]

Received: February 7, 2017 Accepted: April 2, 2017 Published: April 5, 2017
doi:10.5296/jei.v3i1.10699 URL: http://dx.doi.org/10.5296/jei.v3i1.10699

Abstract
Surprisingly, scant information exists regarding the collaboration patterns of mixed methods
researchers. Thus, the purpose of this mixed methods bibliometric study was to examine (a)
the distribution of the number of co-authors in articles published in the flagship mixed
methods research journal (i.e., Journal of Mixed Methods Research [JMMR]) as a function of
article genre (Quantitative Phase); (b) the relationship between the genre of articles published
in JMMR and degree of collaboration in these articles (Quantitative Phase); (c) the difference
between the number of authors in empirical research articles and non-empirical research
84 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
articles published in JMMR (Quantitative Phase); and (d) select leading mixed methods
researchers’ collaboration experiences as a function of genre of article (Qualitative Phase).
An analysis of all articles published in JMMR from 2007 (its inception) to 2015 (the latest
complete year at the time that the study was conducted) revealed (a) a statistically
significantly higher proportion of empirical research articles (63.2%) than non-empirical
research articles (36.8%), (b) that empirical research articles were 1.4 times (95% confidence
interval = 1.10, 1.78) more likely to involve multiple authors than were non-empirical
research articles; and (c) that empirical research articles contained statistically significantly
more authors than did non-empirical research articles. With respect to the qualitative phase,
four themes (i.e., mental perception, mixed methods research, publication and research aids,
and independent/group work) emerged regarding collaboration for empirical articles versus
for non-empirical research articles. Implications of these findings are discussed.
Keywords: Mixed methods bibliometric study, Collaboration patterns, Genre of article,
Empirical research articles, Non-empirical research articles, Theoretical articles,
Methodological articles, Conceptual articles
1. Introduction
In general, works published in journals fall under two broad categories: empirical research
articles and non-empirical research articles. Specifically, as noted by Onwuegbuzie and Frels
(2016), empirical research articles
represent research wherein data are generated via direct observation or experiment in
order to address one or more research questions (i.e., interrogative statements that the
researcher attempts to answer using research techniques) and/or to test one or more
hypotheses (i.e., proposed explanations of observable phenomenon that can be tested via
research). As such, findings from empirical research studies are based on actual evidence,
as opposed to theory, assumptions, or speculations. (p. 4)
Moreover, an empirical research article involves the primary reporting of the conduct of a
study in which quantitative and/or qualitative data are collected and analyzed quantitatively
and/or qualitatively. Thus, an empirical research article involves the reporting of a
quantitative research study (i.e., comprising the collection, analysis, legitimation, and
interpretation of numeric data, with objectives that include: to explore, to describe, to explain,
to control, and to predict phenomena), a qualitative research study (i.e., comprising the
collection, analysis, legitimation, and interpretation of nonnumeric data that naturally occur
from one or more of the following sources: documents, talk, observations, and images), or a
mixed methods research study (i.e., involving the mixing or combining of “quantitative and
qualitative research techniques, methods, approaches, concepts or language … [within] a
single study”; Johnson & Onwuegbuzie, 2004, p. 17). All three types of studies (i.e.,
quantitative research study, qualitative research study, mixed methods research study)
optimally involve the following four phases: research conceptualization (e.g., determining
the goal, objective, rationale, and purpose of the study; developing the research question[s]),
research planning (i.e., selecting the sampling design and research design), research
implementation (i.e., collecting, analyzing, legitimizing, and interpreting data), and research
85 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
dissemination (i.e., sharing the research findings [e.g., orally, visually, in writing]).
In stark contrast, non-empirical research articles represent works that do not involve the
primary reporting of the conduct of a study. Rather, these articles can be conceptual,
theoretical, methodological, or historical in nature. Conceptual articles involve the
introduction of a new or expansion of an existing concept, idea, thought, interpretation,
concern, challenge, schema, or the like. Theoretical articles involve the introduction,
modification, or expansion of one or more theories or models that optimally stem from
concepts, assumptions, ideas, beliefs, propositions, hypotheses, hunches, findings, or the like.
Methodological articles involve the introduction of a new or expansion of an existing
research design or method (e.g., sampling design, instrumentation, procedures, analysis).
Finally, historical articles involve the author(s) situating the extant literature in historical
contexts such that they yield a historical framing of a phenomenon, concept, event,
experience, or other entity. Simply put, for the most part, non-empirical research articles in
general, and conceptual, theoretical, methodological, or historical articles in particular, should
represent persuasive essays wherein these articles contain “a warranted and transparent path
of argumentation in which parts are integrated to form a logical and coherent whole that
exceeds the sum of its parts and wherein conclusions are logically based on appropriate
evaluative frameworks” (Onwuegbuzie & Frels, 2016, p. 363).
Whereas empirical research articles are overwhelmingly characterized by
Introduction/Literature Review, Method, Results, and Discussion sections, non-empirical
research articles do not have these components. Instead, non-empirical research articles are
not so formulaic, and, hence, (potentially) are substantially more diverse. And this gap in
diversity is even more profound for mixed methods research articles because non-empirical
mixed methods research articles can be quantitatively focused (e.g., involving a new
procedure or analysis for the quantitative phase[s] of mixed methods research studies),
qualitatively focused (e.g., involving a new procedure or analysis for the qualitative phase[s]
of mixed methods research studies), or mixed methods focused (e.g., involving a new
procedure or analysis for the quantitative and qualitative phases of mixed methods research
studies). As such, it could be argued that these two categories of research articles (i.e.,
empirical research articles, non-empirical research articles) require a different set of
scholarship skills—especially with respect to the field of mixed methods research. And it is
likely that this different set of scholarship skills, thought processes, and formats necessitates
different levels and types of collaboration among authors representing these two categories of
research articles. Yet, to date, collaboration patterns as a function of genre very rarely has
been the focus of investigation. One notable exception is the bibliometric study (i.e., study of
the academic literature using techniques such as a citation analysis) conducted by Fisher,
Cobane, Ven, and Cullen (1998), which led to the finding that multiple authorship was more
common for empirical research articles wherein sophisticated quantitative research
approaches were used. Notwithstanding, to date, we could not find a single study in which the
degree of collaboration rates was compared as a function of genre (i.e., empirical articles vs.
non-empirical articles) of mixed methods research articles. Further, none of these numerous
bibliometric studies included the use of qualitative research approaches. Yet, information
86 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
about collaboration patterns would be useful for beginning and emergent mixed methods
researchers as they learn how effectively to write these two categories of articles. Thus, this
was the focus of the present mixed methods research article. Specifically, using Plano Clark
and Badiee’s (2010) typology, the research questions in this study represented separate
research questions—specifically two quantitative research questions coupled with one
qualitative research question. The following quantitative research questions were addressed:
1) What is the distribution of articles published in a select mixed methods research journal as
a function of genre?
2) What is the relationship between the genre of articles published in a select mixed methods
research journal and degree of collaboration in these articles?
3) What is the difference between the number of authors in empirical research articles
published in a select mixed methods research journal and the number of authors in
non-empirical research articles published in the same select mixed methods research journal?
The following qualitative research question was addressed:
4) What are select leading mixed methods researchers’ collaboration experiences as a
function of genre of article?
1.2 Genre of Articles Published in Mixed Methods Research
Journal of Mixed Methods Research (JMMR) is the flagship journal for mixed methods
researchers. According to the JMMR submission guidelines—and supporting our earlier
contention regarding two broad genres of articles—“JMMR publishes two types of
manuscripts: empirical research and methodological/theoretical discussions” (Fetters &
Freshwater, 2015, p. 205). Fetters and Freshwater (2015) state that for methodological/
theoretical discussions,
Topics of interest include the following: types of research/evaluation questions, designs,
sampling or measurement procedures, approaches to data analysis and integration,
validity, software applications, paradigmatic stance, writing structure, theoretical lenses,
cultural issues, ethical issues, validity, and quality issues in mixed methods research. The
hourglass model for such papers generally should follow that of a persuasive essay. This
involves reviewing the relevant mixed methods context to illustrate the problem clearly,
and then stating a clear objective or purpose for the paper. Most such papers will address
about three to five points. These should be written in the order laid out in the beginning,
and use illustrative and persuasive examples. At the bottom of the hourglass, authors
briefly summarize the key points or implications, honestly discuss limitations, and
suggest broadly future implications or applications. Structurally, the
introduction–persuasive argumentation–discussion typically has a 1:3-5:1 proportion in
terms of word count, so a 6,000-word manuscript should have roughly a
1,000:4,000:1,000 words breakdown, though the range will be 1,000:3,000-5,000:1,000.
An abstract of 120 words will roughly parallel these word proportions, 20:80:20. (p. 205)
In contrast, “original mixed methods articles are characterized by the collection and analysis
87 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
of both qualitative and quantitative data. Authors should integrate the findings and draw
inferences based on the qualitative, quantitative and mixed methods findings” (Fetters &
Freshwater, 2015, p. 205). Given these directions, the JMMR editors, currently, Professors
Michael D. Fetters (Editor) and José F. Molina-Azorin (Co-Editor), provide a clear
demarcation between empirical research articles and non-empirical research articles.
1.3 Collaboration in Research/Scholarship
Collaboration in research/scholarship occurs when two or more researchers/authors “work
together on a project and contribute resources and effort, both intellectual and physical”
(Subramanyam, 1983, p. 34). More specifically, Katz and Martin (1997) defined
collaboration as the “working together of researchers to achieve the common goal of
producing new scientific knowledge” (p. 7). As noted by Subramanyam (1983), there are
several types of collaborations in research/scholarship, including the following:
 Collaboration between organizations (e.g., universities);
 Collaboration among colleagues;
 Collaboration between an instructor and her/his student(s);
 Collaboration between an advisor/supervisor and her/his doctoral student;
 Collaboration between mentor and mentee(s);
 Collaboration between researcher and consultant;
 Collaboration between consultant (e.g., program evaluation) and client; and
 International collaboration.
Further, collaboration can take many forms that range from providing general ideas, opinion,
advice, or criticism (i.e., theoretical collaboration; Heffner, 1981) to participating actively
through many, most, or all phases of the research project (i.e., technical collaboration;
Heffner, 1981). However, for collaborators to justify having their names on a published work,
they must have engaged in technical collaboration. This point is made clear by the authors of
the American Psychological Association (APA) Publication Manual:
Authorship is reserved for persons who make a substantial contribution to and who
accept responsibility for a published work … Individuals should only take authorship
credit for work they have actually performed or to which they have substantially
contributed (APA Ethics Code Standard 8.12a, Publication Credit). Authorship
encompasses, therefore, not only those who do the actual writing but also those who have
made substantial scientific contributions to a study. Substantial professional contributions
may include formulating the problem or hypothesis, structuring the experimental design,
organizing and conducting the statistical analysis, interpreting the results, or writing a
major portion of the paper. Those who so contribute are listed in the byline. Lesser
contributions, which do not constitute authorship, may be acknowledged in a note (see
section 2.03). These contributions may include such supportive functions as designing or
88 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
building the apparatus, suggesting or advising about the statistical analysis, collecting or
entering the data, modifying or structuring a computer program, and recruiting
participants or obtaining animals. Conducting routine observations or diagnoses for use
in studies does not constitute authorship. Combinations of these (and other) tasks,
however, may justify authorship. (APA, 2010, p. 18)
1.3.1 Measuring Collaboration
A common way of measuring collaborative research patterns is via an index called the degree
of collaboration. According to Subramanyam (1983), the degree of collaboration can be
calculated via the following formula:
CC = NM/(NM + NS) (1)
Where, CC represents the degree of collaboration, NM represents the number of multiple
authored papers, and NS represents the number of single-authored papers. In other words, the
degree of collaboration is the ratio of the number of multi-authored journal articles to the
total number of journal articles (i.e., # of single-authored articles + # of multiple-authored
articles) (Subramanyam, 1983).
1.4 Mixed Methods Bibliometric Study
The present study of collaboration patterns as a function of article genre among mixed
methods researchers involved the conduct of what Onwuegbuzie et al. (in press)
conceptualized as a Mixed Methods Bibliometric Study. These authors posited that in mixed
methods bibliometric studies, qualitative data are mixed or combined with bibliometric
quantitative data for the purpose of helping researchers both to identify the patterns of
publications within a given field, discipline, or body of knowledge (i.e., Quantitative phase[s])
and to understand how to these patterns have emerged (i.e., Qualitative phase[s]) in an
attempt to ascertain the degree of development of various fields/disciplines/knowledge areas.
Accordingly, mixed methods bibliometric studies help researchers better “to evaluate the
processes of production, communication, and use of scientific information” (Vimala &
Dominic, 2013, p. 44). Thus, we deemed it appropriate to conduct a mixed methods
bibliometric study to address our research questions.
1.5 Philosophical Framework
The research philosophical stance for our study was a recently conceptualized stance by
Onwuegbuzie and Frels (2013) that they called a critical dialectical pluralistic stance.
Researchers who adopt a critical dialectical pluralistic stance are concerned with social
injustices that occur at Bronfenbrenner’s (1979) four major levels of environment that impact
a person, namely:
 The microsystem (Level 1): the immediate environment with which the person closely
interacts (e.g., home, recreation center, religious institution);
 The mesosystem (Level 2): the other systems in which the person spends time, such as
learning institution or place of work;
89 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
 The exosystem (Level 3): the systems by which the person might be influenced but of
which he/she is not directly a member, such as the relationships among teachers/instructors,
the institutional administrators, the person’s work supervisors, the person’s parents, or other
close family members; and
 The macrosystem (Level 4): the larger cultural world surrounding the person such as the
society or community at large that includes societal belief systems, cultural norms, ideologies,
policies, laws, or practices that indirectly influence the person.
Such concern leads critical dialectical pluralistic researchers to conduct what Onwuegbuzie,
Collins, and Frels (2013) coined as micro-research studies (i.e., Level 1: research wherein
one or more persons or groups are studied within his/her/their immediate environment[s]),
meso-research studies (i.e., Level 2: research wherein one or more persons or groups are
studied within other systems in which the he/she/they spends time), exo-research studies (i.e.,
Level 3: research wherein one or more persons or groups are studied within systems by which
the he/she/they might be influenced but of which he/she/they is not directly a member), and
macro-research studies (i.e., Level 4: research wherein one or more persons or groups are
studied within the larger cultural world or society surrounding him/her/them). Moreover,
uniquely, critical dialectical pluralistic researchers also are concerned with social injustices
that might occur as a result of the power differential between the researcher(s) and
participants. In addition to a goal of addressing social justice, critical dialectical pluralism
involves incorporating multiple epistemological perspectives (Onwuegbuzie & Frels, 2013).
Accordingly, in the present research study, the researchers assumed a research-facilitator role
that empowered some of the participants to assume the role of participant-researchers, who,
in turn, served as co-researchers at every stage of the mixed methods research process.
2. Method
2.1 Sampling Design and Sample
For the quantitative research phase, a criterion sampling scheme was used, which involved
the selection of the flagship journal for mixed methods researchers, namely, JMMR. For this
journal, all articles published from 2007 (its year of inception) to 2015 (the latest complete
year at the time that the study was conducted) were examined, with the exception of editorials
(which are not refereed). For the qualitative phase, participants were selected via a purposive
sampling scheme—specifically, a maximum variation sampling scheme (i.e., choosing
participants to maximize the range of perspectives investigated in the research study;
Onwuegbuzie & Collins, 2007), wherein participants were selected in order to answer the
qualitative research question (Teddlie & Yu, 2007). Specifically, the participants for this study,
which included two of the researchers who served as participant-researchers, were 14 mixed
methods researchers (6 women and 8 men) from institutions representing various geographic
areas in the United States. As demonstrated by Guest, Bunce, and Johnson (2006), 12
interviews are sufficient to “understand common perceptions and experiences among a group
of relatively homogeneous individuals” (p. 79)—suggesting that our sample size was
adequate for obtaining data saturation (Sandelowski, 2008) and theoretical saturation
(Sandelowski, 2008). Two participants were classified as being complete member participants
90 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
(Adler & Adler, 1987) because they were both researchers and participants in this study.
Overall, the mixed methods sampling design involved a concurrent design using nested
samples for the quantitative and qualitative components of the study (Onwuegbuzie & Collins,
2007) because the quantitative and qualitative data were collected concurrently (i.e.,
independently) and the vast majority (i.e., 80.0%) of the participants selected for the
qualitative research phase had authored/co-authored one or more articles in JMMR—the
journal selected for the quantitative research phase.
2.2 Mixed Methods Research Design
Our mixed methods bibliometric study involved combining postpositivism (i.e., quantitative
phase) and interpretive (hermeneutic) phenomenology (i.e., qualitative phase). This
combination yielded what Mayoh and Onwuegbuzie (2014, 2015) referred to as mixed
methods phenomenological research (MMPR)—specifically, a concurrent MMPR, which
consisted of a dominant interpretive phenomenological phase (i.e., “seek[ing] to elucidate or
make explicit our understanding of human behaviours and actions”; Allen & Jensen, 1990, p.
244) and a less-dominant postpositivist phase (i.e., PHEN+quan).
For the quantitative phase, the researchers determined the number of authors contained in
each article across all the journals. Also, we noted the genre of each article. For the
qualitative phase, the researchers interviewed several leading mixed methods researchers
using a semi-structured format. As recommended by Onwuegbuzie, Leech, and Collins
(2008), the interviewers underwent debriefing interviews themselves in order to promote
reflexivity.
2.3 Analysis
2.3.1 Quantitative Analyses
In the quantitative phase, we used both descriptive analyses and inferential analyses.
Specifically, our descriptive analyses involved the use of descriptive statistics to assess both
the proportion of empirical and non-empirical research articles, and the degree of
collaboration, which comprised a measure of central tendency (i.e., frequency counts) and
measures of distributional shape (i.e., skewness, kurtosis). Our inferential analyses involved
(a) a Fisher’s Exact Test to assess the relationship between the genre of articles published in a
select mixed methods research journal and degree of collaboration in these articles, and (b) a
nonparametric t test to examine the difference between the number of authors in empirical
research articles versus non-empirical research articles published in JMMR. Effect sizes were
reported for all statistically significant findings.
2.3.2 Qualitative Analyses
We used constant comparison analysis (i.e., Glaser, 1965) via QDA Miner Version 4.1
(Provalis Research, 2015), which is a computer-assisted mixed methods data analysis
software program that can be used to code, to annotate, to retrieve, and to analyze both
images and a variety of text-based file formats. Our main objective in conducting a constant
comparison analysis was to generate themes from the participants’ responses (Glaser, 1965).
91 www.macrothink.org/jei
Journal of Educational Issues
ISSN 2377-2263
2017, Vol. 3, No. 1
Using Constas’s (1992) typology, both the locus of theme and the naming of themes
development was investigative, stemming from the intellectual constructions of the
researchers. Also, the verification component of categorization was participative (Constas,
1992), stemming from the participant-researchers who verified each emergent theme. This
verification approach was accomplished a posteriori (Constas, 1992).
2.3.3 Mixed Analyses
Overall, both a concurrent mixed analysis and sequential mixed analysis (Onwuegbuzie &
Combs, 2010; Onwuegbuzie & Teddlie, 2003) were used. With respect to the concurrent
mixed analysis, the analysis of the (quantitative) bibliometric data occurred independently of
the analysis of the (qualitative) interview data. However, as part of the analysis of the
interview data, a sequential mixed analysis ensued. Specifically, after the qualitative coding
process, we transformed the qualitative themes that emerged into data that could be analyzed
descriptively (i.e., quantitizing; cf. Miles & Huberman, 1994; Onwuegbuzie & Teddlie, 2003;
Sandelowski, Voils, & Knafl, 2009; Tashakkori & Teddlie, 1998).
3. Results
3.1 Quantitative Phase
An analysis of the 163 articles published in JMMR from 2007 (its inception) to 2015 revealed
that nearly two thirds (i.e., 63.2%) of the articles represented empirical research. This
difference between these two genres of research articles was statistically significant (p
< .0005). The effect size, as measured by the difference in percentages, was large at 26.4%.
With regard to level of collaboration across these 9 years, the number of authors per
manuscript ranged from 1 to 10 (M = 2.71, SD = 1.68). Interestingly, 71.8% of the articles
involved more than one author—which yielded the degree of collaboration. Further, whereas
28.2% of articles involved one author, 23.3% involved two authors, 23.3% involved three
authors, 11.7% involved four authors, 7.4% involved five authors, 3.7% involved six authors,
1.2% involved seven authors, and 1.2% involved 10 authors—with the latter representing the
number of authors who co-wrote the current article.
Table 1 presents the publication rate by genre for each of the 9 full years of JMMR’s
existence. It can be seen from this table that the proportion of empirical research articles
exceeded the proportion of non-empirical research articles for 7 out of the 9 years. In 2010,
there was an equal number of empirical research articles and non-empirical research articles,
whereas in 2012, there was a higher proportion of non-empirical research articles (i.e.,
58.3%). Overall, a statistically significantly (χ2[1] = 9.95, p = .002, Fisher’s Exact Test = .002,
Cramer’s V = .25) higher degree of collaboration emerged for empirical research articles
(80.6%) than for non-empirical research articles (57.6%). Moreover, empirical research
articles were 1.4 times (95% confidence interval = 1.10, 1.78) more likely to involve multiple
authors than were non-empirical research articles.

92 www.macrothink.org/jei