What counts as ‘Good’ Qualitative Accounting Research? Researchers’ Perspectives on Assessing and Proving Research Quality

Photo by Shutterstock

What counts as ‘Good’ Qualitative Accounting Research? The experiences of authors and reviewers

What makes a good qualitative accounting paper?

There is a lot of “preaching” about what is to be expected from a good accounting paper. Handbooks, articles, workshops, and seminars have proliferated on how to write and defend good qualitative research. They have suggested a plurality of criteria for assessing research quality, more or less different from those used in the quantitative field. However, surprisingly, we know much less about how those indications are “practiced” by reviewers and authors when they assess, and write research; the practical difficulties qualitative researchers experience when having to convince the readers of the quality of their research; and how reviewers assess qualitative research quality in their everyday practices.

To address these issues, this research conducted interviews with 18 accounting scholars from diverse backgrounds, focusing on academics who are not yet professors to capture often overlooked and less represented perspectives.

Through thematic analysis, practical and implicit methods employed by researchers to demonstrate and assess research quality have been identified, particularly in terms of contribution, consistency, confidence, and generalizability. The interviewees highlighted the inherent ambiguity surrounding what constitutes good research in the qualitative accounting community, in contrast to the perceived higher clarity on criteria in the quantitative accounting community. They raised critical issues related to the assessment criteria, highlighting possible flaws to address when developing and writing about their research.

Regarding contribution, researchers emphasized the importance of positioning papers with respect to theories, literature, and practice/policy. They expressed concerns about excessive engagement with theory driven by a desire to please senior academics and editors, which may hinder innovation and theorization. On the other hand, limited engagement with literature was seen as creating blind spots and impeding contribution to interdisciplinary debates. Moreover, many of them noted that research often fails to adequately contribute to practice and policy due to an excessive focus on theoretical development at the expense of engagement with concrete societal challenges.

Regarding consistency, respondents highlighted the need for research to maintain coherence throughout the entire research process, evident in the writing of the paper and the connections between its elements. They emphasized the consideration of tradeoffs between saturation and the role of counter-evidence or counter-narratives in research, stressing the importance of proper representation of voices and the crystallization of results. Contextualization was also deemed crucial in making these tradeoffs clearer to audiences.

In terms of confidence, scholars emphasized that qualitative researchers should not mimic quantitative research but embrace the serendipitous, messy, and unpredictable nature of qualitative research. They encouraged researchers to openly discuss the twists and turns in their research journey, including changes in research questions, findings that prompt project rethinking, and the inevitable limitations of any study.

Generalization emerged as a particularly controversial and challenging aspect. Analytical generalization was perceived as difficult to achieve, contradicting psychological and public policy literature. However, qualitative evidence was seen as particularly influential in shaping audiences' understanding of phenomena and decisions due to the vividness effect.

The study participants suggested that the multiplicity and vagueness of research evaluation criteria reflect its processual, tacit, and dialogical nature. Research assessment is not a static exercise but a socially shaped process highly dependent on the audience's expectations for the paper. Reviewers tacitly discern whether a paper is "good" and evaluate it based on their expectations regarding contribution, consistency, and confidence. This democratic nature of the process is potentially conducive positive impacts on the development of good research. Yet, it also risks promoting a ceremonial approach to paper development, whereby adhering to certain theoretical or methodological practices preferred by specific communities or "tribes" is seen as a shortcut to signal research quality and facilitating publication.

These findings point to several future avenues for intervention for authors, reviewers, and editors.

Authors should increase their awareness of how psychological elements shape research processes, including the roles of cognitive biases, emotions, and "bullshitting" in their work. They should prioritize innovative, less explored theories rather than relying on established ones for ease of acceptance. Additionally, authors should consider engaging more with practice and the broader literature on their themes of interest, even if it spans different disciplines and methods. The focus should be on offering a concrete contribution rather than generating research gaps by narrowing the boundaries of the literature considered.

Reviewers and editors should reflect on whether they are effectively advancing the field or inadvertently discouraging authors from providing an honest account of their research process. They should support authors in fulfilling their ambitions rather than forcing them to conform to a preferred research question or oversimplifying messages from their papers.

Overall, these findings suggest the need to continue debating how we assess the quality of qualitative research in our everyday activities and reflect on how we can promote acceptance and openness to pluralism, in our communities, as well as in data collection, analysis, in the theorizing, and in connecting epistemology and methodology.

Read the full article

Authors at the Department of Management

Ileana Steccolini, Full Professor

Academic disciplines: Accounting and Public Management

Teaching areas: Accounting and Public Management

Research fields:  Public sector accounting and accountability, public management

Breve biografia max 400 caratteri spazi inclusi

Ileana Steccolini is a Professor of Accounting and Public management. She is an Editor of Financial Accountability & Management, President of the International Research Society of Public Management, Founder of the Public Service Accounting and Accountability Group, Chair of the EIASM public sector conference, and Chair of the Standing Scientific Committee of the European Accounting Association.