oSoTL 2(2) Editorial:
Teaching research methods better? Or research methods for better teaching?
Jo Ferrie1, Catriona Forrest1 and Thees Spreckelsen1
1University of Glasgow, Glasgow, UK
This special issue invited papers relating either to the pedagogy of research practice, or how we can harness methods, methodologies, and research design to better evaluate our teaching.
In essence, the paper was derived from personal struggles to find a home for such work within a world-leading research university that was structured around disciplines and the teaching of substantive topics, and around research careers. The editing team for this special edition teach courses which sit alongside this disciplinary structure: teaching either research methods courses spanning a school made up of six disciplines, or across a college made up of five schools and many hundreds of students; or skills-based courses on journalism practice within subject or school-wide media courses. Each of the editors are on learning and teaching contracts despite all being actively involved in research. This is an example of neoliberal management decision-making, a decision that disappears our potential to be recognised as researchers because there is so much teaching for us to do. Within our institution there has been growing investment in research methods teaching teams, just as there has been growing investment and recognition of colleagues on the learning and teaching track. The investment is slow and often concentrated on early career scholars meaning there are leadership vacuums in the ‘middle’ of the institution. Our overarching university structures have created barriers to the building of communities of practice in research methods or learning and teaching.
This journal is something of a beacon and its ethos of critical development of scholarship and of scholars attracted our attention. Early conversations with the oSoTL editors Nathalie Tasler, Matthew Barr, and Vicki Dale, helped us consider what a useful contribution could be. We had a strong desire to expose how tough some things are and how welcome the solidarity found within communities of practice is.
Teaching students to develop their research practice is tough. The Scholarship of Teaching and Learning (SoTL) in this field is relatively new and has focused predominantly on the teaching of numeric data, an area also sometimes known as ‘quantitative methods’ and ‘computational social sciences.’ The difficulty of teaching in this field is fuelled both by the resistance shown by students: on maths anxiety (Williams et al., 2008); on fear of numbers (Scott Jones & Goldring, 2015); on anxiety and devaluing of courses that receive mixed student feedback (Scott Jones & Goldring, 2014); on the devaluing of high value, rather than enjoyable learning (Ryan et al., 2014); and on managers resisting adequate time to learn methods because they’re perceived as unpopular (Macinnes, 2014). Significant social science investment in Q-Step Centres across the UK (over £20million into 17 centres) demonstrated a different ‘way’ was possible. Significant outputs including writing on embedding methods in substantive learning (Buckley et al., 2015; Rosemberg et al., 2022) were produced by Q-Step teams, adding to a growing literature on pedagogy and methods. Most of the literature in the teaching of numeric data is published in journals which examine advances in methodologies rather than in dedicated SoTL journals, and so it can be difficult to find for those coming with pedagogic concerns.
Ralston (2019) argued that the separation of methods in the student’s learning experience creates an insurmountable gap for educators. By removing the learning from disciplinary homes into distinct methods courses which are often taught in pan-disciplinary spaces, the implied message to students is that methods aren’t relevant to their discipline. There is emerging literature on what makes methods teaching different, perhaps best captured by Nind and Lewthwaite (2018) who argued that educators need content knowledge grounded in disciplinary norms, knowledge of method and skills, as well as pedagogical expertise. Where educators are teaching across schools or colleges, content knowledge is particularly difficult to attain or demonstrate to students. Ferrie et al. (2022), in their review of data-driven skill gaps, flagged that few universities recognise and accommodate the additional labour and skill required to teach methods and pointed to other barriers to learning including large class sizes and the oppositional positioning of quantitative and qualitative approaches (see too, Ferrie and Spreckelsen, this issue). Their work identified gaps also in digital skills, needed by all disciplines including those highly associated with computational methods (Barone et al., 2017; Organization for Economic Co-operation & Development (OECD), 2020). Thus, methods educators are challenged by reluctant students, by difficult working environments, and by an area that is rapidly evolving: educators will always need to redevelop materials to future-proof the learning experience.
If time were to be made available for educators to redesign methods teaching materials, this could provide opportunities to engage with pedagogy of research methods, and this special issue aims to support this work. We welcomed papers from all disciplines or inter-disciplinary spaces which have innovated to aid student engagement.
In turn, in developing the rationale for this special issue, we welcomed scholarship around how we can better evaluate our pedagogical work. SoTL has a reputation in some quarters (drawing from our own, admittedly anecdotal experience) for ‘easy’ outputs at odds with the competitive nature of leading research journals. For discussion on this see too the contribution from McEwan et al. in this special issue. Journals dedicated to SoTL are increasingly becoming critical spaces, asking reviewers to focus particularly on the methodology of work submitted. A former Dean of Learning and Teaching at our institution described SoTL journals as increasingly being considered 4*, a borrowing of REF vernacular to say that the quality of some SoTL is world leading. Despite this, academic staff on the learning and teaching track are seldom given space or support to practice their scholarship. With workloads off the scale and a lack of a system to provide mentors, for many of us we do our writing unsupported and in our 'free' time. This has obvious gender and equality implications. Under such pressures we have tended to adopt a position of resilience, of just getting through each year. But we prefer to take a position of resistance whereby we take the time to showcase our practice, and our passion, and our strategies for advancing the learning experience. We take it upon ourselves to deliver the habitus we want, of critical, careful, intellectual scholarship that draws on collaboration. We are either fortunate or cursed; as being learning and teaching academics who have expertise in research methods, we have never considered a quick course evaluation as sufficient for scholarship. It’s not interesting enough to deliver a simple description of ‘what we did.’ In this special issue we have tried to delve more critically into the realities of teaching, of the legacies we deal with, of the structures that restrict our capacity to innovate and flourish. Our editors and authors have puzzled as to why and how things have come to be, and how and why we should seek change.
For learning and teaching colleagues who do not have expertise in knowledge production or who specialise in methods that are not well suited to evaluating pedagogy, scholarship is challenging. For example, scholars who are wedded to disciplines which are particularly numeric may require to learn about more qualitative approaches. And the reverse is true: learning statistical strategies to evaluate pedagogical impact can feel onerous and difficult to scholars used to working with text and words. This special issue encouraged colleagues and peers to submit work which critically and intellectually engaged with methodologies that support pedagogical evaluation. Collectively, these articles represent leadership in two areas: teaching research methods better, and improved use of methods in teaching.
There are some threads that weave in and out of these papers which were not anticipated. The paralysis that seems to pervade writing about teaching and learning is one. This may relate to the ‘invisibility’ of teaching excellence in higher education, or perhaps more accurately, the ‘disappearance’ of bad teaching. Consistently strong or weak student feedback has no consequence on who is valued or promoted by the institution. We ourselves resist making this part of our practice visible, rarely allowing colleagues into our teaching spaces. Our articles are peer reviewed, but our teaching is not. Because of this, making our practice visible feels like a hurdle to many. Also to blame for the paralysis may be the difficulty of and adopting new methodologies to critically evaluate and then improve our teaching and learning. Fortunately for readers, Honeychurch and Austen & Jones Devitt (this issue) offer practical solutions to getting into a pedagogical mindset or to become inspired into performing evaluation.
Related to this is the (self-)critical tone inherent in most articles and authors who are enthusiastic to learn, and to ‘do better’. Honeychurch’s solution of falling back on tried and tested methodologies is as practical as Austen & Jones Devitt’s focus on student outcomes is inspirational. In turn, the careful reflection in McEwan et al., from a team mostly made up of statisticians, shows how critical approaches can be adopted. This special issue begins with their relatively traditional scholarship paper which examines maths anxiety in a statistics course.
The contribution by McEwan et al. examines SoTL as a process, or a methodology. A way of systematically or methodically engaging with reflective practices which illuminate teaching and learning experiences for academic staff who are research elites but, perhaps, may have less time to invest in teaching skills. Reflexivity is practiced throughout the paper, which starts with an exercise in transparency as the authors state their profile allowing readers to ‘imagine’ their roles in the academy. This humanising device is used later in the paper to great effect. In turn, the paper asks the reader to, and carefully argues for the value of, engaging in reflexive practices. Thus, the paper acknowledges that scholarship is not purely shifting attention from a disciplinary-related research question to a class-related research question, but a more demanding shift in practitioner focus from evaluating what is out ‘there’ to evaluating themselves and the learning environments they create. Acknowledging the growth in structures to support this transition, such as the UK Professional Standards Framework (UK PSF), the paper considers the shift from the perspective of those working in science broadly and statistics specifically. Inclusion in the authoring team of a colleague who works in academic development speaks to the communities required to help achieve this transition to SoTL thinking and doing. A key strength of this article lies in the fundamental approach taken by the authors as teachers: in reflecting on their practices, they start by actively considering rather than passively assuming who their students are, and whether their motivations to study impact on their learning capabilities. Further the author’s honest account of how and what they have learned exposes how difficult the transition from research to scholarship is.
Taking a step back from how students react in the classroom, our second paper by Ferrie & Spreckelsen considers how we can help give students insight into our pedagogical approach using the imagery of trees and a dirty greenhouse. The paper argues that disciplinary learning and teaching is markedly different from teaching quantitative methods (sometimes known as social statistics) and qualitative methods. If we don’t help students acknowledge this from the start, then they will utilise strategies that have worked for their disciplinary courses, ultimately making their learning of methods harder. The more pedagogical approaches can be demystified for students, the less likely they are to resist the learning, even if they still feel anxious about it. While the analogies work well with the traditional separation of quantitative and qualitative courses (often taught by different educators), the paper argues that a mixed methods approach would trigger fewer problems for students resistant to learning methods. Like this editorial, the paper points to significant structural barriers that impact on methods courses that make the ‘job’ harder for lecturers. Barriers include relatively large student numbers, relatively small teaching teams, relatively earlier career, and relatively large overall workloads compared with colleagues hired to teach disciplinary courses. In calling for the use of these analogies to help students position their learning, the paper is signifying the importance of time to engineer the pedagogy of our courses. To think beyond what content is required, what skills must be learned or what intended learning outcomes should be used, and to imagine how it will be learned.
From this consideration of research methods teachers’ careful focus on pedagogy, the third paper by Honeychurch shifts our attention to starting our scholarship, utilising methods that we are familiar with. In the course of becoming a scholar of learning and teaching, Honeychurch argues, retaining our practiced and rehearsed methods and methodologies helps us establish rigour in our work: so, while the topic may be new, our practice can feel safe. The bricolage approach is also a strong case for establishing practices that encourage development and skill growth as one critically considers not just what we want to learn about ‘our’ teaching or ‘their’ learning experience but whether we have the methodological toolkit to answer that question. It highlights the choices that we have as scholars, much in the same way that Ferrie and Spreckelsen championed teaching devices that encouraged students to awaken to the research choices they have. Honeychurch’s paper is a significant contribution to SoTL, with their encouragement to scholars to consider how we collectively resist the diminishing of our work by some disciplinary researchers. By utilising a range of methodologies, and particularly by utilising those we know achieve rigour, as scholars of learning and teaching we will undoubtedly produce work commensurate with the strongest outputs from our universities.
Karlsson delivers a praxis paper which gives us a rich insight into their experience of bridging between their (development of) pedagogy and making this visible through scholarship. Early on in their paper, Karlsson talks about management decisions to assign early career scholars to research methods teaching even though they did not have expertise in this nor in the disciplinary field in which the course was situated. The small number of people working in research methods created a leadership vacuum and there was little support or mentorship from senior colleagues, which reinforced the absence of a strong literature in teaching research methods to help them deliver excellence. Further, this course unlike others in the programme had to be pitched for both beginners and for those with existing knowledge and skills who would be expecting to engage with advanced material. Karlsson’s response, evaluated in their paper, was to deliver a learning space where the students acted as participants as well as researchers, and where they as teacher acted as research manager as well as the focus of evaluations. This careful play with power inequalities gave students critical awareness of ethics and how they play out in the ‘field’. The paper adds value to this special edition, engaging with how students become the centre of student-centred learning and how this interacts with learning outcomes.
Austen and Jones-Devitt’s contribution makes a critical argument about engagement in scholarship and pedagogical innovation needing to lead to improvements in student outcomes. The opening to their conclusion says it all, “The world of evaluation research often collides with, instead of complementing, pedagogic practice” (Austen & Jones-Devitt, this issue, pp. 55-66). Drawing on experiences working with Quality Assurance Agency (QAA) Scotland’s Enhancement Themes as a trigger for greater valuing of evaluation practices, the authors consider how colleagues might become ‘liberated’ to deliver evaluation. Austen and Jones-Devitt’s careful work highlights that an evaluation is not just an exercise in measurement (has the course worked or not worked), but of reflexive consideration of why something has or has not worked and how that learning can inform decision-making which ultimately enhances the student experience and in turn, enhances student outcomes. Thus, of the papers in the special issue, it is here where we consider afresh why pedagogy matters and firmly places students in the centre of what we should be doing (well). Specifically, the paper outlines the ‘Universal Evaluation Framework’, and uses four stages: Diagnose; Plan; Gather and Reflect, as action zones. The paper traces journeys through the zones that themselves encourage a reflexive engagement. We encourage readers interested in Scottish Higher Education to read this paper for an excellent overview of the Scottish Funding Council’s investment in universities and the impact is has had on student outcomes.
Humphrey’s contribution focuses very much on the early-career experience, of someone who has recently emerged from methods courses and is concerned with making meaning, while facing numerous barriers to being and doing. It explores otherness and marginal-identities as critical lenses to learn more about normative structures that infiltrate knowledge production, in this case, qualitative research. Humphrey’s voice is rich and emotional as they chart their navigation through research interactions made challenging through power differentials, through insider/outsider complexities and through the content itself being difficult. Using poetry as a communication device, Humphrey reminds us that our academic practice is emotional practice. The paper encourages educators, particularly those who teach research methods, to create learning spaces that allow students to imagine their research fieldwork not in terms of following a recipe (applying an analytical framework for example) but as embodied and complicated. The paper is another call to consider the student at the centre of our teaching, rather than to structure learning around content. We can do better as educators if we ask, not what do students need to know, but who do they need to ‘be’? This paper reminds us that knowledge production is almost always a transformational experience. The disruption implied by this can become harmful if the researcher, particularly those researching for the first time, is unaware that this transformation is part of the process and not a form of failure.
The final paper by Forrest and Ferrie examines the future of research methods training in the UK. The paper has a strong social science focus as it critically assesses the three major pieces of evidence that have informed the Economic and Social Research Council’s (ESRC) 2022 approach to research methods training for post-graduate students (also known in the UK as PhD students and doctoral candidates). The paper provides an overview of the most significant changes heralded in the most recent ESRC guidelines for post-graduate training and considers how well they might work for students who are not pursuing a PhD to launch an academic career, using the field of journalism and the skills training delivered through several media courses as an exemplar. The future direction encompasses many of the themes covered in the other papers in the special issue. We need to invest time in student-centred pedagogies which help learners think, and then do.
The seven papers collectively carry an urgency. They exemplify a community who are desperate to do better and be better, striving for discernible impact on their students’ learning opportunities. Whichever field we come from we are committed to our students flourishing. Equally we are committed to our colleagues flourishing. As those working in academic or educational development or academics on learning and teaching tracks push forward excellence in scholarship, and as we improve the learning experience, it is hoped that university leadership acknowledges and invests in these vital communities of practice.
Barone, L., Williams, J. U., & Micklos, D. (2017). Unmet needs for analyzing biological big data: A survey of 704 NSF principal investigators. PLOS Computational Biology, 13(11). https://doi.org/10.1371/journal.pcbi.1005755
Buckley, J., Brown, M., Thomson, S., Olsen, W., & Carter, J. (2015). Embedding quantitative skills in the social science curriculum: case studies from Manchester. International Journal of Social Research Methodology, 18(5), 495-510. https://doi.org/10.1080/13645579.2015.1062624
Ferrie, J., Wain, M., Gallacher, S., Brown, E., Allinson, R., Kolarz, P., MacInnese, J., Sutinen, L., & Cimatti, R. (2022). Scoping the skills needs in the social sciences to support data driven research. Technopolis Group. https://www.ukri.org/wp-content/uploads/2022/10/ESRC-171022-ScopingTheSkillsNeedsInTheSocialSciencesToSupportDataDrivenResearch.pdf
MacInnes, J. (2014). Teaching quantitative methods. Enhancing Learning in the Social Sciences, 6(2), 1-5. https://doi.org/10.11120/elss.2014.00038
Nind, M., & Lewthwaite, S. (2018). Methods that teach: Developing pedagogic research methods, developing pedagogy. International Journal of Research & Method in Education, 41(4), 398-410. https://doi.org/10.1080/1743727X.2018.1427057
OECD. (2020). Building digital workforce capacity and skills for data-intensive Science. OECD Science, Technology and Industry Policy Papers, No. 90, OECD Publishing, Paris, https://doi.org/10.1787/e08aa3bb-en
Ralston, K. (2019). ‘Sociologists Shouldn’t Have to Study Statistics’: Epistemology and anxiety of statistics in sociology students. Sociological Research Online, 25(2), 219-235. https://doi.org/10.1177/1360780419888927
Rosemberg, C., Allison, R., De Scalzi, M., Bryan, B., Farla, K., Dobson, C., Cimatti, R., Wain, M., & Javorka, Z. (2022). Evaluation of the Q-Step programme. Technopolis Group. https://www.technopolis-group.com/wp-content/uploads/2022/05/Q-Step-evaluation-report-May-2022.pdf
Ryan, M., Saunders, C., Rainsford, E., & Thompson, E. (2014). Improving research methods teaching and learning in politics and international relations. Politics, 34, 85-97. https://doi.org/10.1111/1467-9256.12020
Scott Jones, J., & Goldring, J. E. (2014). Skills in mathematics and statistics in sociology and tackling transition. Advance HE. https://www.advance-he.ac.uk/knowledge-hub/skills-mathematics-and-statistics-sociology-and-tackling-transition
Scott Jones, J., & Goldring, J. E. (2015). ‘I’m not a quants person’; Key strategies in building competence and confidence in staff who teach quantitative research methods. International Journal of Social Research Methodology, 18, 479-494. https://doi.org/10.1080/13645579.2015.1062623
Williams, M., Payne, G., Hodgkinson, L., & Poade, D. (2008). Does British sociology count? Sociology students’ attitudes toward quantitative methods. Sociology, 42, 1003-1021. https://doi.org/10.1177/0038038508094576