From statistics anxiety to SoTL:
How a scholarly enquiry led to professional growth
Michael P. McEwan1, Eilidh Jack1, Craig Alexander1 and Mitchum Bock1
1University of Glasgow, Glasgow, UK
In higher education, academic staff are experts: disciplinary experts. They are often required to have significant research expertise, research degrees and a long track record of formal and experiential learning in their chosen specialist subject area. Then, after a long research apprenticeship, they finally get a teaching role. The UK Professional Standards Framework (UKPSF), Advance HE and even, to some extent, the Teaching Excellence Framework, all exist to professionalise teaching as a legitimate career in higher education but still, on appointment, most early career lecturers are subject experts but novice teachers.
There is a growing professional requirement for higher education academic staff to adopt a scholarly approach to learning and teaching practice, to gain professional recognition as teachers, and to undertake scholarship of teaching and learning as part of ongoing and continuing professional development. Whilst taking a scholarly approach to practice is an expectation of the UKPSF, and scholarly teaching practice is a characteristic of expertise in teaching, what role does the Scholarship of Teaching and Learning (SoTL) play in professional development?
This article explores how a significant first foray into SoTL for early career statistics lecturers has resulted in unexpected learning for those who engaged in it. A study on how statistics anxiety may differ amongst different cohorts of students became a vehicle for deep learning about teaching, about students, about course design and pedagogy, and about SoTL. This led to a ‘critical awakening’ as scholarly teachers through the process of data collection, analysis, and reflection.
The aim of this article is to showcase the value not only of pedagogical evaluation as a scholarly output and source of publication, but also that the process of engaging in SoTL – whether it results in successful outputs or not – is of extreme value in becoming an expert scholarly teacher in higher education.
scholarship, SoTL, statistics anxiety, professional development
The Scholarship of Teaching and Learning (SoTL) is growing in higher education, particularly with the rise of ‘teaching-focused’ academics (HESA, 2021). There is an increasing professional requirement for higher education academic staff to adopt a scholarly approach to learning and teaching practice (Advance HE, 2011), to gain professional recognition as teachers (Browne, 2010), and to undertake SoTL as part of ongoing and continuing professional development (Chalmers, 2011; Hutchings et al, 2011). But engaging in SoTL is difficult. Newly recruited academic staff are experts: disciplinary experts. They have significant disciplinary expertise, research expertise, research degrees and a long track record of formal and experiential learning in their chosen specialist subject area. Often, after a long research apprenticeship, they finally get a teaching role. The UK Professional Standards Framework (UKPSF), Advance HE and even the Teaching Excellence Framework, all exist to professionalise teaching as a legitimate career in higher education but still, on appointment, most early career lecturers are subject experts but are novice teachers and novices of SoTL. It is not until a first academic appointment that early career academics typically start learning about teaching (Smith, 2010) and the language of SoTL, never mind its existence, is alien not just for new academic staff but also for staff from divergent disciplines, such as Science, Technology, Engineering and Mathematics (Kelly, Nesbit & Oliver, 2012).
Whilst early career support for SoTL can be effective in developing understanding of and engagement in SoTL (McEwan, 2022), it is also inconsistent (Tierney, 2016). However, is engaging in SoTL itself an example of practice-based learning and development? What is the role of SoTL in an academic’s initial professional development as a teacher in higher education? Is it a language in which to gain fluency? Is it a skillset to master? Is it a methodology of enquiry to gain experience of? Is SoTL a liminal academic practice to be negotiated?
In order to explore the value in engaging with SoTL, this paper is an account of the reflections on the experiences of four academics (three disciplinary experts in statistics and an academic developer) as they engage in a collaborative SoTL project about ‘statistics anxiety’. An academic developer typically works with academics to support the development of teaching, learning and assessment practice and, in this instance, the academic developer also has a leadership role in promoting and supporting development of SoTL practice across the institution. For the three statisticians, this collaborative SoTL project represents new ground away from their experience in disciplinary research. Accordingly, this paper explores how a significant first foray into SoTL for early career statistics lecturers has resulted in unexpected learning for those who engaged in it.
To meet this aim, this paper begins by setting the scene and briefly reviewing the initial ‘statistics anxiety’ SoTL project. This is followed by a section outlining the methodology for generating our reflections, followed by a more substantial section presenting and analysing our reflections according to Glassick’s framework for the evaluation of scholarship (Glassick et al., 1997). We then close this paper with some overarching reflections on the process of engaging in that first SoTL project in order to highlight our overall experiences, learning and professional growth as a result of that undertaking.
Many students find statistics courses challenging, and ‘statistics anxiety’ has been identified as a significant factor impacting student learning in statistics courses in a variety of contexts and subjects (Chew and Dillon, 2014). This is of particular relevance when students are studying in interdisciplinary or ‘non-specialist’ settings (Birenbaum & Eylath, 1994; Hanna et al, 2008; Onwuegbuzie & Wilson, 2003). Whilst statistics anxiety is generally associated with poorer student outcomes in statistics courses, it is not the only factor (Sandoz et al., 2017). Accordingly, understanding the factors that impact on the levels of statistics anxiety is an important element of statistics education and is vital knowledge for those that teach statistics, particularly in interdisciplinary or ‘conversion’ style courses and programmes.
In order to explore these factors, the authors of the current article devised and implemented an educational enquiry focusing on some key questions relevant to statistics anxiety. Statistics anxiety has an impact on student learning and performance in statistics courses, but is that impact dependent on who the students are? Koemadij and Ailey (2018) note that specialist statistics students are more likely to harbour long-term ambitions related to statistics compared to non-specialists – does this mean specialist students are less anxious? Gordon (2004) notes that students who are required to study a compulsory statistics course demonstrate less willingness to study and are more likely to struggle to find authenticity and applicability in their statistics learning. By contrast, are interdisciplinary students (i.e., non-specialists) more anxious? Does this mean students who voluntarily take statistics courses are also less anxious? Accordingly, it was of interest to explore how ‘specialism’ and ‘compulsion’ related to statistics anxiety. This is what Bock et al. (2020) and Alexander et al. (2022) explored and report on, and that original SoTL enquiry is the focus of our reflections here.
In their book ‘Scholarship Assessed: evaluation of the professoriate’, Glassick, Huber and Maeroff (1997) present a framework for evaluating SoTL. Glassick’s framework demands that SoTL has clear goals, adequate preparation, appropriate methods, significant results, effective presentation, and a reflective critique. This framework is the foundation of an initial course on SoTL that many new academics at the University of Glasgow complete as part of their early career academic development on the Postgraduate Certificate in Academic Practice (PGCAP) and so most early career staff at the institution are familiar with Glassick’s framework. Whilst other frameworks for supporting SoTL also exist, such as Felten’s five principles of good practice in SoTL (Felten, 2013), Glassick’s framework was chosen as the tool to explore SoTL partly due to its familiarity for all staff and partly for its final principle of ‘reflective critique’. The main purpose of this article is to reflect on engagement of SoTL and so to evaluate this budding professoriate, this article adopts critical reflection on each element of Glassick’s framework as an enquiry method (Brookfield, 2017) to critically reflect on the experiences of SoTL undertaken by the four authors of Bock et al. (2020) and Alexander et al. (2022).
In order to critically reflect on the experiences of engaging in SoTL, an interview style approach is adopted whereby the academic developer (AD) interrogates the reasoning and reflections of the statisticians (S1, S2, S3), deconstructing and reconstructing the experiences and reflections therein, in order to interpret and untangle where the benefits and learning from engaging in the original ‘statistics anxiety’ SoTL project have come about. Interview questions were sent electronically to S1, S2 and S3 individually, and responses were independently gathered, thematically analysed and, where appropriate, interpreted and presented by the AD. Glassick et al.’s (1997) framework was adopted to structure these questions around exploring goals, preparation, methods, results, and presentation in turn. The AD’s reflective critique of the responses uses critical reflection as an analysis tool and is presented within this article. This presentation was then checked for accuracy and meaning by the statisticians, and it is this ‘academic developer’s view’ that serves as the critically reflective critique thus completing the evaluation model of ‘scholarship assessed’.
This approach is necessarily interpretivist and inductive in nature and, as such, the interviewer (AD) heavily influences the direction of the questions. Moreover, this approach focuses on the experiences of a single SoTL investigation by a small group of scholars – it is highly contextual and highly limited in scope. Accordingly, this study does not claim to be generalisable but rather it highlights possible interpretations of early career lecturers’ experiences of a first SoTL project.
With regards to the interviewees, S1 and S2 are early career lecturers on the ‘Learning, Teaching and Scholarship’ (LTS) track: an academic career track with a focus on teaching and scholarship. S3 is a Senior Lecturer on the LTS track with a role involving aspects of leadership in learning and teaching in the statistics subject area. The AD is also a lecturer on the LTS track but with a leadership role in academic development alongside a role supporting early career academic staff to gain professional qualification and professional recognition in learning, teaching, and scholarship.
The interviewees were asked a series of questions constructed around Glassick’s framework (Glassick et al., 1997) and the responses, and associated analysis, to these questions form the results, presented as a reflective critique. The structure of the paper hereon follows Glassick’s framework, and so are presented as reflections on goals, methods, results of the original SoTL project, and presentation of that original work in turn. This is followed by some concluding questions and reflections on what each contributor wished they knew beforehand and how they feel about SoTL having now engaged in an enquiry in an attempt to shed light on potential learning through engagement and experience in SoTL.
Glassick’s framework outlines the need for clear goals in a SoTL enquiry. Goals that are well articulated, defined, precise, and answerable but in addition goals that identify a gap in existing knowledge in the field. In this instance, the stated goals of the statistics anxiety project reported in Bock et al. (2020) and Alexander et al. (2022) were to explore the extent that statistics anxiety impacts on student learning when comparing specialist and non-specialist cohorts and voluntary or non-voluntary course requirements.
These goals focus on questions about students: who they are and what their motivations to study statistics are. But where did these questions come from? Were their goals associated with being a teacher, or a scholar? Accordingly, the authors were asked to reflect on their goals for the project with a steer that these goals could be focused on students or be more personal.
S1: The main goal was probably to quantify the level of statistics anxiety amongst our students to ensure we were providing sufficient support. Secondary to that we were looking to test our assumptions about differing levels of statistics anxiety amongst different cohorts of students, with our initial thinking that ‘specialist’ students may show lower levels of statistics anxiety compared to ‘non-specialist’ students. I guess we were looking to see whether the very different ways we teach statistics within our courses aimed at different sets of students was valid and justified.
Personally, a goal for me was to get some experience undertaking a SoTL project. Having just completed my [statistics] PhD and started as a lecturer on the LTS track, I was keen to get involved in something that could help in my development on that track.
Looking back, I don’t think the goals were very clear. We probably all had an idea of what we thought we might get out of a study like this, but in all honesty, we probably rushed into the project.
S2: The goal of our project was to investigate two things. Firstly, to investigate how prevalent statistics anxiety was within learners in our introductory statistics courses and secondly, did the levels of statistics anxiety differ depending on the cohort of student?
My motivation to investigate this study was twofold. Through my own teaching experience and to some extent, my previous learning experience, I found that students tend to disengage with course material when certain elements are introduced, such as probability and coding. I began to wonder whether this was due to ‘statistical anxiety’ causing a barrier to students learning. I wanted to ensure that the course I delivered was engaging for the students and attempted to deliver materials in such a way to minimise the effects of statistics anxiety, which contributes towards disengagement.
Reflecting on the goals after completing the work, I feel these were clear if somewhat broad in focus. Our definition on differing cohorts did not entirely match with what we found in the analysis, and this should have had some more consideration prior to beginning the investigation.
S3: Having taught introductory statistics courses for over 10 years and having recently taken the role of Head of Level 1 Statistics, I was keen to initiate a review of the design and delivery of the introductory statistics courses in our school. The primary goal [of the study] was to inform the review. A secondary goal was to build on earlier work in this area within the school which would hopefully lead to sharing the results more widely via publications.
On a personal level, I hope this would develop evidence of ‘scholarship’ which could enhance [professional and career development].
To an early career teacher, the questions around anxiety and what factors impact on anxiety are vitally important – they focus on ‘who the student is’. Kugel (1993) and Biggs and Tang (2011) highlight that the focus of a relatively early career teacher starts on what the teacher does and then who the students are; but importantly not (yet) on what students are doing in their learning. Therefore, it follows that an important consideration for early career teachers relates to student characteristics that impact on learning. What characteristics can teachers cater for in their approach or, at the very least, what characteristics can teachers know more about to ‘be prepared’? Being able to better understand your students as a cohort, as categorised, helps early career teachers make the educational transition from ‘self’ to ‘student’.
Kugel (1993) also reflects on the intermediate ‘subject’ stage of development, where early career teachers focus strongly on the subject that they teach, learning evermore about the subject and its depth, ways to lay it out, deliver it, structure it that make it more digestible, more exciting, more enticing. Anxiety is clearly a barrier to learning in any teacher’s eyes, and so understanding anxiety, its causes and its impact is a significant element in making the educational transition beyond ‘subject’ to ‘student’.
However, an important theme is also clear from the reflections of the statisticians: a chance to engage in SoTL. The statisticians are all LTS lecturers with an expectation of engaging in scholarship and so an ‘in’ to undertaking an educational enquiry provides a useful tool to begin this journey from subject specialist to educational specialist. What is also clear, however, is that all had multiple goals for the project. To this extent, were the goals truly clear, truly shared? Perhaps not. What is clear, however, is that an opportunity to engage in SoTL was perceived as valuable and developmental both personally and professionally.
Glassick’s framework discusses the need for adequate preparation as part of the SoTL process. In this instance, ‘adequate preparation’ relates to knowledge of existing scholarship and research around statistics anxiety with the aim of establishing a ‘conceptual framework’ for the enquiry. To this end, the authors were asked to what extent they were prepared for the enquiry in terms of knowledge about statistics anxiety alongside their preparation to undertake any scholarly enquiry.
S1: Not as prepared as I would have liked but […] it felt like a bit of a rush to get something out at the start of the semester, otherwise we would have had to wait another year. At that stage I hadn’t completed any PGCAP courses and had no experience in how to undertake a scholarship project. We did base our study on a master’s project supervised by S3, and we did try to engage in some of the stats anxiety literature, but I wouldn’t say I was overly knowledgeable in the field stats anxiety.
S2: Prior to the project, I had discussed with my colleagues regarding potential research areas and one proposition was to extend upon an earlier project undertaken in the department (which one member co-supervised on) that looked at statistics anxiety for a specific class. Our approach was to utilise a similar survey tool to this work [and we believed that the previous work had] spent a lot of time researching appropriate metrics. After the analysis, I do wish I had known more on developed survey tools on statistics anxiety as the tool we used did not consider computational effects on stats anxiety.
S3: I relied heavily on the work that had been done before in the school and on reflection would have liked to have dedicated more time to gaining my own insights into the broader area of ‘statistics anxiety’.
I joined the project team a little later, and one area I thought that I could contribute to was in relation to a conceptual framework and literature review on statistics anxiety. It seems odd to say this now, especially as methods had been implemented and data collected and analysed by this stage, but the literature review felt a little like a ‘retrofit’. It was clear that the team understood statistics anxiety as a body of work, but none of us knew the field with considerable depth. Collection and analysis of data appeared to be the primary focus at this early stage.
To analyse this, it is useful to consider a major part of my own academic development role: teaching a course about the design of SoTL enquiries. I use Glassick’s framework extensively in this role, mainly because it explicitly foregrounds ‘clear goals’ and ‘adequate preparation’. However, in my experience (over 10 years) of supporting SoTL for early career academics through this course, I often see the choice of enquiry methods driving a project’s design instead of goals and preparation. For example, my students often adopt ‘familiar’ methods rather than ‘appropriate’ methods (similar to findings outlined in Hubball, Clarke and Poole, 2010). This elevation of methods to the forefront of SoTL design can relegate ‘goals’ and ‘preparation’ to a secondary, rather than primary, consideration. As a result, SoTL projects on my course are often initially designed ‘backwards’ with goals being manipulated to ‘fit’ into the choice of methods, often without sufficient knowledge of the field.
I feel it is fair to say that happened here also. On reflection, all my co-authors, and I, would have spent more time on preparation, reading relevant literature, critiquing certain approaches in similar studies. By doing this a greater depth of knowledge of the field would have developed, alongside a greater appreciation for the potential tools and, importantly, for the potential goals. This is tangible evidence of the importance of adequate preparation for all SoTL practitioners (SoTLers), but especially those new to the field.
The project reported in Bock et al. (2020) and Alexander et al. (2022) used a survey adapted from the statistics anxiety rating scale (STARS), initially developed by Cruise et al. (1985). The adapted survey used only the first three subscales of STARS, as proposed by Chew and Dillon (2014), to measure students’ anxiety about statistics. Qualitative data on statistics anxiety was also collected using three open-ended questions as part of the survey, and additional questions also collected data that explored self-efficacy and attitude towards statistics. The enquiry drew data from students involved in three introductory statistics courses: one was a first year ‘voluntary’ and ‘specialist’ statistics course, another was a first year ‘voluntary’ and ‘non-specialist’ statistics course, and the third course was a ‘compulsory’ and ‘non-specialist’ statistics course for third year students studying engineering.
To reflect on the methods of the enquiry the authors were asked how they felt now about the methods used in the study. Why was this design used? Did they understand the methods, and do they feel, on reflection, that they were the correct choice? On reflection, how did the authors feel about the instruments and the way in which they were used?
S1: I feel the methods were chosen more out of ease than specifically designed to align with our project goals. As much as I was happy in my understanding of the methods and we felt it was important to use a well-known validated survey for our first attempt at a scholarship project, we were probably limited in our findings based on the methods we used. I feel like we learned more in the open responses (which we designed ourselves) than in the outcomes of the STARS questions. We also felt that the 51 items in STARS, along with questions about self-efficacy, was too long and so we decided to remove some in the hope that we would get a better response rate. In hindsight, I think we should have spent more time on the design of the study, even if this delayed when we could have sent out the survey.
S2: [Since] we were looking to see if statistics anxiety was more prevalent within the non-specialist cohort, and to assess the level of statistics anxiety amongst learners […] [we] designed the study to look at three different cohorts, to gather as comparative a sample as possible to match up to our initial hypotheses.
One area I feel the survey did not cover was computational elements of statistics such as programming, which I feel also contribute to stats anxiety. We did incorporate some follow-on questions not connected to the STARS survey to cover these areas, but they did not naturally integrate into the quantitative measure of anxiety used in STARS. In hindsight, I feel some more in-depth qualitative analysis may have provided additional evidence, e.g., conducting focus groups.
S3: The design of the study itself followed the approach taken in a previous study in the school […]. We felt it would be useful to have another time point to compare our results with (in reality this didn’t prove to be a particularly fruitful aspect of the study). In particular, I would have been more critical of the measurement tools (e.g., STARS, General Self-Efficacy Scale) we used and would have considered alternatives.
As mentioned earlier, being able to better understand your students as a cohort, as categorised, helps early career teachers make the educational transition from ‘self’ to ‘student’ – ‘who are my students’ is an important question (Biggs and Tang, 2011; Ramsden, 2003). The overall sampling design for the study was essentially a convenience sample, both in the sense of a sampling technique and in terms of it being convenient to explore since all three authors taught, and led teaching, with those cohorts. On reflection, the sampling and thus the participant recruitment was sound – that aspect of the design met the project goals and had potential to deliver meaningful results. Interestingly, none of the authors felt there was an issue with the sampling – but then all are experienced researchers, with a numerate background and know how to design a comparative study! It is within their comfort zone.
However, the instruments and the way they were identified and ultimately adapted and used is a point of reflection. All felt more confident using a validated tool, despite modifying that validated tool to fit the specific goals of the enquiry, thus affecting its validity. There is significant doubt expressed about whether the correct tool was used in the first place as a basis for the data collection, but this was not identified as a doubt until after the data collection and analysis (and, indeed the write-up) was completed.
This suggests several things: that the previously discussed doubts around ‘adequate preparation’ also have an impact on enquiry design; that an over-confidence in using a validated tool may exist when moving outside your comfort zone (again related to adequate preparation); and that as a result of doing this enquiry the authors have undergone a critical awakening in terms of SoTL methods! Indeed, Webb (2016) notes that conceptions of what is valid research (method) and epistemological and ontological shifts are essentially threshold concepts within SoTL and so negotiating them involves significant challenge. That liminality in approaches to research, scholarship and validity as threshold concepts further stresses that there should be considerable time spent on researching possible designs before settling on a chosen design – returning to the importance of adequate preparation. This is where the interdisciplinarity comes into effect in SoTL – growing a working knowledge and confidence in multiple research methodologies is vital for the developing SoTLer. That, and the specific specialist knowledge around the enquiry area: the adequate preparation.
Glassick’s framework suggests that good SoTL should have meaningful results whereby the results of an enquiry support the scholar’s ideas or hypothesis and necessitate a change or impact on practice that enhances student learning.
The project reported in Bock et al. (2020) and Alexander et al. (2022) identified several statistically significant results and several meaningful results. Statistically significant results included females enduring higher anxiety scores compared to males, and Chinese learners enduring lower anxiety rates compared to British and Irish learners. Also of statistical significance was that statistics anxiety negatively correlated with exam performance (though other factors also played a role). In contrast, meaningful results further included a finding that student demographics could not be so easily categorised as ‘specialist’ and ‘non-specialist’. In fact, 42% of students in a course intended for specialists were not specialists at all.
This need for significant, or rather, meaningful results, drove a question to the authors: ‘What was the most significant result of doing this project for you?’
S1: The most significant result for me was the fact that the different cohorts of students did not show different levels of statistics anxiety. This did surprise me. I also found the responses to the open questions particularly insightful as it allowed us to identify common areas which students feel anxious about.
S2: For me, the most significant result was that statistics anxiety did not vary across the cohorts. My initial [assumption] was that cohort 3 would show evidence of a stronger effect of statistics anxiety though this was not the case. After discussion, we concluded that our definition of a ‘specialist’ may not necessarily be true and may have impacted the outcome.
S3: The most significant result was the realisation that 42% of students on the course we initially considered ‘specialist’ didn’t list statistics or mathematics as their main subjects. This, perhaps more than any other finding, shaped our approach to the redesign of the Level 1 statistics courses.
As previously discussed, for an early career teacher it is important to understand who your students are before completing the transition from self to student (Kugel, 1993). However, what this study has shown is that categorising students isn’t as easy as it seems, even when the structures (such as course requirements) try to do so! Perhaps this is a specific issue for statistics as it is increasingly seen as a necessary subject for a wide variety of career goals, or for Level 1 in Scotland where the degree structure typically has a broader foundation and so students often pick up ‘additional’ subjects.
As an academic developer, I was particularly struck by my colleagues not stating that one of the most significant results was their learning as SoTLers through experience and reflection (which, to me, is hugely evident!). They identified a need for more preparation, for greater critique of methods and for the ability to use SoTL as a tool for enhancement, yet this was not explicitly stated in response to: ‘What was most significant?’. Perhaps this element of professional development, a critical awakening as a scholarly teacher, needs to be more explicit.
Glassick’s framework calls for SoTL to be effectively presented. Indeed, one distinction from ‘a scholarly approach’ to teaching and SoTL is that SoTL is disseminated, shared, and becomes public and open to critique (Kern et al., 2015; McEwan, 2022). Whether or not such scholarship is public, highly cited or has significant impact is another matter for another debate (e.g., see Canning & Masika, 2020), but for this work an important question for the authors of Bock et al. (2020) and Alexander et al. (2022) was: ‘On reflection, what would you choose to communicate about your project and how?’.
S1: To be completely honest, I’m not sure the results alone are novel enough to make a valid contribution to the field. What might be more useful would be more of a discussion paper on what we learned and how our teaching/materials have changed as a result of that (which would have taken several years to pull together). We were too focussed on the analysis of the responses to the questionnaire, rather than the implications of this for us as teachers on these courses. Perhaps a better route of dissemination would have been to informally share the results at a conference (for example) initially, and then have these results at the forefront when making changes to course design to allow us to write a more reflective piece after that. I feel like we perhaps missed an opportunity to use a scholarship project to directly influence the changes we made to our Level 1 statistics courses, as part of a Level 1 review that took place the following year.
S2: From this work, my main communication would be that irrelevant of a cohorts’ speciality in statistics, the effects of statistics anxiety are prevalent across all students in introductory statistics courses. Looking at particular areas of the qualitative analysis, key phrases that were pulled out were ‘programming’ and ‘probability’ which suggest these are problem areas for students, and design of any such course should take this into consideration.
S3: I think this is a key consideration as we review the design of the study and the instruments used. Little thought beyond ‘publishing’ was given to the options for dissemination but in hindsight any study worth doing should be worth disseminating the results thereof. In particular, I would consider ‘less formal’ methods of dissemination such as pre-prints and seminar talks to help shape and define the goals of the study and the methodology used.
‘Outputs’ of SoTL are somewhat contested! Even the call for papers for this special edition posed a provocative statement around ‘easy outputs’. In a research-intensive university, the cultural expectation around ‘outputs’ centre on high impact, international journal publications – this is what is valued in reward and recognition rounds, and this is what is valued institutionally. These journals need novelty, significant findings, and valid contributions to the field. However, SoTL journals typically have lower impact values (Fanghanel et al., 2016) and so the institutional pressure to publish SoTL in any journal may be significant.
It is clear from some of the comments that (impactful) journal articles were likely at the forefront of the authors’ minds with mention of valid contributions and novelty alongside considering ‘less formal’ routes (i.e., not a journal publication). However, perhaps the work of Trigwell and Shale (2004) would be of value here, where ‘outcomes’ rather than ‘outputs’ frame the effective presentation of SoTL. Outcomes include documentation, but also include learning: student learning and teacher learning. Outputs include articles, reports, presentations, conversations, and curricular materials.
In terms of outcomes, it is clear that learning has taken place through this work – significant teacher learning! Learning about practice, about curriculum, and, critically, about students and about SoTL. Despite this, there is some feeling from the authors that the work needs output as well. This is no bad thing – but support and recognition of SoTL-appropriate outcomes and well as outputs from the institution would be of considerable value, particularly to new SoTLers!
It feels entirely appropriate to finish a reflective critique about engaging in SoTL with some overarching reflections. The teachers that began this project and the authors that now reflect on it are changed. There has been considerable time and experience that has gone into building significant tacit knowledge around teaching, supporting learning and scholarship of teaching and learning. So, with that experience in mind, the authors were asked: ‘What does the ‘new you’ wish the ‘old you’ knew?’.
S1: I wish I knew how different undertaking a scholarship-based project is compared to the research I had done as part of my MSci and PhD. We were definitely focused on the statistical analysis rather than the implications of what we found.
S2: Two things. I wish I had firstly thought more carefully about how we define a ‘specialist’ and construct[ed] more questions which indicated this type of student. I also wish I had been more aware that statistics anxiety is still highly prevalent even in optional courses!
S3: I wish I knew the wider educational context into which our findings could be shared. Indeed, I still feel I’m in the ‘shallow end’ of this pool!
I wish I knew (or perhaps remembered) how much of a learning curve taking part in SoTL for the first time was! I have completed many projects over the years and have published in SoTL journals, given conference presentations and even the odd keynote and I think I have forgotten what it’s like to be ‘new’. Oddly, I still feel ‘new’ and can resonate with S3’s remarks about ‘still’ feeling like I’m in the shallow end, after all developing as a SoTLer takes time (McEwan, 2022). I also now realise the absolute importance of ‘adequate preparation’. This seems very odd – I teach about the absolute importance of adequate preparation to early career lecturers all the time, but this experience has taught me that if you think your preparation is adequate, then it isn’t. Do more. Do it again. Do twice as much. The comments relating SoTL to a PhD highlight something key here: many academics have completed a PhD which typically results from 3-5 years of undergraduate study in a specific discipline followed by 3 further years of highly specialised study and research experience. That’s a lot of formal learning. The experience on which a first SoTL project is built is likely measured in months, not years. So, there is very little preparation already in the bank.
To conclude, the call for contributions to this special edition contained the statement 'SoTL has a reputation in some quarters for ‘easy’ outputs, at odds with the competitive nature of leading journals'. We wanted to write this piece as a result of this statement because, as we hope this reflective article highlights, SoTL is far from easy. Accordingly, we felt it appropriate to close with a response to that statement.
S1: I think there is a place in SoTL for both ‘easy’ outputs and the more traditional competitive journals. In my experience (and this won’t be the experience of everyone engaging in SoTL), it feels that LTS lecturers are expected to start publishing in high-end journals with little-to-no training or experience in SoTL. What I mean by that is that many of us come from completing a PhD (and perhaps post-doc) in a very discipline focused field of research but when we begin our journey into SoTL we are essentially re-training in a different field. In order to be deemed as ‘successful’ (by that I mean in terms of promotion on this track) we are expected to start producing outputs almost immediately or be stalled in our careers. Would a first-year PhD student be expected to publish work to the same standard that a researcher with 5+ years of experience would? People might term some SoTL outputs as ‘easy’, but they aren’t necessarily ‘easy’ to the person who has worked on them. I feel the ‘easy’ outputs are just as important as the more traditional outputs as they allow for LTS lecturers to build that experience that [research-focused] lecturers gained from PhD/postdoc training. Additionally, the style of more traditional journal-based outputs doesn’t always fit in with a piece of SoTL. Maybe a blog-post or case-study or podcast is actually a more impactful way of disseminating the work, but that shouldn’t diminish the work. The phrase ‘horses for courses’ comes to mind.
S2: From my experience in trying to publish this work, I feel somewhat the opposite with this statement! I would say there is something of a culture regarding SoTL outputs as less impactful than subject specific research, but with time I have understood that this is more to do with a misunderstanding of what SoTL research is and [how it] impacts. During the early stages of my career, I have become more aware of the university wide impact of SoTL and some of the interesting and impactful research being done throughout, which I have either tried to incorporate into my own practice or seek further information in constructing similar projects with other colleges and schools.
I would also seriously challenge the supposed ‘easy’ nature of publishing in SoTL, as I have found with this work! The review process in journals I have been found to be very rigorous and journals will require your work to align very closely with their ethos and issue topics closely. I feel the review process is just as difficult and thorough as it is in an applied statistics journal, and your work must be to a very high standard to be accepted and published. As SoTL is a new field for myself, this has been a steep learning curve, but has provided me with far more respect for the quality of publications in the field.
S3: My perception and (limited) experience doesn’t agree with this statement. Perhaps if the statement is referring to the broad range of methods for dissemination of SoTL (e.g., seminars, workshops, pre-prints, blog/twitter posts, etc.) then it could be argued that there is more scope to share insights into learning and teaching (compared to discipline specific research). But if we are comparing ‘like with like’ then I don’t agree that its ‘easy’ to publish in leading SoTL journals. Indeed, for those coming from a different background to, say, educational/social sciences, to get outputs published requires a major realignment and effort and certainly is not ‘easier’ than continuing to publish in their specialist field.
Easy outputs? Every author in this project has a substantial track record of publishing in their ‘subject’ areas with nearly 50 publications amongst them in a variety of formats. Every author has a PhD, postdoctoral research experience and multiple degrees. This is a qualified group (as is any academic author list). Yet everyone here said it was hard. That the learning curve was steep. That the process was rigorous (and frustrating). SoTL should not have a reputation for easy outputs – quite the opposite. There exists a confusing rhetoric around SoTL vs pedagogic research (PedR as it’s often called) and its consideration as research (Tierney, 2020) and that alone means that ‘outputs’ are not easy! In many instances, these outputs (not outcomes) are at odds with the focus of a more teaching focused academic’s role and competing priorities where teaching and administration are often prioritised over SoTL (e.g., see Figure 1 in Tierney, 2020). Indeed, as Webb (2016) notes, there are several threshold concepts involved in ‘learning’ SoTL: breaking away from subject norms and epistemological positions alongside negotiating the liminality between subject expert and SoTL novice. Negotiating threshold concepts is hard, indeed troublesome by their very definition (Meyer & Land, 2003). Those who are engaged in enquiring into their teaching practice for the purposes of professional development, enhancing student learning and sharing those stories are specialists in an area that many are not. They are SoTL experts. They are learning and teaching experts. They have worked very hard to get there.
Advance HE (formerly Higher Education Academy). (2011). The UK Professional Standards Framework. https://www.advance-he.ac.uk/guidance/teaching-and-learning/ukpsf
Alexander, C., Jack, E., McEwan, M.P. & Bock, M. (2022) Exploring statistics anxiety in several introductory statistics courses to understand differences between types of students and types of courses. In S. A. Peters, L. Zapata-Cardona, F. Bonafini, & A. Fan (Eds.), Bridging the Gap: Empowering & Educating Today’s Learners in Statistics. Proceedings of the 11th International Conference on Teaching Statistics (ICOTS11 2022), Rosario, Argentina. International Association for Statistical Education.
Bock, M., Alexander, C., Jack, E., & McEwan, M. (2020, August 25). Study design & quantitative analysis in learning and teaching scholarship: Case study exploring student attitudes to introductory statistics. 13th Annual University of Glasgow Learning and Teaching Conference, Glasgow.
Birenbaum, M., & Eylath, S. (1994). Who is afraid of statistics? Correlates of statistics anxiety among students of educational sciences. Educational Research, 36(1), 93-98. https://doi.org/10.1080/0013188940360110
Biggs, J., & Tang, C. (2011). Teaching for quality learning at university. McGraw-Hill Education (UK).
Brookfield, S. (2017). Becoming a critically reflective teacher. John Wiley & Sons.
Browne, J. (2010). Independent review of higher education funding and student finance. http://www.bis.gov.uk/assets/biscore/corporate/docs/s/10-1208-securing-sustainable-higher-education-browne-report.pdf
Canning, J., & Masika, R. (2020). The scholarship of teaching and learning (SoTL): The thorn in the flesh of educational research, Studies in Higher Education, 47(6), 1084-1096. https://doi.org/10.1080/03075079.2020.1836485
Chalmers, D. (2011) Progress and challenges to the recognition and reward of the scholarship of teaching in higher education. Higher Education Research and Development Journal, 30(1), 25-38. https://doi.org/10.1080/07294360.2011.536970
Chew, P. K., & Dillon, D. B. (2014) Statistics anxiety update: Refining the construct and recommendations for a new research agenda. Perspectives on Psychological Science, 9(2), 196-208. https://doi.org/10.1177/1745691613518077
Cruise, R. J., Cash, R. W., & Bolton, D. L. (1985). Development and validation of an instrument to measure statistical anxiety. Proceedings of the American Statistical Association, Chicago, IL.
Fanghanel, J., Pritchard, J., Potter, J., & Wisker, G. (2016). Defining and supporting the Scholarship of Teaching and Learning (SoTL): A sector-wide study. Executive summary. http://repository.uwl.ac.uk/id/eprint/2065/
Felten, P. (2013). Principles of good practice in SoTL. Teaching and Learning Inquiry, 1(1), 121-25. https://doi.org/10.20343/teachlearninqu.1.1.121
Glassick, C. E., Huber, M. T., & Maeroff, G. I. (1997). Scholarship assessed: Evaluation of the professoriate. John Wiley & Sons.
Gordon, S. (2004). Understanding students’ experience of statistics in a service course. Statistics Education Research Journal, 3(1), 40-59. https://doi.org/10.52041/serj.v3i1.541
Hanna, D., Shevlin, M., & Dempster, M. (2008). The structure of the statistics anxiety rating scale: A confirmatory factor analysis using UK psychology students. Personality and Individual Differences, 45(1), 68-74. https://doi.org/10.1016/j.paid.2008.02.021
HESA (2021, January 19). Higher education staff statistics: UK, 2019/20. https://www.hesa.ac.uk/news/19-01-2021/sb259-higher-education-staff-statistics
Hubball, H., Clarke, A., & Poole, G. (2010). Ten-year reflections on mentoring SoTL research in a research-intensive university. International Journal for Academic Development, 15(2), 117-129. https://doi.org/10.1080/13601441003737758
Hutchings, P., Huber, M. T., & Ciccone, A. (2011) Scholarship of Teaching and Learning reconsidered: Institutional integration and impact. San Francisco: Jossey-Bass
Kelly, N., Nesbit, S., & Oliver, C. (2012). A difficult journey: Transitioning from STEM to SoTL. International Journal for the Scholarship of Teaching and Learning, 6(1), Article 18. https://doi.org/10.20429/ijsotl.2012.060118
Kern, B., Mettetal, G., Dixson, M., & Morgan, R. K. (2015). The role of SoTL in the academy: Upon the 25th anniversary of Boyer’s Scholarship Reconsidered. Journal of the Scholarship of Teaching and Learning, 15(3), 1-14. https://doi.org/10.14434/josotl.v15i3.13623
Koemadji, Z. A., & Ainley, J. M. (2018, July, 8-13) Specialist and non-specialist students’ attitude towards statistics. International Conference on Teaching Statistics. Kyoto, Japan. https://iase-web.org/icots/10/proceedings/pdfs/ICOTS10_C247.pdf?1531364324
Kugel, P. (1993). How professors develop as teachers. Studies in Higher Education, 18(3), 315-328. https://doi.org/10.1080/03075079312331382241
McEwan, M.P. (2022). The Journey to SoTL: Institutionally supporting a transition to scholars of teaching and learning. International Journal for the Scholarship of Teaching and Learning, 16(2), Article 5. https://doi.org/10.20429/ijsotl.2022.160205
Meyer, J., & Land, R. (2003). Threshold concepts and troublesome knowledge: Linkages to ways of thinking and practising within the disciplines (pp. 412 – 424). University of Edinburgh.
Onwuegbuzie, A. J., & Wilson, V. A. (2003). Statistics anxiety: Nature, etiology, antecedents, effects, and treatments - A comprehensive review of the literature. Teaching in Higher Education, 8(2), 195-209. https://doi.org/10.1080/1356251032000052447
Ramsden, P. (2003). Learning to teach in higher education. Routledge. https://doi.org/10.4324/9780203507711
Sandoz, E. K., Butcher, G., & Protti, T. A. (2017) A preliminary examination of willingness and importance as moderators of the relationship between statistics anxiety and performance. Journal of Contextual Behavioral Science, 6(1), 47-52. https://doi.org/10.1016/j.jcbs.2017.02.002
Smith, J. (2010). Forging identities: the experiences of probationary lecturers in the UK. Studies in Higher Education, 35(5), 577-591. https://doi.org/10.1080/03075070903216650
Tierney, A. (2016). More than just a teaching fellow: The impact of REF and implications of TEF on life science teaching-focused academics in UK HEIs (Doctoral thesis, Durham University). http://etheses.dur.ac.uk/11826/
Tierney, A. (2020). The scholarship of teaching and learning and pedagogic research within the disciplines: should it be included in the research excellence framework? Studies in Higher Education, 45(1), 176-186. https://doi.org/10.1080/03075079.2019.1574732
Trigwell, K., & Shale, S. (2004). Student learning and the scholarship of university teaching. Studies in Higher Education, 29(4), 523-536. https://doi.org/10.1080/0307507042000236407
Webb, A. S. (2016). Threshold concepts and the Scholarship of Teaching and Learning. In R. Land, J. H. F. Meyer, & M. T. Flanagan (Eds.), Threshold concepts in practice (pp. 299–308). Sense Publishers. https://doi.org/10.1007/978-94-6300-512-8_22