Evaluation and Reflection

Introduction

As with many new lecturers, I have had little training in teaching and so have often had to adapt as I go along. But I am a scientist at heart and approach teaching with the same critical mindset as I approach research. Remembering well what it is like to be a student, it’s important to me that my methods are effective and if something doesn’t work, I aim to realise it quickly and change it. 

I have been very lucky to have two excellent mentors at UWC who helped me settle into my courses during an extremely turbulent time (the height of the pandemic). CP, a visiting professor from the USA (a Fullbright scholar), had taught the Honours course the year before me. Ian Schroeder,  winner of the “Most innovative teaching during the COVID pandemic” award in 2021, taught the same course several years prior. Both agreed to serve as moderators for the Honours course and the PHY322 practicals. I’m extremely grateful to both of them for their guidance in uncharted waters!

As mentioned in Teaching Practices, I use google forms to obtain feedback from students throughout the course. The regular feedback gives students an opportunity to let me know if something is going wrong. Throughout 2021, I adapted rapidly, often on a weekly timescale. Although I had done a needs assessment which included some evaluation of programming skill and computer literacy, I nonetheless came in with an assumption about how quickly these skills can be obtained. The start to the course was quite rocky before I started learning how to scaffold assignments correctly. As I said, my goal is always to meet the students wherever they are and try to walk with them to achieve the course objectives. This has often required significant flexibility in course design.

One of the most important tools is the final course evaluation. I perform a fairly short evaluation at the end of the PHY322 practicals, since they are only a component of a larger module but I request an extensive feedback form be filled out by the Honours students. Here I summarise some results and reflect on what I learned from them and how I have incorporated the comments into my 2022 course design.

PHY322 Feedback 2020

I only asked students for qualitative feedback, specifically asking them what they found helpful, what they didn't and any further comments. Here's a selection of comments, positive and negative.

What aspect of the course was most useful in helping you learn?

• "I found the resources most useful. When I'm researching these sorts of tutorials without any guidance I become overwhelmed with what is necessary right now and what might be necessary in future. I appreciated the step by step tutorials that slowly transitioned into assignments."

• "I found the guided assignments very helpful"

• "The instructions were always laid out so we knew what to do. And Dr helping us when we came to you with problems"

What aspect of the course was least useful in helping you learn?

• "The online interactive tutorials which were only useful in the beginning"

• "learning by own self"

• "Covid :("

• "Duration of two weeks assignment was too much"

Any other comments or suggestions for this course going forward?

• "I love the set up wished I was introduces to python this way when we started working with it"

• "i suggest to start teaching python platform starting 1st year level"

• "face to face interaction is a must even if its for a computer class"

My reflection on these comments:

Although one student suggested two weeks was too long for an assignment, my experience suggested students struggled with completing the assignments in one week. However that was when all courses were online, my plan is to keep things the same this year and see how the students do. If they find the practicals too short, I can add more content, especially as these students are highly motivated to gain programming expertise.

Students consistently request more training in programming throughout their physics degree. As part of this, I have joined the computational physics working group within the department to try to integrate more changes throughout the degree. Although I do not expect things to change quickly, we may be able to introduce more programming aspects at various points in the undergraduate rather than starting as late as we currently do.

This cohort of students seemed to take well to the guided practicals and appreciate the resources I curated for them.

PHY322 Feedback 2021

I also asked for other comments or suggestions and select a few here:

• "Why does this course count so little towards the semester mark? The Computational Physics part should count more to that since most of us will be making use of this in our futures."

• "I think children who have no idea what coding is would have learned much more if Doc Loc actually thought Python, i did coding in first year and basics being laid by a lecturer really helped a lot."

• "The course was decent. I understand it is a self learning course but more face to face or online face to face with the lecturer and or tutor would be apricated. It also help to show that the support is there instead of just those a WhatsApp message or an email in which you do not really feel the support. (When I say support I don't mean helping with the work and giving answers, I mean that you are actually there for us to come and ask questions and interact more) ."

• "I still believe that computer science should be a compulsory module in 1st year so that we can get use to the language of code and then by 2nd year to start with some of the stuff we did in the course so when we/they get to 3rd year we can start doing more questions such as the question 3 and 4 and 5 in the assignments which is essentially what we need in at honours level to then keep up with that pace as well as help us in the work place since most fields are turning to coding and computational modeling."

• "I learned more code in this few pracs then i did in all my years at uwc. The structure of the pracs were set up perfectly for learning to code in a self learning environment"

My reflection on these comments:

Again, I find students feel they are being under-served by the lack of programming preparation in their degree but there were some positive comments that my "crash course" approach is an effective method of teaching. I did hear the requests for more interaction and am grateful we can be face-to-face in 2022. I suspect some students will benefit in particular from having to sit in the practical sessions, do the work and ask for help when they need it. 

While obviously dominated by small number statistics, there does appear to be a positive correlation between attendance and performance. This was important to test because most of the time in class is actually spent just working on the practical which, some could argue, can easily be done at home. Indeed, there is at least one outlier who did not attend classes but still performed extremely well. However on average, students who attend class do better. Whether this effect is due to higher performing students being the ones who generally attend all their classes or to class attendance assisting in learning is not entirely clear. The class average was 4% higher in 2022 compared to 2021 but I do not yet have enough data to know if this is significant. My conclusion from this and comparing against last year's performance is that third year students are generally responsible enough to manage their time effectively and do not need policing. However, a lack of class attendance can be an early indicator of at-risk students if their marks are also comparatively low.

• "Ive realised that physics is mostly about modeling and not about using equations"

This was a very meaningful comment for me because I find many students think physics is all about finding the right equation to do and then calculating the answer, which is highly disconnected from the way physicists actually do work. I was very happy to have helped at least one student realise this fact.

Computational Physics Honours 2021

For the Honours course, I obtained a standardised assessment form from the department, which I adapted. The final form had 30 questions, most numerical (from strongly disagree to strongly agree) with some written questions. 6 out of 8 students responded. I had positive responses to questions such as "I enjoyed this course", "the lecturer was very organised" and "the lecturer motivated me to do my best".

All students found the live feedback sessions useful, especially making use of interactive tools like AhaSlides. One commented that it helped that these were recorded so they could go back and rewatch. This is difficult to maintain in an in-person environment. I've fallen back on making short videos on any particular concepts that seem generally misunderstood but it seems this is one positive aspect of online learning that may be missed by the students.

What aspects did you enjoy most about Computational Physics?

• "getting support with hard question from lecture"

• "Learning all the new concepts and things we can do with python"

What aspects of the course need to be improved?

• "more time with lecture"

• "More practice questions would be nice. Basically like a tut, questions that dont count for marks but are solely there for practice."

• "There could be more smaller exercises or example problems."

Based on your experience of this course, Computational Physics, how has your view on the nature of science and being a scientist changed?

• "For the most part, numerical methods plays a big role, and coding makes it second nature to be able to do or understand these things."

• "The course has shown me that there are many ways to look at problems and solve them in science."

• "I think Data science needs a dedicated person and always punctual with time"

Please give any other suggestions you like about the course

• "Well done Dr you really do take people out of their comfort zones"

• "problems solving and understanding way to tackle physics questions"

• "Doc Lochner is great!"

• "The course needs more time especially for those who never did programming before"

My reflection on these comments:

Firstly, I think these comments show the students had an authentic learning experience, which was one of my goals. They felt they experienced first hand how to apply these techniques to real physics problems and that they had obtained useful problem solving skills. I considered deeply comments about needing more time and also the request for more example questions. It's difficult to carve extra time for the course so I have opted for trimming it down with more focus on the basics up front. I have my doubts about providing more questions for them to solve, given how constrained for time they already appeared to be. I have provided the current set of students with resources for programming challenges to hone their skills, but this request is one I will keep in mind and discuss with the class.

Overall, I have been pleased with the feedback I have received from students so far. I think my teaching style, which can be considered to be unorthodox, is appreciated and understood. They recognise the value of the skills they are learning and the importance of self-learning. I will continue to refine these courses based on the feedback, providing scaffolding and extra resources as needed.