Full transcription of the interview

Jordan: [00:00:00] Hello, curious minds and a very warm welcome to The Research Beat . With me, your host, Jordan Kruszynski. Today we’re going to be talking about maths with our guest, Dr Charles Roddie. Charles has a BA in Mathematics from Trinity College, Cambridge, and a PhD in economics from Princeton. He’s lectured in mathematics and Economics at Oxford and Cambridge universities, and he’s a fellow at Sydney Sussex College Cambridge. In addition, he’s the founder and CEO of Summatic.

So Charles, welcome to the show.

Charles: Pleasure to be here, Jordan.

Jordan: Wonderful to have you. So, Charles, let’s get started:

Can you tell us what Summatic does? 

Charles: Well, it’s a platform for online assessment and resources for quantitative subjects, so Maths and STEM subjects.

So we work with Maths at school level, and we tend to work with STEM subjects at universities. And, uh, the aim is to present questions and learning materials, which are, are very authentic. So, for example, [00:01:00] student answers, they’re not just multiple choice, but the system could understand formula answers, equation answers, steps of working matrices, etc, and, we also try to present graphical explanations in a, a similar way to how you might present them on a whiteboard. So it’s more interactive than what you would see in a static textbook. So overall, it’s an interactive textbook for, maths and STEM subjects. 

Jordan: And I think the idea with all of these exercises is practicality, accessibility, understanding for students, right?

Charles: That’s, that’s right. , It seems to be a very effective learning process for students so that, they can take in some quite hard concepts in step-by-step, uh, approaches. So really drawing , on, um, the experience , of what you would do in a lecture, building things up a step at a time.

Jordan: How can we actually convey that in an online platform, that’s better than just a, a pdf that’s, that’s put on, on a screen. So why did you found?

Charles: , well, clearly there are [00:02:00] hundreds of , millions of people in education at any, each one time. So it’s, uh, it, it’s clear an enormous challenge to get them all learning effectively with the best, you know, best, uh, teaching methods and approaches and, you know, what, what’s the potential of of technology to actually allow the best, uh, contents, the best learning methods to be conveyed, uh, has always been a question , in my mind.

And so when I was teaching, I would think about, you know, what, what am I doing? Is there a method? But also, uh, could a computer do this? And, uh, that’s not just the question of could a computer take my job? Um, which is a fear of many people, but also, you know, can the best let methods of learning and assessment.

Be reproduced in order to reach, um, many more people than with just it in, in one classroom. So what I, I wanted to do more than, than just have, have the best method I could for, uh, the few people sitting in front of me. But can, can this be actually expanded , and scaled out? Uh, so we can take , the best education and, uh, distribute it [00:03:00] effectively?

Jordan: It’s very nice idea, and do you find generally that students have a really good experience with Summatic?

Charles: Um, yes. , I developed it based, based on, my experiences of , what is needed for a complete solution. And, uh, we’ve had some very good feedback , from students, that actually. Validates that, which I was very, very pleasantly, surprised with. So, um, the, the aspects , that they quite like are, first of all, when they do a question, they have a chance to correct answers and they have a chance to do, new randomized variants.

And, uh, they find that quite, an anxiety-free method of, of, of learning, Cause they, they’re not faced with, they get it right or wrong, and that’s it, . and they also ha they also like the convenience. I think convenience is very important for students, uh, especially these days, uh, whether we’ve got an app and then they do a question and , there are links to the relevant materials that are right there.

So putting things together for them in, in one place. Um, Typical academic task, but, uh, they, they, they like it when it’s done very, uh, very [00:04:00] simply. 

Jordan: So it’s not exam conditions, which, which does take 

Charles: some of the pressure off. Yes. It’s generally, we haven’t been used a lot in exam conditions because exam conditions need a lot of extras.

For online assessment, you need proctoring solutions. So there are people talking to us about using in exams, but , likely to be , later in the year. And most of the work we do is, is on the formative side. Uh, getting students to, to grips with materials, not rather than exams. 

Jordan: So, Charles, tell us:

What are the mathematical challenges that students typically face when they’re moving from school to university?

Charles: Well this, it’s a very, uh, very broad question because there are many different university degrees and there are also different levels of prerequisites.

So some students are coming in with GCSEs, some students are coming in with A levels and they’re going to do different STEM subjects that have different quantitative requirements. But, uh, usually there is, there’s, there’s a very large, um, hill climb wherever that hill is positioned. . Um, for [00:05:00] example, um, students may be coming into a biosciences degree or an economics degree and might have just GCSE mathematics.

So then in the first year, they have to learn, um, how to do advanced statistics, handle data sets, uh, do various hypothesis tests. Um, they may have to learn a whole calculus course. And this is a very, very large jump from GCSE, which they have to make during the first year while they’re doing all sorts of other courses.

And it’s often in just a few weeks, uh, of, of, of teaching. So, when I hear that from academics, I feel like it’s an almost impossible challenge. But, it’s been really enjoyable for me to take part in that challenge and to help students to get up that hill.

Jordan: So on that point:

Almost one in three students in the UK are dissatisfied by the current level of assessment and feedback on their course. Why do you think they’re so dissatisfied? 

Charles: Well, it’s, it’s really hard to give good feedback to a [00:06:00] large, large group of students who are doing lots of work all the time.

 , it’s, it’s very time-consuming to give good feedback. And if, if, if hundreds of students are doing a course, it’s a, it’s often an industrial process of, of marking and, uh, assessing their work. Um, If I get an exam, then I get a huge pile of papers. And because it’s an exam, I take it very seriously.

And if every sentence they make, I try to figure out is this on the right lines? Is it a valid step? , if it’s ambiguous, I try to figure out what is in a student’s mind. But that is a, that’s an enormous amount of time and effort, which, which can’t be maintained through the whole of the year.

So it’s, it’s really hard to solve that challenge, and that’s where I think computers, uh, do come in. And, the ideal situation in my opinion is, if computerized assessment, automated assessments can tackle, um, you know, a majority of the work for assessment, where we’re teaching students how to do methods, solve problems…

They can do that online, [00:07:00] get assessed online, and then for those, uh, questions that require. Um, more complex answers, whether essays, extended reasoning, structured arguments that are being presented by students then bring in, humans, academics to evaluate those. In my opinion, that’s the best balance of time and quality.

Jordan: That sounds like a good balance. And I was going to ask about the explanatory element of some maths problems. Obviously, it’s not possible for, a computer program to be able to give that detailed explanation that might allow a student to understand. 

Charles: Yes. Well, um, these computers, that, that are doing assessment, they’re not thinking themselves.

So, ultimately the explanation is written in some form by, by a, a, a person, an academic who is, programming the system. . So it is possible to give good explanations on, on a question. And even if that question is, is randomized and has different variants, you can give, uh, explanations that are adapt to those settings.

So, [00:08:00] um, I think, you know, seeing explanations and, um, seeing how things are done is, is very possible for a computer system. The only thing that’s quite hard for computers is to evaluate student answers if they are very complex and structured and. and have intricate, uh, layers , of argument and explanation.

So that’s, that’s when I think, humans, uh, need to take over. 

Jordan: and these various problems that you designed Summatic to solve, did you experience or identify these problems during your own academic career? 

Charles: , that’s an interesting question, and funnily enough the answer , is no because, um, I, I think I, I’ve taught at, at places which have the luxury of, of, um, having huge resource.

In the form of college endowments in addition to, uh, a few student fees. Um, and, and with these huge resource, they employ huge manpower to, uh, to tackle , these problems. So we are able to give students here, uh, a really good experience of, of assessment, and we, we can look closely [00:09:00] at their answers and discuss with them in person, but that, that can’t be scaled out to, to, uh, to everyone.

The, uh, the resources are just, just too, too enormous. , for that. So it instead, it, it’s so inspired me , by showing what is, what is the ideal, uh, learning environment, and then how much of that could we actually, um, replicate , in an online setting and, uh, uh,, reproduce , at scale. 

Jordan: It’s a fine idea to take your experiences and , the quality of education that you received.

Put it all together and see if you can present it , to anybody and share it with them, uh, to allow them to learn it, you know, at their own pace in an effective style. Yeah.

So I think you touched on it earlier slightly, but:

How have the last two years of restrictions affected students’ abilities to transition mathematically from school to university? 

Charles: Uh, that’s an interesting question because it’s, um, when, when the, the, the lockdowns happened, um, I would’ve [00:10:00] expected we’ve got, uh, perhaps a, a a three months of learning loss on average, and then we’re gonna find students to just set back academically by three months when they, when they arrive.

Um, but it’s actually a lot more, more complex than that. And there are, there are so many elements that , have been affect. Um, so, I think , we do see students coming into programs a bit less prepared, academically, and we focus on quantitative skills and there is a bit less, less preparation when I talk to universities out there.

But it’s not just the lockdowns and the online. teaching and, the differences between different types of schools. Um, it’s not just that, it’s also that the assessment system was disrupted. So , we had, um, a couple of years of, teacher assessed exams, which meant, meant there was a, a lot of inflation, which meant that, um, students were going to very different universities than where,, where they would’ve gone.

Very large cohorts at universities with higher admission standards. So, um, [00:11:00] that means that there was their students going into harder programs and they would’ve gone to before. So that also exacerbates the gaps that students, and, uh, university departments.

we also have not just academic, learning losses, but also social organizational and, and personally teaching,, teaching, uh, here in, in one of the Cambridge colleges. , I haven’t witnessed a lot of academic gaps, actually. I find students academically have come in, uh, well prepared, but, um, we have found personal organizational, uh, gaps that it’s, it’s, it’s taking time for students to relearn how to.

Um, uh, how to discuss with each other, how to, collaborate, how to organize themselves. I think a lot of people used to have been sitting in, in a, in a bedroom, uh, learning, for a year. Um, and now having to take, take the time to figure out what it’s like to engage with people, uh, in, in person and what it’s like to organize [00:12:00] themselves in ways that they didn’t have to do when everything was online.

So, something I didn’t expect was to find that the non-academic issues are often more important, resulting from disruptions.

Jordan: And did you notice that the situation drove up interest in your platform? Or alternatively, did it make you think that, it was more important that there really was more of a core for platforms like this?

Charles: Yee. I mean, we didn’t catch the, uh, the early wave of it, just launching, a year ago. But certainly, uh, departments are very interested in, in, in preparation for students. And if, and if they are less certain about the level of students coming in, then, uh, then there’s a greater, you know, Greater interest in, in preparing them.

But I think that this is something ongoing universities have been preparing for, uh, the transition to University for, for a long time. And it’s, it’s part of this long, an ongoing process of digitization, uh, the use of online platforms to help, to help that process.[00:13:00] 

because you know, it, it, even before covid, the gap between school and university was a, was a big, big challenge. You got more and more students going into STEM subjects and so more and more students need to be trained in the relevant quantitative skills.

Jordan: So Charles, in this next section we’re going to talk about some of the issues in academia that are most important to you. So:

What have you observed while visiting multiple university departments across the country and indeed across the world?

Charles: Well, it’s been very, uh, it’s been very eyeopening because I was, um, I’ve just worked in a couple of, um, economics departments and I never really, uh, even worked out how the, those departments function cuz I was just giving my lectures, uh, doing the exams, et cetera. Um, so, uh, seeing how departments take, take decisions is quite, uh, it’s quite interesting.

Um, They certainly do it in different ways, and the use of technology is, is very varied. Um, so , if we start with technology, I do see, uh, vastly different levels [00:14:00] of adoption and preparedness when it comes to, uh, uh, digital, digital tools for learning. Some of them have had academics that have worked on creating systems, uh, over over years.

And then, . Have recently been able to deploy, preparatory systems, supporting systems , to give students resources, assessments for preparation and their first year. So a few, a few departments are ahead of the game. Um, a lot of them are starting to think about, digital tools, maybe , using a few from their learning management systems.

And, quite a lot. Haven’t started to  think about this and, um, are interested in a student preparation and resources, but haven’t really, made a start yet. So, Very wide, uh, variety of, levels of resource for, for students, um, that, that I’ve seen.

, I I also find differences in the ways that decisions get taken. , Often the organizational structures, um, are, are good at making certain sorts of decisions, like who should lecture this course? [00:15:00] Um, how, uh, how should we organize our teaching assistants and. Go about the process of being a department, but when it comes to, um, making a change, adopting something new, some departments are certainly more open and have the ability to take those decisions.

Other departments get interested, but the processes aren’t there for decision-making. Um, so, so certainly, certainly a lot of differences around decision-making processes, too, 

Jordan: , can things really get slowed down at the departmental level? 

Charles: Yes. It, it often is slow. I think if we, if we go from, um, fast examples to slow examples, then um, I think business schools tend to be.

Really the fastest and can often manage their own resources. And if they see your needs, they say, let’s, let’s try it. Let’s go ahead. Then, uh, a few weeks later, we see we’re actually preparing the solution and we’ve got all the contracts set up, et cetera. So it is possible for, for business schools and, [00:16:00] and some departments to move, uh, move fast.

Uh, but typically it’s a process that might take a year of thinking. Uh, perhaps an academic that’s responsible for a quantitative course gets interested then, makes a proposal. There, there are some lines of communication that’s, um, sometimes are standards and sometimes are just testing whether colleagues have the rights, the right people to talk to.

Um, and, and that process, can take a long time. And so the important thing there is that we. , we have that process at the right time of the year, so it can go ahead in the next year cuz otherwise it becomes a two-year process. So, it varies from being something that can happen , within a, uh, two or three weeks to something that can take over a year.

Jordan: And we are talking about implementing changes. In your view, does the kind of, the pace of these changes in departments have a knock on effect on students and researchers who are working within a univers?

Charles: I think that departments are, [00:17:00] most interested in identifiable problems, so where they are going from one course to the next, but don’t quite have the right skills to do that. . Um, so if we can fix those problems, that’s the low hanging fruit, I’d say. , so, where the existing core structure, uh, can be maintained, but we just make it easier for, for students to follow, follow through that.

Um, now in theory , they might be able to accelerate a core structure, uh, where, where they say, well actually, um, can, can, can you do this material before you start or, or write it? The write in your first term, uh, with this, computer assisted method. Then maybe we can add more quantitative, um, uh, of course.

Um, it doesn’t usually work like that, but, uh, we are talking to some departments that are deciding whether to take a more quantitative or a non-quantitative approach. Um, you know, economics is a subject, for example, where there are quantitative degrees, non-quantitative degrees, and having a, having [00:18:00] a, um, a platform that can help with quantitative, um, learning and assessments.

Um, I can influence that. 

Jordan: So, we’re going to take a look at a couple of articles and a couple of issues in your field, Charles. This first article using:

Game Theory Mathematics to Resolve Human Conflicts

Now, before we go into this, uh, I wonder if you could just give us a quick overview of what Game Theory is. 

Charles: Well, it’s a, it’s a very general field which, uh, looks at the actions people take in situations where their choices are interacting and their incentives are interacting.

 So how do people, and organizations, um, manage situations, where they are acting together, where their choices are interacting and where they have, you know, similar or conflicting incentive.

Um, now it’s a very general description because this sort of thing is, can, can be applied, uh, to, to many, many fields. Um, [00:19:00] the agents concerned, they could be people in teams, they could be, uh, countries that are interacting in international relations setting. They could be businesses, um, they can even be, um, plants and animals in an evolutionary biology setting where, where they’re interacting in a predatory or they can even be trees in a forest deciding whether to grow high or wide.

So, uh, it’s a very general generalizable setting and that generality has allowed it to take over a lot of economic modeling. Over the last 50 years. 

Jordan: And we have actually a really interesting example here of those conflicting ideas, intentions and standpoint. So Professor Nels Bonfeld, social and environmental scientist at the University of Sterling carried out a series of investigations into conflicts between.

People who on one hand want to use resources for, [00:20:00] biodiversity and on the other hand, who want to use resources for growing food, and, these games were developed and carried out, uh, concerning a variety of situations from Scotland. Toban. This is a very complicated issue and I wonder what you think about using game theory to try to solve these kinds of problems.

How does it work? 

Charles: Well, the way that they used, um, game theory was actually more using games, uh, to, to, to try to solve problems. And it was very entrepreneurial, so not your standard game theory paper that appears in a journal. Uh, they actually wrote, an app that gets people to participate in games where these games, have an ecological, uh, component.

Quite entrepreneurial, an entrepreneurial way of doing this because they built this to be engaging so that they could get lots of data. And, um, it looks like they analyzed that data from the point of view [00:21:00] of, um, uh, qualitative analysis, I think, so that they, could see how do people engage with these games.

How did they interact with each, um, what was enabling cooperation? Um, it’s, I think it’s a mix of qualitative and quantitative. Um, So, um, actually experimental, experimental game theory is a, is a, is a, is a big, uh, is a big thing in the research field because we don’t just want to understand how theoretical agents play games with complete rationality and maximum ability to.

To work out the best strategies. We also want to know how to, how to, to ordinary people, how do real organizations, play games? So, uh, I would say this, this work is more, more in that setting of experimental games. Usually experimental games are done in very controlled settings where , A group of participants that often they’re students, they come into a to a room, they’re give uncertain financial incentives, and then they make [00:22:00] strategies in controlled settings, which gives you some degree of reliability, some degree of scientific valid.

At least within that group of people for rewards that are fairly small. Um, but I think this, the research that was done by, um, Butterfield and others was, a bit wider ranging than that. Um, a bit less controlled, but more data. And I think that that is, that is what we’re, we’re seeing in the world out there.

Lots and lots of data. Perhaps not, not very controlled, but we can, we can try to glean, glean, glean what insights we can from. Well, 

Jordan: The article was very clear that the landowners involved in these various cases had very heated opinions, and it’s, it’s interesting to think about how those, you know, complicated, nuanced opinions can be worked into mathematical modeling and gains, which might have a, a worthwhile outcome in terms of helping people in situations.

Charles: yeah, the trouble with, with, uh, game theory is that it can get very [00:23:00] complex very quickly if you try to add, um, add a lot of features where there are the way in which people form opinions, uh, whether it’s, uh, various sorts of emotions. So, uh, it’s usually good for understanding, uh, one of those things at a time, um, but, but not multiple things because it goes beyond our ability to analyze those models. Um, but I, I, I think they were getting some quite interesting, um, Qualitative results about the effect of, being able to communicate on the ability to resolve problems. And also, uh, they mentioned, they mentioned the effect of, um, incentives, which is, is always, always a good, good thing to measure when we’re, we’re doing, policy and economics, uh, let me see. It was, it was incentives.

Uh, it was incentives of conservation efforts and the, and the effectiveness of that. Uh, so, then we’re getting some quite, quite interesting survey data on, on, on potentially the effectiveness of, of incentives of conservation out of these games where they got lots of people to engage.[00:24:00] 

Jordan: so Charles, you wanted to say something about polarization in this study.

Charles: Yes, cuz there was, there was some other research, um, in, in this article by, um, Oubre and, um, he was looking at, um, what are the dynamics of polarization, uh, which we’re seeing so much of these days that, it’s a massively important, topic. And, what he was finding was that, um, people, people aren’t just in.

Bubbles that exist, but these bubbles were strengthened by the incentives to, to appeal to, and get approval from, similar people. Um, so what, what are, what are the dynamics of, of, of, of bubbles I think is such an important question for the modern world. Um, because these bubbles are, are so, are so limiting to the process of thought.

The process of you are gaining knowledge and objectivity. 

Jordan: And just to clarify, these bubbles are situations or structures where you have a number of people who are [00:25:00] kind of aligned or contained within a particular ideology? 

Charles: Yeah. Contained in a particular ideology.

So they don’t hear the, um, other perspectives. , and information, which doesn’t confirm the bubble that they are, that they are in is, um, is not gonna get, not gonna penetrate.

So, so these things are, are really interesting , to model and try to understand and then see, you know, because the bigger questions is c can, can we continue to live in, democracies in a situation where actual objective information doesn’t reach people? Cuz, they all live in bubbles.

and I think, um, you know, the, their identification of the incentives that are involved in bubbles is really, really, really important. Now, I’m focused on teaching now as an academic and, through my startup Summatic, but, um, This sort of thing actually wants me to make, go back and do research and, and try to, uh, figure this out because, somehow we need to put together why are we having these, these bubbles and [00:26:00] this polarization, and can we.

Can we achieve a situation where we can combine, hierarchy with independence, uh, where the hierarchy is that, that some people have got to the point where they are becoming experts or , have been, um, approved As, as people who understand and understand things well, and , can we have these groups that are trusted and can communicate information more widely? Is that compatible , with the polarization and bubble formation that we see at the moment? 

Jordan: So Charles, we are going to close today with quite a big question for you:

Why does maths matter?

Charles: Well, mass has become the, uh, the, the methodology of, of so many subjects, out there. And, it’s been a gradual, gradual progress really over the, over, over millennia where it’s taken over, first of all, physical sciences, um, [00:27:00] then, then taking over. Um, economics, we, we’ve actually started with game theory was over the last 50 to 70 years.

Um, And now there are so many subjects that req that, uh, previously weren’t mathematical, but, um, now, have a lot of data and required data analysis and require more and more statistics. Um, so this is not a general answer to to, to why, why is it the case that, that that has. Emerged over time, but we are, we are certainly seeing more and more, more quantitative, trends within, within subjects.

Uh, we see rise of STEM subjects that require, requires, um, quantitative skills to, to, to, to build up those, those, those theories and models. And we could require more and more data these days. And that’s a, that’s a very. new thing. Um, where over the last, last, um, 10 years, there’s just been an explosion of data and people who otherwise would be in, in, in fields that, that were more discussive and now having to deal with, uh, with data.[00:28:00] 

And even in the humanities, I’m told that there are, there are now databases of, um, uh, databases of, of, of articles and research and, and research words. And many, uh, researchers spend their time in these databases, uh, querying, searching, um, performing, uh, quite quantitative studies actually. So.

So the most, the most recent answer to that question is that this, this, um, preponderance of data that has that, that is cropping up all over the place. And we have been measuring more and more things and having to analyze those measures. 

Jordan: So researchers, you’re going to need maths far into the future.

Charles: certainly researchers, but uh, I think even people going into to regular jobs. Need to answer the question of how do we use data to do our jobs more effectively? 

Jordan: Definitely. Charles, finally, how can our listeners reach you if they’re interested in getting in contact with you?[00:29:00] 

Charles: You can write to me at, the shortest email is cr250@cam.ac.uk, which is my academic email address.

Um, or , if you’re interested in the work I’m doing, With, uh, quantitative assessment and resources, then go to the summatic.uk page, that’s Summatic and you’ll be able to find my, uh, contact details there as well. 

Jordan: Wonderful. And can they also reach you on social media? 

Charles: Um, yes.

I tend to limit my social media usage these days, but um, you can find me on LinkedIn as well, Charles Roddie. 

Jordan: Fantastic. Charles, thank you so much. It’s been an absolute pleasure to have you today. A wonderful conversation. 

Charles: It’s a great conversation, Jordan. Thank you! 😊