As someone who has regularly trained language models, fine tuned and used LLM's including GPT3 almost daily, I think I can comment.
1. When you prompt the algorithm to "write an essay on" you won't get much success, without showing it some examples first and with a bit of breakdown at the paragraph level. Finding an efficient way of breaking down what "analysis" is and feeding it into the model in a way it understands (few - shot learning) will help it learn what you want.
2. In fact, not just analysis, anything else that fits the marking criteria - quotes form the book, clarity of language. There is an emerging field of AI engineering called "prompt engineering" geared towards how to get these generative models to produce what you want.
3. It is possible to fine tune davinci to generate analytical paragraphs. With enough labelled data, writing an analytical paragraph is possible.
So I guess my question is, if a student can do all that, why don't they just write an essay? I think it has potential but I think you need expert knowledge to make it work. And then you may as well just write.
I guess - but the technical skills gap between GPT3 and students isnt expected to be filled by students, but a software company. Jasper AI, for example, is built on top of GPT3 and writes marketing copy by filling in the gap between what comms departments want and what GPT3 can handle. All I'm saying is that it's certainly possible to close that gap.
I just don't know if essay copying is as luctrative an investment for these companies, which is I guess why tech has generally floundered in the ed space.
on a side note, i've used GPT3 to write a few teachers' reports to give them their weekends back :)
This is kind of how I see it. I don't think it will be something that will revolutionise education because like you say, there isn't any money in it. I am playing with AI for reports too :)
The rumor is that GPT-4 is coming out soon, maybe early next year. It'll be better, but my guess is that it will have some of the same pitfalls of GPT-3.
As a secondary math teacher homework is practically useless for assessing student learning. Between messaging solutions to your friends, solution apps like Photomath and Brainly, and even teacher editions of curricula Googleable you can’t trust what students submit as their work.
Having students do exercises in class has painfully but lucidly revealed what they know and can do; a big hill to climb, but bring your water and boots cause we’re hiking it.
The problem isn’t that AI has surpassed the analytical capability of a focused, articulate, engaged young reader -- it’s that it is very nearly surpassing what many instructors are forced by long experience to accept from the freshman students admitted to large state universities (at least in the US). That generated sample was boring and repetitive, but more robotically competent than many of the essays I have to grade every semester.
Well, perhaps. Depends on what we mean by "education." Something like 34-36% of US adults (from 2018 figures) hold a bachelor's degree. Did the average person's academic aptitude or cognitive horsepower *double* since 1985? Or did we enter a post-industrial, job-offshoring era where most non-elite undergrad education has come to serve as a credentialing mechanism for lower middle-class clerical and care-sector work? The idea of "tracking" younger students into vocational paths (not just blue-collar work, but nursing, accounting, broad swathes of "project management" and service-sector lower management) instead of through four-year colleges would be deeply controversial in the US given our racial history, ideas about egalitarian self-uplift, and given the demographic sorting that would likely occur under a system, e.g., like Germany's. I don't know as much about how this shakes out in Australia, the UK, etc.
As someone who has regularly trained language models, fine tuned and used LLM's including GPT3 almost daily, I think I can comment.
1. When you prompt the algorithm to "write an essay on" you won't get much success, without showing it some examples first and with a bit of breakdown at the paragraph level. Finding an efficient way of breaking down what "analysis" is and feeding it into the model in a way it understands (few - shot learning) will help it learn what you want.
2. In fact, not just analysis, anything else that fits the marking criteria - quotes form the book, clarity of language. There is an emerging field of AI engineering called "prompt engineering" geared towards how to get these generative models to produce what you want.
3. It is possible to fine tune davinci to generate analytical paragraphs. With enough labelled data, writing an analytical paragraph is possible.
So I guess my question is, if a student can do all that, why don't they just write an essay? I think it has potential but I think you need expert knowledge to make it work. And then you may as well just write.
I guess - but the technical skills gap between GPT3 and students isnt expected to be filled by students, but a software company. Jasper AI, for example, is built on top of GPT3 and writes marketing copy by filling in the gap between what comms departments want and what GPT3 can handle. All I'm saying is that it's certainly possible to close that gap.
I just don't know if essay copying is as luctrative an investment for these companies, which is I guess why tech has generally floundered in the ed space.
on a side note, i've used GPT3 to write a few teachers' reports to give them their weekends back :)
This is kind of how I see it. I don't think it will be something that will revolutionise education because like you say, there isn't any money in it. I am playing with AI for reports too :)
The rumor is that GPT-4 is coming out soon, maybe early next year. It'll be better, but my guess is that it will have some of the same pitfalls of GPT-3.
As a secondary math teacher homework is practically useless for assessing student learning. Between messaging solutions to your friends, solution apps like Photomath and Brainly, and even teacher editions of curricula Googleable you can’t trust what students submit as their work.
Having students do exercises in class has painfully but lucidly revealed what they know and can do; a big hill to climb, but bring your water and boots cause we’re hiking it.
Do you think that’s a school culture issue? If students aspire to know rather than aspire to score, that could shift things?
It’s a 2022 generational issue that as a school we must fix.
Pandemic made it worse, but folks want something for nothing because that’s what “the culture” pushes.
Yet, If my generation had access to this tech we’d have done the same. Educators must adapt.
When I was 13-14 in the 70’s I didn’t care about “knowing” but my teachers, and parents, held me accountable for knowing, thus I learned.
Too many teachers today don’t hold students accountable for “knowing”, allowing them to “turn things in for points” vs. demonstrating mastery.
The problem isn’t that AI has surpassed the analytical capability of a focused, articulate, engaged young reader -- it’s that it is very nearly surpassing what many instructors are forced by long experience to accept from the freshman students admitted to large state universities (at least in the US). That generated sample was boring and repetitive, but more robotically competent than many of the essays I have to grade every semester.
You poor thing. Grading is boring enough! Do you think it’s part of a broader devaluing of education across all sectors?
Well, perhaps. Depends on what we mean by "education." Something like 34-36% of US adults (from 2018 figures) hold a bachelor's degree. Did the average person's academic aptitude or cognitive horsepower *double* since 1985? Or did we enter a post-industrial, job-offshoring era where most non-elite undergrad education has come to serve as a credentialing mechanism for lower middle-class clerical and care-sector work? The idea of "tracking" younger students into vocational paths (not just blue-collar work, but nursing, accounting, broad swathes of "project management" and service-sector lower management) instead of through four-year colleges would be deeply controversial in the US given our racial history, ideas about egalitarian self-uplift, and given the demographic sorting that would likely occur under a system, e.g., like Germany's. I don't know as much about how this shakes out in Australia, the UK, etc.