Support from readers like you keeps The Journal open.
You are visiting us because we have something you value. Independent, unbiased news that tells the truth. Advertising revenue goes some way to support our mission, but this year it has not been enough.
If you've seen value in our reporting, please contribute what you can, so we can continue to produce accurate and meaningful journalism. For everyone who needs it.
WHO’D HAVE THOUGHT that news headlines this year would be filled with debate about assessment and examinations, but that’s exactly what we’ve seen over the last few months as new technology has caught everyone’s attention.
ChatGPT - an artificial intelligence chatbot developed by the AI research and development company OpenAI - was made available publicly last year and has led to much controversy given its seeming ability to answer any question in fluent, informed, well-structured language, and to refine its answers based on user feedback.
ChatGPT chat bot screen seen on smartphone and laptop display with Chat GPT login screen on the background. A new AI chatbot by OpenAI. Stafford, Unit Alamy Stock Photo
Alamy Stock Photo
Given that in education we ask our students questions and look for fluent, informed, and well-structured essays in return, you can see the problem.
The technology
ChatGPT is built using a ‘large language model’ (LLM). Essentially, this is a program that estimates the probability of words following others in a sequence, which it “learns” by analysing example text provided to it. When you use auto-complete in a text message, you are using a language model.
ChatGPT was trained to predict the most likely next word, using 500 billion words of text: all of Wikipedia, all public-domain books, huge volumes of scientific articles, and other web text (from many sites, including discussion forums of varied quality) – more than humans could read in a lifetime. It has additional training to enable it to participate in a turn-taking conversation and to recognise requests.
Older language models base their next-word predictions on the previous 5-10 words, but GPT and other new ones use a summary of all previous text as a context. This enables them to generate long sequences of plausible text.
It is intuitive to use, and its structure and scale lead to extremely interesting “emergent properties” – ChatGPT can generate answers to questions on virtually any topic, it can retain context across multiple turns, it can even generate rhymes, adopt any requested writing style, and produce computer code.
Advertisement
However, there is nothing fundamental in its design to ensure it generates truthful outputs, except that factual statements might be more common than non-factual ones in its training text. But it does often write factual material: you can predict what word is likely to follow “the capital of Bolivia is”, even if you don’t really know what a capital city is.
One way to think about ChatGPT is that it is like an improv performer, playing along with the prompts you give, rather than an oracle of truth. This creates risks if we take its outputs at face value.
A typical view of assessment is one-dimensional. There is a focus on a final exam, testing the learner at one fixed date and time to see how good they are (at such tests), or requiring the submission of a final product (an essay, a project report) by a deadline, and then graded out of their sight. Assessment, in other words, is something that is done to the student.
The challenge to assessment integrity is large because many courses include a substantially continuous assessment component. Almost any assessment that is primarily text-based, and completed out of sight of the assessor, is susceptible to ChatGPT.
Educational institutions have a public responsibility to be able to stand over their awards and qualifications and need to be assured that any work a student submits was produced by them and demonstrates their level of knowledge or skill. We have to protect honest students from reputational damage by association.
The homepage of ChatGPT. Artificial intelligence that writes greeting cards, poems or non-fiction texts - and sounds amazingly human. With the chatbot, you can not only talk on the Internet. With the help of artificial intelligence (AI), it also writes essays, poems, letters and all kinds of other texts on command - and impresses with its abilities. DPA / PA Images
DPA / PA Images / PA Images
In recent years we have seen the growth of companies that deliberately target students under pressure, offering them contract cheating services. One view is that using ChatGPT (or similar a tool) is not fundamentally different from using such a ‘service’ or having a friend do your assignment, but it may be more tempting because it is so accessible and easy to use.
Academia’s response
Considerable work is being done nationally and internationally, through groups such as the National Academic Integrity Network, Quality and Qualifications Ireland (QQI) and individual institutions, to counter this threat and to build resilience into assessment and qualifications.
Focusing on assessment integrity doesn’t mean that we can’t also think of assessment as being ‘for’ learning, and even serving ‘as’ learning.
Much of the conversation in educational circles has been around this more holistic perspective. In this view we can look at the artefacts we ask students to produce (essay, report, project) as being something in which they can feel pride and an opportunity for them to demonstrate that they’ve met the learning goals of the course.
Related Reads
ChatGPT creators release new tool aimed at easing concerns over university plagiarism
Students urged to steer clear of ChatGPT essays as bot reopens debate on college assessment
Chatbot capable of writing scenes for a play and dissertations released by US start-up
iPhone with appstore and ChatGPT and Chat bot apps, artificial intelligence time. WriteSmith - AI Essay Writing 01.01.2023 Alamy Stock Photo
Alamy Stock Photo
We can recognise that learning is a developmental journey and so rather than evaluating a student only on a final assignment, we can place value on the process that led to it.
When we write, we think. We have to gather ideas, connect them, reshape and structure an argument. Writing is a thinking tool, not just an ‘output’.
Therein perhaps lies the contrast with tools such as ChatGPT. As LLMs inevitably become embedded in a range of products such as MS Word, they will no doubt have a role in aiding the technical aspects of writing, but they cannot be used to shortcut thinking, insight, and creativity. We have to reward students for these abilities, not just for high-quality prose.
To achieve this, we educators may need to up our game. We shouldn’t aim to reward the “regurgitation” of material but rather aim to develop and assess abilities which build on each other – from memorisation and knowledge of basics to the ability to apply concepts in new situations, provide critiques, and generate new concepts.
Being clearer about how, what and why we assess, as well as ensuring our students have a critical digital literacy will also help position tools such as ChatGPT appropriately.
Of course, in practice, there are limitations imposed by resources and time constraints. Perhaps as a society, we need to be honest about the cost of high-quality education and invest to ensure we have realistic staff/student ratios, appropriate teaching spaces, and the learning supports required for diverse student needs.
But, nonetheless, re-framing learning and assessment to focus more on the learning journey will allow us to redesign courses, finesse our choice of teaching methods and reinforce the importance of student engagement and responsibility for their own learning.
Dr James McDermott is a Lecturer in the School of Computer Science, University of Galway. Professor Michael Madden is Established Professor of Computer Science at University of Galway where he leads the Machine Learning Research Group. Dr Iain MacLaren is the Director of the Centre for Excellence in Learning & Teaching at the University of Galway.
Readers like you are keeping these stories free for everyone...
A mix of advertising and supporting contributions helps keep paywalls away from valuable information like this article.
Over 5,000 readers like you have already stepped up and support us with a monthly payment or a once-off donation.
To embed this post, copy the code below on your site
Close
6 Comments
This is YOUR comments community. Stay civil, stay constructive, stay on topic.
Please familiarise yourself with our comments policy
here
before taking part.
Once schooling has streamlined students through to their initial job in their respective careers, at no point afterwards will they ever need to discuss education in a job interview.
Why bother write a well structured essay if a computer can do it? It “frees up time”.
“Free time” was the seed that started the industrial revolution, a bunch of rectors in GB (a Uni degree back then was necessary to be a rector)who were given a monthly stipend to do essentially nothing not even rectoring. Instead of sitting on their backside they carried out experiments( geology was the big thing back then). Read “bill bryson, short history of nearly everything” for more info.
“Free time” creates opportunity for revolution. [BRAIN DROPPINGS from former nuig d grade eng student, now highly successful]
Its a natural progression in academia, from writing notes on yellow rulers and reading them in exams to whipping articles off Wikipedia to getting an essay farm located offshore for $5. Schools need to be embracing Chat GPT and any other incarnations that appear are here to stay. It should be pretty obvious if someone who is normally poor at homework is suddenly turning in work thats full of hard text and every teacher should be able to spot the differences (z instead of s) between generative AI and written prose. There will just be more written exams instead.
Interesting opinion piece by Professors , a little surprised that the overall tone of the article is seemingly under playing the extent of disruption that technology is about to inflict on education , with respect finding the need to discuss the student teacher ratio / or that fact high quality education will cost more and need resources / I would respectfully suggest that our professors face up to the reality that the scale of changes globally on the role of technology in education is about to ramp up exponentially requiring extensive changing of the curriculum and impacting the very fabric of society / the scale of change will be unprecedented in the next 30 years / North Korea have been educating their youth on computer coding and cyber hacking 10 years / “the times they are a changing”
Really good point that scratches the surface of the problem with our lazy system of transitioning young adults into broader society, a system that needs broadening and a more individually focused order, not just one that makes it easier for institutions to ‘administer’ education. But the better questions are, why do we do jobs? what will we do when there is exponentially more people than work?
The bigger question is, how many of the careers for which third level qualifications are requirements will simply cease to exist, with AI taking over roles that are currently filled by graduates?
There was an awful lot of smugness a few years ago when automation was beginning to replace manufacturing roles, people that had a higher level of education, in a somewhat condescending manner, suggested that all they need do is upskill!
Those that said it was the thin edge of the wedge, were discounted as conspiracy theorists! After all, how could an AI replace a graduate in a high level career with years of experience?
The answer is very easily, not just a graduate, all graduates everywhere, add to this AI controlled robots to carry out physical work and it’s a brave new world..
Gaza famine 'reminiscent of Ethiopia' and 'unlike anything seen this century', says UN's WFP
Updated
26 mins ago
3.4k
Courts
Former garda who impersonated colleague online to encourage men to rape her jailed for seven years
1 hr ago
12.3k
Maguiresbridge
Suspect in shooting of mother and children in Co Fermanagh dies in hospital
16 hrs ago
84.3k
Your Cookies. Your Choice.
Cookies help provide our news service while also enabling the advertising needed to fund this work.
We categorise cookies as Necessary, Performance (used to analyse the site performance) and Targeting (used to target advertising which helps us keep this service free).
We and our 214 partners store and access personal data, like browsing data or unique identifiers, on your device. Selecting Accept All enables tracking technologies to support the purposes shown under we and our partners process data to provide. If trackers are disabled, some content and ads you see may not be as relevant to you. You can resurface this menu to change your choices or withdraw consent at any time by clicking the Cookie Preferences link on the bottom of the webpage . Your choices will have effect within our Website. For more details, refer to our Privacy Policy.
We and our vendors process data for the following purposes:
Use precise geolocation data. Actively scan device characteristics for identification. Store and/or access information on a device. Personalised advertising and content, advertising and content measurement, audience research and services development.
Cookies Preference Centre
We process your data to deliver content or advertisements and measure the delivery of such content or advertisements to extract insights about our website. We share this information with our partners on the basis of consent. You may exercise your right to consent, based on a specific purpose below or at a partner level in the link under each purpose. Some vendors may process your data based on their legitimate interests, which does not require your consent. You cannot object to tracking technologies placed to ensure security, prevent fraud, fix errors, or deliver and present advertising and content, and precise geolocation data and active scanning of device characteristics for identification may be used to support this purpose. This exception does not apply to targeted advertising. These choices will be signaled to our vendors participating in the Transparency and Consent Framework. The choices you make regarding the purposes and vendors listed in this notice are saved and stored locally on your device for a maximum duration of 1 year.
Manage Consent Preferences
Necessary Cookies
Always Active
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work.
Social Media Cookies
These cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.
Targeting Cookies
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
Functional Cookies
These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then these services may not function properly.
Performance Cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not be able to monitor our performance.
Store and/or access information on a device 149 partners can use this purpose
Cookies, device or similar online identifiers (e.g. login-based identifiers, randomly assigned identifiers, network based identifiers) together with other information (e.g. browser type and information, language, screen size, supported technologies etc.) can be stored or read on your device to recognise it each time it connects to an app or to a website, for one or several of the purposes presented here.
Personalised advertising and content, advertising and content measurement, audience research and services development 195 partners can use this purpose
Use limited data to select advertising 158 partners can use this purpose
Advertising presented to you on this service can be based on limited data, such as the website or app you are using, your non-precise location, your device type or which content you are (or have been) interacting with (for example, to limit the number of times an ad is presented to you).
Create profiles for personalised advertising 119 partners can use this purpose
Information about your activity on this service (such as forms you submit, content you look at) can be stored and combined with other information about you (for example, information from your previous activity on this service and other websites or apps) or similar users. This is then used to build or improve a profile about you (that might include possible interests and personal aspects). Your profile can be used (also later) to present advertising that appears more relevant based on your possible interests by this and other entities.
Use profiles to select personalised advertising 120 partners can use this purpose
Advertising presented to you on this service can be based on your advertising profiles, which can reflect your activity on this service or other websites or apps (like the forms you submit, content you look at), possible interests and personal aspects.
Create profiles to personalise content 51 partners can use this purpose
Information about your activity on this service (for instance, forms you submit, non-advertising content you look at) can be stored and combined with other information about you (such as your previous activity on this service or other websites or apps) or similar users. This is then used to build or improve a profile about you (which might for example include possible interests and personal aspects). Your profile can be used (also later) to present content that appears more relevant based on your possible interests, such as by adapting the order in which content is shown to you, so that it is even easier for you to find content that matches your interests.
Use profiles to select personalised content 48 partners can use this purpose
Content presented to you on this service can be based on your content personalisation profiles, which can reflect your activity on this or other services (for instance, the forms you submit, content you look at), possible interests and personal aspects. This can for example be used to adapt the order in which content is shown to you, so that it is even easier for you to find (non-advertising) content that matches your interests.
Measure advertising performance 177 partners can use this purpose
Information regarding which advertising is presented to you and how you interact with it can be used to determine how well an advert has worked for you or other users and whether the goals of the advertising were reached. For instance, whether you saw an ad, whether you clicked on it, whether it led you to buy a product or visit a website, etc. This is very helpful to understand the relevance of advertising campaigns.
Measure content performance 78 partners can use this purpose
Information regarding which content is presented to you and how you interact with it can be used to determine whether the (non-advertising) content e.g. reached its intended audience and matched your interests. For instance, whether you read an article, watch a video, listen to a podcast or look at a product description, how long you spent on this service and the web pages you visit etc. This is very helpful to understand the relevance of (non-advertising) content that is shown to you.
Understand audiences through statistics or combinations of data from different sources 111 partners can use this purpose
Reports can be generated based on the combination of data sets (like user profiles, statistics, market research, analytics data) regarding your interactions and those of other users with advertising or (non-advertising) content to identify common characteristics (for instance, to determine which target audiences are more receptive to an ad campaign or to certain contents).
Develop and improve services 116 partners can use this purpose
Information about your activity on this service, such as your interaction with ads or content, can be very helpful to improve products and services and to build new products and services based on user interactions, the type of audience, etc. This specific purpose does not include the development or improvement of user profiles and identifiers.
Use limited data to select content 51 partners can use this purpose
Content presented to you on this service can be based on limited data, such as the website or app you are using, your non-precise location, your device type, or which content you are (or have been) interacting with (for example, to limit the number of times a video or an article is presented to you).
Use precise geolocation data 65 partners can use this special feature
With your acceptance, your precise location (within a radius of less than 500 metres) may be used in support of the purposes explained in this notice.
Actively scan device characteristics for identification 36 partners can use this special feature
With your acceptance, certain characteristics specific to your device might be requested and used to distinguish it from other devices (such as the installed fonts or plugins, the resolution of your screen) in support of the purposes explained in this notice.
Ensure security, prevent and detect fraud, and fix errors 122 partners can use this special purpose
Always Active
Your data can be used to monitor for and prevent unusual and possibly fraudulent activity (for example, regarding advertising, ad clicks by bots), and ensure systems and processes work properly and securely. It can also be used to correct any problems you, the publisher or the advertiser may encounter in the delivery of content and ads and in your interaction with them.
Deliver and present advertising and content 126 partners can use this special purpose
Always Active
Certain information (like an IP address or device capabilities) is used to ensure the technical compatibility of the content or advertising, and to facilitate the transmission of the content or ad to your device.
Match and combine data from other data sources 94 partners can use this feature
Always Active
Information about your activity on this service may be matched and combined with other information relating to you and originating from various sources (for instance your activity on a separate online service, your use of a loyalty card in-store, or your answers to a survey), in support of the purposes explained in this notice.
Link different devices 67 partners can use this feature
Always Active
In support of the purposes explained in this notice, your device might be considered as likely linked to other devices that belong to you or your household (for instance because you are logged in to the same service on both your phone and your computer, or because you may use the same Internet connection on both devices).
Identify devices based on information transmitted automatically 116 partners can use this feature
Always Active
Your device might be distinguished from other devices based on information it automatically sends when accessing the Internet (for instance, the IP address of your Internet connection or the type of browser you are using) in support of the purposes exposed in this notice.
Save and communicate privacy choices 103 partners can use this special purpose
Always Active
The choices you make regarding the purposes and entities listed in this notice are saved and made available to those entities in the form of digital signals (such as a string of characters). This is necessary in order to enable both this service and those entities to respect such choices.
have your say