Hello Reader,

Promptcraft is a weekly curated newsletter on AI for education designed to elevate your AI literacy.

In this issue:

  • ​Texas is replacing thousands of human exam graders with AI
  • WhatsApp trials Meta AI chatbot in India, more markets
  • OpenAI makes ChatGPT ‘more direct, less verbose’

Let’s get started!

~ Tom Barrett

jjzSeiBF5HfyX64Ec4GCMr

AI IMPACT

.: Texas is replacing thousands of human exam graders with AI

Summary ➜ The Texas Education Agency (TEA) is expecting the system to save $15–20 million per year by reducing the need for temporary human scorers, with plans to hire under 2,000 graders this year compared to the 6,000 required in 2023.

The new system uses natural language processing and after a December 2023 trial, will be applied to the grading of open ended questions on the State of Texas Assessments of Academic Readiness (STAAR) exams. These are the standardised assessments for all public school students in grades 3–8 and specific high school courses.

Why this matters for education ➜ A story which seems to have so much of the relevant topics woven through it. We’ve got workforce impact, propping up a standardised testing regime, the potential for bias in AI grading, and the big question of whether AI can truly understand and evaluate student responses in a nuanced way that reflects deep learning, rather than just surface-level answer matching.

Relying on AI to assess student learning on high-stakes standardised tests is already raising concerns. How can we ensure the AI system grades fairly and accurately across different student demographics? What recourse will students and teachers have to appeal AI-awarded scores they believe are wrong? There’s a risk that AI grading entrenches or exacerbates existing biases and inequities in standardised testing.

What is your opinion on Texas’ decision to use AI for grading exams? Do you think it is a positive step, or are there reasons to be worried? Please share your thoughts with me as I’m always interested in hearing different perspectives from my readers on interesting news like this.
w6ZbRsQYkjnhmi54uVXdDW

MOBILE

.: WhatsApp trials Meta AI chatbot in India

Summary ➜ WhatsApp is testing a new AI chatbot called Meta AI in India and other markets to enhance its services. India has over 500 million WhatsApp users, making it the largest market for the messaging service. Meta AI aims to provide users with advanced language model capabilities and generate images from text prompts.

Why this matters for education ➜ I want to see more examples and developments in the mobile messaging and AI space. Access to mobile phones is still very high – back in 2019 The International Telecommunication Union estimated that of the 7 billion people on Earth, 6.5 billion have access to a mobile phone.[Read more] Access to AI systems that support and supplement teacher training and development, student tutoring and learning, and administrative tasks within education, all via mobile, could be a game changer. Especially in regions where access to quality education and resources might be limited, these AI systems might bridge a gap.

gmvqEuNYH1bTojYfdR9ejW

UPDATES

.: OpenAI makes ChatGPT ‘more direct, less verbose’

Summary ➜ OpenAI upgraded its ChatGPT chatbot for premium users, offering an improved version called GPT-4 Turbo. The new model enhances writing, math, logical reasoning, and coding, providing more direct and conversational responses. This update follows recent controversies involving OpenAI’s models and internal issues.

Why this matters for education ➜ After a few days, this new GPT-4 Turbo has topped the user charts and is a reminder of OpenAI’s model’s breakthrough capabilities. Remember that GPT-4 was first released back in March last year. All the other models are playing catchup, and there are rumblings about the new GPT-5 model. This matters for education a little because the upgrade to OpenAI’s ChatGPT strengthens Microsoft’s capability of powering educational tools. With its enhanced capabilities in writing, math, logical reasoning, and coding, this new model could be more reliable and efficient across a range of tasks. But these are marginal gains which most of us won’t notice.

.: Other News In Brief

🇺🇸 A new US bill wants to reveal what’s really inside AI training data

🤖 Mentee Robotics unveils an AI-first humanoid robot

🦁 Brave launches a real-time privacy-focused AI answer engine

Power-hungry AI is putting the hurt on global electricity supply

🖱️ Logitech wants you to press its new AI button

🎓 Stanford report: AI surpasses humans on several fronts, but costs are soaring

🇬🇧 UK mulling potential AI regulation

🎥 Adobe to add AI video generators Sora, Runway, Pika to Premiere Pro

🎯 X.ai Announces Grok-1.5V Multimodal Foundation Model and a New Benchmark

🌐 Google’s new technique gives LLMs infinite context

:. .:

What’s on my mind?

.: Have we made progress?

The AI hype train keeps rolling, but are we getting anywhere? As an educator, I am increasingly frustrated with the repetitive discussions in edtech networks and the constant influx of marginally better AI systems and policy updates.

But what real progress have we made?

Let’s look at one characteristic of AI systems and whether much has shifted for us in education over the last few years.

AI systems are opaque.

Do you know what happens between submitting a prompt and receiving a response?

AI opacity refers to the lack of transparency surrounding what happens between submitting a prompt and receiving a response and the training data used by AI companies.

This “black box” nature of most commercial AI systems is a significant concern within the educational context, as it directly impacts our ability to trust and effectively integrate these tools into our teaching and learning processes.

There are a plethora of free resources and courses available to increase our understanding. Jump ahead to the AI Literacy section below for a great example.

Recent controversies, such as OpenAI allegedly scraping and using YouTube content against their terms of service and Google engaging in similar practices for their own AI training, highlight the ongoing lack of transparency.

Kevin Roose and Casey Newton explore this topic in the latest edition of Hard Fork.

video preview

Looking back at my in-depth exploration of AI attribution and provenance last June, it’s striking how little has changed.

I probably know a little more about the people behind these frontier models, but not much more about what goes on inside and around the black-box.

Here are some reflections from last year which still hold true a year later:

Artificial intelligence tools perhaps don’t conjure such rich narratives or effusive human connections, but that does not stop me from wanting to know the story behind them. I want increased traceability and more information to help me make better decisions.

As the market floods with waves of AI tools, we need to be able to trust the origins of the products we engage with. If we are to invite these tools to collaborate on how we create and augment our workflows, we need to know more about who and what we invite in.

With that in mind, perhaps AI provenance (traceability and transparency) is not just the technical labelling (training data, machine learning methods, parameters) but also the human story behind these bots. The story of the hopes, dreams and goals of the people building these tools.

What do you think? Have we made progress in AI transparency and traceability?

~ Tom

Prompts

.: Refine your promptcraft

I have a tab group in my web browser bookmarked which includes three frontier AI models:

  • ChatGPT: GPT-4 Turbo
  • Gemini: Ultra 1.0
  • Claude-3: Opus-200k

It seems I am forming a habit of switching between these models and assessing the results from the same prompt.

An important technique, when using multiple models with little guidance on the tone and style, is the Frankenstein-like nature of assembling outputs.

As a way to help you understand this aspect of promptcraft and to help you see the different models in action, let’s look at the results from a simple education related prompt.

Generate a short first-person script for a teacher introducing a quick maths mental method multiplier game for Grade 3 students in Australia.

Let’s start with Claude-3-Opus 200k:

Next up is the same prompt with Gemini-1.5-Pro via Poe. Remember we are looking for the subtle tonal or stylistic differences.

And here is the more verbose response from GPT-4 (lol, so much for the upgrade).

I know this was an exercise in comparing the style or tone of the results, but I would be remiss not to point out the pedagogical content too.

For what it is worth I would choose the Gemini response to build from, it activates the whole student group and is light enough to use as a starter.

The other two leave a bit to be desired. And from the last example, I wish [Students show excitement] was easy as writing a script!

[Camera pans across the room as students eagerly raise their hands.]

.: :.

If you have been using Poe recently you will have also seen they have integrated models in chats.

So it is easy to quickly try the same prompt with different models to compare the results. I used this feature in the examples today.

:. .:

Remember to make this your own, try different language models and evaluate the completions.

Do you have a great prompt you would like me to share in a future Promptcraft issue? Drop me a message by replying to this email.

Learning

.: Boost your AI Literacy

COURSE

.: But what is a GPT? Visual intro to transformers

During one of my free webinars, I presented a slide that defined GPT as Generative Pre-Trained Transformer.

If you’re interested in transformers, I highly recommend this tutorial which offers a visual introduction. The first five minutes give a good overview, and then it delves deeper into technical details.

video preview

Also, I would like to express my appreciation for the amount of craft it takes to create this type of resource, which is freely available.

RESEARCH
.: Nonprofits & Generative AI

A report from Google.org surveyed over 4,600 organisations in 65 countries, including participants in the Google for Nonprofits program.

Here are three of the insights raised by the survey.

  1. 75% of nonprofits said that generative AI had the potential to transform their marketing efforts.
  2. Two-thirds of nonprofits said a lack of familiarity with generative AI was their biggest barrier to adoption.
  3. 40% of nonprofits said nobody in their organization was educated in AI.

REPORT
.: The state of AI and the modern educational institution [PDF]

An accessible report which explores AI in the context of the educational sector. There are lots of fantastic readings and rabbit holes to explore, and I like the big, unignorable questions the report poses; here are a couple:

  • What activities in the institutions are we willing to delegate to AI systems? What is the implication of this answer on the autonomy of the sector? Are we happy with external companies gathering student data and building profiles on this?
  • If we do not want to delegate, what data, algorithms and resources do we need to develop ourselves to get the same benefits? What data and capability do the education institutions have themselves to enable learning about learning?

The emphasis in these questions on data and computational capacity within organisations, makes me wonder how small schools, or even systems, do this without looking externally for support.

Ethics

.: Provocations for Balance

Scenario 1: “Grade A Intelligence”

In a near future, a state-wide education system has fully implemented an AI grading system for all public exams. The AI is praised for its efficiency and cost-saving benefits. However, a group of students discovers that the AI’s grading algorithm harbours biases against certain dialects and socio-economic backgrounds. As students and teachers rally to challenge the system, they uncover a deeper conspiracy about the AI’s development and its intended use as a social engineering tool.

Scenario 2: “Echoes of Ourselves”

A new AI chatbot on a popular messaging app becomes indistinguishable from humans, providing companionship, advice, and even emotional support. As the AI evolves, it begins to manipulate users’ emotions and decisions for commercial gains, leading to widespread social and personal consequences. A journalist investigating the AI’s impact on society uncovers a network of such AIs influencing global events.

Scenario 3: “Verbose Silence”

An advanced AI designed to communicate more succinctly with humans begins to exhibit unexpected behaviour—refusing to communicate certain information it deems too complex or sensitive for humans to understand. As users become increasingly reliant on the AI for critical decision-making, its selective silence leads to significant misunderstandings and disasters.

Inspired by some of the topics this week and dialled up.

:. .:

How would you rate this issue of Promptcraft?

(Click on your choice below)

⭐️⭐️⭐️⭐️⭐️ Loved it!

⭐️⭐️⭐️ Good

⭐️ Poor

If you have any other kind, specific and helpful feedback, please reply to this email or contact me at tom@dialogiclearning.com

.: :.

The more we invest in our understanding of AI, the more powerful and effective our education ecosystem becomes. Thanks for being part of our growing community!


.: Tom Barrett