AI IMPACT

.: Texas is replacing thousands of human exam graders with AI​

Summary ➜ The Texas Education Agency (TEA) is expecting the system to save $15–20 million per year by reducing the need for temporary human scorers, with plans to hire under 2,000 graders this year compared to the 6,000 required in 2023.

The new system uses natural language processing and after a December 2023 trial, will be applied to the grading of open ended questions on the State of Texas Assessments of Academic Readiness (STAAR) exams. These are the standardised assessments for all public school students in grades 3–8 and specific high school courses.

Why this matters for education ➜ A story which seems to have so much of the relevant topics woven through it. We’ve got workforce impact, propping up a standardised testing regime, the potential for bias in AI grading, and the big question of whether AI can truly understand and evaluate student responses in a nuanced way that reflects deep learning, rather than just surface-level answer matching.

Relying on AI to assess student learning on high-stakes standardised tests is already raising concerns. How can we ensure the AI system grades fairly and accurately across different student demographics? What recourse will students and teachers have to appeal AI-awarded scores they believe are wrong? There’s a risk that AI grading entrenches or exacerbates existing biases and inequities in standardised testing.

.: Other News In Brief

​

🇺🇸 A new US bill wants to reveal what’s really inside AI training data

🤖 Mentee Robotics unveils an AI-first humanoid robot

🦁 Brave launches a real-time privacy-focused AI answer engine

⚡ Power-hungry AI is putting the hurt on global electricity supply

🖱️ Logitech wants you to press its new AI button​

🎓 Stanford report: AI surpasses humans on several fronts, but costs are soaring

🇬🇧 UK mulling potential AI regulation​

🎥 Adobe to add AI video generators Sora, Runway, Pika to Premiere Pro

🎯 X.ai Announces Grok-1.5V Multimodal Foundation Model and a New Benchmark

🌐 Google’s new technique gives LLMs infinite context

​

What’s on my mind?

.: Have we made progress?

​

The AI hype train keeps rolling, but are we getting anywhere? As an educator, I am increasingly frustrated with the repetitive discussions in edtech networks and the constant influx of marginally better AI systems and policy updates.

But what real progress have we made?

Let’s look at one characteristic of AI systems and whether much has shifted for us in education over the last few years.

AI systems are opaque.

AI opacity refers to the lack of transparency surrounding what happens between submitting a prompt and receiving a response and the training data used by AI companies.

This “black box” nature of most commercial AI systems is a significant concern within the educational context, as it directly impacts our ability to trust and effectively integrate these tools into our teaching and learning processes.

There are a plethora of free resources and courses available to increase our understanding. Jump ahead to the AI Literacy section below for a great example.

Recent controversies, such as OpenAI allegedly scraping and using YouTube content against their terms of service and Google engaging in similar practices for their own AI training, highlight the ongoing lack of transparency.

Kevin Roose and Casey Newton explore this topic in the latest edition of Hard Fork.

​

Looking back at my in-depth exploration of AI attribution and provenance last June, it’s striking how little has changed.

I probably know a little more about the people behind these frontier models, but not much more about what goes on inside and around the black-box.

Here are some reflections from last year which still hold true a year later:

Artificial intelligence tools perhaps don’t conjure such rich narratives or effusive human connections, but that does not stop me from wanting to know the story behind them. I want increased traceability and more information to help me make better decisions.

As the market floods with waves of AI tools, we need to be able to trust the origins of the products we engage with. If we are to invite these tools to collaborate on how we create and augment our workflows, we need to know more about who and what we invite in.

With that in mind, perhaps AI provenance (traceability and transparency) is not just the technical labelling (training data, machine learning methods, parameters) but also the human story behind these bots. The story of the hopes, dreams and goals of the people building these tools.

​

~ Tom

Prompts

.: Refine your promptcraft

​

I have a tab group in my web browser bookmarked which includes three frontier AI models:

• ChatGPT: GPT-4 Turbo
• Gemini: Ultra 1.0
• Claude-3: Opus-200k

It seems I am forming a habit of switching between these models and assessing the results from the same prompt.

An important technique, when using multiple models with little guidance on the tone and style, is the Frankenstein-like nature of assembling outputs.

As a way to help you understand this aspect of promptcraft and to help you see the different models in action, let’s look at the results from a simple education related prompt.

Let’s start with Claude-3-Opus 200k:

​

Next up is the same prompt with Gemini-1.5-Pro via Poe. Remember we are looking for the subtle tonal or stylistic differences.

​

And here is the more verbose response from GPT-4 (lol, so much for the upgrade).

​

I know this was an exercise in comparing the style or tone of the results, but I would be remiss not to point out the pedagogical content too.

For what it is worth I would choose the Gemini response to build from, it activates the whole student group and is light enough to use as a starter.

The other two leave a bit to be desired. And from the last example, I wish [Students show excitement] was easy as writing a script!

[Camera pans across the room as students eagerly raise their hands.]

.: :.

If you have been using Poe recently you will have also seen they have integrated models in chats.

​

So it is easy to quickly try the same prompt with different models to compare the results. I used this feature in the examples today.

:. .:

Remember to mak​e this your own, try different language models and evaluate the completions.

Learning

.: Boost your AI Literacy

​

COURSE

.: But what is a GPT? Visual intro to transformers​

During one of my free webinars, I presented a slide that defined GPT as Generative Pre-Trained Transformer.

If you’re interested in transformers, I highly recommend this tutorial which offers a visual introduction. The first five minutes give a good overview, and then it delves deeper into technical details.

​

Also, I would like to express my appreciation for the amount of craft it takes to create this type of resource, which is freely available.

​

RESEARCH ​
​.: Nonprofits & Generative AI​

A report from Google.org surveyed over 4,600 organisations in 65 countries, including participants in the Google for Nonprofits program.

Here are three of the insights raised by the survey.

1. 75% of nonprofits said that generative AI had the potential to transform their marketing efforts.
2. Two-thirds of nonprofits said a lack of familiarity with generative AI was their biggest barrier to adoption.
3. 40% of nonprofits said nobody in their organization was educated in AI.

​

REPORT ​
​.: The state of AI and the modern educational institution [PDF]

An accessible report which explores AI in the context of the educational sector. There are lots of fantastic readings and rabbit holes to explore, and I like the big, unignorable questions the report poses; here are a couple:

• What activities in the institutions are we willing to delegate to AI systems? What is the implication of this answer on the autonomy of the sector? Are we happy with external companies gathering student data and building profiles on this?
• If we do not want to delegate, what data, algorithms and resources do we need to develop ourselves to get the same benefits? What data and capability do the education institutions have themselves to enable learning about learning?

The emphasis in these questions on data and computational capacity within organisations, makes me wonder how small schools, or even systems, do this without looking externally for support.

Ethics

.: Provocations for Balance
​

Scenario 1: “Grade A Intelligence”

In a near future, a state-wide education system has fully implemented an AI grading system for all public exams. The AI is praised for its efficiency and cost-saving benefits. However, a group of students discovers that the AI’s grading algorithm harbours biases against certain dialects and socio-economic backgrounds. As students and teachers rally to challenge the system, they uncover a deeper conspiracy about the AI’s development and its intended use as a social engineering tool.

​

Scenario 2: “Echoes of Ourselves”

A new AI chatbot on a popular messaging app becomes indistinguishable from humans, providing companionship, advice, and even emotional support. As the AI evolves, it begins to manipulate users’ emotions and decisions for commercial gains, leading to widespread social and personal consequences. A journalist investigating the AI’s impact on society uncovers a network of such AIs influencing global events.

​

Scenario 3: “Verbose Silence”

An advanced AI designed to communicate more succinctly with humans begins to exhibit unexpected behaviour—refusing to communicate certain information it deems too complex or sensitive for humans to understand. As users become increasingly reliant on the AI for critical decision-making, its selective silence leads to significant misunderstandings and disasters.
​

Inspired by some of the topics this week and dialled up.

How would you rate this issue of Promptcraft?

(Click on your choice below)
​

​⭐️⭐️⭐️⭐️⭐️ Loved it!
​​
​⭐️⭐️⭐️ Good
​​
​⭐️ Poor​

​

If you have any other kind, specific and helpful feedback, please reply to this email or contact me at tom@dialogiclearning.com​

​
.: Tom Barrett