AI makes the humanities more important, but also weirder

Just like AI seems like it's being used as a convenient scapegoat for the layoffs and trimming following the end of ZIRP, now we see it also being used to blame for the failures of our modern education system.

Mostly which are that our system only rewards one thing in education: the grade. Not understanding, knowledge, intelligence, but instead a single number that is more easily gamified than anything. And this single number (your GPA) is the single most important thing for every level from middle school to college where it will unironically determine your entire (academic/academic-adjacent) future.

> Well, now nobody can really even say what jobs will be available in 5 to 10 years with high confidence.

I am very sure that those jobs that have been existing for a long time will continue existing for a long time (and no: even if some disruption occurs, these jobs won't suddenly disappear, but will rather phased out slowly; you thus have sufficient time to make a decent plan for you).

In other words: in my opinion one can predict rather well many jobs that will be available in 5 to 10 years.

The "inconvenient" truth rather is that many high-paying jobs in the new economy sector don't satisfy this criterion of "existing for a long time". So by this criterion you might miss out some hard to predict high-earning opportunities. Thus, if you are the kind of person who tends to easily become envious if your friends suddenly experience a windfall, such a job perhaps won't make you happy.

Trades would collapse if even 10% of the population switched to them. What does no one get about this?

> I don't see how we can blame them rather than ourselves for the educational and career system we've created

I dont know man, you do it every time.

Nobody ever could say with confidence what jobs will pe available. those who know the basics and flexible can find something.

IN RESPONSE TO: There's a deeper educational problem that's been decades in the making which is that students are trained to see school and work as a series of never ending goals to achieve. The ultimate one is to 'get a job'.

This one is really bad. I have waves of local grads begging for unpaid internships or work now. They all were AP students, good GPAs, good schools, "learned to code", took on 200k of college debt -- only to find the promised job is just not frigging there at the end of the line. Meanwhile, they are told there is a "massive shortage of coders" and see overseas workers hired into those same jobs. How do younger people trust the system anymore? Further, in light of this, why would any steel-worker (metaphorically speaking) want to retrain and learn to code if even the geeks learning to code face dismal outcomes?

This is rooted in the idea that a person should need permission to work. The reason we do this is because our capitalist society is structured around competition instead of collaboration. Because all work must be framed as a competition, the only way collaboration is allowed is by explicitly joining a team.

This rule is made explicit with copyright: derivative work is, by default, illegal. The only way that derivative work is allowed to be made is after making a contract with the rightsholder.

This system can only be enforced with incompatibility. You can't actually stop someone from making derivative work unless your help is actually required. The most familiar instance of this requirement is software: it's simply not pragmatic for someone to extend your code, when they only have access to a compiled binary. Software work can only be extended from a shared context: what you write must be compatible with what is written. That's generally unfeasible when what is written is a compile-time optimized binary executable.

Incompatibility is not a perfect barrier. It's possible to decompile software. It's possible to edit text, images, video, audio, etc. Copyright depends on incompatibility, and LLMs are simply the newest, most compelling way to call that bluff.

Most students see education as a video game - the goal is to score as many points as possible in a framework controlled by some apparently arbitrary rules. That educators fail to distinguish school from video games is mostly the fault of educators. And video games are more fun.

> Designing "ChatGPT-proof assignments" feels to me like a similar level of intellectual challenge.

Designing a "mostly ChatGPT-proof class" is in my opinion actually rather simple: just be inspired by the German university system:

Each week you have to complete exercise sheets which are often hard. But being capable of solving a certain quota of the exercises is just the beginning: this only qualifies you for being allowed to take the actual (oral or written) exam.

In this sense (as many professors will tell you), the basically sole purpose of the quota for solving (often hard) exercise sheets is actually preventing students from the "self-inflicted harm" of doing an exam that they are not yet prepared for.

And yes: while it is forbidden to "cheat" (e.g. ChatGPT) on your exercise sheets, this policy is typically not strongly enforced (you will nearly always just get a unequivocal, strongly worded oral reprimand by the tutor). Instead, if you did this, you will for sure not be prepared for the upcoming exam, and flunk it (and most students are very aware of this).

Oh yeah, I didn't mention yet that if you flunked the same exam typically 3 times (depending on the university), you have "finally failed" (endgültig nicht bestanden), and are not allowed anymore to study the same degree course at every German university.

> where all you got was the outline of the island

As a blind person I can't help but notice that this "innovative" question design is inherently inaccessible to people like me. Yes, I know I'm not the norm. However, I can't help but notice that this avoidance of text based assignments is making accessible teaching even more impossible. Say welcome to a new generation of Digital Divide.

Among others, Howard Rheingold has been active in this space. For those interested, check out the Peeragogy Handbook and the post that sparked the idea[0].

> The more I give my teacher-power to students and encourage them to take more responsibility for their own learning, the more they show me how to redesign my ways of teaching

[0]: https://clalliance.org/blog/toward-peeragogy/

> Designing "ChatGPT-proof assignments" feels to me like a similar level of intellectual challenge.

This is indeed true. But a challenge is that few professors are being given the time and training to help do this. When you are on a 4/4 and just keeping your head above water you don't tend to have enough time to adopt experimental pedagogies and completely replan your courses. And professors are largely being tasked with doing this independently rather than having universities offer training or support with adjusting methods to be more AI resistant. And unfortunately the fast speed of development of these tools is making good ideas obsolete quickly. I know some people who switch to having students make podcasts rather than writing papers as a final assignment and then we started seeing "create your own podcast" tools appear and made it roughly as easy to cheat on this assignment as a traditional paper.

If a teacher knows the topic that they're teaching, then a 30 second talk with a student is enough to determine if the student actually knows anything. Maybe "assignments" aren't the best method of building and verifying knowledge.

As a professor of Computer Science, I'm about ready to allow my students to use AI -- if you can't beat 'em, join 'em. But they're not going to like it, because what I'm going to do is significantly raise the bar.

For the last several years I've been allowing students (college sophomores) to do take home exams where they essentially have to build a networked system in C++. I tell them not to use AI but I have to assume they are.

Even still, the solutions are not all that great, despite unlimited time and resources, they still submit work that falls into a roughly B average. Which, if they didn't have AI, maybe that would be a C average, so perhaps the standards just need to go up. Still, it's not like they're all getting 100%.

AI is still really only as good as the person driving it, and it still, despite all the hype, hallucinates like crazy, such that if they don't pay close attention and constantly course correct, they can't expect the project to actually work.

I find it difficult to think of examples that aren't just based on niche information (what did we talk about on this day? what analogy does the teacher make repeatedly?). Maybe multimodal stuff?

That comment struck me as odd if only because Plato is not commonly part of the standard critical thinking curriculum. Introduction to philosophy or history of philosophy yes, but critical thinking no.

This is not to say that Plato didn't think critically. Of course he did, as do all (or most) philosophers. I'm just talking about university courses and textbooks.

Yeah, that does sound like a good idea and a good way to prove one has learned the ideas presented in the course and the relationship between them well

“Cannot be done with chat gpt”

Is a military base the only place “no devices” can be enforced or something? How deep is this addiction? I’m scared to ask what the academic version of fizz-buzz would be at the end of a 4 year degree, “hey write one paragraph describing a simple paradox, an example of irony, or an example of a metaphor”.

Reminds me of https://xkcd.com/810/

[dead]

[dead]

I am a zoomer. Is it really that hard for everyone to NOT use AI? Using AI very clearly atrophies your cognitive skills.

> It's not about how to use history as an aid to prediction.

This is not what academic historians generally do. The study of history tells us about the story of humanity and exists for wider reasons than direct application to decision making today. This is even true for military history, which is among the slowest subfields at adopting new methods and practices.

> But what he describes as teaching history is more like history appreciation. It's not about how to use history as an aid to prediction. It's more about studying the "classics", from Cicero onwards.

How are those things dissociated?

I always saw history as a waht to explain "why things are the way they are". Whatever we are now, this amalgamation of good or bad things, is a consequence of what came before.

Is "studying the classics" from a historical standpoint even done for "appreciation"? I always presumed that one should do so critically, and that's more or less what I found from actual historians.

> It's not about how to use history as an aid to prediction.

Because this isnt the goal of history. You cant use history to predict the future.

> If you want to understand Cicero's era, for example, read [1]. This is a study of Cicero by an experienced and cynical newspaper political reporter. The author writes that too many historians take Cicero's speeches at face value. The author knows better, having heard many politicians of his own day.

No they dont. This is yet another example of an outsider looking at another field, thinking he knows better and making a fool of himself. Also, why would you ignore 80 years of scholarship about Cicero and read a book from 1942???

Studying the winners is also a surefire way to learn only survivorship bias.

It makes a good point by separating history as strategic analysis from history as cultural appreciation. Most teaching today leans toward the latter, which is also easier for AI to replicate. But the most valuable historical thinking often comes from uncomfortable questions failure, unintended consequences, and perspectives that usually get ignored.

> Military officers study much history, but in a different way. They look for mistakes.

Amongst other things...

When history is approached as a pattern of decisions, risks, and consequences (not just narratives) it starts to feel a lot more relevant to other complex domains, including AI itself. And yeah, studying the losers (or the overlooked) often yields more insight than lionizing the winners.

I agree that professors can probably develop ai resistant evaluations.

But there is minimal institutional support for this. Everybody is going it on their own. And the ai tools are changing rapidly. Telling a ntt faculty member teaching a 4/4 that they just need to stop being lazy and redo their entire evaluation method is not really workable.

Iteration is also slow. You do a new syllabus to try to discourage cheating. You run the course for a semester. You see the results with now just a few weeks to turn around for your spring semester courses. At best you get one iteration cycle per six months. More likely it is one per year since it is basically impossible to meaningfully digest the outcome of what you tried and to meaningfully try something different mere weeks later.

What needs to happen is a return to a more Socratic method of teaching the humanities.

Instead of being about the consumption and production of texts -- texts which only the TA and maybe the prof will read ... ever -- the class should be more about the dialectical process of discussing the ideas in the texts and lectures the students consume.

An LLM will be able to easily rattle off an undergrad level paper. But it can't sit in the classroom and have an intelligent conversation with peers.

Of course none of this will happen because it doesn't scale economically nearly as well. It is far more labour intensive.

Yeah, I've been thinking about when the calculator came to bear. I was in high school physics when (rich) kids started bringing in "scientific" calculators. There was a flap then about whether they should be allowed at all.

I'm not sure they are analogous to LLMs, but a compelling argument at the time was that the students were going to use them in the field anyway. (That certainly seems to be the future for software engineering students.)

I think if you removed the economic incentive to cheat then humanities wouldn't have the problem it does with LLMs

equating chatgpt to "a better spell checker" is wild; how is writing not also a skill that should be taught?

The transformer architecture was originally developed for translation tasks, but massive, overfit generative models are awful at translation. (Seriously, part-of-speech classification + dictionary lookup + grammar mapping – an incredibly simplistic system with performance measurable in chapters per second – does a better job, and you even get confidence intervals.) If you want a translation tool, use something like Project Bergamot (https://browser.mt/), not generative AI.

> Unfortunately, so much of the humanities in high school is treated as an exercise rather than real discovery.

I, too, find this exceptionally annoying.

The history we have inevitably passes through bottlenecks. Much is left out, edited away, embellished.

We have no idea what happened over 500 years ago, but the idea of a brilliant scholar translating the Greek texts on behalf of the Medicis, shouldn't simply be accepted as stated. If you are running the world (like the Medicis did) history would be a lever of control. It seems inevitable that the stories should be directed (carefully) towards whatever ends.

History is actually a present day activity - it provides the backdrop to the present. Altering that backdrop has present value.

I can't see how ai will help us the endeavour of trying to get a better handle on what happened in the past. I do see how it would provide modern-day Medicis a way to change the backdrop more quickly.

I think all PhDs have an oral defense. Not only that its common to have Quals where you have to present the state of the art research in your field and answer oral questions about it. Even if you had ChatGPT it would be tough because a lot of questions are like, "Why did XYZ do ABC after seeing result 123?" The problem is that often its an untrue statement they are trying to get you to justify -- XYZ didn't do ABC or didn't do it after seeing 123. This is something I imagine most LLMs to struggle with today and say, "No, that didn't happen -- here's actually what happened" when its in a nuanced field.

> When the basic coding skill (or any skill) is commoditized, it's the people with complementary skills that benefit the most.

I think that "knowing how to ask good questions" that you then solve has always been a valuable skill.

The kids are not even trying to do either. They are already gravitating towards multidisciplinary teams, cause unlike past generations they are dealing with a rate of change at a totally different level. In such an environment, people get to see their own limitations much faster no matter the quality of their training and they end up having to rely on others.

The big challenge is getting very different people with ever growing different skillsets and interests to coordinate, stay in sync and row in one direction.

>The emerging generation of historians will simply take it for granted that they can create their own custom research and teaching tools and deploy them at will, more or less for free.

And they will spend 12 hours trying to figure out which is the fake python library and the citation that the LLM has hallucinated from the real one. Vibe coding is just WYSIWYG on steroids in good and bad. WYSIWYG didn't go anywhere.

The field of history had their - oh, this is a thing - moment quite a few decades ago with Hayden White. It is a good example of the underlying issue.

Hayden White's assertion that "history is fiction" was (and still is) a complex one, not intended to dismiss the factual accuracy of historical narratives (as it is more often than not portraied).

Instead, it highlights the interpretive nature of historical writing and the way historians shape their accounts of the past through literary and rhetorical techniques. White argues that historians, like novelists, use narrative structures and stylistic devices to construct meaning from historical events.

The failures we see in AI systems usually come from neglecting real-world complexity

Basically a reformulation of Joel Spolsky's "commoditize your complement".

That's what would happen if it was a logical system. It's not. This is where it gets interesting.

Instead, it's a statistical model, and including that prompt is more like a narrative weight than a logical demand. By including these words in this order, the model will be more likely to explore narratives that are statistically likely to follow them, with that likelihood determined by the content the model was trained on, and the extra redistribution of weights via training.

We don't really need to worry about technically misstating our objectives to an LLM. It doesn't follow objectivity. Instead, we need to be concerned about misrepresenting the overall vibe, which is a much more mysterious task.

I think the main subject of that hole is objectivity, and it affects much more than education.

By obsessively structuring education around measurement, we have implied that what is taught is objective fact. That's always been bullshit, but - for the most part - it's been consistent enough with itself to function. The more strict your presentation of knowledge is, the more it can pretend to be truly objective. Because a teacher has total authority over the methodology of learning, this consistency can be preserved.

The reality has always been that anything written is subjective. The only way to learn objective fact is to explore it through many subjective representations. This is obvious to anyone who learns mathematics: you don't just see x^2+y^2=z^2, and suddenly understand the implications of the Pythagorean theorem.

Because objectivity can't be written, objectivity is not computable. We can't give a computer several subjective representations of a concept, and have it figure it out. This is the same problem as ambiguity: there is no objectively correct way to compute an ambiguous statement. This is why we write in programming languages: each programming language grammar is "context-free", which means that everything must be completely and explicitly defined. To write a computable statement, we must subject the abstract concept in mind to the language (grammar and environment) we write it in.

Most of the writing in our society is done with software. Because of this, we are implicitly encouraged to establish shared context. If what we write is consistent with the context of others' writing, it can pretend to be objective, and be more computable. Social interactions that are facilitated by software are also implicitly structured by that software's structure. There can be no true free-form dialogue in social media.

The exciting thing about LLMs is that they don't have this problem. They conveniently dodge it, by not doing any logic at all! Now instead of the software subjecting you to a strict environment, it subjects you to a familiar one. There is no truth, only vibes. There is no calculation, only guesses. This feels a lot like objectivity, but it's just a new facade. The difference is that the boundary is somewhere new and unfamiliar.

---

I think the death of objectivity is good progress overall. It's been a long time coming. Strict structure has always been overrated. The best way to learn is to explore as many perspectives as you can find. We humans can work with vibes and logic, together.

For a long time, schools assumed that if a student turned in a written essay, it meant they had learned something. But now AI can write those essays too ,so that assumption doesn’t hold up anymore.

The real question is: if writing alone doesn’t prove learning, then what does?

Maybe we’ll see a return to oral exams or live discussions. Not because they’re perfect, but because they’re harder to fake.

In a way, AI didn’t ruin education it just exposed problems that were already there.

The definition of learning has not changed. One of our first written records is a man complaining that this newfangled writing thing will make the students lazy, they will no longer have to remember their studies.

Education will adapt.

A shockingly large percentage of education in the US is teaching kids to produce bullshit essays, so it's no surprise that AI is blowing a hole in it.

> This makes me doubt that you learn anything if laptops or tablets are involved.

This may be true for you, but it certainly isn't generally true.

I haven't written anything substantial on paper in years, and I certainly have learned new things in that time.

I think this comes down to how you first learnt to learn. If I write notes in nodepad i dont remember them as well as if i write them by hand, but I think that's just a byproduct of years of school, and other people have learnt their own ways and are just as effective.

There's an entire generation that is evidence that your personal case is not the general case.

Gamified learning apps, in a broad sense, seem to be very effective for _practice_.

(source: am a literature student - UK open university)

I just asked get ChatGPT a question about use of language in a specific book (China Miéville's Kraken). It gave a plausible answer but used a common English word ('knackering') instead of the word 'knacking' that is actually used (the word 'knackering' isn't in the text). I then asked it to give me a quote using 'knackering' and it gave me a quote with the word 'knacking'. I asked it to put the quote into Harvard reference style which it did, but didn't provide a page number, which it said I'd have to look up myself.

I think relying on ChatGPT for an essay would be fairly frustrating, and if you used ChatGPT to develop your thesis, you could very quickly get a generic plagiaristic answer (based on the quotes everyone else used) rather than one than that captured your own response to the text, and which contained factual errors.

[Edit] It also said that "gods inhabiting USB sticks are presented matter-of-factly" in Kraken which is false (there are no mentions of USB). When I asked it for a quote to support its assertion, it replied "While the novel doesn't explicitly depict gods inhabiting USB sticks..."

OK, that last sentence would be incredible. I think this is tongue in cheek, but I was an English lit major so I feel the need to defend my honor. XD

I don’t think you’d do well in 400-level classes this way. English Lit isn’t as much of a joke as STEM students make it out to be[1]: it gets a lot harder than the bullshit 101/201 courses everyone is required to take. You’re supposed to try to be original in your analysis, and it has to be rigorously proven within the text itself.

Probably as a grad student you’d start arguing against other critics points, but not undergrad. I think that would hold for almost all schools because no one at that level in that field wants to hear from someone who doesn’t know how to analyze a text in the first place. It’d be like a high school student trying to tell you about software (or systems, network, data, etc.) engineering.

For similar reasons, AI summarizations for past contributions wouldn't work, either. If you’re arguing someone else’s analysis is wrong, you’re going to need to read and understand the whole thing. And if you’re just copying from AI, you’ll have a hard time defending your position.

Although, man, if you can understand the subtext of a book from listening to an audiobook *while gaming*, AND you have time to watch all the online lectures about a book!? I need to talk to you about time management, my friend!

1 - I have been involved in so many forums and subreddits where people try to analyze books, comics, TV, or movies. Based on what I have seen come out of people there —- most of you MFers couldn’t pass 300-level classes.

People can’t analyze literature for shit, and I think it’s because everyone gets such a negative perception of literary analysis because high school and required college classes are junk. It’s actually really hard to read five novels in month, keep track of all the characters and plots and themes and so on, and understand all of them well enough to write a coherent argument. I saw so many kids in my major crying about their GPA getting tanked because they weren’t ready for rigorous analysis. FWIW, I was 25 as a Junior (third year of uni, in the US), and had spent the last few years reading exercise physiology papers while bored at work. Seeing real science changed my life, and I wanted to apply their level of rigor to my own analysis of any kind. It’s why I’m good at my job now, tbh.

> Consume the classics in audiobook form while flâneuring or gaming

I can't tell if you're serious or not, but this would be like saying you've "watched" a movie because you had it on your second monitor while you were playing counter strike. There's a fundamental difference in hearing a book read to you and actually listening and engaging with the audio on a meaningful level. The latter requires focus, and you're not going to be able to do that if you're gaming away at the same time.

Put another way, why would someone who presumably loves reading bother studying literature if they don't actually care enough to pay attention to the books they supposedly loved?

What a wonderful bit of satire.

To those that missed the joke - 'consuming' the classics is the antithesis of a liberal arts education. The value lies in the engagement, the debate, the Hegellian dialectic involved in arriving at a true grok level understanding of the text or topic.

It would be akin to reading the Sparknotes of Ulysses and being able to reference how it draws heavily on Homer's Odyssey, or utilises stream-of-consciousness narrative to great effect; and thinking that, as a result, you have the faintest understanding of the text, its conception, or its impact.

The OP almost hit on this with the 'Listen to all the online lectures from Oxford,Yale,MIT etc...'. Unlike coding bootcamps or similar, universities are not VOCATIONAL TRAINING - no matter how skewed towards that end-goal the American Economy is dictating such. As just about any Educator can attest, no amount of listening to youtube lectures will replace the University experience, nevermind the Oxbridge/Ivy League experience.

The pedagogical benefits are simply unrealisable from an AI prompt 'streamlining'- i.e. being forced to read and engage with topics outside of your comfort zone to maintain your GPA, engaging and working with people from a diversity of outlooks and backgrounds, benefitting from the 1:1 and small group sessions with the Academics who often wrote the literal book on the subject in question.

If the intersection of JSTOR and Machine Learning didn't reduce humanities to Cory Doctorow level script-kiddyism, the hoi polloi throwing prompts into a hallucinatory markov chain isn't likely to advance or diminish Academia anymore than the excess of 'MBA IN 5 DAYS!' or '...for Dummies' titles previously available.

You can do this without being in a formal college program, go for it!

Sounds like all ways to skirt around the actual learning bit. Consuming the classics while gaming? Really?

That is the reverse turing test: https://www.youtube.com/watch?v=MxTWLm9vT_o

Will the AI find ALL the occurrences. Finding hallucinated ones is OK here as the user can check each one.

And for each of those dozens, there are now millions of untalented people who can generate terrible films with a few clicks. How does a Tarantino, or even one of your new talents, ever get discovered?

The friction is a feature, not a bug.

A lot of my engineering exams didn't allow certain calculators, so I guess that's what will happen for humanities, supervised and timed essay writing.

I remember going to museums and seeing all the modern art, and wondering why art went to such weird places compared to the realism of the 1600's, and my conclusion was that invention of photography made realism obsolete. I wonder if something similar will happen to the humanities.

> There's a real risk of LLMs flattening the educational journey into a kind of performance optimization game

It's already been the case for a long time, it's just going to get worse.

We've already begun to normalize the belief that humans hallucinate as often as LLMs, and are categorically less reliable and less trustworthy than LLMs. The political and social ramifications of this are bound to be negative.

That reminds me of a professor that would play various forms of music compressed in different ways. The class actually preferred the distortion produced by mp3 over uncompressed.

The hype cycle is running on FOMO. "as a developer, you will be left behind" And is sustained by fear tactics. "there wont be any developer jobs left"

Giant investments are being made, and a lot of people's outcomes rest on them, which incentivizes them to become evangelists for the cause.

End-of-day it is a productivity boosting tool like a search-engine or a calculator - whats funny to see the human drama playing out around its adoption.

> Can we make a place for the social ladder climber that's separate from a place where real learning happens, where nobody is tempted to cheat because there's nothing to be gained from cheating?

I don't think so. Let me suggest a set of hypotheses:

- the social ladder always exists; it's basically fundamental to human existence, especially when trying to do anything (see tyranny of structurelessness)

- people are always trying to climb the hierarchy

- the examocracy(+) was introduced by ancient Chinese and/or Prussia in an attempt to bring actual competence into the social hierachy, in large part to help the leader fight wars

- if there isn't a "fair" route to the top, competent outsiders will resort to unfair ones, which can be remarkably lethal and destructive

- the university provides its own small academic and intellectual hierarchy, but this exists in dialogue with the needs of a wider society and its demand for non-academic intellectually specialized work. Whether that's priests, lawyers, doctors, army officers, weapons designers, civil servants, or programmers

- this comes down to: does the wider society demand human competence in intellectual operations still, or does ALL of that get AI-automated away? Do we get "humans, but with a smarter Clippy" or do we get "AI runs the world for billionaires and everyone else is surplus peasantry".

- the secret third, worse, case is that AI destroys the concept of competence and things stop working on a large scale.

(+) I do not want to have a huge fight over the word "meritocracy"