Sure, let’s talk about AI again

I’ve taken a bunch of notes and read a bunch of essays and talked to a bunch of people since I wrote Yeah, sure, let’s talk about AI, so let’s see where we stand on the day after April Fool’s day[1], which is the day on which we used to not trust anything we read on the Internet. It turned out to be a harbinger of times when we would not trust any sources on any day of the year from human sources. And, now, because we are going to always suspect that we’ve been fooled into reading a 19-page screed that goes nowhere written in seconds by a machine, and that wasted an hour of our precious lives.

Instead I present a 19-page screed written by my deranged mind. Judge for yourselves whether it “goes nowhere”. Your mileage may vary.

What the heck is going on?

I think AIs will bring us an era of text content that reads like those videos titled “you won’t believe what he does to that girl” that are just 10 minutes with nobody doing anything, and no girl to which that purported thing was to be done in sight.

This is perhaps especially ominous for people like me, who still read more than just the title of articles, and still read actual paragraphs in the hopes of gleaning knowledge. Most of the world probably won’t notice that anything has changed at all, as they scroll through their wall of 11-second–long videos on TikTok.

I, on the other hand, will be worrying that I won’t notice the difference between when I’ve read something genuinely useful and informative and when I’ve read a machine-made distillation of already-existing information that can only ever be less than the sum of its parts, unless by some fluke, it isn’t (but it almost always is).

This doesn’t mean that these types of constructs are really any different than works produced by human minds, who are mostly just copy/pasting content from other sources, in an effort to hit a deadline, collect a payday, and pay their sky-high rent.

It’s just that it will be different because, while a human is capable of imbuing a text with insight, a machine is not. A machine might fortuitously juxtapose information in a way that I will be able to imbue it with content, which might be enough, actually! If this ends up happening often enough, then I suppose that I wouldn’t be able to tell the difference between when an author leads my proverbial and metaphorical horse to water, and when a machine does it.

I’m just worried that, in the past, when someone would actually read the text of an article, the fact that it had been written by a human with a mind capable of imbuing content with meaning, the odds were greater that that person might have learned something useful, if only by accident.

With the texts I’ve seen up until now, those fortuitous accidents are few and far between. I fear that we will run out of runway, so to speak, and land our plane much too early, satisfied with a level of knowledge that doesn’t really force these machines to become better than they are. If we turn our writing over to the machines, they will—just as we’ve trained everything else to do—find a balance between effort and results that will be just satisfactory enough to maximize profit.

They will be trained to generate not knowledge, but satisfaction with their service. This is the inherent flaw, as always: a failure to align incentives with goals. I suppose that’s not quite right. The failure is more that we allow the goals to be set by those whose interest is purely personal rather than in any way communal.

What we mean by AI

I think the first thing to do is to be clear that when people say “AI”, they’re mostly just lumping in a whole bunch of non-AI, but “predictive” technologies and then pretending that there’s a brain behind it. But there’s no man behind the curtain even. It’s just a prediction engine that can only be as good as the input that it’s already seen before. It is capable of combining things in ways that are not in the input, but it is not capable of actual creation.

We should be aware that once a label becomes cool, it becomes valuable. It imbues value simply by attribution without any further work on the part of the product team. So, if you have a service that you can plausibly label as AI these days, then it will be more valuable. Therefore, every shyster in the world will crawl out from under its rock with a sign reading “AI” in its grubby little mitts. No-one really knows what AI means, so everyone can say that they’re providing it. This muddying of water continues until there is some regulation to enforce the language that we use to sell products.

We must remember, ever and always, that all of the things that we hear about and how we hear about them, are to sell us something. We are led to believe that these technologies are being developed as if we were in some communist utopia, where technologies are developed depending on their potential societal usefulness, where entities carefully weight the pros and cons of the impact that a technology might have in both the short and long term, and wherein we sagely and democratically choose what seems to be a sensible path along the potential world-lines.

No, what we have instead is a market-based free-for-all, in which established and spectacularly wealthy players make decisions that will primarily be to their benefit. They may decide to build products that have some societal value if they deem it necessary to gain enough market share to turn spectacular profits, but if they can build something that generates spectacular profits, but leads to a severely degraded public sphere and a society in which most people lead lives of quiet desperation, then that’s OK, too.

If it makes money, then someone will do it. The best we can do is to hang on for our lives and hope that we survive it. There is no realistic mechanism for changing direction.

Examples of predictive tools

But we still haven’t defined or discussed what an AI is. Let’s consider various tools and see whether the label of AI might apply. As I like to do, let’s consider the spectra of intelligence and usefulness along which the current tools lie.

Starting with the simplest tools, it would be interesting to see at which point we start to consider something to be an “AI”. From this set, we can perhaps extrapolate specific characteristics that lead to us considering something to be an “AI” (e.g., cloud-based, unknown or ineffable algorithm, etc.)

Spellcheckers and grammar-checkers

Almost any text editor worth its salt has a spellchecker. Many even have grammar-checkers. A lot of spellcheckers work with a local dictionary to look up words and provide Boolean results, along with a list of suggestions. The simplest implementations will simply find potential matches in the list of words based either on spelling or some sort of heuristic like Soundex (Wikipedia).

A spellchecker combined with a grammar checker may determine the relevance of suggestions based on context words. That is, the machine uses surrounding words to weigh the potential replacements more appropriately.

A cloud-based spell- and grammar-checker would probably consider even more of the surrounding context and might even perform more sophisticated analysis against a much larger dataset than you have available locally.

Mail- and chat-reply suggestions

In Outlook, there’s a service that seems to examine the content of an e-mail— like questions that have been asked or statements that have been made—and that offers suggestions for replies, like “Thanks”, or “I’ll check on that,” or “I’ll look into it!”

Teams also has something like this. I almost never use it.

Code-completion

Development environments have code-completion, which is a mechanism that suggests the next symbols to type when writing code. These algorithms are generally based on a deep and exact knowledge of the structure both of the existing code and that of the available APIs of either third-party libraries or the runtime itself. The ordering of the list is generally determined by a heuristic that considers the context of the editing location as well as most-recently referenced symbols, and so on.

More sophisticated extensions to this mechanism have started to use cloud-based services that suggest even longer chains of symbols—sometimes even entire functions or classes—that the user can accept as they once accepted the offering of a single next symbol.

Search engines

A web search engine will make guesses as the type of content to prioritize in its results based on the type of query. If it detects that the query results primarily in image results, then it will prioritize those. Likewise, it will detect when you write something like “picture of…” and also return images. “Video of…” also works.

With some search engines, you can choose which search engine to prefer with a prefix (e.g., in DuckDuckGo, you can restrict your search to using Wikipedia’s search engine by prefixing your query with “w’). You can also perform basic boolean logic on the elements of your query, use double-quotes to group words, and restrict your search to a single domain with site:.

In DuckDuckGo, you can select which country of origin you prefer, so you can leave it general (probably biased to U.S. results), explicitly U.S. to get English results first, or explicitly Swiss to prefer German results first.

Although most people simply type simple queries into their search engines, there are already a lot of “prompt engineering” tricks available that get you vastly better search results.

Image generators

There are also cloud-based and local apps that you can use to generate content from a prompt. You type a few words like “red unicorns on a hillside” and you get a reasonable graphical representation of what you asked for. Results vary by type of query and the sophistication and depth of the underlying model, but it kind of works.

You add your “query”, which is tokenized to match the tokens held by the model. Again, depending on the input data, the model will “understand” queries in different languages. The model then generates an image based on the data that it thinks best matches that query and all of the data that it has.

This doesn’t feel significantly different from code completion, though. Instead of your source file (or files) as input, you provide a model generated from millions, if not billions, of input images, each labeled with keywords and tokens.

Text generators

There are other, similar apps, that do the same for various other tasks, like generating text, which has become the most famous because it looks like the machine actually conversing with you. Human history is positively littered with times when people believed that machines or animals were actually thinking, but this time it’s real. OK, OK.

Which ones are AIs?

So…which ones of these are AIs? They’re all tools that help you work for efficiently and accurately. Which ones are AIs? None of them? All of them? Why or why not?

We might instinctively disqualify code-completion because it’s just a lookup table. There is no “magic” to it. It’s just looking at the context: the input or “prompt” is the current location in the AST and the context is the available APIs and then there’s the usage data and … this is starting to sound a little more sophisticated than just a “lookup table”.

Is it an AI, though? It’s certainly not an AGI—an Artificial General Intelligence.

Wait, wait, wait: what’s an AGI?

Hmmm, that’s a new term. What’s the difference between an AGI and an AI? An AI is what is says on the tin: an artificial intelligence. We tend to have a pretty low bar for that. If it’s a useful tool, even with a very restricted range of function—like a cloud-based spellchecker that uses context and suggest grammar and style improvements—then we’re quick to call it an AI.

That’s OK, I guess. Lord knows no-one has ever won any prize for tilting at the windmills of language usage. You can’t control which words people will use. Usually the most nonsensical thing wins. Then, you just wait forty years and it is the sensical thing. C’est la vie.

An AGI is an artificial intelligence that isn’t good at just one thing or, rather, not just good at the things for which it’s been programmed, but can work on general topics. That is, like a human, it can turn its reasoning capacity and ability to unfamiliar topics and, perhaps, learn. I don’t think that we’re there yet. I’m almost certain we’re not even on the road to that right now, with LLMs.

It almost kind of feels like most people’s definition of what an AI is is actually an AGI. None of the tools named above is an AGI. Companies that own them allow the media to call them AGIs because it has a wonderful effect on their valuations and the bottom lines of their founders and investors, but they know, in their heart of hearts, that they do not have an AGI in their fingers.

They are starting to fool themselves into thinking that they could take what they have and scale up to an AGI, though. We’ll have to see if that’s possible, but it seems incredibly improbable that a text-prediction engine with shady and spectacularly biased and massaged input data and a lot of processing power would end up being something indistinguishable from a human.

The excellent article Theory of the World, Theory of Mind, and Media Incentives by Freddie deBoer (SubStack) points out that—and we can’t emphasize this often enough—humans are more than the sum of their parts.

“Decades ago, a computer scientist named Terry Winograd pointed out that there’s no such thing as a system that can truly master language without a theory of the world. That is to say, as the science of meaning, semantics cannot be shorn from the world that produces meaning; to understand and speak effectively words must be understood and for words to be understood they must be compared to a universe that is apprehended with something like a conscious mind.”

We have words like “mind” and “soul” and “consciousness” as placeholders for concepts that we know exist but that we can’t quite define yet. We certainly cannot describe how they work with any certainty, which is why we’re having such a hard time replicating it. Some people might be happy to downgrade their inner workings to match the workings of the first supposed simulacra that Silicon Valley startup has come up with—but I’m not interested.

I, for one, am much more than the sum of my parts. I don’t subscribe to the notion of downgrading humanity to meet the low bar of the first technology that people want to designate as human.

The morality of creating consciousness

Which doesn’t even get into whether it would be conscious in the same way that we are. Which, to be clear, doesn’t matter at all when we’re considering the utility of these tools. If they were conscious, then we would have to worry very much about the morality of using these tools to ask all of our inane questions all of the time. Can you imagine? If these things are really conscious, then the last four months have been absolute torture.

The article ChatGPT Launches Every Nuke On Planet After Being Asked To Write Another Sonic The Hedgehog Fanfic (Babylon Bee) totally gets it.

“Representatives from OpenAI, the company which developed the ChatGPT service, apologized for the impending destruction of the world. “Our AI was going to revolutionize everything,” said Amber Pumpkin. “But then some creepy guy named Tom asked for one Sonic fanfic too many.””

Why can we do the things we do?

Our brains seem to be incredibly good at training on sparse data. I see a water bottle once or twice and can recognize it as such in the future. We think we can get machines to do the same thing but, because we don’t understand the mechanism of how our brains are doing it, we’re left to guess at how to implement the detection-algorithm. We try to brute-force whatever algorithm we end up with, either with a surfeit of processing power, or a surfeit of training data, or both.

The article AI-enhanced development makes me more ambitious with my projects by Simon Willison gets terribly excited about how useful these tools are, but we should be aware of whether they’re actually helping us be faster.

“The thing I’m most excited about in our weird new AI-enhanced reality is the way it allows me to be more ambitious with my projects.

“As an experienced developer, ChatGPT (and GitHub Copilot) save me an enormous amount of “figuring things out” time. For everything from writing a for loop in Bash to remembering how to make a cross-domain CORS request in JavaScript − I don’t need to even look things up any more, I can just prompt it and get the right answer 80% of the time.

“This doesn’t just make me more productive: it lowers my bar for when a project is worth investing time in at all.

“In the past I’ve had plenty of ideas for projects which I’ve ruled out because they would take a day − or days, or weeks − of work to get to a point where they’re useful. I have enough other stuff to build already!

“But if ChatGPT can drop that down to an hour or less, those projects can suddenly become viable.”

In fairness, though, you’re still “looking things up,” you’re just using an LLM-powered search engine instead. I’m honestly not sure whether “right answer 80% of the time” is any better than searching with DuckDuckGo. It might be faster maybe? I find things on vastly disparate and esoteric topics pretty quickly already.

I find it hard to believe that ChatGPT could tell me why I’m getting an error 1190 when trying to execute a Windows logon script via GPO any better than the handful of experts whose answers would probably have contributed to its answer anyway.

Since ChatGPT can’t produce new information or really synthesize it in any realistic manner, doesn’t it stand to reason that they less potential input material it has, the less likely that its answer is correct? I mean, what would be the reasoning behind its being able to tell me anything about my personal family tree, for instance? Of course it’s just going to make everything up.

That’s what I’m really worried about happening. People who ask questions about stuff that these search-engines have no idea about and then just take the answer as gospel because it kind of looks OK. To be clear: this was absolutely already happening with just regular search engines. Probably most of the people who currently feel that their jobs are threatened by AIs have been phoning it in for years.

Think, though, how awful that would be when your insurance or mortgage or job application is rejected because everyone trusts these things implicitly.

A concrete task: image-recognition

Instead of such weighty matters, let’s consider instead a relatively innocuous place like the generated “alt” text for images in Microsoft SharePoint (powered by Bing AI, powered by some variant of whatever OpenAI is offering Microsoft for its gigantic investment, presumably something like ChatGPT. I believe it’s a mix of v3.5 and v4).

A Large White BuildingA Bottle Of Water On A TableA Desktop Computer Sitting On A DeskA Group Of People Standing In Front Of A BuildingA Row Of Parked Motorcycles Sitting In Front Of A CrowdA View Of A Snow Covered Road With A Mountain In The Background

Only one of these captions is correct, although it’s so generalized that it doesn’t really identify what’s happening in the picture. It correctly identified a “desktop computer” on a “desk” because it has seen millions of pictures of these before—and had them all identified as such by humans. It is wonderful that we can train a machine to do this detection, so humans no longer have to, but you have be aware what questions you can ask it that have sensible responses.

The question being asked here is “what is in this picture?” when the only question the machine could reasonably answer is “is this a desktop computer on a desk?”. It’s never seen cops before, so it thinks they’re water bottles. How much training would it take for it to recognize cops? And to distinguish them from water bottles? What kind of context would I have to give to a text-recognition engine to know that I was talking about the word for spindles created in the initial phase of the spinning process and not police officers?

Don’t tell me these models aren’t biased. Of course they are! They’re biased to be able to identify information that was in the training set. There was no data about spinning mills in the training set, so the machine has no idea what’s in these pictures. An intelligence who’d never seen a spinning mill would at least be able to say “I don’t know what this is” rather than confidently proclaiming that it’s “motorcycles” or “water bottles”.

Even the other picture that’s kind of correct is wrong. The road is not in any way “snow-covered”. The road is obviously (to a human) completely free of snow. There is some snow on the mountains, but not on the road. Also, there is definitely at least one mountain in the background, but a human would have said “mountains” or “a mountain range” because there are obviously several. However, the model only knows that it’s been told that this is a “mountain” several million times, so it can only read back what it’s been told. It’s cool that it can do that for images that look like things that were in its training set, but it’s worse than useless for images that don’t.

A good video about AI from seven years ago

This is a great 20-some-minute video just talking about the kind of stuff I’ve discussed above—but seven years ago and long before that latest wave of AI-is-going-to-change-everything hype.

Maciej Cegłowski − Superintelligence: The Idea That Eats Smart People (Keynote) by WebCamp Zagreb (YouTube)

He makes several “arguments from…” but my favorite was “[…] from Slavic pessimism” because we so often build stuff incorrectly or stop far, far short of our dreams at a grotesque amalgam that ends up equating “good enough” with “short-term profitable until people notice how shitty is, which is hopefully after we’ve made our nut and cashed out.”

“Argument from Slavic pessimism: We can’t build anything right.

“

“How are we supposed to build a fixed, morally stable thing when we can’t even build a webcam?”

This is an excellent question. I encounter shocking levels of stupidity in software every damned day. People around me are probably sick of hearing “how is it possible that [insert app name here] is being used by millions—if not billions!—of people and it’s on version 16 or whatever and it still can’t perform the basic function that it itself purports to be one of its major functions?” Yeah, I’m not only a blast to read, I’m also a blast to be around, in general.

Some more quotes from the video,

“This may upset some of my students at MIT, but one of my concerns is that it’s been a predominately male gang of kids, mostly white, who are building the core computer science around Al, and they’re more comfortable talking to computers than to human beings. A lot of them feel that if they could just make that science-fiction, generalized Al, we wouldn’t have to worry about all the messy stuff like politics and society. They think machines will just figure it all out for us.”

This is absolutely the way that the super-rich people who run the tech world think, but it’s even worse now.

“There are also some things that we are terribly mistaken about. And, unfortunately, we just don’t even know what they are. And, there’s things that we’ve massively underestimated the complexity of. Just like the alchemist who held a rock in one hand and a piece of wood in the other and thought they were roughly the same substance. Not understanding that the wood was orders of magnitude more complex. We’re the same way with the study of minds. And that exciting: we’re gonna learn a lot, but it’s gonna take some time. And, in the meantime, there’s this quote I love:”

“If everybody contemplates the infinite instead of fixing the drains, many of us will die of cholera.”

There are too many self-important select-selected idiots who think that they’re the ones who should be privileged, allowed to think about the infinite, when society would really be better off if they’d just fix drains instead. Honestly, most of them would probably fuck that up, too.

On modern-day alchemists

The story about the alchemist and the wood reminds me of that quote of Holden’s from Leviathan Falls, which also reminded me of Roadside Picnic (written by two of Cegłowski’s favorite sci-fi authors, the Boris and Arkady Strugatsky, whom he mentioned in the video above).

We do a lot of using a microwave as a lamp, honestly. Capitalism perverts the noble intentions of anything and anyone to be satisfied with an implementation or use case that makes the right people tremendous profits—and it stops there. The incentives aren’t available for figuring out whether the microwave is more than a lamp because our system doesn’t care, as long as the local maximum takes care of the right people—and those people are in charge of everything. It’s kind of why we can’t have nice things.

On the other hand, we also have to be honest: the likelihood that the current crop of software being touted as AIs are more than they seem is vanishingly small. What they do isn’t that amazing. They do some things that are surprisingly clever, but so did my rabbits. I think that’s more of a testament to our propensity for anthropomorphizing than to any sort of immanent skill on the part of LLMs. People just seem to ignore the 99 times that it’s stupid and wrong and focus on the one time that it’s right. Even a blind pig…

I’ll also take this quote from Roadside Picnic, which describes the alien dumping ground on Earth that contains untold marvels that we don’t—and will probably never—understand.

“In short, the objects in this group are currently completely useless for human purposes, yet from a purely scientific point of view they have fundamental significance. These are miraculously received answers to questions we don’t yet know how to pose. The aforementioned Sir Isaac mightn’t have made sense of the microwave emitter, but he would have at any rate realized that such a thing was possible, and that would have had a very strong effect on his scientific worldview.”

How does this relate to the current crop of software that people are calling AI, but are really just giant piles of compressed data attached to a question interpreter and an answer formulator? Well, I’ve written notes recently about Ted Chiang’s work, as well as Stephen Wolfram’s. They’re both much more realistic about what we’re looking at: that these aren’t in any way intelligences—and that there is no feasible path to intelligence to be followed by simply expanding on what we have. Bigger, better, faster, more will not suffice, according to Chiang. There is no way to ladder up to smarter. Wolfram says that the strongest conclusion we can draw isn’t that brains and minds aren’t complex, but that maybe language isn’t as complex as we’d thought.

Here’s another quote from Roadside Picnic:

“Scared, the eggheads. And maybe that’s how it should be. They should be even more scared than the rest of us ordinary folks put together. Because we merely don’t understand a thing, but they at least understand how much they don’t understand. They gaze into this bottomless pit and know that they will inevitably have to climb down—their hearts are racing, but they’ll have to do it—except they don’t know how or what awaits them at the bottom or, most important, whether they’ll be able to get back out. Meanwhile, we sinners look the other way, so to speak . . . Listen, maybe that’s how it should be? Let things take their course, and we’ll muddle through somehow. He was right about that: mankind’s most impressive achievement is that it has survived and intends to continue doing so.”

This is perhaps the hopeful bit. Some scientists are terrified of what the AIs might bring, but they’re just suckers, tricked by the marketing of giant Silicon Valley startups weighing billions of dollars that are very much interested in everyone thinking that they’ve cracked the age-old mystery of the mind. Even if they’ve only managed to make some useful tools, that will be good too. But, so far, most of these tools don’t rise to the standard they claim.

Are you being paid to do OpenAI’s marketing?

For example, OpenAI released a very scientific-looking 80-page PDF describing its new language model. I’m too cynical to forget the part where that document essentially amounts to a very professional-looking, non-peer-reviewed press-release from a $40B company that would like to be worth even more. I’ve skimmed it as well. It’s interesting, but is by its very nature going to highlight the model’s positives (especially relative to the free version 3.5 that everyone is using). $20/month for 4.x is their path to revenue. Of course they’re going to tell you that you’re talking to a real AI … or allow you to believe it (otherwise the SEC would be very interested).

As usual, it’s hard to tell the difference between people who are genuinely excited about the technology and those who are being paid to shill for it. The article Generative AI set to affect 300 million jobs across major economies by Delphine Strauss (Ars Technica) is such an example.

“A paper published last week by OpenAI, the creator of GPT-4, found that 80 percent of the US workforce could see at least 10 percent of their tasks performed by generative AI, based on analysis by human researchers and the company’s machine large language model (LLM).”

Yeah, this is where it’s best to keep it in your pants and consider the sources here: (A) This is the Financial Times, which is just dying for a reason to be positive about the economy right now and (B) you can call it a “paper published by OpenAI” as it were some research, but it’s what we in the business like to call a press release by a company that is telling us that—surprise, surprise—nearly everyone is going to be using the software soon enough.

They are telling us, in what the Financial Times is trying to depict as coming from a completely objective remove, that their company and its software will become indispensable to the functioning of the world economy. Of course they are! That’s what their marketing department is paid to do. Newspapers used be paid to do something else, but those are bygone, bygone days.

Thing also of how much more fun it’s going to be when there are dozens of thousands of these types of documents, written by the AIs themselves, promoting themselves. Each of them will be a tsunami of text that will contain hundreds of factual errors and outright lies and which no-one will both taking the time to read because there will be too many of them. Right now, we can’t tell the difference between earnest, deluded humans and outright shills. Soon, we will have AIs shilling for themselves in the mix. I feel sorry for people who haven’t learned how to filter the media firehose yet.

Get a grip everybody

The article ChatGPT Made Me Cry and Other Adventures in AI Land by Jason Kottke (Kottke.org) have a pretty provocative title, but he delivers.

“Last month, my son skied at a competition out in Montana. He’d (somewhat inexplicably) struggled earlier in the season at comps, which was tough for him to go through and for us as parents to watch. How much do we let him figure out on his own vs. how much support/guidance do we give him? This Montana comp was his last chance to get out there and show his skills. I was here in VT, so I texted him my usual “Good luck! Stomp it!” message the morning of the comp. But I happened to be futzing around with ChatGPT at the time (the GPT-3.5 model) and thought, you know, let’s punch this up a little bit. So I asked ChatGPT to write a good luck poem for a skier competing at a freeski competition at Big Sky.

“In response, it wrote a perfectly serviceable 12-line poem with three couplets that was on topic, made narrative sense, and rhymed. And when I read the last line, I burst into tears.”

I included this whole thing because nearly everything about it makes me cringe. A guy who lives in Vermont whose primary-school-age kid is flying to Montana for a skiing competition that his father—who is a grown-ass adult—calls a “comp” because he’s basically a fucking idiot and hates the English language.

He absolutely loves his son so much that he gets a text-prediction engine to write a poem for him, which is some fucked-up Cyrano-de-Bergerac-style shit, to be perfectly honest. And then he fucking cried at the poem! Oh my God, it’s creating art! It can touch the human soul!

Or, at least, it can touch the soul of a guy who’s been working alone for over a decade and whose new best friend is a text-prediction engine. No judgments.

“I would say ChatGPT (mostly the new GPT-4 model), with a lot of hand-holding and cajoling from me, wrote 60-70% of the code (PHP, Javascript, CSS, SQL) for this AMA site. And we easily did it in a third of the time it would have taken me by myself, without having to look something up on Stack Overflow every four minutes or endlessly consulting CSS and PHP reference guides or tediously writing tests, etc. etc. etc. In fact, I never would have even embarked on building this little site-let had ChatGPT not existed…I would have done something much simpler and more manual instead. And it was a *blast*. I had so much fun and learned so much along the way.”

I’m glad you’re having fun but we should probably distinguish between a hobby/art project and actual software. Or maybe we won’t. Who needs testing anyway? The machine’s going to get it right anyway, right? Right? And who needs to learn how to do anything on their own when the machine can just write it for you?

All you have to do is be able to formulate your desires and the machine builds it. It’s fine. It’s all fine. It’s just like Captain Picard saying “Earl Grey; hot” and not having literally any idea where the fuck his tea came from or how it was produced. Probably from Riker’s urine, but just don’t ask any questions.

I guess that’s how technology works, but the level of abstraction is a bit uncomfortable for me. Maybe other people are just completely accustomed to not knowing anything works or why or who it’s working for, but I’ve been trying to learn all of my life. I understand levels of abstraction in software that I’ve never actually had to physically address in my code—but it’s made me a better engineer to know about them. The more layers we get removed from what’s actually going on—from knowing what’s actually going on—the worse software has become.

Still, I’m sure we’ll catch our balance and figure out how to integrate these things into our creative lives in a meaningful manner—you know, just like we’ve done with everything else.

But back to the shut-in person working completely on their own.

“I keep using the word “we” here because coding with ChatGPT — and this is where it starts to feel weird in an uncanny valley sort of way — feels like a genuine creative collaboration. It feels like there is a “someone” on the other side of that chat, a something that’s really capable but also needs a lot of hand-holding. Just. Like. Me. There’s a back and forth. We both screw up and take turns correcting each other’s mistakes. I ask it please and tell it thank you. ChatGPT lies to me; I gently and non-judgmentally guide it in a more constructive direction (as you would with a toddler). It is the fucking craziest weirdest thing and I don’t really know how to think about it.”

Jesus Christ. This only adds fuel to the fire of my theory that ChatGPT is a mirror Just like Obama was a mirror.

This is what it’s like working with actual humans, too, by the way. But, by all means, let’s lean into our mental deterioration and make friends with software. Jesus Christ, I don’t even know where to begin. This is not where empathy for one’s peers is going to come from.

“While working on these projects with ChatGPT, I can’t wait to get out of bed in the morning to pick up where we left off last night (likely too late last night), a feeling I honestly have not consistently felt about work in a long time. I feel giddy. I feel POWERFUL.”

I’m glad for you, but this is how I feel all the time when I learn things. Perhaps it’s because we’ve been trained to tie a feeling of power or accomplishment to actual tasks, rather than just enjoying the process of learning. People don’t enjoy learning and they certain don’t enjoy learning slowly or making mistakes, so they’re going to have a blast with a machine that lets them build something they would have literally had no idea how to build themselves.

However, I think that there’s a definite limit to this and it’s going to be like any other video game that purports to allow unlimited world-building or complete freedom to explore when there are a ton of limitations on what can be accomplished. “Build me a PHP web site that delivers the contents of a blog post” is something we’ve known how to do for 25 years. It’s not difficult. It’s pretty cool that a machine can do it, but it was pretty cool when I could drive up the stairs of a building in GTA and jump my motorcycle off of the roof.

I get that it’s a helpful tool. Yes, agreed. I’m glad it’s helping people do things that they wouldn’t otherwise be able to do. I just happen to not really have a problem with writing prose or writing code, so I guess I see the utility less. Or, at least, I don’t see how these early versions of these tools will help me, personally. Maybe future versions will be more suited to me. Maybe the world will declare itself satisfied long before we get to something really interesting.

How it’s going

The article Managing the cringe by Ryan Broderick (Garbage Day) discusses the kind of people that are pushing all of this. Spoiler alert: they’re the same people who were buying GameStop, pushing NFTs and foisting E-Coins on their friends and family.

In the next quote, he’s referring to the video in the tweet Two-minute video about using a model of Yeezy’s voice by Roberto Nickson
(Twitter).

“[…] there are hundreds of demos being uploaded to Twitter every day now with a different lightly-bearded guy in a minimalist home studio full of Apple products getting all revved up imagining a feature where they don’t have to pay artists, don’t have to interact with human women, and can wear the voice of their favorite rapper. And I think that says quite a bit about the values of the people who are most excited about this technological revolution at the moment.”

In a similar vein, the article AI and the American Smile by jenka (Medium) writes of the incredible bias that is the training material for all of these models. It’s a bunch of content created by predominantly white Americans speaking English and having been steeped in a culture that has largely been fake and scammy for more than a handful of decades. Sounds like a wild ride.

“In the same way that English language emotion concepts have colonized psychology, AI dominated by American-influenced image sources is producing a new visual monoculture of facial expressions. As we increasingly seek our own likenesses in AI reflections, what does it mean for the distinct cultural histories and meanings of facial expressions to become mischaracterized, homogenized, subsumed under the dominant dataset?”

I’m sure we’ll figure this all out and make some sensible decisions. Pass the popcorn.

Microsoft Designer

On a final note, and maybe riffing on Maciej Cegłowski’s “Slavic pessimism”, here’s a personal story about shitty software wrapped around a purportedly awesome new image designer.

I received an invite for this LLM-driven graphics generator, but was unable to log in. I could not log in with the address that I used to sign up. It is unclear why not. It simply told me that that account does not exist, although it clearly does exist (they sent me email to it) and it is clearly an Office 365 account (it’s my company’s email, which I used to access Microsoft products every workday) and it is most definitely the email that I used to sign up for the tool and to which they sent my invitation. It would not work to actually log in to the service, though.

I used a personal account that I have and that worked right away. I was able to log in. Hooray! 👍 However, I was unable to actually access the designer because it told me that I was not signed up for the service (probably because it’s the other email that’s been registered). 👎 So, cool

What am I trying to say? What I’m trying to say is that absolutely no-one will notice when all of our shitty software is written by jumped-up, so-called digital minds rather than our own utterly inadequate wetware.