Distrust and Verify

[JM]: So the other day I came across an article by a friend of mine, Mark Smith, titled "Using Fortune to Reinforce Habits".

[JM]: And this is a cool article.

[JM]: You should check it out.

[JM]: It is about something that happens to me a lot.

[JM]: I really like finding interesting new things,

[JM]: command line programs, command line tools to run in the terminal,

[JM]: and I am always experimenting with new ones.

[JM]: Sometimes they stick, sometimes they don't.

[JM]: But a problem that I run into very frequently is that I forget that they are there.

[JM]: I've installed them, I may even have notes somewhere that I installed them and why I did it.

[JM]: But the reality is in everyday usage, I just forget that they're there and I forget to use them.

[JM]: And Mark came up with a rather creative solution in which he documents the various tools that he's installed.

[JM]: And when he launches a new terminal session, something pops up to remind him that this tool exists.

[JM]: So at the end of the post, there's a screenshot of the end result.

[JM]: And there's a little box that says, don't forget about Hexel, a handy binary file viewer.

[JM]: So not only does it remind you, hey, this is what the command is,

[JM]: there's also a reminder of

[JM]: what you would use it for and, and why you might want to use it.

[JM]: And he explains that he could address this problem in a different way for certain tools that are like one-for-one replacements.

[JM]: So for example, there is an `ls` replacement tool called `eza` as a I don't actually know how one would call that.

[JM]: But anyway, it's spelled E-Z-A.

[JM]: And when you run it, you get more or less the same output as `ls`, there's a lot of similar flags and options.

[JM]: So you could

[JM]: alias `ls` to `eza` so that when you ran `ls`, you're actually getting `eza`.

[JM]: And as Mark points out, that's fine when you have commands that are largely backwards-compatible, and the command line flags are more or less the same.

[JM]: But a lot of times you get tools that aren't.

[JM]: So there is a `duf` command, which is a replacement for `du`.

[JM]: There's `sd`, which is a replacement for `sed`.

[JM]: And

[JM]: There's `fd`, which is a replacement for `find`.

[JM]: And a lot of times, these don't all have the exact same command line flag options.

[JM]: And so aliasing often isn't the best way to solve this.

[JM]: And I love this solution that Mark came up with.

[JM]: And he goes into great detail as to how you can implement the same thing for yourself.

[JM]: I'm excited to do this.

[JM]: I haven't had a chance yet, because I just came across this, but I'm definitely going to do it.

[JM]: And he,

[JM]: like me, uses Fish shell.

[JM]: So there's even a little bit of detail in here for how to do it for fellow Fish-heads like me.

[JM]: Okay, moving on.

[JM]: And this isn't really follow-up.

[JM]: We've talked about Substack before, but there's a new development.

[JM]: So it's follow-up, I suppose, in that way.

[JM]: And what is very much in the news at the moment is that Substack sent out a push alert promoting a racist blog.

[JM]: And I'm going to use the word "racist" here.

[JM]: You can substitute that word for other words that are perhaps more accurate.

[JM]: I will leave that up to the listener.

[JM]: In one of the articles that I've read about this topic, the article said, "Substack is primarily funded by Andreessen Horowitz, a firm whose founders have pushed extreme far-right rhetoric".

[JM]: And it's worth noting that there was an actual swastika in the push alert.

[JM]: Substack says that the alert was issued by mistake.

[JM]: And my response to that is kind of like, oh, well, in that case! I feel like I am not alone in this being yet another in a long string of reasons why if you are a content creator who is on Substack, it is time, if you haven't already, for you to get off of Substack.

[JM]: And if you are someone who subscribes or otherwise patronizes creators on Substack,

[JM]: I would invite you to consider whether you want to continue doing that, whether you want to maybe even reach out to the creator in question and gently and kindly ask that maybe it is time for them to find a different place to hang their hat.

[JM]: But of course, that is just my opinion.

[JM]: What I think is unequivocally and objectively clear is that Substack has made it very, very obvious that

[JM]: that they are totally okay with racist content.

[JM]: Like that is just not even debatable at this point.

[JM]: That is just objective fact.

[JM]: They've gone on record saying that they're not going to "censor things based on content".

[JM]: At which point I would point to the parable of the racist bar and you can replace racist with the...

[JM]: appropriate word.

[JM]: And the parable goes something like if racists frequent your bar and you don't kick them out, more racists will come,

[JM]: the rest of your clientele will leave, and your bar will become a racist bar.

[JM]: And that's how I feel about Substack.

[JM]: I feel like that is essentially what they have become.

[JM]: And I was very pleased to come across a site this week called LeaveSubstack.com in which the same message is being made, right?

[JM]: The message of you should probably leave Substack.

[JM]: Here's how to do that.

[JM]: Here's the reasons why you should do that.

[JM]: And the list seemingly is just growing one bullet point at a time as more and more evidence piles up.

[JM]: And there's some good suggestions in here.

[JM]: Like the ones that we have mentioned in our previous episode about "don't call it a Substack", such as Beehiiv, ButtonDown, Ghost, etc.

[JM]: There's examples of who else has already migrated away from Substack.

[JM]: So I think this is a great site, great resource, and kudos to the person who made it their mission to put this up as a resource for other people.

[DJ]: I have to admit that previous similar issues on Substack had not pushed me completely away from the platform as a reader.

[DJ]: As a publisher, I would never use it in the first place for...

[DJ]: a different set of reasons that are also described on that Leave Substack website, namely that it's not a platform you own.

[DJ]: Substack is the next in a long line of publishing websites that say, "Hey, individual who wants to make money by publishing their content to an audience on the internet, we will help you do that.

[DJ]: And we have accumulated an audience that

[DJ]: you want to reach them, don't you?

[DJ]: Well, come on in here and we will help you build a big audience and possibly make a living off of your writing, which is something that a lot of people want to do and it's hard to do."

[DJ]: Like, it's hard to build an audience.

[DJ]: It's easy to publish in this day and age, but it's hard to build an audience, especially an audience of people who will pay you.

[DJ]: So there have been a lot of web sites like Substack and

[DJ]: personally, as a publisher, I would not publish on them, not merely because of the risk of having my content adjacent to extremely horrible content, but because I don't really control the platform.

[DJ]: And ultimately...

[DJ]: There's a question of like, are those customers really my customers?

[DJ]: Is the brand really my brand?

[DJ]: But those are all the considerations you should have if you are trying to build a business by publishing.

[DJ]: If you are merely a reader, you have the question of, well, I really want to pay this writer for the writing that I like, but I don't really want that transaction to go through this platform whose behavior I find objectionable.

[DJ]: That's a tough one to me.

[DJ]: There is maybe just one writer in particular that I currently subscribe to who publishes on Substack.

[DJ]: And in light of this most recent crap on Substack's part, there's a part of me that wants to yank my eight Canadian dollars a month or whatever it is.

[DJ]: But quite frankly, that hurts the writer a lot more than it hurts Substack.

[DJ]: And yet, as you've brought up in several contexts in the past,

[DJ]: voting with your wallet is important.

[DJ]: It's one of the few ways that an individual can try to make some kind of dent in the world is by choosing where they spend their time and attention and money.

[DJ]: So I haven't decided what I'm going to do in the case of subscribing to somebody on Substack.

[DJ]: The person that I subscribe to has actually written about this problem in the past and

[DJ]: although in the past they have generally made an argument for why they will remain on Substack, I'm not going to try to paraphrase it right now off the top of my head because I don't want to misrepresent their position.

[DJ]: Suffice to say, if I wrote to them and said, "Hey, I really like reading your stuff, but oh boy, this platform makes me increasingly uncomfortable, so I might have to go, I don't know that that's going to make a dent, but it's what I can do."

[JM]: I completely empathize with creators who want to monetize their work and who believe that Substack offers the best option for them in terms of enabling them to make a living by publishing their content.

[JM]: And perhaps they've searched and tried to find tools and other alternatives and just haven't found something that delivers the same degree of income or ease of use or whatever their evaluation criteria are.

[JM]: Totally get it.

[JM]: I do think it's worth asking oneself, "At what cost", right?

[JM]: I think it's worth saying, okay, perhaps Substack does offer the best monetization strategy and one needs to eat.

[JM]: And I completely understand that.

[JM]: I do think it's at least worth asking the question, like, "Is it worth it to me?"

[JM]: Is it worth it?

[JM]: Whatever this delta is, whether it's more money or more ease of use, or this is where the audience is.

[JM]: And sometimes maybe the answer is yes.

[JM]: Sometimes maybe the answer is it is indeed worth it.

[JM]: And it is probably from a position of privilege that I say personally for me, it just would not be worth it.

[JM]: And at the same time, like I said, I recognize that I am probably in a different position from other folks who decide that they either want or need to stay with Substack.

[JM]: And I don't think I would necessarily judge them for that.

[JM]: But I agree with what you said, which is you do have agency, you can vote with your wallet, vote with your feet.

[JM]: And I think it actually

[JM]: is meaningful if you know a creator who you respect and whose work you appreciate to reach out to them and say, "Hey, this is something that's important to me.

[JM]: And I just wanted you to know that.

[JM]: And I don't know if I will continue to be here for these reasons."

[JM]: Maybe it has an effect.

[JM]: Maybe it doesn't.

[JM]: But it is what you can do, as you said.

[JM]: And to me, I think that's worth doing.

[DJ]: I want to even go a little further when it comes to creators because I've seen this problem before in other places.

[DJ]: I'm part of the indie author community, which is to say I wrote a novel and I published it myself and distributed it myself as opposed to going through a publishing company.

[DJ]: And this is a path towards being an author that has evolved over the last 15 years, like roughly since the Amazon Kindle came into existence and created the first market where people could easily publish digital books.

[DJ]: And there is lots and lots and lots of talk in the indie author community about owning your platform and, as they call it, "selling wide", which is an alternative to just put your book on the Kindle store and pour all of your time and effort and resources into selling through that one platform.

[DJ]: And I think this is a good analogy to people feeling like Substack is their only hope.

[DJ]: I want to quote the article that we've been discussing because I think this captures the heart of this topic.

[DJ]: "Small creators can't afford to move off Substack because it's the only place anyone makes money doing this stuff," one creator posted to Blue Sky.

[DJ]: "I've never heard of a complete app that covers all the functionality of Substack.

[DJ]: For people to shift en masse, another platform needs to actually cover those bases.

[DJ]: It needs a good word editor with built-in referencer, proper monetization, and an effective user front-end that aids finding content."

[DJ]: I think that last phrase is at the heart of this, because I've seen this in so many other places, like indie authors who feel beholden to Amazon and the Kindle store.

[DJ]: Creators are trapped by this problem of discovery.

[DJ]: It's easier than ever to publish your work on the internet.

[DJ]: but it's still hard to actually get people to find it, and then it's even harder to get people to pay you for it.

[DJ]: There are a lot of systems that need to be in place, and it's hard to build those systems by yourself as an independent creator.

[DJ]: However, as I've seen many people say in the indie author space, and it's true in this other small publishing space as well, when you're trying to make a business out of your writing, you have to build a business.

[DJ]: And this is hard to do.

[DJ]: And honestly, look, a lot of us creators don't want to do it.

[DJ]: We want to be writers.

[DJ]: We don't want to be entrepreneurs.

[DJ]: But I'm going to say you don't have a choice for better or worse.

[DJ]: If you want to make a living off of your creative work,

[DJ]: it behooves you to learn how to build a business around your creative work.

[DJ]: And one of the principles of building a successful business is do not make your entire business completely reliant on one other external party.

[DJ]: So the fundamental fallacy contained in that quote that I read, although like I sympathize very much with the person who wrote it, is they say, "I've never heard of a complete app

[DJ]: that covers all the functionality of Substack.

[DJ]: For people to shift, another platform has to cover those bases."

[DJ]: And I'm going to say, no, you're missing the point.

[DJ]: The solution is you need to be able to build your own platform.

[DJ]: Because the problem of Substack, as we're describing it now, is that Substack builds this business and they gain all the attention.

[DJ]: And then they say, "Hey, you, writer.

[DJ]: If you actually want to make a living, you got to play by our rules, whatever those rules are.

[DJ]: And if those rules are that your website can't be djjacobson.com, it has to be yourthing.substack.com.

[DJ]: And you don't get to pick the colors and font that go on your writing.

[DJ]: We're going to do that.

[DJ]: So in other words, you don't own your brand.

[DJ]: We do.

[DJ]: Those are just the rules.

[DJ]: And then if it turns out the rules are, and by the way, we're just as likely to push to send a push notification about some racist content as we are about your content.

[DJ]: Too bad.

[DJ]: These are the rules."

[DJ]: So the real solution for creators is to build their own platforms.

[DJ]: That's hard to do, but it's not impossible to do.

[DJ]: And so I want to push back a little on that quote and say that for people to shift, it's not the case that some other platform has to come along and force you to play by their rules instead of Substack's.

[DJ]: And ideally, the tools and the services around those tools get better and easier that can help creators build their own platforms.

[DJ]: And that LeaveSubstack.com web site provides

[DJ]: mentions some of those tools that have gotten popular.

[DJ]: So they include things like Ghost, which is a content management system that you can host yourself or host in various other places that does not lock you into a certain branding and a certain network.

[DJ]: But of course, on the other hand, you have to find other ways of building an audience.

[DJ]: I'm a big fan of whatever you call it, the indie web, the open web,

[DJ]: the ability to build things on the Internet that are not beholden to that one big venture capital-driven tech company that ultimately doesn't care if you are successful as a creator.

[DJ]: It only cares about maximizing returns to its shareholders.

[DJ]: And what I can see is the tools for the open web continue to get better, even as it seems like these giant silos are more and more dominant.

[DJ]: There is a groundswell of independent tools and places.

[DJ]: Manton Rees, who's the founder and maintainer of micro.blog, which is one such space, just posted a blog post called "We Will Survive Google Zero".

[DJ]: And he's referring to something that's going on with Google right now,

[DJ]: where it's seeming like web traffic is really starting to fall off from search, being driven by, among other things, the increasing prominence of AI scraping the web and AI being used as a search engine,

[DJ]: are going to disintegrate, and that is rough for a lot of people, those models had some substantial downsides.

[DJ]: Just like the model of, yeah, people will pay for your email newsletter, but the only effective place to host it is on this one web site with apparently extremely questionable values.

[DJ]: And Manton points out that there's an opportunity here for us to build tools that better empower creators.

[DJ]: And I'm with him on this.

[DJ]: I'm hopeful that the outcome of things like hopefully Substack actually being held to account for their ugly behavior will be more and more uptake of tools that better empower independent creators.

[JM]: This is going to sound weird, but I was actually quite pleased when I saw this news.

[JM]: I'm sure it was horrible for the people that received this push alert.

[JM]: I'm sure it was very triggering for a lot of people that were exposed to this racist content.

[JM]: I'm pleased because I'm hoping that this will be the impetus that some people need to finally get off this platform.

[JM]: And for people who...

[JM]: read or otherwise support content creators on that platform,

[JM]: I hope this will be the push to get people off this platform.

[JM]: I hope this will be a push for exodus, and also in the process, creating this opportunity for the alternatives, whether it's ones we've already discussed, whether it's ones that are going to be born by people who see news like this and say, "This is terrible.

[JM]: That's it.

[JM]: I've had it.

[JM]: I'm going to go create the next great Substack alternative."

[JM]: Because that's what we need.

[JM]: We have tools, you don't have to go and build your own platform.

[JM]: We have tools out there today, like ones we've already talked about.

[JM]: So you don't have to know how to build your own platform.

[JM]: But you do have to know, at some point, that you have to build your own audience. This discovery feature that Substack has, that helps people discover content that's similar to their interests...

[JM]: That's something you're going to probably have to figure out until such time as there are other platforms that aren't this kind of walled garden.

[JM]: You're going to have to figure out how to do that.

[JM]: And I agree with Manton when in his article, he says, this is a great time to build a relationship with readers.

[JM]: Go start a blog, start a newsletter, start linking to your posts on social media, wherever makes sense.

[JM]: This is great advice.

[JM]: And it's never as easy as it sounds.

[JM]: It's incredibly difficult.

[JM]: But in the end, I think it will pay off.

[JM]: And it's a lot better than hitching your wagon to companies like Substack.

[JM]: It is really hard to build an audience.

[JM]: And it's very much like the tree parable, right?

[JM]: The best time to plant a tree is 20 years ago.

[JM]: When's the second best time?

[JM]: Today.

[JM]: Mastodon and the Fediverse are very new and represent a tiny fraction of the number of users and engagement and just overall social media usage that we saw in Twitter's heyday.

[JM]: And yet in this short amount of time, and despite the fact that most people on the planet have no idea what Mastodon or the Fediverse are, there are people with tens of thousands of followers.

[JM]: That's a huge achievement.

[JM]: And how did they do that in such a short amount of time, relatively speaking?

[JM]: They did that just one day at a time, posting things that were interesting to other people.

[JM]: So I do empathize with how difficult it is to do.

[JM]: But there's plenty of examples out there of how people are doing it.

[JM]: It's not magic.

[JM]: It's just hard work.

[JM]: And it's something that people can do if they find themselves in a situation where they're trying to build an audience.

[JM]: There's no shortcuts.

[JM]: There's no super easy path.

[JM]: There's just one small step at a time that eventually ends up achieving results if you're persistent, lucky and some combination thereof.

[JM]: Okay, moving on.

[JM]: One of the members of the original team that created the Django web application development framework went on to help create a site called SoundSlice.

[JM]: And one day he noticed something strange in the site's error logs a few months ago.

[JM]: He noticed that users kept uploading ASCII tablature, which is a music notation system for guitars, despite the fact that SoundSlice doesn't actually accept ASCII tablature and process it, and had never advertised or stated on its site that it could do that.

[JM]: And nonetheless, the error logs included images of what the users had uploaded,

[JM]: and a lot of them were screenshots of ChatGPT conversations, where ChatGPT had generated these ASCII tabs and told the users to upload them to SoundSlice.

[JM]: So over a period of a month or two, on average, SoundSlice was getting five to ten images per day, which is enough for folks at SoundSlice to say, "Okay, what is happening here?"

[JM]: "Why are people uploading these tab files?"

[JM]: And looking at these images, it was clear to them what was happening that ChatGPT was telling users to do this.

[JM]: And so they had a few options in terms of how they were going to handle this one, they could just ignore it and know that five to ten times a day, roughly, people were doing this and being frustrated when it didn't do what this large language model told them it could do.

[JM]: ... or they could do.

[JM]: Another thing that they could do in response was to put notices on the site saying, "Hey, if you're using ChatGPT, and it tells you that you can do this, this feature doesn't actually exist."

[JM]: Neither of those two things is particularly satisfying as someone who is working on say, a product like this and is excited about it and has pride in what they're building.

[JM]: So

[JM]: Perhaps unsurprisingly, but also I think something that's just awesome is that instead they decided to spend a few hours and actually develop this feature.

[JM]: So to recap, ChatGPT hallucinated a SoundSlice feature.

[JM]: The developers noticed that that's what was happening and decided to go ahead and build that feature because clearly the demand was demonstrated.

[JM]: This is what users apparently wanted to do.

[JM]: And there's no way for them to easily reach out to the people at OpenAI and say, "Hey, your large language model is hallucinating a feature that doesn't exist.

[JM]: Maybe tell your model to stop doing that."

[JM]: That's not something they could easily do.

[JM]: So instead, something that was relatively easy for them to do, which is they could just implement the feature.

[JM]: So I love that someone encapsulated this phenomenon as "gaslight-driven development".

[JM]: Yeah,

[JM]: which might be my favorite term this week.

[DJ]: You know what stands out to me about this, Justin, is that the headline of the article contains a very specific word.

[DJ]: "ChatGPT hallucinated a SoundSlice feature, *forcing* human developers to add it."

[DJ]: ... which sort of makes it sound like ChatGPT showed up at their office with baseball bats and loomed over them until they had added the feature.

[DJ]: Or I suppose what it's saying is now that this unreliable information system exists,

[DJ]: so popular people feel pressure to make their products do what ChatGPT claims it can do. Or the way I think someone in the article mentioned, it's similar to what's practically a cliché and software product development where the sales team enthusiastically assures a potential customer that the software has a feature that it doesn't really have.

[DJ]: And now, well, you guys better add that feature because we just sold a $200,000 deal based on the notion that it exists.

[DJ]: That's kind of a classic.

[DJ]: What actually stands out to me about this story, and I think you mentioned to you too, is...

[DJ]: There's actually something kind of cool about the SoundSlice developers realized there was demand for a feature that was within their capability to add.

[DJ]: So they did.

[DJ]: I think the slight issue I take with the use of the word "forcing" in the article headline is I presume that if it was totally infeasible for SoundSlice to add that feature, they would not have felt forced to do it.

[DJ]: Right?

[DJ]: They would have found some other way.

[DJ]: And that other way might be a customer support burden, where now people are writing in being like, "Why doesn't this feature work?"

[DJ]: And you have to write back and be like, "Look, I'm sorry that that feature doesn't exist.

[DJ]: And ChatGPT is not reliable."

[DJ]: Right?

[DJ]: The thing this reminded me of was a sort of anecdote or parable called "Paths in the Grass" that I heard.

[DJ]: I think it was related to Steve Jobs and Apple.

[DJ]: This was many years ago.

[DJ]: It was supposedly at some American college campus when they were sort of building out a new campus, and instead of paving pathways between buildings, they just planted grass.

[DJ]: And then for the first year, the students had to just walk through these grassy fields to get to the buildings.

[DJ]: And so by the end of the semester, there were these natural paths worn through the grass in the places where people would naturally choose to walk.

[DJ]: And then that summer or whenever, they just paved those so that instead of trying to plan ahead of time for where people would walk, they just watched the way people actually behaved and then accommodated that behavior.

[DJ]: I like that parable when it comes to things like product design, because that is often a valuable strategy.

[DJ]: And so I actually like things like this where, you know, they could have said like, "Boy, it's so annoying.

[DJ]: The ChatGPT is just making stuff up and then people expect it to exist."

[DJ]: And I mean, I agree that is in fact annoying.

[DJ]: However, it's also illuminating because if ChatGPT had said, "Hey, SoundSlice can do whatever", and then no one had tried to do that, it wouldn't matter, right?

[DJ]: There's no demand.

[DJ]: The fact that people were *trying* to do it suggests that this is something our users want to be able to do.

[DJ]: That's actually valuable information.

[JM]: Yeah, I think it's pretty clear that they were very much annoyed, just to be very clear about how they perceived it.

[JM]: And as an example of that, this person when interviewed said, "ChatGBT is making product recommendations for existent and non-existent features alike to massive audiences with zero transparency into why it made those particular recommendations and zero recourse."

[JM]: So I had a similar reaction when I saw the headline.

[JM]: I'm like, "forcing" human developers to add this feature?

[JM]: What?

[JM]: I don't know about "forcing".

[JM]: And the more I thought about it, I realized, yeah, actually, they kind of are being forced because the other options that they evaluated

[JM]: are so unpalatable, right?

[JM]: Like just ignoring the problem like it's not there and just letting users be frustrated by it or putting up some banner on the site that says, "Hey, this thing that you clearly have demonstrated that you want to do, like, yeah, that feature doesn't exist.

[JM]: And sorry that this silly large language model hallucinated it."

[JM]: Neither of those options are very appealing.

[JM]: And so I think they...

[JM]: very much were forced to have this feature.

[JM]: But despite the fact that it's annoying for I think both users to be told that there's this feature that doesn't exist, and for developers to have to be on the receiving end of that and have to deal with it...

[JM]: The good news, I suppose, in the end, is that here was something that users wanted to do.

[JM]: And eventually, the developers realized that users wanted to do this.

[JM]: And so they spent a little bit of time and they built a feature.

[JM]: And you know,

[JM]: everyone's happy in the end.

[JM]: So I think there is like a positive take on this whole story.

[DJ]: Yeah, I think both things can be true that there's that positive take.

[DJ]: And also, this is an obnoxious state of things that we live in, at least now, because as a user, it's also annoying when you use a system like ChatGPT, the model for how it presents information just lends itself to being misleading.

[DJ]: And we've discussed this in the past in a variety of times when we've talked about

[DJ]: how large language models behave.

[DJ]: Because these systems are designed to predict the appropriate output for your input, they are biased towards, they seem to be anyway, biased towards like agreement and not rejection.

[DJ]: Fundamentally, they seem to be bad at saying no.

[DJ]: So if you say, "Can I do this thing with this product?"

[DJ]: They are more likely, it seems, to say, "Yeah, you can.

[DJ]: Here's how you do it."

[DJ]: Not because they necessarily have good or bad information about the product itself, even, but because...

[DJ]: an affirmative response might just be weighted as more likely given your input than a negative response.

[DJ]: But I've encountered this before where I would ask, you know, "I'm running a certain version of the SQLite database library, for example, and I'm shown an example for how to do a certain thing.

[DJ]: And that doesn't work."

[DJ]: It's like, say here, "Call this function with two arguments" ...

[DJ]: and the error message I get is, this function doesn't accept two arguments.

[DJ]: So I look into it, and I look at the actual documentation and realize, oh, they added the second argument in a newer version.

[DJ]: And so as a test, I'd gotten this code snippet from ChatGPT, and I went back to ChatGPT and said, "Hey, I'm on version...

[DJ]: whatever of SQLite, and it doesn't support this.

[DJ]: When did this get added?"

[DJ]: And, you know, it caught quote confidently, but it just says basically like, "Oh yeah, of course that was added in version, you know, 25 that was released in...

[DJ]: whenever, like July of 2024". So then I go over to SQLite's release notes and find out that's wrong -- it was actually added in a totally different version on a different date, so at this point I'm just kind of doing this for my own edification because I have the information I need.

[DJ]: I put into ChatGPT, "Actually, it seems like, you know, that was added in version 36" or something like that.

[DJ]: And I got a response like, "Oh, that's right.

[DJ]: It was added in version 36, which was released in July."

[DJ]: And I look at the release notes.

[DJ]: I'm like, it was November, actually.

[DJ]: But OK, close enough.

[DJ]: But that's the kind of the position I've adopted now is that I find these chatbots useful, but you got to be skeptical.

[DJ]: And it's inherently difficult to be because of just like the way our brains process natural language when this chatbot, like I said, it appears to respond differently.

[DJ]: "Confidently."

[DJ]: I put confident in scare quotes because as far as I know, there is no such concept inherent to the computer program.

[DJ]: At least it doesn't show you one.

[DJ]: It doesn't tell you how confident it is.

[DJ]: But the problem is,

[DJ]: it generates natural language, and our brains naturally interpret that as having a level of confidence.

[DJ]: If ChatGPT's output said things like, "What you really want is to be able to upload ASCII tablature to SoundSlice, but to be perfectly honest with you, I...

[DJ]: I don't remember if they actually let you do that or not.

[DJ]: You should probably go to their web site and check."

[DJ]: We would have a very different article on our hands, probably.

[DJ]: But the system is not built to do that.

[DJ]: So I've found I just have to train myself to be... How does the saying go?

[DJ]: Trust but verify.

[DJ]: Take ChatGPT's like, "Oh, you should do X and Y and Z", and go, okay, that sounds like a good idea, but...

[DJ]: Remember to double-check that that's actually true or correct.

[DJ]: What I wonder is, are most people going to learn that as they use as this thing gets more popular and as people use it over time?

[DJ]: Is the average person going to learn what its limitations are or not?

[JM]: There have been more than a few times where I have thought about that same "trust but verify" phrase as it relates to like how you should approach large language models.

[JM]: And then I realized just how ill-suited that phrase is for this situation, because you should absolutely *not* trust them.

[JM]: It's not a question of "trust but verify" -- you should just *not* trust what you're being told, and also verify it.

[JM]: Because, like you said, there is probably some degree internally, right, to which this tool can assess some degree of confidence about the answer that it's giving.

[JM]: I don't know.

[JM]: Maybe it can.

[JM]: Maybe it can't.

[JM]: I'm over here pontificating about it as if I know that that's true.

[JM]: I don't know.

[JM]: But let's just assume for the sake of argument that it can come up with some degree of like, "Okay, I feel I am 90% confident."

[JM]: "I am 27% confident", whatever it is.

[JM]: But if they could do that, it would be really interesting if it could say, as you suggested, "Well, I think that ...", or "I believe ...", or...

[JM]: "I'm not sure, but maybe ...", but if we've learned anything about human nature, we thrive on confidence and confidence is what we look for in humans.

[JM]: Who's surprised that that's probably what we're looking for in our large language models.

[JM]: And so that is what is delivered to us.

[JM]: Another thing that occurs to me is in this particular case, right,

[JM]: because I keep wanting to take like a positive approach to this particular piece of news.

[JM]: And in this case, the solution...

[JM]: was there, it only took a few hours, everyone's happy.

[JM]: But the reality is, the solution only took a few hours to implement to build *this* feature.

[JM]: But what if the large language model confabulated something that's wildly infeasible to implement. Then there's no great solution, is there?

[JM]: Now the developers are back to square one in terms of the other options that they have, right?

[JM]: Which is, well, we can't really reach out to the company that makes this large language model because that's unlikely to go anywhere.

[JM]: We could just ignore it.

[JM]: We could throw up a banner.

[JM]: None of these are great solutions.

[JM]: And it's probably even more likely in the general case that some large language model is going to confabulate something that is not easy to implement.

[JM]: And that's not great.

[JM]: I suppose if that happens, then what folks can do on the development side, on the site that, say, is noticing that some large language model is hallucinating something that the product doesn't actually do, is they can put up a banner that says, "Hey, this large language model hallucinated this feature, this doesn't actually exist.

[JM]: And here's how you can reach out to the people that make this tool to let them know about it."

[JM]: And if enough people do this kind of thing,

[JM]: then maybe this becomes a competitive advantage and certain large language models will get used more than others.

[JM]: And maybe folks will, once again, vote with their feet and go use a different tool.

[JM]: So maybe there is a way to take a positive spin on this.

[JM]: All right, that's all for this episode.

[JM]: Thanks everyone for listening.

[JM]: You can find me on the web at justinmayer.com and you can find Dan at danj.ca.

[JM]: Please reach out with your thoughts via the Fediverse at justin.ramble.space.