Into the Disintegration Chambers

[JM]: I want to mention a new tool that I've been trying called Whispering.

[JM]: And my motivation with this tool is I spend a lot of time typing on a keyboard, and I like typing, and I like keyboards as we've talked about before, but a lot of typing does at some point get tiring.

[JM]: It has a physical effect on the body, at least for me.

[JM]: I want to spend more of my time, when possible, dictating, and less time typing out words.

[JM]: Sometimes, depending on what I'm doing, that's not really feasible.

[JM]: If I'm writing code, if I'm typing things that dictation software is not particularly likely to record very well,

[JM]: then typing is gonna be a much better way to do it.

[JM]: But composing email, text messages, the type of text I would often compose for a web form, like a GitHub issue, for example, these are kinds of things where dictation software can really do a lot.

[JM]: And it's really transcription, right?

[JM]: At the end of the day, it's taking my voice and it is transcribing it into text.

[JM]: And there are many tools that do this.

[JM]: One of the more popular ones for MacOS is called MacWhisper, which I believe is a application wrapper around an open-source transcription tool called Whisper, which is why you have MacWhisper and you have this tool that I'm talking about now called Whispering.

[JM]: And I've heard great things about MacWhisper.

[JM]: I think it's about 60 US dollars, give or take.

[JM]: But when I can, I prefer to reach for an open-source solution first and see if it can meet my needs.

[JM]: And sometimes it does, and that's great.

[JM]: And sometimes it doesn't.

[JM]: And I'm more than happy to pay someone to provide a tool that's going to be sufficiently improved over, say, the open-source alternative.

[JM]: In this case, so far, I'm really enjoying Whispering.

[JM]: The way it works is I have my cursor in a text field, whether that's an email composition window or in the Messages app where I'm composing a text message, and I hit command-shift-semicolon, which is I think the default key binding, and I will hear this audible sound...

[JM]: that means that it is listening.

[JM]: I'll then dictate what it is that I want to say.

[JM]: I push that same key combination again and then hear another sound.

[JM]: And then depending on how long I was talking, I will either very shortly see the transcription or I will have to wait for a while and then I will see the transcription.

[JM]: And it both puts the transcription on the clipboard as well as pastes it wherever the cursor is when you originally push the key combination.

[JM]: I feel like the transcription is very good.

[JM]: I'm using an open-source model, or an open-weights model, from Nvidia called Parakeet TDT, which provides a pretty good balance of resource consumption like RAM, transcription speed, and accuracy.

[JM]: I do wish the transcription were faster because there is lag for longer recordings and it does seem proportional to the recording length regardless of how much silence there was, because sometimes I'll say a sentence and then I'll think for a second.

[JM]: I should probably just go ahead and push the key combination and have it generate where I am so far.

[JM]: But it seems like if there's long periods of silence, that it takes just as long than if I were speaking that whole time.

[JM]: So if you are gonna talk for say 20 seconds and then push the button, you're gonna wait for a while.

[JM]: Don't expect it to just like pop up immediately.

[JM]: But if you say something relatively quickly, then you'll get a much quicker response.

[JM]: That's just been my experience so far.

[JM]: This Parakeet TDT model handles capitalization and punctuation automatically.

[JM]: So for me, it's a bit of an adjustment.

[JM]: If I say something something comma something something period, then I end up with the words "period" and "comma" in my transcription, which is not how it works with, for example, MacOS and iOS dictation.

[JM]: So if I frequently am going back and forth between these two different tools,

[JM]: then I have to remember which one I'm using, because on one of them I have to say those periods and commas. On the other one...

[JM]: I definitely shouldn't unless I want those words polluting my transcription.

[JM]: The development velocity of this project is great.

[JM]: I'm really excited to see how much work is going into it, how new features are being added at a rapid pace.

[JM]: The interface is definitely interesting.

[JM]: It's not particularly standard.

[JM]: So for example, I'm used to being able to hit command-comma to get to the settings and that does not work in this application.

[JM]: And I frequently have to look at this main user interface and stare at it and think, where are the settings, then I

[JM]: instinctively want to get to them fast so I hit command-comma and nothing happens, but I eventually see the little gear icon and okay there it is

[DJ]: You'd think for an app like this you could just say the word settings and it would open the settings menu like come on that just seems obvious

[JM]: So far it's just a transcription tool it doesn't allow you to use voice to control its own behavior but it does have like a always-listening mode and then like the key-command mode that I use where it I guess detects...

[JM]: when you're trying to dictate, I don't really know.

[JM]: I mean, I guess it just, if you're in a field and you start talking, it'll just start putting stuff in the field.

[JM]: I don't really know.

[JM]: I haven't tried that mode.

[JM]: The latest version that just came out is interesting because instead of an embedded database, there's a shift now to storing all of the data, the recordings, the transcriptions,

[JM]: and the transformations, which I'll get to in a moment, as regular Markdown files with YAML front-matter.

[JM]: And the idea is that instead of using IndexedDB, which was the tool that was used initially when the idea was that Whispering was gonna be a web-first tool, but now that the focus is more on a desktop app,

[JM]: it doesn't really make sense to be using this browser-based database to store this data.

[JM]: And so the author recognized this is now the wrong tool for the job.

[JM]: The data is opaque.

[JM]: Making backups requires special tooling. Import/

[JM]: export is complicated.

[JM]: You don't really own your data.

[JM]: That's the main thing that the author was focused on and wanted to get it out of this database and into plain text files that you can easily read, edit, backup, and move around freely.

[JM]: So I think this is a really cool change and sets the stage for easier syncing between devices so that you could sync your recordings, transcriptions, and transformations from one computer to another,

[JM]: makes it easier to backup your data, restore that data from backups and a bunch of other things.

[JM]: I mentioned transformations.

[JM]: I haven't used this yet, but it seems interesting because the idea is that you can apply a set of transformations to your transcription.

[JM]: So if for example,

[JM]: you are a non-native speaker of English, maybe this allows you to automatically give it some kind of LLM-based command, like review the transcription for grammatical errors and fix them.

[JM]: Again, I haven't used the transformation feature.

[JM]: So I don't really understand what it does and how it works and what those use cases are.

[JM]: But it does sound interesting that you can do that. I haven't found a particularly notable use case for it yet, but I'm sure they exist.

[JM]: If you visit the GitHub repository for this tool, the first thing you'll notice is the location of the app and its README is not in the project root where you would expect it.

[JM]: It's somewhat buried inside other folders.

[JM]: And the reason is that Whispering is part of a larger envisioned project called Epicenter.

[JM]: With the idea being that there's going to be, or I guess there already is, this Epicenter Assistant, which is this local-first assistant that you can chat with.

[JM]: And the idea is that it will become the access point to everything you've ever written, thought, or built.

[JM]: And on the web site, it says, "A database for your mind built on plain text, one shared context across all your apps."

[JM]: "Your transcripts inform your notes."

[JM]: "Your notes guide your AI."

[JM]: "Everything connects through a single folder of plain text."

[JM]: And the README says that this Assistant is currently unstable and is waiting for a pull request in another project that I think it depends upon to be merged.

[JM]: And I think this is interesting from the context of, here's an agent that is running locally, that isn't making external API calls unless you decide that's how you want to do it, that can look at, say, all of your Markdown-based notes

[JM]: and other projects that you work on and can be queried in interesting ways.

[JM]: I'm not really sure exactly how I would use this Epicenter Assistant if and when it becomes more stable, but it does seem interesting that Whispering is a part of it.

[JM]: And I look forward to seeing what it might eventually be able to do.

[DJ]: I'm interested in checking this one out because I'm a big fan of so-called local-first software, where the ethos behind the design of a piece of software is that it's based around using the resources of your computer, things like your file system, etc., instead of what

[DJ]: often feels like this sort of automatic and perhaps even inevitable way that software is being built where all the data is on a server somewhere and you probably access it through your web browser.

[DJ]: So it always makes me happy to come across projects like this that offer an alternative to a popular category of apps because this idea of like put all your data somewhere and we'll help you organize it

[DJ]: has been around for a while and it's very popular it reminds me of these so-called like second brain apps that got popular over the last five years apps, like Notion is a popular example, and then a more recent addition to that idea of "put all your data somewhere and we'll help you organize it" of course is,

[DJ]: "Put all your data somewhere and we'll use an LLM to help you organize it."

[DJ]: But again, I don't really want to use an application like that if it requires all my data to be possessed by some company and stored in a data center somewhere.

[DJ]: That's not how I'd prefer to roll.

[DJ]: So I'm really excited to see apps like this and the rise of tools for using large language models locally on our own hardware.

[DJ]: And of course, those models becoming more and more able to run on local hardware because it means projects like this that can give us a local-first alternative.

[DJ]: I think that's really exciting.

[JM]: Okay, moving on.

[JM]: In other news, Apple has released the 26.1 series of operating systems.

[JM]: So that includes the iPhone, MacOS, iPadOS, all of the various OSs.

[JM]: And I don't think either of us have a whole lot to say about 26.1.

[JM]: I'm not running any of the 26 OSs.

[JM]: The only reason that I thought this was notable really comes down to two things.

[JM]: One, it sounds like

[JM]: Apple has heard all of the many complaints about their Liquid Glass design language and has decided to give folks the option to tone it down a bit by adding some opacity, somewhat of a

[JM]: frosted effect on the Liquid Glass and giving folks perhaps a less-glass option to make certain visuals a little bit easier to read, providing more contrast, providing perhaps less instances in which some floating control that is transparent makes that control unreadable because it is floating above some other thing and making the effects mostly illegible.

[JM]: The other thing that I find interesting potentially about this is, as we have talked about many times, I do not turn on automatic updates.

[JM]: But I know folks who do have automatic updates turned on in part because they won't go and

[JM]: do it otherwise.

[JM]: And so I wonder now that Apple has released this point release whether they will turn on — or maybe they have already turned on — automatic updates to this version because there's always like this grace period where Apple will automatically update you

[JM]: to essentially last year's version until they feel like the current one is baked enough to foist it on you.

[JM]: And at least up until yesterday, that had not been the case.

[JM]: If you had automatic updates turned on, you would get automatic updates to 18 point whatever.

[JM]: with this big banner telling you, hey, if you want to, you can manually update to the 26 version.

[JM]: But at some point, Apple is going to flip this switch and people are going to wake up the next morning and see this very significant change to their everyday environment.

[JM]: And I don't know, maybe...

[JM]: folks will be like, "Oh, this is so cool."

[JM]: I don't know.

[JM]: But I know that loved ones that I know who have automatic updates turned on, I will be sending them all a message today that says, "Hey, you might want to turn it off for like, I don't know, a month or two to give Apple enough time to work out perhaps even more kinks, visual or otherwise, to the 26 line of operating systems."

[JM]: Whether they choose to do that or not, is obviously up to them.

[JM]: But I feel like it is my duty to at least let them know that this will happen unless they take action at some point.

[JM]: Moving on to one of my favorite news stories of late is the introduction of the Neo humanoid Robot, a $20,000 humanoid robot that will do your household chores with some fine print involved.

[DJ]: What is the fine print involved?

[DJ]: Like, also it watches you sleep?

[DJ]: Yeah.

[DJ]: May turn on humanity.

[DJ]: It might end up having a switch that's set to evil, like in that old Simpsons Halloween episode with the Krusty doll that tries to kill Homer.

[DJ]: I assume all of that is in the fine print.

[JM]: No doubt.

[JM]: Well, for one thing, it's not just $20,000.

[JM]: It's $20,000 and it's $500 per month.

[JM]: And there is a six month commitment.

[JM]: So you're already looking at a minimum of $23,000 over the first six months.

[JM]: That's your minimum investment to try this thing.

[DJ]: Sorry.

[DJ]: Oh no, Justin, I think I've entered a fugue state, and I suddenly woke up and I was negotiating for the purchase of a car.

[DJ]: What happened?

[JM]: My favorite part about this is that this so-called robot is not fully autonomous.

[JM]: There are certain tasks that perhaps it doesn't quite have the fine motor skills to accomplish.

[JM]: And when it is presented with say one of these tasks, a human will take over and the robot will be controlled remotely by a human wearing a VR headset

[DJ]: Well, okay, hold on.

[DJ]: Hold on.

[DJ]: Sorry.

[DJ]: A human wearing a VR headset is funny enough, but you said that there are tasks for which it lacks the fine motor skills.

[DJ]: And when it can't do that, a human will take over.

[DJ]: But if it lacks the fine motor skills to do a task, that means they actually have to send a person to your house, right?

[DJ]: Like to do the task instead.

[DJ]: Because I don't see how a remote operator with a VR headset makes up for a lack of fine motor control.

[JM]: I don't fully understand how this mode works.

[JM]: Like what things can it do autonomously?

[JM]: Like, is there a human whose job it is, 24 hours a day to monitor each and every one of these, like in real time and essentially babysit it and then be like, "Oh wait, it's having trouble with this thing."

[JM]: And then it just like

[JM]: flips the manual override switch and then suddenly takes control over it.

[JM]: I don't really understand how this switch occurs, but the fact that it occurs at all to me is just hilarious.

[DJ]: I have so many questions.

[DJ]: So do they have a warehouse full of people sitting next to each other in chairs wearing VR helmets like in some horrible Japanese anime dystopia?

[DJ]: Or is it like a gig economy thing where it's like, operate this robot from the comfort of your own home sometimes when it can't put a plate away?

[JM]: I'm fairly confident that is how this is designed to work, yes.

[JM]: This thing is controlled remotely by humans wearing VR headsets looking at you in your pajamas through the robot's cameras while it tries and mostly fails to load the dishwasher.

[JM]: And I saw someone, maybe the founder of this company, but someone who otherwise represents the company, comment on this saying, "For the product to be useful, you have to be okay with this."

[JM]: And "this" being the fact that a human is looking through the robot's cameras,

[JM]: at your home, the inside of your home.

[JM]: So, you know, I feel like the tagline for this thing should be: "$20,000, $500 per month, and the privacy of your home are a small price to pay for a digital peeping-Tom who can almost do your chores."

[DJ]: Oh man, I think that marketing copy could fit in a tweet as well, so we're basically set.

[DJ]: This is such a stupid idea, and it's amazing and stupid, and it also feels inevitable and stupid.

[DJ]: It's such a perfect combination of everything I feel like we complain about with regards to the world of technology that we live in.

[JM]: The tech *business*, right?

[DJ]: Yeah, exactly.

[DJ]: The tech business.

[DJ]: And the other thing is that someone would make that statement, and there's such a thing as telling on yourself, right?

[DJ]: Whatever the CEO or whoever thinks they're accomplishing with that statement, you can turn it around, because it's like, now, in order for this to be useful, you have to be okay with random people spying on you through the creepy eyes of a humanoid robot.

[DJ]: Like, okay, the inverse of that is, since many of us are not okay with that, therefore, your product is not useful.

[DJ]: So, good luck with your Series A round, I guess.

[JM]: Yeah, because to me, for a robot to be useful, it should be a robot, which this is arguably not.

[JM]: If a human has to remotely control it, it's not really a robot.

[JM]: It is a remote-controlled machine.

[DJ]: Yes.

[DJ]: And it's also like so many of the creepiest parts of, again, the tech business in the 21st century is that it's like a super awkward recreation of almost the social mores of a different era.

[DJ]: Because people used to hire people to live in their houses and unload their dishwasher.

[DJ]: Well, they didn't have a dishwasher, but you take my point, right?

[DJ]: Like people used to have servants, right?

[DJ]: And so the promise ever since, like the cartoon The Jetsons, I guess, was that someday, you know, there are lots of problems with having other people as servants.

[DJ]: We don't have to get into those here.

[DJ]: I think that's well covered elsewhere for the last hundred years.

[DJ]: But wouldn't it be nice if you didn't have to do your chores?

[DJ]: Therefore, robots.

[DJ]: That's been a longtime science fiction fantasy.

[DJ]: So now we're trying to make it a reality.

[DJ]: But how do we make it a reality?

[DJ]: Well, the only way we can do that right now is to have this simulacrum that's actually backed by a human being.

[DJ]: So again, you're essentially paying twenty grand plus $500 a month, plus taxes and fees probably...

[DJ]: to have a servant. It's just that the servant is presumably on a couch somewhere with a VR helmet,

[DJ]: so that they don't even have to be in your house, which is like the most Millennial thing I can imagine, right?

[DJ]: Which is that like, of course you want a servant, but the worst thing you can imagine is having to interact with another human being.

[DJ]: So no, we'll just pop this creepy-looking felt and metal mannequin in there.

[DJ]: So the person can be somewhere else while they try and fail to put away your dishes and see you in your pajamas.

[DJ]: They can still see you in your pajamas, Justin.

[DJ]: And that's the real problem.

[DJ]: I mean, frankly, if I bought one of these things, I'd have to seriously up my pajama game and I'm just going to leave it at that.

[JM]: It's so emblematic of the whole "Fake it till you make it" tech phenomenon, right?

[JM]: Like if you can't make it work, then just fake it.

[JM]: I wish this were the first time that we've seen a report of some thing that's amazing, some automated thing that we find out later on...

[JM]: is not actually automated, but a human is doing it manually behind the scenes.

[JM]: We've seen this happen.

[JM]: Well, I suppose at least in this particular case, it's up-front, right?

[JM]: They're not saying, "Hey, here's this robot that's operating autonomously."

[JM]: Like they're at least saying from the get-go, clearly this thing is at least in part controlled remotely by a human.

[JM]: And I feel like recently, before this announcement, I saw someone on the Internet saying,

[JM]: who's a programmer who essentially said something along the lines of, "I don't want AI to write my code, generate my books, music, movies, or other cultural content."

[JM]: "I just want it to do the menial tasks and chores that I don't want to do."

[JM]: And ask and you shall receive, I guess.

[JM]: Like, it seems like not too long after I saw this, this product was announced.

[JM]: And of course,

[JM]: I just love the footnote.

[JM]: Yeah, well, guess what?

[JM]: It's still a human doing it anyway.

[DJ]: Yeah, well, I've seen similar complaints where it was like the promise of automation was supposed to be take care of the menial, busy work of human life so that we can spend our time being creative.

[DJ]: And instead, what has been delivered is the exact complete opposite of that.

[DJ]: Where now AI generates music and books and code and everyone who would like to do those things themselves is like, "So do I still have a purpose in the world, or is it time to head into the disintegration chambers?"

[DJ]: And meanwhile, to put your dishes away, you still need a person, presumably not a very well-paid person, by the way.

[DJ]: How much do you think the people in the VR headsets are getting paid to at least sometimes operate these creepy robots in the houses of people that I presume probably make ten times as much as they do?

[DJ]: Yeah, we must continue to be frustrated and perhaps outraged at the world that has been delivered to us and try to build a better world that does not include these weird felt-faced robots.

[JM]: Seems like a good banner to carry.

[JM]: I can just picture this team, right, developing this robot and thinking, "Man, we're so close."

[JM]: "If only we could get this robot to do something useful."

[JM]: "But we can't."

[JM]: "So until we can, we'll just get a human to control it remotely."

[JM]: "And then eventually..."

[JM]: "we'll be able to figure out how to do it programmatically."

[JM]: "Eventually, we'll make it so that this robot delivers its stated mission and operates autonomously."

[JM]: "But until then, we can at least ship something."

[JM]: That seems like what's actually happening here.

[JM]: And there's just something about it that is just so both sad, amusing...

[JM]: I don't know how else to describe it.

[JM]: There's probably other adjectives that I'm missing here, but I don't know.

[JM]: The whole thing is just, I think it's mostly amusing.

[DJ]: Ridiculous.

[DJ]: I think "ridiculous" is also a good one.

[DJ]: Yeah.

[DJ]: And the thing about that, what you just described is it's totally the minimum viable product ethos, but that works a lot better when you're talking about like a web app

[DJ]: where certain features don't quite work yet, or they're like implemented, but they're implemented in a really clunky way.

[DJ]: And once the company gets a thousand customers and starts generating some revenue, they can re-write their backend or something like that, right?

[DJ]: Like software is super malleable in that way.

[DJ]: That model feels, I think, again, like it feels a lot more ridiculous when it's applied to so-called, like, hard, like

[DJ]: physical-world problems like this.

[DJ]: Whereas it's like, we want these autonomous robots, but they're not autonomous.

[DJ]: So they're just going to be remote-control robots for now until we can make them autonomous.

[DJ]: But it's like, well, for starters, do you see any clear path to actually accomplishing that?

[DJ]: Or are you just hoping that someone will hand you another quarter of a billion dollars so you can keep picking away at it, I guess?

[JM]: Okay, moving on.

[JM]: In a recent announcement, OpenAI said, "Today we're introducing ChatGPT Atlas, a new web browser built with ChatGPT at its core."

[JM]: "With Atlas, ChatGPT can come with you anywhere across the web, helping you in the window right where you are, understanding what you're trying to do and completing tasks for you all without copying and pasting or leaving the page."

[JM]: "Your ChatGPT memory is built-in, so conversations can draw on past chats and details to help you get new things done."

[JM]: "ChatGPT Atlas is launching worldwide on MacOS today to Free, Plus, Pro, and Go users."

[JM]: "Experiences for Windows, iOS, and Android are coming soon."

[JM]: My first thought seeing this was, "Experiences?"

[JM]: Gross.

[JM]: Why not say "Atlas for Windows, iOS, and Android are coming soon"?

[JM]: What do you mean, "experiences"?

[JM]: What is with this marketing language?

[JM]: I don't understand it.

[JM]: Also, I think it's interesting that Atlas is out on MacOS before any other platform.

[JM]: Remember our conversation from last week about how MacOS used to be the nerdy kid at school that was different and weird and got picked on?

[JM]: Well...

[JM]: Look how much has changed since those days.

[JM]: Now all the other platforms are out in the cold, and MacOS is taking center stage for this announcement.

[JM]: Anyway, Atlas is a Chromium-based browser that can't be used at all without signing into ChatGPT.

[JM]: And Atlas is by no means unique.

[JM]: There are a increasing number of browsers being shipped these days with large language model capabilities built into them.

[JM]: Apparently Atlas has an "agent mode" somewhere that's not super easy to find.

[JM]: And I haven't heard anyone really talk about this agent mode very much, particularly in a code generation and testing context, which I think is perhaps the only use case that I personally have any interest in.

[JM]: So at this point, I don't really have any motivation to even install and try this product.

[JM]: If it did, for example,

[JM]: have the ability to generate, run, and test code, all in a browser context in a super easy way, I could see that being useful.

[JM]: But outside of that, I personally don't see many interesting use cases for this class of product.

[JM]: Looking at the announcement, it seems like even the folks at OpenAI struggle to find compelling use cases.

[JM]: To me, it feels like Chromium with a chat button bolted onto it.

[JM]: I don't understand at this stage why someone would prefer this to a combination of a standard web browser and, say, the native ChatGPT Mac application. Combining them into one thing? That...

[JM]: doesn't seem to provide much utility.

[JM]: Before I dig into some of the problems that I have with this class of product... Dan, do you see lots of interesting use cases here that I'm missing?

[DJ]: I don't, but I can't say that that's because I couldn't see them.

[DJ]: It's more that I haven't looked for them.

[DJ]: It's very clear to me that this type of product is not for me.

[DJ]: I assume it's targeted at...

[DJ]: the average human as opposed to a technology enthusiast, because ChatGPT itself is like massively popular with the mainstream as I understand it, right?

[DJ]: I think they said they have like 700 billion accounts or something like that.

[DJ]: There's like 20 trillion humans that use ChatGPT every day or something.

[DJ]: Those sound like totally real numbers.

[DJ]: Yes.

[DJ]: Yeah.

[DJ]: I mean, no, those numbers are commensurate when you look at like how much money OpenAI is like promising to spend versus the amount of revenue they make.

[DJ]: Yeah.

[DJ]: I mean, it all just kind of adds up anyway.

[DJ]: I guess you could call me a bit of an OpenAI-skeptic at this point, both for their financial future,

[DJ]: and also just, as I said before, with local-first software, um,

[DJ]: I'm just reaching a point where every new window I open that says enter your username and password or create an account starts to feel like an amount of friction I don't want to take on.

[DJ]: I start asking, "All right, do I really need this actually?"

[DJ]: I actually have several web browsers already on my computer that let me browse the web without needing to sign into an account.

[DJ]: I don't get why I would want this.

[DJ]: ChatGPT bolted onto my web browser.

[DJ]: Like I use ChatGPT.

[DJ]: It's very useful actually.

[DJ]: And in a way, querying a large language model instead of doing a Google search and then trolling through web pages is a more effective technique I've found for like finding certain classes of information or solving certain classes of problem.

[DJ]: Especially in a professional context.

[DJ]: Like I ran into some weird bug and my app isn't building and I don't understand what this error means.

[DJ]: Well, I used to paste an error message into a search engine and then click on the first six Stack Overflow posts and hope that the information in there would give me the solution, which can often be effective but is not that easy.

[DJ]: A lot of the time, ChatGPT provides an experience that is at least no worse and often better than that.

[DJ]: But that being the case, I don't see why I need to combine the chatbot with the web browser.

[DJ]: But so I'm going to have to leave it up to other people to explain why this is awesome.

[DJ]: The fact is, I'm probably never going to try it.

[JM]: Okay, so it sounds like you don't really see lots of other use cases that are missing here.

[JM]: And setting aside the question of whether LLM-and-browser mashups provide sufficient utility, I see three primary problems with the entire concept of LLM-infused browsers, regardless of which company is shipping them.

[JM]: The first one is that this feels distinctly anti-web to me because these things are browsers and yet they are not browsers.

[JM]: They are more anti-web browsers than anything else because they don't really behave much like browsers.

[JM]: You ask questions to the chat bot and it gives you answers but it doesn't feel like actually using a web browser.

[JM]: It feels designed to supplant web browsers

[JM]: more than making a web browser more useful.

[JM]: And large language models in general feel like they are designed to supplant web browsers.

[JM]: So I just don't really feel in general that this is something that's good for the web.

[JM]: My next concern with this class of product is security.

[JM]: Because if you think of two things that have been in the news a lot lately, it's AI-powered browsers and NPM supply chain attacks.

[JM]: NPM is a JavaScript package manager that frequently is used to provide software that runs in web browsers.

[JM]: And the intersection of large language model-powered browsers and JavaScript supply chain attacks feels like it's not going to end very well.

[JM]: Jim Nielsen wrote an article on the web, and I'm going to read a portion of it because it explains this much better than I could.

[JM]: "Imagine for a second something like the following.

[JM]: You're an attacker and you stick malicious instructions, not code mind you, just plain-text English language prose in your otherwise helpful library and let people install it.

[JM]: No malicious code is run on the installing computer.

[JM]: Bundlers then combine third-party dependencies with first-party code in order to spit out application code, which gets shipped to end users.

[JM]: At this point, there's still zero malicious code that has executed on anyone's computer.

[JM]: Then end users with AI browsers end up consuming these plain-text instructions that are part of your application bundle.

[JM]: And boom, you've been exploited.

[JM]: At no point was any malicious code written by a bad actor executed by the browser engine itself.

[JM]: Rather, it's the bolted-on AI agent running alongside the browser engine that ingests these instructions and does something it obviously shouldn't.

[JM]: In other words, it doesn't have to be code to be an exploit.

[JM]: Plain-text human language is now a weaponizable exploit, which means the surface for attacks just got way bigger."

[JM]: And I think Jim has really succinctly summarized my concerns about the security risks

[JM]: of this entire class of LLM-infused browsers.

[JM]: My third concern with this whole class of product is privacy.

[JM]: In their announcement, OpenAI says the following, "By default, we don't use the content you browse to train our models.

[JM]: If you choose to opt-in this content, you can enable include web browsing in your data control setting.

[JM]: Note, even if you opt into training, web pages that opt-out of GPTBot will not be trained on.

[JM]: If you've enabled training for chats in your ChatGPT account, training will also be enabled for chats in Atlas.

[JM]: This includes web site content you've attached when using the Ask ChatGPT sidebar and browser memories that inform your chats."

[DJ]: I've noticed that when it comes to LLM companies and privacy, the thing a lot of people seem to have gotten in their heads is I don't want them to train their models on something, my information, my whatever.

[DJ]: I actually think there's a simpler concern even than that, which is merely, I just don't want one more gigantic tech company to have any insight whatsoever into what I'm doing on the web.

[DJ]: I'm just not interested in sending a bunch more information about myself to one more of these companies, because I have no reason to trust their motives or behavior.

[DJ]: So, so I agree with you about like all of the problems that you've just said, but especially with this privacy thing.

[DJ]: Yeah, for me, it doesn't even have to do with, are they going to train a large language model?

[DJ]: Like, I don't think I feel as strongly about that as a lot of people do.

[DJ]: For me, it really is much more just like OpenAI saying like,

[DJ]: "Hey, we want to just have like an open network connection on your computer more often so that we can, you know, and more of the data, more of the things you do on your computer are going to travel down that connection to us."

[DJ]: "Sound good?"

[DJ]: And my response is "No, it doesn't."

[JM]: My biggest concern with this is that people will opt-in to this feature.

[JM]: They will say, I want to include web browsing because of some perceived benefit that this will provide them greater capabilities in an Atlas browsing context.

[JM]: And I think one of the primary motivations for OpenAI to ship Atlas is not to so much ship a browser, but instead to ship something that will circumvent scraping-blocking attempts.

[JM]: To me, it feels more like a distributed human-based scraper than anything else, because large language model scraping is reviled enough that bots from companies like OpenAI, and the IP ranges of companies like OpenAI, often end up in block lists.

[JM]: And plenty of servers outright terminate any connections incoming from these perceived bot scrapers.

[JM]: But what if you didn't need to play a game of cat-and-mouse and could instead convince people to enable this, and then you can use regular residential, civilian IP addresses with organic-looking traffic patterns

[JM]: that can use humans to bypass CAPTCHAs, provide proof-of-work challenges... all of this for free.

[JM]: You don't even have to pay people to do it.

[JM]: People will just volunteer and enable these opt-in LLM browser features.

[JM]: And think of the people who are accessing, like, closed forums that are closed for good reasons.

[JM]: So to me, more than anything, it feels like a Trojan horse where this is going to enable people to access knowledge and information that the scraper bots can't access.

[JM]: And it also potentially means that online safe spaces can be at risk.

[JM]: This idea that they say, "Okay, well, web pages can opt out of the GPTBot and we won't train on that stuff."

[JM]: I don't find this particularly satisfying.

[DJ]: I don't see why we should take a company like this at their word.

[JM]: And I think of all of the places that either don't even know that they can do that, or don't know *how* to do that.

[JM]: So to me, there's this disconnect where the person using the Atlas browser, they're the ones opting into the training and then ingesting data from people who *didn't* opt into it.

[JM]: They have to opt *out* of it.

[JM]: That to me feels like the most sinister, unstated motivation behind this class of product.

[JM]: As for what you can do, you can just use a regular browser and not use Atlas.

[DJ]: Yeah, you can not use Atlas.

[DJ]: Let us introduce you to other browsers you may not have heard of.

[DJ]: Like, well, I don't really want to say Chrome, but at least then you're only feeding your data to Google, which you're probably already doing.

[JM]: Don't use Chrome.

[JM]: Just use Safari, Firefox, Vivaldi, Opera... insert anything else really other than, say, Chrome and browsers powered by large language models.

[DJ]: You might have, like, a retro virtual machine running Internet Explorer 6 just to let you re-live the glory days of 2002.

[DJ]: That could be fun.

[DJ]: Take Justin's advice for what browsers to use.

[DJ]: Mine is bad. Bad advice.

[DJ]: Don't use Atlas and don't do any of the things Dan suggests either.

[JM]: All right, everyone.

[JM]: Thanks for listening.

[JM]: We hope you enjoyed the show.

[JM]: You can find me on the web at justinmayer.com and you can find Dan on the web at danj.ca.

[JM]: Please reach out and share your thoughts about this episode via the Fediverse at justin.ramble.space.