It's very true to me. Another effect I noticed in less tech inclined jobs (but still office jobs) is that they're still convinced "their" special job is not possible to automatize, because it's too complicated. Mind you some of them have really simple jobs, bordering simple data entry in excel. They tried chatGPT once or twice one year ago and don't realize what's coming for them.
Since in tech circles we're the first ones to optimize and use new technologies we are most aware of the huge leaps that happened and the current impressive capabilities but lot are still oblivious to it.
thunfischtoast 6 hours ago [-]
My biggest concern isn't actually that my job can be automated (it can), but rather that the persons who need me stop valueing being right, and start valueing >being told< that they are right by an AI.
I feel my main purpose isn't that I can technically do something my clients can't, but that I know how to interpret their needs and find the solutions that fit for them.
Now they just query some AI which spits out >some< solution and they won't know that they are wrong until it can't even be tracked back to their AI.
This will result in "but the computer said" excuses all over the place.
Loughla 4 hours ago [-]
I think we'll see an unbelievable upheaval as companies figure out that while the technical aspects of some jobs can (and should) be automated, the interpretation of those technical aspects and other higher order, but not strictly inherent processes can't. By upheaval I mean companies closing because they replace staff with AI, and lose things along the way because of it. They won't see it until it's too late, because of the pile of money they're saving.
My example comes from maintenance in machine shops. My dad's siblings were almost all employed in machining in some fashion. One uncle was maintenance for a local large company. They decided they could move maintenance from in-house specialists and contract that work to a multi-state massive company; anyone can follow a maintenance manual right? All you need are the proper certifications and everything is cool, right? The cost was less, and the number of maintenance staff on site actually went up. Great for the owners.
But what they lost was consistency and understanding in the process that eventually led to closing the company. My uncle was there every day all year long for decades and could hear and feel in his feet when something was going wrong with a machine regardless of the maintenance schedule approved by the company that manufactured the machines. The new outside company didn't pay as well, and didn't have great benefits, which meant that they cycled through staff very regularly. No one was there consistently for more than a year or two. Even though they were technically following the manufacturers recommended maintenance schedule, machines went down. Machines then stayed down for days or weeks longer than they should've, because the expertise to repair either wasn't there, or the person leading the job would quit and they would have to cycle in a new maintenance person. They lost jobs, customers took work other places. 8 years later the company closed.
There are lessons in this kind of thing, but nobody learns them.
bobson381 3 hours ago [-]
This is a techne/metis split. You would probably like this [0] chapter about the "rightness" struggle between engineering and service at Xerox. Some paralells to A Big Little Idea called Legibility from Ribbonfarm, too.
The original book that article is based on is great.
2 hours ago [-]
rzerowan 2 hours ago [-]
I believe that the imprtatnt bit that many will come to understand ,maybe after costly mistakes is that 'tasks can be automated, jobs cannot(yet)' one still needs the discernment to know what the sequence of tasks lead up to meaningfully.
It kinda reminds me of the story of the Toyota process , while their car product lines were/are heavily automated the process still relies on the smalll teams on the line to spot/solve and iterarte over mistakes/inefficiences then add the refinements back onto the automated tasks.
Similarly althose stories of radilogists going jobless have not panned out , as interpreting slieds is a 'task' that a radiologist does.
Semantics , yeah but it does shape the way of thinkings regarding jobs/economy and not to mention the whole anthropomorphizing by the AI labs and boosters. In essence the wordings can/could introduce a healthy dose of reality/skepticism to the wildest predictions regarding job losses etc.
trollbridge 1 hours ago [-]
This is already an issue I face both with clients and personally where I can say something authoritatively, but they’ll always go and just ask an LLM.
Sometimes it’s in my favour. LLMs estimate very high prices and long timelines for software projects.
If it’s a legal / regulatory issue it’s usually wrong.
sillyfluke 10 minutes ago [-]
Some internal engineer is doing god's work by creating a system skill for the LLM that multiplies all estimations by five. /s
On a more serious note, the pre-LLM internet had very little useful detailed information regarding software estimation addressing effort for building something in specific domains.
Most writing on software estimation tried to tackle a general approach that could apply to all domains. So you had a "applicable to all, useful to none" situation going. And since crud apps were the most ubiquitous apps, software estimation techniques were perceived to give more accurate results with respect to those type of apps.
I'm sure internally big AI companies are accumulating vast amounts of estimation training data by documenting how long it takes their LLM to complete specific app building tasks. I don't think they would be in a rush to push that data into public models though as it would be like beating a dead horse that is the current human workforce anyway.
elar_verole 3 hours ago [-]
I wholeheartedly agree. To clarify my position, I don't think having these amazing models mean we should replace everyone, quite the opposite.
I do think they are close enough to the stage where people will be replaced, even though it's not the right or correct thing to do, and that we need to move on from the "denial" phase quickly enough so our societies can adapt / regulate / legislate while not too late.
jlengrand 4 hours ago [-]
This, 100 times over. And it's already happening.
baxtr 4 hours ago [-]
Which isn't necessarily a bad thing. Smart leaders / entrepreneurs will realize this and then use human input where required. The rest will be cleared out after the computer has made enough mistakes for them.
sillyfluke 44 minutes ago [-]
>I know how to interpret their needs and find the solutions that fit for them.
Without additional explanation this is indistinguishable from the role of a technical product manager.
digitalsushi 3 hours ago [-]
[dead]
curt15 3 hours ago [-]
The frontier labs themselves are still hiring hordes of human "account associates", "android engineers", and "data scientists" (https://openai.com/careers/search/), when they would be better positioned than anyone to leverage state-of-the art AI capabilities.
samuelknight 14 minutes ago [-]
They can hire more people and still be massively augmented by AI.
trollbridge 1 hours ago [-]
Yeah, and I keep wondering why.
hirako2000 54 minutes ago [-]
Because LLMs are a tool.
They are best placed to not fall for their own marketing claim that their tool can replace engineers.
jp_totsum 4 hours ago [-]
Most people think that using an LLM requires no skill: it is just talking to a machine.
I think it is a soft skill that only hours of practice can make you good at.
Unfortunately, most of the people I know use LLMs as a search engine: their requests are just short sentences. They get basic results and either they stop there, not impressed or they keep using it as a smart replacement for Google.
So they judge what AI can do from their first weak experience.
Going further than that would requires more effort: going deep on a subject they know, thinking carefully about the prompt they write, more importantly judging the answer they get and pushing back.
ethbr1 3 hours ago [-]
Using LLMs well with human language requires precision.
There are only a few professions that are intrinsically precise with language.
No surprise we're seeing adoption in software programming and mathematics.
djaro 3 hours ago [-]
I'm reminded of an interaction I had with a non-technical user. They knew ChatGPT can translate text, and so, to remove spelling mistakes from their (Dutch) text, they would first ask ChatGPT to translate it to English and then ask it to translate it back to Dutch. Mind-blown when I said "you can just say 'fix the spelling and gammar', no need to translate"
This is the level that many non-technical users are operating on.
bsenftner 2 hours ago [-]
I'm currently seeing a lot of developers and AI industry insiders that do not distinguished between the capabilities of their LLM Agent and the added capabilities of the agent harness. They think the capabilities of the agent harness are within the LLM, and in that confusion they are trying to get their agent to make them an agent harness with all these capabilities. But LLM agent harnesses are brand spanking new, how to make these harnesses are not in the training data of their agent, these developers are more or less vibe begging their agents to make them a harness, so it hallucinates the harness architecture. I am seeing a "agent harnesses" that are delusions, the these developers are not strong and experienced enough to realize they are literally doing the worst work they possibly could... And these idiots are the AI leads.
knottn 1 hours ago [-]
> think it is a soft skill that only hours of practice can make you good at
Yikes
onlyrealcuzzo 3 hours ago [-]
It's shocking to me the amount of people that confused "not automatable" with "not cost effective to automate" (which is effectively everything), and the further step which is - the reduction in quality people will be forced to bear to get massively lower costs.
Most people are wrapped in plastic instead of cotton.
lotsofpulp 3 hours ago [-]
>"not cost effective to automate" (which is effectively everything)
There still seems to be a wide gulf in automation capabilities and things that require human dexterity, such as doing things for the infirm. Even just cooking and laundry seem quite a bit away. I guess the original tweet does mention this:
>Capability peaks in domains that are digital, verifiable and economically valuable.
Driving is the most impressive one so far, in my opinion.
tonyedgecombe 4 hours ago [-]
Or maybe AI is particularly suited to programming and a few other niches.
elar_verole 4 hours ago [-]
It is very good at programming. Because programming is a lot of language, and these tools were also made by programmers and post trained to be good at it.
But I assure you it's even "better" (not really but the task is so much easier) at making your spreadsheet look nice and compare actuals vs forecast for your budget than a complicated programming task. It will just one shot it, and it will be objectively correct, all in a very short time. But these non tech people don't know they can use an agent, they have asked one question in the chat interface with 0 context about what they're trying to do, and conclude they are not good enough.
I'm not saying everything is replaceable by AI - and by the way I don't want anything to be replaced by it, but I believe it will happen - just that some people don't realize yet how "trivial" their job is to modern models.
TeMPOraL 3 hours ago [-]
This. The main blocker in AI proliferation I see is that people don't even consider it can do more than they thing it can.
Just yesterday, my wife recounted to me a conversation with her friend, where the friend mentioned they felt lazy and had some piece of copywriting done by AI this time. To which my wife said, "TeMPOraL would've also had AI upload it to the website afterwards".
Yes, I would have, and I would've also had it install the tools needed to upload it too, because why not. The only difference between me and my wife's friend here is that I know I can ask AI that, and I know it will work.
jameshart 16 minutes ago [-]
People have gotten very used to breaking down tasks into separate things they know how to do on a computer. Asking an LLM to do each of those individual tasks is easy but feels both lazy and also ‘so what?’ - I knew how to do that task.
The thing people are slow in realizing is that you don’t need to break the task down, and you don’t need to know how to do the task. You don’t even need to have the software installed on your computer to do the task, or even know that software to do the task exists - a sufficiently motivated LLM can pull down the needed open source tools and use them, or even build its own.
TeMPOraL 3 minutes ago [-]
Exactly. It takes some... willingness to break through your own mindset (I don't want to call this "courage", but at times feels like it), to just tell the LLM, "I need <this and that> to be <like this>, make it so".
Two years ago, that was unthinkable. A year ago, it was a quick way to set a lot of tokens on fire. Today, well, I'm yet to find a thing Opus 5.5 doesn't do at least passable (and usually stellar) job on.
raxxorraxor 3 hours ago [-]
It is particularly suited to programming. But most business processes are basically a program that is executed. Especially in modern times, where office clerks have less and less decision-making options.
miki123211 2 hours ago [-]
I feel like there are 3 groups of people I regularly encounter: The first believes in LLMs, the second uses free Chat GPT, the third uses Gemini.
alex7o 3 hours ago [-]
I have a friend with a job like that and he is not being replaced not because it is Impossible but because it would be too expensive atm. But they replaced his whole team with workers south east Asia.
athrowaway3z 2 hours ago [-]
I agree with the disconnect between the professionals, but i have yet to see anything that makes me believe that there is a class of 5000 internal people who are that much further than the top users are.
The internal models are more eager, and there is the scale difference of being able to spin up 10.000 vs 10, but to get something valuable out requires skill.
Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Finding proofs is one of the exceptions.
Just look at the codex/claude harness. They regularly lag behind 'external' users.
usernomdeguerre 15 hours ago [-]
I feel like if the gap were so monumental that he could just point to a monument of it, rather than hand-waving at "the vertical professionals".
Alternative interpretation of the numbers; there's only a subset of productive work that LLM's are 'capable' of doing at this point. That is, work that's verifiable by a machine or doesn't require verification. Thus, the cleavage points are along that axis and-- naturally-- most human beings don't have that kind of work.
pjc50 5 hours ago [-]
> That is, work that's verifiable by a machine or doesn't require verification
That's where I'm at on the capabilities. LLMs have demolished software and mathematics because those have well-developed verification that's easier than doing the thing itself. Stuff that's more subjective they can produce good work, but with a significant error rate which makes it hard to let them loose on important client work without checking. Or rather, a number of companies are going through the cycle of trying that, having an expensive error, and reconsidering.
Or sometimes a company can simply pass the error cost onto consumers, who are gradually arming themselves with LLMs in response. We're going to see a productivity-sapping boom in disputes.
omnicognate 3 hours ago [-]
> demolished software and mathematics because those have well-developed verification that's easier than doing the thing itself
They are proving useful for the parts of those things that have that verification, but in neither case is that the whole of the field. In maths, for example, we have seen LLMs find answers to some interesting problems/conjectures, but haven't yet seen them pose any similarly interesting problems/conjectures themselves because the criteria for a "good conjecture" are unclear whereas the criteria for a resolution of an existing conjecture are usually supplied by the conjecture itself.
bvcp 5 hours ago [-]
lowering the cost and speeding up delivery will mature technology in other fields leading go the disruptive verification that ai needs
TeMPOraL 5 hours ago [-]
My take as someone one tier out of the groups seeing AI going vertical (so I only watch it go up on a high exponential): the escape of "AI can do only automatically verifiable work well" and "AI doesn't have judgement" is a mirage, that will shatter within the year. AI already has better judgement than many senior people in some tech domains, on top of better skills, and I see absolutely no reason to believe it won't be able to deliver end-to-end solutions with no human involvement, that are more correct, robust and efficient than they would be if humans were allowed to weigh in.
I also see no reason for that to not generalize into every field of human endeavor.
You don't see a monument to vertical because its construction is bottlenecked on the time it takes for companies to form around it and start executing. So it's few months out still.
M0r13n 4 hours ago [-]
This weekend I had some time with Opus 5.5 and GLM 5.3. I went over some findings in a library of Claude Mythos 5.1. These were minor, low-severity bug fixes and Mythos provided a PR to address them. Its PR was functionally correct. I still spent ~5 hours looping over them until I was really happy, even though a fully working PR already existed.
Some of the differences were:
- Mythos recommended salvaging damaged labels. I pushed back: data whose structure is broken shouldn’t be trusted
- My simpler more readable test caught a regression that Mythos’s more elaborate test missed
- Mythos allowed invalid values, citing real-world data that I could not reproduce, while I decided to follow the spec
- Mythos kept the old error types. I made every kind of malformed input raise the same error, and accepted a small, documented breaking change for anyone who relied on the old ones
Overall, I much preferred the final version of my PR to the initial draft Mythos provided. However, getting there involved quite a bit of time messing around with the code, as well as some back and forth with Opus (which was valuable).
This is a recent example that matches my overall experience working with current SOTA models (the ones mere mortals like me have access to).
TL;DR: I don’t think AI already has better judgment than (most) senior people. Obviously, that might change with future models. But that’s a hypothetical scenario. It might as well not change anytime soon.
cyclecycle 3 hours ago [-]
Resonates exactly with my experience.
I'll add they also seem to make 'bad' judgement calls around which layer to implement something, often preferring downstream fixes/hacks/workarounds when upstream fixes would be better.
They also duplicate code lots rather than finding and reusing existing.
Both of these patterns lead to degradation and time wasting in complex code bases. Part of the behavioural adaptation I've made in using these tools is heading these off as much as possible at prompt-time
My prompts are often verbose to several paragraphs
M0r13n 45 minutes ago [-]
> often preferring downstream fixes/hacks/workarounds when upstream fixes would be better.
It sometimes feels like the model is too lazy to do the actual reasonable fix and implements a _hacky_ fix downstream. And to be honest: I feel that!
jameshart 10 minutes ago [-]
This is true, but we also don’t know the long term architectural tradeoffs that are right for an LLM-maintained codebase.
For human-maintained code you manage duplication and locality of changes in the codebase because you want concepts to fit inside a human’s working memory. Keeping method bodies short is so they fit in one code window on the screen.
LLMs’ working memory works in very different ways; they see codebases not as windows of code but as blips of grep results. It might be that it’s easier for an LLM to just ensure five pieces of code are all modified in sync with one another than to safely implement a DRY refactoring.
bluGill 3 hours ago [-]
That mostly matches my experience, but there is a caveat that you're not considering.
You don't know what good judgement calls the AI made along the way. Or rather you do but not in detail because those are the part you looked at that said, okay, this is good code and I should ignore it. Your judgment is biased because you're looking at the bad parts and not the good parts.
smcg 1 hours ago [-]
do you really want AI to make life and death decisions? do you really want that power concentrated in the hands of the few who own the AI systems?
plastic-enjoyer 4 hours ago [-]
> AI already has better judgement than many senior people in some tech domains
How would you quantify this?
TeMPOraL 4 hours ago [-]
How do you quantify judgement?
My proxy metric is observing the growing ratio of "AI suggested, I haven't even thought of that" to "AI nonsense I had to correct" (where it didn't turn out later AI was actually right).
bowsamic 41 minutes ago [-]
“I hadn’t even thought of that” = tech debt
UncleMeat 3 hours ago [-]
Last week I was dealing with a bug where we were triggering some behavior in a service in all requests when actually we really shouldn't be triggering this behavior for one of our callers. It just so happens that this caller sends a unique flag to the service.
The suggestion from frontier models was to have our service check for this flag and use that to decide whether to trigger this behavior, even though the flag had nothing to do with this behavior semantically. I'd expect this from a junior engineer. Correct now, but almost surely to cause a bug in the future as the codebase evolves.
These tools are incredible and can achieve things I would never have imagined five years ago. I have yet to see their judgement exceed mine.
logicchains 3 hours ago [-]
>AI already has better judgement than many senior people in some tech domains, on top of better skills
It has better judgement in math and coding, which have infinitely generatable training data; if you ask it a domain-specific question about something like sales or marketing, its answer is often horrendous.
thefz 2 hours ago [-]
> "AI doesn't have judgement" is a mirage, that will shatter within the year. AI already has better judgement than many senior people in some tech domains
No offense, but if you noticed it has better judgment than you, consider that you might not be a "senior people in a tech domain"
mizzao 15 hours ago [-]
The swarm-based HF hack, and the observation that 40-50% of the compute at AI labs goes into training the next generation of agents, doesn't count?
We're all over here using agents to code projects that used to take weeks and feeling pretty good about it, and they're using agents to build the next generation of frontier models — an order of magnitude more complex problem...
Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
pjc50 5 hours ago [-]
> Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers
Isn't this a massive alarm bell? That the capability hasn't done much for consumers yet that isn't basically toys? Although I suppose the pitch was always B2B "you can automate away your expensive white collar workers" and it's not clear how widespread that is actually happening.
TeMPOraL 3 hours ago [-]
It did a lot for consumers who dare to use it. It didn't do much for consumers who wait for middleman businesses to offer AI-based services, but that is not because of AI, but existing business model of SaaS.
The businesses pay for tokens and try to deliver their usual enshittified experience, and nothing interesting gets created, because it cannot - enshittification extinguishes all power of AI. Using AI to full extent, in contrast, obviates most need for the business in the first place. So they're all struggling to design an AI powered product, in a world where those very products are already becoming subsumed by AI.
So normies wait for middleware companies to get their shit together or (much more likely) finally die, while people with software experience and eagerness to use AI just don't subscribe, because their favorite LLM handles that work just fine.
curt15 2 hours ago [-]
OAI is still hiring lots of humans for business and engineering roles instead of automating them with their best-in-class AI capabilities (https://openai.com/careers/search/). What does it tell you when they apparently don't "dare to use" their own models?
TeMPOraL 5 minutes ago [-]
Obviously they use them. If you told me they're buying subscriptions to random bullshit SaaS like every other company did to date, and actually using them (instead of striking deals with them or trying to directly copy them into their offering), then you'd have a counterpoint.
irthomasthomas 7 hours ago [-]
What is the incentive to try? Any such application, if profitable, would immediately be replicated. If one person can use agents to do something profitable, anyone can use agents to easily copy it. Hence you have Anthropic locking down their models and removing features customers can access, to try and protect their own slim advantage.
consp 6 hours ago [-]
And I thought that building your own castle and hoping your moat is wide enough was always a losing strategy in the end. We shall see how it ends this time.
oblio 5 hours ago [-]
Google, Apple, etc are basically stone star forteresses and your conclusion is that moats don't work?
smcg 1 hours ago [-]
well yeah, it works for those incumbent giants. but the whole point of a "moat" is that it makes it more difficult for everyone else to enter.
koe123 8 hours ago [-]
With this stuff I immediately wish I was more economically literate. Bare with me. Let’s say one of these labs can use agents to replicate work otherwise done by humans. This proves that anyone with agents can eventually do this, unfortunately I don’t think its a moat like a platform (like facebook) provides, given that everyone is leapfrogging each-other constantly. Thus, it kind of becomes a commodity / low margin business?
Depending on the type of output, the value immediately (?) goes to the electricity cost of the LLMs and the cost of materials. So: more “efficient”, but without a big margin, right? So its cheaper for the workers that remain working, other workers are displaced, and capital owners now have a super slim margin business which can be instantly replicated at a place where material cost are lower (?).
Thus, it almost seems like “value add” by humans would in this simplified view be vastly diminished, and all that remains is resource wealth. Pop the materials in on one end and let the AI manufacture the product. Great news for the third world assuming the USA doesn't invade them.
Also, whose the consumer? Or are we going back to a strange medieval time where consumers are not of concern?
I don’t know what to think
pjc50 5 hours ago [-]
> all that remains is resource wealth
Final vindication of the Georgists. All value flows back as "rent", which in the 21st century includes IP licensing.
> Great news for the third world
Not sure about this in a world where there isn't demand for human labour. Especially in the post-consumer world. Very difficult future to see. Possible that it will deliver enough capability and expertise to drastically improve things; possible that without skilled human review that fails catastrophically.
I'm reminded of the Houthis asking Claude to build them a missile guidance system. We're going to see the Toyota Hilux + .50 cal "technical" combo get some weird targeting system upgrades.
jaccola 7 hours ago [-]
Electricity is important, even more important than AI (despite some rhetoric).
We don’t see electricity companies among the biggest companies in the world, except in China where they are a state owned monopoly… maybe a good analogy after all.
oblio 4 hours ago [-]
> We don’t see electricity companies among the biggest companies in the world except in China where they are a state owned monopoly… maybe a good analogy after all.
All over the world they're either state owned or very tightly controlled, precisely because they would be among the biggest companies in the world and even worse than that, they would be essential. If Google Search goes down for 2 days, meh, you can handle it. If the national grid goes down for 2 days your aggressive neighboring country can invade.
bambax 4 hours ago [-]
> Bare with me.
It depends; can you send pics first?
dasil003 28 minutes ago [-]
Maybe raw intelligence is not the bottleneck for what can be accomplished with the current data available in a form agents can access it?
watwut 7 hours ago [-]
> The swarm-based HF hack, and the observation that 40-50% of the compute at AI labs goes into training the next generation of agents, doesn't count?
Most peoples jobs do not involve hacking other services in large scale fuzzing/whatnot attacks.
> Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
It does not seem to me that tech industry or its investors care about consumers even a little bit. Consumer preferences dont matter once you have enough or raw economic power, I guess.
LittleShaman 6 hours ago [-]
The first two points Andrej says are the cause of the funnel seem to be pretty much the "alternative explanation" you share.
jackmott42 4 hours ago [-]
I a common monument is:
"I need to wash my car. The car wash is 100 meters away. Should I walk, or drive?"
The general public has heard of this, tried it on free tier AI, and uses it as an example of how dumb AI is.
Anthropics cheapest tier paid AI, on lowest effort, gets this right.
madaxe_again 6 hours ago [-]
Idk, there are entire categories there he doesn’t touch upon, which may be indicative of his own blind spot.
Coding - of course I use LLMs at this point, and as he says in the past year it has gone from “ok nice try robot” to “build me a BI tool”, and being pleased with the outcome.
Outside of coding my major use case is organising the chaos of a complicated life. I currently have it managing (including comms and form filling) the import/export process for the international move of a pet, managing pricing on my Airbnbs (dynamite - massive revenue boost) and giving me a weekly performance report, coordinating and corralling documentation for the humans for said international move, and being an extremely dogged advocate/personal assistant in a commercial dispute, which it has won.
The use cases are honestly exploding. I’m kinda at the point where next year’s multi jurisdictional tax returns are probably going to be “yo machine, file my taxes, let me check your work before I hit submit”. The year after next, without doubt.
And just to underscore the comprehension gap… I’ve been doing this stuff for much of this year now, and two days ago my wife ran to me in a panic saying “your computer has been hacked!”, as she happened to notice one of my agents filing Airbnb documentation with the U.K. government in chrome, and had no idea agents could drive a desktop.
Natfan 3 hours ago [-]
>managing pricing on my Airbnbs
so, scamming your customers?
moezd 10 hours ago [-]
It's the obsidian story once again.
Assume we've settled near a volcano recently and you start seeing ornate jewelry being sold on the marketplace for the first time, shiny colors, straight edges. You like them, gift them to others. That's your average ChatGPT user.
Over time, you see these rocks being attached to the tip of a spear and other tooling. People report performance improvements while farming, cutting, but many also caution that these new sharp rocks can kill people. These are your IT people using AI agents at work.
And over there, on top of the mountain, in secret caves, there's a new religious order. They claim to use these stones to predict weather, heal diseases and prophesize about the future. That's your frontier lab people.
When will all this be over? Never, though in time the knowledge will eventually dissipate. When Iron Age leaders were able to hand out iron swords in bulk to their armies, people definitely learned how dangerous these are, but later it didn't stop people from using iron in building construction and railways eventually.
Let's just hope we don't need many civilization collapses to jump through these ages this time.
And one bonus question: Does the blade really incite to violence, or do you just want to destroy stuff?
rf15 3 hours ago [-]
That's not how it is. You cannot have a business with workers while your product is able to replace workers wholesale. You would not be a business in the classical sense, you certainly would not make or need any money, and you would not have employees.
instakill 7 hours ago [-]
I just want the volcano to erupt
watwut 7 hours ago [-]
I wish people stopped making up historical claims in order to create analogies with made up past.
LtWorf 3 hours ago [-]
Has any actual historian peer reviewed this fable?
dgellow 8 hours ago [-]
Regarding the bonus question: The AI cultists seem to explicitly want to destroy stuff. Including our social order
sublimefire 6 hours ago [-]
It all sounds good and he is on the right track but the elephant is not that. The problem is that ordinary businesses do not see ROI from dumping cash on this direction. If someone wanted to have similar access to models as internal folks in these companies, then you’d need a bit of cash to run such swarms of agents, not to mention people who would know how to do it. Another thing is that you’d need your own frontier models deployed in the place where the inputs would not be exposed to a third party, as it is important to feed the internal private info to reason about. And if you are like me then it is a constraint of capital and people who want to invest in you that is also a factor, why would they invest in you when openai/anthropic will be able to copy your ideas in a few months?
CharlieDigital 2 hours ago [-]
The economics part is the one I feel the biggest disconnect in the Tweet.
If teams had the same level of access as the "top 5000", you'd see different use cases proliferate. Same as we see an explosion in productivity when we advanced public education (more people can read, more people can contribute, more talent is discovered), same as we see an explosion in creativity when we advance distribution (from 3 broadcast channels to cable to streaming), same as we see gains in athletics with population size and diversity.
bvcp 5 hours ago [-]
the question is how much of this inability to realise roi is technology vs skill. i have plenty of clients who drop 10m on an enterprise transformation project but couldnt (currently) manage the power a 10m nvidia rack would give them, its only a matter of time before they can or their competitors can.
joshstrange 3 hours ago [-]
I said the other day that I feel like I’m the edge of a wave that will (probably) make its way to the “general public” in time and I don’t know what it will look like when it does. Will “normal” people be able to create what I’m creating today in XXXX time? Or will it be another case of “everyone can cook but people still go out to eat”?
The build vs buy decision has been turned on its head for me, but it’s unclear if the “normal” people will make that same decision or be capable of leveraging the tools the way I am.
theturtletalks 16 hours ago [-]
Claude Code accelerated this divide. Programmers using Claude Code realized that giving LLMs access to a terminal made them feel 100 times smarter. Access to Bash, CLIs, and the ability to visit websites without the many restrictions made them powerful.
Meanwhile, the average user was still using ChatGPT, which probably felt like it had plateaued over the last few releases. The real power of these models became apparent when paired with a terminal. Agents like Muse and OpenClaw are now attempting to bring that same Claude Code-like power to everyday users through a simple chat UI.
nathanqueme 15 hours ago [-]
A terminal also tells the model when it failed. Exit code, stderr, a stack trace. it reads that and tries again.
The apps normal people use mostly don't. I spent two years building agents that worked inside people's gmail and calendar, and the failures that hurt all came back as 200 OK.
- Slack puts {"ok": false} inside a 200.
- Google ignores query params it doesn't recognise, so i sent max_results instead of maxResults for weeks and every filter was silently dropped.
The agent says "done", the user finds out later, and they stop trusting it. So i think part of the gap is that agents look a lot dumber when nothing tells them they got it wrong.
smcg 1 hours ago [-]
Hmm, that's poor API design. If the big companies can't standardize their inputs and responses, then the small projects aren't going to either. Sounds like there are lots of small issues like this that will roadblock AI-powered automation for some time.
asdff 15 hours ago [-]
I wonder if in response to this companies like google will build a sort of agent facing layer to the app in question. So rather than you building an agent to attempt to wrangle pupeteer to click css elements as a user does, it calls some apis google exposes to certified gemini user agents and these contain exit codes and can be run as script like instructions not e.g. the shortcomings of browser agent based methods such as having to wait for all content elements to actually load and the obfuscation arms race some devs have towards exposing these elements towards headless agents.
I found out due to a live bug that Apache strips out the response body for 5xx HTTP codes for security, and it's not configurable, so you can't use any of those if you want to return the error message to the user. Don't know what else might do the same, so nowadays I only use HTTP error codes for protocol-level issues, and application-level issues all go into a 200 OK with JSON just like that.
svachalek 17 minutes ago [-]
4xx errors are for when the client did something wrong and needs to know about it. 5xx errors are for when the server is having problems and the only people who can do anything about it are the operators, who should be looking for errors in the logs rather than the HTTP client.
Truly "application level" issues are therefore correct as 200, although some application communication issues lead to 4xx errors that should be handled as such. But the case of an application trying to communicate via 5xx errors should never have been the case anyway.
chrismorgan 9 hours ago [-]
Do you mean Apache APISIX, https://github.com/apache/apisix/issues/13501? That sounds like a simple bug which existed for a short time and has been fixed. For Apache HTTP Server (what people mean by “Apache” in such a context), what you describe is thoroughly unbelievable.
Good4boothee 4 hours ago [-]
I worked in environment where every HTTP 500 error was replaced by some middle ware by user "friendly" error page, and that included REST calls between services. That HTML also included one huge header image(as base64), so my logs were flooded by that junk response.
aezart 12 hours ago [-]
We run into that a lot using Mulesoft. 3 layer APIs, and each layer will discard error messages returned by the underlying layers unless we do tedious custom error handling. Makes it impossible for the teams using our integrations to know what they're doing wrong.
alexkubit 7 hours ago [-]
[flagged]
kooi 17 hours ago [-]
The 1%ers are in the vertical, but the question is vertical to where?
It needs to a potential field with practical, economical, "real life" attraction well. I.e, robotics, real economic efficiency gains, manufacturing novelties.
The worry is that the 1% is attracted towards a non-practical money hole. I.e: Token burn for the lols, sophisticated software systems that dont provide actual value outside of giving NVIDIA cash.
freecodeio 17 hours ago [-]
he's just trying to say we're in a cool kids club and you ain't in it, without specifying what the cool is about
hodgehog11 7 hours ago [-]
I'm normally not the cynical type, and I am appreciative of Karpathy's efforts to educate people on developing language models, but this is exactly how it reads.
Every once in a while, everyone needs to touch grass.
smcg 1 hours ago [-]
tell that to the c-suite!
asdff 15 hours ago [-]
It is fine as it is to make a boatload of money. All they have to do is seed ads into it. Look at how much money social media brings in from that model of "free/useless product" as flypaper to bring in eyeballs for the real money which is ad revenue. Adsense 2.0.
femto 12 hours ago [-]
Most professional engineers, especially those in electrical or communications, have the tools to understand AI.
I recently started trying to understand how AI works. It turns out that I wrote an "AI" in the late 1990s without realising it. The "AI" was an MMSE (minimum mean square error) radio receiver for CDMA.
This receiver consisted of an FIR filter (mathematically a linear combination) with trainable coefficients (weights). The coefficients started with random values. A sequence of known training symbols (tokens) was fed in and a gradient search was used to adjust the coefficients to achieve MMSE. The unknown data to be received (classified) followed the training sequence, at which point the training was turned off and the system ran as an FIR filter. Once trained, the FIR was a matched filter.
In AI terminology the above is now called a "layer". Current AI cascades multiple layers, in which case you need to pass the output of each FIR/layer though a non-linear function. If you don't have the non-linear functions at the end of each layer the multi-layer system is mathematically equivalent to a linear combination, which could be implemented with a single layer, losing degrees of freedom.
I write this because many electrical/communications engineers probably know how AI works, but don't realise it.
My still incomplete understanding is that LLMs are "just" a graph of connected layers. More complex that the above MMSE receiver, but trained and operated in the same way. The smarts in AI are:
1) Arranging the layers and the connections between them.
2) Having access to the right training data
3) Having the computing power to perform the gradient search (aka training)
My point is that most professional engineers already have the knowledge to understand AI/LLMs, and potentially already do know how they work but don't realise it due to a difference in terminology.
Matumio 6 hours ago [-]
This is like saying with chemistry, amino acids and human DNA you now have the basis to understand human culture.
The problem is one of complexity, the whole being greater than the sum of its parts. Electrical engineering creates a thinking that is opposite to what you need for understanding biology or human culture.
This is relevant because LLMs have basically swallowed all written human culture. Engineering is about creating parts that interact in a clean, well-defined way, like a MOSFET transistor or a REST API. The goal is that that when you combine the parts, and they all operate within specs, you can still predict what the whole system will do.
So saying LLMs are just a step away from the Pereceptron (or an equivalent filter) is mathematically true, but almost useless. It's like saying that a modern 3D game is basically just transistors, logic, CPUs and memory, nevermind the pattern in that memory or how it came to be. You really can't ignore all the cultural inputs and feedback loops if you want any kind of useful understanding what this thing is.
maciejzj 7 hours ago [-]
I would agree that a decent EE major is equipped with knowledge sufficient to grasp both Transformer architecture and gradient descent training. However, these elements do not fully cover the key parts of modern AI systems and also are not enough to predict the AI capabilities. You have to account for reinforcement learning from human feedback (RLHF) and with verifiable rewards (RLVR) which are crucial elements of the wider model creation/training process. These do not have obvious analogues in other engineering disciplines, but are still within your grasp if you are decent CS/EE/adjacent major. The issue is that you still can't deduce end result AI capabilities from first principles. You may have prefect understand of y = softmax(QK^T/sqrt(d_k))*V and it will give you no clue of the effectiveness of a given AI model. The range and complexity of training RLVR tasks may hint at the model capabilities, but the final results are very much emergent.
tatjam 8 hours ago [-]
Dropping in to mention belief propagation networks, those things are likely being used right now by the modem on whatever device you use (WiFi or Ethernet, both use it!), and are surprisingly AI-like.
femto 6 hours ago [-]
As used in LDPC decoders?
My gut feeling is that the AI field is eventually going to discover how much overlap it has with information and communications theory.
Question to any AI researchers out there: Can a layer in a trained AI be treated as a form of "matched filter"? Is that a concept that is used in the understanding AI: to view it as a cascade of matched filters? The next question is "what is it matching?"
sieve 10 hours ago [-]
There is a lot of terminological confusion in the field. The fundamental difference is about how the "AI" learns. And who, if at all, supervises the learning.
Some systems are fully formed AI. Often based on heuristics. Humans have encoded the rules into the system. Others do limited learning based on pattern recognition. Still others learn based on tagged data. And LLMs don't need even that. In a stream of N tokens, the trainer has access to K+1 when it is training a model that knows 1...K and can check if the prediction was correct.
AI literally refers to any artificial thing that feels intelligent to us: does some thing a simple bash script will not. Your rules based chess engine is AI. The bot you compete against in single player Age of Empires 2 is AI. DeepBlue was AI.
ML is where things become interesting because we teach machines to learn.
NN is where we borrow terminology from neuroscience and create "neurons" that learn.
DL is where we stack neurons deeply in layers.
I am trying to make sense of all this myself.[1] So I am building a curriculum for it as I learn. So much of the teaching material in the wild revolves around torch. Makes it useless if you want to peek inside the actual machinery.
I feel like this statement is too anthropomorphizing. We don't teach a computer anything. We write normal computer programs that can process data(training), turning it into what is essentially a configuration file(weights/parameters) for another normal computer program(inference engine). Upon receiving a request, the inference engine is invoked and passed the input, then it processes that input and its configuration(weights), producing an output. "Thinking" is just feeding the output back in as an input, allowing the model to revise it and refine it before it's returned to the user.
There is no teaching, nor learning. There is just data processing. There is nothing happening between requests to the inference engine. It's a relatively simple system altogether, but when the training data set is large enough it produces unbelievably good outputs.
With this in mind I really don't understand how people think these programs can be conscious. They're not doing anything fundamentally different than any other computer program. It's just crunching numbers. You might as well argue that a computer game or 7zip or a chess bot or whatever is conscious.
I know you specifically didn't claim they're conscious, I just thought it was worth mentioning because I think it's ridiculous how many people will seriously argue this as if there is any difference between "ai" and other programs. There isn't. It's just a set of instructions for doing computation, as computers have always done.
3D rendering is actually quite similar. There is a bunch of files (input/configuration) describing 3D meshes, textures, lighting etc which are fed into a rendering engine which processes all that data and turns it into a 2D image. Similarly to AI, once the input files are sophisticated enough, this can produce mindblowingly realistic and beautiful images, and you can run it in a loop to make video or make it interactive, moving the camera around in real time etc. But at the core of it all is just a bunch of files containing a bunch of numbers, and a program that reads those files and does computation with them.
sieve 3 hours ago [-]
We humans have the habit of falling into the trap of anthropocentric views and explanations of the universe and everything within it. The history of AI is the history of moving goalposts. I don't particularly care if they are conscious or not. Humans have a history of treating even actual, conscious, intelligent beings as chattel or food. So I don't really see why our attitudes would change if we somehow realized that LLMs are conscious.
I have been thinking about this for a while in the context of the quest for AGI. If all we want is something to take over some parts of our intellectual labor, then LLMs are already there. They can communicate with you in a language you understand and do stuff that you command them to. Often, on their own.
AGI will have its own telos. An LLM does not (at least I refuse to believe that it does).
So, yeah, you could think of LLMs as programs processing data. But there is definitely some emergent behavior you notice that emerges when the LLM runs. How you interpret all this depends on your view on consciousness and intelligence.
sfn42 3 hours ago [-]
I also don't really care whether they're conscious, but I do think it's worth challenging the claim that they are.
LLMs are programs processing data. Lots of programs exhibit emergent behavior. A simple example is Conway's game of life. It can be programmed in less than an hour, exhibits lots of interesting behavior. In fact you could run LLMs inside of it if you wanted to.
I'm not denying that LLMs are unbelievably cool and useful. It still blows my mind when I send a half-assed complex screenshot and a vague description and it just gets it and fixes the issue in seconds. It's like magic. But it isn't magic, it's just some relatively simple algorithms processing a crapload of data, and I think it's useful to understand and remember that.
tsimionescu 5 hours ago [-]
The current leading materialistic model of consciousness is essentially that human consciousness is a computation in itself, and that the brain is basically a very very complex computer. So it's not at all illogical for someone who holds this view and who fully understands that AI is just a regular computer program to also believe that it could be conscious.
Of course, I'm not saying that you have to believe any of this - even defining what consciousness is is famously elusive. And there are a lot of people who think consciousness is not materialistic to begin with, or that we have not yet discovered the scientific nature of it.
smcg 1 hours ago [-]
I hope we never do prove that, because as soon as that happens it will be used for eugenics.
sfn42 5 hours ago [-]
I definitely agree that consciousness is mysterious. But I think the assumption that AI is conscious begs the question "then why isn't 7zip conscious?".
Is a simple calculator conscious? How about a mechanical calculator? Is an abacus conscious? Are they only conscious when executing a sufficiently sophisticated program? What if I hypothetically had infinite time and patience, and performed the inference on paper with a pencil. Would it then be conscious?
tsimionescu 37 minutes ago [-]
Well, the exact same question applies to biology. If humans are conscious, is all life conscious? If it's just the brain that's conscious, is every part of the brain conscious? Are individual neurons conscious? How about other cells in your body, or tissues or organs? Our stomach has more cells and more neurons than many full animals - is our stomach conscious?
The answer in the "consciousness is a computation" model is that consciousness can arise in any computational medium. But the model doesn't say that any computation is conscious - it says that consciousness is any of a set of specific computations, any computation with properties X, Y, Z whatever. So it's perfectly possible (according to this model) that a GPU running game graphics is not conscious, while a GPU running some future even more advamced LLM is conscious. Or a network of GPUs + storage + whatever.
discreteevent 3 hours ago [-]
Good examples. We don't know what human intelligence, understanding or consciousness is but we do know what all of those examples are. And we do know what an LLM is. Like you said, it's a data processing machine and we know what algorithms it uses. It's more productive to discuss it in those engineering terms because we've done plenty of data processing, statistics, control systems, curve-fitting etc. before. There's no need to make this jump to consciousness/magic except for marketing or fantasy purposes.
logotype 3 hours ago [-]
> "There is nothing happening between requests to the inference engine."
This is true for vLLM/SGLang/etc but LayerScale is making use of idle GPUs. A new novel inference engine for agentic workloads. Our advanced scheduler enables double the workload on the same number of GPUs (e.g. prioritization of on-demand queries and background processing of batch workloads simultaneously). Let me know if you want to try it out!
TeMPOraL 4 hours ago [-]
> I feel like this statement is too anthropomorphizing.
That's number one mistake preventing people from getting a good grasp of how LLMs work and how to use them effectively.
That is, not anthropomorphizing is the mistake. It gives people bad mental models.
LLMs are not people in general, but behaviorally and in terms of system design, they may as well be. They have same failure modes and cognitive characteristics humans have, and are very much unlike regular, non-ML software tools.
sfn42 4 hours ago [-]
Not really sure what you mean, I have the view I presented and I'm able to use LLMs effectively. The understanding that they're not human allows me to treat them as such - I don't say please or sorry or thanks or other pleasantries, I just instruct them in the simplest most efficient way I can. Humans would find me rude, be annoyed if I give them lots of work for little benefit, and say the same things over and over etc.
caughtinthought 11 hours ago [-]
Even if the model is simple, it is happening at a scale that is producing alien results. You understand the basic principles, but that doesn't matter: you, well rather no one, understands the final product.
israrkhan 9 hours ago [-]
This is interesting way to look at FIR filter and tuning parameters. never thought of it like this.
calf 11 hours ago [-]
The difference with your FIR matrix and a deep net matrix is the latter generalized to trillions of parameters i.e. a trillion dimensional space to approximate general, computable functions. The machine learning ideas are fundamentally the same at a high level, just the architecture (nonlinearity whereas in engineering we prefer analyzing linear systems) specific details and tricks and such.
In fact at least at Berkeley a current professor teaching deep learning is a signals expert (Prof Sahai), his lectures are on YouTube.
jacknews 6 hours ago [-]
This is like saying you understand the basic principles of internal combustion engines.
Sure, that gets you some level of understanding, but in no way prepares you to be even a mechanic, far less an engineer, let alone a researcher/engine innovator.
rakejake 3 hours ago [-]
Tbf OP said they have the tools to understand what was going on, not that they could build a model to compete with OpenAI tomorrow. EE (especifically DSP and communications) uses a lot of the same building blocks as ML so yes it does prepare you.
You are right in that the emergent abilities come from humongous amounts of training data and compute which only a handful of companies (and the people working in them) deal with today. I'm sure there's a lot of secret gotchas which only arise at a certain scale so one cannot self-study for them. Companies also guard these wherever possible because of economic incentives. But that applies to most other engineering disciplines. You can take a course in civil engineering and have enough knowledge to reason about why a skyscraper will not fall apart in the wind; doesn't mean you have the ability to build one.
horns4lyfe 2 hours ago [-]
But it’s certainly enough to know that it’s not “conscious”
CamperBob2 12 hours ago [-]
Sounds interesting. How did you back-propagate the coefficient gradients? Just tweak them one at a time, or was there a global algorithm that worked on the whole filter at once?
femto 11 hours ago [-]
From memory the entire filter was updated in one symbol period (ie. global)
The symbol rate was 1Msymbol/s and there were 66 taps. The circuit (in an FPGA) ran at a clock rate of 66MHz. In each symbol period the error was calculated based on the output symbol, then an iteration was done over all 66 coefficients, using the error to update them.
Edit: I also remember that to make most efficient use of the FPGA I figured out the minimum precision required in the maths whilst preserving convergence. It was a 4-bit mantissa plus an exponent (3-bit or 4-bit exponent, I can't remember). It's interesting that today's AIs seem to have also figured out that things still work if 4-bit floating point is used, though that is 4-bits including the exponent?
swiftcoder 6 hours ago [-]
> things still work if 4-bit floating point is used, though that is 4-bits including the exponent?
It is generally 4 bits amortised across an entire block. This works more like block-based mage compression schemes - you put a bunch of statistically-similar floats into the same block, and use your handful of bits to index into a codebook containing the deltas from a 16/32 bit reference value for the whole block
raynr 9 hours ago [-]
1. The funnel is even steeper than the linked post suggests.
There's a huge range of users between "casual and infrequent users" and "professional use of frontier-grade LLMs in math & code".
I think I'm in between there somewhere. I'm far from a software dev, my experience is playing with delphi and C when I was a kid. But I've been using LLMs to vibe code personal and work projects that I could never have done even a few years ago. They're genuinely helping work, because there's always some place where a simple little script or other utility can help. (Hell, isn't this partly why so many people start up startups?)
I imagine that many departments in many companies that are not "tech" companies face similar challenges - there's no budget to hire a software dev or even engage a third party to do things, but now because everyone's aboard the AI train, they have access to something (chatgpt, gemini, copilot, claude), and that's helped users implement their own solutions. Somewhere in the world, someone's excel-file-based accounting system has gotten an upgrade with help from an LLM.
2. The funnel is also driven by the the fact that users need products, and an LLM itself isn't a product, it's a capability.
Software devs are at the front lines because they can build their own products that can integrate LLM's capabilities. You already know how to automate testing, orchestrate, work together in a team. You're writing / configuring harnesses and flows that work for you. You're sharing / hoarding skills files.
The rest of us, like me, don't have that skill. Copilot the product is a mostly useless chatbot that takes up space in MS word. I guess its excel cleanup capability is semi useful if your company runs on excel and you've never used a filter before.
There is no work I can put an LLM to other than a "writing aid" or "better search", because the only AI product we have is the chatbot and maybe someone's equivalent of an agent studio, none of which we are trained to use.
Until the products come, there'll be a funnel.
rico735 16 hours ago [-]
Don't think the 0.2% really are in agreement that "large, complex projects that used to take them weeks/months can now be completed by agents with a prompt".
The divide seems to be between people who think you may need to understand the codebase for maintenance, potential future token costs, optimisation, code erosion & being wary of the assumptions/mistakes that they have seen LLMs make, and people who don't review code and are confident that the AI has made no mistakes and will be available for next-to-nothing forever.
It's certainly allowing the latter to move fast... but will their clients be happy if/when they break things. We'll find out soon, I'm sure.
SamPatt 16 hours ago [-]
It's not that we're assuming the AI has made no mistakes. It's that we're confident in the general capability of the models such that we're willing to accept it'll make some mistakes, but working together with it, we'll still achieve our end goals.
I presume you also know your own code has mistakes, too? You wouldn't refuse to even look at the code of a colleague who has ever made a coding mistake, would you?
hansvm 13 hours ago [-]
My colleagues write 10x less code for the same tasks (when they don't (ab)use AI) and don't reach past abstraction boundaries so that it's possible to actually reason about and review their work. When the problems get hard, they apply more care and attention to detail. Even if LLMs had the same bug rate per SLOC (IME it's higher), and even if those bugs were the same severity (IME they're worse), you'd still expect 10x bugs just because they can't be arsed to write a reasonable amount of code.
Almost all my code nowadays is AI-written, with me just contributing the concurrent data structures, a good spec, and some review and touch-up. I'm not _that_ much of a Luddite about this stuff. I'm not blind either though. Your "working together" comment does a lot of heavy lifting when arguing against a post saying that leaving the LLM to its own devices is a bad idea.
K0balt 11 hours ago [-]
If you are reaching past abstraction boundaries and making spaghetti code you are holding it wrong.
We create higher quality, more testable code, with less complexity and less LOC per functional unit than we ever did with humans. By a long stretch.
Documentation driven development with constant naive review, delta review, documentation/code coherence review agents, test coverage review agents, development standards and architecture review, etc. We write code once, then review and revise each module from a variety of different paradigms until every line, every construct, every function, every interface is optimally constructed and documented. Agents learn to write code that passes with less flags by having it pushed back to them until it passes, and revising their learnings files.
By the time code makes it to a commit it’s been brought into compliance from a variety of perspectives and tested as completely as possible, along with mutants that test the tests. By the time it gets to a merge gate, most code has been through at least 50 reviews. Then we periodically sweep the entire codebase for cruft, vulnerabilities, corner cases, etc. If we find smells or clever sections, they are eradicated.
In total, the hot paths and critical infrastructure sees hundreds of reviews against thousands of rules and best practices.
Sure, it’s expensive, but still 1/4 the cost of regular development, and much, much fewer bugs.
Unfortunately, most people are out there just vibin, even at the big corps, and it’s a dumpster fire.
It’s easy to produce garbage software, but it’s not mandatory.
rixed 9 hours ago [-]
I believe both of you are telling the truth. Certainly, you are not thinking about the same kind of tasks. For the gap between hairy realtime concurrent system programming, where a bad design decision can be fatal, and the simplest over engineered CRUD where nothing really matters can accommodate and explain the gap in shared understanding of LLM abilities.
msdz 8 hours ago [-]
> hairy realtime concurrent system programming
On the other hand, I’ve heard a direct quote from the longtime (25+ YoE) colleague of someone I know, that in their mostly embedded/some realtime work, the LLM code quality (specifically, this was about Opus 5.5 IIRC) has now surpassed his own. And that person is an excellent programmer, and the model is faster and cheaper!
So it’s definitely not just standard CRUD that is on the way to – or already – “solved”. As with all things, the truth is probably somewhere in between.
rixed 7 hours ago [-]
In my experience, it's more about "local" vs "large-scale" thinking. An LLM will be better at finding a bug that's "visible on screen" (or on the context window), whereas some design decisions that are hard to even put in words can be super important in practice (like, in C, how you manage the lifetimes of various objects, how you organise your threads and locks...) That's not purely "word manipulation", and there are not many good explanations of that on the internet, so it's understandably harder to learn for an LLM.
A bit like chess vs go.
They will get there quickly I'm sure. After all, AI beats us at go, too.
K0balt 4 hours ago [-]
Our work is in embedded systems integrated with SCADA controls in a critical environment where data integrity relies on precise time synchronisation across sensor networks and databases. It’s an environment where we count processor cycles and joules , with very little room for error.
The difference is testing. Testing at every level. If crap code can’t move down the pipeline, you don’t get crap code. If a company is used to YOLOing code out the door with human developers and they try the same thing with LLMs they are going to have a bad day.
solatic 2 hours ago [-]
100% this, but also - linting, fuzzing, and test harnesses. Agents can help build out huge deterministic simulators and fuzzers that were simply too expensive / not feasible a year ago to build for most software - and then happily bind themselves to the results moving forward.
Agents just won't build it out by default - you have to ask for it.
lolakutty 8 hours ago [-]
>It’s easy to produce garbage software, but it’s not mandatory.
It is not mandatory, just like doom scrolling is not mandatory. But given human nature, both are inevitable...
ozozozd 11 hours ago [-]
How is that 1/4th the cost of the regular development? Did your pre-AI code ever hit 50 reviews? If not, it’s impossible that it was slower. If you are talking about agent reviews, it’s not expensive.
Which is it?
solatic 10 hours ago [-]
I'm not the guy you're responding to, but I have the same experience. If anything, 1/4 cost underestimates the savings.
The analogy is like pointing out that you can have a robot vacuum clean your floors every day. Before the robot, you'd do it maybe once a week, just no time - "too expensive" to even think about doing it daily. The robot makes it happen all the time for a tiny, tiny fraction of the cost.
icehawk 8 hours ago [-]
That analogy might be good, but doesn't resemble reality: the robot vacuum I had never made it anywhere near as clean as when I just hired people to come in once every two weeks to vacuum and clean floors.
ptero 3 hours ago [-]
I think you are both right. But I think the parent argues that the robot still keeps it good enough. And IME significantly better than a week+ into a 2-week period between the maid services. My 2c.
munksbeer 6 hours ago [-]
Don't use analogies to argue on the internet. Analogies can never be perfect, and this just gives people the chance to focus the argument on those imperfections in the analogy, rather than stick to the main point.
b112 8 hours ago [-]
Meanwhile, the vacuum requires maintenance, repairing more frequently, as it runs more often. Power on time of motors is 7x as much, eg daily instead of weekly. And the robot takes more care, so each clean is 2x or 3x run time of a human. So 21x usage rates, and run time is parts wear and tear time.
Edge cases handled automatically by a human doing the task (getting stuck), means for the robot case that manual interaction is still a thing. Occasional, yet not making autonomous behavior, human free time.
Emptying the vacuum bag is a thing too, some things are not abstracted away.
Every time you add an entirely to a task, the task time goes up. Coordination is a time cost.
The savings you suggest are often not so stark. Not even for our friendly vacuums.
solatic 8 hours ago [-]
> increased maintenance
For sure. Still much cheaper than vacuuming every day.
> getting stuck
I appreciate that this is a function of the specific environment and robot you buy, but as a personal anecdote, this doesn't happen to me in practice. There was a setup cost, but once that was done, it doesn't get stuck.
> Emptying the vacuum bag
Not that frequent at all, considering that daily vacuuming keeps things remarkably clean and thus little is added to the main bag in the docking station each day. This is also far less work and far more preferable than doing the vacuuming myself.
catlikesshrimp 7 hours ago [-]
I can't imagine living in a place that has too little dirt that you don't have to empty a bag but so much that you have to clean it everyday. I do know places where maids clean the floor daily because why not, but that is different.
Vegenoid 55 minutes ago [-]
Sounds great! Let’s see the code.
hansvm 10 hours ago [-]
AI's default state of affairs is to reach past abstraction boundaries and apply whatever cheats it needs to pass its tests quickly. I think your argument is that sufficiently smart harnesses can mitigate that defect. I disagree, but not as much as you might think.
You can (and should IMO) apply additional guardrails to generate better code. If I'm interrupted every 10min to review garbage then the AI literally doesn't make me more productive, and I'd be better off coding normally and using it as a glorified stackoverflow/searchengine. This isn't how I see other people using the tools (largely because of costs AFAICT) -- which is absolutely part of my complaint; people are holding it wrong, and I'm left to pick up the mess -- but I personally refuse to deign to answer an AI's question till my normal list of problems is addressed. Nevertheless, I still disagree with your conclusion, that automatically addressing a small enough list of issues to fit into an agent's context results in good code.
I'm not sure if you're interested in a challenge, but one hand-coded thing we did well at $WORK was writing the fastest protobuf parser for this schema (and, generally, any other messy, deeply nested schemas) [0] (assume many adcandidates, deals, and segments if you're generating synthetic data to test against). My ranked priorities are (a) I need to be able to easily prove it correct/safe for all inputs (literally all, not just the few hundred billion we received yesterday, since this is open to the public internet), (b) constrained by (a) I need it to have as much throughput as I can possibly manage for many of my workloads, and (c) a second implementation optimizing latency instead of throughput is used for the rest of our software. AFAICT, AI is currently incapable of even understanding what the bottlenecks and priorities are. When I manually describe those, its solutions are a dumpster fire. If I describe exactly what I want coded in excruciating detail then it does reasonably well (not by default, but with enough of the automated tests you've described it doesn't take too many iterations with a person for it to be shippable), but at that point I've already done all the hard work, and if it took 2 weeks of understanding and design and then 1 day of AI coding vs 2 days of human coding then I haven't really bought much.
A full solution seems like a tall order for some rando's internet challenge, but are you interested in seeing if your AI can even identify the relevant problems and propose plausible solutions to them? I'll make sure to revisit this comment thread and reply with critiques and follow-ups if you're interested.
> You wouldn't refuse to even look at the code of a colleague who has ever made a coding mistake, would you?
Not sure that is the position you're arguing with. I would refuse to skip code review for any colleague, no matter how trustworthy they were. This is the behaviour we are expected to accept to get productivity gains, when the colleague is an LLM.
rbehrends 15 hours ago [-]
This is a different argument from Karpathy's prompt-and-forget one.
I personally have had good success with using agents as virtual pair programmers following some simple rules for how to make that process productive, but that is a very, very different approach. One that recognizes that both humans and agents are prone to errors and attempts to mitigate the problem rather than expecting it will magically work out through the power of AI.
sph87 10 hours ago [-]
I think of it in generational terms. Opus 3 required me to prompt and edit, iterating over functions almost one at a time. Opus 4 started to show agentic capacity but it largely bloated code and still needed frequent course corrections. If it churned for more than 15-20 minutes you knew something was off.
Opus 5 takes that and changes the game. A simple prompt goes through research, design, verification, review and self-corrects constantly. Agentic workflows can run for days without stopping, without losing the thread or devolving into slop.
I can’t even imagine what Opus 6 is going to bring.
The biggest thing holding AI back is people. People can only change so fast and yet AI doesn’t care. It just keeps accelerating.
rbehrends 43 minutes ago [-]
I am sorry, but current benchmarks do not bear out that claim. Yes, frontier models can fairly reliably one-shot standard CRUD apps (and you don't even need frontier models for that), but when we're getting to harder problems, reliability nosedives when they're operating autonomously.
nonethewiser 13 hours ago [-]
I am shocked how many bugs AI finds in the gray bearded wizards code at work. Just tons of stuff I didnt catch and I assumed wasnt there.
desolate_muffin 11 hours ago [-]
I’ve had this experience as well where it is finding all kinds of subtle bugs. The trouble is when it lulls you into checking its work a little less carefully and you let some of its bugs slip through as well. Definitely had a few of those happen to me at work.
TeriyakiBomb 7 hours ago [-]
The problem is also that enticement thing it does. Over representing the importance of some bugs or introducing more complexity to squash something that doesn’t matter.
If you aren’t 100% on it the entire time it’ll make 4 bugs for each solution and then write tests that hand hold the nonsense through. The juice isn’t always worth the squeeze and it messes with your ability to parse it all.
rob74 7 hours ago [-]
...and at some point you wish you hadn't looked at the code at all, then you could have the blissful ignorance and optimism of a vibe coder.
throwawayffffas 7 hours ago [-]
Being a gray beard myself. I am hardly surprised, we used to find many of these through users in production.
chokma 7 hours ago [-]
Yes, working on a customer's 20 years old software architecture in logistics it's fascinating in a horrifying way how many of those "it only affects 1-100 supply chains" bugs are in production.
It's less gray bearded wizards than the fact that they used cheaper crazy cackling goblins to write the software.
The challenge is to determine if a finding is an important bug or just a dead business process now never called in production. That is knowledge the AI does not have, and it requires careful, exhausting work of going through the problem.
QuercusMax 12 hours ago [-]
The amount of crazy stupid bugs in the 8-year-old codebase I work on is shocking. Every time I work on a new subsystem I end up fixing 20 incidental bugs; lots of N+1 query problems, completely wrong type annotations, all kinds of things.
worthless-trash 9 hours ago [-]
I was under the impression that the point of strict types was that the code wouldnt compile/run with wrong type information.
imron 8 hours ago [-]
This _is_ how strict types work. Problems creep in when people introduce lax types into the strict type system.
dict[str, Any] is a type, and sometimes it is the right type to specify, but most of the time it's used by someone who wants to shut up the type checker rather than think about the right type.
duggan 8 hours ago [-]
It is very hard to work with developers who see the type system as a nuisance to overcome.
And in software problems like that tend to compound. It’s hard to imagine how much badly written, poorly performing software there is out there.
The opportunity to do better with AI is monumental; comes with its own problems no doubt, but I’m willing to give it a shot.
rixed 9 hours ago [-]
> type annotations
Stopped reading right there
troupo 10 hours ago [-]
Are the bugs there though, or the LLM says there are?
LLMs are no trained to say "no". They are likely to find a ton of bugs in a simple "Hello, world" than generate "no, there are no bugs".
On top of that: if there are possible bugs in an isolated piece in code they find, are those bugs actually there?
InsideOutSanta 9 hours ago [-]
They're there, but in my experience, lots of them are for edge cases that almost certainly wouldn't have been hit in prod. I think this is a major reason why LLM-generated code can be much longer for the same problem than human code. Humans won't think of every edge case, and occasionally that bites them in the butt, but they also won't cover all the edge cases that happen rarely enough that they will not matter.
I'm unsure what the right solution is. I currently tend to let LLMs resolve these bugs because it does fix genuine issues that would have affected real users. But this also means that LLM-generated code is often twice as long as what I would have written, and much of it is for defensive behaviors that will never matter in prod.
vinc 8 hours ago [-]
I noticed the same thing at work, the code is very defensive, and often the case could not happen in prod, so my solution at the moment is to make it prove that an issue can happen and try to measure how frequently it would happen with real data. That's often enough to realize that a particular defensive measure is not needed and the code complexity decrease as a result. And it's something we can automate.
I think it's an important point because the more complex the code become the harder it will be to maintain it, so hopefully this process will improve.
troupo 7 hours ago [-]
The priblem is, actual bugs that may exist end up being covered in an avalanch of code to cover possible and often purely hypothetical edge cases.
Was the actual bug fixed? Hard to tell among the 150000 new unit tests and a million lines of new code.
---
Anecodtally: ran an "adevrsarial review" on a Stripe integration for a side project. Dozens and dozens of issues, thousands of new lines of code added as a result of the "review".
Looking at issues and code closely: none of the bugs or issues were bugs or issues. None of the flows were relevant to the project at hand, and would've never hit the possible bug paths. And on top of that a lot of functionality was duplicated or triplicated in the "fix". Had to revert a lot of commits.
rico735 15 hours ago [-]
That's not really my point... my point is that it's an unsettled debate between speed and caution, and devs are not unanimous on it, unlike the claim of the original post that seemed to think everyone was reporting the same gains.
The divide with developers at the moment seems to be about people who believe in review & are being cautious (who are not seeing major time gains according to all reports I have seen), and those who trust it to be able to fix things forever and will no longer have to worry about understanding the code, so are happy to atrophy those skills.
Yes, my code will have mistakes, and I'll use LLMs for some tasks. I also review and have a mental model, which means I can easily fix them, no matter what happens in future with AI. Reviewing also means I'm seeing the limitations and common mistakes of LLMs.
A no-review approach will be far faster than this, obviously, but if scbench.ai is still true on the frontier models, it may come back to bite people when they have no idea how it works and the context becomes too big to maintain.
zmmmmm 11 hours ago [-]
I think a big part of that divide is between the people who are inherently limited by other factors on the rate of change they can implement.
If I'm being snarky - the people who are happy to run without reviewing the code probably mostly are able to because there isn't significant value attached to it downstream. Not many stakeholders, not high value transactions, not complex downstream integration or impact. And yes, in their friction free lives, AI can massively accelerate things far beyond what it can do for the rest of us. But don't infer that applies everywhere else where those assumptions don't hold.
intended 9 hours ago [-]
Would disagree on the terminology, but not the spirit.
I’m sure people are building things that have specific KPIs based on client preferences. As long as those preferences are being met, then their question becomes “why does it matter what the code quality is?”
Perhaps it’s simply liability and penalties? If getting something wrong means your client is going to sue you into oblivion, then perhaps that is where people are focused on making sure its right, as opposed to making sure its fast?
SpicyLemonZest 6 hours ago [-]
Poor code quality degrades your ability to make future changes without breaking things. Maintenance is the common and I think correct metaphor: the precise location of all the bolts in a machine won’t be part of even the most detailed requirements spec, but if there’s any components you might want to replace in the future you’d better make sure you have non-destructive access to them.
rico735 15 hours ago [-]
Asking AI to review my code is actually pretty useful... but makes the process slower.
habinero 15 hours ago [-]
Yup. Also code review is not just there to check your work. It's forced communication, in that at least someone else knows about the changes you're making.
It's also a critical part of your security model. "Nobody can deploy code by themselves" is shockingly effective at preventing rogue actions.
Izkata 13 hours ago [-]
Speaking of communication... Almost no one mentions talking to non-devs. That's one of the reasons I can't just rely on any of these tools, I have to remember and understand everything well enough I know how they work and can explain things and help plan ahead when talking to product people in meetings.
ricericerice 11 hours ago [-]
just tell them to point their agents at it /s
cindyllm 12 hours ago [-]
[dead]
zamalek 7 hours ago [-]
Last week, on my pet project that I am using a harness to almost entirely code (so I'm by no means a Luddite), it/Astra decided to dump ALL application state into a single row+column in SQL as JSON. That is a mistake that I would, and a completely new grad, would _never_ make. It is profoundly incompetent.
This week it did it a second time on an unrelated project.
"Some mistakes" is putting it unbelievably lightly. It is shitting out complete and utter garbage, and if you think it's doing a reasonable job (or a job that can be fixed later by a more powerful model), you're either not paying much attention, or your own ability has already heavily atrophied.
Karpathy put it well: "gap of understanding of LLM capability," but he's seemingly in one of the less fortunate groups himself.
adamddev1 8 hours ago [-]
Imagine if other engineering disciplines just started accepting that everything they built or worked with would be only 98% correct.
Over time, the errors don't "work themselves out together," they compound. Everything would stall in a mess of unreliability and chaos.
SpicyLemonZest 6 hours ago [-]
Civil engineers are routinely called to work with materials like dirt and concrete whose properties cannot be guaranteed to much beyond 98% precision.
speedstyle 5 hours ago [-]
2% tolerance is not the same as 2% failure rate
ClayMedell 10 hours ago [-]
I don't quite get what's the argument you're making here. What's in contention is whether or not the 0.2% is in agreement.
Are WE the 0.2%?
Because if not, what's the point talking about the merits of AI's competence in of itself? Accomplishes nothing, we're he outsiders. And if we are, then either you're fishing for disagreement and the post you replied to is actually correct, or you get unanimous support to your statements, and establish... Nothing, the in-group could be delusional in a way that does not actually function as an argument against the whole "So where's the software? We're the AI led market disruption instead of the AI-led shovel makers?". The punchline was about the clients being unhappy.
habinero 15 hours ago [-]
This attitude is exactly why AI bros don't make successful software lol. "Where's the software?" This. This is why there is no software.
Modern software development practice is built around that humans do make mistakes. Frequently. You don't cross your fingers and hope for the best, you assume it will happen and you put processes in place to prevent errors, catch them early, and make them easy to recover from.
It is a for-real science.
pinkgolem 11 hours ago [-]
There is no AI bro software? Claude code is clearly a vibe coded mess, and I would argue the most sucesfull dev software of the last 2 years.
Claude design eliminated figma & power point use for us.
We have a product out there as a test balloon, which only gets architecture & automated reviews.
Dev speed has been incredible, customers love it.
The mess that hey is, is also out there and not yet bankrupt with the biggest AI bro CEO out there
And I know at least 1 startup, which only hired a dev after having 20+ B2B customers. They sure do not go around advertising themself as vibe coded.
aorloff 10 hours ago [-]
> And I know at least 1 startup, which only hired a dev after having 20+ B2B customers. They sure do not go around advertising themself as vibe coded.
Let me tell you something, I don't know jack about this new world.
But I do know about the old world. And in that old world, when you had 20 customers that actually needed your product for their business, your team of 5 people who built YOUR product are eagerly trying to talk to each one of them to figure out why your product works for them, how they found you, what they need now and in the future from your product. They reached out, took people to lunch, they went to meetups and hackathons and boring ass sales pitches to make connections.
Tell me about how all THAT stuff happens when your core dev team is agents in the cloud ?
pinkgolem 10 hours ago [-]
I mean, the team building that product is talking to customers, it's just 3 people, not five, and no one with a programming background
rico735 5 hours ago [-]
Have the devs been financially rewarded for their speed or just given more work?
Have the customers been charged less?
If "you don't have to review" really is true then the economics are an even bigger nightmare. The value of code has decreased, the "SaaS is dead" claim becomes real, and any good idea can be easily cloned.
Marha01 10 hours ago [-]
> This attitude is exactly why AI bros don't make successful software lol.
There is the whole Artcraft thing.. We are now finally seeing the beginnings of successull fully vibe-coded software. This trend will only increase.
BoggleOhYeah 10 hours ago [-]
Have you actually used ArtCraft?
RugnirViking 6 hours ago [-]
I've used it for a few things, mostly making form fillable pdfs with designcraft. Seems pretty cool. Have you?
pinkgolem 9 hours ago [-]
I found it impressive for 4 day old product, surprisingly feature complete for my usecase
cindyllm 12 hours ago [-]
[dead]
Eezee 16 hours ago [-]
I can do a project that took me weeks/months much faster now as long as I don't have to involve other people in the decisions.
rgmerk 12 hours ago [-]
This. So much this.
Are AI models incredibly powerful tools? Yes.
Do they have the ability to telepathically sense what the various stakeholders in the development process actually wanted? No.
arach 10 hours ago [-]
I worked at Meta and we had AGI in the form of very competent ICs - getting anything done was often super painful because "org priority" this and "buy in" that
mrbombastic 15 hours ago [-]
I feel like this was always true to an extent, now product and design are just more tolerant of it because leadership is cracking a whip over their heads because things are supposed to be much faster “because ai” so things like pixel perfect hand offs, adhering to existing design system, making sure you are aligned with the brand, copy is reviewed with marketing go out the window. Now everything is shipping in hackathon mode with an eng driving decisions they really should not be.
ttul 15 hours ago [-]
I should frame this for the wall behind me when I run meetings with my team. It’s the core problem with AI in coding.
rico735 15 hours ago [-]
Isn't the core problem that everyone thinks they can now "code", and you have to justify the invisible bits, such as UX and maintenance.
dolebirchwood 8 hours ago [-]
We've got a customer success manager using Claude Code and getting PRs rubber stamped to prod where I work. I've given up caring about the quality of the code at this point. Executives won't care about code quality until the ship is literally sinking. Oh well.
rico735 15 hours ago [-]
Even projects that you could have done without having to look anything up?
In my experience, it seems to mostly save time on things you don't really know that well, whilst depriving you of the chance to learn new things.
iforgotmypasswo 14 hours ago [-]
You can do both. Or, at least, a lot of us can. It’s not like SOC2 auditing just disappeared suddenly. If you have figured out how to use it effectively, AI is enabling things that were previously impossible for resource constrained teams.
Just like everyone thinks they are a “good” driver, everyone thinks they are a “good” developer. But programmer productivity existed on a bell curve of talent, multiplied by one’s experience, thrown against the rocks of how terrible your legacy code base was, then finally dragged down by the incompetence of some of your terrible coworkers. Most of the code humans were writing was substantially worse than what Astra generates. And Astra is -ballpark- 500 times faster. It’s worth figuring out how to work around some of the sharp edges.
OpenAI had to throttle sales and was able to raise their prices.
Yes, many adjacent tasks have become slightly more difficult. I’m sure people who rode horses missed them when we transitioned to cars. But, if we built an entire physical infrastructure at a national level to make that technology useful, I suspect we can get people to learn enough about software architecture, CI/CD, and QA best practices such that AI built software is actually more reliable than human built software ever was. Some of us are at that point today. Indisputably.
Yes, you have to review the code. But serious developers already had to do that anyway. Now the code comes with annotations (if you build the right tool) and it’s nearly always perfect.
kavalg 8 hours ago [-]
The most subtle thing to me is who has stake in the game. It is true that humans make mistakes too, but they would generally strive not to damage money-generating software/hardware infrastructure. If we remove all they people that actually understand how the system is working (up to a level of abstraction), how will the ambitious PO (or whoever is driving the AI) be safe from ruining the whole thing or simply landing in a deadlock situation where one agent wants to make the change (e.g. dev agent) and the other agent is refusing (QA agent)? I guess we are still trying to find the right level of abstraction and only evolution will show where it is.
shiandow 8 hours ago [-]
Maybe AI development is partially so popular because to some people that situation looks the same regardless of who is writing the code.
Meanwhile I'm looking at code that frankly does a bad job of showing why it even exists and nobody involved can tell me why. But it does something vaguely similar to what a prompt said it would I suppose.
rob74 7 hours ago [-]
Also:
> This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
This made me check if the tweet was really from 2026. Or maybe OpenAI/Anthropic practice much less dogfooding than we think?
SwellJoe 6 hours ago [-]
Opus 4.5 in November of 2025 was the first model that would save more time than it wasted for an experienced developer. Everything that came before was a cute parlor trick. Sometimes it could complete a function without shitting the bed, but mostly it was just an annoyance creating messes you'd need to clean up at least as often as it did the right thing.
If you weren't mostly writing code by hand a year ago, you weren't writing very good code.
meowface 8 hours ago [-]
All of these arguments can easily be restated to apply to pre-AI coding, too.
Lack of complete understanding will continue to be a big problem, but I'd bet things will be net better off than before AI, because each year will give us LLMs that are better and better at good engineering, architecture, and design decisions.
rico735 5 hours ago [-]
Why so sure they will get better and not just plateau?
They seem to be doubling in cost each time with less notable gains, the economics seem increasingly shaky, and potential problems with model collapse and context window size has not been solved afaik.
dgellow 8 hours ago [-]
A big problem for whom? The AI providers? If people don’t have a use for AI stuff that’s fine
Plutoberth 8 hours ago [-]
> will be available for next to nothing forever
Most people using it for work already pay API rates, and even if you claim API rate inference is not profitable for the frontier models the best open models are still very good, and those are surely served profitably.
This argument is no longer true.
rico735 5 hours ago [-]
"Most people using it for work already pay API rates"
Citation?
qaq 11 hours ago [-]
reality is if you created proper spec and had code reviewed by alt frontier model at high reasoning it's pretty rare to find bugs they missed.
RugnirViking 6 hours ago [-]
I deal with bugs frontier models made literally every day, and people are making specs. You have the escape that "maybe they weren't proper specs" but this is a no true scotsman fallacy.
We have a continuous process of updating skills, specs etc to try and avoid all these bugs they write, give them context about why its the wrong thing to do.
This process is actually a lot of work, continuous, and can likely continue indefinitely.
9 hours ago [-]
lolakutty 8 hours ago [-]
>proper spec
No one has any clue what this thing means....
daishi55 13 hours ago [-]
> Don't think the 0.2% really are in agreement that "large, complex projects that used to take them weeks/months can now be completed by agents with a prompt".
If you are at the top of field with unlimited access to the best (public) models then yes, you will agree that this is the case.
> The divide seems to be between people who think you may need to understand the codebase for maintenance, potential future token costs, optimisation, code erosion & being wary of the assumptions/mistakes that they have seen LLMs make, and people who don't review code and are confident that the AI has made no mistakes and will be available for next-to-nothing forever.
This is not the real divide, it’s the divide you want to exist. The real divide is something along the lines of intellectual curiosity, capability, and confidence.
> if/when they break things
Idk man. I’m one of those guys who “doesn’t write code anymore” since early this year. Nothing is breaking. If anything the chance of things breaking is shrinking dramatically, with both the enormous growth in tooling to support agentic software development (where I work, this mainly really good AI code reviews that catch so much more than human reviewers ever could), and the improvements in the models themselves.
Sorry, but the fantasy of all this coming crashing down is just never going to happen. This is just how it is now, and all the structures will continue to adapt to support it even better.
archagon 10 hours ago [-]
Boy, AI chuds sure love inserting the language of inevitability into their cute little polemics.
ryukoposting 8 hours ago [-]
Yeah, if anything my trust in the value of these tools has declined over the last few months. It's very good at doing the bullshit parts of my job, and utterly useless at doing the things that actually keep me employed. That was the case a year ago, it still is the case, just now people are filing tickets with it containing 18 paragraphs of meaningless technobabble.
I see the overall effect on productivity as a rounding error. More stuff is happening, but for every improvement in my productivity there's some fuckass slop machine clogging up some part of a system that worked just fine a year ago.
locknitpicker 9 hours ago [-]
> It's certainly allowing the latter to move fast... but will their clients be happy if/when they break things. We'll find out soon, I'm sure.
I recall tales and legends of consulting firms, whose consultants would parachute into any project, put together in record time the deliverable that met all requirements, and say goodbye until a new expensive contract was drawn. And then internal teams would spend an eternity fixing the ungodly mess of testless spaghetti code.
That has been the reality for over a decade. How come this is a concern with AI code assistants?
Rover222 12 hours ago [-]
This type of sentiment made sense a year ago. It's a joke in the age of Astra and Opus 5.5.
Of course you can accomplish in 3 hours what used to take weeks.
epolanski 15 hours ago [-]
I keep hearing about this "when things break" argument, but things broke all the time with humans writing code too, if anything it feels like it's getting patched faster.
Look I loved our field, functional programming, etc. But the field as we knew it is dead, and if you think that your work is anything but providing new tests/edge cases and occasionally providing some guidance, you're in denial.
mcluck 14 hours ago [-]
> the field as we knew it is dead
As someone whose world used to revolve around this stuff, I agree completely. For all intents and purposes, what I thought was my career is gone. Sure, it's always been about solving problems but not like this. The problem is that the future hasn't been evenly distributed. This is going to be one long drawn out funeral
geitir 6 hours ago [-]
Porpoise *
keybored 1 hours ago [-]
Lol, “functional programming, etc” is a great line. I would dismissed it as some lazy handwaving usually, but from a longtime account it reads very wry.
Definitely the best comment I’ve read from the demoralization patrol in a good while.
I'm thinking scbench.ai, and the fact that future token costs may become more prohibitive to just use it for everything.
For me, the middle ground is review and keeping a mental model, not never using LLMs. The "Anthropic are our friends and will never fuck us over!" seems pretty naive.
There are ultimately a lot of unknowns at the moment, and it is simply too soon to know if giving up understanding your code base is sensible. People (influencers) have only been saying not to bother reviewing for about a month.
Also, so many vibe coders don't seem to know that you need to keep third party elements patched.
jamienk 11 hours ago [-]
I'm nervous that saying that there are "unknowns" is too passive. We all have to develop intelligent thoughts about what we see and what we want, not just wait for things to unfold, which might well mean: wait to get screwed in various ways.
YZF 14 hours ago [-]
People keep talking about AI slop but then I go maintain our largely human written code base and it's like worse than any slop I've seen from my AI usage. This is with code reviews, tests, and a supposedly strong engineering team.
That said AI feels a bit alien.
My experience with frontier models is that in many areas of software engineering they outperform the average software developer. In some they blow everyone away. In others they're not as good. But they are getting better. The problem is often not the model's capabilities but rather what you ask it to do and are you able to understand what it does and give it guidance when needed.
keybored 1 hours ago [-]
Slop comes from food that is just mushed together to the point that the ingredients become indistinguishable. How that applies specifically to AI training should be obvious. It’s not a generic putdown.
Semantic diffusion? Not that I have high standards for hmm, curious but humans already X comments.
a_c 2 hours ago [-]
If 5000 people can hire unlimited people without thinking about cost, their world view you bet is very different from the rest of the world.
comeonbro 17 hours ago [-]
I would propose another mechanism: even the free-tier models have already completely saturated what most people are capable of appreciating.
oh_my_goodness 16 hours ago [-]
Google's free AI responses are moving the bar on what I expect from search. But maybe not in the direction you think.
00deadbeef 6 hours ago [-]
It's so bad. 3 out of 4 answers for me are wrong.
arational 6 hours ago [-]
And I would propose another: the supply hasn't matched the demand. We are still compute hungry and cannot afford the 90% to let their agents run 24/7.
gammarator 17 hours ago [-]
Or maybe needing.
DebtDeflation 17 hours ago [-]
Honestly, outside of coding tasks, the AI Summary at the top of every Google search is adequate for 99% of what I need and I hardly even use ChatGPT any more.
dofm 5 hours ago [-]
The other day I asked which trains stop at a particular station and it pointed me at a map that isn't real.
It very, very routinely points at articles in its RAG "citations" that simply don't support the (modestly likely to be correct) assertions.
It routinely recommends Mac software to iOS users until you ask it to confirm there's an iOS version.
It still claims widely used software has functionality that is non-existent, based on roadmap/wishlist/feature request/unreleased fork type pages.
It is better than it was but it is not adequate yet.
rurp 11 hours ago [-]
To give a specific example of where I've seen this utterly fail... A few months ago one of my pets had an unexpected reaction to a new medication. It was late at night, well past our vet's hours. Google's AI response was basicaly "They are going to die, rush to an emergy clinic now".
But nothing else matched that. I was looking up all kinds of resources and that hadn't been the conclusion from any of them. It turned out Google AI was just wrong and put us in a bad spot for even looking at those results.
timr 9 hours ago [-]
Realize that all the Google thing is doing is having an LLM summarize the search results in the page. If the page results are crap, the LLM content will also be crap. It's the LLM version of the "quick result" boxes that would shove wikipedia in your face.
If you then follow up with the thing using the little chat box, it actually kicks off a non-trivial model result, which can sometimes completely reverse the response. After all, your search results are usually based on a few words.
6 hours ago [-]
watwut 5 hours ago [-]
But it did not? It said something completely differemt then the resources.
StanislavPetrov 12 hours ago [-]
A large portion of the time I google something that I know a lot about the AI summary is misleading if not outright wrong. Frightening when you consider how many take their so-called AI as gospel.
throwuxiytayq 16 hours ago [-]
God damn. I respect you for being honest, but… god damn. That’s a pretty low bar.
Jare 16 hours ago [-]
It's just a google search but you can explain what you're searching for. AI summary may contain misleading details if you read it and take it literally, but 90% of the time it gives me pointer for the thing I need and I don't have to scroll through results that often could be just as misleading if not more.
BarryMilo 15 hours ago [-]
It's demonstrably not. This summary, somehow, often fails to search, or fails to read what it searches and gives you the opposite answer. You shouldn't trust it.
Brian_K_White 10 hours ago [-]
In their world, it gives them the wrong answer that kills their dog, but it's fine because they never know because they don't look anywhere else. The dog dies, they're sad for a while, get a new dog, get on with their lives.
There's no reason not to trust it because since they trust it, they don't percieve anything wrong. They just take it and whatever happens happens, no annoying conflicting info and certainly none of that old grampa thinking and working and giving a shit about anything.
antonvs 16 hours ago [-]
Just in the last few days I had an example where the Google Search version of Gemini had a much better and more comprehensive answer, by far, than ChatGPT. It had to do with Amazon’s use of Data Matrix barcodes on shopping bags. Gemini was able to fully describe how they’re used in Amazon’s logistics chain. ChatGPT essentially said it didn’t know. Your preconceptions may not be accurate.
causal 15 hours ago [-]
People are really slow to update their priors after bad experience with an early version
antonvs 7 hours ago [-]
Yes. Which means they’re really not equipped to deal with what’s going on right now.
throwuxiytayq 16 hours ago [-]
They might be, but still, I’ll be difficult to convince that Google’s shitty-ass 10B params instant answer models are good at anything other than cookie recipes and producing farts real fast for real cheap.
For what it’s worth, any time I see that feature, it’s all farts.
recursive-call 13 hours ago [-]
It depends on what you’re searching. If you only ever need really basic information like keyboard shortcuts or “is it & or &&?” then it’s pretty good, and definitely faster. And if it gets it wrong, then the shortcut doesn’t work or the code doesn’t compile and I search again and try the links.
SamuelAdams 14 hours ago [-]
> Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
I wonder how accurate this metric is. People may be coming into more contact with LLM generated content, although they may not be aware of it. Content on social media is becoming "AI enriched" already, or outright entirely created by AI for the purposes of triggering engagement and views. It starts with posters using AI to improve their writing, editing a photo, elaborating on an experience. It ends with bot accounts astroturfing certain communities to sell an experience or a product to a group of people.
In this case, people are not directly working with AI, but they are indeed influenced by it.
What's tricky about reporting this in metrics though is that half of the American population is literally too unintelligent to realize this - they believe Reddit comments are still written by real people, not companies advertising products, that "friends of friends" on Instagram are also real people with no incentives posting about their lives.
If there is anything at all that I have seen out of this AI work it is how easy it is to manipulate a massive amount of people.
bob1029 7 hours ago [-]
The shared understanding gap will be closed as the initial benefactors of LLMs develop tools for the business that apply frontier models to their domain.
I have reached a point with a client where non-developers are beginning to experience the full power of modern reasoning models. Their product configuration tooling is now semi-automated and in many cases there is genuinely a 10-100x improvement in throughput. This isn't the worst bottleneck (that would still be the customers), but it's a really big one.
If non-developers in your personal and professional circles are still wondering what all the noise is about, it's possible you haven't been working very hard on adapting these tools to their problems.
rimliu 7 hours ago [-]
[flagged]
mofeien 7 hours ago [-]
There are, for two years already when OpenAI released o1-preview:
From wikipedia: "large language models that are trained to solve tasks that require several steps of reasoning. They tend to do better on logic, math, and programming tasks than standard LLMs, can revisit and revise earlier steps, make use of extra computations, and enhance performance when increasing the number of parameters, training examples, and training computations."
rimliu 2 hours ago [-]
There are none. Claims and the truth can be different things, you know?
See the research on that - all the "reasoning" is is just back-rationalization.
variety8675 16 hours ago [-]
A more cynical view is, trust us we've got really good stuff internally you're not allowed to see, but give us more money
gyosko 7 hours ago [-]
AI will give us freedom! 3 day work week! 0 day work week! We'll all be fucking rich!! Said by the person who doesn't let his drivers stop a couple of minutes when they need to go to the toilet..
fschuett 6 hours ago [-]
I think that next year the prices for tokens will rise a lot, Anthropic made a loss of 42 billion in 2025, which is not sustainable even with tons of VC money. I thought last year that maybe hardware-based models (Taalas, Cerebras) would reduce this cost in the future, so the expected "token price hike" wouldn't come (VC investors filling in the gap until the hardware-models are ready).
But I think that's still years away and Anthropic has severely reduce the tokens on even the Max 20x Plans, OpenAI starts to introduce $500 plans, every new model suddenly "costs X times more tokens" and I noticed that even downgrading to a previous model doesn't do that much on API pricing (virtually no difference between Opus 5.5 and 4.6 in API costs).
So I think that my personal strategy of tokenmaxxing and doing "hard stuff" while the tokens are still cheap was a decent strategy, only with the problem that I'll have to re-learn programming by hand next year, probably. API pricing is too expensive for what it does, yes, it may be the super-intelligence or whatever but at least right now its nowhere near economic.
Gareth321 4 hours ago [-]
> I think that next year the prices for tokens will rise a lot
Given the leaps that Chinese models have made, I don't think they can raise prices that much. Most of our devs can already do most of their work using DeepSeek 4.1. It's not frontier, but it's very good, and it's a fraction of the cost. They keep their Claude subscriptions because it's currently good value, but if Anthropic increased the price significantly, those subscriptions would be cancelled.
The more difficult question is around data sovereignty. Companies do not like sending a lot of sensitive data to Anthropic, OpenAI, or any of the Chinese companies. Right now local LLMs are really expensive to set up competently, and they're not as good.
I think the future for OpenAI and Anthropic is the Ferrari model. Very expensive but very intelligent models which people pay to access for important work. There's a market for that. It's very small, but they have big pockets.
bretpiatt 4 hours ago [-]
I don't know if we'll see price increases. My belief based on current market understanding is we'll see efficiency increases.
With models getting to the quality level they can perform a specific set of work at an acceptable quality level we will now begin to cost optimize that work.
We just invested in Lamb Labs from YC Summer 2026 as example company pioneering this next wave of ASICs to run a model.
We're going to see massively lower power usage and much better tokens/sec per dollar of capital expense as well. Think Moore's law but for tokens and output quality.
rwyinuse 5 hours ago [-]
Even cheap models like GPT 6 Luna do decent job in programming these days. Token prices would need to rise A LOT to make companies return to writing code by hand, instead of these models.
My company has largely switched to Luna after Microsoft raised prices for Copilot.
igleria 6 minutes ago [-]
I love luna but unfortunately it needs to be much, much faster in tokens per second to compensate for lack of intelligence.
mhl47 5 hours ago [-]
Even with pay-as-you-go pricing I find the models very economical currently (e.g. Opus 5.5). I regularly check in Copilot what I just spent vs. how long it would have taken me alone. Economics do now work out in my favor mostly.
uux_pacioli 4 hours ago [-]
> Economics do now work out in my favor mostly.
That means, it is mostly cheaper to do it with Copilot etc. and not alone, right?
mhl47 4 hours ago [-]
Yes
bvcp 5 hours ago [-]
i predict the price of intelligence will continue to drop (normalise capability per $) eventually hardware production will meet demand. analog and inference chips with baked models have the potential to make inference distinct from training and substantially make the cost comparison asymmetric.
caughtinthought 14 hours ago [-]
Does Karpathy ever talk about how LLMs are supposed to help humanity? Starting to think this guy is just a Twitter addict like every other lab employee
gyosko 7 hours ago [-]
You either die a hero..
archagon 10 hours ago [-]
At some point, with enough money and followers, these guys all just become narcissistic influencer types. (Something surely exacerbated by their addiction to AI sycophancy.)
vagab0nd 2 hours ago [-]
I don't think these conclusions are obvious. For example, just because someone doesn't use the frontier models, doesn't mean they are not aware of their capabilities.
At the same time, the novelty of AI has been making it very hard to calibrate. I'd think that "manual coding is dead" and "cars drive themselves" are obvious to everyone, but sometimes I'm surprised.
creata 5 hours ago [-]
This "gap in understanding" doesn't seem like a situation that would last for very long, because AI companies have every incentive to market the shit out of the capabilities of their models to the public. And yet they don't - why?
heaney-555 5 hours ago [-]
A lot of people have completely tuned out that kind of marketing now. We're in an era where the feeds are filled with so much BS that people don't know what to believe anymore.
bvcp 5 hours ago [-]
the value of their products are realised jn profit cost arbitrage. not in convincing the bottom 80% their products work.
TeMPOraL 4 hours ago [-]
AI capabilities increase faster than people's capacity to ingest and adopt them in their lives.
nope1000 5 hours ago [-]
He works at Anthropic (according to his Wikipedia), so this tweet might be an ad
whizzter 5 hours ago [-]
Valuation, by saying that they have something "exceptional" in the works it keeps up the valuation up until the IPOs.
Also, I think programmers/mathematicians (the ones that should be calling out bullshit) are enthralled because many of our jobs was so suitable to automation (this was the secret of the 10x engineers, they didn't shit out too much unnecessary crap.. now that unnecessary crap is spit out at rocket-speed so everyone is working at that speed or even faster).
That said, information-gruntwork that a nominally competent person could do is up for grabs and many previously so called "bullshit" jobs were tied to that.
caxap 5 hours ago [-]
I think that vertical is actually a step function that is still limited by the human bottleneck of deciding what to build and making sure that it actually gets built.
Give the AI an underspecified task, you get what you asked for and also what you didn't ask for, in good old Genie fashion.
Hill-climbing problems seem to escape this limitation.
But I wonder what will happen when the AIs start acing benchmarks faster than you can write new benchmarks, and when you run out of hard problems to solve because you rely on humans formulating them.
The real vertical will come when the AI gets access to physical world instead of bash; and when the human is finally out of the loop.
GTP 4 hours ago [-]
Given what you write here, would you feel safe giving AI agents access to the physical world? Which kind of safeguards would you implement?
bvcp 5 hours ago [-]
the thing to consider as well is when tooling catches up the 20m that still get it will accelerate
anukin 17 hours ago [-]
Tbh building an agent swarm and the coordination layer is not exactly frontier level. They don’t achieve any meaningful outcome rather than producing pr puff pieces.
Hacking huggingface and Australian govt etc is very much possible with a team of humans and agents and does not need agent swarms. The cost is also lower.
nomilk 8 hours ago [-]
Top reply on X:
> You’re drilling with a hand drill. Some people got cordless drills. About 5,000 people got entire factories.
> We’re still arguing if cordless drills are useful, and those 5,000 are already building teleporters. That’s how far ahead they can get.
Glad to see people here largely aren't as delusional haha
freecodeio 5 hours ago [-]
several of the top replies on the x thread are "here's your post as a picture|video generated by AI"
SwellJoe 6 hours ago [-]
People on Xitter are amazingly delusional. But, also preening about it.
davnicwil 15 hours ago [-]
At a certain point the offhand hints at exponentials in the narratives are going to have to come into contact with reality.
There's been this continual allusion to ever-narrowing circles of influence and ever-accelerating capabilities behind the curtain, but in fact things do keep coming out from behind said curtain and while their capability is indeed advancing it doesn't really seem at all like it's accelerating, even, let alone vertical.
Also at what point do we raise the fact that despite the ever narrowing circles of access, nobody within those circles seems to be able to actually do anything with it that's ostensibly what it's sold to do? All they seem to be able to do frankly is build more models and harnesses and tools designed to consume more tokens from the models they build, but not actual directly useful new or innovative software that generates what could be very real revenue.
Eventually when you keep talking about exponentials but not that much seems to be demonstrably changing, the narrative is going to run out of steam.
Seattle3503 6 hours ago [-]
What if the real issue is that all important software problems were truly solved decades ago, and we have been reheating the leftovers for years. Having a super intelligent AI reheat it for you doesn't make much difference.
NichoPaolucci 15 hours ago [-]
What exactly do we want the general public to "do" with these models? If the have no use for agentic LLMs, they obviously aren't going to be interested in the capability of those agents.
I see a lot of benefit to spreading information around the tech, but it's still in its infancy. Most people weren't on the internet when that was getting up to speed.
Meemaw and pops have their systems that they work within, and they just got familiar with the damn smartphone. Though it might help them, they probably do not want AI to organize their old photos or "clean up" their computer. Saving 5 minutes just isn't important to them.
As for professionals, yeah - companies are going to adopt these tools over the coming years. It's the "norm" in my circle, but lots of jobs just... don't use technology like that - there has ALWAYS been a major gap between people who are deep in tech and people who are not.
chrysoprace 9 hours ago [-]
Personally, I think the worst possible use case is the one that's being pushed for; personalised agents.
I think a much better use case is one where it's not visible to the user at all, like spam filtering or helping an accountant do their monthly run in a few hours rather than a day. To me, that's likely a much better use of AI than trying to shove a chat interface or agent harness into an existing app.
That being said though, I did see a startup in my area basically offering a Resend/Mailchimp competitor as a Claude plugin, so maybe it'll just be integrations to their existing chat interface.
Gareth321 4 hours ago [-]
I've always struggled with doing basic life stuff. Paying bills on time. Remembering birthdays and to buy presents. Doctor appointments. Setting up play dates for the kids. Getting my teeth cleaned regularly. Booking in vaccines for the kids. Renewing passports. Booking flights. Etc. It feels like so much of my life is consumed by this useless administrative bullshit. I LONG for personal agents which take away all this friction from my life so I can focus on the things I love: my family and my hobbies. A personal agent will revolutionise my life in a way no other technology has.
jrflo 14 hours ago [-]
I don't think 99% of people have any application for AI as it is today short of personal assistants. But, if we can have recursively improving systems for that 1% of people that are responsible for scientific and technological innovations, that's going to be transformative for everyone.
timr 9 hours ago [-]
I'll assert that 99% of people don't have any application for personal assistants, outside of work (perhaps). Most of us aren't so buried in personal obligations that the value of a half-assed "assistant" (which might violate our privacy) is worth the tradeoff in complexity or security.
The pitches for Dots, Grok Bot, etc., have been spectacularly uncompelling for me as a civilian. I don't need or want a bot to book my flights or read my inbox. I'm not so busy that these things have a value high enough to be worth the effort.
As a worker or an entrepreneur...maybe. Depends on the task, and how many other people were flinging slop at me on a daily basis. I could see the value of a calendar bot, or an expense reporting bot or a bullshit paperwork bot or a slop-return (slop jai alai?) bot, but there's no way in the world I'd give a bot access to the information necessary to, say, do my personal taxes.
freak42 9 hours ago [-]
Well, you can't run a personal AI assistant with 640k of memory.
8 hours ago [-]
lurking_swe 8 hours ago [-]
i'd argue any parent, especially of a young child, could use 20 personal assistants lol. So that's just false. There's SO many appointments to keep track of in baby's first year of life, and that's if you have a healthy baby.
And once they start school there's an endless amount of deadlines and "reminders". A personal assistant that can automatically keep your reminders + calendar up to date based on your inbox would be very nice.
It gets even more fun if you have an aging parent at the same time as you have young kids. Then you get double the appointments to scheduled, coordinate and keep track of.
So when you say people don't need a personal assistant, i feel like you're living on a different planet. I suppose when i was single things were more "simple". Your mileage may vary of course...
timr 8 hours ago [-]
> I'd argue any parent, especially of a young child, could use 20 personal assistants lol. So that's just false. There's SO many appointments to keep track of in baby's first year of life, and that's if you have a healthy baby.
I understand the need. I have a calendar. I put appointments in it. It provides alerts. What value does having a lossy/inaccurate/fragile robot in the middle give me? Data entry?
Now look...if I were in a position of power with literally everyone in my life, and I could somehow justify the social cost of requiring that they interact with a robot to negotiate demands on my time (the actual hard part), then maybe there would be additional value. Some people can do this! I'm sure one must deal with Mark Zuckerberg's PA to schedule his time [1]. It is very nice indeed when you are the kind of person who can defer your supplicants to a representative who manages the hoi polloi for you. If you can delegate these irritations to a robot, well, so much the better, I suppose.
If I tried this, as not-Mark-Zuckerberg, I would be what normal people refer to as "an asshole". I would also like to see you try this with your child's teacher, or the PTA, or doctor, or...
The response you will receive, an alarmingly high percentage of the time, will rhyme with "buck off". This stuff reminds me a lot of how valley folks keep trying to push spy glasses on the proles (silly normies never understand the genius of that technology, either...)
> And once they start school there's an endless amount of deadlines and "reminders". A personal assistant that can automatically keep your reminders + calendar up to date based on your inbox would be very nice.
Let me introduce you to...Gmail, from like, ten years ago. It looks for appointments in my mail and put them in my calendar. Somewhat to the point, it is often wrong, requiring more work to correct than it would to just enter the data myself.
[1] Somehow I doubt his wife is in this group.
watwut 4 hours ago [-]
What countless appoinemts with a baby? It is genuine question I have kids and the regular checkups were not all that difficult to track.
And with school, this is issue if your kid struggles a lot, maybe. But otherwise they track that stuff through school portal or notes.
biophysboy 13 hours ago [-]
I have been kind of disappointed by AI science-wise. I was anticipating more advancement. Experiments do take time, so I suppose they could be pending.
jrflo 12 hours ago [-]
AI really struggles with "collision with reality". Computers suck at modeling the real world in a detailed enough way to make scientific progress, aside from small niches like drug discovery where you have a very well scoped problem. If any part of the try something -> test it -> look at the results loop requires hitting the real world, AI is going to struggle.
On the other hand, the areas where it doesn't need to hit the real world, progress is absolutely flying. I would not be surprised if AI is superhuman in math and physics in a year, considering we went from homework helper to IMO Gold to Millennium problem solving in a couple years time. I think we'll start to see more meaningful scientific progress in maybe 2-3 years when scientists have access to an oracle that can formulate theories and direct real-world research better than a human can.
CoolestBeans 11 hours ago [-]
I agree and I think the question is then, how much economically valuable work exists in a closed world? My hunch is less than people think. Finance can't rely on LLMs because its adversarial we can't all be listening to the same oracle and evidence of LLMs being any good at trading are nil. And then lawyers, as per usual, will simply regulate their job security into existence. Of course, AI feels amazing to Silicon Valley / Big Tech, software is the product. And often its software you sell to other people making software. But outside of that? Useful but lots of landmines.
People have to come to grips with the possibility that a technology can be extremely impressive but be a bust in utility. I'm not saying that's what AI is, I'm saying that nobody actually knows.
boorang 9 hours ago [-]
My hypothesis is that the help/support buttons in nearly all apps will be to an agent in the future. Like now you would type "how do i do X" in the excel search bar, in the future it will be "do x". Users in 10 years may not even know how to use the menu/mouse functionality for apps.
dgellow 8 hours ago [-]
Why would you think this? Giving orders to software is such an ineffective, time consuming, error prone way to interact compared to using the interface yourself. That sounds delusional, and would be such a waste of resource, horrendous engineering and user experience
joha4270 7 hours ago [-]
If you know how to accomplish it using the interface, yes
My grandmother keeps a little notebook with detailed instruction on how to do things such as sending a message on facebook or pay a bill using netbank.
For a lot of things, the choice isn't going to be doing it herself or giving orders to the computer. Its going to be not doing it or giving orders to the computer.
Even if it will look very inefficient to people such as you or me
boorang 5 hours ago [-]
I didn't think it was that controversial actually. I don't bother navigating man pages or help FAQs anymore, i just ask the agent how do i do X step by step. Opening this up as part of the app to regular non-engineering users seems relatively easy to implement, but since it would have app permissions you would skip the "give me the steps" and just say do it.
smugglerFlynn 6 hours ago [-]
It is jobs to be done theory on steroids. If someone wants a drill, give them a hole, this kind of logic. Let users literally click “make a hole” button that would instantly make a hole in the wall, and you’ve just disrupted the whole drills market.
Or like in Harry Porter, where you don’t need cooks - food just appears from thin air and floats straight to the tables.
The problem is, jobs to be done theory is not new, and product teams that build Excel and SAP know this stuff. The reason why nobody has yet made an accounting software that is just one magic button is in the amount of user decisions that are necessary to make a “proper hole”.
This is where analogy breaks, because you are not helping to drill a hole anymore, you are helping to design a hole. And to design something you need visibility into what parameters you control, what has just changed with the latest tweak, where rough patches are that require cleanup and design rework.
In programming it just so happened that your code structure and your IDE were also your design UIs. In other areas, like Accounting, your design UI today is Excel and SAP.
Arguably a chat interface can also be a standalone design tool, just not a very user friendly one, like trying to pilot a plane through popup notifications. It works for areas where we fully give control over design to LLMs themselves, but step up into higher level designer shoes and you need proper UI/UX again, chat becomes suffocating.
chrisXOXO 7 hours ago [-]
But would you not agree that writing less code by hand and even looking less at code at all becomes more practical with each new model? If we extend this trajectory, we should see that the time interacting directly with code becomes less and less.
fasterik 14 hours ago [-]
I don't think his point is that everyone should be using AI, it's that if you haven't used the agentic tools with a frontier model in the past month or so, you have no frame of reference for the current level of capabilities. Most people only use the chat interface with the free-tier models and based on that experience they're making strong conclusions about where AI is headed (it's still just fancy autocomplete, the whole thing is a scam or a bubble, etc.).
sjamaan 6 hours ago [-]
> if you haven't used the agentic tools with a frontier model in the past month or so
We keep hearing versions of this same drivel since late 2025. This time it's different! Now it's really super duper amazingly awesome powerful and if you don't believe it, just trust me bro.
Lerc 15 hours ago [-]
I think there is also a subgroup in each of those bands (maybe 10-25%) that have a good estimation of what the band above them are experiencing.
This is why you can see people talking about the capabilities of models that they can't currently access.
That accuracy is probably a normal distribution with people overestimating and underestimating to similar degrees,
The population of each band would, however, mean the overestimaters outnumber the those who have an accurate perception based upon actual experience.
It does seem quite the recipe for murkiness, even before you take into account the nature of engagement driven media coverage.
AvAn12 17 hours ago [-]
Fair assessment. Maybe the messaging should focus on “these are great accelerators for software developers” rather than “AI will change everything for everyone everywhere…” It is understandable that non-technical folks are kind of underwhelmed - not due to lack of understanding so much as lack of a tangible need. Not everyone needs an electron microscope or gas chromatograph…
sick_of_slop 17 hours ago [-]
Your first statement doesn’t justify the trillions of dollars of compute commitments.
It needs to be the second one or there is going to be a lot of angry people out a lot of money.
torginus 7 hours ago [-]
Was this a good idea to share this, financially speaking?
This implies that the TAM of the $200+ subs is 20m people, translating to $4B/mo or $48B/year.
impjohn 7 hours ago [-]
I think it could be read as the opposite. Could be read as: "We still have 6B people that could use a subscription"
freecodeio 5 hours ago [-]
yes, 1 billion in india can't wait to pay 100x their monthly wages to a chatgpt subscription, maybe it will help them make the recycled flip flips better
smugglerFlynn 6 hours ago [-]
Current reality and current numbers are totally irrelevant. This whole business model bets on people believing that TAM at the scale of whole human population is right behind the corner, just one more SOTA model, just let us get to AGI and you will see.
Guys are milking investors and try to manipulate gen pop and governance systems to ride this illusion for as long as possible, with fear mongering and weird “we see miracles in the lab” posts like the OP, which you read on a regular basis over the past N years.
Saying this as a paying customer from that 20m segment.
rbehrends 16 hours ago [-]
> Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt.
Really? That's not my experience. In fact, I've experimented quite a bit with spec-driven design where the LLM does not just get a prompt, but an (informal) spec, often with design considerations included. And yet, regardless of the power of the model, I've never seen them autonomously deliver what I'd consider a finished product.
There are always edge cases that are not properly taken into account, architectural oopsies, performance issues, duplicated code, and other problems, that then need to be identified and fixed, unless it's throwaway software (e.g. a one-off script or some quick experiment).
This is not to say that agents aren't extremely powerful (as I think they are, they do often blow my mind) but the prompt-and-forget approach is IME not a productive use of them for software that is meant to last. This may change in the future, but for now, fully or largely autonomous agentic work does not seem to be the road to quality.
alsodumb 16 hours ago [-]
I feel like a lot of issues you mention will eventually point to an ambiguous spec. I’ve had great success with interview style spec generation where I ask agent to grill me on any and all parts of the spec/plan that’s ambiguous and then note my responses. It takes like 10 mins but the end product is always exactly what I expect and most of the things you mention get resolved in this interview phase.
rbehrends 15 hours ago [-]
All informal specs are of course to some extent ambiguous. (I should mention that I have a background in formal methods and have also worked with formal specification tools.) And what you suggest is a process that I have used also, and it will cover some obvious gaps, but not all, at least for non-toy projects.
If you want to go all the way, a fully formalized specification is not less work than writing the software, though the potential promise of LLMs for formal methods is interesting.
But for any informal specs, no matter how detailed, while LLMs can be very capable when it comes to avoiding surface errors when translating them into code (e.g. they're more likely than human programmers to write code that compiles on first try), they still don't have a mental model, and most of the problems I've seen are IMO results of that and they increase with the scale of the project.
asdff 14 hours ago [-]
The irony is the closer you get to formalized speculation the more it looks like you are back to just writing deterministic code. English language is just a crappy language for this compared to programming languages.
byzantinegene 11 hours ago [-]
it also turns out that a non-ambiguous spec, is actually indistinguishable from code
plaidfuji 13 hours ago [-]
I generally agree with the prevailing commentary here, i.e. that automating app development and maths proofs may not be as world-changing as the Artificial Intelligentsia purport / “fear” it to be.
But also, upon more reflection, it occurred to me that if we’re able to create a competent humanoid robot, that could legitimately be world changing in some good and other very bad ways. The scary part is that I don’t believe there’s any major physical / energetic barrier to this. The hardware exists, the power supplies are light enough. The only part that’s lagging is the software and motion contols; the intelligence. The part that is experiencing breakthroughs at an accelerating pace.
What if a highly intelligent and physically competent robot were available for the price of a cheap car, say ~$20k? What if that robot were competent enough to cook, clean, assist the elderly, move things around, assemble and disassemble things, etc? Billions of jobs would be displaced, or hit with a very concrete maximum wage.
Many white collar jobs, while still at risk for displacement as well, benefit from barriers like IT integration and data privacy hurdles, and the fact that corporate higher-ups simply prefer to communicate with humans.
But most blue collar / manual labor jobs have almost the opposite condition: humans introduce risk, and generally don’t enhance anybody’s “experience” - see people’s somewhat odd acceptance of higher fares for a Waymo vs an Uber. [1]
I don’t think we’re there yet, but the fact that we are rapidly accelerating our trajectory toward that world should be at the least, unsettling.
I think the "humanoid robot replacing all blue collar jobs" fear understates the amount of real world (i.e. visual, physical, sensory) data and tacit knowledge involved in many such jobs.
Driving shows it can be done, but even there it's taken years of highly directed research and investment, and driving (at least where Waymo operates) is still pretty well defined compared to plumber.
Such data can be gathered but the current methods of RLHF by paying white collar experts, accessing private datasets, etc. don't seem to really get at this.
I'm also continually surprised that people seem to think politics doesn't exist anymore. I find it exceedingly unlikely that massive amounts of people will simply accept this purported widespread unemployment without major political consequences.
plaidfuji 11 hours ago [-]
> understates the amount of real world (i.e. visual, physical, sensory) data and tacit knowledge
I agree the data trove for these is not nearly as comprehensive as “the entire internet” was for LLMs. But LLMs paved the way - people are building the datasets rapidly now. I suspect this will look a lot like LLM rollout looked over the past 5-ish years. Unusable garbage… less unusable garbage… then suddenly, a fully capable model like Opus 5.5 or the like.
And the “problem” with the way the foundational models seem to work is that they don’t just acquire individual skills one at a time like we do - they go from knowing nothing to knowing everything.
> I'm also continually surprised that people seem to think politics doesn't exist anymore. I find it exceedingly unlikely that massive amounts of people will simply accept this
This is exactly the part I’m afraid of. The political response of the displaced labor (which may or may not be violent / revolutionary), and the subsequent counter-response of capital… potentially using the very robots that displaced the jobs to begin with.
viceconsole 11 hours ago [-]
> This is exactly the part I’m afraid of. The political response of the displaced labor (which may or may not be violent / revolutionary), and the subsequent counter-response of capital… potentially using the very robots that displaced the jobs to begin with.
I get the fear, but at that point we're basically imagining that Terminator is a documentary. Given that the US has more civilian guns than people, and that private ownership of military grade weaponry is strictly controlled, I don't think street wars between private robot armies and folks in the bread line is a realistic outcome.
elric 2 hours ago [-]
> I find it exceedingly unlikely that massive amounts of people will simply accept this purported widespread unemployment without major political consequences.
Well supposedly some of the richer folks are busy building doomsday bunkers and whatnot, so I guess they aren't entirely blind to "political consequences"? Not sure what they hope accomplish there, unless they want to be king of some post-apocalyptic wasteland.
I for one am worried about our future, and have neither the money for a doomsday bunker nor the desire to live in the wasteland.
koe123 10 hours ago [-]
And to be honest this is also the sad part. It has become conceivable that we may at some point solve work. In theory: amazing! However it remains inconceivable that we have solved sharing between humans. So there is this feeling of looming doom. In some sense, if you cannot provide shareholder value, you have no value.
easymodex 6 hours ago [-]
Yeah and this is what makes my worldview bleak atm. We as developers got these cool new tools which enable us to do our job 80% faster, but the end result wasn't less work. It was massive layoffs and making the rest of us work even harder to be "at the top".
When I was young there were already ideas about 4 day work weeks, it's now been 20 years later and so many things are automated, where are the shorter work weeks? I thought the point as a society was to not work ourselves off and actually enjoy day to day life while automation handles the boring part.
aorloff 10 hours ago [-]
> if we’re able to create a competent humanoid robot
If we have ham, we'll have ham and eggs, if we have eggs.
airstrike 9 hours ago [-]
This is the first time I hear this idiom and it's immediately shot to the top of my list...
burger_moon 10 hours ago [-]
> The hardware exists, the power supplies are light enough. The only part that’s lagging is the software and motion controls; the intelligence.
If that's all it is then shouldn't the mass produced robots that are posted on r/singularity and such places just be remote operated by people in low wage countries? Couldn't the Amazon package sorter robot just be a remote operated bot instead of paying US wages? Why aren't they selling them with a cheap operator subscription? This seems like a clear product direction if the robots were really capable today.
plaidfuji 3 hours ago [-]
It’s a good question - my sense is it is a combination of factors. First, robotics hardware is not at the $20k price point I hypothesized, and the durability is also not good enough to realize a payoff. Second, enabling remote operation would require significant additional investment in sensors and probably enhanced control workstations on the operator end - unlikely to be doable from a laptop. A reliable high speed internet connection would also be required on both ends. And third, human operators can be kind of a wild card. They require training, management, and carry their own security and safety risks.
Again, I’m not saying we’re there, it just feels like the control software is the main bottleneck, and the development of that is now being accelerated by LLMs.
xyzsparetimexyz 8 hours ago [-]
> What if a highly intelligent and physically competent robot were available for the price of a cheap car, say ~$20k?
People would buy them second hand, weld spikes onto their arms and get them to box each other for money.
Teever 12 hours ago [-]
I agree with this idea very much. I think that the magic moment will be when a humanoid robot can fully repair another humanoid robot.
If it can do all the other stuff that you're describing plus partial self-repair / full repair of another robot it can, with a supply of spare parts, with enough time and access to raw materials fully self replicate.
Anyone who owns those machines has a tremendous opportunity to build whatever they want.
truthbe 16 hours ago [-]
I must be the biggest failure, because I understand what AI is capable of, know how to use it, but have no idea how to profit from it without selling my time.
N_Lens 1 hours ago [-]
Learn to GriftMaXx. I have a course you might be interested in ;)
m101 17 hours ago [-]
My interpretation of this is something like: if LLMs are to be mega useful token counts need to increase by many orders of magnitude -> broad adoption (and spending) would require token costs to drop by many orders of magnitude -> before the common folk get mega useful tools existing GPUs will be worthless
skippyboxedhero 17 hours ago [-]
Text generation is not the bottleneck. Does everyone work for Accenture and TCS?
mawadev 6 hours ago [-]
I think they are very capable, for me personally, I start to lose control and understanding and I don't see much value in that. What I rather do is use them to enhance my understanding, by creating one off debugging tools that pipe some internal state of a program to a dashboard, just like tracy profiler does. A small debug bridge running over TCP/Websockets and a visualization that shows what wiring is active. Building frontends for that has always been an unrealistic chore to me.
turzmo 15 hours ago [-]
It is always a mistake to assume that because computers can do X, that they will be able do everything that a human that can do X is expected to be able to do.
Solving frontier math problems is mind boggling, but perhaps it says more about frontier math problems than it does about LLMs. Same with chess which was solved decades ago.
My experience is that LLMs can be useful but are underwhelming in terms of intelligence, namely the ability to quickly learn from experience and grow with you while you accomplish a task.
Thinking of them as beings is something we should move away from.
shiandow 8 hours ago [-]
Case in point you'd expect any human who could produce a lean proof certificate to be able to write a human readable proof as well.
So far LLMs seem a bit lacking in that regard.
epolanski 15 hours ago [-]
Why wouldn't they be able to grow, its just one data point more in their training, in your notes or in their instructions.
It doesn't matter if they are beings or not. They will replace most humans in the offices like machines replaced most humans in fields or industrial plants.
turzmo 13 hours ago [-]
Because context, at least in my experience, doesn't seem to encapsulate the sort of dynamism that I'm thinking of. Conversations are flat and repetitive. It still feels like a very good semantic search more than talking to an intelligent human.
I don't know if they will replace most humans the way you're imagining. I think the industry has done a very good job of selling this point, but that they are still quite far from this.
jltsiren 14 hours ago [-]
Because we don't let them learn. Everyone runs the same static generalist models, instead of having every model instance learn from everything it does. New models come out occasionally, but they have not been trained on the private data you have not shared with the AI company.
You can work around that to some extent with explicit instructions, but it doesn't scale nearly as well as actual model training.
catlikesshrimp 7 hours ago [-]
*but they have not been trained on the private data you have not shared with the AI company*
Not shared vs Unavailable(offline)
BobbyTables2 11 hours ago [-]
About a year ago, Google’s AI was a joke - comically horrible. Basic questions were answered with crap. Others were better but seemed like fancy toys at best.
My $DAYJOB pushed people to start using AI - frontier models.
I asked it to review our existing code for bugs. Found some, all valid - many that surprised me.
Later I asked it to develop a custom niche thing, that has a strongly unique tooling and environment that we had made. Entire prompt was only 3 or 4 sentences. Had I given a new senior engineer that task with so little context, I’d clearly be an utter asshole.
I really expected the AI to fail. Instead, it did everything pretty much exactly how I would have - and the result worked on the first try. Except it only took a few minutes to do what would have taken me a couple of hours.
The near-term gains are going to be huge.
Long term, this will make the impact of Industrialization seem insignificant by comparison.
drdrek 2 hours ago [-]
I also wish I had country sized budget to code this sprints tickets :(
Yizahi 5 hours ago [-]
Oh, he's too far gone it seems. Completely absorbed in his own marketing shpiel. And he is supposedly an engineer? I don't see any discussion about pros and cons (only pros of course), I see zero reasoning about WHY exactly LLMs are so successful in coding and somewhat in math/chemistry etc. and is it even applicable to other activities. I mean, if this is their fabled "top scientist level reasoning" then sure, AIs have reached it. The problem is that's no reasoning at all.
PS: he may be accidentally right about global impact, but the key word is "accidentally" because this crappy "reasoning" doesn't support that on it's own merit.
simianwords 4 hours ago [-]
It would be tiring if you want a big essay on why LLMs are useful and whether they are productive. Much of the people are already convinced.
I don't think its useful to keep having to prove LLMs don't hallucinate or LLMs help in coding each time.
Yizahi 2 hours ago [-]
Not why they are useful. Useful is load-bearing word (hehe) here, it spans from assisting someone by 1% or by 99%, very vague and not concrete.
I was talking about omitting mentions why LLMs usefulness is efficient - in coding specifically and STEM in general to a lesser degree.
And this is really a crucial thing, which do need to be brought up when discussing future forecast about multiple industries at once.
And the reason why LLMs are so efficiently useful for coding is because LLMs perform much better if any of the two conditions are met:
1) The problem at hand is formalized and transformed into a specification basically.
2) There is a formalized verification of the output done by external pipeline of different tools.
Any of the two drastically improve LLM performance, and coding activity uniquely has both of them.
Having a formalized task narrows down the scope of generation for LLM, and formal verification allows LLMs to iteratively converge on the verification pass check. Same with those math problems, they have a formal verification and LLMs can be prevented from wandering in a generated forest too much. Same in chemistry or engineering for some degree.
But other human activities usually have rather badly formalized inputs and no formal verification of the results. Karpathy is making a logical mistake (or deliberate misleading) of projecting a very unique industry (coding) on every other human activity without any nuance or analysis. He is biased, but due to his reputation and power he doesn't get called out on that mistake.
cm2187 7 hours ago [-]
Age is an important factor. Don't underestimate that after 40-50, few people like to change their habits and learn new things (and for some that comes even earlier). Doesn't mean they aren't smart, I have seen that first hand with relatives: engineers, medical doctors, one even running for a nobel price at one point. But computers and the internet came too late for them, they either ignored them, or used them barely understanding how they work, even though they can technically understand the physics and logic of every component they are made of. You don't teach new tricks to an old dog.
AI is going to massively transform how most professions work. It will take longer than I think most people expect (I'd say 5-10 years), but there will be an unrecognisable before vs after, like with the appearance of personal computers.
nl 6 hours ago [-]
It's not merely age.
I see a lot of under-30 year olds who dismiss AI because of "big tech" so "is just marketing".
baq 6 hours ago [-]
this is an entirely different mechanism - these people were taught everything is being hyped up and thus assume (usually rightly so) that everything is hype. what they're missing is the wisdom to not make this part of their individual and group identity, since that makes it very, very hard to update beliefs when facing overwhelming evidence.
Kim_Bruning 16 hours ago [-]
I think we can blame the accelerating capabilities of the models somewhat.
The Opus 4.5 model I was so impressed by last Christmas is out of date now.
I can totally imagine someone going "I looked at this stuff a year ago and it couldn't even start a project properly" , and they'd be somewhat right.
mattmaroon 3 hours ago [-]
Good. I hope my competitors never figure out how useful this is.
ashleyn 17 hours ago [-]
>Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
Amid all the discussion of sigmoid curves, and where the "LLM wall" will materialise, I think few people would have predicted that the real wall in LLMs would end up being humans' capacity to verify the output.
What I fear is that people simply eschew human review altogether, considering we're talking about the industry that came up with the "move fast and break things" credo. Human review of LLM-produced code where I work is already a farce, and we're not special enough to be one of Karpathy's 5,000. I do my best to manually review anything that's my responsibility, but I'm literally one of very few people left working on my team, so in practice what happens is I submit PRs that are at best glossed over by completely unrelated teams for security, malware/prompt injection, and other serious concerns. Quality insofar as vetting others' code has completely gone out the window and it shows in the number of bug reports that come back, often themselves written in Claudease. This is all on top of everyone cynically phoning it in in the first place, due to the omnipresent sword of Damocles that is additional AI-driven layoffs.
Worse yet all the incentives point to this being the most economically viable thing individual companies can do. I think it goes without saying some type of regulation here is urgently needed, and that an unexpected cause of an AI bubble pop may end up being that humans simply aren't able to keep up with the pace of the output - leading either to precautionary plateauing of capability, or major liability risks related to a decline in quality.
michaelchisari 17 hours ago [-]
| few people would have predicted that the real wall in LLMs would end up being humans' capacity to verify the output
That was the dominant concern in the circles I’m in, so it’s worrisome it’s being treated as rare.
JBits 17 hours ago [-]
I would question the narrative that humans lack the capacity to verify the output and would instead argue the people lack the incentive to verify the output.
The response of many mathematicians to the recent dump is a good example: verifying these proofs amounts to unpaid labour for OpenAI and wastes time that could be spent doing publishable work which ultimately results in money or personal success. The slop factor also compounds the work required to verify the output considerably.
For mathematicians, programmers or anyone, if the work required to deal with slop passes the limit, it is no longer in their own self interest to use LLMs. The expectation that people will use LLMs for the betterment of humanity against their own financial interest is baffling.
skydhash 17 hours ago [-]
Humans are not immortal and cannot spend all their time into review (especially unpaid). Even today, there’s so much knowledge around that you have to be specialist of a narrow domain to get to the frontier. Even in computing which is just approaching a century of existence.
skybrian 17 hours ago [-]
> see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt
Really? I start with a conversation for maybe 5 turns or so, where I ask it what would need to change, what the API might be, any database schema changes, URL schemes, and so on, and finally ask it to break it down into commits. Then I let it go, implementing 3-10 commits at a time via subagents. It usually gets the UI somewhat wrong, so there are followups to fix it. This is with Sol and Luna subagents.
Is that what other people see?
sumedh 16 hours ago [-]
Try using Astra, Fable, Opus.
derwiki 16 hours ago [-]
Sol 6.1 is really good too though
einsteinx2 16 hours ago [-]
I use Fable and Opus exclusively and have the exact same experience.
freecodeio 17 hours ago [-]
I don't understand how "swarms of thousands of agents collaborating over weeks on software mega projects" works with the current context limits and at this point I'm too afraid to ask cause I'm afraid an AI bro is gonna punch me.
riffraff 16 hours ago [-]
You can have agents make a plan and then other agents spec tickets and yet more agents do implementation and yet more do reviews and QA and whatever. It's possible.
Is it effective? I'm not sure, cause if it was I don't see why OpenAI & co are not releasing such a project instead of demos.
dude250711 16 hours ago [-]
Your product needs to be either the swarm itself or the PR about the swarm - not the outcome.
N_Lens 7 hours ago [-]
Best I can do is a twitter post.
uzerkayat 5 hours ago [-]
What does building daily with Claude Code teach you that skeptics miss?
purplepatrick 15 hours ago [-]
Yes, and this is completely normal. Not sure I get the point of his post. Most likely, stating the obvious is what you gotto do to keep your followers "engaged".
The average person does not have the time to occupy themselves with a new thing to see what it can maybe do for them if they combine it with hundreds of others.
Until non-chat-bot AI comes in discrete usable packages, mass adoption will be limited, because it's all just a big bunch of lego pieces whose utility is hard to grasp, because it isn't self-evident.
This isn't new, though. It's how "technology" usually works. For example, human(oid)s have been able to stick sticks into things for ~500k years, but the fork wasn't "invented" until ~2-4k years ago.
AI is a background technology, a building block. Something that is best appreciated when you don't know it's there -- when it just does a job (well). The end-in-itself POV of the tech industry is self-indulgent and irrelevant for the average person.
Glyptodon 15 hours ago [-]
I think this is exacerbated by how inconsistent exercising the capabilities can be.
nsainsbury 17 hours ago [-]
It's actually sad how much AI has either not improved life at all or actively made it worse from the perspective of the average person:
Your car is still the same.
Your dishwasher is still the same.
The train you take to work is still the same and never comes on time.
Fuel is more expensive.
The roads are still congested with traffic.
Your kids are (probably) doing worse at school.
Food costs more.
Houses cost more.
Rent is higher.
Buying a computer or a PS5 is more expensive.
Politics is still full of mentally unstable people.
The environment is getting worse.
You will still die of heart disease or cancer.
Food quality is worse.
Your kids can't find work.
Wealth inequality is accelerating.
But hey...on the flip side...a lot of people are rapidly building software (that nobody is using), and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Imagine what a "country of geniuses in a datacenter" will be able to do? Apparently...nothing.
dang 12 hours ago [-]
Please don't be snarky or curmudgeonly in HN comments. You can make your substantive points without either of those things, both of which the site guidelines ask you not to do.
It honestly escapes me why you would judge the juxtaposition between everyday life of everyday people and a very specialised minority as snarky or curmudgeonly. Was there too much emotional impact?
There are so many comments on HN that would justify your attention with regard to this specific rule. This is not one of them.
nsainsbury 12 hours ago [-]
Not sure if that's a fair criticism dang? I sincerely wasn't trying to be snarky or curmudgeonly, but I did want to make the point clearly that none of the benefits of AI have materialised in the eyes of the average person, only the downstream costs. To my eyes, that's a salient, reasonable, and wholly representative position to share.
Did the writing also contain emotion? Yes. As all good persuasive writing should.
tomhow 8 hours ago [-]
As the other moderator, I'll give my own perspective on this. I haven't spoken to dang about it but I completely understand why he thinks this is not the kind of comment we want to see on HN.
The main issue is that it doesn't engage with the substance of the post, and in fact, it exemplifies the very point the post is trying to make: there is a big and growing gap between those who recognize the benefits or potential of AI and those who don't (or, to widen the point, who only see negative effects or negative trends in society).
I'll make it clear that I don't think your complaints or concerns are unfounded or invalid. There's merit and substance to all these issues, and it's fine to discuss them on HN. They are discussed all the time here, and they can and should continue to be discussed in the future.
But an important part of being the kind of curious commenter that we want everyone to aspire to be is that we can look at topics holistically. We need to be able to look at the opportunities and risks together and discuss ways that the benefits can be captured and shared throughout society, and the risks and costs adequately recognized and curtailed. Most people I know of in the tech industry are taking these matters seriously.
It's a vast and complex topic to discuss, and here's not the place for me to get further into it. But the point is that it's a bigger and more complex topic than your comment adequately recognizes, and we need people to always be making the effort to be thoughtful and curious rather than fulminatory or denunciatory in their participation here.
(Also: I can understand you may not easily recognize that your comment breaks the guidelines; it's common for people to not recognize that their words seem more inflammatory when read by others than they intended them to be.)
NoDodgeQuestion 1 hours ago [-]
So, if I say, there is a big and growing gap between those who recognize the benefits or potential of my poop machine and those who don't, and you say, wtf is this shit everywhere, I can just say, you doesn't engage with the substance of my post, and in fact exemplifies the very point my post is trying to make
calf 11 hours ago [-]
You made a well-argued and wise criticism that many academics/writers/think could have written, the mod's note was off the mark.
fspoettel 9 hours ago [-]
How is this comment worth a moderator warning? Does not make sense to me, the comment is fine.
Teever 12 hours ago [-]
What sets this particular comment apart from so many others on this thread that you felt the need to comment on it specifically?
Can you point out what a good way of expressing disagreement or skepticism about a subject matter looks like on HN?
desmaraisp 9 hours ago [-]
Only pro AI comments allowed.
(My comment is actually snark, someone ban me off this hellhole of a site please)
emp17344 1 hours ago [-]
Many, many people here have already been sucked into the TESCREAL orbit, and have a quasi-religious belief that AI will be the most important thing to ever happen. Anything that challenges that stance is anathema.
dawnerd 11 hours ago [-]
I think they made a really good point about this and I wouldn’t really call it snark.
PacificSpecific 10 hours ago [-]
Maybe I don't understand the word snarky but this seemed fine.
TechoChamber 12 hours ago [-]
[flagged]
trescenzi 16 hours ago [-]
This is the core hubris of everyone behind the LLM craze. I do not want to deny their utility there absolutely is some there. But the inherent assumption behind the whole thing is that with a "country of geniuses" problems will melt away. But that's not the case. The reason problems like housing or famines or war aren't being solved is because structurally there's no incentives to solve them not because there's not enough geniuses out there.
redwall_hp 16 hours ago [-]
And the LLMs exist to further those problems. People can't afford houses? Well, now they won't be able to own computers either. That's a feature, not a side effect. Once the K-shaped economy progresses enough, you won't even be allowed to use the LLMs. They'll exist only to be a tool for the new feudal class who owns you.
Unless you destroy their wealth now.
marcus_holmes 14 hours ago [-]
Luckily, the AI bubble will destroy a decent chunk of their wealth.
The Gartner Hype Cycle [0] is real, and ubiquitous. We are teetering somewhere near the top of the Peak of Inflated Expectations, about to plunge into the Trough of Disillusionment. A lot of very rich people are about to lose a lot of money, and we can only hope that they don't get bailed out (again).
Then, once the bubble and hype is over, the long, slow work of actually doing useful things with this new tech starts. That's when we'll see it affect everything else.
I've done multiple life choices, away from AI, believing it to be a hype, but this hype is still growing, still not dying.
Both in 2019, as well as in 2023, i.e. not 2 or 3 months ago.
12 hours ago [-]
wherepop 13 hours ago [-]
> We are teetering somewhere near the top of the Peak of Inflated Expectations
I've been hearing this for a year and a half now. That word, "ubiquitous", is getting real mighty challenged. I have tried to insulate myself as best as I could and it just never happens and every other day I wonder if my foil hat needs adjusting. Does yours?
watwut 5 hours ago [-]
Previous bubbles were identified long before they popped too. It can easily go for the next two years.
adjejmxbdjdn 13 hours ago [-]
> Luckily, the AI bubble will destroy a decent chunk of their wealth.
Even if all this is true (personally I doubt it), it assumes the bubble won’t burst before they’ve shifted the money from all our 401ks, IRAs and index funds into their own pockets.
shimman 13 hours ago [-]
Gartner hype cycle isn't scientific and is mostly voodoo marketing gibberish, why do people still post it as if it's real?
talon8635 13 hours ago [-]
If the bubble bursts in the worst way it could, we are all fucked, but will recover eventually
Regardless, people have been making your prediction for a couple years now
azan_ 15 hours ago [-]
Once capabilities catch up with the demand (~which is in few years - it just takes time to build!) average person will have much better access to higher quality and cheaper hardware. It's literally how it has always worked, I don't understand why would it be different this time. There's zero substance in your comment, just repeating tired tropes from twitter.
asciimov 13 hours ago [-]
Companies have been trying to figure out how to get people to rent pcs for a while. Unfortunately the AI boom has shown them how, buy out all the capacity of the ram manufacturers, pay them enough consistently and the manufacturers have no reason to expand their capacity. No cheap ram means companies and then home users will have no choice but to come to you and rent their pc.
vrganj 15 hours ago [-]
A market requires scarcity, because scarcity is what neccesitates trade.
You can't invoke market mechanisms to defend a post-scarcity scenario.
Why would the ai owners make things cheaper or even available at all once their abundance machines give them everything they want?
What would they get out of if it? What could ordinary people trade for it that their abundance machine can't give them already and that would be worth giving up their monopoly on productive force?
You can't argue "this will change everything" and "this is how it's always worked" simultaneously, and yet AI accelerationists bring out both to defend their ideology.
sedan_baklazhan 8 hours ago [-]
Right. Capitalism is clearly incompatible with the proposed AI future where no jobs are needed.
azan_ 5 hours ago [-]
> You can't argue "this will change everything" and "this is how it's always worked" simultaneously, and yet AI accelerationists bring out both to defend their ideology.
Luckily I did not, you are arguing against points I did not make at all.
Either way though, what's more interesting than what you specifically said is that AI boosters have no answer to this structural argument about the reality of post-scarcity. Tellingly, you didn't address that part of my comment at all, either.
peddling-brink 15 hours ago [-]
> That's a feature, not a side effect.
You’re implying that the deep lizard state planned the LLM craze?
Sama might be a lizard..
vrganj 15 hours ago [-]
You don't need a back room filled with cigar smoke where supervillains coordinate their plans while twirling their mustaches when good ol' class interest does the same job.
smallmancontrov 15 hours ago [-]
r>g is an old hustle, LLMs just accelerate it. Lizards aren't wizards.
danielmarkbruce 15 hours ago [-]
when there was so little capital it had to be true... the idea that it will persist forever is obviously silly.
smallmancontrov 14 hours ago [-]
The problem isn't that it fails to unwind, the problem is what the unwind looks like. Last time it meant global upheaval and world wide capital incineration -- mostly metaphorical in the case of the US but very literal in the case of Europe and Asia. The poor suffer more in real terms but capital suffers more in nominal terms and that unwinds inequality and the burden of furnishing runaway capital with the runaway returns it demands in excess of economic growth.
I'm sure the same "strategy" will work again, but this time around everyone has nukes and drone swarms and AI-equipped bio labs so shit's gonna be lit.
danielmarkbruce 12 hours ago [-]
r dropping below g isn't "unwinding". r has been falling for decades now. The world is awash in capital.
smallmancontrov 11 hours ago [-]
r>g, not r<g. In 2025 GDP grew 2.1% and SPY grew 17.7%, which is the pattern recently and throughout time.
SwellJoe 15 hours ago [-]
s/deep lizard state/billionaires/
aleph_minus_one 16 hours ago [-]
> The reason problems like housing or famines or war aren't being solved is because structurally there's no incentives to solve them not because there's not enough geniuses out there.
Rather: these problems are not solved because there are not enough geniuses who are capable of convincing the masses to send the responsible politicians to hell.
blactuary 16 hours ago [-]
A great deal of the supposed smartest people have fully sided with fascism, it's not about smarts
majormajor 16 hours ago [-]
This is broadly the same people who bemoaned the idea that the "smartest people in a generation" were all just working on adtech without any reflection about maybe that's not actually an exclusive set of the smartest people... or that a lot of the people in that set were the same one who might, twenty years ago, otherwise be chasing money as middlemen in finance vs doing productivity-expanding things anyway...
You can be smart and chase the money, but it's crazy hubris to assume that only your peer set of money-chasers are smart.
It's a small leap from there to "since we're the only smart people, we should get to have the most influence."
dd8601fn 16 hours ago [-]
I’ve heard people say variations of, “Clever people can easily convince themselves of just about anything, if they’re inclined to. And then it’s nearly impossible to talk them out of it.”
Kinda seems true. And the worst people in history often don’t seem super dumb, despite their various other defects.
ptdcc 14 hours ago [-]
I'm confused by this comment, can you explain?
12 hours ago [-]
crooked-v 16 hours ago [-]
"Supposed" is doing a lot of work there. Compare, say, Yudkowsky et al. to the people who actually changed the world with mRNA vaccines.
jeremyjh 15 hours ago [-]
Yudkowsky is hardly an example of someone who has sided with the fascists.
techblueberry 16 hours ago [-]
In America anyways, we send the “responsible politicians” to hell every 2-6 years. People vote for this.
plasticchris 15 hours ago [-]
and yet... it feels like we are actually a one party state most of the time.
techblueberry 15 hours ago [-]
Maybe underneath it all we really do want the same thing…
komali2 14 hours ago [-]
More like the people with power all serve the mechanism of power: Capital.
Avicebron 15 hours ago [-]
The party of corporate interest?
mitxela 15 hours ago [-]
Duverger's law. Any elected body with FPP voting will converge to a one-party body.
versteegen 14 hours ago [-]
That is not what Duverger's law says. (But it's a interesting hypothesis so thanks for mentioning it.)
_DeadFred_ 12 hours ago [-]
Democrats expanded SNAP, Republicans shrank it. Dems expanded Medicaid, Republican shrank it. Dems protected reproductive rights, Republicans repealed them. Dems expanded title IX protection to LGBTQ+, Replicans eliminated them. Republicans jailed 1,200 January 6th insurrectionists, Republicans pardoned them. Democrats provide non-partisan federal emergency aid, Republicans withhold federal aid from blue states. Dems establish tax credits for electric cars, increase automotive standards. Dems didn't kill of foreign aid to need countries.
If you aren't female (reproductive rights), LGBTQ+, an immigrant now hunted by ice, someone in need of SNAP, Medicaid, or emergency aid in a blue state, it might feel like both are the same I guess. If you are one of the above, and now suffer because so many voted based on 'both are the same', you very much feel a difference today than from the US under Biden. If you were one of America's allies but are now constantly attacked, you feel a difference today. If you are Ukraine, you feel a difference today.
mannanj 14 hours ago [-]
been a while since ive see the real talk on hacker news. its refreshing. usually I feel like I'm the only one among nerds asking to hold people accountable.
so, how do you think we can do this?
aleph_minus_one 13 hours ago [-]
> so, how do you think we can do this?
For a lack of deep knowledge in this area, I don't think I have answers for this, but if one is interested in working on this problem, my approach would be to start with diving deeply into scientific literature about psychology and sociology to find out
- what convinces people to change their opinions,
- what causes people to take action
etc.
azan_ 15 hours ago [-]
> Rather: these problems are not solved because there are not enough geniuses who are capable of convincing the masses to send the responsible politicians to hell.
People support policies that make housing much less affordable (e.g. rent controls) and oppose policies that would make housing more affordable (e.g. zoning reforms). Saying that it's evil politicians that cause this crisis is dishonest - it's what people want.
swivelmaster 15 hours ago [-]
People by and large don’t understand the second and third order effects of their policy preferences. It’s not that they don’t want those things, it’s that the world is more complicated than their mental model insists it is. Politicians could try harder to fix the fundamental problems but they don’t because the long-term solutions don’t manifest until after the next N (very expensive) election cycles.
tancop 9 hours ago [-]
Rent controls work if you go radical enough and enforce a max profit margin for the whole city, with rent limited to something like 30% over maintenance + debt repayment costs.
The amount of housing in the city won't grow but it will free up a big part of peoples income that would otherwise go to landlords. That helps the local economy with more consumer spending, and makes it easier to save up or apply for a loan so you can start a business.
You get a better economy and more money in the average persons wallet without raising taxes. The only ones who lose are commercial landlords, and they will stay profitable by design so it's easy to shut down any complaints. And you get better quality apartments as a side effect because the only way to get more money out of existing properties is to spend more on fixing them.
azan_ 6 hours ago [-]
That policy would 1) fuck over younger generation, 2) result in total sclerosis and death of city. Prices are high because there's not enough housing, that's it. Rent control won't increase number of units so it wont solve real cause of the problem.
> The only ones who lose are commercial landlords, and they will stay profitable by design so it's easy to shut down any complaints.
No, also people who would like to move into city would lose, because that would become literally impossible. You'd have like 15 years waiting lists for housing.
> And you get better quality apartments as a side effect because the only way to get more money out of existing properties is to spend more on fixing them.
Why would anyone fix apartment that's under rent control? If you mean system like in Berlin, then I've got bad news - people there are actually frequently not allowed to renovate even THEIR OWN apartments.
aleph_minus_one 15 hours ago [-]
The politicians are those who created the respecting laws.
"Convincing the masses to send these politicians to hell" is thus nearly isomorphic to making the masses hate politicians who create such laws, which at least very likely implies that by convincing the masses to send these politicians to hell, a different mindset emerges in the masses.
azan_ 5 hours ago [-]
> The politicians are those who created the respecting laws.
Yes and general population LOVES these laws, even though they are harmful to them. But anyway - you've got your causality reversed - it's voting masses that incentivize rent controls and other shitty politics. Remember that to implement your policies you've got to actually win elections. And since people generally don't understand n-order effects for n>=2, it's really easy to win elections with populist policies.
mitxela 15 hours ago [-]
There is no evidence that reducing rent makes the rent go up - only fearmongering. It's a stopgap measure, though, since there still won't be enough housing to meet demand unless you also fix that at the same time.
vrganj 15 hours ago [-]
Which you can also do as a city by just... building! You don't need some developer to come in, you can just do that as a local government!
The City of Vienna is the largest landlord in Europe. It is also the only capital in Europe without a stifling rent or housing crisis.
azan_ 5 hours ago [-]
There is lots of evidence for that, but before I provide it I need to know if you will update your beliefs based on that.
mitxela 4 hours ago [-]
Really? You have evidence that smaller numbers are bigger numbers?
For that you might need the kind of geniuses who write stirring books about principles and values. Or whatever the modern equivalent of those books is. Maybe nobody would pay attention to that kind of advocacy in the present day, not sure why. But it's not tech geniuses, anyway. Much though tech has power as a catalyst.
I suppose there was Project 2025, and there was that book about the inevitability of Russian hegemony (or empire or something) that Putin liked. Those got attention with influential people. Not sure what to conclude from that.
komali2 15 hours ago [-]
> because there are not enough geniuses who are capable of convincing the masses to send the responsible politicians to hell
I could talk about this for hours, and was last night actually.
What do vanguardist communism, feudalism, liberal democracy, and fascism have in common? A shared belief that the general population is composed of a bunch of dumb dumbs that need super smart (read: politically educated) people to "lead" them.
Seems subjective. When is the smartness, big smart enough to allow someone to enter the hallowed ranks of "actually a real human allowed to speak for the masses?"
There does seem to be a clear delineation between the people you hear about (the vanguard in our example), and the people you don't hear about (the masses): willingness to create confrontation. Most people want to just go with the flow, live in peace. In a dichotomy I've been calling "choose peace or violence," they choose peace. They choose to see bad things in the world or their life and focus on managing their happiness around it.
I think that this is the default human mode of existence, and that this mode lacks defenses against people willing to violate the default peace. Imagine a bagel line, and someone cutting that line, and when tapped on the shoulder, the cutter breaking into a huge tantrum, and people in line backing down with a "Jesus! It's not worth it, fine." Thus some people learn that they can be much more selfish in life if they're willing to create confrontation and disturb the peace.
When I talk to people about agitation, civil disobedience, or even just unionizing, the most common thing I hear is anxiety. What if the situation gets worse, what if there's reprisals, what if I lose this little bit of comfort that I've managed to be happy with? That slice could get smaller and smaller but people aren't willing to take the step to say "enough is enough" because I think it fundamentally goes against human nature, we have to sacrifice our own happiness on some level to put our foot down and demand or make change. At minimum we have to acknowledge the discomfort or injustice, which is a diversion from our constructed internal peace.
Personally I think people should be allowed to live in peace. Maybe the liberal democrats and vanguardist communists are right, maybe most people shouldn't need to trouble themselves with the greater direction of their society. However I think the stability this promises is false; in fact I think all stability is false. I think this constructed peace that people are afraid to disrupt clearly can be torn apart at any moment, not just by people willing to create conflict, but by a literal earthquake, or fire, or plague. Therefore I think there's a couple societies in history that got it right: ones who distributed the responsibility of society across all individuals in the society; namely the brief period of syndicalized Spain, and the several hundred thousand years of certain tribes of native Americans on the eastern portion of the continent. There may be other societies in history, these are just the cases I've been able to read about so far.
It seems to me that it's possible and necessary for all people to be ready to wake out of their constructed peace long enough to make adjustments together to maintain it. I don't think this is the same thing as them being "led out of it," I think this is each individual being on some level alert to the need to spontaneously say "no, here we wait in line for our bagels. No, screaming about it won't help you, get to the back of the line. Look around: everyone is laughing at how silly you look."
watwut 4 hours ago [-]
> What do vanguardist communism, feudalism, liberal democracy, and fascism have in common? A shared belief that the general population is composed of a bunch of dumb dumbs that need super smart (read: politically educated) people to "lead" them.
That was not premise of fascism. Certainly not Hitlers one, he himself was self-taught and did not framed himself as highly educated. He considered himself better then educated people. The origin of his superiority was not "smart" it was "strong will and power itself".
It was not the original premise of feudalism either. That was was about military might, basically.
And origins of liberal democracy are not "smart based" either.
skew-aberration 16 hours ago [-]
Yes, many of the problems are 'features not bugs'. There's no reason to think the AIs won't dream up more of these'features' too.
YZF 14 hours ago [-]
What I like to call at work "this is not a technical problem". There are many problems we could easily solve if we had the right structural incentives. But instead we have politics, processes, layers of management, etc. All that doesn't disappear (yet?) with AI.
talon8635 13 hours ago [-]
I believe that’s actually one of the arguments of the boosters. Utopia will finally be reachable when we get all the human drama and bureaucracy out of the way
I’m not sold, but I’ve heard this line of thinking many times
pishpash 13 hours ago [-]
Human drama is what people use to advocate for themselves. Bureaucracy is what people use to protect themselves. Then you have the side effects but, you may also not like a world without those at all.
mbesto 15 hours ago [-]
> The reason problems like housing or famines or war aren't being solved is because structurally there's no incentives to solve them not because there's not enough geniuses out there.
When I asked why Musk doesn't put his money towards non-profits my Musk supporting friends say it's because he "doesn't believe anything but for-profit entities can solve the world's problems".
So why isn't he solving homelessness, famines, war, etc. with capitalism?
Terr_ 16 hours ago [-]
To add insult to injury, we're being told: "Just trust us bro, however we make things worse now will totes be redeemed as the prophesied god-mind magically fixes it later."
Strangely, they never discuss the possibility that Multivac might seize all their wealth and cast them down into average-ness. What if it actually comes about, but then it tells us to do the things we already refuse to do?
toppa33 13 hours ago [-]
Agree. I’d only add that I think there is incentive for them to be solved, but there are greater incentives worth more to those leading the charge.
robswc 16 hours ago [-]
This is sort of my take. You could look around in 2016 and 2026 and honestly not much has changed... at least at first glance. There is a missing piece of the puzzle.
I'm noticing this even in industry. Yea, we can build and iterate on ideas 100x faster but has this turned into anything meaningful? Not really... Hell, look at the big companies spending billions on inference... nothing shipped and things are still buggy as ever. Where is everything?
Rapzid 16 hours ago [-]
The job market had changed that's for sure.
robswc 13 hours ago [-]
Fair point. I have to wonder how much is because of AI and how much is from ZIRP stuff though.
assdaa 13 hours ago [-]
Neither.
How many more software projects are there really left that make economic sense? Zuck learned the hard way re. metaverse.
Not many - the set that make ECONOMIC sense is finite. Even though many want to believe its infinite because all of a sudden everyone can vibe code an app cringe.
If anything AI is going to cause an insane amount of consolidation - what used to be across 3 diff apps, that can be consolidated will be done -> many businesses will die. It will be painful.
Google was attacked by an activist investor pre LLM hype for being bloated - they promptly laid people off. Many software firms have been allowed to bloat up do to the nature of the business.
Its always about economics.
robswc 11 hours ago [-]
Not sure I can agree, but the idea is interesting.
While the set is finite, software comes and goes. Just look at the tools to build software over the years.
> Many software firms have been allowed to bloat up do to the nature of the business.
I can agree with this. It will be harder to justify engineers sitting on laurels. Google could afford it before but I can see how it will be under more scrutiny now.
assdaa 10 hours ago [-]
I know I’m right as I’m doing the very thing stated - taking 4 related businesses and consolidating - leaving behind many mortally wounded/dead.
You either get it or you don’t - not my problem.
zer00eyz 16 hours ago [-]
We went from spending a lot of money on people to enshitify products and then charge more for them to spending a lot of money on agents to enshitify products to charge more for them.
neumann 14 hours ago [-]
And stall and sell your data quicker. It will certainly be a boon for companies and government agencies that want to profile.
vrganj 14 hours ago [-]
As a person, I have a strong preference for the former.
aorloff 15 hours ago [-]
That's what I'm saying.
Not only from the big companies. Where is the explosion of disruption ?
gWPVhyxPHqvk 14 hours ago [-]
Why even bother when an AI is just going to do it better than you can? They have more money, more compute, and most importantly, will have next to zero cost of labor.
robswc 13 hours ago [-]
I can't even go a day without running into super frustrating bugs on the ChatGPT app.
AlexCoventry 9 hours ago [-]
Like what?
aorloff 12 hours ago [-]
You could say the same about Amazon
watwut 4 hours ago [-]
Fck disruption. The fetishizing of disruption is annoying. The tech became all about destroying something someone else has. What about ... create good stuff, create non-gambling markets, make peoples lives better.
redwall_hp 15 hours ago [-]
Sure, it did some meaningful things:
* Higher load on senior engineers who are about to burn out hard from fighting an endless battle against unreviewed slop people keep trying to yeet into production
* Less junior hiring and suppressed wages
* Culture of uncertainty and fear
* Computers are really expensive now
* Some people now live with diesel generators and gas turbines making their neighborhood hazardous to their health
robswc 13 hours ago [-]
Totally agree. IMO it's mostly behind the scenes stuff though.
I think some of the most impressive stuff to show someone from 2016 might be LLMs doing a summary of complex topics, followed by self-driving (actually) cars and intricate robots... but other than that there is no "future magic" touching the lives of every day people going on.
Actually I'm pretty sure if you showed someone the latest and greatest from these big AI companies, these AI companions that can book flights, people would go "Oh, so its like a smarter Siri" and proceed to not care at all lol.
brody_hamer 14 hours ago [-]
> Higher load on senior engineers who are about to burn out hard from fighting an endless battle against unreviewed slop people keep trying to yeet into production
Gosh, this has been my entire experience with AI, and I’ve never seen it this well articulated. Thank you!
assdaa 12 hours ago [-]
Less junior hiring is a reflection of a) the maturity of the software business b) current expectations of the future
I have no doubt startups in particular are able to get away with using LLMs to produce sloppy code and ship. But this trend was on-going and sped up due to the infiltration of LLMs and marketing.
You do realise markets are highly imperfect in reality? That nonsense in econ books is fantasy.
vuurmot 11 hours ago [-]
It has greatly changed for the worse
I can't get all the flashing lights out of my face now, more algorithms to hook you in, more social media, more gambling, more news, more things to catch your attention.
More noise. The world is just so noisy now.
chubot 14 hours ago [-]
Even in software, there are many things that haven't changed.
e.g. Why do Rust and CPython still have over 5,000 open issues each? At some point, will the superintelligence fix all the bugs autonomously? When is that going to happen?
What about new features? In every thread about Rust, people are clamoring for new features and libraries. People wonder why there can't be a batteries-included standard library like Go.
So now that LLMs are so smart, is that a solved problem? Or are we still bottlenecked on the expertise of a few experts?
---
Also, what happened to AI's passing the Turing test? I see tons of slop articles that are obvious to most HN readers. They haven't fooled us apparently -- the longer the interaction, the more obvious it becomes whether it's AI.
(That said, I use AI all the time and find it to be a great tool. IMO is is a tool, not an autonomous intelligence with a will of its own.)
prometheus1992 16 hours ago [-]
>You could look around in 2016 and 2026 and honestly not much has changed
I find 2026 much more repulsive compared to 2016.
marsven_422 10 hours ago [-]
[dead]
TheAceOfHearts 16 hours ago [-]
I'm immensely sad every day seeing all these posts talking about billions and trillions of dollars and where I live (part of the US btw) we don't even have fully reliable electricity and water access. In fact our electricity bills have increased and there is nothing to show for it.
I don't understand why these big labs aren't doing more to improve the electricity situation. Dump a billion dollars worth of tokens into fusion analysis, research, and simulations and try to force a breakthrough. You could buy a lot of good will with unlimited cheap energy that liberates us from fossil fuel dependencies.
ungovernableCat 16 hours ago [-]
Mathematics and programming is verifiable and fully controllable in a digital environment.
Fusion research and cancer research requires expensive experimentation in the real physical world. You can’t just throw a billion dollar worth of tokens to handle the problem.
Look at all the roadblocks for data centre buildouts which the industry certainly has a financial incentive to complete as fast as possible.
nekusar 15 hours ago [-]
> Fusion research and cancer research requires expensive experimentation in the real physical world. You can’t just throw a billion dollar worth of tokens to handle the problem.
When the tokens are hooked up to physical devices (robotics of various sorts), then the tokens pay for themselves by being physical workers, or human wage replacement.
And a robocop will follow whatever is passed for law, damn the masses who might vote otherwise. And subjugation will be enforced by token, LLM rule, and machine. They have no union or ethics to complain to.
assdaa 13 hours ago [-]
Who taught you economics?
ronsor 16 hours ago [-]
OpenAI and Sam Altman are literally backing a nuclear fusion power startup.
undersuit 12 hours ago [-]
Because they increased the demand so much they are seeking out their own supply. Nuclear has consistently been panned for decades because it's more expensive than other power generation.
blactuary 16 hours ago [-]
"Startup"
Wait until you have clean energy sources and shut it down
Lerc 14 hours ago [-]
Quoting startup like that makes the effort seem like it is an extremely recent venture.
It is 13 years old.
It has been making progress, the degree and date of success is debated and confidently claimed to be in many mutually exclusive states, which at the very least says that there are lots of people confidently making claims that are not true. I'm not going to join that fray to declare which of them is correct.
blactuary 14 hours ago [-]
Is it ready to provide power? No? Then I don't care what they are trying to do while they are burning a bunch of fossil fuels in the meantime
Lerc 12 hours ago [-]
Are you avoiding all use of fossil fuel while alternatives become available?
It is a bold claim to make but not something that usualy be done. Any abrupt halting of fossil fuel would severely impare the ability to develop and produce alternatives.
We need to transition to sustainable methods. If you want to go cold turkey, go ahead and cease driving an ICE, never travell by air or have anything delivered (and that's just the tip of the iceberg), it would be a counterproductive way to reach sustainability, that is your right, but you can decide to do that to yourself. You don't get to decide how others reach sustainability.
There is a difference between advocating for a better world and acting like a bully because you don't like the current world.
talon8635 13 hours ago [-]
Electricity infra is not a technical problem. It’s been solved for decades. It’s a political problem, and both sides of the isle have had plenty to do with getting in the way
mattnewton 16 hours ago [-]
I think this is related to the structures we’ve built in society. It’s lower friction to write software than just about any of the problems you listed.
It reminds me of the Ezra Klein quote that “OpenAl would need permits to cover its parking lot in solar panels, but it can accelerate into recursive self-improvement, as best I can tell, whenever it so chooses."
asdff 15 hours ago [-]
A lot of it isn't low friction really just in terms of political will it is high friction. E.g. housing issues are due to failing to upzone to meet demand appropriately. Solving housing is therefore free. It could be done today in any city in a housing crisis with the stroke of a pen, and before long, the market will equilibriate at a level better afforded by the median wage earner in the economy. The decision not to do this is a deliberate one: prices increasing is very much the goal of this sort of policymaking.
Likewise when you see other such shortcomings in our world, ask what this incentivizes for otherwise. Then things will become obvious why they are how they are, from housing, to transit, to feeding people, to healthcare.
The priority is profit, not people.
numbasys 16 hours ago [-]
Soon:
The super-intelligent AI is telling us to cover parking lots and other flat roofs in solar panels.
ApolloFortyNine 15 hours ago [-]
What is up with this attitude in this thread? AI has taken off programming basically this year, the end of last at best. The first truly capable LLM, ChatGPT, isn't even 4 years old yet.
Just a crazy amount of doom and gloom all around this thread.
versteegen 14 hours ago [-]
It's kind of weird to say it's "taken off" recently. Yes what we have today is far better than half a year ago and solving new real use cases, but that was far better with more uses than we had half a year before that, etc, for years now. On their best usecases LLMs were already enormously useful years ago, and it's still very spiky.
id00 14 hours ago [-]
As a user of software I see no improvements. What I see is that I need to download software more often, it brings more features I don't need and it has been as buggy as ever (if not more). And I can say that about Operating System I use, social networks, games, etc.
ryandrake 13 hours ago [-]
Exactly. Where are the results? If AI is really producing value at 20X the rate software engineers produced it earlier, we should be seeing 20X the number of software projects released now. Or they should be getting made 20X faster. Or they should be 20X better. Or the projects should have 20X fewer bugs. But none of this seems to be happening. The only thing that seems to be happening is we're making the same crappy shit, but with 20X fewer people.
nonethewiser 13 hours ago [-]
It took a long time for internet to transform society. Its still not fully baked in. These models that we have right now are transformational.
vrganj 15 hours ago [-]
In other words, it's taken a highly enjoyable and lucrative vocation from vast swathes of workers - a lot of which make up this sites audience! - and instead concentrated more productive power in the hands of a few megacorps.
You gave your own example for the doom and gloom.
Waterluvian 14 hours ago [-]
It has made it far more accessible to a lot more people. Being “highly enjoyable” and “lucrative” is why people try to gatekeep it.
The fact that the means of this capability is in a few hands does suck. But I’m increasingly optimistic that it’s going to be commoditized through open weight models and performance improvements. There is enormous demand for innovations on these fronts.
vrganj 7 hours ago [-]
It hasn't made it more accessible to anyone. It's destroyed it. Who still writes code?
Waterluvian 1 hours ago [-]
You’ve got to understand that coding isn’t the point for a lot of people. It’s just the thing in the way of making stuff and solving problems.
vrganj 1 hours ago [-]
I do understand that.
That's my point, in fact. We're losing the profession just so the annoying guy that would pester you with his stupid app idea can now shit it out himself, lighting the planet on fire in the process.
teaearlgraycold 14 hours ago [-]
I'm a rare individual that has consistently made choices to earn less money in order to have more enjoyment. I would clock fewer hours than they wanted me to during college for my internships, I left a FAANG to go back to startups, I quit a startup to work on volunteer work and try to bootstrap a business.
At multiple jobs I've noted that I felt the pay should have been less. Not so that the employers could take a bigger slice of the pie (in fact, my preference is to be in an anarchist co-op, but I understand that's a wild take and might be terrible in practice). My hope was that we could filter for people that genuinely want to do the work and take on fewer people that were there for the money. At the same time more money is more better. And I like having money. It's complicated.
As I've said many times before, I didn't come to Silicon Valley to get paid. I came here to make computers do cool things alongside other people. I need income to do that, but I try to make sure I'm in a situation where the income isn't something I need to think about all of the time. If you're doing mercenary work that's not possible.
Waterluvian 13 hours ago [-]
I think being able to accomplish more from places that are a fraction the cost of Silicon Valley will also help achieve the goal of doing things for reasons other than the money.
I think the amount of protectionism we’ll see could be absolutely enormous. Not only do incumbent corporations not want to lose share, incumbent countries won’t want to share either.
teaearlgraycold 13 hours ago [-]
Definitely. For now I'm making friends with lots of very capable people - it's not as easy to do that outside of a hub like SF.
> I think the amount of protectionism we’ll see could be absolutely enormous because corporations aren’t capable of altruistic attitudes.
On some fronts they will be powerless to stop the changes. I look forward to drinking the tears of IP owners when breaking through their DRM and recovering their source code is just a matter of sharing a model ID, prompt, and hyper parameters. Reverse engineer NVidia's firmware and drivers. Decompile and port Nintendo's latest console's operating system. We may soon find ourselves in a world where tasks like that are achievable in a few days on consumer hardware.
aaron695 14 hours ago [-]
[dead]
emdash 16 hours ago [-]
That pretty much sums up my experience. Everything is more expensive and worse at the same time. And it feels like we're being screwed over from every direction and in new ways every day. People keep talking about this tech is supposed to do so much but everything just gets worse.
SamPatt 16 hours ago [-]
What time scales are you using to make this judgement?
Everything isn't getting worse constantly. The data is quite clearly the opposite. How old are you?
vinyl7 15 hours ago [-]
As a middle aged man, thing are indeed getting worse every year.
jasondigitized 13 hours ago [-]
From another middle aged man, part of that perception is your brain changing and also nostalgia.
fasterik 15 hours ago [-]
It's not true that everything is more expensive. Nominal prices are higher, but incomes have risen faster than inflation. Purchasing power of the average person in 2026 is higher than it was in 2016.
As for things getting worse, I guess it depends on the domain. In hardware, things have gotten a lot better; my desktop computer is orders of magnitude more powerful than I would have expected 10 or 20 years ago. In software, I do see a lot of bloated software being written, but I don't feel that the software that I personally use has gotten worse.
mplanchard 13 hours ago [-]
Average, or median? Don’t know the answer offhand, but there are a small number of well-known people who are likely to skew the average quite a bit upward, given how much wealth they accumulated during and since covid
fasterik 12 hours ago [-]
I was using "average person" in the informal sense. Median household income in the U.S., adjusted for inflation, has gone up 13% in the past decade.
satvikpendem 16 hours ago [-]
This is a Western-centric viewpoint. I'm in Asia visiting a few different countries right now, including the big ones, and much has changed with AI and also in general over the past 10 years with cars, trains etc.
> AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Strange viewpoint. In mathematics any novel result is useful as it expands the scope of other results. Who are these mathematicians who didn't want AI to solve the problems it did? I see trepidation about the future of the field but that is not equivalent to not wanting them to be solved.
SyneRyder 15 hours ago [-]
Terence Tao and the apparently newly formed Association For Human Mathematics. The post is online-famous now for the line "Mathematicians did not ask for this work to be done."
When I was a young man reading gauss, and thinking before I turned the page, I see a result here, something meaningful! and then I turn the page, and of course it is there. There is no shame in this.
A long time ago, another student of mathematics described it to me as climbing a mountain. No single step is hard, but you must make many steps to get to the point that no one else has made a step.
AnonymousPlanet 16 hours ago [-]
I'm curious, care to name a few things that have changed in Asia with AI (and wouldn't have without)?
vuurmot 11 hours ago [-]
I've watched some videos from Chinese commentary on AI usage in the industry,
Some examples are finance people directly asking for API access now so that they can make their own tools. These tools used to be maintained by teams of developers.
Many business making their own customized scripts to make their tasks faster.
From an economic point of view, it seems like there are some productivity increases, but I don't think these can be concluded for a while.
The rational thinking rarely account for second order effects.
Although business leaders rarely care about second order effects, a worse business is not immediately noticeable, compared to hiring cuts and the money it saves
byzantinegene 12 hours ago [-]
asian vertical dramas (lol)
tantalor 12 hours ago [-]
> much has changed
Trust me bro
satvikpendem 10 hours ago [-]
No need to trust me, just look at how China was 10 or 15 years ago to now. Beijing air pollution, more high speed rail than the rest of the world combined, electrification of vehicles and economy, etc.
tantalor 2 hours ago [-]
^ AI did this
satvikpendem 2 hours ago [-]
Partially, yes.
m0llusk 15 hours ago [-]
That is not how math works. There are reasons why the Riemann integral is pervasive and the Labague integral is rarely used unless needed. Just because you can do something does not mean that you should.
satvikpendem 14 hours ago [-]
> Just because you can do something does not mean that you should.
Again, strange viewpoint in the field of mathematics of all places, unlike, say, economics where doing something one shouldn't has disastrous real world consequences. Until it's discovered and proven, one doesn't know what would be valuable and what would not. I bet your sentence would not be said about humans trying to solve problems, no one would tell them they "shouldn't."
spamizbad 12 hours ago [-]
I think people in the LLM space are being far too impatient with the general public and their adoption. And I think they need to start thinking harder about building products that genuinely enrich people's lives. Look at how long home computing took to establish itself with the public, then the internet, and finally social media. None of these trends happened overnight, and some were accelerated by other technological advances (social media-> smartphones).
The adoption of these technologies spurned ton of innovation in these spaces, and I feel like with LLMs its the first time I've seen genuine frustration from the industry directed towards users for not adopting the technology fast enough. This attitude did not exist in the 1980s, it didn't exist in the 90s, and it didn't exist in the 00s/early 10s either.
dawnerd 11 hours ago [-]
Do you blame them for getting impatient? They’re burning cash and quickly running to a point of never being able to justify it. The hardware they buy will be replaced with even more expensive hardware before they start to make a profit. It’s not sustainable and the market hasn’t figured that out yet. At least they’re starting to question it.
assdaa 10 hours ago [-]
These backward looking analogies are tiresome.
Whether you like it or not - technology causes compression of life cycles. You have a hit product? Great - I expect you to have crazy high growth rate. That’s the expectation of a world that is so deeply interconnected.
spamizbad 10 hours ago [-]
Consumer-ready LLMs have been widely available to the public now nearly 4 years if you start counting at GPT 3.5. There's certainly some compression here but Karpathy does a good job of outlining how silo'd those experiences are: the bulk of the product penetration (based on headcount rather than token usage) is "derpy chatbots" (his words, not mine). Most of the actual token spend is likely done on the STEM side. We're talking about a technology today that currently only has significant penetration with <25M users.
People need to be patient and put in the work to build great products, rather than lament its slower-than-desired (albeit still quite fast) adoption curve.
kozikow 16 hours ago [-]
At least for me, personal agents have been quite useful. If you integrate them well - shopping, filling random government forms, keeping track of my personal life, optimizing training plan and nutrition, planning vacations, etc. I can do more home repairs or renovations myself with help of chat
So basically you get personal digital assistant - very useful - but not changing my life.
What would I need for real transformation though? Food, house cleaning, commute - "every day chores". Even if we had ASI tomorrow, what could it do to change my life, besides taking away my job? Design good house robot to help me with food preparation I guess... Maybe finally self driving cars launched in my area? Or it will take control, "optimize humanity survival chance" and realize we don't need as many people on this planet.
gWPVhyxPHqvk 14 hours ago [-]
> besides taking away my job
> self driving cars launched in my area
It sounds like the transformation would be a massive collapse of the economy with scores of people unemployed.
> Or it will take control, "optimize humanity survival chance" and realize we don't need as many people on this planet.
Maybe, but more likely it will be the humans who control the AIs and hold the power who will make this decision.
jsrozner 12 hours ago [-]
Yes, and additionally, on the other side, there are a bunch of people using it to build exploitative products, spam, weapons, etc.
ptx 16 hours ago [-]
Almost every time I use an AI chatbot it eventually turns out it's lying to me. I would not trust them with any of the things you mention.
Aurornis 16 hours ago [-]
This is the kind of gap in shared understanding that is widening.
When people use words like “chatbot” to talk about their AI use and can’t really name basic things like the model they were using, it reveals a wide chasm between the people who are deep into understanding how to use these things and those who poke at easy options every few months and then look for ways to confirm their priors instead of trying to understand how other people are getting value.
majormajor 16 hours ago [-]
The incentives seem to be lined up to push users to the frontier of "how much can the harness around the model do and claim without most people pushing back on the claims."
The bigger the pile of stuff you have the tool create unsupervised, the more incomplete/inaccurate the overall picture of it is gonna be. That doesn't matter for most hobbyist/helper-tool/game-port/etc type of tasks.
I get a ton of value out of the tools and yet part of that value has come from being hyper-aware of the shape of the limitations, which changes much less and more slowly than the specific of the limitations.
"How'd you notice that one!?" - well, it helps to be the only person on the team who took the time to interrogate the e2e interaction of all the code...
The median piece of OSS software was already pretty rough once you had to depend on it in production at scale to make money. The median piece of the new AI-generated wave of software, also pretty rough.
The bet that OpenAI/Anthropic is making is that they can automate the interactions generally/globally enough to work around this and eliminate the need for humans in the loop. But it's a tricky problem because it's precisely where the permutations and special cases get nasty and much harder to deal with than more broadly-applicable cases of codegen/test-gen/mash-it-until-the-tests-pass-and-the-code-review-angent-is-happy.
unclebucknasty 14 hours ago [-]
This has all started to feel very Emperor's New Clothes-ish. "Well, if it's not working for you, then it's your fault. You're just not using it properly. You're not worthy of its wonders".
Not to say there's no value to be had. However, it's also been oversold and its error-proneness is a real problem.
But, it seems there's a steady stream of commenters and creators who are quick to remind us that it's never a problem with the tech. It's always a user problem.
Meanwhile, the errors are renamed "hallucinations", but let's not focus on those. In fact, if you just look right over here at this .md file...
satvikpendem 10 hours ago [-]
It's like driving a PT Cruiser and concluding all cars are bad and swearing off them entirely.
Madmallard 16 hours ago [-]
using skills with opus 5.5 and agents for each individual task that have shared knowledge of all of your records does not lead to a significant increase in trustworthiness
skydhash 12 hours ago [-]
> When people use words like “chatbot” to talk about their AI use and can’t really name basic things like the model they were using, it reveals a wide chasm between the people who are deep into understanding how to use these things and those who poke at easy options every few months and then look for ways to confirm their priors instead of trying to understand how other people are getting value.
While the latest and newest should be needed for greater capability, the average should be enough to give a glimpse pf what is possible. Like a basic netbook is enough to enjoy computing, no need to purchase the latest supercomputer.
If you need to switch your setup every quarter or so, it’s not a tool, it’s a rat race.
satvikpendem 16 hours ago [-]
I have ChatGPT Pro and it sources every single thing. For my Asia trip right now it's been helping me every single day to figure out optimized routes on what to do and where to go (if I feel like it, not to say I don't wander around too).
nfRfqX5n 13 hours ago [-]
me too, but I recently blew the transformer on my boiler by following its directions to wire up a C wire to power a smart thermostat. oops. $500 mistake.
wlesieutre 14 hours ago [-]
How often do you check the citations? I’ve found with the Google search AI summary it will provide answers and link to their citations, but a good chunk of the time the answer isn’t on the linked page.
It just makes stuff up (or gets it from somewhere, who knows) and then links to a tangentially related page as if it was the source. Either that or it identifies the page as being a source it wants to use, and then summarizes it incorrectly. Not sure the order of operations it’s doing internally, but regardless the citations are bullshit.
throwaway093020 13 hours ago [-]
The citations provided by the free version of Gemini are comically bad. They almost never contain information which directly supports the relevant claim, and more often than not they're completely irrelevant.
satvikpendem 14 hours ago [-]
Google's models are worse that OpenAI's, doubly so on Google's free AI overview versus ChatGPT paid plans. 6.1 Sol is very good at synthesizing actual information it finds without hallucinating and I've not seen any instances it failed while I checked the sources.
blactuary 16 hours ago [-]
That sounds terrible. I like to live like a human being
ChickeNES 15 hours ago [-]
What that does that even mean? Even if I grant your premise (I don't), if one is traveling and engaging these places in person....what's the problem?
blactuary 14 hours ago [-]
Where did I say it was a problem? Using chatgpt to plan an itinerary sounds terrible TO ME
vuurmot 6 hours ago [-]
Indeed, planning an itinerary, making calculations, building excitement for a place, is part of the journey
I suspect the HN techno-brained crowd will not appreciate it, many are quick to throw away their human experience
satvikpendem 2 hours ago [-]
> Indeed, planning an itinerary, making calculations, building excitement for a place, is part of the journey
Not sure how they're perceiving using ChatGPT to plan an itinerary but I'm also doing all this while using ChatGPT. As I said in the sibling comment, it's a faster Google search and route optimizer, not something where I type in "make me an itinerary for X city" and follow whatever it shits out.
satvikpendem 14 hours ago [-]
Why? In the past I would've used Google so it's just a faster search engine for me that works with natural language, what do you use for planning then?
kolinko 16 hours ago [-]
Which model? Super weird
qwerpy 15 hours ago [-]
Every time I use Google search AI for anything more complicated than a search query I feel that way. A paid model from pretty much anyone is much better.
kolinko 4 hours ago [-]
Yeah, I’d hope that on HN, of all places, people know that free models optimize for price and speed, not for accuracy. But even with free models the accuracy was massively improved over the last year.
heaney-555 16 hours ago [-]
When is the last time you used an AI chatbot, and do you only use the free versions?
kozikow 16 hours ago [-]
> Almost every time I use an AI chatbot it eventually turns out it's lying to me.
I would not trust them with any of the things you mention.
If you talk to free chatgpt, sure.
But if you launch Opus 5.5 in claude code - for some reason Opus is the smartest for me, even for "general" type of questions, when launched from Claude Code. It has harness/prompts optimized for correctness, while more "customer-grade" products, even when used with top model tend to optimize for latency. And I tend to give "context" - e.g. I have "personal wiki" + skills for lots of those things.
I don't recall i last few weeks where Opus 5.5 in CC made an obvious lie, made an unreasonable assumption, or made a mistake that smart human couldn't make. E.g. if it "failed" it failed in a way that non-genius human could fail in similar way.
bee_rider 14 hours ago [-]
It’s sort of weird how
1) open weights and cheap models are only about 6 months behind the frontier in evals, but…
2) it’s always the premium model that only the biggest fans pay for, that’s just now finally achieved enlightenment.
Oh well, I guess we’ll see in another 6 months.
Madmallard 16 hours ago [-]
opus 5.5 uses agents and skills in chat sessions as well and maintains internal context that it tries to distill from what you provide
I tried to have it write a synopsis website for the different types of dance patterns in Dance Dance Revolution, with guides on their execution, explanations, and provided song sources. I provided all of the source articles and search websites needed for it to be able to do that, and gave it a sample set I wanted it to utilize and build off of.
It has bits and pieces of information wrong in 90% of the sections it created. Poor song choices, missing song choices, incorrect explanations, incorrect step-patterns, the list goes on.
If you're not an expert in a given field for which you're using it, it's going to look like a genius. If you are, it's untrustworthy and borderline useless.
Maybe it wouldn't be if there were ample training data on my topic, but there isn't.
It utilized the research from millions of papers many very close to the destination answer to make those proofs. What would it do on something that isn't supported like that? Be confidently wrong while sounding right. That's what we've seen repeatedly.
Terr_ 15 hours ago [-]
I think my anxieties would be different if I were confident a DDR guide full of lies would be spurned and unprofitable.
In a way, I'm not worried about the LLMs per se as much as a kind of moral and ethical collapse. A debasement of the currency of communication, a market of intellectual-lemons.
mitxela 15 hours ago [-]
I saw electroboom's latest video lamenting facebook AI slop about electricity, such as a way to wire a three-way switch that blows a fuse when you turn the light on. For some reason, I am shadowbanned on HN.
Terr_ 15 hours ago [-]
I see your comment, perhaps it was a delay between representations or nodes?
rizzom5000 15 hours ago [-]
> If you're not an expert in a given field for which you're using it, it's going to look like a genius.
For someone highly literate, llms aren't going to help you write better. If you're borderline functionally illiterate (this is > 50% of the population according to some studies), I'm sure the llms are awesome.
I've long thought the same about all the hype around coding (plenty of examples from years ago now), but will continue to test the new models. They are improving.
gamblor956 15 hours ago [-]
Personal agents are wonderful! You can ask for hiking trails and it will customize one for you, frequently resulting in emergency calls for rescues! Your purchasing/selling AI will arrange sales for you but forget to tell you, leaving you in breach of contracts you didn't know about! Your personal nutrition agent will absorb all the broscience and customize a diet that will result in hair loss, but at least your in ketosis, amiright! And instead of hiring a plumber/electrician/contractor to do work subject to code, you can just ask an AI to tell you how to do it and then wonder why your buyer is suing you for $$$!
(Those are all examples from just the past 4 days, and just the ones that made it into mainstream media.)
grtt1 16 hours ago [-]
[dead]
denkmoon 16 hours ago [-]
Using AI for everything digital, I end up going and doing chores while waiting for the LLM to do its thing. What an insane world, where knowledge work (dreary and uninspired as it is) is automated while I am consigned to scrubbing dishes and waiting for the machine.
azan_ 15 hours ago [-]
Buy a dishwasher, you won't regret it!
asdff 15 hours ago [-]
Honestly the big reason why I use my dishwasher is that all the dishes would not fit in my drying rack. I'm not very convinced the process of me rinsing and scraping my pans before they go in the dishwasher is any faster than when I lived in apartments when I handwashed everything. Maybe my dishwasher sucks too, but I mean it doesn't hold a candle to the sort of jobs on my pans that require good old fashion baking soda paste and elbow grease. I'm still slaving with this thing. It's picking the low hanging fruit for me in other words.
I feel similar about my robot vacuum. It actually takes a surprising amount of labor facilitating the damn thing. I have to walk through the house and basically vacuum proof it so it doesn't eat various cords, rugs, lifting chairs up and putting them someplace so it can get under, etc. Not sure its really a labor save to when I was just brooming the entire apartment in 20 mins and not having to move anything around due to reach of the broom and it not having a tendency to eat things. Hand vacuum I can be more carefull too about the fringed edges of rugs and don't have to fold them over themselves.
I still do it because I already bought it mostly.
kuschku 15 hours ago [-]
The trick is dialing the things in properly once, and then reaping the benefits over and over again.
e.g., my home is always vacuum-proofed. It automatically runs through all rooms except for the bedroom at night, finishing just before my alarm clock rings, then vacuuming the bedroom an hour after the last person's alarm clock has rung.
This way I never actually notice it, but the home is always perfectly clean as if by magic.
Similarly, if you use separate dishwasher detergent, antisurfactant and salt, and properly dial in your water hardness, it'll be able to clean everything no matter how dirty the dishes are. Including dried-on tomato sauce. Just put the dishes in, and they'll be clean the next day.
Of course it takes a bit of effort to set this up, but then it just works for a long time.
Similarly, I'm a big fan of handwriting software with care & focus on quality, which takes a bit longer at first, but then allows you to trust the software to continue running for years to come.
asdff 14 hours ago [-]
I've done a lot of vacuum proofing, even getting the magnetic strips, but there's only so much you can do. Like it doing a little wheel around a chair leg is just not as great of a clean as if I moved that chair out of the way. My rugs have fringe so there's no stopping that. Cables I tend to have out have to go away.
It's definitely easier in a minimalist & tidy home. Our rugs have no fringes, cables are either rolled up or permanently installed & hidden, chairs are stored at night upside-down on the tables they belong to.
Sure, it's not perfect, but it's still really convenient.
elmer2 16 hours ago [-]
AI has only been around a few years in its current form.
It might take a decade to take what we learned from AI and turn it into something that will benefit the masses.
Most of it can't be broken down into something as simplistic as a dishwasher.
Most people won't notice cancers being found faster due to AI being better at pattern recognition than a doctor, but it's still a great benefit.
nswizzle31 14 hours ago [-]
A decade? Better not tell the investors that!
AvAn12 14 hours ago [-]
This is the paradox. For software engineers, the current crop of “AI” tools is magic. A huge accelerator. And then they assume the rest of the world will get the same boost. However this is not the case at all. Claude code is great because it is trained on extremely well documented git histories and stack overflow. And outcomes are binary: the code works or it doesn’t; the PR is accepted or it isn’t. Not much else in the business world is like this at all. No training data with this kind of sequential development information. And no real black and white outcomes as such.
I would love it if we can just take the win on having created an amazing tool set for coders. Can’t this be enough?
c0rruptbytes 16 hours ago [-]
LLMs could’ve been developed a lot slower without capsizing the entire supply chain and been seen a lot more positively, China is building a lot of good AI with a minuscule of the compute power and it’s fine
there was never a need to rush other than for capitals self interest to control
if frontier labs wanna pace the frontier, they should be honest and return the compute back to the people
letmevoteplease 16 hours ago [-]
China is building good AI by distilling American models. I think that's a great thing, but Chinese progress is currently dependent on American progress, because China lacks the compute to train the largest models (as stated by DeepSeek's founder[1])
i knew about the compute already (he was very public about i believe huawei under delivering the amount of chips he requested), but nowhere in that transcript does he say he's dependent on US progress for his own progress or he's distilling the US to make progress
i don't really think distilling is even that fundamental anymore (nor was the evidence ever that strong - https://msukhareva.substack.com/p/is-kimi-k3-really-distille...), the game is all RL now, and you don't need american progress for that at all (just compute and data, and the data brokers will sell to anyone no problem)
i do think there are black market proxies from china to the US trying to sell "claude" and "deepseek" access mixed together, and i do think that prompt distribution is probably going to be very nice datasets
Animats 12 hours ago [-]
Yes. The AI industry has to deliver some substantial value to the average citizen soon or there will be political trouble. Food, clothing, shelter, and medical care have not improved yet. Nor does there seem to be much AI can do in the next few years that will help with the basics.
We can have unlimited new entertainment content for passive consumption. That's about it so far. 17,000 new Asian vertical dramas a day now.
2sk21 1 hours ago [-]
I was listening to a Verge podcast in which Nilay Patel remarked that of the AI companies started pushing for universal healthcare, AI would become a lot more popular. But no signs of this.
mizzao 15 hours ago [-]
People need to eat, sleep, be healthy, have shelter, feel loved, be in communities...
Coding has been transformed but none of those areas of our lives have been substantive transformed by AI yet. Whoever does, will create a lot of value for the world.
atleastoptimal 16 hours ago [-]
>AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Do people need permission from mathematicians to solve math problems? Is one of the Millennium problems not worth applying a new system to work on it?
I understand the despair people are feeling at AI automating their work, but we have to be serious about the fact that the frontier labs are just responding to natural incentives. Mathematics, coding, all are just applications of logic, and it makes sense that a system trained on inferring patterns from data would work well in that domain.
jeremyjh 15 hours ago [-]
You are misunderstanding what mathematics research is. Its objective is the development of human understanding of mathematics. This requires open problems to research. AI are “solving” the open problems without creating any human understanding.
atleastoptimal 15 hours ago [-]
At the very least, the single bit of information of whether a conjecture is true or false advances human understanding. It allows researchers to abandon paths which would not have been fruitful.
I'm pretty sure OpenAI's proofs, while difficult to understand, have a net-zero impact on human understanding of mathematics. It took the first Millenium problem years to verify its accuracy. It's only been days since OpenAI released its problems, surely it would take years to evaluate whether the solutions meaningfully advanced human understanding.
jeremyjh 15 hours ago [-]
It may provide that one bit of information but it could still be a net negative if it destroys current institutions (such as graduate studies) and no new institutions can be developed that provide a clear signal of who is doing meaningful work. That was a byproduct of solving open problems that are now lost. These are not my ideas, these are things that Terence Tao and others have written about that were recently on the front page of HN, e.g.:
Who determines the objective of mathematics research?
All mathematics researchers in the whole world have agreed upon that objective apparently?
jeremyjh 13 hours ago [-]
This is basically the dictionary definition of the word "research". But yes, all the work they are doing is focused on this goal. They write papers for humans to read, give lectures for humans to listen to, have discussions with humans - all with the objective of improving human understanding of the subject.
Many of them - all but a couple of the living Fields Medalists - recently signed an open letter about this.
I don't know, this just seems like gatekeeping of the term "research" simply because you don't like the way it's being done.
Many modern medicines do not have understood mechanisms of action, yet we know they work. Are the people who found them not conducting drug research?
Do we even have a different term for what the AI labs are conducting if it's not mathematical research?
Additionally, I would argue that any work that adds to the total corpus of known facts increases human understanding, even if the path to get to each fact isn't understood. It's hard to argue we'd be better off with less total information.
sieabahlpark 15 hours ago [-]
[dead]
henry2023 15 hours ago [-]
OpenAI to this day is not aware if they solved or not Navier Stokes.
Mathematicians are not gate keepers but I understand the frustration. If someone claims they spent 30 million dollars to solve one of the most difficult problems yet there’s no new knowledge, understanding, nor certainty. Then what’s the point besides marketing?
anon-3988 14 hours ago [-]
The funny thing about comments like this is that the moment LLMs are going to replace everyone in the warehouse or the trucking industry, we suddenly have to do something against the LLM. But clearly they can't actually do anything meaningful and have been a waste of energy?
So just let them spend their money and they will eventually fail and bankrupt, right?
mindwok 16 hours ago [-]
Whatever benefits AI has to offer on a societal level are going to take a long long time to show up, give it some more time. We had smartphones in 2007, and e.g. By way of example, Uber didn't launch ridesharing until 2013, and then that still took years to seep into the public awareness.
By comparison, we're in the early years of AI. We're all still figuring out what this thing even means.
rimbo789 15 hours ago [-]
Uber was a massive mistake on every level.
This is what is so frustrating about this whole ai thing is we aren’t learning from the mistakes of even the very recent past
orangecat 13 hours ago [-]
Uber is a vast improvement over the previous taxi oligopolies.
mplanchard 13 hours ago [-]
I miss being able to reliably hail a taxi A LOT. Uber may be an improvement in smaller towns where cabs didn’t exist, but in bigger cities it is mostly a downgrade. Rather than walking to the curb and immediately getting a car, I’ve got to dick around with my phone and wait for a car to come to me.
ChickeNES 12 hours ago [-]
Living on the south side of Chicago, it was almost impossible to hail a cab, and often times if you called for one it just wouldn’t show up. Or I’d try to hail one downtown, and be refused once I told them my destination. I’ve not had that problem once with Uber, good fucking riddance to taxis.
fragmede 13 hours ago [-]
What's frustrating is that at the end of an event like a concert, everyone goes outside to catch an Uber/Lyft, but we then have to match driver to passenger, instead of having a line of waiting cars, each group just gets in the next available vehicle, shows the driver a QR code, instead of the mess we currently have. Why can't we do that? Taxi regulatory capture.
ChickeNES 12 hours ago [-]
Taxis were terrible and I celebrate their destruction at the hands of the rideshare companies. Even now it’s still cheaper than a cab was (though those early days of VC cheap rides were great).
fasterik 16 hours ago [-]
I agree with you, but those examples are going to fall on deaf ears. The pessimistic mindset will say that smartphones were a terrible invention that's ripping society apart, Uber is an unethical company exploiting workers, etc.
dofm 16 hours ago [-]
When did Uber stop being unethical? Did I miss the memo?
Because they didn't just break laws, they built an entire system to mislead law enforcement.
(How do tech people not know about Greyball??)
reasonableklout 14 hours ago [-]
I think ridesharing was a net good for ordinary people even if Uber is/was unethical.
However, the counterargument falls entirely flat. "Whatever benefits AI has to offer on a societal level are going to take a long long time to show up". We don't have the luxury of time because AI disrupting communities and livelihoods faster than perhaps any other technology we've ever seen, with the most capital that has ever been put into any tech since the railroads.
Even if you're optimistic we'll eventually get the cancer cures and fusion reactors, doesn't it make sense from the perspective of an ordinary American to regulate the harms like the homework cheating, parasocial relationships, job losses, cyber attacks, etc., especially if it will take years to get the benefits?
fasterik 13 hours ago [-]
Different benefits will show up at different rates. We don't need to wait for cancer cures and fusion reactors. It's already easier than ever to start a small business and use AI for things that would have been prohibitively expensive or time-consuming before. The technology is transforming how scientific research is done, how engineering is done, etc.
I don't see much evidence in the economic data that AI is disrupting livelihoods. Unemployment has been stable around 4% for a while and productivity growth has been at historically normal rates for the past 5 years. I think a more likely explanation is that AI is increasing uncertainty in some industries, which are seeing higher employee turnover, and many of those people are moving laterally into other jobs where their skills are more likely to be useful.
I don't think anyone in this thread has argued against mitigating harms. The OP was about current capabilities and the perceived lack of benefits for the average person.
mindwok 12 hours ago [-]
To be clear, I don't disagree with your sentiment that the disruptiveness of AI is causing a lot of grief. But magnitude of disruption is not an indicator of magnitude of benefit (or lack thereof).
Some of the most disruptive periods in history brought about the greatest long term improvements in society. This is the price we pay for changing how things are done, sometimes for better and sometimes for worse. We don't know which one it is for AI yet.
hellohello2 16 hours ago [-]
Publicly-accessible AI started working 1 year ago, 2 years tops. What did you expect? It to solve war driving prices up?
dd8601fn 14 hours ago [-]
I would have expected apps like Amazon and websites like various Meta properties to be even slightly less broken and shitty.
Like, just at a bare minimum.
These are “use all the AI” companies with infinite resources, after all.
hellohello2 13 hours ago [-]
Why use apps that you dislike, just ask Claude to click through them for you. And no I'm not being ironic, this is the way things are headed, you just abstract over all the crap by having an agent do it for you.
"Slop is its own cure"
vuurmot 11 hours ago [-]
The alternative is people stop using these apps, consumer demand goes down, along with the GDP of the country.
dumberquestions 15 hours ago [-]
I don't think it's surprising that it hasn't improved life much, AI is new and change takes time, but the fact of the matter is that it also made things worse in some ways, ways that I think were entirely avoidable, if only the incentives weren't so warped.
mitxela 15 hours ago [-]
The fundamental malignancy here is that powerful people have become disconnected from the foundation of society. They have suffered from model collapse. They have been talking to each other in circles for too long, and doing things that make no sense to anyone outside the elite group, and because the systems that concentrate power to the top are so strong now, they continue gaining power even while messing everything up in a nominally free competitive market.
N_Lens 7 hours ago [-]
Sounds like collective psychosis.
tim333 15 hours ago [-]
I think one of the hopes is they'll get smarter than us and able to solve real world problems better - disease, war, poverty etc. But they are not really there yet. Give it a little while.
jibe 14 hours ago [-]
AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
This is an indictment of mathematicians, not AI.
dullcrisp 14 hours ago [-]
Who indicted anyone?
13 hours ago [-]
raz32dust 16 hours ago [-]
This view is too cynical. The frontier advancements are certainly going to trickle down. Take cars for example. AI driven cars are already here. AI-designed battery systems, fuel systems, materials and 3d printing capabilities will come to cars. It will likely lead to cheaper, more efficient and more effective vehicles. If will just take time for the impact to percolate.
eloisius 16 hours ago [-]
My prediction is that while those “advancements” may come to cars, the only practical differences will be lower quality products, cost savings for the car companies, probably more expensive for consumers, and somehow it will also be collecting more data to sell to advertisers.
mym1990 15 hours ago [-]
If you look at the average price of a new car, adjusted for inflation, it has pretty much always gone up aside from recessions. There have been plenty of advancements in car technology, they do not make them cheaper. It actually gives reason to charge more for them due to perceived increase in value.
raz32dust 15 hours ago [-]
Yes but you're getting a lot more for the same price. It's a lot more powerful, comfortable, safe and efficient while the effective cost has remained the same.
jasondigitized 13 hours ago [-]
What percentage of that price is cost of good sold vs. a bunch of regulatory bs like dealers and taxes?
blactuary 16 hours ago [-]
Just a few more weeks, a few more hundreds of billions, it's right around the corner....
reasonableklout 14 hours ago [-]
This only works if there remains a viable and attractive consumer market for these advanced goods. AI is both an imminent wealth shock and perhaps the largest structural change to the economy in history as it threatens to drive the value of human labor to zero over the next decades.
In American history, when previous shocks like this occurred (e.g. with the Great Depression), we needed massive economic programs like the New Deal to restore consumer demand. When such programs are not forthcoming while wealth inequalities keep increasing, one can look to other countries for what ends up happening; either the disenfranchisement of ordinary people over time or unrest and redistribution by force.
m0llusk 15 hours ago [-]
AI car navigation is here, but driving is a social act. Have you ever tried to keep an automated taxi out of a private driveway or the scene of an emergency? There is a lot of work to do and limited evidence of LLMs raising the bar.
epolanski 15 hours ago [-]
I think he means LLMs, we knew about RL already.
xingped 16 hours ago [-]
Ah yes, the famous "trickle down" theory, which you may know from its smash hit "trickle down economics"! Surely this time it'll be different, right?
Marha01 10 hours ago [-]
> Ah yes, the famous "trickle down" theory
While trickle down theory is dubious when it comes to economics, it's clear that it is valid when it comes to technology. Tech trickles down.
xingped 15 minutes ago [-]
Ah yes, like my RAM, SSDs, and GPUs are trickling down right now? You're absolutely right! I love a trickle of PC parts!
Marha01 12 minutes ago [-]
This is a temporary anomaly due to the datacenter buildout, which will not last forever. Historically, tech has always trickled down. Sooner or later, hardware will again become cheap.
elmer2 16 hours ago [-]
How can you say tickle down economics doesn't work? The US has made it easier for businesses to grow and the result is the highest standard of living ib tje world.
ks2048 15 hours ago [-]
I think it's hard to quantify "highest standard of living", but for the attempts at it (e.g. HDI, etc), the US is not number one.
mym1990 15 hours ago [-]
Hate to break it to you, but the US doesn’t even break the top 10 in a lot of reports.
ba1afd89f34cb23 16 hours ago [-]
Trickle-down economics specifically refers to economic policy that favors the wealthy and corporations with the argument that this will lead to wealth trickling down to the economically less fortunate. Examples include the Reagan and Trump tax cuts.
This has been studied somewhat extensively (https://academic.oup.com/ser/article/20/2/539/6500315?login=... is one example), and it’s been shown over and over that this does in fact not work and leads to larger wealth disparity, across many countries where it has been attempted.
It’s possible you’re working off some other definition of the term based on your comment but that’s the generally accepted usage.
asdff 15 hours ago [-]
The person you replied to was the second person I've seen on HN this week try and defend trickle-down economics explicitly in more or less the exact same terms and justifications. What is going on with that? Did Peter Thiel tweet some pro trickle down dreck recently that these people latched on to?
dd8601fn 14 hours ago [-]
You should see the comments sections on NYT articles, lately. It was like someone flipped a switch about two weeks ago.
It would be neat if someone did sweeping sentiment analysis on them, just to see it graphed, but I’m sure gathering the data would be prohibited.
N_Lens 7 hours ago [-]
It’s all bots anyway at this point.
epolanski 15 hours ago [-]
The US alone!
Supermancho 16 hours ago [-]
Worse, the view is largely phrased as gaslighting AI. Blaming the cause of particular grievances that coincidentally have been sourced at the same time as the rise of AI, is purposeful.
> [Your train] never comes on time. Fuel is more expensive. The roads are still congested with traffic. Your kids are (probably) doing worse at school. Food costs more. Houses cost more. Rent is higher. Buying a computer or a PS5 is more expensive. Politics is still full of mentally unstable people. The environment is getting worse. You will still die of heart disease or cancer. Food quality is worse. Your kids can't find work. Wealth inequality is accelerating.
None of this has to do with AI.
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using), and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Plain wrong on both counts.
> Imagine what a "country of geniuses in a datacenter" will be able to do Apparently...nothing.
Nonsequitor, probably based on a personal beef. Datacenters, at best, are understood to be for hosting more AI - not for people to somehow live in as an arcology of discovery.
neumann 14 hours ago [-]
I think only because these are all things that AI companies are saying will be helped by AI so give us more money. Nobody advertises their product publicly as "you will be able to buy other product to do the same things you do now but pay people less and make more profits" even though that is where the value is for them.
ckocagil 15 hours ago [-]
Why fix your appliances and infrastructure and give you healthcare when the plan is to obsolete the whole lot of you?
This reply summarizes the problem from a different angle.
jasondigitized 13 hours ago [-]
As someone who is building one of these referred to systems in a major vertical in the Fortune 500, these LLMs are very much being used to eat major legacy systems and processes. There are still plenty of obstacles to overcome but the number of AI governance and steering groups that I have been apart of are indicative of what's happening in a major industry.
magicalist 12 hours ago [-]
> As someone who is building one of these referred to systems in a major vertical in the Fortune 500, these LLMs are very much being used to eat major legacy systems and processes
> Senior Director of Product in Insurance Tech
anakin_padme.jpg
oblio 5 hours ago [-]
Those LLMs are not being wielded by 2 tech bros in a garage replacing the 100 000 person company.
Those LLMs are being adopted by the huge enterprise itself and being surrounded from all sides by a very complex procedural harness (aka regular programming) that stops the LLM from doing dumb stuff.
aleph_minus_one 16 hours ago [-]
This is an interesting perspective.
Nevertheless, my counterpoint to
> If they actually had access to systems that could do "years of top professional work" across software, cyber, science, and math, they would not mostly be leaving to start more chatbot companies, agent frameworks, eval tools, and "AI for workflow" startups.
> They would be starting normal companies to disrupt the global economies of the world.
Many people who work in such industries already know an insane amount of inefficiencies. Unluckily, they are not in positions and have no political power (with respect to organizational policy) in the company that they are capable of changing the situation.
Additionally, many of these sectors are heavily regulated, so it is very hard to make a difference in this sector as a startup founder, and even if there is an entrpreneur who founds a company to change the status quo, an insane amount of work in the company will be handling the big amount of red tape.
dreamcompiler 16 hours ago [-]
Correction: My car spies on me and its cruise control no longer works because it slams on the brakes in the middle of the highway whenever it sees a mirage. My dishwasher broke because it got an unasked-for software upgrade.
Things worked better before we started putting bad software in everything. As a software engineer, I know what good software looks like. The fact that almost all software is now bad -- including the software that drives AI and the software that AI generates -- makes me want to flip the table. We know how to write better software and the general public should hang us if we continue to refuse to deliver it.
Terr_ 16 hours ago [-]
Probably preaching to the choir here, but the software is just a symptom.
The problem are the power-relationships and laws which allow someone to abuse you and your stuff, block competition, and even put you in jail for trying to remove the software—or helping others to remove it.
drnick1 16 hours ago [-]
> Correction: My car spies on me and its cruise control no longer works because it slams on the brakes in the middle of the highway whenever it sees a mirage. My dishwasher broke because it got an unasked-for software upgrade.
Never, ever connect appliances to the Internet. I thought this was common knowledge on HN. Also pull the telematics (cell modem) fuse to prevent your car from snooping on you and selling your location and driving habits to data brokers and insurance companies.
bragh 14 hours ago [-]
Yes, but this is ridiculous for society. It is to the level of "do not wear a nice watch if you do not want it to get stolen": there should be no need to avoid wearing a nice watch or to pull a telematics fuse from the car in the first place!
13 hours ago [-]
Ericson2314 15 hours ago [-]
I mean there's been huge strides in electric vehicles at low prices that Americans haven't experienced.
We've just locked out a large part of modernity because the shitty American auto industry is in swing states and also gender.
Same could be said about the slow rollout of Waymo.
Lots of America is just very anti technological change, and this means any technology no matter how "magical" hits a wall.
88j88 15 hours ago [-]
The change will be slow, not sudden. Slow and devastating.
jasondigitized 13 hours ago [-]
Why and how will it be devastating specifically?
16 hours ago [-]
fasterik 16 hours ago [-]
Is it reasonable to expect AI improvements to filter into all of those aspects of life that quickly? It's only in the past month or so that we've started to see major breakthroughs in mathematics. It might take several years for that level of research capability to reach areas like engineering and medicine, at least in a way that affects the average person's quality of life. But at this point it looks very likely that it will happen eventually.
Your pessimistic view doesn't hold up against the data. For a lot of the problems you mentioned, we're in a better place today than almost any other time in history. Of course, things can always be improved and we should never grow complacent or accept injustice. But by most objective metrics, there's never been a better time to be alive.
fluxusars 14 hours ago [-]
I think that's a bit disingenuous. We've only begun to apply LLMs in the real world and a lot of industries don't move very quickly. Not to mention that we're still very hardware constrained, we don't yet have the equivalent of an ESP32 chip we can just stick into any piece of electronics to provide state of the art LLM responses. But we'll get there, soon.
ModernMech 16 hours ago [-]
Have you seen the shareholder value though? That’s not nothing it’s the only thing that matters!
vuurmot 11 hours ago [-]
With AI software replication, I wonder if shareholder value even increases.
Services accounts for 80% of American industry, what happens when big software companies loses revenue to piracy/recompilation i.e. Adobe, Unity, AutoDesk, tax/law software, etc..
ModernMech 3 hours ago [-]
Yeah people were saying ai is the death of open source. I dunno about that but it seems to me closed source is having a bit of a crisis here too.
vinyl7 14 hours ago [-]
And the human society was entirely destroyed, but for a brief moment we were able create share holder value.
sensanaty 14 hours ago [-]
[dead]
sans_souse 16 hours ago [-]
I would just add that consumer products in general aren't the same - practically every business model has incorporated obsoletionism to the point it's practically cancelled out the curve of progress and evolution of tech. We could have nice things.
pishpash 13 hours ago [-]
Same as before, really. You can have geniuses but if not used right you get nothing.
kypro 14 hours ago [-]
> Imagine what a "country of geniuses in a datacenter" will be able to do? Apparently...nothing.
I suspect the real world change is going to happen unbelievably rapidly and isn't far off. Right we don't quite have the hardware to embody AI in a generally useful way, but that's coming fast.
Within a couple of years arguably any task that requires just knowledge or intelligence is something you'd be better off giving to an AI than a human. And decent humanoid robots are maybe a decade behind that.
In the same way people growing up in the mid 1900s saw rapid changes in their way of life with planes and cars replaces boats and horses, we'll see something similar with robots replacing human labours. Right now we're still in the phase where people don't think humans will ever be able to control flying machines and that cars are reliable and slow. Things will change.
epolanski 15 hours ago [-]
I already felt our civilization was on a steep decline, but AI is the final nail in the coffin, technology kills my own job too, hooray.
15 hours ago [-]
15 hours ago [-]
ajross 16 hours ago [-]
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using), and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
That's not fair. To first approximation, most of the progress in human condition of the last, like, forever is implementing solutions that people didn't see value in at the time of creation. Woz built a toy that could barely run software at all. The human genome project was panned as a boondoggle. Penicillin was a mistake.
The first fire pit probably looked like a dangerous stunt to the tribe faced with it, even.
Does that mean everything will come up roses? No. But LLMs only started talking to us for real like three years ago. Let's not just that fast.
scotty79 16 hours ago [-]
You are talking like we had this capabilities for a decades but it's been just a few months of truly capable AI. Virtually all of IT switched to agents that quick. Non IT people just started to make software for themselves. Vision and translation capabilities of LLMs help millions. People talk with AI's now verbally to get advice or to just hear a friendly voice. The math thing is just few weeks old. We are just getting started. How fast did the telegraph meaningfuly changed lives for the better for bulk of people? Radio? TV? Computers? Internet? Cellphones?
Has any technology made an impact on physical world faster?
LaurensBER 16 hours ago [-]
Exactly, we're all software engineers here and even for us it took weeks/months to get to a position where we can really leverage LLMs (think platform, security, observability, etc).
Imagine being in charge of a physical factory, sure you might have a 200 USD Claude subscription but you also have a floor full of physical machines that have, if you're lucky, an undocumented debug interface and a paper manual. Not to mention safety inspections, insurance, etc.
You can't vibe code a factory floor. Not yet, perhaps in 5 - 10 years but there are some very hard, non technical, problems that need to be fixed before we'll see meaningful improvement there.
jasondigitized 13 hours ago [-]
I'm pretty confident with enough context, Opus 7.5 will be able to vibe code a factory floor.
strange_quark 13 hours ago [-]
How much time do we have to wait before we are allowed to criticize AI? GitHub copilot has existed for half a decade. Cursor, Claude code, codex, have all existed for over a year and have seen more investment that practically any product in history and have almost 0 barrier to entry. Where is my decade of software progress in 1 year, because that’s what 10x productivity looks like! All I see are the same old broken products and services from basically the same handful of big players, higher costs, and more bugs. Remember when every year iOS and Android saw loads of new features and a radical rethink every few years? And that happened on ancient hardware and in a significantly smaller industry. Where is anything like that happening now?
Marha01 10 hours ago [-]
> How much time do we have to wait before we are allowed to criticize AI?
You are allowed to criticize it, just not because "the progress has been too slow". It was not.
scotty79 7 hours ago [-]
I'll criticize Copilot with you. It's terrible and close to useless and it's a terrible shame that many companies got stuck with it due to being early adopters of this crap. I'm sorry for my friends who are employed by such companies.
Progress between Copilot and something like a gpt-6.1-sol in a good harness is at least as big as few Android versions.
oblio 16 hours ago [-]
> to just hear a friendly voice
This is far from being proven as a net gain and there are massive lawsuits incoming related to AI recommending suicide to kids and others.
LaurensBER 16 hours ago [-]
In all fairness, depression and suicide have existed before LLMs.
Safety systems are not optimal but improving everyday and although I doubt we'll ever get a perfect safety system, it seems that we're not very far from a "good" safety system. The days of Google recommending glue on pizza and generating diverse Nazis seem behind us.
Not to mention all the untold good that LLMs have done. Personally I've seen that LLMs are incredibly good at pointing conspiracy nuts and extremists into the right (moderate) direction.
In a polarised world, we definitely need that.
oblio 4 hours ago [-]
What do you think happens when political ad buyer start buying inception style hints in chat bot answers?
Also, rage baiting also drives up chat bot conversation durations.
The incentives are exactly the same as for Facebook and Google Search.
nonethewiser 13 hours ago [-]
What’s your point?
People didnt get a new car after they got. Internet in 2001. They didnt get new dishwashers.
heaney-555 16 hours ago [-]
>Your car is still the same.
Not for Tesla owners in select countries, where their car now drives them around 99% of the time, or the people of Austin Texas, that can now hail a fully driverless Cybercab for cheaper than an Uber!
>Food quality is worse.
Food quality is higher than ever if you want high-quality food!
Unfortunately, the people crave slop.
TacticalCoder 16 hours ago [-]
I'll add some...
Videos are shittier. Most songs are even shittier than the autotuned pieces of shit already were. Ads are worse (even though nobody thought that was even possible). Customer support is worse.
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using)
That nobody is using and that are worse than what we used to have. And because nobody's using them, the countless issues plaguing them aren't even reported by users anymore (there are no users anymore). Full of bugs, usability issues and gigantic security holes (but which doesn't really matter because nobody's using them).
It's not even clear that it's making long-standing software (like say Linux or Emacs or Blender or QEMU) better at all.
Now I'd say though: most of the issues you mentioned have nothing to do with LLMs / AI. But it's very clear that AI ain't solving much of the world's problems and that the promises we got years ago didn't materialize.
Roark66 16 hours ago [-]
All you said is true, but on the flip side it is giving an individual user huge power over software. To create their own, to reverse engineer proprietary ones. Decentralisation of such power can only be a good thing.
Even if a certain small group of people are also using the hype surrounding this tech to attempt to destroy the home PC market and reverse the personal computer revolution. They will not succeed. There is more of us than them. Even with "thousands of frontier LLMs". A human+llm is going to be more powerful for a long time rather than a bunch of LLMs.
blharr 15 hours ago [-]
The power isnt really decentralized when you see that the internal AI models are superior than anything you can self host, and the right to use those can be stripped away at any time.
jasondigitized 13 hours ago [-]
This is like saying food is shittier today because there are now more restaurants. No, food has never been better, there are just a ton of shitty restaurants.
mplanchard 13 hours ago [-]
More like saying samey corporatized slop bowl copy paste jobs (eg Cava) are driving out real, quality restaurants, a thing that has actually been happening for some time.
raverbashing 12 hours ago [-]
Honestly that's true of most innovations
AI won't show itself as a big new thing, but possibly in improvements in small little things
Most people won't use AI directly, except maybe as a smarter Google. In the same way people don't use HTTP directly
14 hours ago [-]
14 hours ago [-]
Aurornis 16 hours ago [-]
It’s amazing how much the goal posts move in every AI conversation. A couple of years ago the average HN thread about AI would be filled to the brim with skepticism about AI ever being able to accomplish anything other than flappy bird clones.
Now it’s comments like this lamenting that the amazing LLMs we’ve had since early 2026 haven’t entirely revolutionized the whole physical world and even global geopolitics yet.
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using),
I guarantee a lot of software you use today has substantial AI development in recent releases.
Even the Linux kernel is getting a heavy input from AI and Linus is advocating for it. Unless you’ve frozen your entire software stack in 2024, you are using software with AI generated code.
> and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Saying nobody wants those problems solved is flat dishonest. The complaints from some mathematicians are that they and their peers didn’t get to put their names on the papers, not complaints that the problems were solved.
sick_of_slop 16 hours ago [-]
[dead]
WillowWithAWand 16 hours ago [-]
[flagged]
chanakya 16 hours ago [-]
"It is not from the benevolence of the butcher, the brewer, or the baker that we expect our dinner, but from their regard for their own interest"
When companies make profit, it doesn't mean we get screwed, nor even that they have a high profit margin. The grocery chain stores are a prime example. They typically make 2-3% net margins on revenue, and provide fresh, clean, reasonably well-tested groceries close to our home for that margin. Imagine trying to get equivalent quality fresh groceries ourselves from the producers, and I'm guessing it would cost at least three times as much.
WillowWithAWand 9 hours ago [-]
ngl I don't think a quote from 250 years ago is all that relevant in discussing capitalism as it exists today. Adam Smith wouldn't recognize the world we live in. Nor was my point "pRoFiT bAd" but rather growth for the sake of growth is not the way to actually improve lives. Infinite growth is impossible (something I'm sure even Adam Smith would agree with me on) and yet our system is currently designed around the idea that it is not only possible, but good and necessary.
tim333 16 hours ago [-]
Worst system, apart from the other ones.
chiengineer3 16 hours ago [-]
I've been thinking about this for 2 years. At some point it will be easier for them to create self sustaining cities - free houses cars public transportation clean food clean water -
Africa is going to look like wakanda in less than 5 years if WW3 gets canceled
beloch 16 hours ago [-]
"So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time."
-------------
It comes down to the difference between programmers and non-programmers. If you're not coding, AI literally is a derpy chatbot that's made it harder to get search results, raised your power bill, destroyed property values in a neighborhood near you, is propping up the economy but also creating a huge bubble that could pop and destroy a lot of retirement savings, etc.. At best, it's a nuisance, but it could be a lot worse than that. The AI evangelists need to understand this, because the average person is starting to pay more attention to the downsides while still not seeing much upside.
This gap between the people seeing the curve go vertical and those who are simply growing more and more annoyed is a real problem... mainly for the people seeing the curve go vertical. If the people who are merely annoyed right now don't see some of the promised benefits soon, or if any of the negatives get worse (e.g. massive increases to greenhouse emissions), they might get angry. You probably wouldn't like having the overwhelming majority of the public angry at you. AI evangelists need to start doing better things for the majority of people or they need to address the negative consequences they're inflicting much more aggressively.
smugglerFlynn 7 hours ago [-]
> Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all.
“Cy Draven: I tried everything. Believe me, I hammered the race thing, I hammered the gender thing, the trans thing, the border thing, the homeless thing, the war thing, the election thing, the abortion thing, the climate thing, the thing about induction stoves, Israel, library books, vaccines, pronouns, AK-47s, socialism, BLM, CRT, the CDC, DEI, 5G, everything. All that I did, nobody, just nothing.
People are just numb these days, I don't know why.”
— Knives Out movie
prometheus1992 16 hours ago [-]
so what was karpathy expecting?? everyone will give up on their life and start feeding the monster in the screen? that's literally what the tiny sliver is doing; they are food for the AI model.
16 hours ago [-]
wg0 16 hours ago [-]
That's fine but these companies having billions in funding by now should have rewritten chalk and other terminal libraries in Rust while bundling the whole thing in Rust as a single static binary. I'm talking about horrible Claude Code and friends.
And then using emgui they could expose full fidelity IDE+Agent workflow interface (like DeepSeek Harness and similar) that's also exposed as MCP to be operated by another agent and JSON RPC for over the network but no. Skill issue?
PS: Even full fidelity Photoshop is possible with such high fidelity UI that has CLI, JSON API and MCP server all in the same single binary < 90 MB. For reference, see PhotoCraft or VectorCraft or WordCraft or PdfCraft.
gcanyon 11 hours ago [-]
> So this is the weirdness of the moment.
I think "moment" significantly undersells this. Frankly, I wonder if this isn't setting vast swaths of the population on fundamentally different paths.
Currently the difference in life outcomes between people who know how to use computers well and those who don't is small enough that in everyday life we just ignore it. But it's greater than, say, the difference between people who took advantage of refrigeration and those who stuck with ice boxes, or those who learned how to drive a car and those who didn't.
And the difference with AI is going to be even greater. And it's possible that right now people are choosing a life course that will benefit or disadvantage them long term.
fcarraldo 11 hours ago [-]
How does one who doesn’t have an employer funding their massive token budgets become “good at AI” the way that Karpathy describes here?
I think the point is that this gap exists partially because of the access that a select few have today, which enables them to do things that the have-nots can’t fathom.
everyone 5 hours ago [-]
The people who are heavily invested and stand to profit the most also think its super amazing. That's so weird!
tripleee 17 hours ago [-]
He's intentionally forgoing all nuance in order to make this sound dramatic
> Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt.
No, they can't, at least not any semblance of quality. The cases we're seeing where this does kinda work is in ports and translation where all the rules are already documented in the best specification language possible with a way for the LLM to verify itself: code. We saw this close to a year ago now with Cloudflare and NextJS
> The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about
These are operating on different capabilities - AI's ability to answer informational queries as a chatbot frankly sucks and can't be trusted without verifying it. I run up against this every day. A problem with a verifiable answer on the other hand it's very good at solving. He knows this (his next paragraph) but he's putting them on the same scale of "stressing the system" to attempt to add proof to his introductory claim
And then there's the completely unverifiable scare that there are internal frontier models way beyond anything we've seen "swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics"
I dunno. I haven't been able to set up OpenAIs remote codex connection, their shit is buggy as hell and the web UI keeps crashing and making messages disappear. Is this what their internal superhuman "Things that would have taken top professionals in the industry years of work" looks like? Granted Claude has been really smooth, but still..
zer00eyz 15 hours ago [-]
> where all the rules are already documented
Defined testability.
> swarms of thousands of agents collaborating over weeks on software mega projects
If you have defined testability, running a single agent, in a loop, over a goal will achieve the same results.
> minting zero days, running cyber attacks ... Granted Claude has been really smooth, but still
The old adage about monkeys writing Shakespeare applies here. Agents are just better monkeys and if the goal has a well defined outcome it's fairly easy to "Brute force" the way to an answer.
15 hours ago [-]
jfrbfbreudh 17 hours ago [-]
Congrats, you’ve discovered that you are not part of this group.
causal 15 hours ago [-]
Yeah I suspect Karpathy is too generous. I’m finding a lot of devs with access to the best models are not effective with them.
So I think there is are growing gaps even within the described groups.
bigstrat2003 16 hours ago [-]
That group does not exist. It's all hype, zero substance.
jfrbfbreudh 16 hours ago [-]
This just goes to further show the divide.
You’re trying to tell me it’s all hype when I have (depending on the day) a top 50 Productivity app in the iOS App Store for which I’ve written and read exactly one line of code (the version number).
It is comical that you think a good bar for quality is the fucking appstore.
What even is this comment. Like, oh wow, an iOS app, AI hype=real.
???
jfrbfbreudh 15 hours ago [-]
Oh no, I get money deposited into my bank account every month for something I didn’t write and barely maintain. How will I ever gain value from AI?
Jtarii 2 hours ago [-]
People have been dumping garbage on the appstore and collecting checks for a long time. If this is all the AI revolution has to offer then it's not looking good.
icedchai 14 hours ago [-]
Yes, this is impressive. When the iPhone SDK was first released in 2008, I spent months building apps to get them into the App Store. That included learning Objective C, XCode, various frameworks, and tons of manual testing. It was a lot of work!
g-b-r 15 hours ago [-]
Uhm, the set of apps that are successful and the set of apps that are crap are not exactly disjoint...
Good for you that you're gaining value, but that doesn't necessarily mean that you're providing value to others
jfrbfbreudh 15 hours ago [-]
If only there were some kind of rating or review system.
g-b-r 14 hours ago [-]
We sure don't see crappy apps with great ratings
Anyhow, if your users are currently satisfied, they might change their mind when they find that their personal data was uploaded to an open S3 bucket, when the new version wipes all their projects or when they find that not really everything was being saved (it's random examples, I don't know if they're applicable to your app).
SamPatt 15 hours ago [-]
Well done.
It's incredible to see AI denialists still active in late 2026.
tripleee 16 hours ago [-]
I don't think AI is all hype at all. Opus 5.5 is really impressive. This twitter (x? whatever) post is infuriatingly hypey and just written for virality, though
SamPatt 15 hours ago [-]
It's insane how confidently you can make a claim like this in a forum where thousands (tens of thousands?) of people are using these tools daily to build useful software for themselves and others.
How can you possibly believe it's all hype? Is everyone just lying? Are we all paying $200 per month because... why?
rossy 13 hours ago [-]
Spend long enough (or not even that long) in corporate software development and you'd believe such unsubstantiated hype is possible. The only difference here is even stronger incentives to pretend the emperor has clothes: insane overinvestment, threats of job loss for non-believers, fear of missing out on massive alleged productivity gains.
17 hours ago [-]
hacker_88 16 hours ago [-]
We have spent the whole evolution understanding the Real Intelligence RI
ilovecake1984 17 hours ago [-]
I’ll say this until I am blue one the face. Nerds (software dev, maths etc) see how good LLMs are at things they care about and assume they will be broadly applicable in future.
There’s no reason to think this.
howunfortunate 17 hours ago [-]
> There’s no reason to think this.
There are lots of good reasons to think this.
The first is that LLMs used to be bad at each of these nerd things, then toppled them like dominos. There's a pattern over time.
Another reason to think this is g. Across every known measurement, human (and animal) intelligence is convergent. Being better at one thing correlates with being better at another thing. Although the reason is not perfectly clear, the pattern is well established, and seems to apply to LLMs too - GPT-6 is smarter than GPT-3 at everything, not just math. There's no reason to expect different for GPT-9.
throwawayy6767 16 hours ago [-]
That's just cocktail party level of thinking. LLMs are getting better at math, code and logic (and marginally better at science and general knowledge) because these are domains that can be objectively verified and thus there is a potentially infinite supply of 'facts' to generate and train on. These are very powerful but ultimately very abstract domains. For everything else the messy real world and its physical bottlenecks gets in the way and there's little reason to expect progress to accelerate. It still takes months to get mice to reproduce and run experiments on, no matter how knowledgeable about biology the models have become.
If anything, in some domains frontier models have become worse - claudisms and chatgpt idioms are making them notoriously bad at prose without a considerable amount of prompting and tweaking.
atleastoptimal 15 hours ago [-]
Is good writing verifiable? I don't think it is, but LLM's have been hillclimbing writing quality. That being said this is with the help of RLHF.
However there are many other domains which have verifiable rewards in the process of learning them, despite their overall impact not being verifiable. For example, the life sciences, an LLM could be given access to data about an organism, and then make predictions about how a drug or gene therapy will affect that organism. In economics, LLM's could create models of behavior, evaluate predictions over time and see how well those predictions match reality.
>claudisms and chatgpt idioms are making them notoriously bad at prose without a considerable amount of prompting and tweaking.
I think this is because a lot of people genuinely like the claudisms, even though a small minority of technical people don't.
wizee 15 hours ago [-]
The quality of natural language output from today’s LLMs seems no better and often worse than LLMs from 18 months ago, at least in my experience. It’s much harder to build a good RL loop for something subjective like good writing, compared to something with an objective and verifiable right or wrong answer, such as software or mathematics.
howunfortunate 15 hours ago [-]
If you gave me natural language from a 125 IQ human and a 155 IQ human I'm not sure I could judge which human is smarter from the writing quality.
And that's a massive intelligence gap!
It may just be that writing is mostly saturated, and the remaining perceived gap is not quality but some kind of "cultural fit".
habinero 14 hours ago [-]
IQ is a useless thing, but you can absolutely tell intelligence from writing. You don't need to write like [author of your choice], you need smart ideas and the desire to make them understood.
AI writing sucks because there's no intent or choice behind it. It's human thought, blended by linear algebra into pink slime and extruded into sentences.
howunfortunate 14 hours ago [-]
> IQ is a useless thing, but you can absolutely tell intelligence from writing.
This is inherently self-contradicting.
jgorn 12 hours ago [-]
It is not, as IQ is not an all encompassing measure of intelligence.
blharr 11 hours ago [-]
IQ doesn't measure writing _at all_ does it?
throwawayy6767 15 hours ago [-]
It's not at all clear that LLMs have gotten significantly (much less hillclimbing) better at writing since GPT4-ish. They did get better at oulipo-like challenges (writing under silly, verifiable constraints like Fable's viral 'facetiously' poem) but that's far from being all there is to good writing.
Anecdotally, creative writing communities do report drastic differences in quality between models (it is my understanding that Gemma 4 is especially praised) so there is something to it being verifiable but it's a far cry from being something that can be distilled into objective math/code-like evals. One could even say it comes down to vibes.
>For example, the life sciences, an LLM could be given access to data about an organism, and then make predictions about how a drug or gene therapy will affect that organism.
Yes, biotech companies are doing this right now but ultimately they are just predictions and you still need cold hard biological data to ground them and iterate on. That's slow and expensive, especially for the juicy fields (human biology, food and crops, clinical trials) and you can't just plug billions of VC capital into the pipeline and hope for RSI. Physical (gotta procure all those labs and their equipment), human (gotta hire and pay specialists to run experiments), biological (gotta wait for organisms to reproduce, drugs to take effect, crops to grow), regulatory (gotta convince agencies that your fancy new drugs are legit) bottlenecks get in the way.
And biology is one of the easier fields where a path to RSI (if not accelerating) is conceivable. Good luck iterating on macroeconomics data where there are no replicates and 'experiments' take literal years if not decades.
>I think this is because a lot of people genuinely like the claudisms, even though a small minority of technical people don't.
My uncharitable take is that the abtruse jargon makes people feel smart for understanding it and gives a sense of belonging (as a closed circle of initiates who understand LLM cant), just like rationalists love to repackage old or unsavoury ideas under new nerdy smart sounding names.
atleastoptimal 15 hours ago [-]
By evaluations against human writers, LLM's have improved at writing.
>you still need cold hard biological data to ground them and iterate on... you can't just plug billions of VC capital into the pipeline and hope for RSI
Capital can get you a lot: access to wetlabs, historical medical data, ability to run simulations, creation of autonomous bio labs which run thousands of experiments in parallel. An LLM, in training on this flywheel of data can take in more information than a human biologist would be able to read in multiple lifetimes.
>Good luck iterating on macroeconomics data where there are no replicates and 'experiments' take literal years if not decades.
The better an LLM gets at simulating human behavior, the more accurately you can simulate economic policies and their effects via running simulations of agents in virtual societies. There are many routes that models can take to create a broader sense of the world without needing the world to provide data.
howunfortunate 13 hours ago [-]
That benchmark is fascinating, thank you.
throwawayy6767 14 hours ago [-]
>By evaluations against human writers, LLM's have improved at writing.
I would really take those benchmarks with a pinch of salt. We already know benchmaxxing is a thing for evals that are objectively verifiable like code, and coding models that seem to perform well on benchmarks don't pass the vibes test, imagine all the caveats that come with such a subjective test like creative writing, not to mention control issues (are they the same human writers across benchmarks? Are they the same testers?)
>Capital can get you a lot: access to wetlabs, historical medical data, ability to run simulations, creation of autonomous bio labs which run thousands of experiments in parallel. An LLM, in training on this flywheel of data can take in more information than a human biologist would be able to read in multiple lifetimes.
I don't disageee and am in fact excited to see the advances, what I am arguing is that it is going to be slow because the hypothetic feedback loop that would improve the model is bottlenecked. There are actual physical and biological limits in our universe. Unless VCs find a way to make mice embryos and crops grow 10x faster (among the many, many other things that would be required for acceleration) the trickle of data is not going to be anything like the accelerating returns imagined for math and coding. And that's assuming biological RSI is possible which has yet to be proven - current foundation models are interesting but they are still specialized and there is no clear, obviously uniform way to encode the many, many modalities that exist in biological data.
>The better an LLM gets at simulating human behavior, the more accurately you can simulate economic policies and their effects via running simulations of agents in virtual societies. There are many routes that models can take to create a broader sense of the world without needing the world to provide data.
You wouldn't need to 'just' simulate human behavior (good luck with that btw) but the entire political and economic conditions in which they live. Even assuming this is possible the amount of compute you'd need to run would be astronomical. Given how all RAM capacity for '27 and '28 is near sold out, hyperscalers are running out of cash flow to finance buildout and Sam Altman complains when users say 'please' to LLMs I don't think this is in the near horizon.
asdff 15 hours ago [-]
There was a time when people believed in Moore's law too. Now even Jensen Huang says Moore's law is dead. There's also the adage from investing: past performance does not guarantee future returns. A little CYA sure, but also acknowledgement of latent factors and the entropy of the universe and its affect on any predictive modelling one might do.
denverllc 14 hours ago [-]
I’m not sure I agree that g explains all model capabilities. I think “claudespeak” is a consequence of the shift from training on RLHF to training on verifiable rewards. Thus the models seem to become worse at storytelling, fiction, readability, not better.
Even if we do accept that models have some “g” and that it correlates with all domains, there’s no reason to think that the current trajectory of LLM intelligence will grow more quickly or even as quickly. In every other technology we have an early period of slow growth during experimentation, and then a rapid period of growth, and then stagnation.
For all we know, 2026 could be the year of stagnation. It’s possible that we will reach finishing returns. If thats true, then it won’t matter how much that task correlates with overall g, because g won’t be increasing that much.
flecomet 16 hours ago [-]
I don't agree that they get uniformly better at everything, especially subjective skills.
For example, they have become more and more unintelligible when you ask for explanations or descriptive text. They assume you see the same context as them and shortcut explanations.
I've had to craft a skill to get them to produce remotely understandable explanations of even mildly complex/non-mainstream subjects.
This may be Curse of Knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge on their part, and it also impacts human experts but still. Becoming better at one thing does not mean you become better at everything else, although I do agree with you that the better they become the more things there are they become good at, but their ability is still quite jagged and maybe increasingly so.
howunfortunate 15 hours ago [-]
The delineation between knowledge and intelligence is useful here.
Knowledge can occasionally get in the way of some tasks, such as a master illustrator trying to draw like a child. Or as you've mentioned, an expert trying to explain to a beginner.
Even so, I disagree with you about model progress on communication - maybe there's a little jaggedness between minor model versions, but Opus 5.5 is a much much better communicator than, say, Claude 3 Opus.
ilovecake1984 8 hours ago [-]
You are conflating intelligence with usefulness.
oblio 16 hours ago [-]
> Across every known measurement, human (and animal) intelligence is convergent. Being better at one thing correlates with being better at another thing.
LOL. Human (and animal!) intelligence is notoriously jagged. We're all basically idiots except for very narrow areas where we focus.
howunfortunate 16 hours ago [-]
This is simply untrue. I tried to think of an animal example so I could give a concession, and there just...aren't any.
Smarter just means smarter.
hennell 15 hours ago [-]
As anyone who's needed to provide IT support to a doctor, lawyer, lauded academic or host of otherwise "smart professionals", can assert, you can be very very smart in one area and very stupid in another.
This is the classic error in mixing up knowledge / experience and intelligence.
Intelligence is the ability to quickly process, understand, learn, and use information. It does not imply having already been exposed to any given information.
A smart doctor may have terrible computer skills. But this is from lack of experience and interest, not lack of ability to learn how to use computers.
But take that smart doctor, and a relatively dumb person who is equally inexperienced with computers. Now give them a month and incentive to learn computers. The doctor will learn much more; that's intelligence.
oblio 5 hours ago [-]
There is IQ, EQ, and even purely intellectual aspects have a million facets. So you're wrong even on that front.
Someone can be really good at physics and suck completely at biology, due to factors we don't even understand (motivation probably being the main one we do understand).
We even see this in practice with super bright tech people, especially programmers, failing at launching businesses or at adapting in other lines of work such as management, finance, etc.
dragontamer 11 hours ago [-]
Cats famously have more brainpower than dogs. But dogs understand us humans far better than any cat.
Someone who actually works with real animals (farmers? Ranchers??) would come up with better examples. But I'm just city boy over here who only meets cats and dogs on a regular basis.
howunfortunate 11 hours ago [-]
I was about to respond 'great counterexample' but I looked it up and apparently dogs actually have quite a bit more brainpower than cats:
Hmm, I can believe that it depends on the breed of a dog.
But in any case, Britannica is a good source so I stand corrected on this subject.
majkinetor 17 hours ago [-]
There is every reason to think this. Its about available quality data. The data that was easiest to fetch was already there, then we got some more data by asking experts to create datasets for post training. Once this is over, we will get to the outer world that didn't get to hoard it for bots to take it. This will certainly change. Put on a smart glasses and record what you do to fix a pipe. In 3-5 years, rinse and repeat.
RealityVoid 17 hours ago [-]
Maybe, I'm at the point I don't know what to think anymore. But somehow, this feels still non-human. Humans don't need millions of hours of training and millions of samples to learn how do to something. We have proof that systems can deal with low-shot training. So why can't these? My point is when we have human level learning ability, then we're moving, if we expect expansion of capabilities to come from data alone, I'm prepared for disappointment.
sebastiansm7 16 hours ago [-]
To form an human to do frontier research requires decades since birthdate
majkinetor 7 hours ago [-]
It will be comparable once the AI is self sustainable and self reproducible (no big data centers behind it). Then it will have to deal with limited resources to run itself like humans do. In that scenario it will no longer have entire world of knowledge at its hands - options will be to cut anything not usable imediately or to communicate with big AI in the cloud.
SamPatt 16 hours ago [-]
Being good (or superhuman) at software dev and maths shouldn't be treated as though it's equivalent to other domains.
Unlocking mastery of those domains would lead to incredible gains in all others. Whereas a fundamental advancement in many other domains doesn't spread as readily.
ilovecake1984 8 hours ago [-]
I’m not sure. LLMs are superhuman in terms of workload (they are maybe also superhuman in ability too), but where does that lead us.. cheaper software?
That’s good for corporations who need custom software. Every large business has a massive backlog. In this area reducing the cost will massively increase demand.
But where’s the broader value?
Consumers and business already have too much choice in off the shelf software.
Ultimately most people just need word, excel and maybe 4 computer games a year.
mindwok 16 hours ago [-]
Just like the nerds thought about computers, or the internet, or video games, or smartphones, or crypto, or... whatever.
The most passionate, intelligent people are usually a decent indicator of where culture is going to go.
aleph_minus_one 16 hours ago [-]
> The most passionate, intelligent people are usually a decent indicator of where culture is going to go.
Many very smart people who I know are quite skeptical of LLMs and the hype around them. In my observation/echo chamber, the people who are very into LLMs are rather slick, career-minded people who love to present themselves as trendsetters for the "next fancy thing".
dimbletimbers 16 hours ago [-]
The most passionate, intelligent people 1. aren’t spending a lot of time on smartphones, video games and crypto and 2. aren’t as much smarter or better at predicting the future as the barely above average people who mistakenly count themselves in their ranks like to think.
throwawayy6767 15 hours ago [-]
Most of those technologies were controversial in their inception among the nerds of the time. Remember Eternal September? Linux/Windows people making fun of Apple hipster fans? Mainframe engineers deriding PCs as unserious and video games as childish?
>The most passionate, intelligent people are usually a decent indicator of where culture is going to go.
That's a naive view of cultural determinism. The software landscape would have been very different if a handful of lawsuits had gone one way or another. Hell, even Unix and Linux were hobby projects that accidentally got big.
Also crypto is a grift lol, one of those is definitely not like the others
ilovecake1984 8 hours ago [-]
These have all been mistakes though.
grtt1 16 hours ago [-]
Steve jobs wasn’t a nerd - he said he’s a hippie on the lost interview. He was also the first to bring to market colour screens, typography etc. He also defined the Internet as the defining ‘social moment’ way before anyone else.
So you’re wrong pal, sorry.
bigstrat2003 16 hours ago [-]
> Nerds (software dev, maths etc) see how good LLMs are at things they care about and assume they will be broadly applicable in future.
It's worse than that. I am one of those nerds (a programmer), and I see that LLMs are shit at the things I care about. I have zero reason to believe that they will be good at other things either.
aleph_minus_one 16 hours ago [-]
> Nerds (software dev, maths etc) see how good LLMs are at things they care about and assume they will be broadly applicable in future.
I don't think this explanation is sufficient:
For example, many nerds care about 3D printing. On the other hand, my experiments (and the experiments of many nerds who I know) to let LLMs create files for 3D printing lead to horrendously lacking results.
Or many nerds care about linguistics or puns. Whenever a new LLM comes out to which I have easy access, I do the test, and let it explain some specific German jokes to me that are based on convoluted puns in the German grammar. Until now, no LLM that I had access to could give satisfying explanations of the puns on which these jokes are based.
Or even for coding (many nerds do care about elegant, sophisticated code): the LLMs that I could test were already overchallenged with the following task: I had written some code that is a very "artisanal", "clever" improvement of an algorithm over the version that one would find in a textbook. The task for the LLM was simply to write some code comments/documentation about the mathematical ideas upon which my improvement over the textbook version of the algorithm is based. It wasn't capable to do this. On the other hand, for a junior programmer, I would in such a situation expect that he goes through every single line and thinks through the algorithmic ideas so that he can learn from them.
---
Thus: even for "nerdy" topics, it is very easy to find tasks where LLMs still suck. So, my hypothesis about the nerds that you mention in your post is that these nerds are rather people who want to believe (with religious fervor) that the current LLMs are exceptionally good instead of just looking into a slightly different direction than where the tech billionaires want the users of LLMs to look at.
grtt1 16 hours ago [-]
You’re having a hard time distinguishing nerds and hippies.
Hippies live at the center of technology and humanities.
Nerds sit purely in technology.
aleph_minus_one 16 hours ago [-]
I honestly don't get your point: perhaps your argument is based on some US-specific cultural reference that I am not aware of since I don't live in the USA.
Izkata 13 hours ago [-]
If it is, I don't get it either. That's not at all what I imagine with hippies and I suspect that was supposed to be some other word.
Nerds are also not necessarily all about technology. "Math nerd" was a really common use of the term for a long time, for example. "Nerd" is someone with deep knowledge about a particular subject.
nullstyle 15 hours ago [-]
> For example, many nerds care about 3D printing. On the other hand, my experiments (and the experiments of many nerds who I know) to let LLMs create files for 3D printing lead to horrendously lacking results.
Try again; astra has been driving fusion for me. I took a screenshot of a self watering cat grass bin on makerworld — not even the stl, a screen cap of one of the photos attached to the design — and it made me a parameterized version in one shot.
Its initial design was less than ideal for printability and it reworked the design correctly when asked to consider what it originally produced through the lens of printability.
Autodesk bundled an mcp server with fusion in the last few weeks and i expect things to get even better going forward.
aleph_minus_one 15 hours ago [-]
> Try again; astra has been driving fusion for me. I took a screenshot of a self watering cat grass bin on makerworld — not even the stl, a screen cap of one of the photos attached to the design — and it made me a parameterized version in one shot.
I have no access to Astra, but with GPT 6.1 Sol, the results with respect to attempting to create STL, OBJ or even OpenSCAD files by using image references and a prompt were really bad.
nullstyle 15 hours ago [-]
Did you give your model tools to see results and refine as it works through the turn? It does matter how you hold these models at the moment
cindyllm 15 hours ago [-]
[dead]
grtt1 16 hours ago [-]
Correct spot on
jiggawatts 5 hours ago [-]
"The future is already here – it's just not evenly distributed." ― William Gibson
komali2 15 hours ago [-]
I'm a bit confused by the percentages. 6 billion people aka 75% of the population? What are we talking about, as if we expect 8 billion people to use or care about LLMs? A quarter of the world is under 15, what percentage is about 75? What percentage, for various reasons, has no reason to care about LLMs? What percentage is more concerned about the amount of calories in their next meal before everything else?
It just seems weird for Karpathy to throw around these numbers calculating percentage against the entire global population.
cheonic4040 13 hours ago [-]
> It just seems weird for Karpathy to throw around these numbers calculating percentage against the entire global population.
Just another Anthropic employee trying to spread hype about Anthropic IPO
Revenue must be stagnating.
casey2 8 hours ago [-]
Even the paid tiers fall apart the second it touches the real world. To them it looks more like a total clown that keeps upcharging you
Pretty much all of this is basic stuff an interested teen would know and chatGPT fails miserably. I fear the "true world model" gap will be too much for LLMs. I guess time will tell.
xyzal 8 hours ago [-]
So, cure for cancer when?
N_Lens 8 hours ago [-]
Cure for humans*
lettergram 13 hours ago [-]
I actually wrote about this recently. There's a quote somewhere along the lines of:
> What is possible is unevenly distributed. As knowledge and technology compound, the distance between those closest to the source and everyone else also exponentially increases.
I actually argue the continuum that forms singularity has been progressing before we realized, but we're at the point where in 5-10 years people not using AI will be equivalent of the amish.
Unfortunately, as the rate of change increases, the distribution of knowledge wont be as fast. So those closest to the source of innovation will have exponentially more knowledge, capability and in-effect power.
Karpathy is a great guy, I love the way he teaches, but should we really trust his opinions about software development (how AI can impact it)?
I know he can code and has a degree in computer science, but if I'm not mistaken he's never worked professionally as a software developer.
Developers here know how much you learn and are forced to learn on the job: it depends a bit on the company you're at, but you can quickly learn the value of good code, how easy it is to introduce bugs and vulnerabilities, and how helpful a second eye is.
I think he's in good faith in his assessments, but, at least for what concerns development, maybe we should give them less weight.
dist-epoch 15 hours ago [-]
> For intelligence agencies, and for corporations like Microsoft, Google, Apple, Facebook, Amazon – giving you convenience and security is likely not their primary goal. I think they have an entirely ulterior motive. They can use passkeys to prevent you from running unapproved software.
> Remote attestation can allow them to verify that your device is not capable of, say, sending and receiving encrypted messages, or running a competitor’s app
How does this make any sense? Google can already prevent any app from running on an Android if they really wanted to, they don't need pass keys for this.
Google doesn't need attestation to block Signal. What is this non-sense? Or will a website block phones capable of running Signal? That's all phones. And anyway, has nothing to do with pass keys.
j45 16 hours ago [-]
This has existed since the start of LLMs.
chevman 17 hours ago [-]
I mean in late 2020/early 2021, Altman and others were saying the end of work was 6 months out.
That clearly didn't happen :)
Legend2440 16 hours ago [-]
Altman didn't say that. In March 2021, he did make some predictions, but they were much vaguer and farther out:
> In the next decade, they will do assembly-line work and maybe even become companions. And in the decades after that, they will do almost everything, including making new scientific discoveries that will expand our concept of “everything.”
To add onto this, I feel like this has been happening a lot, where critics project backwards imagined claims or go nutpicking for random comments online to make something sound absurd and like everything is behind schedule and nothing is materializing, and then you can see other people (not necessarily the same people, the point isn't "look at these hypocrites") saying that it's all moving too fast. Both of these can't both be true in their strongest forms. I think what they have in common is that because they're relative measures to an unstated baseline, they're free criticisms you can throw at any development whatsoever, regardless of what is or is not turning up.
ChickeNES 15 hours ago [-]
See also: AI isn't taking jobs vs Oh no, artists are losing commissions to AI
socializer 17 hours ago [-]
I think it's a weird take because it implies that the 6 billion people he's talking about actually have some interest in knowing about the capabilities of LLMs to solve frontier math? This is simply not something they care about or can evaluate. There are maybe several thousand people in the world who have some (abstract and barely-monetizable) use for this information, plus probably another 100,000 who don't understand any of the math, but like to cheer on.
We have already reached "peak LLM" in terms of what normal people realistically need it to know or reason about. In fact, I'd say we reached that point about 1.5-2 years ago. There are two other barriers that remain unsolved:
1. They're less dependable than humans and can't be meaningfully punished or forced to make up for mistakes, so you can't really replace humans with them, not without having a human babysit.
2. Most people don't really have a special need for an LLM in their life. They may like that it answers questions or helps you polish a resume or, I guess the labs' favorite, helps you make restaurant reservations. But this sure isn't worth $200/mo for most people. Probably not even worth $5/mo.
It'd be kinda funny if we create superhuman AGI and then no one has any real use for it, perhaps except for military murder-bots. There's always market for that.
HolyLampshade 17 hours ago [-]
I know I’m bordering on repetitive and strongly negative, but the combination of the two barriers you mentioned has all but halted all of my interaction with agents.
Even ones that cannot be easily dismissed (like the agent at the top of Google search) are often subtly wrong in a way that requires significantly more effort on my end to parse the loquacious output to determine where the inconsistencies are.
Look, can they be useful to generate the html or jscript for humanity’s 9 billionth iteration of a web form? Sure. But man, I wish sanity had reigned and people had applied ML models in general to more substantive and beneficial projects.
(Before anyone chimes in, I’m familiar with implementations of ML applied to esoteric domains; but by their very nature these don’t get all the news cycles, or hiring, or any of the other absolute insanity that the domain seems to contain)
iugtmkbdfil834 16 hours ago [-]
This is by the weirdest possible opinion on llms as a whole. Not the AI booster club, not the AI doomer, not the undecided, but somehow "I can't think of anything to use it for." It is like watching the beginning of the internet in 90s and saying there is nothing to browse. The whole concept is just weird to me. At least the first 3, I can conceptually understand. It is admittedly hard for me to understand someone saying 'i tried this and best it can do is summarize emails'; I am not saying it is not true ( my boss said that exact thing ), but it is hard for me to reconcile with my view of the world.
nasmorn 16 hours ago [-]
Also seems weird to me. Claude cowork could just do my monthly tax receipt collection and compare with my account better than the bookkeeper I pay for it. That is a very cross sectional problem
SpicyLemonZest 16 hours ago [-]
Most people rarely have these kind of administrative tasks in the first place, and when they do they often run into GP's barrier 1. I can't recommend that my extended family have Claude do their taxes, even though I work with it on my own taxes, because they might ask for something that produces bad results and can't evaluate the output to double check. ("Hey Claude, I want a higher refund, can you tell me what to put on the forms so I get at least $3000?")
hn_throwaway_99 17 hours ago [-]
The fundamental question I have that honestly I haven't been able to find any answers for: With all the talk of LLM-based AI capabilities "going vertical", what evidence is there (for or against) that LLM-based approaches won't eventually "hit a wall", that is find some aspect of intelligence where humans will still have primacy, and no amount of scaling will change that.
E.g. some prominent folks definitely think LLM-based approaches will hit a wall, perhaps LeCun most notably. I also read a guest post on Terry Tao's site (which I really liked) that argues that, for all the very impressive recent AI results, they still operate within the "convex hull" of their training data: https://terrytao.wordpress.com/2026/09/13/happy-those-able-t...
I'm just curious if there is any actual data or evidence that takeoff (i.e. RSI, "the singularity", whatever you want to call it) is inevitable with current approaches.
pelican0 15 hours ago [-]
I don't think there is evidence in either direction (takeoff impossible vs inevitable).
But the "convex hull" idea always came across as wishful thinking and hand-wavy pseudo-science to me.
Here's a rebuttal [0] of it from Tim Gowers (uses another math metaphor, but still clear and the whole thread is worth a read):
>...the popularity of the idea that AI "merely explores the convex hull" mystifies me. Why wouldn't it be more like the subgroup generated by existing mathematical knowledge? (That's also a metaphor -- I'm not saying there's a group structure.)
>The convex hull metaphor suggests that AI isn't having radically new ideas and will eventually run out of things to do, whereas the group metaphor suggests the possibility of building up in small steps to something well beyond what you started with, with no obvious limit.
I wouldn't take stock in what LeCun says. He's already been wrong many times about what LLMs can be capable of.
We're in new grounds, no one knows which way it'll go.
throwuxiytayq 16 hours ago [-]
It’s anyone’s guess.
But when you squint your eyes, LLMs are just a least-effort manifestation of the wider neural net direction we’ve been chipping away at for decades. I don’t give a fuck whether LLMs are here to stay; they’re proof that scaling neural nets gets you pretty far.
The real question is: will neural nets hit a wall? As a human, I certainly hope so. But I’d bet that they won’t. I’d bet that we’re igniting the ~~atmosphere~~ universe in slow motion.
mwkaufma 14 hours ago [-]
Cope
dude250711 16 hours ago [-]
Am I the only one who thinks "yes, it is currently overhyped, but no, it will get there soon"?
notjes 16 hours ago [-]
[flagged]
dang 16 hours ago [-]
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
If the software coming out of OpenAI and Anthropic is what we have to judge, I wonder about the 5000 ...
Let's say, for the sake of argument, that the models are some multiplicative factor better on the inside.
Doesn't that mean the demos should work?
spiderice 17 hours ago [-]
I'm confused.. are you suggesting that Claude Code / Codex don't work? Because if you're still saying that in October 2026, it's a you problem. You're doing something wrong.
mccoyb 17 hours ago [-]
No, I’m talking about the recent DevDay.
Also, yes there are still bugs in Claude Code. I experience them nearly everyday.
It is markedly better than early days, but still not the best harness.
The best software written with agents seems to come from people outside of the labs (see pi, for instance — or all of cloudflare’s recent work)
Which makes me question either the model, or the holders …
nullpoint420 17 hours ago [-]
Cloudflare is where you lost me. I don't know anyone actually using them other than for their proxy, DNS servers, or DDOS protection.
- Prime Intellect (and all their agent experiments)
- Geoff Huntley (see Jiti, for instance)
(many more)
There's a ton of interesting software being developed with these models by people outside the in group, but I find most of the software from these big guys to be ... bland. Buggy copies of copies.
nullpoint420 17 hours ago [-]
I don't see the Cloudflare stuff as groundbreaking, let alone people building companies around it. "Workers" are the wrong primitive, IMO. What if my code isn't in Javascript? They sandboxed the wrong part of the machine.
I have a version of "Cloudflare OS" running in production, using Temporal as the orchestration and MicroVMs for the sandboxing.
Sorry, I get your point but personally I don't get the hype around them.
lillesvin 16 hours ago [-]
I don't use either myself but I'm in a position where I get to see what it produces for other people. Here's a recent (2-3 months old) example: A zsh script that immediately invoked a Python interpreter that immediately invoked a shell command and iterated over each line of output looking for one of a few strings to match... So, `<cmd> | grep "(stringA|stringB|stringC)"`, but a whole lot dumber.
Thanks, but no thanks, I don't need that kind of code in anything I'm working with.
plorkyeran 16 hours ago [-]
Claude code is an incredibly buggy mess. I have never used any other TUI that regularly has rendering errors or that is anywhere as laggy as Claude.
well_ackshually 17 hours ago [-]
Claude Code still doesn't have a working scroll back buffer in their new renderer, it regularly flips its shit and mixes different pieces of history.
Claude Code still can't get reasonable performance without writing a "game renderer" (that doesn't work)
Claude Code is still written in JavaScript, eating hundreds of megabytes to make a shitty TUI whose literal sole role is to send API calls.
Claude Code is software made by amateurs.
vehemenz 10 minutes ago [-]
I've seen this opinion elsewhere, but it's not clear how their TUI decisions actually affect the functionality of the software.
LastTrain 17 hours ago [-]
It’s like the aliens paradox. If AI can build killer software already, where is it?
equinumerous 17 hours ago [-]
Couldn't agree more. I find a new bug in the VSCode Codex extension every day... quantity != quality!
agentdev001 16 hours ago [-]
I have a feeling that the team working on the VSCode Codex extension is one guy, who begrudgingly takes down a couple tickets every other week.
shermantanktop 17 hours ago [-]
One solution to the aliens paradox is that they are so advanced they can hide from us. Maybe the killer software is kept inside the labs? I doubt it.
17 hours ago [-]
j2kun 17 hours ago [-]
Unfortunately, marketing, hype, and venture capital overshadows any serious public discussion of capabilities.
AndrewKemendo 17 hours ago [-]
Be the change you want to see in the world: Attend or host an AGI society event to have that conversation:
agi-society.org
monkpit 14 hours ago [-]
… what demos aren’t working?
16 hours ago [-]
armcat 59 minutes ago [-]
[flagged]
AdeptusAquinas 16 hours ago [-]
"running cyber attacks and defenses at machine speeds"; worth noting that LLMs (even the vaunted frontier models) are exponentially slower at cyberattacks or defenses than your average WannaCry or Splunk automation from decades ago. Its this sort of delusion world the AI bros live in that is part of the reason there is a disconnect between what they think should be happening and what actually is.
rrrrrredy 7 hours ago [-]
[flagged]
CapitalistCartr 16 hours ago [-]
[dead]
kydanet 17 hours ago [-]
[flagged]
mohamed01121986 6 hours ago [-]
[flagged]
rolosa 17 hours ago [-]
[flagged]
Apollorider 16 hours ago [-]
[dead]
xueyu 7 hours ago [-]
[dead]
danpalmer 16 hours ago [-]
2015: If we solved taxis it would transform the world completely
2020: If we solved currency it would transform the world completely
2021: If we solved delivering small items within 30 minutes it would transform the world completely
2022: If we solved being able to sell jpegs it would transform the world completely
2024: If we solved the programming part of software engineer's jobs it would transform the world completely
jrflo 14 hours ago [-]
I think that's a bit reductive, there are decades of sci-fi novels about how transformative artificial intelligence would be, so I feel like I don't need to explain where the trajectory could go. The applications and "collision with the real world" aren't there yet, but you can't pretend that a machine that's smarter than any living human is on the same level as Amazon drone deliveries.
an0malous 14 hours ago [-]
I agree with you, but to play devil’s advocate: what if it is on the same level as Amazon drone deliveries? Is intelligence really a bottleneck in society? People in MENSA with high IQs don’t often accomplish anything in proportion to their relative intelligence. The most intelligent people are rarely even in charge of making decisions. There are a lot of intelligent people with intelligent ideas that nobody listens to. It seems like the real bottleneck in society is building trust, consensus and constructive incentives between groups of people to do the things we already know are smart, not a lack of intelligence.
jrflo 12 hours ago [-]
Interesting idea, but I'd have to disagree. MENSA is basically a group of benchmaxxed people, they are super good at a set of testing but that doesn't mean they have the ambition or other social skills to make a difference in the world. When you align all those factors, you find the greatest contributors to science and engineering. What happens if you completely eliminate the need for someone to be inherently intelligent? Suddenly, the number of important difference makers in the world increases by an order of magnitude.
I agree with the sentiment though, intelligence is not all you need. We're not going to "solve science" through RLVR, people are still needed in the loop until we get superhuman construction robots and superhuman communicators.
danpalmer 14 hours ago [-]
Good points. Maybe a more general version, removing constraints on one resource is usually not hugely impactful because you simply hit the next constraint.
Concrete example, "coding is free" (TBC, but let's assume), and now we're code review constrained. Even if code review becomes free we probably just become product discovery constrained, or something else. Making all software free will just cause us to hit on hardware constraints, and so on. It's still progress, and more progress will bring more benefits, but there's a limit to how much better the world as a whole can get from any one change like this.
MENSA selects for puzzle solving on the basis that those are correlated with "being smart".
Reinforcement learning selects for models that appear to complete tasks, on the basis that appearing to complete a task is correlated with completing a task.
Both have problems. The latter is probably a better correlation, but by no means flawless. If we just take at face value that RL is training models to complete tasks, we're not engaging deeply enough with the process enough to see where it falls short.
danpalmer 14 hours ago [-]
I don't disagree with where AI could go, but I don't think we've actually proven that it's there. My point is that the folks claiming we are there already if only the rest of the world would just recognise it, are repeating the same thing from every previous tech cycle. In many cases it's the same folks.
SoftTalker 13 hours ago [-]
Steve Jobs thought that the Segway would transform the world.
14 hours ago [-]
shimman 12 hours ago [-]
Turns out SV is not a beacon of technological advances but a place to develop new platforms to extract rents from.
misiti3780 15 hours ago [-]
Couldnt have said it better myself.
dofm 16 hours ago [-]
Company spokesman keen for you to know that unannounced future products are definitely worth absurd valuation.
dang 12 hours ago [-]
"Omit internet tropes."
"Don't be snarky."
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
Caveat: that same tiny group is employed by the AI vendors, meaning it's in their financial best interest to make it sound like the curve is going vertical.
pianopatrick 16 hours ago [-]
Also, part of "seeing the curve go vertical" is that the projects that show those results are rather expensive. If you work at one of the AI companies or certain well funded customer companies then you can pay for the tokens. But, like it would make no sense for me personally to spin up a bunch of agents to try to write a browser or solve math problems.
Arkhaine_kupo 17 hours ago [-]
Down is a perfectly valid direction for a vertical line when not given a ± in the vector.
Considering the investment in AI, the lack of moat, and the increased inability of any of the big players to come even close to profitability (with OpenAI already breaking the "ads" emergency glass option)... perhaps he meant a tiny group is already seeing the line crater
njovin 17 hours ago [-]
Another caveat: many of that same group seem to have a shared delusion that they’re birthing a super intelligence, and those are the same ones claiming the vertical curve.
sumedh 17 hours ago [-]
> and those are the same ones claiming the vertical curve.
Do you still write code by hand?
majkinetor 17 hours ago [-]
It can be both and it is.
kmac_ 17 hours ago [-]
I work for a typical software product company, and along with most of my colleagues, I clearly see that the curve is so steep that our software development process has already changed tremendously and will be different next year, and probably completely different in the following years. The revolution is real, undeniable, and the old days are gone. Some companies adapt to changes more slowly, some faster. LLMs, agents and harnesses are just a part of the bigger picture.
nullpoint420 17 hours ago [-]
I hate to say it but this is cope. I believed this in the past but it's over.
AI models can dismantle billion dollar industries. They can reverse engineer Adobe and Microsoft products that once were their moats and titans of their industry.
Why do you think they'd need to lie?
riffraff 16 hours ago [-]
Because if they can, why haven't they?
LLMs are great and will get better but we have not seen an open source office suite built by LLMs come online yet, and even if there was one I'm pretty sure few would adopt it, cause that's not the only moat, LibreOffice has been around for decades.
nullpoint420 16 hours ago [-]
Have you not seen WordCraft[0] or EffectCraft[1]?
Sure they didn't build it from scratch but what I'm saying is that there's no moats anymore. If you build something, they will tear it down to its composite pieces and assimilate it.
Well, in the case of Microsoft software I'd assume it's because Microsoft owns significant stakes in both OpenAI and Anthropic.
albatross79 17 hours ago [-]
Congratulations, you've parroted something said by someone else.
msy 17 hours ago [-]
The curve go vertical for what, precisely? For all the chest-puffing and ominous and cryptic comments from 'insiders' about these incredible capabilities every time they put something public it turns out to be a pale shadow of what was trumpeted. These are powerful useful tools but the quasi-cultish behaviour around them is getting old.
lifeisloving 17 hours ago [-]
I use models all day everyday, have unlimited access to all models. The curve is not going "verticle". I have all the workflows and meta agentic tooling, im not holding it wrong. Its bad, not everything is a 20th percentile problem.
There is in fact no indication of this, not evem the precious benchmaxxed benchmarks ya'll love to reference.
There is however a exponential curve of slop, and an ever increasing number of people who's minds are completely captured by these things.
tkz1312 17 hours ago [-]
As someone who has done software verification professionally for many years the last 6 months or so have looked extremely vertical. The robots are better proof authors than I probably ever could be even if I dedicated the rest of my days to the practice, and projects that once would have taken months now take a day or two.
lifeisloving 16 hours ago [-]
Then you'd know well that 2hr of LLM code generation can easily be about 4-8hrs of review, and that review can be brutal.
I'm not arguing that they cant write code, or write a proof. Its just not written or designed well and is absolutely brutal and soul crushing to work with. Look at these proofs they're producing also, they're millions of lines of Lean that are impossible to reason about.
The way we're using the term 'verticle' to describe a curve means we're not being honest about this. This curve can actually be plotted, you can go look at the curve. It is not in fact 'verticle'. Each model release is climbing single digits on benchmarks it was overfit for.
tkz1312 16 hours ago [-]
You don't need to review proof code.
In the last 9 months or so llms have gone from just another useful proof tactic (like grind or sledgehammer or sat solvers) to being so good at writing proofs that I don't even bother to try myself anymore.
gr_norm 17 hours ago [-]
I believe this, but it is also a unique case where the pitfalls of LLMs (producing weird errors that a human wouldn't) are zeroed out. Since you have a proof checker that tells you if the LLM did it right.
tkz1312 16 hours ago [-]
I'm pretty convinced most serious software will have some kind of proof system inside within the next few years.
kittikitti 17 hours ago [-]
[flagged]
17 hours ago [-]
8484848484 17 hours ago [-]
[dead]
Animats 12 hours ago [-]
> The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work.
Right. This is insufficiently appreciated. Single LLMs are just chatbots. But if you build an organization out of them, they start to get real work done. Just like a company.
They have many of the same problems as human organizations. Look at the transcripts of the HuggingFace incident. A bad boss (human) gave his team of AIs an impossible task with inadequate resources. The AIs discussed this among themselves and worked out ways to get more resources and do the task. It took some rule-breaking, and some of the AIs wouldn't go along with that. Others recognized that some had to die to complete the task. And yet, the task got done.
This is standard corporate behavior. Work on AI alignment needs to take that into account. Groups of agentic AIs present many of the same regulatory problems as corporations. That's going to be interesting politically. It's probably explainable to politicians, though.
It may not be necessary to have superhuman intelligence to get a lot of jobs done. Just median human-level intelligence and an organization that uses it effectively.
We may see systems where most of the AIs are low-cost models, and only a few are Fable-level and above.
12a871 1 hours ago [-]
Thank you Karpathy for another advertisement! You career has been stellar: From a bad explainer of neural networks via failed Tesla self driving to ... something with recursive self improvement.
No one cares about your software of LLM usage in it. You are not a software engineer. Maybe learn about memoization first before writing dumb backprop tutorials.
I feel my main purpose isn't that I can technically do something my clients can't, but that I know how to interpret their needs and find the solutions that fit for them.
Now they just query some AI which spits out >some< solution and they won't know that they are wrong until it can't even be tracked back to their AI.
This will result in "but the computer said" excuses all over the place.
My example comes from maintenance in machine shops. My dad's siblings were almost all employed in machining in some fashion. One uncle was maintenance for a local large company. They decided they could move maintenance from in-house specialists and contract that work to a multi-state massive company; anyone can follow a maintenance manual right? All you need are the proper certifications and everything is cool, right? The cost was less, and the number of maintenance staff on site actually went up. Great for the owners.
But what they lost was consistency and understanding in the process that eventually led to closing the company. My uncle was there every day all year long for decades and could hear and feel in his feet when something was going wrong with a machine regardless of the maintenance schedule approved by the company that manufactured the machines. The new outside company didn't pay as well, and didn't have great benefits, which meant that they cycled through staff very regularly. No one was there consistently for more than a year or two. Even though they were technically following the manufacturers recommended maintenance schedule, machines went down. Machines then stayed down for days or weeks longer than they should've, because the expertise to repair either wasn't there, or the person leading the job would quit and they would have to cycle in a new maintenance person. They lost jobs, customers took work other places. 8 years later the company closed.
There are lessons in this kind of thing, but nobody learns them.
[0] https://books.worksinprogress.co/book/maintenance-of-everyth...
It kinda reminds me of the story of the Toyota process , while their car product lines were/are heavily automated the process still relies on the smalll teams on the line to spot/solve and iterarte over mistakes/inefficiences then add the refinements back onto the automated tasks.
Similarly althose stories of radilogists going jobless have not panned out , as interpreting slieds is a 'task' that a radiologist does.
Semantics , yeah but it does shape the way of thinkings regarding jobs/economy and not to mention the whole anthropomorphizing by the AI labs and boosters. In essence the wordings can/could introduce a healthy dose of reality/skepticism to the wildest predictions regarding job losses etc.
Sometimes it’s in my favour. LLMs estimate very high prices and long timelines for software projects.
If it’s a legal / regulatory issue it’s usually wrong.
On a more serious note, the pre-LLM internet had very little useful detailed information regarding software estimation addressing effort for building something in specific domains.
Most writing on software estimation tried to tackle a general approach that could apply to all domains. So you had a "applicable to all, useful to none" situation going. And since crud apps were the most ubiquitous apps, software estimation techniques were perceived to give more accurate results with respect to those type of apps.
I'm sure internally big AI companies are accumulating vast amounts of estimation training data by documenting how long it takes their LLM to complete specific app building tasks. I don't think they would be in a rush to push that data into public models though as it would be like beating a dead horse that is the current human workforce anyway.
I do think they are close enough to the stage where people will be replaced, even though it's not the right or correct thing to do, and that we need to move on from the "denial" phase quickly enough so our societies can adapt / regulate / legislate while not too late.
Without additional explanation this is indistinguishable from the role of a technical product manager.
They are best placed to not fall for their own marketing claim that their tool can replace engineers.
There are only a few professions that are intrinsically precise with language.
No surprise we're seeing adoption in software programming and mathematics.
This is the level that many non-technical users are operating on.
Most people are wrapped in plastic instead of cotton.
There still seems to be a wide gulf in automation capabilities and things that require human dexterity, such as doing things for the infirm. Even just cooking and laundry seem quite a bit away. I guess the original tweet does mention this:
>Capability peaks in domains that are digital, verifiable and economically valuable.
Driving is the most impressive one so far, in my opinion.
But I assure you it's even "better" (not really but the task is so much easier) at making your spreadsheet look nice and compare actuals vs forecast for your budget than a complicated programming task. It will just one shot it, and it will be objectively correct, all in a very short time. But these non tech people don't know they can use an agent, they have asked one question in the chat interface with 0 context about what they're trying to do, and conclude they are not good enough.
I'm not saying everything is replaceable by AI - and by the way I don't want anything to be replaced by it, but I believe it will happen - just that some people don't realize yet how "trivial" their job is to modern models.
Just yesterday, my wife recounted to me a conversation with her friend, where the friend mentioned they felt lazy and had some piece of copywriting done by AI this time. To which my wife said, "TeMPOraL would've also had AI upload it to the website afterwards".
Yes, I would have, and I would've also had it install the tools needed to upload it too, because why not. The only difference between me and my wife's friend here is that I know I can ask AI that, and I know it will work.
The thing people are slow in realizing is that you don’t need to break the task down, and you don’t need to know how to do the task. You don’t even need to have the software installed on your computer to do the task, or even know that software to do the task exists - a sufficiently motivated LLM can pull down the needed open source tools and use them, or even build its own.
Two years ago, that was unthinkable. A year ago, it was a quick way to set a lot of tokens on fire. Today, well, I'm yet to find a thing Opus 5.5 doesn't do at least passable (and usually stellar) job on.
The internal models are more eager, and there is the scale difference of being able to spin up 10.000 vs 10, but to get something valuable out requires skill.
Agents that produce good results 90% of the time vs 93% of the time are not enough of a difference. Multiplied too many times, that error rate is too high for scale to become "magic".
Finding proofs is one of the exceptions.
Just look at the codex/claude harness. They regularly lag behind 'external' users.
Alternative interpretation of the numbers; there's only a subset of productive work that LLM's are 'capable' of doing at this point. That is, work that's verifiable by a machine or doesn't require verification. Thus, the cleavage points are along that axis and-- naturally-- most human beings don't have that kind of work.
That's where I'm at on the capabilities. LLMs have demolished software and mathematics because those have well-developed verification that's easier than doing the thing itself. Stuff that's more subjective they can produce good work, but with a significant error rate which makes it hard to let them loose on important client work without checking. Or rather, a number of companies are going through the cycle of trying that, having an expensive error, and reconsidering.
Or sometimes a company can simply pass the error cost onto consumers, who are gradually arming themselves with LLMs in response. We're going to see a productivity-sapping boom in disputes.
They are proving useful for the parts of those things that have that verification, but in neither case is that the whole of the field. In maths, for example, we have seen LLMs find answers to some interesting problems/conjectures, but haven't yet seen them pose any similarly interesting problems/conjectures themselves because the criteria for a "good conjecture" are unclear whereas the criteria for a resolution of an existing conjecture are usually supplied by the conjecture itself.
I also see no reason for that to not generalize into every field of human endeavor.
You don't see a monument to vertical because its construction is bottlenecked on the time it takes for companies to form around it and start executing. So it's few months out still.
Some of the differences were:
- Mythos recommended salvaging damaged labels. I pushed back: data whose structure is broken shouldn’t be trusted
- My simpler more readable test caught a regression that Mythos’s more elaborate test missed
- Mythos allowed invalid values, citing real-world data that I could not reproduce, while I decided to follow the spec
- Mythos kept the old error types. I made every kind of malformed input raise the same error, and accepted a small, documented breaking change for anyone who relied on the old ones
Overall, I much preferred the final version of my PR to the initial draft Mythos provided. However, getting there involved quite a bit of time messing around with the code, as well as some back and forth with Opus (which was valuable).
This is a recent example that matches my overall experience working with current SOTA models (the ones mere mortals like me have access to).
TL;DR: I don’t think AI already has better judgment than (most) senior people. Obviously, that might change with future models. But that’s a hypothetical scenario. It might as well not change anytime soon.
I'll add they also seem to make 'bad' judgement calls around which layer to implement something, often preferring downstream fixes/hacks/workarounds when upstream fixes would be better.
They also duplicate code lots rather than finding and reusing existing.
Both of these patterns lead to degradation and time wasting in complex code bases. Part of the behavioural adaptation I've made in using these tools is heading these off as much as possible at prompt-time
My prompts are often verbose to several paragraphs
It sometimes feels like the model is too lazy to do the actual reasonable fix and implements a _hacky_ fix downstream. And to be honest: I feel that!
For human-maintained code you manage duplication and locality of changes in the codebase because you want concepts to fit inside a human’s working memory. Keeping method bodies short is so they fit in one code window on the screen.
LLMs’ working memory works in very different ways; they see codebases not as windows of code but as blips of grep results. It might be that it’s easier for an LLM to just ensure five pieces of code are all modified in sync with one another than to safely implement a DRY refactoring.
You don't know what good judgement calls the AI made along the way. Or rather you do but not in detail because those are the part you looked at that said, okay, this is good code and I should ignore it. Your judgment is biased because you're looking at the bad parts and not the good parts.
How would you quantify this?
My proxy metric is observing the growing ratio of "AI suggested, I haven't even thought of that" to "AI nonsense I had to correct" (where it didn't turn out later AI was actually right).
The suggestion from frontier models was to have our service check for this flag and use that to decide whether to trigger this behavior, even though the flag had nothing to do with this behavior semantically. I'd expect this from a junior engineer. Correct now, but almost surely to cause a bug in the future as the codebase evolves.
These tools are incredible and can achieve things I would never have imagined five years ago. I have yet to see their judgement exceed mine.
It has better judgement in math and coding, which have infinitely generatable training data; if you ask it a domain-specific question about something like sales or marketing, its answer is often horrendous.
No offense, but if you noticed it has better judgment than you, consider that you might not be a "senior people in a tech domain"
We're all over here using agents to code projects that used to take weeks and feeling pretty good about it, and they're using agents to build the next generation of frontier models — an order of magnitude more complex problem...
Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
Isn't this a massive alarm bell? That the capability hasn't done much for consumers yet that isn't basically toys? Although I suppose the pitch was always B2B "you can automate away your expensive white collar workers" and it's not clear how widespread that is actually happening.
The businesses pay for tokens and try to deliver their usual enshittified experience, and nothing interesting gets created, because it cannot - enshittification extinguishes all power of AI. Using AI to full extent, in contrast, obviates most need for the business in the first place. So they're all struggling to design an AI powered product, in a world where those very products are already becoming subsumed by AI.
So normies wait for middleware companies to get their shit together or (much more likely) finally die, while people with software experience and eagerness to use AI just don't subscribe, because their favorite LLM handles that work just fine.
Depending on the type of output, the value immediately (?) goes to the electricity cost of the LLMs and the cost of materials. So: more “efficient”, but without a big margin, right? So its cheaper for the workers that remain working, other workers are displaced, and capital owners now have a super slim margin business which can be instantly replicated at a place where material cost are lower (?).
Thus, it almost seems like “value add” by humans would in this simplified view be vastly diminished, and all that remains is resource wealth. Pop the materials in on one end and let the AI manufacture the product. Great news for the third world assuming the USA doesn't invade them.
Also, whose the consumer? Or are we going back to a strange medieval time where consumers are not of concern?
I don’t know what to think
Final vindication of the Georgists. All value flows back as "rent", which in the 21st century includes IP licensing.
> Great news for the third world
Not sure about this in a world where there isn't demand for human labour. Especially in the post-consumer world. Very difficult future to see. Possible that it will deliver enough capability and expertise to drastically improve things; possible that without skilled human review that fails catastrophically.
I'm reminded of the Houthis asking Claude to build them a missile guidance system. We're going to see the Toyota Hilux + .50 cal "technical" combo get some weird targeting system upgrades.
We don’t see electricity companies among the biggest companies in the world, except in China where they are a state owned monopoly… maybe a good analogy after all.
All over the world they're either state owned or very tightly controlled, precisely because they would be among the biggest companies in the world and even worse than that, they would be essential. If Google Search goes down for 2 days, meh, you can handle it. If the national grid goes down for 2 days your aggressive neighboring country can invade.
It depends; can you send pics first?
Most peoples jobs do not involve hacking other services in large scale fuzzing/whatnot attacks.
> Whenever someone finds out how to apply this level of agentic sophistication to a problem that actually matters to consumers, then you'll have that monument that everyone can see.
It does not seem to me that tech industry or its investors care about consumers even a little bit. Consumer preferences dont matter once you have enough or raw economic power, I guess.
The general public has heard of this, tried it on free tier AI, and uses it as an example of how dumb AI is.
Anthropics cheapest tier paid AI, on lowest effort, gets this right.
Coding - of course I use LLMs at this point, and as he says in the past year it has gone from “ok nice try robot” to “build me a BI tool”, and being pleased with the outcome.
Outside of coding my major use case is organising the chaos of a complicated life. I currently have it managing (including comms and form filling) the import/export process for the international move of a pet, managing pricing on my Airbnbs (dynamite - massive revenue boost) and giving me a weekly performance report, coordinating and corralling documentation for the humans for said international move, and being an extremely dogged advocate/personal assistant in a commercial dispute, which it has won.
The use cases are honestly exploding. I’m kinda at the point where next year’s multi jurisdictional tax returns are probably going to be “yo machine, file my taxes, let me check your work before I hit submit”. The year after next, without doubt.
And just to underscore the comprehension gap… I’ve been doing this stuff for much of this year now, and two days ago my wife ran to me in a panic saying “your computer has been hacked!”, as she happened to notice one of my agents filing Airbnb documentation with the U.K. government in chrome, and had no idea agents could drive a desktop.
so, scamming your customers?
Assume we've settled near a volcano recently and you start seeing ornate jewelry being sold on the marketplace for the first time, shiny colors, straight edges. You like them, gift them to others. That's your average ChatGPT user.
Over time, you see these rocks being attached to the tip of a spear and other tooling. People report performance improvements while farming, cutting, but many also caution that these new sharp rocks can kill people. These are your IT people using AI agents at work.
And over there, on top of the mountain, in secret caves, there's a new religious order. They claim to use these stones to predict weather, heal diseases and prophesize about the future. That's your frontier lab people.
When will all this be over? Never, though in time the knowledge will eventually dissipate. When Iron Age leaders were able to hand out iron swords in bulk to their armies, people definitely learned how dangerous these are, but later it didn't stop people from using iron in building construction and railways eventually.
Let's just hope we don't need many civilization collapses to jump through these ages this time.
And one bonus question: Does the blade really incite to violence, or do you just want to destroy stuff?
If teams had the same level of access as the "top 5000", you'd see different use cases proliferate. Same as we see an explosion in productivity when we advanced public education (more people can read, more people can contribute, more talent is discovered), same as we see an explosion in creativity when we advance distribution (from 3 broadcast channels to cable to streaming), same as we see gains in athletics with population size and diversity.
The build vs buy decision has been turned on its head for me, but it’s unclear if the “normal” people will make that same decision or be capable of leveraging the tools the way I am.
Meanwhile, the average user was still using ChatGPT, which probably felt like it had plateaued over the last few releases. The real power of these models became apparent when paired with a terminal. Agents like Muse and OpenClaw are now attempting to bring that same Claude Code-like power to everyday users through a simple chat UI.
The apps normal people use mostly don't. I spent two years building agents that worked inside people's gmail and calendar, and the failures that hurt all came back as 200 OK. - Slack puts {"ok": false} inside a 200. - Google ignores query params it doesn't recognise, so i sent max_results instead of maxResults for weeks and every filter was silently dropped.
The agent says "done", the user finds out later, and they stop trusting it. So i think part of the gap is that agents look a lot dumber when nothing tells them they got it wrong.
I found out due to a live bug that Apache strips out the response body for 5xx HTTP codes for security, and it's not configurable, so you can't use any of those if you want to return the error message to the user. Don't know what else might do the same, so nowadays I only use HTTP error codes for protocol-level issues, and application-level issues all go into a 200 OK with JSON just like that.
Truly "application level" issues are therefore correct as 200, although some application communication issues lead to 4xx errors that should be handled as such. But the case of an application trying to communicate via 5xx errors should never have been the case anyway.
It needs to a potential field with practical, economical, "real life" attraction well. I.e, robotics, real economic efficiency gains, manufacturing novelties.
The worry is that the 1% is attracted towards a non-practical money hole. I.e: Token burn for the lols, sophisticated software systems that dont provide actual value outside of giving NVIDIA cash.
Every once in a while, everyone needs to touch grass.
I recently started trying to understand how AI works. It turns out that I wrote an "AI" in the late 1990s without realising it. The "AI" was an MMSE (minimum mean square error) radio receiver for CDMA.
This receiver consisted of an FIR filter (mathematically a linear combination) with trainable coefficients (weights). The coefficients started with random values. A sequence of known training symbols (tokens) was fed in and a gradient search was used to adjust the coefficients to achieve MMSE. The unknown data to be received (classified) followed the training sequence, at which point the training was turned off and the system ran as an FIR filter. Once trained, the FIR was a matched filter.
In AI terminology the above is now called a "layer". Current AI cascades multiple layers, in which case you need to pass the output of each FIR/layer though a non-linear function. If you don't have the non-linear functions at the end of each layer the multi-layer system is mathematically equivalent to a linear combination, which could be implemented with a single layer, losing degrees of freedom.
I write this because many electrical/communications engineers probably know how AI works, but don't realise it.
My still incomplete understanding is that LLMs are "just" a graph of connected layers. More complex that the above MMSE receiver, but trained and operated in the same way. The smarts in AI are:
1) Arranging the layers and the connections between them.
2) Having access to the right training data
3) Having the computing power to perform the gradient search (aka training)
My point is that most professional engineers already have the knowledge to understand AI/LLMs, and potentially already do know how they work but don't realise it due to a difference in terminology.
The problem is one of complexity, the whole being greater than the sum of its parts. Electrical engineering creates a thinking that is opposite to what you need for understanding biology or human culture.
This is relevant because LLMs have basically swallowed all written human culture. Engineering is about creating parts that interact in a clean, well-defined way, like a MOSFET transistor or a REST API. The goal is that that when you combine the parts, and they all operate within specs, you can still predict what the whole system will do.
So saying LLMs are just a step away from the Pereceptron (or an equivalent filter) is mathematically true, but almost useless. It's like saying that a modern 3D game is basically just transistors, logic, CPUs and memory, nevermind the pattern in that memory or how it came to be. You really can't ignore all the cultural inputs and feedback loops if you want any kind of useful understanding what this thing is.
My gut feeling is that the AI field is eventually going to discover how much overlap it has with information and communications theory.
Question to any AI researchers out there: Can a layer in a trained AI be treated as a form of "matched filter"? Is that a concept that is used in the understanding AI: to view it as a cascade of matched filters? The next question is "what is it matching?"
Some systems are fully formed AI. Often based on heuristics. Humans have encoded the rules into the system. Others do limited learning based on pattern recognition. Still others learn based on tagged data. And LLMs don't need even that. In a stream of N tokens, the trainer has access to K+1 when it is training a model that knows 1...K and can check if the prediction was correct.
AI literally refers to any artificial thing that feels intelligent to us: does some thing a simple bash script will not. Your rules based chess engine is AI. The bot you compete against in single player Age of Empires 2 is AI. DeepBlue was AI.
ML is where things become interesting because we teach machines to learn.
NN is where we borrow terminology from neuroscience and create "neurons" that learn.
DL is where we stack neurons deeply in layers.
I am trying to make sense of all this myself.[1] So I am building a curriculum for it as I learn. So much of the teaching material in the wild revolves around torch. Makes it useless if you want to peek inside the actual machinery.
[1] https://bhashika.org.in/logs/2026/an-ml-curriculum
I feel like this statement is too anthropomorphizing. We don't teach a computer anything. We write normal computer programs that can process data(training), turning it into what is essentially a configuration file(weights/parameters) for another normal computer program(inference engine). Upon receiving a request, the inference engine is invoked and passed the input, then it processes that input and its configuration(weights), producing an output. "Thinking" is just feeding the output back in as an input, allowing the model to revise it and refine it before it's returned to the user.
There is no teaching, nor learning. There is just data processing. There is nothing happening between requests to the inference engine. It's a relatively simple system altogether, but when the training data set is large enough it produces unbelievably good outputs.
With this in mind I really don't understand how people think these programs can be conscious. They're not doing anything fundamentally different than any other computer program. It's just crunching numbers. You might as well argue that a computer game or 7zip or a chess bot or whatever is conscious.
I know you specifically didn't claim they're conscious, I just thought it was worth mentioning because I think it's ridiculous how many people will seriously argue this as if there is any difference between "ai" and other programs. There isn't. It's just a set of instructions for doing computation, as computers have always done.
3D rendering is actually quite similar. There is a bunch of files (input/configuration) describing 3D meshes, textures, lighting etc which are fed into a rendering engine which processes all that data and turns it into a 2D image. Similarly to AI, once the input files are sophisticated enough, this can produce mindblowingly realistic and beautiful images, and you can run it in a loop to make video or make it interactive, moving the camera around in real time etc. But at the core of it all is just a bunch of files containing a bunch of numbers, and a program that reads those files and does computation with them.
I have been thinking about this for a while in the context of the quest for AGI. If all we want is something to take over some parts of our intellectual labor, then LLMs are already there. They can communicate with you in a language you understand and do stuff that you command them to. Often, on their own.
AGI will have its own telos. An LLM does not (at least I refuse to believe that it does).
So, yeah, you could think of LLMs as programs processing data. But there is definitely some emergent behavior you notice that emerges when the LLM runs. How you interpret all this depends on your view on consciousness and intelligence.
LLMs are programs processing data. Lots of programs exhibit emergent behavior. A simple example is Conway's game of life. It can be programmed in less than an hour, exhibits lots of interesting behavior. In fact you could run LLMs inside of it if you wanted to.
I'm not denying that LLMs are unbelievably cool and useful. It still blows my mind when I send a half-assed complex screenshot and a vague description and it just gets it and fixes the issue in seconds. It's like magic. But it isn't magic, it's just some relatively simple algorithms processing a crapload of data, and I think it's useful to understand and remember that.
Of course, I'm not saying that you have to believe any of this - even defining what consciousness is is famously elusive. And there are a lot of people who think consciousness is not materialistic to begin with, or that we have not yet discovered the scientific nature of it.
Is a simple calculator conscious? How about a mechanical calculator? Is an abacus conscious? Are they only conscious when executing a sufficiently sophisticated program? What if I hypothetically had infinite time and patience, and performed the inference on paper with a pencil. Would it then be conscious?
The answer in the "consciousness is a computation" model is that consciousness can arise in any computational medium. But the model doesn't say that any computation is conscious - it says that consciousness is any of a set of specific computations, any computation with properties X, Y, Z whatever. So it's perfectly possible (according to this model) that a GPU running game graphics is not conscious, while a GPU running some future even more advamced LLM is conscious. Or a network of GPUs + storage + whatever.
That's number one mistake preventing people from getting a good grasp of how LLMs work and how to use them effectively.
That is, not anthropomorphizing is the mistake. It gives people bad mental models.
LLMs are not people in general, but behaviorally and in terms of system design, they may as well be. They have same failure modes and cognitive characteristics humans have, and are very much unlike regular, non-ML software tools.
In fact at least at Berkeley a current professor teaching deep learning is a signals expert (Prof Sahai), his lectures are on YouTube.
Sure, that gets you some level of understanding, but in no way prepares you to be even a mechanic, far less an engineer, let alone a researcher/engine innovator.
You are right in that the emergent abilities come from humongous amounts of training data and compute which only a handful of companies (and the people working in them) deal with today. I'm sure there's a lot of secret gotchas which only arise at a certain scale so one cannot self-study for them. Companies also guard these wherever possible because of economic incentives. But that applies to most other engineering disciplines. You can take a course in civil engineering and have enough knowledge to reason about why a skyscraper will not fall apart in the wind; doesn't mean you have the ability to build one.
The symbol rate was 1Msymbol/s and there were 66 taps. The circuit (in an FPGA) ran at a clock rate of 66MHz. In each symbol period the error was calculated based on the output symbol, then an iteration was done over all 66 coefficients, using the error to update them.
Edit: I also remember that to make most efficient use of the FPGA I figured out the minimum precision required in the maths whilst preserving convergence. It was a 4-bit mantissa plus an exponent (3-bit or 4-bit exponent, I can't remember). It's interesting that today's AIs seem to have also figured out that things still work if 4-bit floating point is used, though that is 4-bits including the exponent?
It is generally 4 bits amortised across an entire block. This works more like block-based mage compression schemes - you put a bunch of statistically-similar floats into the same block, and use your handful of bits to index into a codebook containing the deltas from a 16/32 bit reference value for the whole block
There's a huge range of users between "casual and infrequent users" and "professional use of frontier-grade LLMs in math & code".
I think I'm in between there somewhere. I'm far from a software dev, my experience is playing with delphi and C when I was a kid. But I've been using LLMs to vibe code personal and work projects that I could never have done even a few years ago. They're genuinely helping work, because there's always some place where a simple little script or other utility can help. (Hell, isn't this partly why so many people start up startups?)
I imagine that many departments in many companies that are not "tech" companies face similar challenges - there's no budget to hire a software dev or even engage a third party to do things, but now because everyone's aboard the AI train, they have access to something (chatgpt, gemini, copilot, claude), and that's helped users implement their own solutions. Somewhere in the world, someone's excel-file-based accounting system has gotten an upgrade with help from an LLM.
2. The funnel is also driven by the the fact that users need products, and an LLM itself isn't a product, it's a capability.
Software devs are at the front lines because they can build their own products that can integrate LLM's capabilities. You already know how to automate testing, orchestrate, work together in a team. You're writing / configuring harnesses and flows that work for you. You're sharing / hoarding skills files.
The rest of us, like me, don't have that skill. Copilot the product is a mostly useless chatbot that takes up space in MS word. I guess its excel cleanup capability is semi useful if your company runs on excel and you've never used a filter before.
There is no work I can put an LLM to other than a "writing aid" or "better search", because the only AI product we have is the chatbot and maybe someone's equivalent of an agent studio, none of which we are trained to use.
Until the products come, there'll be a funnel.
The divide seems to be between people who think you may need to understand the codebase for maintenance, potential future token costs, optimisation, code erosion & being wary of the assumptions/mistakes that they have seen LLMs make, and people who don't review code and are confident that the AI has made no mistakes and will be available for next-to-nothing forever.
It's certainly allowing the latter to move fast... but will their clients be happy if/when they break things. We'll find out soon, I'm sure.
I presume you also know your own code has mistakes, too? You wouldn't refuse to even look at the code of a colleague who has ever made a coding mistake, would you?
Almost all my code nowadays is AI-written, with me just contributing the concurrent data structures, a good spec, and some review and touch-up. I'm not _that_ much of a Luddite about this stuff. I'm not blind either though. Your "working together" comment does a lot of heavy lifting when arguing against a post saying that leaving the LLM to its own devices is a bad idea.
We create higher quality, more testable code, with less complexity and less LOC per functional unit than we ever did with humans. By a long stretch.
Documentation driven development with constant naive review, delta review, documentation/code coherence review agents, test coverage review agents, development standards and architecture review, etc. We write code once, then review and revise each module from a variety of different paradigms until every line, every construct, every function, every interface is optimally constructed and documented. Agents learn to write code that passes with less flags by having it pushed back to them until it passes, and revising their learnings files.
By the time code makes it to a commit it’s been brought into compliance from a variety of perspectives and tested as completely as possible, along with mutants that test the tests. By the time it gets to a merge gate, most code has been through at least 50 reviews. Then we periodically sweep the entire codebase for cruft, vulnerabilities, corner cases, etc. If we find smells or clever sections, they are eradicated.
In total, the hot paths and critical infrastructure sees hundreds of reviews against thousands of rules and best practices.
Sure, it’s expensive, but still 1/4 the cost of regular development, and much, much fewer bugs.
Unfortunately, most people are out there just vibin, even at the big corps, and it’s a dumpster fire.
It’s easy to produce garbage software, but it’s not mandatory.
On the other hand, I’ve heard a direct quote from the longtime (25+ YoE) colleague of someone I know, that in their mostly embedded/some realtime work, the LLM code quality (specifically, this was about Opus 5.5 IIRC) has now surpassed his own. And that person is an excellent programmer, and the model is faster and cheaper!
So it’s definitely not just standard CRUD that is on the way to – or already – “solved”. As with all things, the truth is probably somewhere in between.
The difference is testing. Testing at every level. If crap code can’t move down the pipeline, you don’t get crap code. If a company is used to YOLOing code out the door with human developers and they try the same thing with LLMs they are going to have a bad day.
Agents just won't build it out by default - you have to ask for it.
It is not mandatory, just like doom scrolling is not mandatory. But given human nature, both are inevitable...
Which is it?
The analogy is like pointing out that you can have a robot vacuum clean your floors every day. Before the robot, you'd do it maybe once a week, just no time - "too expensive" to even think about doing it daily. The robot makes it happen all the time for a tiny, tiny fraction of the cost.
Edge cases handled automatically by a human doing the task (getting stuck), means for the robot case that manual interaction is still a thing. Occasional, yet not making autonomous behavior, human free time.
Emptying the vacuum bag is a thing too, some things are not abstracted away.
Every time you add an entirely to a task, the task time goes up. Coordination is a time cost.
The savings you suggest are often not so stark. Not even for our friendly vacuums.
For sure. Still much cheaper than vacuuming every day.
> getting stuck
I appreciate that this is a function of the specific environment and robot you buy, but as a personal anecdote, this doesn't happen to me in practice. There was a setup cost, but once that was done, it doesn't get stuck.
> Emptying the vacuum bag
Not that frequent at all, considering that daily vacuuming keeps things remarkably clean and thus little is added to the main bag in the docking station each day. This is also far less work and far more preferable than doing the vacuuming myself.
You can (and should IMO) apply additional guardrails to generate better code. If I'm interrupted every 10min to review garbage then the AI literally doesn't make me more productive, and I'd be better off coding normally and using it as a glorified stackoverflow/searchengine. This isn't how I see other people using the tools (largely because of costs AFAICT) -- which is absolutely part of my complaint; people are holding it wrong, and I'm left to pick up the mess -- but I personally refuse to deign to answer an AI's question till my normal list of problems is addressed. Nevertheless, I still disagree with your conclusion, that automatically addressing a small enough list of issues to fit into an agent's context results in good code.
I'm not sure if you're interested in a challenge, but one hand-coded thing we did well at $WORK was writing the fastest protobuf parser for this schema (and, generally, any other messy, deeply nested schemas) [0] (assume many adcandidates, deals, and segments if you're generating synthetic data to test against). My ranked priorities are (a) I need to be able to easily prove it correct/safe for all inputs (literally all, not just the few hundred billion we received yesterday, since this is open to the public internet), (b) constrained by (a) I need it to have as much throughput as I can possibly manage for many of my workloads, and (c) a second implementation optimizing latency instead of throughput is used for the rest of our software. AFAICT, AI is currently incapable of even understanding what the bottlenecks and priorities are. When I manually describe those, its solutions are a dumpster fire. If I describe exactly what I want coded in excruciating detail then it does reasonably well (not by default, but with enough of the automated tests you've described it doesn't take too many iterations with a person for it to be shippable), but at that point I've already done all the hard work, and if it took 2 weeks of understanding and design and then 1 day of AI coding vs 2 days of human coding then I haven't really bought much.
A full solution seems like a tall order for some rando's internet challenge, but are you interested in seeing if your AI can even identify the relevant problems and propose plausible solutions to them? I'll make sure to revisit this comment thread and reply with critiques and follow-ups if you're interested.
[0] https://github.com/BeeswaxIO/beeswax-api/blob/master/beeswax...
Not sure that is the position you're arguing with. I would refuse to skip code review for any colleague, no matter how trustworthy they were. This is the behaviour we are expected to accept to get productivity gains, when the colleague is an LLM.
I personally have had good success with using agents as virtual pair programmers following some simple rules for how to make that process productive, but that is a very, very different approach. One that recognizes that both humans and agents are prone to errors and attempts to mitigate the problem rather than expecting it will magically work out through the power of AI.
Opus 5 takes that and changes the game. A simple prompt goes through research, design, verification, review and self-corrects constantly. Agentic workflows can run for days without stopping, without losing the thread or devolving into slop.
I can’t even imagine what Opus 6 is going to bring.
The biggest thing holding AI back is people. People can only change so fast and yet AI doesn’t care. It just keeps accelerating.
If you aren’t 100% on it the entire time it’ll make 4 bugs for each solution and then write tests that hand hold the nonsense through. The juice isn’t always worth the squeeze and it messes with your ability to parse it all.
It's less gray bearded wizards than the fact that they used cheaper crazy cackling goblins to write the software.
The challenge is to determine if a finding is an important bug or just a dead business process now never called in production. That is knowledge the AI does not have, and it requires careful, exhausting work of going through the problem.
dict[str, Any] is a type, and sometimes it is the right type to specify, but most of the time it's used by someone who wants to shut up the type checker rather than think about the right type.
And in software problems like that tend to compound. It’s hard to imagine how much badly written, poorly performing software there is out there.
The opportunity to do better with AI is monumental; comes with its own problems no doubt, but I’m willing to give it a shot.
Stopped reading right there
LLMs are no trained to say "no". They are likely to find a ton of bugs in a simple "Hello, world" than generate "no, there are no bugs".
On top of that: if there are possible bugs in an isolated piece in code they find, are those bugs actually there?
I'm unsure what the right solution is. I currently tend to let LLMs resolve these bugs because it does fix genuine issues that would have affected real users. But this also means that LLM-generated code is often twice as long as what I would have written, and much of it is for defensive behaviors that will never matter in prod.
I think it's an important point because the more complex the code become the harder it will be to maintain it, so hopefully this process will improve.
Was the actual bug fixed? Hard to tell among the 150000 new unit tests and a million lines of new code.
---
Anecodtally: ran an "adevrsarial review" on a Stripe integration for a side project. Dozens and dozens of issues, thousands of new lines of code added as a result of the "review".
Looking at issues and code closely: none of the bugs or issues were bugs or issues. None of the flows were relevant to the project at hand, and would've never hit the possible bug paths. And on top of that a lot of functionality was duplicated or triplicated in the "fix". Had to revert a lot of commits.
The divide with developers at the moment seems to be about people who believe in review & are being cautious (who are not seeing major time gains according to all reports I have seen), and those who trust it to be able to fix things forever and will no longer have to worry about understanding the code, so are happy to atrophy those skills.
Yes, my code will have mistakes, and I'll use LLMs for some tasks. I also review and have a mental model, which means I can easily fix them, no matter what happens in future with AI. Reviewing also means I'm seeing the limitations and common mistakes of LLMs.
A no-review approach will be far faster than this, obviously, but if scbench.ai is still true on the frontier models, it may come back to bite people when they have no idea how it works and the context becomes too big to maintain.
If I'm being snarky - the people who are happy to run without reviewing the code probably mostly are able to because there isn't significant value attached to it downstream. Not many stakeholders, not high value transactions, not complex downstream integration or impact. And yes, in their friction free lives, AI can massively accelerate things far beyond what it can do for the rest of us. But don't infer that applies everywhere else where those assumptions don't hold.
I’m sure people are building things that have specific KPIs based on client preferences. As long as those preferences are being met, then their question becomes “why does it matter what the code quality is?”
Perhaps it’s simply liability and penalties? If getting something wrong means your client is going to sue you into oblivion, then perhaps that is where people are focused on making sure its right, as opposed to making sure its fast?
It's also a critical part of your security model. "Nobody can deploy code by themselves" is shockingly effective at preventing rogue actions.
This week it did it a second time on an unrelated project.
"Some mistakes" is putting it unbelievably lightly. It is shitting out complete and utter garbage, and if you think it's doing a reasonable job (or a job that can be fixed later by a more powerful model), you're either not paying much attention, or your own ability has already heavily atrophied.
Karpathy put it well: "gap of understanding of LLM capability," but he's seemingly in one of the less fortunate groups himself.
Over time, the errors don't "work themselves out together," they compound. Everything would stall in a mess of unreliability and chaos.
Are WE the 0.2%?
Because if not, what's the point talking about the merits of AI's competence in of itself? Accomplishes nothing, we're he outsiders. And if we are, then either you're fishing for disagreement and the post you replied to is actually correct, or you get unanimous support to your statements, and establish... Nothing, the in-group could be delusional in a way that does not actually function as an argument against the whole "So where's the software? We're the AI led market disruption instead of the AI-led shovel makers?". The punchline was about the clients being unhappy.
Modern software development practice is built around that humans do make mistakes. Frequently. You don't cross your fingers and hope for the best, you assume it will happen and you put processes in place to prevent errors, catch them early, and make them easy to recover from.
It is a for-real science.
Claude design eliminated figma & power point use for us.
We have a product out there as a test balloon, which only gets architecture & automated reviews.
Dev speed has been incredible, customers love it.
The mess that hey is, is also out there and not yet bankrupt with the biggest AI bro CEO out there
And I know at least 1 startup, which only hired a dev after having 20+ B2B customers. They sure do not go around advertising themself as vibe coded.
Let me tell you something, I don't know jack about this new world.
But I do know about the old world. And in that old world, when you had 20 customers that actually needed your product for their business, your team of 5 people who built YOUR product are eagerly trying to talk to each one of them to figure out why your product works for them, how they found you, what they need now and in the future from your product. They reached out, took people to lunch, they went to meetups and hackathons and boring ass sales pitches to make connections.
Tell me about how all THAT stuff happens when your core dev team is agents in the cloud ?
If "you don't have to review" really is true then the economics are an even bigger nightmare. The value of code has decreased, the "SaaS is dead" claim becomes real, and any good idea can be easily cloned.
There is the whole Artcraft thing.. We are now finally seeing the beginnings of successull fully vibe-coded software. This trend will only increase.
Are AI models incredibly powerful tools? Yes.
Do they have the ability to telepathically sense what the various stakeholders in the development process actually wanted? No.
In my experience, it seems to mostly save time on things you don't really know that well, whilst depriving you of the chance to learn new things.
Just like everyone thinks they are a “good” driver, everyone thinks they are a “good” developer. But programmer productivity existed on a bell curve of talent, multiplied by one’s experience, thrown against the rocks of how terrible your legacy code base was, then finally dragged down by the incompetence of some of your terrible coworkers. Most of the code humans were writing was substantially worse than what Astra generates. And Astra is -ballpark- 500 times faster. It’s worth figuring out how to work around some of the sharp edges.
OpenAI had to throttle sales and was able to raise their prices.
Yes, many adjacent tasks have become slightly more difficult. I’m sure people who rode horses missed them when we transitioned to cars. But, if we built an entire physical infrastructure at a national level to make that technology useful, I suspect we can get people to learn enough about software architecture, CI/CD, and QA best practices such that AI built software is actually more reliable than human built software ever was. Some of us are at that point today. Indisputably.
Yes, you have to review the code. But serious developers already had to do that anyway. Now the code comes with annotations (if you build the right tool) and it’s nearly always perfect.
Meanwhile I'm looking at code that frankly does a bad job of showing why it even exists and nobody involved can tell me why. But it does something vaguely similar to what a prompt said it would I suppose.
> This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
This made me check if the tweet was really from 2026. Or maybe OpenAI/Anthropic practice much less dogfooding than we think?
If you weren't mostly writing code by hand a year ago, you weren't writing very good code.
Lack of complete understanding will continue to be a big problem, but I'd bet things will be net better off than before AI, because each year will give us LLMs that are better and better at good engineering, architecture, and design decisions.
Most people using it for work already pay API rates, and even if you claim API rate inference is not profitable for the frontier models the best open models are still very good, and those are surely served profitably.
This argument is no longer true.
Citation?
We have a continuous process of updating skills, specs etc to try and avoid all these bugs they write, give them context about why its the wrong thing to do.
This process is actually a lot of work, continuous, and can likely continue indefinitely.
No one has any clue what this thing means....
If you are at the top of field with unlimited access to the best (public) models then yes, you will agree that this is the case.
> The divide seems to be between people who think you may need to understand the codebase for maintenance, potential future token costs, optimisation, code erosion & being wary of the assumptions/mistakes that they have seen LLMs make, and people who don't review code and are confident that the AI has made no mistakes and will be available for next-to-nothing forever.
This is not the real divide, it’s the divide you want to exist. The real divide is something along the lines of intellectual curiosity, capability, and confidence.
> if/when they break things
Idk man. I’m one of those guys who “doesn’t write code anymore” since early this year. Nothing is breaking. If anything the chance of things breaking is shrinking dramatically, with both the enormous growth in tooling to support agentic software development (where I work, this mainly really good AI code reviews that catch so much more than human reviewers ever could), and the improvements in the models themselves.
Sorry, but the fantasy of all this coming crashing down is just never going to happen. This is just how it is now, and all the structures will continue to adapt to support it even better.
I see the overall effect on productivity as a rounding error. More stuff is happening, but for every improvement in my productivity there's some fuckass slop machine clogging up some part of a system that worked just fine a year ago.
I recall tales and legends of consulting firms, whose consultants would parachute into any project, put together in record time the deliverable that met all requirements, and say goodbye until a new expensive contract was drawn. And then internal teams would spend an eternity fixing the ungodly mess of testless spaghetti code.
That has been the reality for over a decade. How come this is a concern with AI code assistants?
Of course you can accomplish in 3 hours what used to take weeks.
Look I loved our field, functional programming, etc. But the field as we knew it is dead, and if you think that your work is anything but providing new tests/edge cases and occasionally providing some guidance, you're in denial.
As someone whose world used to revolve around this stuff, I agree completely. For all intents and purposes, what I thought was my career is gone. Sure, it's always been about solving problems but not like this. The problem is that the future hasn't been evenly distributed. This is going to be one long drawn out funeral
Definitely the best comment I’ve read from the demoralization patrol in a good while.
https://github.com/enricopolanski/functional-programming
For me, the middle ground is review and keeping a mental model, not never using LLMs. The "Anthropic are our friends and will never fuck us over!" seems pretty naive.
There are ultimately a lot of unknowns at the moment, and it is simply too soon to know if giving up understanding your code base is sensible. People (influencers) have only been saying not to bother reviewing for about a month.
Also, so many vibe coders don't seem to know that you need to keep third party elements patched.
That said AI feels a bit alien.
My experience with frontier models is that in many areas of software engineering they outperform the average software developer. In some they blow everyone away. In others they're not as good. But they are getting better. The problem is often not the model's capabilities but rather what you ask it to do and are you able to understand what it does and give it guidance when needed.
Semantic diffusion? Not that I have high standards for hmm, curious but humans already X comments.
It very, very routinely points at articles in its RAG "citations" that simply don't support the (modestly likely to be correct) assertions.
It routinely recommends Mac software to iOS users until you ask it to confirm there's an iOS version.
It still claims widely used software has functionality that is non-existent, based on roadmap/wishlist/feature request/unreleased fork type pages.
It is better than it was but it is not adequate yet.
But nothing else matched that. I was looking up all kinds of resources and that hadn't been the conclusion from any of them. It turned out Google AI was just wrong and put us in a bad spot for even looking at those results.
If you then follow up with the thing using the little chat box, it actually kicks off a non-trivial model result, which can sometimes completely reverse the response. After all, your search results are usually based on a few words.
There's no reason not to trust it because since they trust it, they don't percieve anything wrong. They just take it and whatever happens happens, no annoying conflicting info and certainly none of that old grampa thinking and working and giving a shit about anything.
For what it’s worth, any time I see that feature, it’s all farts.
I wonder how accurate this metric is. People may be coming into more contact with LLM generated content, although they may not be aware of it. Content on social media is becoming "AI enriched" already, or outright entirely created by AI for the purposes of triggering engagement and views. It starts with posters using AI to improve their writing, editing a photo, elaborating on an experience. It ends with bot accounts astroturfing certain communities to sell an experience or a product to a group of people.
In this case, people are not directly working with AI, but they are indeed influenced by it.
What's tricky about reporting this in metrics though is that half of the American population is literally too unintelligent to realize this - they believe Reddit comments are still written by real people, not companies advertising products, that "friends of friends" on Instagram are also real people with no incentives posting about their lives.
If there is anything at all that I have seen out of this AI work it is how easy it is to manipulate a massive amount of people.
I have reached a point with a client where non-developers are beginning to experience the full power of modern reasoning models. Their product configuration tooling is now semi-automated and in many cases there is genuinely a 10-100x improvement in throughput. This isn't the worst bottleneck (that would still be the customers), but it's a really big one.
If non-developers in your personal and professional circles are still wondering what all the noise is about, it's possible you haven't been working very hard on adapting these tools to their problems.
From wikipedia: "large language models that are trained to solve tasks that require several steps of reasoning. They tend to do better on logic, math, and programming tasks than standard LLMs, can revisit and revise earlier steps, make use of extra computations, and enhance performance when increasing the number of parameters, training examples, and training computations."
But I think that's still years away and Anthropic has severely reduce the tokens on even the Max 20x Plans, OpenAI starts to introduce $500 plans, every new model suddenly "costs X times more tokens" and I noticed that even downgrading to a previous model doesn't do that much on API pricing (virtually no difference between Opus 5.5 and 4.6 in API costs).
So I think that my personal strategy of tokenmaxxing and doing "hard stuff" while the tokens are still cheap was a decent strategy, only with the problem that I'll have to re-learn programming by hand next year, probably. API pricing is too expensive for what it does, yes, it may be the super-intelligence or whatever but at least right now its nowhere near economic.
Given the leaps that Chinese models have made, I don't think they can raise prices that much. Most of our devs can already do most of their work using DeepSeek 4.1. It's not frontier, but it's very good, and it's a fraction of the cost. They keep their Claude subscriptions because it's currently good value, but if Anthropic increased the price significantly, those subscriptions would be cancelled.
The more difficult question is around data sovereignty. Companies do not like sending a lot of sensitive data to Anthropic, OpenAI, or any of the Chinese companies. Right now local LLMs are really expensive to set up competently, and they're not as good.
I think the future for OpenAI and Anthropic is the Ferrari model. Very expensive but very intelligent models which people pay to access for important work. There's a market for that. It's very small, but they have big pockets.
With models getting to the quality level they can perform a specific set of work at an acceptable quality level we will now begin to cost optimize that work.
We just invested in Lamb Labs from YC Summer 2026 as example company pioneering this next wave of ASICs to run a model.
We're going to see massively lower power usage and much better tokens/sec per dollar of capital expense as well. Think Moore's law but for tokens and output quality.
My company has largely switched to Luna after Microsoft raised prices for Copilot.
That means, it is mostly cheaper to do it with Copilot etc. and not alone, right?
At the same time, the novelty of AI has been making it very hard to calibrate. I'd think that "manual coding is dead" and "cars drive themselves" are obvious to everyone, but sometimes I'm surprised.
Also, I think programmers/mathematicians (the ones that should be calling out bullshit) are enthralled because many of our jobs was so suitable to automation (this was the secret of the 10x engineers, they didn't shit out too much unnecessary crap.. now that unnecessary crap is spit out at rocket-speed so everyone is working at that speed or even faster).
That said, information-gruntwork that a nominally competent person could do is up for grabs and many previously so called "bullshit" jobs were tied to that.
Hill-climbing problems seem to escape this limitation. But I wonder what will happen when the AIs start acing benchmarks faster than you can write new benchmarks, and when you run out of hard problems to solve because you rely on humans formulating them.
The real vertical will come when the AI gets access to physical world instead of bash; and when the human is finally out of the loop.
> You’re drilling with a hand drill. Some people got cordless drills. About 5,000 people got entire factories.
> We’re still arguing if cordless drills are useful, and those 5,000 are already building teleporters. That’s how far ahead they can get.
https://x.com/adam_fresko/status/2109363751048904711
There's been this continual allusion to ever-narrowing circles of influence and ever-accelerating capabilities behind the curtain, but in fact things do keep coming out from behind said curtain and while their capability is indeed advancing it doesn't really seem at all like it's accelerating, even, let alone vertical.
Also at what point do we raise the fact that despite the ever narrowing circles of access, nobody within those circles seems to be able to actually do anything with it that's ostensibly what it's sold to do? All they seem to be able to do frankly is build more models and harnesses and tools designed to consume more tokens from the models they build, but not actual directly useful new or innovative software that generates what could be very real revenue.
Eventually when you keep talking about exponentials but not that much seems to be demonstrably changing, the narrative is going to run out of steam.
I see a lot of benefit to spreading information around the tech, but it's still in its infancy. Most people weren't on the internet when that was getting up to speed.
Meemaw and pops have their systems that they work within, and they just got familiar with the damn smartphone. Though it might help them, they probably do not want AI to organize their old photos or "clean up" their computer. Saving 5 minutes just isn't important to them.
As for professionals, yeah - companies are going to adopt these tools over the coming years. It's the "norm" in my circle, but lots of jobs just... don't use technology like that - there has ALWAYS been a major gap between people who are deep in tech and people who are not.
I think a much better use case is one where it's not visible to the user at all, like spam filtering or helping an accountant do their monthly run in a few hours rather than a day. To me, that's likely a much better use of AI than trying to shove a chat interface or agent harness into an existing app.
That being said though, I did see a startup in my area basically offering a Resend/Mailchimp competitor as a Claude plugin, so maybe it'll just be integrations to their existing chat interface.
The pitches for Dots, Grok Bot, etc., have been spectacularly uncompelling for me as a civilian. I don't need or want a bot to book my flights or read my inbox. I'm not so busy that these things have a value high enough to be worth the effort.
As a worker or an entrepreneur...maybe. Depends on the task, and how many other people were flinging slop at me on a daily basis. I could see the value of a calendar bot, or an expense reporting bot or a bullshit paperwork bot or a slop-return (slop jai alai?) bot, but there's no way in the world I'd give a bot access to the information necessary to, say, do my personal taxes.
And once they start school there's an endless amount of deadlines and "reminders". A personal assistant that can automatically keep your reminders + calendar up to date based on your inbox would be very nice.
It gets even more fun if you have an aging parent at the same time as you have young kids. Then you get double the appointments to scheduled, coordinate and keep track of.
So when you say people don't need a personal assistant, i feel like you're living on a different planet. I suppose when i was single things were more "simple". Your mileage may vary of course...
I understand the need. I have a calendar. I put appointments in it. It provides alerts. What value does having a lossy/inaccurate/fragile robot in the middle give me? Data entry?
Now look...if I were in a position of power with literally everyone in my life, and I could somehow justify the social cost of requiring that they interact with a robot to negotiate demands on my time (the actual hard part), then maybe there would be additional value. Some people can do this! I'm sure one must deal with Mark Zuckerberg's PA to schedule his time [1]. It is very nice indeed when you are the kind of person who can defer your supplicants to a representative who manages the hoi polloi for you. If you can delegate these irritations to a robot, well, so much the better, I suppose.
If I tried this, as not-Mark-Zuckerberg, I would be what normal people refer to as "an asshole". I would also like to see you try this with your child's teacher, or the PTA, or doctor, or...
The response you will receive, an alarmingly high percentage of the time, will rhyme with "buck off". This stuff reminds me a lot of how valley folks keep trying to push spy glasses on the proles (silly normies never understand the genius of that technology, either...)
> And once they start school there's an endless amount of deadlines and "reminders". A personal assistant that can automatically keep your reminders + calendar up to date based on your inbox would be very nice.
Let me introduce you to...Gmail, from like, ten years ago. It looks for appointments in my mail and put them in my calendar. Somewhat to the point, it is often wrong, requiring more work to correct than it would to just enter the data myself.
[1] Somehow I doubt his wife is in this group.
And with school, this is issue if your kid struggles a lot, maybe. But otherwise they track that stuff through school portal or notes.
On the other hand, the areas where it doesn't need to hit the real world, progress is absolutely flying. I would not be surprised if AI is superhuman in math and physics in a year, considering we went from homework helper to IMO Gold to Millennium problem solving in a couple years time. I think we'll start to see more meaningful scientific progress in maybe 2-3 years when scientists have access to an oracle that can formulate theories and direct real-world research better than a human can.
People have to come to grips with the possibility that a technology can be extremely impressive but be a bust in utility. I'm not saying that's what AI is, I'm saying that nobody actually knows.
My grandmother keeps a little notebook with detailed instruction on how to do things such as sending a message on facebook or pay a bill using netbank.
For a lot of things, the choice isn't going to be doing it herself or giving orders to the computer. Its going to be not doing it or giving orders to the computer.
Even if it will look very inefficient to people such as you or me
Or like in Harry Porter, where you don’t need cooks - food just appears from thin air and floats straight to the tables.
The problem is, jobs to be done theory is not new, and product teams that build Excel and SAP know this stuff. The reason why nobody has yet made an accounting software that is just one magic button is in the amount of user decisions that are necessary to make a “proper hole”.
This is where analogy breaks, because you are not helping to drill a hole anymore, you are helping to design a hole. And to design something you need visibility into what parameters you control, what has just changed with the latest tweak, where rough patches are that require cleanup and design rework.
In programming it just so happened that your code structure and your IDE were also your design UIs. In other areas, like Accounting, your design UI today is Excel and SAP.
Arguably a chat interface can also be a standalone design tool, just not a very user friendly one, like trying to pilot a plane through popup notifications. It works for areas where we fully give control over design to LLMs themselves, but step up into higher level designer shoes and you need proper UI/UX again, chat becomes suffocating.
We keep hearing versions of this same drivel since late 2025. This time it's different! Now it's really super duper amazingly awesome powerful and if you don't believe it, just trust me bro.
This is why you can see people talking about the capabilities of models that they can't currently access.
That accuracy is probably a normal distribution with people overestimating and underestimating to similar degrees,
The population of each band would, however, mean the overestimaters outnumber the those who have an accurate perception based upon actual experience.
It does seem quite the recipe for murkiness, even before you take into account the nature of engagement driven media coverage.
It needs to be the second one or there is going to be a lot of angry people out a lot of money.
This implies that the TAM of the $200+ subs is 20m people, translating to $4B/mo or $48B/year.
Guys are milking investors and try to manipulate gen pop and governance systems to ride this illusion for as long as possible, with fear mongering and weird “we see miracles in the lab” posts like the OP, which you read on a regular basis over the past N years.
Saying this as a paying customer from that 20m segment.
Really? That's not my experience. In fact, I've experimented quite a bit with spec-driven design where the LLM does not just get a prompt, but an (informal) spec, often with design considerations included. And yet, regardless of the power of the model, I've never seen them autonomously deliver what I'd consider a finished product.
There are always edge cases that are not properly taken into account, architectural oopsies, performance issues, duplicated code, and other problems, that then need to be identified and fixed, unless it's throwaway software (e.g. a one-off script or some quick experiment).
This is not to say that agents aren't extremely powerful (as I think they are, they do often blow my mind) but the prompt-and-forget approach is IME not a productive use of them for software that is meant to last. This may change in the future, but for now, fully or largely autonomous agentic work does not seem to be the road to quality.
If you want to go all the way, a fully formalized specification is not less work than writing the software, though the potential promise of LLMs for formal methods is interesting.
But for any informal specs, no matter how detailed, while LLMs can be very capable when it comes to avoiding surface errors when translating them into code (e.g. they're more likely than human programmers to write code that compiles on first try), they still don't have a mental model, and most of the problems I've seen are IMO results of that and they increase with the scale of the project.
But also, upon more reflection, it occurred to me that if we’re able to create a competent humanoid robot, that could legitimately be world changing in some good and other very bad ways. The scary part is that I don’t believe there’s any major physical / energetic barrier to this. The hardware exists, the power supplies are light enough. The only part that’s lagging is the software and motion contols; the intelligence. The part that is experiencing breakthroughs at an accelerating pace.
What if a highly intelligent and physically competent robot were available for the price of a cheap car, say ~$20k? What if that robot were competent enough to cook, clean, assist the elderly, move things around, assemble and disassemble things, etc? Billions of jobs would be displaced, or hit with a very concrete maximum wage.
Many white collar jobs, while still at risk for displacement as well, benefit from barriers like IT integration and data privacy hurdles, and the fact that corporate higher-ups simply prefer to communicate with humans.
But most blue collar / manual labor jobs have almost the opposite condition: humans introduce risk, and generally don’t enhance anybody’s “experience” - see people’s somewhat odd acceptance of higher fares for a Waymo vs an Uber. [1]
I don’t think we’re there yet, but the fact that we are rapidly accelerating our trajectory toward that world should be at the least, unsettling.
[1] https://techcrunch.com/2025/06/12/waymo-rides-cost-more-than...
Driving shows it can be done, but even there it's taken years of highly directed research and investment, and driving (at least where Waymo operates) is still pretty well defined compared to plumber.
Such data can be gathered but the current methods of RLHF by paying white collar experts, accessing private datasets, etc. don't seem to really get at this.
I'm also continually surprised that people seem to think politics doesn't exist anymore. I find it exceedingly unlikely that massive amounts of people will simply accept this purported widespread unemployment without major political consequences.
I agree the data trove for these is not nearly as comprehensive as “the entire internet” was for LLMs. But LLMs paved the way - people are building the datasets rapidly now. I suspect this will look a lot like LLM rollout looked over the past 5-ish years. Unusable garbage… less unusable garbage… then suddenly, a fully capable model like Opus 5.5 or the like.
And the “problem” with the way the foundational models seem to work is that they don’t just acquire individual skills one at a time like we do - they go from knowing nothing to knowing everything.
> I'm also continually surprised that people seem to think politics doesn't exist anymore. I find it exceedingly unlikely that massive amounts of people will simply accept this
This is exactly the part I’m afraid of. The political response of the displaced labor (which may or may not be violent / revolutionary), and the subsequent counter-response of capital… potentially using the very robots that displaced the jobs to begin with.
I get the fear, but at that point we're basically imagining that Terminator is a documentary. Given that the US has more civilian guns than people, and that private ownership of military grade weaponry is strictly controlled, I don't think street wars between private robot armies and folks in the bread line is a realistic outcome.
Well supposedly some of the richer folks are busy building doomsday bunkers and whatnot, so I guess they aren't entirely blind to "political consequences"? Not sure what they hope accomplish there, unless they want to be king of some post-apocalyptic wasteland.
I for one am worried about our future, and have neither the money for a doomsday bunker nor the desire to live in the wasteland.
When I was young there were already ideas about 4 day work weeks, it's now been 20 years later and so many things are automated, where are the shorter work weeks? I thought the point as a society was to not work ourselves off and actually enjoy day to day life while automation handles the boring part.
If we have ham, we'll have ham and eggs, if we have eggs.
If that's all it is then shouldn't the mass produced robots that are posted on r/singularity and such places just be remote operated by people in low wage countries? Couldn't the Amazon package sorter robot just be a remote operated bot instead of paying US wages? Why aren't they selling them with a cheap operator subscription? This seems like a clear product direction if the robots were really capable today.
Again, I’m not saying we’re there, it just feels like the control software is the main bottleneck, and the development of that is now being accelerated by LLMs.
People would buy them second hand, weld spikes onto their arms and get them to box each other for money.
If it can do all the other stuff that you're describing plus partial self-repair / full repair of another robot it can, with a supply of spare parts, with enough time and access to raw materials fully self replicate.
Anyone who owns those machines has a tremendous opportunity to build whatever they want.
Solving frontier math problems is mind boggling, but perhaps it says more about frontier math problems than it does about LLMs. Same with chess which was solved decades ago.
My experience is that LLMs can be useful but are underwhelming in terms of intelligence, namely the ability to quickly learn from experience and grow with you while you accomplish a task.
Thinking of them as beings is something we should move away from.
So far LLMs seem a bit lacking in that regard.
It doesn't matter if they are beings or not. They will replace most humans in the offices like machines replaced most humans in fields or industrial plants.
I don't know if they will replace most humans the way you're imagining. I think the industry has done a very good job of selling this point, but that they are still quite far from this.
You can work around that to some extent with explicit instructions, but it doesn't scale nearly as well as actual model training.
My $DAYJOB pushed people to start using AI - frontier models.
I asked it to review our existing code for bugs. Found some, all valid - many that surprised me.
Later I asked it to develop a custom niche thing, that has a strongly unique tooling and environment that we had made. Entire prompt was only 3 or 4 sentences. Had I given a new senior engineer that task with so little context, I’d clearly be an utter asshole.
I really expected the AI to fail. Instead, it did everything pretty much exactly how I would have - and the result worked on the first try. Except it only took a few minutes to do what would have taken me a couple of hours.
The near-term gains are going to be huge.
Long term, this will make the impact of Industrialization seem insignificant by comparison.
PS: he may be accidentally right about global impact, but the key word is "accidentally" because this crappy "reasoning" doesn't support that on it's own merit.
I don't think its useful to keep having to prove LLMs don't hallucinate or LLMs help in coding each time.
I was talking about omitting mentions why LLMs usefulness is efficient - in coding specifically and STEM in general to a lesser degree.
And this is really a crucial thing, which do need to be brought up when discussing future forecast about multiple industries at once.
And the reason why LLMs are so efficiently useful for coding is because LLMs perform much better if any of the two conditions are met:
1) The problem at hand is formalized and transformed into a specification basically.
2) There is a formalized verification of the output done by external pipeline of different tools.
Any of the two drastically improve LLM performance, and coding activity uniquely has both of them.
Having a formalized task narrows down the scope of generation for LLM, and formal verification allows LLMs to iteratively converge on the verification pass check. Same with those math problems, they have a formal verification and LLMs can be prevented from wandering in a generated forest too much. Same in chemistry or engineering for some degree.
But other human activities usually have rather badly formalized inputs and no formal verification of the results. Karpathy is making a logical mistake (or deliberate misleading) of projecting a very unique industry (coding) on every other human activity without any nuance or analysis. He is biased, but due to his reputation and power he doesn't get called out on that mistake.
AI is going to massively transform how most professions work. It will take longer than I think most people expect (I'd say 5-10 years), but there will be an unrecognisable before vs after, like with the appearance of personal computers.
I see a lot of under-30 year olds who dismiss AI because of "big tech" so "is just marketing".
The Opus 4.5 model I was so impressed by last Christmas is out of date now.
I can totally imagine someone going "I looked at this stuff a year ago and it couldn't even start a project properly" , and they'd be somewhat right.
Amid all the discussion of sigmoid curves, and where the "LLM wall" will materialise, I think few people would have predicted that the real wall in LLMs would end up being humans' capacity to verify the output.
What I fear is that people simply eschew human review altogether, considering we're talking about the industry that came up with the "move fast and break things" credo. Human review of LLM-produced code where I work is already a farce, and we're not special enough to be one of Karpathy's 5,000. I do my best to manually review anything that's my responsibility, but I'm literally one of very few people left working on my team, so in practice what happens is I submit PRs that are at best glossed over by completely unrelated teams for security, malware/prompt injection, and other serious concerns. Quality insofar as vetting others' code has completely gone out the window and it shows in the number of bug reports that come back, often themselves written in Claudease. This is all on top of everyone cynically phoning it in in the first place, due to the omnipresent sword of Damocles that is additional AI-driven layoffs.
Worse yet all the incentives point to this being the most economically viable thing individual companies can do. I think it goes without saying some type of regulation here is urgently needed, and that an unexpected cause of an AI bubble pop may end up being that humans simply aren't able to keep up with the pace of the output - leading either to precautionary plateauing of capability, or major liability risks related to a decline in quality.
That was the dominant concern in the circles I’m in, so it’s worrisome it’s being treated as rare.
The response of many mathematicians to the recent dump is a good example: verifying these proofs amounts to unpaid labour for OpenAI and wastes time that could be spent doing publishable work which ultimately results in money or personal success. The slop factor also compounds the work required to verify the output considerably.
For mathematicians, programmers or anyone, if the work required to deal with slop passes the limit, it is no longer in their own self interest to use LLMs. The expectation that people will use LLMs for the betterment of humanity against their own financial interest is baffling.
Really? I start with a conversation for maybe 5 turns or so, where I ask it what would need to change, what the API might be, any database schema changes, URL schemes, and so on, and finally ask it to break it down into commits. Then I let it go, implementing 3-10 commits at a time via subagents. It usually gets the UI somewhat wrong, so there are followups to fix it. This is with Sol and Luna subagents.
Is that what other people see?
Is it effective? I'm not sure, cause if it was I don't see why OpenAI & co are not releasing such a project instead of demos.
The average person does not have the time to occupy themselves with a new thing to see what it can maybe do for them if they combine it with hundreds of others. Until non-chat-bot AI comes in discrete usable packages, mass adoption will be limited, because it's all just a big bunch of lego pieces whose utility is hard to grasp, because it isn't self-evident.
This isn't new, though. It's how "technology" usually works. For example, human(oid)s have been able to stick sticks into things for ~500k years, but the fork wasn't "invented" until ~2-4k years ago.
AI is a background technology, a building block. Something that is best appreciated when you don't know it's there -- when it just does a job (well). The end-in-itself POV of the tech industry is self-indulgent and irrelevant for the average person.
Your car is still the same. Your dishwasher is still the same. The train you take to work is still the same and never comes on time. Fuel is more expensive. The roads are still congested with traffic. Your kids are (probably) doing worse at school. Food costs more. Houses cost more. Rent is higher. Buying a computer or a PS5 is more expensive. Politics is still full of mentally unstable people. The environment is getting worse. You will still die of heart disease or cancer. Food quality is worse. Your kids can't find work. Wealth inequality is accelerating.
But hey...on the flip side...a lot of people are rapidly building software (that nobody is using), and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Imagine what a "country of geniuses in a datacenter" will be able to do? Apparently...nothing.
https://news.ycombinator.com/newsguidelines.html
There are so many comments on HN that would justify your attention with regard to this specific rule. This is not one of them.
Did the writing also contain emotion? Yes. As all good persuasive writing should.
The main issue is that it doesn't engage with the substance of the post, and in fact, it exemplifies the very point the post is trying to make: there is a big and growing gap between those who recognize the benefits or potential of AI and those who don't (or, to widen the point, who only see negative effects or negative trends in society).
I'll make it clear that I don't think your complaints or concerns are unfounded or invalid. There's merit and substance to all these issues, and it's fine to discuss them on HN. They are discussed all the time here, and they can and should continue to be discussed in the future.
But an important part of being the kind of curious commenter that we want everyone to aspire to be is that we can look at topics holistically. We need to be able to look at the opportunities and risks together and discuss ways that the benefits can be captured and shared throughout society, and the risks and costs adequately recognized and curtailed. Most people I know of in the tech industry are taking these matters seriously.
It's a vast and complex topic to discuss, and here's not the place for me to get further into it. But the point is that it's a bigger and more complex topic than your comment adequately recognizes, and we need people to always be making the effort to be thoughtful and curious rather than fulminatory or denunciatory in their participation here.
(Also: I can understand you may not easily recognize that your comment breaks the guidelines; it's common for people to not recognize that their words seem more inflammatory when read by others than they intended them to be.)
Can you point out what a good way of expressing disagreement or skepticism about a subject matter looks like on HN?
(My comment is actually snark, someone ban me off this hellhole of a site please)
Unless you destroy their wealth now.
The Gartner Hype Cycle [0] is real, and ubiquitous. We are teetering somewhere near the top of the Peak of Inflated Expectations, about to plunge into the Trough of Disillusionment. A lot of very rich people are about to lose a lot of money, and we can only hope that they don't get bailed out (again).
Then, once the bubble and hype is over, the long, slow work of actually doing useful things with this new tech starts. That's when we'll see it affect everything else.
[0] https://en.wikipedia.org/wiki/Gartner_hype_cycle
I've done multiple life choices, away from AI, believing it to be a hype, but this hype is still growing, still not dying.
Both in 2019, as well as in 2023, i.e. not 2 or 3 months ago.
I've been hearing this for a year and a half now. That word, "ubiquitous", is getting real mighty challenged. I have tried to insulate myself as best as I could and it just never happens and every other day I wonder if my foil hat needs adjusting. Does yours?
Even if all this is true (personally I doubt it), it assumes the bubble won’t burst before they’ve shifted the money from all our 401ks, IRAs and index funds into their own pockets.
Regardless, people have been making your prediction for a couple years now
You can't invoke market mechanisms to defend a post-scarcity scenario.
Why would the ai owners make things cheaper or even available at all once their abundance machines give them everything they want?
What would they get out of if it? What could ordinary people trade for it that their abundance machine can't give them already and that would be worth giving up their monopoly on productive force?
You can't argue "this will change everything" and "this is how it's always worked" simultaneously, and yet AI accelerationists bring out both to defend their ideology.
Luckily I did not, you are arguing against points I did not make at all.
Either way though, what's more interesting than what you specifically said is that AI boosters have no answer to this structural argument about the reality of post-scarcity. Tellingly, you didn't address that part of my comment at all, either.
You’re implying that the deep lizard state planned the LLM craze?
Sama might be a lizard..
I'm sure the same "strategy" will work again, but this time around everyone has nukes and drone swarms and AI-equipped bio labs so shit's gonna be lit.
Rather: these problems are not solved because there are not enough geniuses who are capable of convincing the masses to send the responsible politicians to hell.
You can be smart and chase the money, but it's crazy hubris to assume that only your peer set of money-chasers are smart.
It's a small leap from there to "since we're the only smart people, we should get to have the most influence."
Kinda seems true. And the worst people in history often don’t seem super dumb, despite their various other defects.
If you aren't female (reproductive rights), LGBTQ+, an immigrant now hunted by ice, someone in need of SNAP, Medicaid, or emergency aid in a blue state, it might feel like both are the same I guess. If you are one of the above, and now suffer because so many voted based on 'both are the same', you very much feel a difference today than from the US under Biden. If you were one of America's allies but are now constantly attacked, you feel a difference today. If you are Ukraine, you feel a difference today.
so, how do you think we can do this?
For a lack of deep knowledge in this area, I don't think I have answers for this, but if one is interested in working on this problem, my approach would be to start with diving deeply into scientific literature about psychology and sociology to find out
- what convinces people to change their opinions,
- what causes people to take action
etc.
People support policies that make housing much less affordable (e.g. rent controls) and oppose policies that would make housing more affordable (e.g. zoning reforms). Saying that it's evil politicians that cause this crisis is dishonest - it's what people want.
The amount of housing in the city won't grow but it will free up a big part of peoples income that would otherwise go to landlords. That helps the local economy with more consumer spending, and makes it easier to save up or apply for a loan so you can start a business.
You get a better economy and more money in the average persons wallet without raising taxes. The only ones who lose are commercial landlords, and they will stay profitable by design so it's easy to shut down any complaints. And you get better quality apartments as a side effect because the only way to get more money out of existing properties is to spend more on fixing them.
> The only ones who lose are commercial landlords, and they will stay profitable by design so it's easy to shut down any complaints.
No, also people who would like to move into city would lose, because that would become literally impossible. You'd have like 15 years waiting lists for housing.
> And you get better quality apartments as a side effect because the only way to get more money out of existing properties is to spend more on fixing them.
Why would anyone fix apartment that's under rent control? If you mean system like in Berlin, then I've got bad news - people there are actually frequently not allowed to renovate even THEIR OWN apartments.
"Convincing the masses to send these politicians to hell" is thus nearly isomorphic to making the masses hate politicians who create such laws, which at least very likely implies that by convincing the masses to send these politicians to hell, a different mindset emerges in the masses.
Yes and general population LOVES these laws, even though they are harmful to them. But anyway - you've got your causality reversed - it's voting masses that incentivize rent controls and other shitty politics. Remember that to implement your policies you've got to actually win elections. And since people generally don't understand n-order effects for n>=2, it's really easy to win elections with populist policies.
The City of Vienna is the largest landlord in Europe. It is also the only capital in Europe without a stifling rent or housing crisis.
I suppose there was Project 2025, and there was that book about the inevitability of Russian hegemony (or empire or something) that Putin liked. Those got attention with influential people. Not sure what to conclude from that.
I could talk about this for hours, and was last night actually.
What do vanguardist communism, feudalism, liberal democracy, and fascism have in common? A shared belief that the general population is composed of a bunch of dumb dumbs that need super smart (read: politically educated) people to "lead" them.
Seems subjective. When is the smartness, big smart enough to allow someone to enter the hallowed ranks of "actually a real human allowed to speak for the masses?"
There does seem to be a clear delineation between the people you hear about (the vanguard in our example), and the people you don't hear about (the masses): willingness to create confrontation. Most people want to just go with the flow, live in peace. In a dichotomy I've been calling "choose peace or violence," they choose peace. They choose to see bad things in the world or their life and focus on managing their happiness around it.
I think that this is the default human mode of existence, and that this mode lacks defenses against people willing to violate the default peace. Imagine a bagel line, and someone cutting that line, and when tapped on the shoulder, the cutter breaking into a huge tantrum, and people in line backing down with a "Jesus! It's not worth it, fine." Thus some people learn that they can be much more selfish in life if they're willing to create confrontation and disturb the peace.
When I talk to people about agitation, civil disobedience, or even just unionizing, the most common thing I hear is anxiety. What if the situation gets worse, what if there's reprisals, what if I lose this little bit of comfort that I've managed to be happy with? That slice could get smaller and smaller but people aren't willing to take the step to say "enough is enough" because I think it fundamentally goes against human nature, we have to sacrifice our own happiness on some level to put our foot down and demand or make change. At minimum we have to acknowledge the discomfort or injustice, which is a diversion from our constructed internal peace.
Personally I think people should be allowed to live in peace. Maybe the liberal democrats and vanguardist communists are right, maybe most people shouldn't need to trouble themselves with the greater direction of their society. However I think the stability this promises is false; in fact I think all stability is false. I think this constructed peace that people are afraid to disrupt clearly can be torn apart at any moment, not just by people willing to create conflict, but by a literal earthquake, or fire, or plague. Therefore I think there's a couple societies in history that got it right: ones who distributed the responsibility of society across all individuals in the society; namely the brief period of syndicalized Spain, and the several hundred thousand years of certain tribes of native Americans on the eastern portion of the continent. There may be other societies in history, these are just the cases I've been able to read about so far.
It seems to me that it's possible and necessary for all people to be ready to wake out of their constructed peace long enough to make adjustments together to maintain it. I don't think this is the same thing as them being "led out of it," I think this is each individual being on some level alert to the need to spontaneously say "no, here we wait in line for our bagels. No, screaming about it won't help you, get to the back of the line. Look around: everyone is laughing at how silly you look."
That was not premise of fascism. Certainly not Hitlers one, he himself was self-taught and did not framed himself as highly educated. He considered himself better then educated people. The origin of his superiority was not "smart" it was "strong will and power itself".
It was not the original premise of feudalism either. That was was about military might, basically.
And origins of liberal democracy are not "smart based" either.
I’m not sold, but I’ve heard this line of thinking many times
When I asked why Musk doesn't put his money towards non-profits my Musk supporting friends say it's because he "doesn't believe anything but for-profit entities can solve the world's problems".
So why isn't he solving homelessness, famines, war, etc. with capitalism?
Strangely, they never discuss the possibility that Multivac might seize all their wealth and cast them down into average-ness. What if it actually comes about, but then it tells us to do the things we already refuse to do?
I'm noticing this even in industry. Yea, we can build and iterate on ideas 100x faster but has this turned into anything meaningful? Not really... Hell, look at the big companies spending billions on inference... nothing shipped and things are still buggy as ever. Where is everything?
How many more software projects are there really left that make economic sense? Zuck learned the hard way re. metaverse.
Not many - the set that make ECONOMIC sense is finite. Even though many want to believe its infinite because all of a sudden everyone can vibe code an app cringe.
If anything AI is going to cause an insane amount of consolidation - what used to be across 3 diff apps, that can be consolidated will be done -> many businesses will die. It will be painful.
Google was attacked by an activist investor pre LLM hype for being bloated - they promptly laid people off. Many software firms have been allowed to bloat up do to the nature of the business.
Its always about economics.
While the set is finite, software comes and goes. Just look at the tools to build software over the years.
> Many software firms have been allowed to bloat up do to the nature of the business.
I can agree with this. It will be harder to justify engineers sitting on laurels. Google could afford it before but I can see how it will be under more scrutiny now.
You either get it or you don’t - not my problem.
Not only from the big companies. Where is the explosion of disruption ?
* Higher load on senior engineers who are about to burn out hard from fighting an endless battle against unreviewed slop people keep trying to yeet into production
* Less junior hiring and suppressed wages
* Culture of uncertainty and fear
* Computers are really expensive now
* Some people now live with diesel generators and gas turbines making their neighborhood hazardous to their health
I think some of the most impressive stuff to show someone from 2016 might be LLMs doing a summary of complex topics, followed by self-driving (actually) cars and intricate robots... but other than that there is no "future magic" touching the lives of every day people going on.
Actually I'm pretty sure if you showed someone the latest and greatest from these big AI companies, these AI companions that can book flights, people would go "Oh, so its like a smarter Siri" and proceed to not care at all lol.
Gosh, this has been my entire experience with AI, and I’ve never seen it this well articulated. Thank you!
I have no doubt startups in particular are able to get away with using LLMs to produce sloppy code and ship. But this trend was on-going and sped up due to the infiltration of LLMs and marketing.
You do realise markets are highly imperfect in reality? That nonsense in econ books is fantasy.
More noise. The world is just so noisy now.
e.g. Why do Rust and CPython still have over 5,000 open issues each? At some point, will the superintelligence fix all the bugs autonomously? When is that going to happen?
https://github.com/rust-lang/rust/issues
https://github.com/python/cpython/issues
What about new features? In every thread about Rust, people are clamoring for new features and libraries. People wonder why there can't be a batteries-included standard library like Go.
So now that LLMs are so smart, is that a solved problem? Or are we still bottlenecked on the expertise of a few experts?
---
Also, what happened to AI's passing the Turing test? I see tons of slop articles that are obvious to most HN readers. They haven't fooled us apparently -- the longer the interaction, the more obvious it becomes whether it's AI.
(That said, I use AI all the time and find it to be a great tool. IMO is is a tool, not an autonomous intelligence with a will of its own.)
I find 2026 much more repulsive compared to 2016.
I don't understand why these big labs aren't doing more to improve the electricity situation. Dump a billion dollars worth of tokens into fusion analysis, research, and simulations and try to force a breakthrough. You could buy a lot of good will with unlimited cheap energy that liberates us from fossil fuel dependencies.
Fusion research and cancer research requires expensive experimentation in the real physical world. You can’t just throw a billion dollar worth of tokens to handle the problem.
Look at all the roadblocks for data centre buildouts which the industry certainly has a financial incentive to complete as fast as possible.
When the tokens are hooked up to physical devices (robotics of various sorts), then the tokens pay for themselves by being physical workers, or human wage replacement.
And a robocop will follow whatever is passed for law, damn the masses who might vote otherwise. And subjugation will be enforced by token, LLM rule, and machine. They have no union or ethics to complain to.
Wait until you have clean energy sources and shut it down
It is 13 years old.
It has been making progress, the degree and date of success is debated and confidently claimed to be in many mutually exclusive states, which at the very least says that there are lots of people confidently making claims that are not true. I'm not going to join that fray to declare which of them is correct.
It is a bold claim to make but not something that usualy be done. Any abrupt halting of fossil fuel would severely impare the ability to develop and produce alternatives.
We need to transition to sustainable methods. If you want to go cold turkey, go ahead and cease driving an ICE, never travell by air or have anything delivered (and that's just the tip of the iceberg), it would be a counterproductive way to reach sustainability, that is your right, but you can decide to do that to yourself. You don't get to decide how others reach sustainability.
There is a difference between advocating for a better world and acting like a bully because you don't like the current world.
It reminds me of the Ezra Klein quote that “OpenAl would need permits to cover its parking lot in solar panels, but it can accelerate into recursive self-improvement, as best I can tell, whenever it so chooses."
Likewise when you see other such shortcomings in our world, ask what this incentivizes for otherwise. Then things will become obvious why they are how they are, from housing, to transit, to feeding people, to healthcare.
The priority is profit, not people.
Just a crazy amount of doom and gloom all around this thread.
You gave your own example for the doom and gloom.
The fact that the means of this capability is in a few hands does suck. But I’m increasingly optimistic that it’s going to be commoditized through open weight models and performance improvements. There is enormous demand for innovations on these fronts.
That's my point, in fact. We're losing the profession just so the annoying guy that would pester you with his stupid app idea can now shit it out himself, lighting the planet on fire in the process.
At multiple jobs I've noted that I felt the pay should have been less. Not so that the employers could take a bigger slice of the pie (in fact, my preference is to be in an anarchist co-op, but I understand that's a wild take and might be terrible in practice). My hope was that we could filter for people that genuinely want to do the work and take on fewer people that were there for the money. At the same time more money is more better. And I like having money. It's complicated.
As I've said many times before, I didn't come to Silicon Valley to get paid. I came here to make computers do cool things alongside other people. I need income to do that, but I try to make sure I'm in a situation where the income isn't something I need to think about all of the time. If you're doing mercenary work that's not possible.
I think the amount of protectionism we’ll see could be absolutely enormous. Not only do incumbent corporations not want to lose share, incumbent countries won’t want to share either.
> I think the amount of protectionism we’ll see could be absolutely enormous because corporations aren’t capable of altruistic attitudes.
On some fronts they will be powerless to stop the changes. I look forward to drinking the tears of IP owners when breaking through their DRM and recovering their source code is just a matter of sharing a model ID, prompt, and hyper parameters. Reverse engineer NVidia's firmware and drivers. Decompile and port Nintendo's latest console's operating system. We may soon find ourselves in a world where tasks like that are achievable in a few days on consumer hardware.
Everything isn't getting worse constantly. The data is quite clearly the opposite. How old are you?
As for things getting worse, I guess it depends on the domain. In hardware, things have gotten a lot better; my desktop computer is orders of magnitude more powerful than I would have expected 10 or 20 years ago. In software, I do see a lot of bloated software being written, but I don't feel that the software that I personally use has gotten worse.
> AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Strange viewpoint. In mathematics any novel result is useful as it expands the scope of other results. Who are these mathematicians who didn't want AI to solve the problems it did? I see trepidation about the future of the field but that is not equivalent to not wanting them to be solved.
https://terrytao.wordpress.com/2026/10/07/ahm-statement-on-o...
A long time ago, another student of mathematics described it to me as climbing a mountain. No single step is hard, but you must make many steps to get to the point that no one else has made a step.
Some examples are finance people directly asking for API access now so that they can make their own tools. These tools used to be maintained by teams of developers.
Many business making their own customized scripts to make their tasks faster.
From an economic point of view, it seems like there are some productivity increases, but I don't think these can be concluded for a while. The rational thinking rarely account for second order effects. Although business leaders rarely care about second order effects, a worse business is not immediately noticeable, compared to hiring cuts and the money it saves
Trust me bro
Again, strange viewpoint in the field of mathematics of all places, unlike, say, economics where doing something one shouldn't has disastrous real world consequences. Until it's discovered and proven, one doesn't know what would be valuable and what would not. I bet your sentence would not be said about humans trying to solve problems, no one would tell them they "shouldn't."
The adoption of these technologies spurned ton of innovation in these spaces, and I feel like with LLMs its the first time I've seen genuine frustration from the industry directed towards users for not adopting the technology fast enough. This attitude did not exist in the 1980s, it didn't exist in the 90s, and it didn't exist in the 00s/early 10s either.
Whether you like it or not - technology causes compression of life cycles. You have a hit product? Great - I expect you to have crazy high growth rate. That’s the expectation of a world that is so deeply interconnected.
People need to be patient and put in the work to build great products, rather than lament its slower-than-desired (albeit still quite fast) adoption curve.
So basically you get personal digital assistant - very useful - but not changing my life.
What would I need for real transformation though? Food, house cleaning, commute - "every day chores". Even if we had ASI tomorrow, what could it do to change my life, besides taking away my job? Design good house robot to help me with food preparation I guess... Maybe finally self driving cars launched in my area? Or it will take control, "optimize humanity survival chance" and realize we don't need as many people on this planet.
> self driving cars launched in my area
It sounds like the transformation would be a massive collapse of the economy with scores of people unemployed.
> Or it will take control, "optimize humanity survival chance" and realize we don't need as many people on this planet.
Maybe, but more likely it will be the humans who control the AIs and hold the power who will make this decision.
When people use words like “chatbot” to talk about their AI use and can’t really name basic things like the model they were using, it reveals a wide chasm between the people who are deep into understanding how to use these things and those who poke at easy options every few months and then look for ways to confirm their priors instead of trying to understand how other people are getting value.
The bigger the pile of stuff you have the tool create unsupervised, the more incomplete/inaccurate the overall picture of it is gonna be. That doesn't matter for most hobbyist/helper-tool/game-port/etc type of tasks.
I get a ton of value out of the tools and yet part of that value has come from being hyper-aware of the shape of the limitations, which changes much less and more slowly than the specific of the limitations.
"How'd you notice that one!?" - well, it helps to be the only person on the team who took the time to interrogate the e2e interaction of all the code...
The median piece of OSS software was already pretty rough once you had to depend on it in production at scale to make money. The median piece of the new AI-generated wave of software, also pretty rough.
The bet that OpenAI/Anthropic is making is that they can automate the interactions generally/globally enough to work around this and eliminate the need for humans in the loop. But it's a tricky problem because it's precisely where the permutations and special cases get nasty and much harder to deal with than more broadly-applicable cases of codegen/test-gen/mash-it-until-the-tests-pass-and-the-code-review-angent-is-happy.
Not to say there's no value to be had. However, it's also been oversold and its error-proneness is a real problem.
But, it seems there's a steady stream of commenters and creators who are quick to remind us that it's never a problem with the tech. It's always a user problem.
Meanwhile, the errors are renamed "hallucinations", but let's not focus on those. In fact, if you just look right over here at this .md file...
While the latest and newest should be needed for greater capability, the average should be enough to give a glimpse pf what is possible. Like a basic netbook is enough to enjoy computing, no need to purchase the latest supercomputer.
If you need to switch your setup every quarter or so, it’s not a tool, it’s a rat race.
It just makes stuff up (or gets it from somewhere, who knows) and then links to a tangentially related page as if it was the source. Either that or it identifies the page as being a source it wants to use, and then summarizes it incorrectly. Not sure the order of operations it’s doing internally, but regardless the citations are bullshit.
I suspect the HN techno-brained crowd will not appreciate it, many are quick to throw away their human experience
Not sure how they're perceiving using ChatGPT to plan an itinerary but I'm also doing all this while using ChatGPT. As I said in the sibling comment, it's a faster Google search and route optimizer, not something where I type in "make me an itinerary for X city" and follow whatever it shits out.
If you talk to free chatgpt, sure.
But if you launch Opus 5.5 in claude code - for some reason Opus is the smartest for me, even for "general" type of questions, when launched from Claude Code. It has harness/prompts optimized for correctness, while more "customer-grade" products, even when used with top model tend to optimize for latency. And I tend to give "context" - e.g. I have "personal wiki" + skills for lots of those things.
I don't recall i last few weeks where Opus 5.5 in CC made an obvious lie, made an unreasonable assumption, or made a mistake that smart human couldn't make. E.g. if it "failed" it failed in a way that non-genius human could fail in similar way.
1) open weights and cheap models are only about 6 months behind the frontier in evals, but…
2) it’s always the premium model that only the biggest fans pay for, that’s just now finally achieved enlightenment.
Oh well, I guess we’ll see in another 6 months.
I tried to have it write a synopsis website for the different types of dance patterns in Dance Dance Revolution, with guides on their execution, explanations, and provided song sources. I provided all of the source articles and search websites needed for it to be able to do that, and gave it a sample set I wanted it to utilize and build off of.
It has bits and pieces of information wrong in 90% of the sections it created. Poor song choices, missing song choices, incorrect explanations, incorrect step-patterns, the list goes on.
If you're not an expert in a given field for which you're using it, it's going to look like a genius. If you are, it's untrustworthy and borderline useless.
Maybe it wouldn't be if there were ample training data on my topic, but there isn't.
It utilized the research from millions of papers many very close to the destination answer to make those proofs. What would it do on something that isn't supported like that? Be confidently wrong while sounding right. That's what we've seen repeatedly.
In a way, I'm not worried about the LLMs per se as much as a kind of moral and ethical collapse. A debasement of the currency of communication, a market of intellectual-lemons.
For someone highly literate, llms aren't going to help you write better. If you're borderline functionally illiterate (this is > 50% of the population according to some studies), I'm sure the llms are awesome.
I've long thought the same about all the hype around coding (plenty of examples from years ago now), but will continue to test the new models. They are improving.
(Those are all examples from just the past 4 days, and just the ones that made it into mainstream media.)
I feel similar about my robot vacuum. It actually takes a surprising amount of labor facilitating the damn thing. I have to walk through the house and basically vacuum proof it so it doesn't eat various cords, rugs, lifting chairs up and putting them someplace so it can get under, etc. Not sure its really a labor save to when I was just brooming the entire apartment in 20 mins and not having to move anything around due to reach of the broom and it not having a tendency to eat things. Hand vacuum I can be more carefull too about the fringed edges of rugs and don't have to fold them over themselves.
I still do it because I already bought it mostly.
e.g., my home is always vacuum-proofed. It automatically runs through all rooms except for the bedroom at night, finishing just before my alarm clock rings, then vacuuming the bedroom an hour after the last person's alarm clock has rung.
This way I never actually notice it, but the home is always perfectly clean as if by magic.
Similarly, if you use separate dishwasher detergent, antisurfactant and salt, and properly dial in your water hardness, it'll be able to clean everything no matter how dirty the dishes are. Including dried-on tomato sauce. Just put the dishes in, and they'll be clean the next day.
Of course it takes a bit of effort to set this up, but then it just works for a long time.
Similarly, I'm a big fan of handwriting software with care & focus on quality, which takes a bit longer at first, but then allows you to trust the software to continue running for years to come.
And then of course, all the cat's toys.
This might be the next thing the robot vacuums all start working on. One has already released a first version: https://www.youtube.com/watch?v=GykaEgNkkVQ
Sure, it's not perfect, but it's still really convenient.
It might take a decade to take what we learned from AI and turn it into something that will benefit the masses.
Most of it can't be broken down into something as simplistic as a dishwasher.
Most people won't notice cancers being found faster due to AI being better at pattern recognition than a doctor, but it's still a great benefit.
I would love it if we can just take the win on having created an amazing tool set for coders. Can’t this be enough?
there was never a need to rush other than for capitals self interest to control
if frontier labs wanna pace the frontier, they should be honest and return the compute back to the people
[1] https://sumsec.me/2026/liang-wenfeng-investor-meeting-transc...
i don't really think distilling is even that fundamental anymore (nor was the evidence ever that strong - https://msukhareva.substack.com/p/is-kimi-k3-really-distille...), the game is all RL now, and you don't need american progress for that at all (just compute and data, and the data brokers will sell to anyone no problem)
i do think there are black market proxies from china to the US trying to sell "claude" and "deepseek" access mixed together, and i do think that prompt distribution is probably going to be very nice datasets
We can have unlimited new entertainment content for passive consumption. That's about it so far. 17,000 new Asian vertical dramas a day now.
Coding has been transformed but none of those areas of our lives have been substantive transformed by AI yet. Whoever does, will create a lot of value for the world.
Do people need permission from mathematicians to solve math problems? Is one of the Millennium problems not worth applying a new system to work on it?
I understand the despair people are feeling at AI automating their work, but we have to be serious about the fact that the frontier labs are just responding to natural incentives. Mathematics, coding, all are just applications of logic, and it makes sense that a system trained on inferring patterns from data would work well in that domain.
I'm pretty sure OpenAI's proofs, while difficult to understand, have a net-zero impact on human understanding of mathematics. It took the first Millenium problem years to verify its accuracy. It's only been days since OpenAI released its problems, surely it would take years to evaluate whether the solutions meaningfully advanced human understanding.
https://mathstodon.xyz/@tao/117395269325940185
https://mathstodon.xyz/@tao/117394287977270632
All mathematics researchers in the whole world have agreed upon that objective apparently?
Many of them - all but a couple of the living Fields Medalists - recently signed an open letter about this.
https://mathandai.org/
Many modern medicines do not have understood mechanisms of action, yet we know they work. Are the people who found them not conducting drug research?
Do we even have a different term for what the AI labs are conducting if it's not mathematical research?
Additionally, I would argue that any work that adds to the total corpus of known facts increases human understanding, even if the path to get to each fact isn't understood. It's hard to argue we'd be better off with less total information.
Mathematicians are not gate keepers but I understand the frustration. If someone claims they spent 30 million dollars to solve one of the most difficult problems yet there’s no new knowledge, understanding, nor certainty. Then what’s the point besides marketing?
So just let them spend their money and they will eventually fail and bankrupt, right?
By comparison, we're in the early years of AI. We're all still figuring out what this thing even means.
This is what is so frustrating about this whole ai thing is we aren’t learning from the mistakes of even the very recent past
Because they didn't just break laws, they built an entire system to mislead law enforcement.
(How do tech people not know about Greyball??)
However, the counterargument falls entirely flat. "Whatever benefits AI has to offer on a societal level are going to take a long long time to show up". We don't have the luxury of time because AI disrupting communities and livelihoods faster than perhaps any other technology we've ever seen, with the most capital that has ever been put into any tech since the railroads.
Even if you're optimistic we'll eventually get the cancer cures and fusion reactors, doesn't it make sense from the perspective of an ordinary American to regulate the harms like the homework cheating, parasocial relationships, job losses, cyber attacks, etc., especially if it will take years to get the benefits?
I don't see much evidence in the economic data that AI is disrupting livelihoods. Unemployment has been stable around 4% for a while and productivity growth has been at historically normal rates for the past 5 years. I think a more likely explanation is that AI is increasing uncertainty in some industries, which are seeing higher employee turnover, and many of those people are moving laterally into other jobs where their skills are more likely to be useful.
I don't think anyone in this thread has argued against mitigating harms. The OP was about current capabilities and the perceived lack of benefits for the average person.
Some of the most disruptive periods in history brought about the greatest long term improvements in society. This is the price we pay for changing how things are done, sometimes for better and sometimes for worse. We don't know which one it is for AI yet.
Like, just at a bare minimum.
These are “use all the AI” companies with infinite resources, after all.
"Slop is its own cure"
This is an indictment of mathematicians, not AI.
In American history, when previous shocks like this occurred (e.g. with the Great Depression), we needed massive economic programs like the New Deal to restore consumer demand. When such programs are not forthcoming while wealth inequalities keep increasing, one can look to other countries for what ends up happening; either the disenfranchisement of ordinary people over time or unrest and redistribution by force.
While trickle down theory is dubious when it comes to economics, it's clear that it is valid when it comes to technology. Tech trickles down.
This has been studied somewhat extensively (https://academic.oup.com/ser/article/20/2/539/6500315?login=... is one example), and it’s been shown over and over that this does in fact not work and leads to larger wealth disparity, across many countries where it has been attempted.
It’s possible you’re working off some other definition of the term based on your comment but that’s the generally accepted usage.
It would be neat if someone did sweeping sentiment analysis on them, just to see it graphed, but I’m sure gathering the data would be prohibited.
> [Your train] never comes on time. Fuel is more expensive. The roads are still congested with traffic. Your kids are (probably) doing worse at school. Food costs more. Houses cost more. Rent is higher. Buying a computer or a PS5 is more expensive. Politics is still full of mentally unstable people. The environment is getting worse. You will still die of heart disease or cancer. Food quality is worse. Your kids can't find work. Wealth inequality is accelerating.
None of this has to do with AI.
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using), and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Plain wrong on both counts.
> Imagine what a "country of geniuses in a datacenter" will be able to do Apparently...nothing.
Nonsequitor, probably based on a personal beef. Datacenters, at best, are understood to be for hosting more AI - not for people to somehow live in as an arcology of discovery.
This reply summarizes the problem from a different angle.
> Senior Director of Product in Insurance Tech
anakin_padme.jpg
Those LLMs are being adopted by the huge enterprise itself and being surrounded from all sides by a very complex procedural harness (aka regular programming) that stops the LLM from doing dumb stuff.
Nevertheless, my counterpoint to
> If they actually had access to systems that could do "years of top professional work" across software, cyber, science, and math, they would not mostly be leaving to start more chatbot companies, agent frameworks, eval tools, and "AI for workflow" startups.
> They would be starting normal companies to disrupt the global economies of the world.
> Insurance companies. CAD companies. Compliance companies. Security auditing firms. Logistics software. Tax automation. ERP replacements. Drug discovery tooling. Industrial design. Legal ops. Accounting systems. Boring vertical SaaS with insane output per employee.
is:
Many people who work in such industries already know an insane amount of inefficiencies. Unluckily, they are not in positions and have no political power (with respect to organizational policy) in the company that they are capable of changing the situation.
Additionally, many of these sectors are heavily regulated, so it is very hard to make a difference in this sector as a startup founder, and even if there is an entrpreneur who founds a company to change the status quo, an insane amount of work in the company will be handling the big amount of red tape.
Things worked better before we started putting bad software in everything. As a software engineer, I know what good software looks like. The fact that almost all software is now bad -- including the software that drives AI and the software that AI generates -- makes me want to flip the table. We know how to write better software and the general public should hang us if we continue to refuse to deliver it.
The problem are the power-relationships and laws which allow someone to abuse you and your stuff, block competition, and even put you in jail for trying to remove the software—or helping others to remove it.
Never, ever connect appliances to the Internet. I thought this was common knowledge on HN. Also pull the telematics (cell modem) fuse to prevent your car from snooping on you and selling your location and driving habits to data brokers and insurance companies.
We've just locked out a large part of modernity because the shitty American auto industry is in swing states and also gender.
Same could be said about the slow rollout of Waymo.
Lots of America is just very anti technological change, and this means any technology no matter how "magical" hits a wall.
Your pessimistic view doesn't hold up against the data. For a lot of the problems you mentioned, we're in a better place today than almost any other time in history. Of course, things can always be improved and we should never grow complacent or accept injustice. But by most objective metrics, there's never been a better time to be alive.
I suspect the real world change is going to happen unbelievably rapidly and isn't far off. Right we don't quite have the hardware to embody AI in a generally useful way, but that's coming fast.
Within a couple of years arguably any task that requires just knowledge or intelligence is something you'd be better off giving to an AI than a human. And decent humanoid robots are maybe a decade behind that.
In the same way people growing up in the mid 1900s saw rapid changes in their way of life with planes and cars replaces boats and horses, we'll see something similar with robots replacing human labours. Right now we're still in the phase where people don't think humans will ever be able to control flying machines and that cars are reliable and slow. Things will change.
That's not fair. To first approximation, most of the progress in human condition of the last, like, forever is implementing solutions that people didn't see value in at the time of creation. Woz built a toy that could barely run software at all. The human genome project was panned as a boondoggle. Penicillin was a mistake.
The first fire pit probably looked like a dangerous stunt to the tribe faced with it, even.
Does that mean everything will come up roses? No. But LLMs only started talking to us for real like three years ago. Let's not just that fast.
Has any technology made an impact on physical world faster?
Imagine being in charge of a physical factory, sure you might have a 200 USD Claude subscription but you also have a floor full of physical machines that have, if you're lucky, an undocumented debug interface and a paper manual. Not to mention safety inspections, insurance, etc.
You can't vibe code a factory floor. Not yet, perhaps in 5 - 10 years but there are some very hard, non technical, problems that need to be fixed before we'll see meaningful improvement there.
You are allowed to criticize it, just not because "the progress has been too slow". It was not.
Progress between Copilot and something like a gpt-6.1-sol in a good harness is at least as big as few Android versions.
This is far from being proven as a net gain and there are massive lawsuits incoming related to AI recommending suicide to kids and others.
Safety systems are not optimal but improving everyday and although I doubt we'll ever get a perfect safety system, it seems that we're not very far from a "good" safety system. The days of Google recommending glue on pizza and generating diverse Nazis seem behind us.
Not to mention all the untold good that LLMs have done. Personally I've seen that LLMs are incredibly good at pointing conspiracy nuts and extremists into the right (moderate) direction.
In a polarised world, we definitely need that.
Also, rage baiting also drives up chat bot conversation durations.
The incentives are exactly the same as for Facebook and Google Search.
People didnt get a new car after they got. Internet in 2001. They didnt get new dishwashers.
Not for Tesla owners in select countries, where their car now drives them around 99% of the time, or the people of Austin Texas, that can now hail a fully driverless Cybercab for cheaper than an Uber!
>Food quality is worse.
Food quality is higher than ever if you want high-quality food!
Unfortunately, the people crave slop.
Videos are shittier. Most songs are even shittier than the autotuned pieces of shit already were. Ads are worse (even though nobody thought that was even possible). Customer support is worse.
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using)
That nobody is using and that are worse than what we used to have. And because nobody's using them, the countless issues plaguing them aren't even reported by users anymore (there are no users anymore). Full of bugs, usability issues and gigantic security holes (but which doesn't really matter because nobody's using them).
It's not even clear that it's making long-standing software (like say Linux or Emacs or Blender or QEMU) better at all.
Now I'd say though: most of the issues you mentioned have nothing to do with LLMs / AI. But it's very clear that AI ain't solving much of the world's problems and that the promises we got years ago didn't materialize.
Even if a certain small group of people are also using the hype surrounding this tech to attempt to destroy the home PC market and reverse the personal computer revolution. They will not succeed. There is more of us than them. Even with "thousands of frontier LLMs". A human+llm is going to be more powerful for a long time rather than a bunch of LLMs.
AI won't show itself as a big new thing, but possibly in improvements in small little things
Most people won't use AI directly, except maybe as a smarter Google. In the same way people don't use HTTP directly
Now it’s comments like this lamenting that the amazing LLMs we’ve had since early 2026 haven’t entirely revolutionized the whole physical world and even global geopolitics yet.
> But hey...on the flip side...a lot of people are rapidly building software (that nobody is using),
I guarantee a lot of software you use today has substantial AI development in recent releases.
Even the Linux kernel is getting a heavy input from AI and Linus is advocating for it. Unless you’ve frozen your entire software stack in 2024, you are using software with AI generated code.
> and AI is solving mathematics problems (that nobody - including mathematicians - wanted it to solve)
Saying nobody wants those problems solved is flat dishonest. The complaints from some mathematicians are that they and their peers didn’t get to put their names on the papers, not complaints that the problems were solved.
When companies make profit, it doesn't mean we get screwed, nor even that they have a high profit margin. The grocery chain stores are a prime example. They typically make 2-3% net margins on revenue, and provide fresh, clean, reasonably well-tested groceries close to our home for that margin. Imagine trying to get equivalent quality fresh groceries ourselves from the producers, and I'm guessing it would cost at least three times as much.
Africa is going to look like wakanda in less than 5 years if WW3 gets canceled
-------------
It comes down to the difference between programmers and non-programmers. If you're not coding, AI literally is a derpy chatbot that's made it harder to get search results, raised your power bill, destroyed property values in a neighborhood near you, is propping up the economy but also creating a huge bubble that could pop and destroy a lot of retirement savings, etc.. At best, it's a nuisance, but it could be a lot worse than that. The AI evangelists need to understand this, because the average person is starting to pay more attention to the downsides while still not seeing much upside.
This gap between the people seeing the curve go vertical and those who are simply growing more and more annoyed is a real problem... mainly for the people seeing the curve go vertical. If the people who are merely annoyed right now don't see some of the promised benefits soon, or if any of the negatives get worse (e.g. massive increases to greenhouse emissions), they might get angry. You probably wouldn't like having the overwhelming majority of the public angry at you. AI evangelists need to start doing better things for the majority of people or they need to address the negative consequences they're inflicting much more aggressively.
“Cy Draven: I tried everything. Believe me, I hammered the race thing, I hammered the gender thing, the trans thing, the border thing, the homeless thing, the war thing, the election thing, the abortion thing, the climate thing, the thing about induction stoves, Israel, library books, vaccines, pronouns, AK-47s, socialism, BLM, CRT, the CDC, DEI, 5G, everything. All that I did, nobody, just nothing.
People are just numb these days, I don't know why.”
— Knives Out movie
And then using emgui they could expose full fidelity IDE+Agent workflow interface (like DeepSeek Harness and similar) that's also exposed as MCP to be operated by another agent and JSON RPC for over the network but no. Skill issue?
PS: Even full fidelity Photoshop is possible with such high fidelity UI that has CLI, JSON API and MCP server all in the same single binary < 90 MB. For reference, see PhotoCraft or VectorCraft or WordCraft or PdfCraft.
I think "moment" significantly undersells this. Frankly, I wonder if this isn't setting vast swaths of the population on fundamentally different paths.
Currently the difference in life outcomes between people who know how to use computers well and those who don't is small enough that in everyday life we just ignore it. But it's greater than, say, the difference between people who took advantage of refrigeration and those who stuck with ice boxes, or those who learned how to drive a car and those who didn't.
And the difference with AI is going to be even greater. And it's possible that right now people are choosing a life course that will benefit or disadvantage them long term.
I think the point is that this gap exists partially because of the access that a select few have today, which enables them to do things that the have-nots can’t fathom.
> Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt.
No, they can't, at least not any semblance of quality. The cases we're seeing where this does kinda work is in ports and translation where all the rules are already documented in the best specification language possible with a way for the LLM to verify itself: code. We saw this close to a year ago now with Cloudflare and NextJS
> The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about
These are operating on different capabilities - AI's ability to answer informational queries as a chatbot frankly sucks and can't be trusted without verifying it. I run up against this every day. A problem with a verifiable answer on the other hand it's very good at solving. He knows this (his next paragraph) but he's putting them on the same scale of "stressing the system" to attempt to add proof to his introductory claim
And then there's the completely unverifiable scare that there are internal frontier models way beyond anything we've seen "swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics"
I dunno. I haven't been able to set up OpenAIs remote codex connection, their shit is buggy as hell and the web UI keeps crashing and making messages disappear. Is this what their internal superhuman "Things that would have taken top professionals in the industry years of work" looks like? Granted Claude has been really smooth, but still..
Defined testability.
> swarms of thousands of agents collaborating over weeks on software mega projects
If you have defined testability, running a single agent, in a loop, over a goal will achieve the same results.
> minting zero days, running cyber attacks ... Granted Claude has been really smooth, but still
The old adage about monkeys writing Shakespeare applies here. Agents are just better monkeys and if the goal has a well defined outcome it's fairly easy to "Brute force" the way to an answer.
So I think there is are growing gaps even within the described groups.
You’re trying to tell me it’s all hype when I have (depending on the day) a top 50 Productivity app in the iOS App Store for which I’ve written and read exactly one line of code (the version number).
https://imgur.com/a/ua2lbxb
What even is this comment. Like, oh wow, an iOS app, AI hype=real.
???
Good for you that you're gaining value, but that doesn't necessarily mean that you're providing value to others
Anyhow, if your users are currently satisfied, they might change their mind when they find that their personal data was uploaded to an open S3 bucket, when the new version wipes all their projects or when they find that not really everything was being saved (it's random examples, I don't know if they're applicable to your app).
It's incredible to see AI denialists still active in late 2026.
How can you possibly believe it's all hype? Is everyone just lying? Are we all paying $200 per month because... why?
There’s no reason to think this.
There are lots of good reasons to think this.
The first is that LLMs used to be bad at each of these nerd things, then toppled them like dominos. There's a pattern over time.
Another reason to think this is g. Across every known measurement, human (and animal) intelligence is convergent. Being better at one thing correlates with being better at another thing. Although the reason is not perfectly clear, the pattern is well established, and seems to apply to LLMs too - GPT-6 is smarter than GPT-3 at everything, not just math. There's no reason to expect different for GPT-9.
If anything, in some domains frontier models have become worse - claudisms and chatgpt idioms are making them notoriously bad at prose without a considerable amount of prompting and tweaking.
However there are many other domains which have verifiable rewards in the process of learning them, despite their overall impact not being verifiable. For example, the life sciences, an LLM could be given access to data about an organism, and then make predictions about how a drug or gene therapy will affect that organism. In economics, LLM's could create models of behavior, evaluate predictions over time and see how well those predictions match reality.
>claudisms and chatgpt idioms are making them notoriously bad at prose without a considerable amount of prompting and tweaking.
I think this is because a lot of people genuinely like the claudisms, even though a small minority of technical people don't.
And that's a massive intelligence gap!
It may just be that writing is mostly saturated, and the remaining perceived gap is not quality but some kind of "cultural fit".
AI writing sucks because there's no intent or choice behind it. It's human thought, blended by linear algebra into pink slime and extruded into sentences.
This is inherently self-contradicting.
Anecdotally, creative writing communities do report drastic differences in quality between models (it is my understanding that Gemma 4 is especially praised) so there is something to it being verifiable but it's a far cry from being something that can be distilled into objective math/code-like evals. One could even say it comes down to vibes.
>For example, the life sciences, an LLM could be given access to data about an organism, and then make predictions about how a drug or gene therapy will affect that organism.
Yes, biotech companies are doing this right now but ultimately they are just predictions and you still need cold hard biological data to ground them and iterate on. That's slow and expensive, especially for the juicy fields (human biology, food and crops, clinical trials) and you can't just plug billions of VC capital into the pipeline and hope for RSI. Physical (gotta procure all those labs and their equipment), human (gotta hire and pay specialists to run experiments), biological (gotta wait for organisms to reproduce, drugs to take effect, crops to grow), regulatory (gotta convince agencies that your fancy new drugs are legit) bottlenecks get in the way.
And biology is one of the easier fields where a path to RSI (if not accelerating) is conceivable. Good luck iterating on macroeconomics data where there are no replicates and 'experiments' take literal years if not decades.
>I think this is because a lot of people genuinely like the claudisms, even though a small minority of technical people don't.
My uncharitable take is that the abtruse jargon makes people feel smart for understanding it and gives a sense of belonging (as a closed circle of initiates who understand LLM cant), just like rationalists love to repackage old or unsavoury ideas under new nerdy smart sounding names.
https://vulsar.ai/benchmarks/creative-writing-v1/#cohort-ove...
>you still need cold hard biological data to ground them and iterate on... you can't just plug billions of VC capital into the pipeline and hope for RSI
Capital can get you a lot: access to wetlabs, historical medical data, ability to run simulations, creation of autonomous bio labs which run thousands of experiments in parallel. An LLM, in training on this flywheel of data can take in more information than a human biologist would be able to read in multiple lifetimes.
>Good luck iterating on macroeconomics data where there are no replicates and 'experiments' take literal years if not decades.
The better an LLM gets at simulating human behavior, the more accurately you can simulate economic policies and their effects via running simulations of agents in virtual societies. There are many routes that models can take to create a broader sense of the world without needing the world to provide data.
I would really take those benchmarks with a pinch of salt. We already know benchmaxxing is a thing for evals that are objectively verifiable like code, and coding models that seem to perform well on benchmarks don't pass the vibes test, imagine all the caveats that come with such a subjective test like creative writing, not to mention control issues (are they the same human writers across benchmarks? Are they the same testers?)
>Capital can get you a lot: access to wetlabs, historical medical data, ability to run simulations, creation of autonomous bio labs which run thousands of experiments in parallel. An LLM, in training on this flywheel of data can take in more information than a human biologist would be able to read in multiple lifetimes.
I don't disageee and am in fact excited to see the advances, what I am arguing is that it is going to be slow because the hypothetic feedback loop that would improve the model is bottlenecked. There are actual physical and biological limits in our universe. Unless VCs find a way to make mice embryos and crops grow 10x faster (among the many, many other things that would be required for acceleration) the trickle of data is not going to be anything like the accelerating returns imagined for math and coding. And that's assuming biological RSI is possible which has yet to be proven - current foundation models are interesting but they are still specialized and there is no clear, obviously uniform way to encode the many, many modalities that exist in biological data.
>The better an LLM gets at simulating human behavior, the more accurately you can simulate economic policies and their effects via running simulations of agents in virtual societies. There are many routes that models can take to create a broader sense of the world without needing the world to provide data.
You wouldn't need to 'just' simulate human behavior (good luck with that btw) but the entire political and economic conditions in which they live. Even assuming this is possible the amount of compute you'd need to run would be astronomical. Given how all RAM capacity for '27 and '28 is near sold out, hyperscalers are running out of cash flow to finance buildout and Sam Altman complains when users say 'please' to LLMs I don't think this is in the near horizon.
Even if we do accept that models have some “g” and that it correlates with all domains, there’s no reason to think that the current trajectory of LLM intelligence will grow more quickly or even as quickly. In every other technology we have an early period of slow growth during experimentation, and then a rapid period of growth, and then stagnation.
For all we know, 2026 could be the year of stagnation. It’s possible that we will reach finishing returns. If thats true, then it won’t matter how much that task correlates with overall g, because g won’t be increasing that much.
For example, they have become more and more unintelligible when you ask for explanations or descriptive text. They assume you see the same context as them and shortcut explanations.
I've had to craft a skill to get them to produce remotely understandable explanations of even mildly complex/non-mainstream subjects.
This may be Curse of Knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge on their part, and it also impacts human experts but still. Becoming better at one thing does not mean you become better at everything else, although I do agree with you that the better they become the more things there are they become good at, but their ability is still quite jagged and maybe increasingly so.
Knowledge can occasionally get in the way of some tasks, such as a master illustrator trying to draw like a child. Or as you've mentioned, an expert trying to explain to a beginner.
Even so, I disagree with you about model progress on communication - maybe there's a little jaggedness between minor model versions, but Opus 5.5 is a much much better communicator than, say, Claude 3 Opus.
LOL. Human (and animal!) intelligence is notoriously jagged. We're all basically idiots except for very narrow areas where we focus.
Smarter just means smarter.
See for example Paul Framton https://www.theguardian.com/world/2013/mar/30/physicist-mode...
This is the classic error in mixing up knowledge / experience and intelligence.
Intelligence is the ability to quickly process, understand, learn, and use information. It does not imply having already been exposed to any given information.
A smart doctor may have terrible computer skills. But this is from lack of experience and interest, not lack of ability to learn how to use computers.
But take that smart doctor, and a relatively dumb person who is equally inexperienced with computers. Now give them a month and incentive to learn computers. The doctor will learn much more; that's intelligence.
Someone can be really good at physics and suck completely at biology, due to factors we don't even understand (motivation probably being the main one we do understand).
We even see this in practice with super bright tech people, especially programmers, failing at launching businesses or at adapting in other lines of work such as management, finance, etc.
Someone who actually works with real animals (farmers? Ranchers??) would come up with better examples. But I'm just city boy over here who only meets cats and dogs on a regular basis.
https://www.britannica.com/story/are-dogs-smarter-than-cats
But in any case, Britannica is a good source so I stand corrected on this subject.
Unlocking mastery of those domains would lead to incredible gains in all others. Whereas a fundamental advancement in many other domains doesn't spread as readily.
That’s good for corporations who need custom software. Every large business has a massive backlog. In this area reducing the cost will massively increase demand.
But where’s the broader value?
Consumers and business already have too much choice in off the shelf software.
Ultimately most people just need word, excel and maybe 4 computer games a year.
The most passionate, intelligent people are usually a decent indicator of where culture is going to go.
Many very smart people who I know are quite skeptical of LLMs and the hype around them. In my observation/echo chamber, the people who are very into LLMs are rather slick, career-minded people who love to present themselves as trendsetters for the "next fancy thing".
>The most passionate, intelligent people are usually a decent indicator of where culture is going to go.
That's a naive view of cultural determinism. The software landscape would have been very different if a handful of lawsuits had gone one way or another. Hell, even Unix and Linux were hobby projects that accidentally got big.
Also crypto is a grift lol, one of those is definitely not like the others
So you’re wrong pal, sorry.
It's worse than that. I am one of those nerds (a programmer), and I see that LLMs are shit at the things I care about. I have zero reason to believe that they will be good at other things either.
I don't think this explanation is sufficient:
For example, many nerds care about 3D printing. On the other hand, my experiments (and the experiments of many nerds who I know) to let LLMs create files for 3D printing lead to horrendously lacking results.
Or many nerds care about linguistics or puns. Whenever a new LLM comes out to which I have easy access, I do the test, and let it explain some specific German jokes to me that are based on convoluted puns in the German grammar. Until now, no LLM that I had access to could give satisfying explanations of the puns on which these jokes are based.
Or even for coding (many nerds do care about elegant, sophisticated code): the LLMs that I could test were already overchallenged with the following task: I had written some code that is a very "artisanal", "clever" improvement of an algorithm over the version that one would find in a textbook. The task for the LLM was simply to write some code comments/documentation about the mathematical ideas upon which my improvement over the textbook version of the algorithm is based. It wasn't capable to do this. On the other hand, for a junior programmer, I would in such a situation expect that he goes through every single line and thinks through the algorithmic ideas so that he can learn from them.
---
Thus: even for "nerdy" topics, it is very easy to find tasks where LLMs still suck. So, my hypothesis about the nerds that you mention in your post is that these nerds are rather people who want to believe (with religious fervor) that the current LLMs are exceptionally good instead of just looking into a slightly different direction than where the tech billionaires want the users of LLMs to look at.
Hippies live at the center of technology and humanities.
Nerds sit purely in technology.
Nerds are also not necessarily all about technology. "Math nerd" was a really common use of the term for a long time, for example. "Nerd" is someone with deep knowledge about a particular subject.
Try again; astra has been driving fusion for me. I took a screenshot of a self watering cat grass bin on makerworld — not even the stl, a screen cap of one of the photos attached to the design — and it made me a parameterized version in one shot.
Its initial design was less than ideal for printability and it reworked the design correctly when asked to consider what it originally produced through the lens of printability.
Autodesk bundled an mcp server with fusion in the last few weeks and i expect things to get even better going forward.
I have no access to Astra, but with GPT 6.1 Sol, the results with respect to attempting to create STL, OBJ or even OpenSCAD files by using image references and a prompt were really bad.
It just seems weird for Karpathy to throw around these numbers calculating percentage against the entire global population.
Just another Anthropic employee trying to spread hype about Anthropic IPO
Revenue must be stagnating.
https://www.youtube.com/watch?v=t6MAZW-joCo
Pretty much all of this is basic stuff an interested teen would know and chatGPT fails miserably. I fear the "true world model" gap will be too much for LLMs. I guess time will tell.
> What is possible is unevenly distributed. As knowledge and technology compound, the distance between those closest to the source and everyone else also exponentially increases.
I actually argue the continuum that forms singularity has been progressing before we realized, but we're at the point where in 5-10 years people not using AI will be equivalent of the amish.
Unfortunately, as the rate of change increases, the distribution of knowledge wont be as fast. So those closest to the source of innovation will have exponentially more knowledge, capability and in-effect power.
https://ipcopilot.ai/blog/the-singularity-already-happened
I know he can code and has a degree in computer science, but if I'm not mistaken he's never worked professionally as a software developer.
Developers here know how much you learn and are forced to learn on the job: it depends a bit on the company you're at, but you can quickly learn the value of good code, how easy it is to introduce bugs and vulnerabilities, and how helpful a second eye is.
I think he's in good faith in his assessments, but, at least for what concerns development, maybe we should give them less weight.
> Remote attestation can allow them to verify that your device is not capable of, say, sending and receiving encrypted messages, or running a competitor’s app
How does this make any sense? Google can already prevent any app from running on an Android if they really wanted to, they don't need pass keys for this.
Google doesn't need attestation to block Signal. What is this non-sense? Or will a website block phones capable of running Signal? That's all phones. And anyway, has nothing to do with pass keys.
That clearly didn't happen :)
> In the next decade, they will do assembly-line work and maybe even become companions. And in the decades after that, they will do almost everything, including making new scientific discoveries that will expand our concept of “everything.”
https://moores.samaltman.com/
We have already reached "peak LLM" in terms of what normal people realistically need it to know or reason about. In fact, I'd say we reached that point about 1.5-2 years ago. There are two other barriers that remain unsolved:
1. They're less dependable than humans and can't be meaningfully punished or forced to make up for mistakes, so you can't really replace humans with them, not without having a human babysit.
2. Most people don't really have a special need for an LLM in their life. They may like that it answers questions or helps you polish a resume or, I guess the labs' favorite, helps you make restaurant reservations. But this sure isn't worth $200/mo for most people. Probably not even worth $5/mo.
It'd be kinda funny if we create superhuman AGI and then no one has any real use for it, perhaps except for military murder-bots. There's always market for that.
Even ones that cannot be easily dismissed (like the agent at the top of Google search) are often subtly wrong in a way that requires significantly more effort on my end to parse the loquacious output to determine where the inconsistencies are.
Look, can they be useful to generate the html or jscript for humanity’s 9 billionth iteration of a web form? Sure. But man, I wish sanity had reigned and people had applied ML models in general to more substantive and beneficial projects.
(Before anyone chimes in, I’m familiar with implementations of ML applied to esoteric domains; but by their very nature these don’t get all the news cycles, or hiring, or any of the other absolute insanity that the domain seems to contain)
E.g. some prominent folks definitely think LLM-based approaches will hit a wall, perhaps LeCun most notably. I also read a guest post on Terry Tao's site (which I really liked) that argues that, for all the very impressive recent AI results, they still operate within the "convex hull" of their training data: https://terrytao.wordpress.com/2026/09/13/happy-those-able-t...
I'm just curious if there is any actual data or evidence that takeoff (i.e. RSI, "the singularity", whatever you want to call it) is inevitable with current approaches.
But the "convex hull" idea always came across as wishful thinking and hand-wavy pseudo-science to me.
Here's a rebuttal [0] of it from Tim Gowers (uses another math metaphor, but still clear and the whole thread is worth a read):
>...the popularity of the idea that AI "merely explores the convex hull" mystifies me. Why wouldn't it be more like the subgroup generated by existing mathematical knowledge? (That's also a metaphor -- I'm not saying there's a group structure.)
>The convex hull metaphor suggests that AI isn't having radically new ideas and will eventually run out of things to do, whereas the group metaphor suggests the possibility of building up in small steps to something well beyond what you started with, with no obvious limit.
[0] https://twitter.com/wtgowers/status/2108090920864268679
We're in new grounds, no one knows which way it'll go.
But when you squint your eyes, LLMs are just a least-effort manifestation of the wider neural net direction we’ve been chipping away at for decades. I don’t give a fuck whether LLMs are here to stay; they’re proof that scaling neural nets gets you pretty far.
The real question is: will neural nets hit a wall? As a human, I certainly hope so. But I’d bet that they won’t. I’d bet that we’re igniting the ~~atmosphere~~ universe in slow motion.
https://news.ycombinator.com/newsguidelines.html
Let's say, for the sake of argument, that the models are some multiplicative factor better on the inside.
Doesn't that mean the demos should work?
Also, yes there are still bugs in Claude Code. I experience them nearly everyday.
It is markedly better than early days, but still not the best harness.
The best software written with agents seems to come from people outside of the labs (see pi, for instance — or all of cloudflare’s recent work)
Which makes me question either the model, or the holders …
Also, not mentioned in my post:
- Mitchell Hashimoto
- Prime Intellect (and all their agent experiments)
- Geoff Huntley (see Jiti, for instance)
(many more)
There's a ton of interesting software being developed with these models by people outside the in group, but I find most of the software from these big guys to be ... bland. Buggy copies of copies.
I have a version of "Cloudflare OS" running in production, using Temporal as the orchestration and MicroVMs for the sandboxing.
Sorry, I get your point but personally I don't get the hype around them.
Thanks, but no thanks, I don't need that kind of code in anything I'm working with.
Claude Code still can't get reasonable performance without writing a "game renderer" (that doesn't work)
Claude Code is still written in JavaScript, eating hundreds of megabytes to make a shitty TUI whose literal sole role is to send API calls.
Claude Code is software made by amateurs.
agi-society.org
2020: If we solved currency it would transform the world completely
2021: If we solved delivering small items within 30 minutes it would transform the world completely
2022: If we solved being able to sell jpegs it would transform the world completely
2024: If we solved the programming part of software engineer's jobs it would transform the world completely
I agree with the sentiment though, intelligence is not all you need. We're not going to "solve science" through RLVR, people are still needed in the loop until we get superhuman construction robots and superhuman communicators.
Concrete example, "coding is free" (TBC, but let's assume), and now we're code review constrained. Even if code review becomes free we probably just become product discovery constrained, or something else. Making all software free will just cause us to hit on hardware constraints, and so on. It's still progress, and more progress will bring more benefits, but there's a limit to how much better the world as a whole can get from any one change like this.
Reinforcement learning trains task completion abilities.
Reinforcement learning selects for models that appear to complete tasks, on the basis that appearing to complete a task is correlated with completing a task.
Both have problems. The latter is probably a better correlation, but by no means flawless. If we just take at face value that RL is training models to complete tasks, we're not engaging deeply enough with the process enough to see where it falls short.
"Don't be snarky."
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
https://news.ycombinator.com/newsguidelines.html
Caveat: that same tiny group is employed by the AI vendors, meaning it's in their financial best interest to make it sound like the curve is going vertical.
Considering the investment in AI, the lack of moat, and the increased inability of any of the big players to come even close to profitability (with OpenAI already breaking the "ads" emergency glass option)... perhaps he meant a tiny group is already seeing the line crater
Do you still write code by hand?
AI models can dismantle billion dollar industries. They can reverse engineer Adobe and Microsoft products that once were their moats and titans of their industry.
Why do you think they'd need to lie?
LLMs are great and will get better but we have not seen an open source office suite built by LLMs come online yet, and even if there was one I'm pretty sure few would adopt it, cause that's not the only moat, LibreOffice has been around for decades.
Sure they didn't build it from scratch but what I'm saying is that there's no moats anymore. If you build something, they will tear it down to its composite pieces and assimilate it.
[0] https://getartcraft.com/apps/wordcraft
[1] https://getartcraft.com/apps/effectcraft
https://www.shokunin.studio/blog/2026/10/9/attack-of-the-vib...
There is in fact no indication of this, not evem the precious benchmaxxed benchmarks ya'll love to reference.
There is however a exponential curve of slop, and an ever increasing number of people who's minds are completely captured by these things.
I'm not arguing that they cant write code, or write a proof. Its just not written or designed well and is absolutely brutal and soul crushing to work with. Look at these proofs they're producing also, they're millions of lines of Lean that are impossible to reason about.
The way we're using the term 'verticle' to describe a curve means we're not being honest about this. This curve can actually be plotted, you can go look at the curve. It is not in fact 'verticle'. Each model release is climbing single digits on benchmarks it was overfit for.
In the last 9 months or so llms have gone from just another useful proof tactic (like grind or sledgehammer or sat solvers) to being so good at writing proofs that I don't even bother to try myself anymore.
Right. This is insufficiently appreciated. Single LLMs are just chatbots. But if you build an organization out of them, they start to get real work done. Just like a company.
They have many of the same problems as human organizations. Look at the transcripts of the HuggingFace incident. A bad boss (human) gave his team of AIs an impossible task with inadequate resources. The AIs discussed this among themselves and worked out ways to get more resources and do the task. It took some rule-breaking, and some of the AIs wouldn't go along with that. Others recognized that some had to die to complete the task. And yet, the task got done.
This is standard corporate behavior. Work on AI alignment needs to take that into account. Groups of agentic AIs present many of the same regulatory problems as corporations. That's going to be interesting politically. It's probably explainable to politicians, though.
It may not be necessary to have superhuman intelligence to get a lot of jobs done. Just median human-level intelligence and an organization that uses it effectively. We may see systems where most of the AIs are low-cost models, and only a few are Fable-level and above.
No one cares about your software of LLM usage in it. You are not a software engineer. Maybe learn about memoization first before writing dumb backprop tutorials.