Rendered at 17:29:26 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
sailingparrot 18 hours ago [-]
500B Luna-equivalent tokens, just for a little fun experiment, which they'll never be able to share widely for very obvious trademark reasons. Even if they could share, not sure exactly what the point would be.
Side projects can be cool and fun even if they have no points to them. Sometimes it's even better when there is no point.
But 500B Luna tokens, that's about 15 SF -> London round trips, or the average CO2 equivalent emissions of an american for about 15 months, based on napkin maths assuming average US energy mix to power the DCs.
Was it really worth it? Should we all start burping 18 tons of CO2 just because we came up with a fun little prompt idea? Should a side project really pollute as much as your neighbor's entire life for more than a year?
I know worrying about climate change is not trendy anymore but there as to be a limit to the amount of not-caring one can have, surely?
Congrats, i guess.
glimshe 7 hours ago [-]
While I don't think this is a good use of the money, you'll go crazy thinking like this.
There are uncountable things I don't value such as fast cars or college football that people drop billions of dollars on. I also bet you spend money on certain things others consider not worthy of the CO2 footprint.
The price system is the best mechanism we have, for all of its faults, to regulate whether something is "worthwhile". The best we can do here is make sure externalities are priced in.
GuB-42 4 hours ago [-]
He didn't just create externalities regarding CO2 emissions. He also created externalities in the price system.
By taking advantage of "unlimited" plans, he paid way less than what his usage has cost OpenAI and Anthropic. Someone, somewhere paid for the real cost of it.
Anyways, is it worth it? Don't know. He wasn't alone on the project, he made a blog post out of it, and others got interested in his experiment. So instead of thinking "15 round trips from London to LA" as someone going back and forth, think more about "15 people going on a round trip from London to LA". Since you mentioned football, we can think of it like a sports team attending an international competition.
jareklupinski 6 hours ago [-]
yea i hope they never see what Red Bull gets up to with their ad spend :P
oefrha 14 hours ago [-]
Guess I also have more sympathy for GitHub’s abysmal uptime now. Thousands of issues and PRs and god knows how many tens/hundreds of thousands comments and events for something that can be entirely local, and that no human will ever read.
dr_dshiv 13 hours ago [-]
I’d love to understand where you get the 18 tons of CO2 for 500B tokens. A source would be really helpful.
GuB-42 5 hours ago [-]
Let's say 1 joule per token, 500B token => 500 GJ => 150 MWh
If we assume 1/3 of a ton of CO2 per MWh (typical in most of the world, including the US), you get 50 tons of CO2 for that experiment.
This is an order of magnitude approximation, but it is enough to conclude that that 18 tons number make sense.
The unknowns are the energy per token but 1 joule per token should get you in the correct order of magnitude. If you want to verify it independently, run a local LLM, measure the power at the plug (in watts), divide it by the number of tokens/second your LLM runs at and you get a rough approximation of the energy use per token in joules. Luna is smallish model, and there is some caching going on, so it may be less, which may explain the 18 tons number GP gave instead of my 50 tons.
The second unknown is the tons of CO2 per MWh. Some datacenters claim they are near zero, because they use renewables or nuclear, but that clean energy they are using is clean energy that is not used to reduce CO2 emissions elsewhere, so I prefer to think in terms of averages.
WheatMillington 15 hours ago [-]
Doesn't that entirely depend on the energy mix going into the data centres in question? How can you possibly know that?
sailingparrot 14 hours ago [-]
i did say
> assuming average US energy mix to power the DCs.
dyauspitr 17 hours ago [-]
Well, that means we should make more clean energy. The answer to a problem is never reduce energy usage.
foopod 15 hours ago [-]
The answer to many problems is reducing energy usage. Progress often comes from using less energy, whether that’s jamming circuits closer together on a silicon wafer or swapping incandescent bulbs for LEDs. More energy spent doesn’t guarantee a better result.
sailingparrot 16 hours ago [-]
This is a false dilemma, yes we absolutely need more clean energy, and yes reducing energy usage is also a good thing to do in many instances. Automotive and aeronautic engineers can tell you about that given how many billions have been invested in making engines consume less.
Should I leave my AC at 65 while I’m on vacation for a month just to save myself the trouble of pressing a button, then tell anyone who calls me out that it’s society’s fault for not producing enough clean energy and I bear no responsibility whatsoever?
Burning 500B token for a fun unsharable side project is a simple choice someone made with (presumably) no gun to their head.
iforgotmypasswo 16 hours ago [-]
I agree with you in spirit, but using less energy is undeniably the answer to numerous problems.
The human brain runs on ~20 Watts (last time I checked). I suspect we’ll find some energy efficiency gains in regard to this technology in the next couple of decades. Computers have a lot less vacuum tubes in them lately, after all.
dyauspitr 16 hours ago [-]
Yeah. There’s two legs to this argument. One is make everything more efficient, which is great and the other is curtail progress, which isn’t.
kennywinker 13 hours ago [-]
> which is great and the other is curtail progress, which isn’t.
Population is growing plus electrification of transport plus more energy for compute plus we want to phase out fossil fuels.
chrysoprace 15 hours ago [-]
We can do both - like running your air con while the sun's out (with solar panels) and turning it off at nighttime.
7 hours ago [-]
nojito 17 hours ago [-]
Still less energy than farming almonds, making clothes, the fridge in your kitchen, etc.
sailingparrot 17 hours ago [-]
> Still less energy than farming almonds, making clothes
I don't even know how to react to this. Pretty sure farming one almond uses a little bit less energy, and farming all the almonds in the world a little bit more, yes. I think you can also eat those almonds, but am not sure either.
> the fridge in your kitchen
I feel like if my own fridge in my own kitchen was consuming on the order of 50MWh over a period a 4 months, I would have noticed, given that that would be over $20,000 at my current residential PG&E rate in SF, or about $60k per year for a fridge.
brandonpelfrey 2 days ago [-]
Unless you explicitly need byte-matching decompilation, there are significantly faster ways to produce a decompilation/C which is functionally equivalent. I need to post about this. What's been working for me is that for every function, Agent A is tasked with writing some code which is semantically equivalent to the original assembly, but not necessarily exactly the same. Agent A also writes tests. Agent A submits the implementation of the function and tests to the harness for it to judge. The harness runs both the original function and the submitted function in a virtual machine/simulator/emulator (the tests define function inputs and starting state). The harness will only accept the implementation if 1) the read/write sequence to RAM is identical to the original function's, and 2) there must be complete line and branch coverage of the original function being decompiled.
I've found this to be robust for decompiling games, while giving the agents enough freedom to write code that is readable and not waste a ton of time making sure e.g. instruction ordering, register assignments, etc. are all exactly the same. For me, having byte-matching decompilation is only one way to produce a decompilation I know is faithful to the original. This "high-level decompilation" process I just described is something agents can do much more quickly.
j2kun 2 days ago [-]
Functional equivalence here, of course, depends on the completeness of the test suite, where byte-identical compiled artifacts does not.
(For example, your approach would not necessarily catch all the same overflow behaviors; the OP expressly claimed that "replicating all bugs" was also important, and many bugs are caused by certain overflow behaviors)
nine_k 2 days ago [-]
> catch all the same overflow behaviors
So you're looking not just for functional but also dysfunctional equivalence %)
j2kun 24 minutes ago [-]
I'm going to start using that term :)
8note 2 days ago [-]
no, thats still functional here. the bugs have to be the same, such that speed runs could still run correctly
SubiculumCode 2 days ago [-]
Byte exact seems only of interest to preserve known bugs etc for cheats/shortcuts/etc.
j2kun 2 days ago [-]
That may be true, but I hate it when people repeat the false idea that functional equivalence requires only a test suite that has full branch/line coverage. Call me triggered :)
That said, I would probably follow this same approach if I were to do this, but with extensive randomized testing as well.
hedgehog 2 days ago [-]
You can do the process in stages. Do the first decompilation mechanically (no LLM), use a SMT solver to show it builds to an equivalent binary to the original, and then use LLM to clean up the code into something idiomatic with the benefit of a correct binary built with the new toolchain. This helps when you want to port across languages or toolchains, and helps protect against toolchain bugs.
burcsahinoglu 24 hours ago [-]
In my project I've been practicing a similar loop as you describe: an agent writes code, SMT solver checks against a spec. A few things I observed:
1. Solver is great at the bugs tests miss (overflow, off-by-one, empty inputs...), its output can be used by the agent to fix it.
2. Solver struggles where you'd assume: loops, non-linear arithmetic and pointer aliasing blow up solve time, floating point is painful.
So your step 2 would cover some of it well, but since the game code is a lot of floating point, SMT would probably have a hard time (get stuck/timeout).
hedgehog 13 hours ago [-]
It depends, the way I set the problem up I define equivalence as Z3 finds a solution within an rlimit of 8M, anything that fails gets machinery added to subdivide or reduce the problem (e.g. assume an index is in the bounds of the buffer it's associated with and add a bounds check). Floating point will in many cases require software emulation but I think that's unavoidable. That might be less of an issue if source and destination are both fairly conformant to IEEE 754 and use modern sizes.
kg 2 days ago [-]
For anything with recorded replays or multiplayer you need to preserve known and unknown bugs for compatibility reasons, not just for cheating.
cowlevel 1 days ago [-]
Or a speedrunning community
momo5502 1 days ago [-]
Read/write sequences to RAM are inconclusive regarding semantic equivalence.
You can have identical sequences and semantics can still be wrong (e.g. when working with the FPU and the float stack, or instructions operating purely on registers). You can also have diverging sequences when having identical semantics, e.g. variables occupying different stack slots, despite computing the same thing.
Also, it feels like just fuzzing the function is slow, compared to simply checking bytes. Fast feedback allows agents to iterate much faster. At some point I wrote a tool that simply proves semantic equivalence, while ignoring irrelevant changes, e.g. register selection or stack slot changes. Turns out it was way too slow as a harness. I still used it to verify some of the remaining functions.
It turns out that getting functions byte-matching is actually not that complicated. Most functions are small and can be one shotted, and over time, agents figured out rules how to control code generation for more complex functions. They wrote down techniques and tricks on how to control register selection for example, or how certain control flow operations may impact basic block ordering, etc. Super insteresting what they figured out. So the further we got into the project, the easier it was to get functions exact.
ndiddy 1 days ago [-]
Games depend enough on stuff like global state or externally provided data that in practice this often isn't enough to guarantee the code actually working the same as the original. You can have tens of thousands of tests that exercise every branch in the game code, and the game still may not behave the same. You really won't know for sure that your code behaves the same under all conditions unless you're able to reproduce the same executable.
cedws 1 days ago [-]
Could you talk more about how you’ve used this technique? I want to start on a similar kind of AI-driven decomp project and I’m looking for any kind of edge I can get.
yababa_y 18 hours ago [-]
I used a similar methodology for https://github.com/emberian/don feel free to dig thru anything in there I think all the notes are in the repo itself... of course it helps having a game that was explicitly designed around networked deterministic lockstep. forces the agent to learn a lot of lessons from reality instead of my typing :)
cedws 15 hours ago [-]
Thank you!
18 hours ago [-]
SubiculumCode 2 days ago [-]
So you restricted it to implementing the same function (same inputs,outputs, dependencies as original?) and prevented the agents from making design decisions by keeping it's scope restricted?
brandonpelfrey 2 days ago [-]
Yes. It can gain more context, but this has been enough. Note, there is also a notion of adversarial review layered on top in which it tries to poke holes in the test plan "you didn't handle this case of XYZ". It isn't actually perfect as a parallel thread said it may miss things like wrapping behaviors. In practice, it's very effective.
cowlevel 1 days ago [-]
Only an LLM can give reasonably meaningful names to most functions and variables automatically, though.
dyauspitr 13 hours ago [-]
That’s not decompilation that’s poorly copying.
nvme0n1p1 2 days ago [-]
> The avid reader of my blog might have noticed that I had previously written two posts that have since been removed. Everyone else might now be wondering which game I am talking about. To both of you I can only say that corporate America was here to ruin our fun.
Let us note without irony that Corporate America is indeed here to ruin your fun. It is their call of duty to wage this type of modern warfare.
asimovDev 10 hours ago [-]
Without context I instantly recognized the crude drawing in the current blog post as MW2 2009 cause the caricature of the sniper rifle looks like the Intervention sniper rifle from the game. It really has become an iconic weapon, a BFG or Quake rocket launcher for Gen Z and Millennials.
TeMPOraL 1 days ago [-]
A C&D-130 in the air, heading for you.
ares623 2 days ago [-]
Friendly reminder that corporate America came at Aaron Swartz.
aetherspawn 2 days ago [-]
The reason this cost so much is because the AI has the ridiculous goal of getting identical assembly output.
The agents would have had to mess around with compiler versions, optimisation options, and the phase of the moon as well.
If you just went for functional equivalence, it would probably cost 10x or 100x less tokens.
Another false economy was using Sonnet instead of a more intelligent model like Sol 6.1 (1), which would have cost more per token, but is 100x or so better at reverse engineering and coding and therefore can chew through the source code much quicker and make fewer mistakes, meaning less work needing to be scrapped.
In my testing doing a similar task, I ran multiple sonnet for weeks and burnt through ~$1000 in tokens to get 20% completion and output that was pretty bad. After switching to Sol 6.1, it finished the whole task in around 2 days, cost around $50, and it did it with zero supervision and a single /goal.
(1): struggle to use Opus for reverse engineering, too many safeguards. OAI has virtually none, and uses way less tokens so is more economical.
Gigachad 2 days ago [-]
Identical output isn’t ridiculous. It’s pretty much a requirement to ensure the game actually is the same in every way. These decomp projects are pitched as a high performance alternative to emulation.
No one is going to use it if it’s a kind of close but not really reimplementation.
Testing functional equivalence is also pretty much impossible. How would you for example test the new one has exactly the same bugs which haven’t been discovered yet. Or doesn’t introduce new ones? This stuff matters for speed runners.
aetherspawn 2 days ago [-]
It’s ridiculous because something as simple as the compiler picking different registers is going to make zero functional difference but give a false negative on assembly compare.
Yet the C code can’t pick what registers to use, so the poor agent is probably shuffling the code around randomly for hours or days until it matches.
That’s probably why the agent dropped down into inline assembly in the first place (the author complained about this), because I bet it’s thinking trace was that this is futile.
Compilers themselves are not even deterministic and running them multiple times creates different assembly.
hypercube33 1 days ago [-]
Isn't MW2 rooted in id tech engines? Last I checked there was raw assembly inside the code base. I would have started with quake 2/3 whatever source code and tried to match that up with Claude first - some of it has to have survived (though I doubt much)
boricj 1 days ago [-]
> Yet the C code can’t pick what registers to use, so the poor agent is probably shuffling the code around randomly for hours or days until it matches.
Humans too do that for matching decompilation. Or at least I imagine so, given that I personally refuse to do that.
People have different goals and will use different techniques to achieve them. The video game reverse-engineering/decompilation community isn't a hive-mind, I went ahead and created ghidra-delinker-extension because I had my own ideas on how to do that.
dezgeg 1 days ago [-]
How do you know the original code didn't use some undefined variable on stack, but happened to consistently work because something else spilled a register containing "suitable" value?
Neywiny 1 days ago [-]
I think I disagree when it comes to bug reproduction. Doesn't seem I'm alone in that.
momo5502 1 days ago [-]
I think you're overlooking an important point here. While the token consumption might feel extremely high, I think it's a bit deceiving.
Due to this strict harness, we were able to use extremely dumb models efficiently.
I think at least 200-300B of tokens (maybe even more) were processed by Luna.
Given that Astra for example is 100x more expensive per token than Luna, a high token consumption doesn't necessarily mean high cost.
So these 300B Luna tokens would equate to 3B Astra tokens and roughly 15B Sol tokens.
Unfortunately a huge part of the logs is lost, so I can't do a detailed analysis on which models were used, how much they cost, etc.
lukewarm707 15 hours ago [-]
Interesting that you used ida mcp.
I tried giving glm 5.3 ghidra. I found that it frequently did better just reading the actual assembly, where needed writing decompilers in real time, in the terminal.
The reasoning showed that it was able to write ghidra as needed and indeed where necessary simply read assembly tokens.
Trying to think about ghidra probably confused it.
I also think that there is a gradient of losing intelligence in refusals, opposed to a simple boolean refuse. I suspect that using an abliterated model would boost the intelligence way up.
easterncalculus 18 hours ago [-]
> Due to this strict harness, we were able to use extremely dumb models efficiently.
With all due respect I seriously doubt that. What are you doing that's different? Almost every test between just Opus and Opus conducting cheaper agent swarms ends up with Opus getting better results at a lower cost. People set up harnesses just like yours and see this.
RussianCow 15 hours ago [-]
> Almost every test between just Opus and Opus conducting cheaper agent swarms ends up with Opus getting better results at a lower cost.
Do you have any links to these tests? I've had a hunch that this was true for a while but haven't found any benchmarks showing it.
aetherspawn 20 hours ago [-]
I don’t recommend always using Astra for RE but in my experience I’d use Sol as minimum. As of recently, perhaps upgraded Terra is good value, but in general it seems like you want the smartest model that is economical to run 24/7 and Astra isn’t economical.
Ohentis 21 hours ago [-]
I wouldn't consider it an unreasonable goal when it's been the standard nearly all previous pre-ai re projects have held themselves to.
stavros 1 days ago [-]
Identical assembly output is the only way to get functional equivalence, otherwise you're getting equivalence in some percentage of situations, but not 100%. Your bug is my feature etc.
ChoosesBarbecue 18 hours ago [-]
It’s probably worth noting that the author is decompiling a game that he’s had ~10 years of experience modding. And has been involved in finding vulnerabilities in [0].
makes me feel old recognising momo from the alteriwnet/fourdeltaone days
edg5000 2 days ago [-]
I sense the approach overcomplicates things. I wonder how long this would have taken in a single session. Maybe this is actually a textbook example of something where subagents make sense, but when I first started LLMs I was often overcomplicating the workflow with all kinds of orchestration. Now I just use one agent, it better allows controlling the output even if the agent works slightly longer. Most time is spent by me writing prompts and reviewing work anyway (for me at least).
hansvm 2 days ago [-]
The best models have O(1k tokens per second). A billion tokens, sequentially, takes 1-2 weeks. 500B takes 500x that ... not fast. The problem might not have required 500B tokens, but if it needed anything within a couple orders of magnitude then something like the given approach (or anything else yielding equivalent results in exchange for parallelism) was mandatory.
speedstyle 2 days ago [-]
Yeah, I don't think the problem needed billions of tokens
hansvm 2 days ago [-]
That's plausibly true. I've definitely seen absurd token costs abused and wasted. I've also seen simple problems require absurd token counts regardless of prompt quality. Which factors made this problem require 1000x fewer tokens than they used?
WASDx 1 days ago [-]
When you see these absurd numbers, they are re-counting cached input for every turn. So a simple tool call when your context size is at 500k counts as another 500k to the sum.
Total Opus 5.5 token usage on OpenRouter last week is 5000B, presumably not counting cached input.
WheelsAtLarge 2 days ago [-]
Interesting, if all software can be decompiled and copied what is the future of software. Will all software be SaaS? A time where the majority of PCs will be terminals? Game consoles are almost there. It's only a small jump for all software to go that way.
Edit:
Here's a possibility.
The future of software is agent only software. We ask for a result, agent asks questions from us, agent uses the specialized software, user gets result. We subscribe to an AI assistant and specialized agents. We are almost there,at least the start. The future of PC's as we know them are numbered. OSs,CLI,compilers and whatever will melt into AI assistants. Say goodbye to writing software for people.
vagab0nd 2 days ago [-]
Non-SaaS, close sourced software is already dead, no? I think pretty soon we'll all be using bespoke software written by the AIs. I already use throwaway software designed for each job because it's so cheap to do so.
edg5000 2 days ago [-]
I wonder about this too. Although, cracking was always possible. So what changed is that people can now modify, improve proprietary software with zero technical knowledge. I can now tell Astra or Fable (maybe not Fable because of safeguards?) to add a feature to Adobe Photoshop and it might actually succeed even when prompting purely on a functional level (e.g. the functionality I want out of the program). It would take days, but I think it would succeed. Call it PhotoBench?
patrickk 1 days ago [-]
Eventually Fable-level models will be abliterated and hosted on Openrouter too, like so many others have been. Remember Chinese models are only a few months behind American ones at this point and are trained on their outputs, and can be abliterated after release.
Adding features to software is still kind of hard unfortunately, even with source code.
unsnap_biceps 2 days ago [-]
A future of all SaaS only works if you can't just tell a LLM what you want and get a custom implementation. I've always done a lot of personal projects, but my velocity has increased dramatically and I've replaced a number of projects that I used to pay for with custom ones that work well enough. I can't imagine that SaaS is going to be a viable model unless it involves a community that wants a unified experience, like a multiplayer game.
bitwize 2 days ago [-]
Even then, interact with a SaaS enough and you may be able to "distill" a spec out of it that's sufficient enough for an LLM to replicate it closely enough. If you feed the LLM screenshots or video of people using it, it will be able to recreate it even more accurately.
georgemcbay 2 days ago [-]
> If you feed the LLM screenshots or video of people using it, it will be able to recreate it even more accurately.
You could just have the LLM agent use the software itself and then duplicate the functionality it finds, don't even need the screenshots or video.
Unless the software is doing something extremely novel, the LLM can just do this all itself after you point it toward a url for the software with instructions to clone what it finds there.
It'll be interesting to see if SaaS companies and/or providers of internet services like Cloudflare try to stop this sort of agentic cloning of SaaS products.
Like, obviously they would kinda want to if/when this sort of thing becomes commonplace, but how do you do it while allowing your software to be agent-friendly for non-cloning uses, and who is going to want to tell their customers they can't use agents with their SaaS for fear of their software being cloned? Kind of a bad situation with either option.
Daishiman 2 days ago [-]
Most software organizations pay for is effectively a SaaS or something where software is a minor part of the artifact and support is where the real money is.
dex0shubham 3 hours ago [-]
This is a really interesting project, I wanted to try this myself but at times the agent produces code that looks right but it isn't. Is there a way to automate the the process of finding these kind of mistakes ?
codedokode 16 hours ago [-]
I think proprietary software will move into cloud. For example, a photo editor will keep processing algorithms on the server and client will be just caching the results and displaying them.
If you make a program with a simple logic, anyone can vibe-copy it in a day. You cannot sell it.
If you make a program with advanced algorithms and complicated math, anyone can clean-room reverse-engineer it (by 2 different AIs) and vibe-copy. Furthermore, reverse-engineered data, such as algorithms and secrets, will be embedded into weights of a next LLM version.
Yes cloud won't work when you have unstable Internet due to certain events, but having a program sometimes not working is better than having a program anyone can copy. Am I wrong?
Also, vibe-copy is a made up word.
jorl17 14 hours ago [-]
I am starting to believe this as well.
But I'm not mad that we can reverse engineer stuff (well, reverse engineer is too nice, in a way: we can sort of clone). In fact, I love it. A friend recently shared that it feels illegal, but I find it to be a dream. The teenager in me who loved open-source would be celebrating intensely for the fact that now we can "open-source" other software. Or modify it. I've got mods in one way or another for almost every game I've been playing recently, it's truly a dream come true. I haven't open-sourced most of it, but mostly because I imagine the world is filled with vibed stuff.
I may make money by building software, and that certainly has included keeping things proprietary, but I don't ideologically love that. If the world progresses towards this, I think we're better off.
(But I'll admit I think I'm in the minority here).
mk89 13 hours ago [-]
You will need a sub-second network speed, in some cases < X ms, if you want to offer an acceptable UX for "heavy user interaction" software, like the photo editor you mentioned.
Even the cloud won't save you, though: some SaaS companies are far too easy to replicate internally. It kind of sucks, because these used to be small companies with a couple of use cases in mind that nobody ever wanted to "reimplement internally" and they did give work to real people. But now, throw some agents at it, and a few 100s $ later you have replicated their product and can run it in your (internal) network.
pornel 15 hours ago [-]
Even if you run code server-side, it may be hard to keep algorithms secret if there's anything that can act as an oracle for a reverse-engineering agent.
...or competitors will just burn their own tokens on reinventing the same thing from scratch. It's going to get really hard to come up with some secret sauce that can't be replicated by a swarm of agents that never sleeps.
BTW, GPL is also totally screwed by vibe-copying.
Capricorn2481 13 hours ago [-]
> If you make a program with a simple logic, anyone can vibe-copy it in a day. You cannot sell it.
I just don't think this is true. People have had the option to Pirate for many years, but many don't. Balatro was in Love2D, so it was effectively open source. But it was one of the most successful indie games in recent memory. Someone made an entire recreation of the the World of Warcraft back end up to wrath of the lich King, effectively letting you play on custom servers for free, and yet people would rather pay for wow classic
Most people don't want to download a random exe outside of the steam store, and even less people will try reverse engineering themselves. Maybe if one of the modifications is really good, or somebody makes it extremely convenient.
I could see one of these engineers getting a lot of attention in the media, and then maybe a company will step in and insist that anthropic put some restrictions on this type of work. But I don't think Activision is going to see this as humongous threat to their bottom line.
stevefan1999 1 days ago [-]
Reverse engineering dynamic dispatches like vtable and fat pointers/slices and traits would be hell. Different padding and compiler settings also contribute a lot.
One of the few problem is that the decompiler is not always reliable. That forces you to go read the assembly, and it is not apparent to decipher the right kind of feng shui, not even from human before the LLM era. I used to play CTFs and my conclusion is exactly that.
This is extremely apparent when there are self-modifying code (e.g. JIT) is involved. You need a stepping debugger to read the right control flow, because the code will diverge based on the instruction pointer and regions you jumped into. At this point static analysis like IDA and Ghidra stopped working.
saagarjha 15 hours ago [-]
I think the idea is that if you lift perfectly it doesn’t really matter that you understand the branch target
bob1029 1 days ago [-]
Reverse engineering existing game binaries seems kind of silly when a lot of the most important AAA knowledge is already available to the general public.
The LLMs have presumably already consumed information like this as part of their training sets. Converting between Hammer and Unity scale is a fairly trivial linear operation. You can dump the BSPs to recover geometry and rapidly accelerate map development using timings that are already known to work.
Attacking the raw binary directly is certainly impressive, but it's totally unnecessary.
petetnt 1 days ago [-]
Great article that really demonstrates why the copious amounts of AI generated PC ports do not fit under ”preservation” guise, which to me has been the point of doing these ports in the first place and what many of the enthusiasts before the LLM wave were aiming for.
> There are no noticeable bugs and all features of the original game are present.
>
> The remaining functions have been reworked repeatedly. While they still don’t match byte for byte, we believe their semantics are correct.
As there are no way to confirm this outside of ”believinh”, what you are left is with a end product that might contain thousands of micro changes that essentially make the game something else than it was intended to be.
fxtentacle 1 days ago [-]
Fully agree.
Since even Fable 5.1 cannot reliably remember what it wrote into "memory" files before the last compaction, why would we trust it to not forget about some side-effect when handling assembly that is so verbose it'll surely exceed the context and, hence, necessitate compaction.
momo5502 1 days ago [-]
Well, I received a few DMCAs and thus stopped. We would've likely brought this as close to 100% as we can, but it's just pointless given the circumstances. So our belief and a few final tests were enough.
jakubmazanec 18 hours ago [-]
> Architectural decisions were not questioned, as long as they aligned with the goal.
Yes, that's the main problem I'm having with agentic workflows. Sometimes you don't care about architecture of something if it just works, but more complex stuff can't work well without good architecture. And currently it seems that there is still no substitute for good human taste.
thway15269037 2 days ago [-]
I struggle to understand what legal leverage they used to threaten him to remove every detail about the game. Can someone post the game name and company name?
So, if you reverse-engineer game X and post reverse-engineered code, what exactly do you infringe, how and in which jurisdiction? What changes if it is done via LLM?
(I understand that LLM decompilation is absolutely out of hand right now and something surely will come to trample the fun. But what and when? I suppose american LLMs will have their system prompt updated to forbid any reversing help and report suspicious activity straight to legal hotline)
xnx 2 days ago [-]
Call of Duty: Modern Warfare 2 (2009) (from a previous version of the page)
bitwize 2 days ago [-]
> So, if you reverse-engineer game X and post reverse-engineered code, what exactly do you infringe, how and in which jurisdiction? What changes if it is done via LLM?
In 1986 there was a federal court case, Whelan Associates Inc. v. Jaslow Dental Laboratory, in which it was ruled that the "structure, sequence, and organization" of a computer program was protected by copyright, and thus independently produced software could be found infringing if it copied these elements, even if the code were not copied (or mechanically translated) verbatim. This led to a six-year period in which computer software enjoyed generous copyright protection, such that "clones" of copyrighted software were effectively infringing. It wouldn't be until the early nineties that other court rulings would tighten the rules again, notably Computer Associates International, Inc. v. Altai Inc.. The 3-step "abstraction-filtration-comparison" test has been used by most courts since 1992 to determine whether nonliteral parts of program code are eligible for copyright protection, and whether another, independently written program is infringing.
HOWEVER, the Whelan standard was never actually overturned or stricken from U.S. law due to legislation or litigation! And companies have sued and won under the Whelan standard! Most notably, Oracle in their copyright and patent case against Google regarding Java APIs in Android. Google ultimately prevailed, but only because the Supreme Court ruled that Google's use of the APIs was fair use; they did not overturn the Federal Circuit's finding that the "structure, sequence, and organization" of the Java declaration code was ineligible for copyright! And I doubt that the Supreme Court would similarly smile on a reverse-engineered complete video game!
I believe that these reverse-engineered projects infringe copyright, under the Whelan standard and perhaps under the stricter Altai standard as well. You do not get a free pass because the original code was in C and yours is in Rust; or because you created a slightly different version of each function in the original code.
awongh 18 hours ago [-]
I didn't realize that the google - oracle decision was just decided over fair use!
But this seems to hinge on the "clean room"- is this a separate set of case law or is it related? As in, if they had been more judicious about the way the game was implemented could they have claimed it was not under copyright?
nullpoint420 2 days ago [-]
Yet AI companies distilling copyrighted books and repositories is somehow okay?
And let's say there's a future where an AI model could zero-shot the game itself. What then?
thway15269037 2 days ago [-]
That implies US jurisdiction plus that was made at the time where "hey, let's actually reverse-engineer and copy our competitor" was very expensive and time-consuming, so slapping one company would discourage everyone else to sink money in it. Currently it seems the latter is becoming either automatic (still somewhat expensive) or even free.
bitwize 2 days ago [-]
Committing copyright infringement against software vendors was automatic and free before, when it involved merely copying files, maybe after cracking the copy protect. It was still illegal. What makes you think this time would be any different? The fact that an LLM did it? Whatever an LLM produces humans assume responsibility for.
thway15269037 2 days ago [-]
When you copy a file, it's trivially simple to prove it's a copy and not an original work.
How can one prove that X is a copy when none of the source code match original? Clean room re-implementations are not illegal after all, if we go that way (yeah, yeah, I know about clean room argument).
While I personally do not think it would be different, but a lot of LLM folks are, and even some companies (rushing to blatantly clone and de-compile stuff). Kinda strange feeling sitting and looking around in the midst of it, y'know.
imtringued 1 days ago [-]
Doing a clean room implementation would require you to build a test suite using an AI against the binary and then you build a test server against which other people, not you, can run their own implementations and you just report how many tests pass.
rasz 2 days ago [-]
>what legal leverage
PIF owns EA, when they invite you to a meeting you might start to worry about foil lined room and carpentry tools lying around
CSm1n 16 hours ago [-]
This isn't EA, it's the other franchise.
mawadev 1 days ago [-]
It is wild how AI ends up in the trenches of plagiarism, going from art to text, from software to games...
sashank_1509 15 hours ago [-]
What is the level of a game that can be decompiled this way. Can a game like say assassins creed or Crisis be decompiled this way? How are the art assets replicated (do we provide it separately). It seems crazy if this is possible!
Capricorn2481 13 hours ago [-]
A lot of people replicating these games are not replicating the art assets. They are making a new engine that points to the old art assets, as redistributing the art assets could get them in a lot more trouble.
sashank_1509 13 hours ago [-]
Are they replicating physics and the rendering pipeline, or are they basically using known entities like Unreal, Havoc etc but letting the agent tune it to match the game?
Capricorn2481 8 minutes ago [-]
That I'm not sure of, because this isn't my area, my info comes directly from youtubers working on this.
But from what I understand, Bevy doesn't come with those things and they did not use something like Havoc, so AI had to translate them as close as it could to match the movement correctly.
To answer your question, it seems like these decompiled game engines are not humongous by themselves. So I wouldn't be surprised if it could be done with Assassin's Creed. But a Youtuber did mention the games they are doing this with (MW2, Skate 3, Minecraft) are ideal starting points.
Is it the same Call of Duty: Modern Warfare 2 decompilation project? If yes, then why author says he/she does no release source code? If I remember correctly, it has to be open sourced Modern Warfare 2 modernized engine and so on? Or do I miss something?
ChoosesBarbecue 18 hours ago [-]
The Bevy rewrite is an independent project from someone else.
eigenblake 16 hours ago [-]
As an interesting legal question, I wonder if the author would be clear to share the Lean/TLA+ specification of the system's behavior...
Neywiny 1 days ago [-]
I do wonder if LTO would obfuscate this a bit more. Presumably if you can't chunk it into small compilation units, you can't find a way to make C from the machine code.
2 days ago [-]
georgemcbay 2 days ago [-]
In case anyone is curious about the obvious question of which game they are talking about... based on months-old reddit posts (which seem like links to prior progress reports of the same project) the game in question appears to be Call of Duty: Modern Warfare 2 (the original 2009 version).
A previous version of the page said the game was Call of Duty: Modern Warfare 2 (2009).
Fizz43 1 days ago [-]
the 80% is a meaningless AI generated % by the way.
momo5502 1 days ago [-]
It's not. Those 80% are calculated by a simple script that just compares bytes.
vivzkestrel 2 days ago [-]
- i want to very very badly see a post of this using LM studio and one of the open source models
- please someone do it
asimovDev 9 hours ago [-]
I am attempting this with a 100 MB GameMaker (HoloCure) game to make mobile and console ports for fun. Because it uses YYC[0] instead of running in a VM which is how GameMaker games usually work (to my crude understanding of the issue) I have to reverse engineer it instead of just extracting assets and putting them into a project template.
I am using Qwen 3.8 27B-8bit in oMLX and Pi agent with GhidraMCP on my MacBook M3 Max. So far it ran for like 10 hours total and I haven't checked in on it much but it seems to have extracted the audio assets successfully and wrote a whole bunch of helper scripts for Ghidra to extract YYC functions which took several hours to run. I put the project on hold cause I didn't want to keep running it at 100% fans for hours every day. It blew through ~200k context of the model 3 times by now. I am not confident in this project, especially because I have very little experience doing grown up reverse engineering that isn't decompiling a C# or Android apps so I can't really guide it.
Interesting read and a thread here. I am working on decompiling two obscure abandoware games from my childhood and it's been a difficult and quirky process...
Even after building whole infra for that and not pursuing identical byte output it's been pretty token hungry process
otagekki 21 hours ago [-]
what if someone does this to the Windows OS?
esafak 2 days ago [-]
Can anyone think of any lessons to draw from this for normal development, where we don't have oracles to serve as guardrails? I write specs but the agents still find ways to insert bugs between the lines. Oh well, job security.
ashdnazg 1 days ago [-]
It's not really comparable. When decompiling you only care about a one time result. Code quality doesn't matter much. Stability doesn't matter much. Only whether the result is accurate. Once you have the matching decompiled code, all tools used until that point will be usually thrown away.
Normal development isn't so lucky, you do care about code quality. So you have to review the code the agents write and handle the testing yourself. The agents will not do what you want, they'll do what they think you want, and you're the only person who knows what you think.
momo5502 1 days ago [-]
The obvious lesson is that you need some kind of test/harness, even for normal development. Your spec will always leave room for interpretation in a sense that is undesired.
Obviously matching functions byte-byte is not it, but I think with enough creativity, you can have guardrails and oracles almost everywhere.
E.g. before diving into feature development, you can have agents generate a test suite based on your implementation first. Then, when developing features, this test suite shows what broke and agents can double check if that was intended.
How that test suite looks like obviously highly depends on the environment.
caniload 15 hours ago [-]
[flagged]
robertbobrowski 1 days ago [-]
[flagged]
Uptrenda 1 days ago [-]
I'm more interested in how the OP managed to convince claude to do what was probably against Activison's TOS. I am assuming they didn't hand specifics of the game but just a blank binary. Because if you had of given specifics claude would have checked the TOS of the IP holder, found that the company doesn't want people to decompile their software, and refused to work on it. Or am I missing something here?
Capricorn2481 13 hours ago [-]
It doesn't seem to care. I had it do this for a small game the other day just to try it out. I fed it decompiled Ghidra code. It doesn't seem to have the same restrictions that developing malware or hacking web servers has.
Maybe companies will ask anthropic to crack down on it at some point.
AtlasBarfed 17 hours ago [-]
Are there focused and tuned decompilation AI agents, outside of these llm ones which I think are going to be more expensive?
The compilation recompilation and automatic porting represents such a quantum leap in terms of software, preservation and software freedom.
Almost like every single piece of software written in the last 50 years can be repurposed by the end users for what they need, not the oppression copyright laws and judgements handed out by a corrupt judiciary and legislative branch
llj123 5 hours ago [-]
[flagged]
WillAdams 2 days ago [-]
To save folks looking this up:
>At current 2026 API rates, 500 billion AI tokens would cost roughly $100,000–$750,000 depending on the model, with most flagship models in the $150–$400 per million input tokens range
Gigachad 2 days ago [-]
This stuff is all done on highly subsidized subscription plans. I suspect Antropic is quite happy to sell these people $100k of compute for $4k because it boosts their growth numbers and they can tell investors once they stop subsidizing, this will grow to $100k. Despite the fact that most of this stuff simply wouldn't be done without the subsidization.
chii 2 days ago [-]
the exact same arguments were said about uber's business at the start.
Yet, it is now profitable.
The bet is that people realize how valuable these services are, and despite complaining, they still would pay the higher price. This realization would not happen without this initial subsidy from investors.
It isn't too different from drug dealer's first sample free...
thway15269037 2 days ago [-]
Uber business model wasn't a subscription "for 10 bucks you can travel 900 lightyears a week"
nmfisher 2 days ago [-]
Last I checked, Uber underperformed the S&P index since its IPO. Even Softbank didn't make a great return on its investment. The growth story was definitely oversold.
Also, Uber was peak ZIRP + COVID, money was cheap and growth was easy. I think the mountain to climb is a lot steeper now.
Gigachad 2 days ago [-]
There’s also examples where this didn’t work. Moviepass for example.
Taxis were an established profitable business model and the uber subsidisation wasn’t anywhere near as much as AI subsidies.
0x457 1 days ago [-]
Moviepass failed because they thought it's going to be like Gym membership - people buy and don't go, but guess what? People love going to movie theaters. On its own its not bad, see AMC Stubs, but AMC owns the theater, they sell you popcorn and soda. OpenAI an Anthropic is closer to AMC than Moviepass.
xienze 1 days ago [-]
> Moviepass failed because they thought it's going to be like Gym membership - people buy and don't go, but guess what? People love going to movie theaters.
You think developers won't/don't stretch subscriptions to the absolute limit? The AI subscription model is like the gym model except a large percentage of the customers work out 24/7/365.
0x457 23 hours ago [-]
You're missing a piece: MoviePass was selling subscriptions while buying tickets from theaters at retail prices. That's why I mentioned AMC A-List, their subscription model works pretty well for them because they control per ticket and subscription price.
The price these companies charge per token is irrelevant to a discussion about subscriptions. You're comparing two selling prices, when what matters is the actual cost. It's like those ads saying, “Normally $20 million, but today it's only $99!” for something that costs $10 to make.
Sure, they're losing money now. But the question is whether the tokens you use through your subscription will eventually cost them less to generate than you pay for that subscription. Their per-token selling price doesn't tell you that.
isubkhankulov 2 days ago [-]
Disagree with your second paragraph. Uber/lyft subsidized into deep negative margin territory around ~2015 or so. Anthropic (and OpenAI) are subsidizing but not losing money on these consumer plans.
edg5000 2 days ago [-]
> subsidizing but not losing money
????
HWR_14 1 days ago [-]
The claim is that uber and lyft lost money on each ride but that openai and anthropic make money on inference. Just not enough to pay the cost of developing the models. The difference is that uber and lyft had to change their pricing (or payment) models to make money, where anthropic or openai could just sell enough inference (at some level of sales).
isubkhankulov 1 days ago [-]
to clarify, I mean that the big model companies are charging consumers much less than equivalent API pricing but they're not actually losing money so its more of a steep at-cost discount for inference. It likely does not fully cover amortized R&D just like the other reply stated but it does still cover marginal inference cost (GPU/power/etc)
Uber and Lyft were paying drivers $X but charging users way less than $X so they were literally burning investor money to get market share.
xienze 1 days ago [-]
> It likely does not fully cover amortized R&D just like the other reply stated but it does still cover marginal inference cost (GPU/power/etc)
Why does this point come up over and over again, pretending that you can truly separate training and inference costs. Yes, they are separate things but the value OpenAI and Anthropic are presenting to the world is "we're the absolute best, no one else comes close." Well, to keep that up you can't just not train for extended periods of time. You have to keep that engine going non-stop when there's free Chinese models nipping at your heels. You can be profitable on "just inference" all you want but if training expenses dwarf that, you're not going to be profitable overall, and that's the bottom line.
edg5000 1 days ago [-]
> does still cover marginal inference cost
Simply comparing to the larger models on OpenRouter implies that the pure hosting costs (equipment + power + minimal overhead) still exceed plan pricing if we assume all users always use their full weekly allowance.
So my conclusion is that it only works because the majority of users doen't fully utilize their allowance. Last month I used almost nothing of my Claude 20x plan (did use Codex though).
Gigachad 2 days ago [-]
It's profitable if you don't count the expenses.
oblio 1 days ago [-]
I think you're joking but that's Amodei & co are claiming with a straight face.
Also, 500 billion is 500,000 millions. At the lower end of your 140/mil estimate that's 70 million dollars. Even at my 2/mil lookup for Chinese frontier models that's one million. Show your math for 100k-750k, please.
WillAdams 2 days ago [-]
It was a search for "average cost of 500 billion ai tokens" which may or may not have been the correct phrasing, but seemed straight-forward enough that at first blush, accepting the AI-generated answer seemed reasonable.
j2kun 2 days ago [-]
I struggle to believe that someone would find it worth that much money to have a decompiled version of a game they also commit to keeping private.
overfeed 13 hours ago [-]
> I struggle to believe that someone would find it worth that much money to have a decompiled version of a game they also commit to keeping private.
The tone of the blog suggests they are using the game to build out tooling and expertise; the capability to decompile large projects on the scale of AAA games is not worth nothing, e.g. security or "competitor analysis" that lays trade secrets bare.
AI is a great at laundering code, I thought open-source software was the prime prey, but I should have thought bigger.
girvo 2 days ago [-]
> they also commit to keeping private
Reading between the lines:
"The avid reader of my blog might have noticed that I had previously written two posts that have since been removed. Everyone else might now be wondering which game I am talking about. To both of you I can only say that corporate America was here to ruin our fun."
Keeping it private likely wasn't the original plan...
GaggiX 2 days ago [-]
The vast majority of these tokens are cache input tokens so even at API price the cost would be much lower.
thway15269037 2 days ago [-]
Even with 95% cache hit, and on cheapest chinese model, it would still be in tens of thousands of dollars (I assume that of 500B tokens at least 10B would be in "output").
If we switch gears to Kimi K* then if would very quickly escalate in hundred thousands USD.
Side projects can be cool and fun even if they have no points to them. Sometimes it's even better when there is no point.
But 500B Luna tokens, that's about 15 SF -> London round trips, or the average CO2 equivalent emissions of an american for about 15 months, based on napkin maths assuming average US energy mix to power the DCs.
Was it really worth it? Should we all start burping 18 tons of CO2 just because we came up with a fun little prompt idea? Should a side project really pollute as much as your neighbor's entire life for more than a year?
I know worrying about climate change is not trendy anymore but there as to be a limit to the amount of not-caring one can have, surely?
Congrats, i guess.
There are uncountable things I don't value such as fast cars or college football that people drop billions of dollars on. I also bet you spend money on certain things others consider not worthy of the CO2 footprint.
The price system is the best mechanism we have, for all of its faults, to regulate whether something is "worthwhile". The best we can do here is make sure externalities are priced in.
By taking advantage of "unlimited" plans, he paid way less than what his usage has cost OpenAI and Anthropic. Someone, somewhere paid for the real cost of it.
Anyways, is it worth it? Don't know. He wasn't alone on the project, he made a blog post out of it, and others got interested in his experiment. So instead of thinking "15 round trips from London to LA" as someone going back and forth, think more about "15 people going on a round trip from London to LA". Since you mentioned football, we can think of it like a sports team attending an international competition.
This is an order of magnitude approximation, but it is enough to conclude that that 18 tons number make sense.
The unknowns are the energy per token but 1 joule per token should get you in the correct order of magnitude. If you want to verify it independently, run a local LLM, measure the power at the plug (in watts), divide it by the number of tokens/second your LLM runs at and you get a rough approximation of the energy use per token in joules. Luna is smallish model, and there is some caching going on, so it may be less, which may explain the 18 tons number GP gave instead of my 50 tons.
The second unknown is the tons of CO2 per MWh. Some datacenters claim they are near zero, because they use renewables or nuclear, but that clean energy they are using is clean energy that is not used to reduce CO2 emissions elsewhere, so I prefer to think in terms of averages.
> assuming average US energy mix to power the DCs.
Should I leave my AC at 65 while I’m on vacation for a month just to save myself the trouble of pressing a button, then tell anyone who calls me out that it’s society’s fault for not producing enough clean energy and I bear no responsibility whatsoever?
Burning 500B token for a fun unsharable side project is a simple choice someone made with (presumably) no gun to their head.
The human brain runs on ~20 Watts (last time I checked). I suspect we’ll find some energy efficiency gains in regard to this technology in the next couple of decades. Computers have a lot less vacuum tubes in them lately, after all.
Progress can be great. It's often used as a weapon against people. See the story of the luddites https://en.wikipedia.org/wiki/Luddites
Population is growing plus electrification of transport plus more energy for compute plus we want to phase out fossil fuels.
I don't even know how to react to this. Pretty sure farming one almond uses a little bit less energy, and farming all the almonds in the world a little bit more, yes. I think you can also eat those almonds, but am not sure either.
> the fridge in your kitchen
I feel like if my own fridge in my own kitchen was consuming on the order of 50MWh over a period a 4 months, I would have noticed, given that that would be over $20,000 at my current residential PG&E rate in SF, or about $60k per year for a fridge.
I've found this to be robust for decompiling games, while giving the agents enough freedom to write code that is readable and not waste a ton of time making sure e.g. instruction ordering, register assignments, etc. are all exactly the same. For me, having byte-matching decompilation is only one way to produce a decompilation I know is faithful to the original. This "high-level decompilation" process I just described is something agents can do much more quickly.
(For example, your approach would not necessarily catch all the same overflow behaviors; the OP expressly claimed that "replicating all bugs" was also important, and many bugs are caused by certain overflow behaviors)
So you're looking not just for functional but also dysfunctional equivalence %)
That said, I would probably follow this same approach if I were to do this, but with extensive randomized testing as well.
1. Solver is great at the bugs tests miss (overflow, off-by-one, empty inputs...), its output can be used by the agent to fix it.
2. Solver struggles where you'd assume: loops, non-linear arithmetic and pointer aliasing blow up solve time, floating point is painful.
So your step 2 would cover some of it well, but since the game code is a lot of floating point, SMT would probably have a hard time (get stuck/timeout).
Also, it feels like just fuzzing the function is slow, compared to simply checking bytes. Fast feedback allows agents to iterate much faster. At some point I wrote a tool that simply proves semantic equivalence, while ignoring irrelevant changes, e.g. register selection or stack slot changes. Turns out it was way too slow as a harness. I still used it to verify some of the remaining functions.
It turns out that getting functions byte-matching is actually not that complicated. Most functions are small and can be one shotted, and over time, agents figured out rules how to control code generation for more complex functions. They wrote down techniques and tricks on how to control register selection for example, or how certain control flow operations may impact basic block ordering, etc. Super insteresting what they figured out. So the further we got into the project, the easier it was to get functions exact.
Call of Duty: Modern Warfare 2 (2009)
https://web.archive.org/web/20260925153118/https://momo5502....
https://web.archive.org/web/20260925153131/https://momo5502....
Come at me, corporate America.
The agents would have had to mess around with compiler versions, optimisation options, and the phase of the moon as well.
If you just went for functional equivalence, it would probably cost 10x or 100x less tokens.
Another false economy was using Sonnet instead of a more intelligent model like Sol 6.1 (1), which would have cost more per token, but is 100x or so better at reverse engineering and coding and therefore can chew through the source code much quicker and make fewer mistakes, meaning less work needing to be scrapped.
In my testing doing a similar task, I ran multiple sonnet for weeks and burnt through ~$1000 in tokens to get 20% completion and output that was pretty bad. After switching to Sol 6.1, it finished the whole task in around 2 days, cost around $50, and it did it with zero supervision and a single /goal.
(1): struggle to use Opus for reverse engineering, too many safeguards. OAI has virtually none, and uses way less tokens so is more economical.
No one is going to use it if it’s a kind of close but not really reimplementation.
Testing functional equivalence is also pretty much impossible. How would you for example test the new one has exactly the same bugs which haven’t been discovered yet. Or doesn’t introduce new ones? This stuff matters for speed runners.
Yet the C code can’t pick what registers to use, so the poor agent is probably shuffling the code around randomly for hours or days until it matches.
That’s probably why the agent dropped down into inline assembly in the first place (the author complained about this), because I bet it’s thinking trace was that this is futile.
Compilers themselves are not even deterministic and running them multiple times creates different assembly.
Humans too do that for matching decompilation. Or at least I imagine so, given that I personally refuse to do that.
People have different goals and will use different techniques to achieve them. The video game reverse-engineering/decompilation community isn't a hive-mind, I went ahead and created ghidra-delinker-extension because I had my own ideas on how to do that.
Due to this strict harness, we were able to use extremely dumb models efficiently. I think at least 200-300B of tokens (maybe even more) were processed by Luna.
Given that Astra for example is 100x more expensive per token than Luna, a high token consumption doesn't necessarily mean high cost. So these 300B Luna tokens would equate to 3B Astra tokens and roughly 15B Sol tokens.
Unfortunately a huge part of the logs is lost, so I can't do a detailed analysis on which models were used, how much they cost, etc.
I tried giving glm 5.3 ghidra. I found that it frequently did better just reading the actual assembly, where needed writing decompilers in real time, in the terminal.
The reasoning showed that it was able to write ghidra as needed and indeed where necessary simply read assembly tokens.
Trying to think about ghidra probably confused it.
I also think that there is a gradient of losing intelligence in refusals, opposed to a simple boolean refuse. I suspect that using an abliterated model would boost the intelligence way up.
With all due respect I seriously doubt that. What are you doing that's different? Almost every test between just Opus and Opus conducting cheaper agent swarms ends up with Opus getting better results at a lower cost. People set up harnesses just like yours and see this.
Do you have any links to these tests? I've had a hunch that this was true for a while but haven't found any benchmarks showing it.
[0]: https://github.com/momo5502/cod-exploits
Total Opus 5.5 token usage on OpenRouter last week is 5000B, presumably not counting cached input.
Edit: Here's a possibility.
The future of software is agent only software. We ask for a result, agent asks questions from us, agent uses the specialized software, user gets result. We subscribe to an AI assistant and specialized agents. We are almost there,at least the start. The future of PC's as we know them are numbered. OSs,CLI,compilers and whatever will melt into AI assistants. Say goodbye to writing software for people.
Someone has vibe coded the entire Adobe suite btw: https://getartcraft.com/apps
https://abliteration.ai/
You could just have the LLM agent use the software itself and then duplicate the functionality it finds, don't even need the screenshots or video.
Unless the software is doing something extremely novel, the LLM can just do this all itself after you point it toward a url for the software with instructions to clone what it finds there.
It'll be interesting to see if SaaS companies and/or providers of internet services like Cloudflare try to stop this sort of agentic cloning of SaaS products.
Like, obviously they would kinda want to if/when this sort of thing becomes commonplace, but how do you do it while allowing your software to be agent-friendly for non-cloning uses, and who is going to want to tell their customers they can't use agents with their SaaS for fear of their software being cloned? Kind of a bad situation with either option.
If you make a program with a simple logic, anyone can vibe-copy it in a day. You cannot sell it.
If you make a program with advanced algorithms and complicated math, anyone can clean-room reverse-engineer it (by 2 different AIs) and vibe-copy. Furthermore, reverse-engineered data, such as algorithms and secrets, will be embedded into weights of a next LLM version.
Yes cloud won't work when you have unstable Internet due to certain events, but having a program sometimes not working is better than having a program anyone can copy. Am I wrong?
Also, vibe-copy is a made up word.
But I'm not mad that we can reverse engineer stuff (well, reverse engineer is too nice, in a way: we can sort of clone). In fact, I love it. A friend recently shared that it feels illegal, but I find it to be a dream. The teenager in me who loved open-source would be celebrating intensely for the fact that now we can "open-source" other software. Or modify it. I've got mods in one way or another for almost every game I've been playing recently, it's truly a dream come true. I haven't open-sourced most of it, but mostly because I imagine the world is filled with vibed stuff.
I may make money by building software, and that certainly has included keeping things proprietary, but I don't ideologically love that. If the world progresses towards this, I think we're better off.
(But I'll admit I think I'm in the minority here).
Even the cloud won't save you, though: some SaaS companies are far too easy to replicate internally. It kind of sucks, because these used to be small companies with a couple of use cases in mind that nobody ever wanted to "reimplement internally" and they did give work to real people. But now, throw some agents at it, and a few 100s $ later you have replicated their product and can run it in your (internal) network.
...or competitors will just burn their own tokens on reinventing the same thing from scratch. It's going to get really hard to come up with some secret sauce that can't be replicated by a swarm of agents that never sleeps.
BTW, GPL is also totally screwed by vibe-copying.
I just don't think this is true. People have had the option to Pirate for many years, but many don't. Balatro was in Love2D, so it was effectively open source. But it was one of the most successful indie games in recent memory. Someone made an entire recreation of the the World of Warcraft back end up to wrath of the lich King, effectively letting you play on custom servers for free, and yet people would rather pay for wow classic
Most people don't want to download a random exe outside of the steam store, and even less people will try reverse engineering themselves. Maybe if one of the modifications is really good, or somebody makes it extremely convenient.
I could see one of these engineers getting a lot of attention in the media, and then maybe a company will step in and insist that anthropic put some restrictions on this type of work. But I don't think Activision is going to see this as humongous threat to their bottom line.
One of the few problem is that the decompiler is not always reliable. That forces you to go read the assembly, and it is not apparent to decipher the right kind of feng shui, not even from human before the LLM era. I used to play CTFs and my conclusion is exactly that.
This is extremely apparent when there are self-modifying code (e.g. JIT) is involved. You need a stepping debugger to read the right control flow, because the code will diverge based on the instruction pointer and regions you jumped into. At this point static analysis like IDA and Ghidra stopped working.
https://github.com/ValveSoftware/halflife/blob/master/pm_sha...
The LLMs have presumably already consumed information like this as part of their training sets. Converting between Hammer and Unity scale is a fairly trivial linear operation. You can dump the BSPs to recover geometry and rapidly accelerate map development using timings that are already known to work.
Attacking the raw binary directly is certainly impressive, but it's totally unnecessary.
> There are no noticeable bugs and all features of the original game are present.
>
> The remaining functions have been reworked repeatedly. While they still don’t match byte for byte, we believe their semantics are correct.
As there are no way to confirm this outside of ”believinh”, what you are left is with a end product that might contain thousands of micro changes that essentially make the game something else than it was intended to be.
Since even Fable 5.1 cannot reliably remember what it wrote into "memory" files before the last compaction, why would we trust it to not forget about some side-effect when handling assembly that is so verbose it'll surely exceed the context and, hence, necessitate compaction.
Yes, that's the main problem I'm having with agentic workflows. Sometimes you don't care about architecture of something if it just works, but more complex stuff can't work well without good architecture. And currently it seems that there is still no substitute for good human taste.
So, if you reverse-engineer game X and post reverse-engineered code, what exactly do you infringe, how and in which jurisdiction? What changes if it is done via LLM?
(I understand that LLM decompilation is absolutely out of hand right now and something surely will come to trample the fun. But what and when? I suppose american LLMs will have their system prompt updated to forbid any reversing help and report suspicious activity straight to legal hotline)
In 1986 there was a federal court case, Whelan Associates Inc. v. Jaslow Dental Laboratory, in which it was ruled that the "structure, sequence, and organization" of a computer program was protected by copyright, and thus independently produced software could be found infringing if it copied these elements, even if the code were not copied (or mechanically translated) verbatim. This led to a six-year period in which computer software enjoyed generous copyright protection, such that "clones" of copyrighted software were effectively infringing. It wouldn't be until the early nineties that other court rulings would tighten the rules again, notably Computer Associates International, Inc. v. Altai Inc.. The 3-step "abstraction-filtration-comparison" test has been used by most courts since 1992 to determine whether nonliteral parts of program code are eligible for copyright protection, and whether another, independently written program is infringing.
HOWEVER, the Whelan standard was never actually overturned or stricken from U.S. law due to legislation or litigation! And companies have sued and won under the Whelan standard! Most notably, Oracle in their copyright and patent case against Google regarding Java APIs in Android. Google ultimately prevailed, but only because the Supreme Court ruled that Google's use of the APIs was fair use; they did not overturn the Federal Circuit's finding that the "structure, sequence, and organization" of the Java declaration code was ineligible for copyright! And I doubt that the Supreme Court would similarly smile on a reverse-engineered complete video game!
I believe that these reverse-engineered projects infringe copyright, under the Whelan standard and perhaps under the stricter Altai standard as well. You do not get a free pass because the original code was in C and yours is in Rust; or because you created a slightly different version of each function in the original code.
It reminded me of this: https://malus.sh/ "cleanroom as a service" - which I guess was kind of a joke, but looks pretty serious now.... (discussion back then: https://news.ycombinator.com/item?id=47350424)
But this seems to hinge on the "clean room"- is this a separate set of case law or is it related? As in, if they had been more judicious about the way the game was implemented could they have claimed it was not under copyright?
And let's say there's a future where an AI model could zero-shot the game itself. What then?
How can one prove that X is a copy when none of the source code match original? Clean room re-implementations are not illegal after all, if we go that way (yeah, yeah, I know about clean room argument).
While I personally do not think it would be different, but a lot of LLM folks are, and even some companies (rushing to blatantly clone and de-compile stuff). Kinda strange feeling sitting and looking around in the midst of it, y'know.
PIF owns EA, when they invite you to a meeting you might start to worry about foil lined room and carpentry tools lying around
But from what I understand, Bevy doesn't come with those things and they did not use something like Havoc, so AI had to translate them as close as it could to match the movement correctly.
To answer your question, it seems like these decompiled game engines are not humongous by themselves. So I wouldn't be surprised if it could be done with Assassin's Creed. But a Youtuber did mention the games they are doing this with (MW2, Skate 3, Minecraft) are ideal starting points.
https://github.com/vladtrc/iw4L
https://www.reddit.com/r/ReverseEngineering/comments/1vxig19...
- please someone do it
I am using Qwen 3.8 27B-8bit in oMLX and Pi agent with GhidraMCP on my MacBook M3 Max. So far it ran for like 10 hours total and I haven't checked in on it much but it seems to have extracted the audio assets successfully and wrote a whole bunch of helper scripts for Ghidra to extract YYC functions which took several hours to run. I put the project on hold cause I didn't want to keep running it at 100% fans for hours every day. It blew through ~200k context of the model 3 times by now. I am not confident in this project, especially because I have very little experience doing grown up reverse engineering that isn't decompiling a C# or Android apps so I can't really guide it.
0 - https://manual.gamemaker.io/lts/en/Settings/YoYo_Compiler.ht...
Even after building whole infra for that and not pursuing identical byte output it's been pretty token hungry process
Normal development isn't so lucky, you do care about code quality. So you have to review the code the agents write and handle the testing yourself. The agents will not do what you want, they'll do what they think you want, and you're the only person who knows what you think.
Obviously matching functions byte-byte is not it, but I think with enough creativity, you can have guardrails and oracles almost everywhere.
E.g. before diving into feature development, you can have agents generate a test suite based on your implementation first. Then, when developing features, this test suite shows what broke and agents can double check if that was intended.
How that test suite looks like obviously highly depends on the environment.
Maybe companies will ask anthropic to crack down on it at some point.
The compilation recompilation and automatic porting represents such a quantum leap in terms of software, preservation and software freedom.
Almost like every single piece of software written in the last 50 years can be repurposed by the end users for what they need, not the oppression copyright laws and judgements handed out by a corrupt judiciary and legislative branch
>At current 2026 API rates, 500 billion AI tokens would cost roughly $100,000–$750,000 depending on the model, with most flagship models in the $150–$400 per million input tokens range
Yet, it is now profitable.
The bet is that people realize how valuable these services are, and despite complaining, they still would pay the higher price. This realization would not happen without this initial subsidy from investors.
It isn't too different from drug dealer's first sample free...
Also, Uber was peak ZIRP + COVID, money was cheap and growth was easy. I think the mountain to climb is a lot steeper now.
Taxis were an established profitable business model and the uber subsidisation wasn’t anywhere near as much as AI subsidies.
You think developers won't/don't stretch subscriptions to the absolute limit? The AI subscription model is like the gym model except a large percentage of the customers work out 24/7/365.
The price these companies charge per token is irrelevant to a discussion about subscriptions. You're comparing two selling prices, when what matters is the actual cost. It's like those ads saying, “Normally $20 million, but today it's only $99!” for something that costs $10 to make.
Sure, they're losing money now. But the question is whether the tokens you use through your subscription will eventually cost them less to generate than you pay for that subscription. Their per-token selling price doesn't tell you that.
????
Uber and Lyft were paying drivers $X but charging users way less than $X so they were literally burning investor money to get market share.
Why does this point come up over and over again, pretending that you can truly separate training and inference costs. Yes, they are separate things but the value OpenAI and Anthropic are presenting to the world is "we're the absolute best, no one else comes close." Well, to keep that up you can't just not train for extended periods of time. You have to keep that engine going non-stop when there's free Chinese models nipping at your heels. You can be profitable on "just inference" all you want but if training expenses dwarf that, you're not going to be profitable overall, and that's the bottom line.
So my conclusion is that it only works because the majority of users doen't fully utilize their allowance. Last month I used almost nothing of my Claude 20x plan (did use Codex though).
https://developers.openai.com/api/docs/pricing https://platform.claude.com/docs/en/about-claude/pricing
OpenAI and Anthropic are both 10/mil in.
https://openrouter.ai/z-ai/glm-5.3#providers https://openrouter.ai/moonshotai/kimi-k3#providers Other frontier models are like 1-3/mil in.
Also, 500 billion is 500,000 millions. At the lower end of your 140/mil estimate that's 70 million dollars. Even at my 2/mil lookup for Chinese frontier models that's one million. Show your math for 100k-750k, please.
The tone of the blog suggests they are using the game to build out tooling and expertise; the capability to decompile large projects on the scale of AAA games is not worth nothing, e.g. security or "competitor analysis" that lays trade secrets bare.
AI is a great at laundering code, I thought open-source software was the prime prey, but I should have thought bigger.
Reading between the lines:
"The avid reader of my blog might have noticed that I had previously written two posts that have since been removed. Everyone else might now be wondering which game I am talking about. To both of you I can only say that corporate America was here to ruin our fun."
Keeping it private likely wasn't the original plan...