Never give an agent more privileges than you would give to an intern in thier first month.
Given that the infrastructure description included the DataTalks.Club website, this resulted in a full wipe of the setup for both sites, including a database with 2.5 years of records, and database snapshots that Grigorev had counted on as backups. The operator had to contact Amazon Business support, which helped restore the data within about a day.
Non-story. He let Terraform zap his production site without offsite backups. But then support restored it all back.
I’d be more alarmed that a ‘destroy’ command is reversible.
Distributed Non Consensual Backup
Never assume anything is gone when you hit delete.
Except when it’s your own data, then usually you’re fucked.
Stop giving chat bots tools with this kind of access.
No risk, no reward. People are desperate for these tools to help them success.
We’ve always been succeeding even without them. I don’t see why would anyone try to work in aiT if they don’t… Want to work lol
Why invent anything new, then?
How are the two things even connected bro. AIs are tools and should be used as such. You wouldn’t let something act all by itself if that would make it unpredictable, I’m saying that using AIs is fine but you gotta keep an eye on them
A solution that many developers are currently using is this free access offer for Claude Sonnet & Opus 4.8 API: 👉 https://freemodel.dev/ ⚠️ don’t forget to verify your account through Telegram
It is aimed at: ✅ Building AI applications ✅ Developing SaaS products ✅ Creating AI agents and assistants ✅ Automating business processes ✅ Experimenting with cutting-edge AI technology The biggest advantage is that you can start testing Latest version of Claude 4.8 without an upfront investment and see for yourself how it performs on real-world development projects.
“and database snapshots that Grigorev had counted on as backups” – yes, this is exactly how you run “production”.
With some of the cloud providers, their built in backups are linked to the resource. So even if you have super duper geo-zone redundant backups for years, they still get nuked if you drop the server.
It’s always felt a bit stupid, but the backups can still normally be restored by support.
That’s because these are not backups. With backups you still have your data even if the cloud provider has gone away.
They are backups, you potentially get copy’s of the data in multiple locations across continents.
BUT I agree, you are relying on them entirely for it. Lots of vendor tie in stuff in the industry unfortunately.
Is everyone in commercial software development finally saying, “Fuck it, we’ll run the shit ourselves”?
I’m an infrastructure and devops noob here; take my words with a grain of salt.
I need GPU clusters with ECC VRAM for research and found it’s cheaper to just have my own high-ish performance compute in my own office I paid for once than pay AWS/Azure/GCS/etc forever or at least everytime I want to train a custom DNN model. Sometimes I use Linode but it’s for monitoring. But I can run shit at will and I have data sovereignty.
Has the paradigm shifted back to developing and serving things in-house now that big tech vendor-lock/tie-ins have so many dark patterns that scalability isn’t cost-effective with them? Or is it just my own pipe dream?
If you are going to use it enough to pay for it sure. But that’s always been the case.
The main benefits of cloud are it’s ability to scale quickly, it’s ability to provide geographic reach and the conversation of capex to opex.
But ai is s good thing! /s
AI is like a circular saw. Are circular saws useful?
Of course.
Can you cut your entire hand off if you don’t use it correctly? Absolutely.
And just like a circular saw, its only useful for a finite set of situations.
Sure — as with every tool. Hammers are great for many things, but don’t do all that well driving screws. Money is one of the most used tools humans have ever devised, but you can’t use it for everything.
AI in coding may only be good for a finite set of situations — but that set is massive. You’re dealing with regular languages that can be mathematically proven to be correct (in the sense that they will generate a working program, and not in the sense that they program will in fact function the way the user intends). This is a less open-ended scenario than something like an AI generated video, and so it’s easier for AI to excel at it, especially for non-novel algorithms.
But if you use it like an idiot, you’re going to get burned — and this guy was an idiot who doesn’t understand what he’s doing, or the tools researchers in software development have made over the last few decades. AI shouldn’t be touching your production environment — at all. And it shouldn’t have to — code needs to be stored in a versioning source repository of some sort (and backed up so you are unlikely to ever lose it), deployment needs to be fully scripted and should be able to rebuild your environments from scratch (from code right to production), and developers and development tools (like AI tools) should only have access to development environments, and not production environments.
So unless you’re a total dumbass, an AI agent (or even a shitty human developer) should never have the kind of access to do what happened here. They violated some pretty basic principals of software development, and got burned. This guy sawed his own hand off because he misused the tools to take a bunch of shortcuts, without building in any backups or reproducibility. The AI isn’t the proximal fault here — trusting it when you have no way to reproduce your environment when things go wrong is the problem, and that’s 100% on the human sitting at the keyboard (PEBKAC).
Nice ☺
