Site integrity incident, 2026-11-13

[The War (ITA)]

So that's why Yesterday when i tried to enter QQ It only showed me a error message saying that It couldn't load the Page.

 

[JohnWolfie34]

Thank you for your hard work QQ staff. I was terrified we'd have an AO3 situation!

 

[Yuhabahha]

Well, that's a disaster avoided.

Ao3 going down was 'meh'.

Losing QQ too would...hurt.

So, adding my voice, thanks to the staff entire for keeping the site running!

 

[NumberSix]

Well, that's a disaster avoided.

Ao3 going down was 'meh'.

Losing QQ too would...hurt.

So, adding my voice, thanks to the staff entire for keeping the site running!

Click to shrink...

As somebody undergoing a severe case of AO3 withdrawal I have to disagree, it going down is a massive disaster.

But QQ, which is one of the few remaining fun internet forums left going down for any reason would be even worse.

Even if I hate reading fiction on BBSes and if they make extremely lousy archives because of the ergonomics and features.

Also, might be a good time for all of us to show our appreciation via some donations to this pace.

 

[Top Sun]

AO3 withdrawal

Click to shrink...

I feel for you, it's hard to find gems on this pile of shit that get pumped daily there, but when you find something it's generally impossible to find somewhere else.

But when QQ got down I started to get frustrated as fuck (lmao) because SB and SV are not the best in terms of quality lately. And a lack of variety too. Unless for Quests, they got diamonds sometimes.

 

[D'znn-Xy'lll MetaChthonia]

is to, say, take down MS365.

Click to shrink...

I can't tell if you specifically chose that Example or aren't aware that Outlook recently was taken down because it was actually a surprisingly easy target that was a core of a lot of Microsoft's services. Excel got partly fucked by it.

Now that you mentioned it yeah that's suspicious as hell but lets say that its a coordinated attack. however that brings up the question of why bring down fanfiction sites of all things

Click to shrink...

From what I heard AO3's likely is hardware based on the time it's taking to come back not software. So broken things need replacing and they don't have the parts. Otherwise I have no clue how it could take 4 weeks to fix software problems without literally rewriting the entire database or being profoundly lazy.

"How dare you have fun?!"

Click to shrink...

You Joke but that's a legitimate stance certain kinds of people have.

What moron searches for fics on the bases of #no beta we die like men or #i wrote this in 3 in the morning instead of sleeping.

Click to shrink...

Thought those Tags were made by the Authors not necessarily actual official ones.

As somebody undergoing a severe case of AO3 withdrawal I have to disagree, it going down is a massive disaster.

But QQ, which is one of the few remaining fun internet forums left going down for any reason would be even worse.

Even if I hate reading fiction on BBSes and if they make extremely lousy archives because of the ergonomics and features.

Click to shrink...

I have 'Fics that only are on AO3 and several I both am waiting to finally Download and ones I want to get back to Reading. Luckily Critian has been motivated by this bullshit to create a Snip Thread on SB. All the MahoAko 'Fics are on AO3.

Also, might be a good time for all of us to show our appreciation via some donations to this pace.

Click to shrink...

Please do.

Spoiler: M0ney pl0x?

 

[Aleh]

If that's the accurate translation:

HA. ABSOLUTE SCRUBLORDS.

Click to shrink...

This was about three years ago, so not about the current attack at all. TBH, we don't know who exactly was responsible for the current attack or what their motives were. The same goes for this bit at QQ.

What we do know is that these attacks can be tied in unexpected ways to the sort of complex geopolitical considerations that we can't even discuss here on QQ without violating Rule 8.

That's why I wasn't comfortable even posting that post without mod review (hence most of the delay in me responding), and I actually revised the post at mod suggestion in order to steer further away from that territory (e.g. my original version of the post mentioned what government sponsored the group).

For our purpose here, however, we can simply note that hacker groups have their own version of street cred, and attacking even hobbyist sites like this one can impact that. We're, for better or for worse, the low-hanging fruit there.

I can't tell if you specifically chose that Example or aren't aware that Outlook recently was taken down because it was actually a surprisingly easy target that was a core of a lot of Microsoft's services. Excel got partly fucked by it.

Click to shrink...

I am aware, yes. The whole thing is why I mentioned it.

And while it may have been surprisingly easy, it was likely a lot more difficult than penetrating QQ or AO3.

 

[Rakdos92]

information

Click to shrink...

Oh damn, that's why the site was down?! I'm glad that the staff was on top of that right from the start. Hopefully we won't see a repeat of that kinda thing anytime soon.

 

[NumberSix]

From what I heard AO3's likely is hardware based on the time it's taking to come back not software. So broken things need replacing and they don't have the parts. Otherwise I have no clue how it could take 4 weeks to fix software problems without literally rewriting the entire database or being profoundly lazy.

Click to shrink...

it is either a database restore and/or them trying to set up the whole environment from scratch. However, in their very last donation drive they said they had spent over 300k on new hardware so I doubt it was a hardware fault. I mean, I could imagine that somehow someone managed to get their hands on all their server credentials and loceked them out or they had to nuke everything from orbit and reprovision everything from scratch but that should be near impossible if they had followed simple security and architectural best practices.
Ah, who am I kidding, they probably did not and the ILM's passwords were probably the default ones.

As I said, there are tools which make all this faster and easier, and simple tools and security policies and design choices that would make it easier.Since they probably have a massive mess and want to deploy their servers manually that is why it is a slow shit show.

Like, for example:
- Using hypervisors so you can spin up a brand new set of DB, backend or other servers out of a golden image/template, overhead below 5%. This also improves frigging segmentation and fail over, because you can seamlessly move workloads from one hypervisor node to another or add more nodes and spin up mor3e VMs fast and easy. This approach is referred to the livestock infrastructure approach. Which also enables easy spin-up of canary and blue-green deployments. (Canry means shifting a small percentage of the traffic to the new version, blue green means having two deployments and gradually upsacling the green version while downscaling the blue one)
- configuration management tools like, ansible, salt, puppet and chef, some of which are agentless and just require an inventory file, dynamic inventory tied to terraform or VM tags and ssh keys, etc.They can all be used for fast bootstrapping of the hypervisor nodes depending on what you end up using. Or if they are so concerned with the muh overhead they can just frigging grab the rancher government ansible role that can deploy a fully CIS hardened, US federal government approved RKE2 cluster to run their workloads.
- Arcitecture - one of the dumbest thing is keeping your authentication and authorization in the same frigging ancient monolithic app.
They should have moved all that to a dedicated microservice that ran a streamlined, highly vetted OIDC provider like Keycloak which is massively deployed and highly vetted with the Ruby on Rails crap just verifying the tokens and enforcing permissions boundaries, hell, there should be gems that integrate with that shit out of the box. And the credentials would of course be stored in a differnet database that the ruby crap can't even connect to, let alone log into or touch.


My brain hurts and my heart bleeds whenever I see a situation where one of the top internet sites I am addicted to doesn't fucking do even the basics right!!!
#literallyshaking

 

[darkxhadow]

Thanks for the hard work

 

[Mairimashita]

Huh, guess explains the weird 'Array' thing that popped up on the side. Thanks for the hard work guy.

Also,

Ao3 going down was 'meh'.

Losing QQ too would...hurt.

Click to shrink...

Truer words have never spoken, but for me it's a double whammy cause there was some good stuff on Ao3 I was reading. Honestly this reminds of the time mangadex got hacked.

 

[CosmicBigfoot]

AO3 is now back in limited capacity.

 

[NumberSix]

AO3 is now back in limited capacity.

Click to shrink...

Yay, just over half a month and slow as a snail marinated in booze and throwing errors for me when I try to log in...
#thisissarcasm.