WLUG Wiki -- “Service Unavailable”?
I was checking a link on the WLUG Wiki <https://wiki.wlug.org.nz/>, but it is currently returning “503 Service Unavailable”. I hope this is temporary ...
I was checking a link on the WLUG Wiki <https://wiki.wlug.org.nz/>, but it is currently returning “503 Service Unavailable”. I hope this is temporary ...
Just talked to Daniel, who is kindly hosting the wiki. Turns out that AI bot crawlers were hammering the wiki and he had to turn it off this morning. He's investigating how to mitigate that to bring it back online. Cheers, Peter
On Fri, 29 Aug 2025 13:21:11 +1200, Peter Reutemann wrote:
Just talked to Daniel, who is kindly hosting the wiki. Turns out that AI bot crawlers were hammering the wiki and he had to turn it off this morning. He's investigating how to mitigate that to bring it back online.
*Sigh* That figures. The damn things are causing trouble everywhere. The “big” services can afford complex (read: expensive) measures to keep the creepy-crawlers out, but ordinary folk ... not so much.
Just heard back from Daniel and the wiki is up and running again. After spending quite some time on various approaches, he settled on using Anubis to keep scrapers out: https://anubis.techaro.lol/ In case you're seeing a page briefly flashing before seeing the actual content, that's Anubis checking your request. Big thanks to Daniel for keeping things going! Cheers, Peter Aug 29, 2025 15:15:05 Lawrence D'Oliveiro <ldo@geek-central.gen.nz>:
On Fri, 29 Aug 2025 13:21:11 +1200, Peter Reutemann wrote:
Just talked to Daniel, who is kindly hosting the wiki. Turns out that AI bot crawlers were hammering the wiki and he had to turn it off this morning. He's investigating how to mitigate that to bring it back online.
*Sigh* That figures. The damn things are causing trouble everywhere.
The “big” services can afford complex (read: expensive) measures to keep the creepy-crawlers out, but ordinary folk ... not so much. _______________________________________________ wlug mailing list -- wlug@list.wlug.org.nz To unsubscribe send an email to wlug-leave@list.wlug.org.nz
On Fri, 29 Aug 2025 21:35:12 +1200 (GMT+12:00), Peter Reutemann wrote:
Just heard back from Daniel and the wiki is up and running again.
After spending quite some time on various approaches, he settled on using Anubis to keep scrapers out: https://anubis.techaro.lol/
Good there is a simple, practicable technique for this.
In case you're seeing a page briefly flashing before seeing the actual content, that's Anubis checking your request.
Yes, I see that. Though I notice the page doesn’t come up right the first time, you have to refresh before it displays properly. In Firefox, at least.
Big thanks to Daniel for keeping things going!
Thanks from here, too.
participants (2)
-
Lawrence D'Oliveiro -
Peter Reutemann