Rate Limiting Introduced

[PeterJ]

The image below shows CPU usage of the server over the last 7 days. You will notice that whilst it's normally very low, we occasionally get loads of 100%. This slows down the website for everyone. Other metrics provided by Digital Ocean increase in-line with CPU usage.



I have been reading up on a Cloudflare feature called Rate Limiting and have set this up this morning.

The limit is currently set to one download of a zip file every 10 seconds.

I know that changing things in Cloudflare can have unforeseen implications, so if you notice anything odd please PM me.

This is not meant to limit legitimate users, instead it's meant to impact people using download managers or similar.

[Pobulous]

I remember that the WOS archive used to ban your IP for doing a mass download - I may have triggered it once when I was trying to hoover up a load of magazines, before they were more conveniently available in other places.

Rate limiting is a much more civilised approach.

[spider]

Should be an effective solution hopefully.

I employed a different method (but that was software specific) but the principle is about the same.

Not punish the good for the bad etc.

[Pobulous]

Not sure if it's related, but the forum seems quite slow this morning.

[Pobulous]

Not sure if it's related, but the forum seems quite slow this morning.

edit, and now it's not, haha.

[SteveSmith]

It does seem a bit suspicious how the CPU is mostly flat, and them hammered for a 24h period.

[Lady Eklipse]

I need to update my tool called ZX Pokemaster and also update TOSEC for the new release. I usually do this one in several years and I do download files with a scraper. Can we arrange something so I can download the files once in several years without being banned?
Thank you.

[PeterJ]

@Lady Eklipse,

I have sent you a PM. Don't worry you won't get banned.