Page 1 of 1
Website issues
Posted: Mon Dec 30, 2019 12:15 pm
by PeterJ
Hello,
I'm getting a lot of alerts over the last 24 hours from Digital Ocean that CPU usage is running high.
Sorry if things are running slow for anyone. We may eventually need to move to a larger virtual server. I will continue to monitor.
Peter
Re: Website issues
Posted: Mon Dec 30, 2019 12:38 pm
by moroz1999
I can suggest doing some steps for this:
1. You can enable file cache for zxImage library, this will save a lot of CPU resources.
2. Profiling SQL requests to find the slowest ones and setting up appropriate indexes would help a lot.
3. You can also set up some data-caching for the heaviest SQL requests: if some popular SQL request cannot be improved further by indexes, you can serialize it and put it into file system (or Redis if it's available on your hosting) for some hours or even days.
4. You can cache the whole HTML responses, that would be the most efficient. Use output buffering to gather a response, make some hash from request uri and save the response with the name of this hash. Next time you get such request uri, just take an HTML from cache and serve it to user or invalidate the cache if it's too old.
Re: Website issues
Posted: Mon Dec 30, 2019 12:57 pm
by Ast A. Moore
Haven’t noticed any slowdown on my end. Granted, I rarely used the main site; mostly the forums.
Re: Website issues
Posted: Mon Dec 30, 2019 1:14 pm
by moroz1999
For the first suggestion:
$converter is an instance of ZxImage class.
Code: Select all
//make some folder for file cache
if (!is_dir($folderForCache)) {
mkdir($folderForCache);
}
//set cache path to converter
$converter->setCachePath($folderForCache);
//enable file caching
$converter->setCacheEnabled(true);
After this all converted images would be automatically stored in the created folder by ZxImage converter. Same converter will also purge the cache automatically from time to time basing on cache file creation dates.
Re: Website issues
Posted: Mon Dec 30, 2019 1:36 pm
by Einar Saukas
moroz1999 wrote: ↑Mon Dec 30, 2019 12:38 pm
I can suggest doing some steps for this:
1. You can enable file cache for zxImage library, this will save a lot of CPU resources.
2. Profiling SQL requests to find the slowest ones and setting up appropriate indexes would help a lot.
3. You can also set up some data-caching for the heaviest SQL requests: if some popular SQL request cannot be improved further by indexes, you can serialize it and put it into file system (or Redis if it's available on your hosting) for some hours or even days.
4. You can cache the whole HTML responses, that would be the most efficient. Use output buffering to gather a response, make some hash from request uri and save the response with the name of this hash. Next time you get such request uri, just take an HTML from cache and serve it to user or invalidate the cache if it's too old.
If you can collect query statistics, I will gladly analyze and optimize them for you.
Although I agree that caching (options 1 and 4) are your best bet.
Re: Website issues
Posted: Mon Dec 30, 2019 5:15 pm
by moroz1999
I'm using all four suggestions in commercial projects, they are all helpful on their levels
Re: Website issues
Posted: Mon Dec 30, 2019 5:28 pm
by PeterJ
Thanks both, I will see what data I can get and will share it with you both.
Peter
Re: Website issues
Posted: Tue Dec 31, 2019 1:34 am
by RMartins
Also check if the website machine, was not infected, with some kind of digital money harvesting bot.
It can also be someone exploring the system, like crawling the page and the search features to extract data.
Which is kind of dumb, since the data (ZX-DB) is open and free.
Re: Website issues
Posted: Wed Jan 01, 2020 12:12 am
by moroz1999
RMartins wrote: ↑Tue Dec 31, 2019 1:34 am
Also check if the website machine, was not infected, with some kind of digital money harvesting bot.
That's actually a nice advise. You can check it this way:
1. Download all files to separate directory
2. Compare it to repository using some bin comparison tool like windiff or beyond compare, this will show quickly any modifications in any scripts.
BTW, are you using 7.4 already? Each PHP update seems to have some real performance improvements in the core, and soon there would be a JIT available as well. You can also try to set up a script preloading list, this would help a lot as well.
But my guess is:
1. ZX images converter needs a file cache enabled, this one is CPU-demanding if done on a fly.
2. SQL requests indexes.