gunicorn high CPU usage and alarmdecoder-webapp unresponsive

General Discussion

gunicorn high CPU usage and alarmdecoder-webapp unresponsive

Postby cw76 » Sun May 01, 2016 10:50 pm

Hello,

I've got a problem that starting occurring whereby the alarmdecoder-webapp no longer functions. I try going to the web URL and I get a 504 error, gateway timeout.

I'm running this on a Raspberry Pi and have been successfully for months. There are a couple things that I've done recently. One is that I attempted to set up cameras in it and had trouble. So, I followed the manual instructions to enable cameras (though the option was already there, it wasn't working for me--when it attempted to show the camera, it would show a broken image icon and just reload rapidly). That process, after a long time of trying to build things, eventually failed. The webapp continued to work fine from that point, however.

The other thing I did was that I set up OpenRemote to query the REST API every 2 seconds for two different calls. Both were just the GET /alarmdecoder API--once to get the armed status, the other to get faulted zones. This seemed to work fine for a day or two.

Then, for whatever reason, I couldn't access the webapp--nor could OpenRemote.

When I SSH'd into the Pi, I ran Top and noticed that 'python' was using 75 to 90-something % CPU. 'ps auxww | grep python" showed that it was gunicorn running the webapp.

I tried to enable debug logging, but I don't see anything interesting in the instances log, other than it having a debug message "1 workers" every second.

Any thoughts on how I can get more telling logs of why this is sucking up so much CPU? Ideally we could figure out what is wrong and fix it directly, and learn from the experience. But, if wiping this and reinstalling the webapp can be done pretty easily, I'd be open to that, as well.

Oh, and I started from the AlarmDecoder-provided image previously.

Thanks for any help!
Carlin
cw76
newt
newt
 
Posts: 15
Joined: Thu Oct 29, 2015 2:32 pm

Re: gunicorn high CPU usage and alarmdecoder-webapp unrespon

Postby kevin » Sat May 07, 2016 11:56 am

Camera thread needs some work, try deleting your camera from the database. It is a known issue that adding cameras to the webapp increases CPU usage exponentially - it is in our list to look at and fix, and make more robust. Currently it only supports mjpeg streams with http basic auth
Not an employee of the company. Just here to help and keep things clean.
kevin
Platinum Nut
Platinum Nut
 
Posts: 994
Joined: Fri Aug 16, 2013 10:10 am

Re: gunicorn high CPU usage and alarmdecoder-webapp unrespon

Postby cw76 » Sat May 07, 2016 12:18 pm

Hi, Kevin,

OK. I actually reimaged to get it working again, but I kept the old image for debugging.

In case I run into this in the future, how do I delete cameras from the database manually? In that state, I couldn't access the web pages.

Thanks,
Carlin
cw76
newt
newt
 
Posts: 15
Joined: Thu Oct 29, 2015 2:32 pm

Re: gunicorn high CPU usage and alarmdecoder-webapp unrespon

Postby kevin » Sat May 07, 2016 9:51 pm

You would have to shut down gunicorn and manually browse the database on the command line with the sqlite tools
Not an employee of the company. Just here to help and keep things clean.
kevin
Platinum Nut
Platinum Nut
 
Posts: 994
Joined: Fri Aug 16, 2013 10:10 am


Return to General

Who is online

Users browsing this forum: No registered users and 11 guests

cron