alarmdecoder.com

As discussed in this thread http://www.alarmdecoder.com/forums/viewtopic.php?f=3&t=755, if a SMTP service is down for some reason, SMTP notifiers can block recording of events in the AD log, throw off their timestamps, and also block other notification vectors. As a quick and inelegant solution to the problem, I modified the EmailNotification class in the /opt/alarmdecoder-webapp/ad2web/notifications/types.py module to spin off a separate thread to handle the sending whenever an SMTP notification is needed. This prevents blocked SMTP from delaying the processing of other notifications (but does nothing to solve the same problem if other notification vectors block).

You can adjust the variables retryinterval_sec (socket timeout on a send attempt) and finaltimeout_sec (how long to keep attempting to send) as you see fit. If a send finally times out, it will post the error in the App Log.

I'm not submitting this as a pull request to the developers because it doesn't address the issue in a particularly clean way, or for all notifiers. After posting I'm going to alter my sig to appropriately disclaim "I am not a programmer".

Code: Select all

class EmailNotification(BaseNotification):
    def __init__(self, obj):
        BaseNotification.__init__(self, obj)

        self.source = obj.get_setting('source')
        self.destination = obj.get_setting('destination')
        self.subject = obj.get_setting('subject')
        self.server = obj.get_setting('server')
        self.port = obj.get_setting('port', default=25)
        self.tls = obj.get_setting('tls', default=False)
        self.ssl = obj.get_setting('ssl', default=False)
        self.authentication_required = obj.get_setting('authentication_required', default=False)
        self.username = obj.get_setting('username')
        self.password = obj.get_setting('password')
        self.suppress_timestamp = obj.get_setting('suppress_timestamp',default=False)

    def send(self, type, text):
        message_timestamp = time.ctime(time.time())
        message_timestamp = " [" + datetime.datetime.now().strftime("%I:%M:%S%p %Z %a %m/%d/%y") + "]"  #bwm

        if self.suppress_timestamp == False:
            text = text + "\r\n\r\nMessage sent at " + message_timestamp + "."

        if check_time_restriction(self.starttime, self.endtime):
            msg = MIMEText(text)

            if self.suppress_timestamp == False:
                self.subject = self.subject + " (" + message_timestamp + ")"

            msg['Subject'] = self.subject
            msg['From'] = self.source
            recipients = re.split('\s*;\s*|\s*,\s*', self.destination)
            msg['To'] = ', '.join(recipients)

            sself=self
            t=threading.Thread(target=self.sendthread,args=(sself,current_app._get_current_object(),recipients,msg,message_timestamp,))
            t.start()


    def sendthread(self, sself, caller_app, recipients, msg, message_timestamp):
        s = None
        retryinterval_sec = 60*1                    #set as you see fit
        finaltimeout_sec = retryinterval_sec * 5    #set as you see fit
        timeout_exception = False
        other_exception = False

        numretries = int(finaltimeout_sec/retryinterval_sec)

        for x in range(numretries):
            timeout_exception = False
            other_exception = False

            # Since AD won't catch any exceptions in this separate thread, put everything
            # important in the try loop and catch so a warning can be sent to the AD log file.
            try:
                if sself.ssl:
                    s = smtplib.SMTP_SSL(sself.server, sself.port, timeout=retryinterval_sec)

                else:
                    s = smtplib.SMTP(sself.server, sself.port, timeout=retryinterval_sec)

                if sself.tls and not sself.ssl:
                    s.starttls()

                if sself.authentication_required:
                    s.login(str(sself.username), str(sself.password))

                s.sendmail(sself.source, recipients, msg.as_string())

            except socket.timeout:
                errinfo = sys.exc_info()
                timeout_exception = True

            except:
                errinfo = sys.exc_info()
                other_exception = True

            if (other_exception is True) or (timeout_exception is False):
                break

        if (timeout_exception is True) or (other_exception is True):
            caller_app.logger.error('Notification Email send failed: '
                                    + msg['Subject'] + ' '
                                    + message_timestamp + ' ('
                                    + str(errinfo[0]) + '; '
                                    + str(errinfo[1]) + ')')
        s.quit()