An update: zmNinja, ES 6.x and notifications

Update Log:

• Jun 26, 2021: Updated that if you enable certain flags in zmeventnotification.ini I will be storing sensitive information but that is ONLY if you explicitly enable it in your config (turned off by default)
• Oct 24: Updated that I am now storing first 10 characters of your alarm title and body (seems in some cases they are being sent as empty, which causes issues)

Around 2 months ago, I posted that ES users should brace for breaking changes. I released ES 6 and zmNinja 1.6 together last week.

Please note that you need to update to ES6 for notifications to work in zmNinja 1.6.

I also wanted to post some updates on changes to how push now works:

• In the past, your ES would directly send a push notification to Google FCM servers and your devices would get a push notification
• In ES 6, I migrated to FCMv1 which has changed how authentication works. FCMv1 uses OAuth which is more secure than long standing API keys. What I then had to do was route your ES messages to an ES cloud server (which I had to write) and my ES cloud server would forward the message forward to FCM. This is not unique — every 3rd party push notification service like PushOver, PushWoosh, etc. does the same thing. It just so happens that I preferred not to do it so far because it was more work. Now I have to do it.

What that now means for you is:

• You are limited to 8000 push messages per month, per device. It is pretty easy to disable this limit, but please don’t. This number is quite generous and more than what many (most?) 3rd party push providers provide. The reason I put this up is for those folks who don’t care about proper zone configuration and generate gobs of pushes per second. My intermediary server is hosted as a google cloud function and I will be paying for it on a monthly basis (I guess I’ll figure out over time how much that is)
• Please note that given your ES push now routes through my intermediary, I store some key log elements so I can map errors if they occur. Specifically, here is what I store:

android device push:

2021-06-26 10:31:54.188 EDT
send_push9l7rt5hrddpz token: ...ez5LZK6wya os:android image: key message elements:{'android': {'channel': 'zmninja', 'icon': 'ic_stat_notification', 'priority': 'high', 'ttl': None, 'image': '<present>', 'tag': None}, 'non_platform_specific': {'log_id': 'NONE', 'title': 'Ended:Back...', 'body': 'Motion: Ba...', 'log_raw_message': 'NO'}}

ios device push:

2021-06-26 10:32:14.798 EDT
send_pushhnczmca1858d token: ...kAak5qgxb0 os:ios image: key message elements:{'ios': {'headers': {'apns-priority': '10', 'apns-push-type': 'alert'}, 'thread_id': 'zmninja_alarm', 'image': '<present>'}, 'non_platform_specific': {'log_id': 'NONE', 'title': 'CatGenie 2...', 'body': 'Motion All...', 'log_raw_message': 'NO'}}

What you see above is an actual push, some user sent using ES 6. I log the last 10 digits of your token (the token is a very long string), the OS and key android and iOS headers the ES sends out. I also store whether this push included an image or not, but don’t store the image as it may contain a private URL with username and password. I also store the timestamp of when this push was sent (actually, google does that in my log view). Finally, I store the first 10 characters of your alarm title and body.

On Jun 26, 2021: I added a few options that you can turn on in zmeventnotification.ini that can help me in debugging if you enable them

# Make this yes if you are having push notification issues 
# and want me to take a look at what is going on on my server side. # Default is no
# WARNING: If you enable this, I'll get to see your push notification # details including your full token and possibly your ZM user and 
# password that you provide as part of the picture URL. 
# I try and automatically replace it, but no guarantees if the regex
# fails in certain conditions. SO KEEP THIS OFF unless you've 
# chatted with me
# and you know we have exchanged notes. You should turn this off once debugging is over
fcm_log_raw_message=no

# If you want me to help you debug what is going wrong in FCM, it 
# helps if you add a unique ID here so that I know what to look for # it can be anything. Default is NONE
fcm_log_message_id=I_LOVE_LAMBSHANKS

Here is an example log that I get when you enable fcm_log_raw_message AND fcm_log_message_id (Note you can enable one and disable the other)

2021-06-26 10:17:52.684 EDT
send_pushhncz39yg6a42 token: ...LbGcpb30fU os:ios image: key message elements:{'ios': {'headers': {'apns-priority': '10', 'apns-push-type': 'alert'}, 'thread_id': 'zmninja_alarm', 'image': '<present>'}, 'non_platform_specific': {'log_id': 'I_LOVE_LAMBSHANKS', 'title': 'Front Door...', 'body': '[s] detect...', 'log_raw_message': 'yes'}}
token: ...LbGcpb30fU os:ios image: key message elements:{'ios': {'headers': {'apns-priority': '10', 'apns-push-type': 'alert'}, 'thread_id': 'zmninja_alarm', 'image': '<present>'}, 'non_platform_specific': {'log_id': 'I_LOVE_LAMBSHANKS', 'title': 'Front Door...', 'body': '[s] detect...', 'log_raw_message': 'yes'}}
2021-06-26 10:17:52.684 EDT
send_pushhncz39yg6a42 RAW message ID:PLIABLEPIXELS being sent to FCM:{"apns": {"headers": {"apns-priority": "10", "apns-push-type": "alert"}, "payload": {"aps": {"badge": 1, "mutable-content": 1, "sound": "default", "thread-id": "zmninja_alarm"}}}, "data": {"eid": "62091", "image_url_jpg": "https://<MY_SERVER_NAME>/zm/index.php?view=image&eid=62091&fid=objdetect&width=600&username=something&password=something "mid": "2", "notification_foreground": "true"}, "notification": {"body": "[s] detected:(coral) person:77% Motion Front Path at 10:17 AM, 26-Jun", "title": "Front Door Alarm (62091)"}, "token": "d7QzQrsBD6A:APA9<REST_OF_MY_TOKEN>"}

Notes:

• <MY_SERVER_NAME> will be the actual server IP/hostname
• <REST_OF_MY_TOKEN> will be your full device token
• username=something&password=something is after my code replaces your passwords, so most likely your login details will be protected but NO GUARANTEES — there may be some condition where the regex fails

Cloud Function details & source code

Finally, everything in the ES is open source. So it is proper that I also tell you what exactly the cloud function code does. What I can’t tell you however is the key I use to sign authentication, because that is private to me. The details of this cloud function are here.