JQuery to the rescue
It has been a while since I have looked at JQuery, most of my spare time has been spent using Angularjs.
So when writing a simple page (well it started off simple but is now pretty complex like all development), I had a list of elements and each element contained some elements (could be <p>, <i>, <a>, text etc) and also an excellent bootstrap toggle button as shown here:

Now, what I wanted was to catch the clicks for the li element and the toggle buttons. What I started with was this (I can hear you scream!!!):

I know it is bad. Initially (in my defense) I only had a div element to check but then a few days later, it was an li, then a b, and then maybe an image in the future. When would the madness stop.
So after googling a bit, I came up with:

So all I did was put a couple of classes in my html and then I used the jquery closest() method to find the first ancestor of that class. If I find a toggle then I must have pressed the toggle, if not then I pressed somewhere inside the li element.
That was it.