It totally makes sense and is such a difficult place to be. Regarding a calling, if you want my advice: find a nearby forest or other natural space, and talk to God about it until you feel what’s right for you. If you don’t live close to nature, just go somewhere where you can talk out loud alone. Maybe look at a photo of Earth taken from space — this helps me remember that what people think about me doesn’t matter. Do whatever you can to put yourself in touch with the Divine Creator of your spirit, and then talk it out until the agitation subsides and you feel at peace. You’ll either know that you shouldn’t take a calling, or that you should, or that it’s not something that God really cares about, so you can choose what works for you right now. And then you can stop worrying. (Which allows you to move on to the next thing to worry about.) :)
About your children… that’s so hard. I don’t know what your goals are for them, but if you want them to stay in the Church without having a terribly painful faith crisis, I believe there are ways of doing that. I remember being heartened and inspired by James McConkie’s interview on Mormon Stories and A Thoughtful Faith podcast. Maybe give it a listen and see if any of it is helpful.
Best of luck!