Anytime I'm invited to dinner or a party, I always tell the host well in advance about my dietary restrictions, and then I immediately offer to help, such as bringing something. If I go and there is still nothing to eat, I don't eat. It's that simple. I have to help myself, not someone else. I'm not going to feel good about myself or my choices if I do something I don't want to do out of peer or societal pressure. It's also possible to decline and be polite about it... I've said things like "Thank you so much for preparing this, I really appreciate it, but I don't eat [food item]" but telling them BEFORE hand helps prevent the situation altogether.