It sounds like you haven't been honest with your husband since the beginning. Dishonesty and deceit will only role into more lies and deceit. 

 You need to be honest with yourself first, then with your family. You need to make a choice. Do you want to drag your family through the heartache that will inevitable ensue if you leave your husband, and follow what you think your heart wants? Or do you get some self-control, tell this man that it ain't going to happen, and stick by your husband and family? Keep in mind that there are 3 families involved in this mess. Your husband and kids, his wife and kids, and your husband and his brother's family. You need to decide if it is worth it. But if it is, then why didn't you two wait for each other before you married?

I have seen this exact issue within my ex husbands family.  I have seen it destroy aman's confidence and left the other man with trust issues with his next partner.  It has destroyed a family without there being any hope of recovery.  You need to work out which brother you really need to be with before you destroy both families.  If it already is not too late. All the best in your endeavour to find the answers that you really need

I guess if its worrying you, you suspect that he would leap at the opportunity to have a 'roll in the hay' with you & you're not sure you have the strength to resist. There might even be an element of the forbiddenness of it making it more exciting (in your &/or his head).

Since you reckon the feelings were impossible to hide last time he was around, there's a fair chance your hubby knows about this dynamic. (Tho he may be too alarmed to bring it up.) So you need to bring it up w/ him: say that you aren't ready to have to deal w/ his brother & you really want to avoid seeing him if at all possible... Reassure hubby of your commitment to your marriage & ask him to help you to not see or at least not be at close quarters with or alone with his brother. This will make your husband your ally & draw you closer together. 

In the long run tho, you will need to get over brother in law... So you'd best make a start on it ASAP. For one thing, a guy who'd play up on his wife is exhibiting pretty scummy behaviour; all the moreso if he's willing to break his brother's heart as well... To say nothing of the mess both your children & your nieces & nephews would have to live thru... Then there's the mess to your in-laws... So you can start by picturing all these people crying & being devastated every time you start thinking warmly of him.. You can have a go at getting chummy with his wife (& she'll make an ideal chaperone too) b/c its harder to think that way about a fella when you really like & respect his missus.. If you're feeling brave, you can even let her in on the plan to avoid him; she'll help even more effectively than your husband could (after all; she lives w/ him & knows his ways). If you don't know her very well, your husband might be able to advise you on how to approach her (he may have known her longer). Make yourself think about some of b-i-l's less-pleasant habits (all men have them) & focus on them so that (in your head) he becomes a walking burping/farting/stink-out-the-loo/mess-in-the-kitchen/whatever thing. Really its no different to avoiding the bakery or the lolly-aisle when you're on a diet. You can do it.... Ok; now take a deep breath & go ask your beloved hubby for some help w/ this.

My fiance's ex girlfriend slept with his brother and it broke up an entire family you cant just think about what it would do to your partner you need to think about everyone invoved if you really love your partner you wouldnt even be worried about something happening

I don't mean to be a party pooper but if you knew you had unfinished business and unresolved feelings, why did you get married? You should have sorted them out before bringing a family and another person in to the scene. Don't ruin another persons family, your own family and the relationship that exists between two brothers. You're a married woman now and are headed down the path of destruction if you put your own needs before the people you care about. If you can't hide feelings then you need to either sort out whether you still want to be with your husband and ruin a lot of lives or be the stronger woman and stop wondering "What if" and move on. Move away if you have to. Don't play roulette with other people's lives. If you still love your husband I would sit down with him and work out what you need to do in order to make things right again and move away. No emails, no phone calls, no letters, no contact.  You've already said that it has been fine when you don't see each other so don't see him then! Talk to your partner and see if you can move elsewhere. Play with fire and you'll end up burnt.

Hi,

If you really care about your husband's feeling and you do love him, and care about your family, just tell the other man that you don't love him any more, the love you have to your husband and your family will strength you, and will help you to face it and end that relation, as you said he's your brother in law so i think you will see him every now and then, see him and be normal, don't give him attention, ignore him and let the past die.

i really wish you the best, but remember that your family and husband love you, so plz don't let them down ...