he fears and i dont know what to do... :(

well i have been talking with this guy about 2 years by now, we met online when we were living different countries. I wasnt in love with him when we were talking but I moved to his country because of my education. Additionally the conversation was awesome and we were talking a lot..We met once when I moved his country and we had a really great time and enjoyed so much. The thing is he had a long distance relationship before (he's being in uk and she's being in south africa), and he said it lasted 6 years, and he said when they were about get marry he changed his mind. He said he didnt see her much and he was young. Now we are living 1 hour away but there was always something keeping him not seeing me, but he always kept in touch with me and even we have opened our feelings to each other at last.

One day, I dont know how it happened, but we started to talk about sex and decided to have it but he was about to go on a holiday. He went and turned back, I texted him while he was on holiday but he didnt reply and I felt so bad. But after he turned back he sent me an email and said sorry and will speak soon. I was still angry and didnt reply 3 days and then send him a short reply saying 'as you like'. He got angry this time and said forget it. But I didnt stop and told him to speak to me whatever the reason is. He said ok and we talked the day after and he said he changed his mind because I live far away and someday may turn back to my country and he is sensitive and cannot just have sex without emotions and if I move away he would be sad. He is the kind of guy that I am so much attracted to and always wanted to be with but because of his fears we couldnt move forward.

Now I feel terrible and even cried so much which I wasnt expected, we are still friends and we can even meet again but I dont know if I should fight for it or forget about him totally. I cannot find the power in myself to stop the friendship now, we still like each other, I am totally stuck and dont know what to do.. Please advice :(