Intruding on our family...

I recently attended a custody trial for my 2 year old son. My girlfriend and I have been together since I was 5 months pregnant. The father was NEVER interested and at one point almost signed his rights over to avoid ever having to pay child support. Once my girlfriend and I moved out of the state to live near her family, the father filed for custody and visitation. His main request was for the child and I to return to his state. We just went to trial and his testimony was horrible. In my opinion, my atty tore him apart on the stand. However, the judge seemed to be unaware of the father's shortcomings and almost seemed to favor him. We are awaiting the ruling still. My question is, do you believe this "father" has the right to dictate where we live and come in now, after 2 years of absence and ambivalence, at the behest of his mother and my mother (whom I do not have a relationship with because of her utter disgust at my being gay)??? Thoughts??? Thnx;)