He moved out,space needed,we have a kid? help?

We have been together 5 years, have a 4 year old ---, we never had a real relationship because we had a kid first. we didnt have that dating period.we moved in together right away and it was always work baby work. we never really did things with EACH other but as a family. He worked, i worked,he always made more than me. He did support me a big whole lot. He cheated me on me 2 years later and came back,ofcourse i took him back for the sake of our family to be together. things got better, we moved out of state together to go do what he wanted in life and i followed cus i supported him, didnt work out we moved back, less than a month later with all the money has he told me he was moving out, because-Im supposedly childish,immature, and we have a communicatin problem and i neeed to be alone more and need to grow up and if this other girl would take him back he'd go who lives an hour away.alot of stuff happened in the next week, he apologized begged he wanted to fix it,swore he loved and miss me and our daughter. said that we were what he wants.but he wasnt going to fake anything since he personally didnt know when he lost feelings for me. since he got his new place within the last month, weve gone to movies,eat dinner and everything with the baby and tells me i can stay over, when i try to conversate with him, he does but not alot to keep it going its always simple.i asked him if he hated me(due to some actions) he said no he loved me.but he gets angry easily at me and i dont yell back.but sometimes he does turn the table to make it my fault in something.Now he tells me he needs space and time for himself,he thinks im being immature because i get mad easily because "i dont get what i want" but its not that, im trying to "save" our family, am i trying to much by calling/trying to talk to him/ask him out ALONE without our daughter to have what we didnt have in our relationship? i told him if he's not going to try than he needs to tell me, but he hasnt said anything like that he doesnt want to or he doesnt want anything to do with me, he just says space? i know space can be bad but what should i do? we have a kid, how can i ignore it. Im not immature and i dk what he sees as immature, maybe my job? ways of doing things? but when he first broke up he didnt say he wanted space until we got into a huge fight and i kicked him out of my place and he had nowhere to go...ahh im lost.