Is it time to let go when his family despises me?

I was seeing a man a year younger than me. Some time back, his mother and sister came to my home and told my parents they didn't approve of me and that I should keep away from their son. My family is very strict, so I lost my phone for about nine months and faced harsh punishments. During that period, I gradually moved on. Months later, his male cousins reached out saying he was miserable, crying, and had stopped his studies. I said no at first, but they kept asking, so I eventually talked to him again. He apologized a lot for his family's actions, but I never truly trusted him again. We would fight because I wanted him to concentrate on his education and future so he could marry me without relying on his parents. He lied to me about various things, and once I checked with one of his cousins about something he said, and the cousin admitted it was a lie. Over time, during arguments, he began saying my situation was just fine, that he wished I’d never appeared in his life, and other painful words. Still, I grew strongly attached to him again. When we weren’t talking, I’d occasionally ask his cousins about him or things I was afraid to ask him directly, mainly from fear he’d get his mother involved again. All those chats were strictly about him. I later found out he isn’t close with those cousins and has no clue I ever spoke to them. He’s also told me he’d never forgive me if I ever communicated with another man. Two weeks ago, I performed istikhara. That night he stayed out until 5 a.m. and sent me a photo where I spotted a piece of a woman’s clothing. He first denied knowing her, then said she was a waitress. He gave me a wrong café name, but I found the actual place myself. It was a gaming café that’s like a bar with music and mixed gatherings. I even contacted the café, and they told me their waitresses don’t wear the kind of uniform the girl in the photo had. When I confronted him, he accused me of fake crying, brought up my family problems, and said that if my dad treated me right I wouldn’t be running after him. He threatened to contact my father and tell him to “control his daughter.” He even told his mother I was chasing him and shared my tearful voice messages with her before blocking me. Two days later, he said sorry, claimed he didn’t know what got into him, and I forgave him because I have love for him. Now he says he wants to marry me and promises his family won’t hurt me, though he wants us to live with his parents. My greatest worry now is that his cousins might tell him I used to talk to them. I’ve begged them not to, and they both said it would only bring trouble for them, so they wouldn’t. But if he ever finds out, I’m sure he’d tell my parents everything. He has even told me that if I had treated him like he’s treated me, he’d never forgive me because my love for him is bigger than his for me. Should I walk away and put my trust in Allah? And do you think there’s a real chance his cousins will one day reveal our conversations?