Understanding the Etiquette of Lowering the Gaze & Interacting with Non-Mahrams

Assalamu alaikum wa rahmatullahi wa barakatuh I'm a Muslim brother in my early twenties. For quite some time now, I've been really trying to practice lowering my gaze from non-mahram women properly. I'm always mindful of Allah's command in the Quran: “And tell the believing men to lower their gaze and guard their private parts. That is purer for them. Indeed, Allah is Acquainted with what they do.” (Surah An-Nur, 24:30) And the Prophet's ﷺ advice to Ali (RA) where he said: “Don't let one glance follow another - the first (unintentional) glance is permissible for you, but not the second.” (Jami` at-Tirmidhi 2777) So yeah, I make sure not to look at non-mahrams unnecessarily. Even when I have to talk to them for work or studies, I keep eye contact to the minimum so I don't come across as rude or distracted. I've even started going to the gym super early to avoid mixed environments, and I stay away from gatherings where people might not be dressed modestly - just trying to protect myself from what's called the “zina of the eyes.” *I really want to get this right - not going to extremes, but not being careless either.* A few things I'm wondering about: 1) When exactly is it okay to look at a non-mahram? 2) Can we look at a non-mahram sister who's wearing proper hijab, if there's no attraction or desire? 3) Is looking considered wrong only if there's desire, or is repeatedly looking without desire also not allowed? 4) Some people say it's fine as long as your intentions are pure. Is that correct, or is it too simplified? Does permissibility depend only on intention, or are there clear limits in the Quran and Sunnah regardless of intention? 5) What about online stuff? Like on social media where non-mahram women might appear without proper hijab - is looking at that content wrong even if there's no desire? I'm not asking how to control myself or anything - I'm genuinely trying to understand what's actually allowed and what's not when it comes to interacting with non-mahrams, whether in real life or online, based on proper Islamic sources. Would really appreciate some guidance from brothers or sisters who have knowledge in fiqh or reliable scholarly references. Jazakum Allahu khayran.