I've been working on this myself! I came to the PS2 pretty late as a kid so I only really got into the Big Hits back then. Here's what I've really liked so far:

Drakengard and
God Hand really don't need any introduction as some of the best known "cult classics" on the system. But they're both must-plays to me. Pure artistry, uncompromising vision.

I will never be able to shut up about
SkyGunner. I've been thinking lately about how there's no Star Fox game that really hits the kind of "cartoon dogfighting" highs of Star Fox 64 but SkyGunner actually does it. It's like a playable banger one-off OVA. Ghibli sensibilities but with an indie soul. And there's a little bit of that Armored Core resource management mixed in too. Every missile and bullet costs money and you're fighting not just to win the battle but make the most dough. There's definitely a learning period in grappling with the controls and the game's internal logic, but once it clicks it feels amazing.
Sakura Wars 5 is exactly the kind of goofy dumb haremslop that hits for me. I love an ensemble cast of insane girlies teaming up to take the piss out of a short king! I love when there's a new monster every week! There's nothing groundbreaking about it but it's a big-time comfort game to me. They ported this to the Wii too, but you can only play it with the Japanese dub (and lip sync (and original character names)) on the PS2.
Flipnic is completely bonkers. If you like pinball, you need to play this. And even if you don't you might still want to? I really don't want to say too much about this because I feel like it's better to come into it without expectations. It's one of the funniest games I've ever played.
Ar Tonelico II I haven't played myself, but it's my girlfriend's absolute #1 favorite game. It shipped with a really shoddy translation, then got
a better fan translation, and now there's supposedly an
even better fan translation in the works.
...which is why I haven't played it yet lol. Anyway those girls in the back seem really close, don't they? Interesting.

The PS2 got a lot of Densha de Go games that range from good to great, but
Densha de Go: Ryojou-hen is my favorite of them all, and my favorite in the series at large. This entry shifts the focus from driving commuter trains to instead driving streetcars and light rail, moving at lower speeds and contending with pedestrian and car traffic. But at the same time it also places a great emphasis on the aesthetic experience of its locales. The four lines correspond to the four seasons--Matsuyama in spring, Enoshima in summer, Kyoto in autumn, and Hakodate in winter--and you can pick what time of day you'd like to drive between morning, noon, evening, and night. It's tranquil and meditative, but with enough plates to keep spinning to never be boring.
And the music is just sublime. Someday if I have more money I want to splurge for this game's special controller, but it plays just fine on a gamepad too.