Hmm...checking out the latest NVIDIA Studio Drivers, it doesn't list the 5000 Series of cards and were last updated a year ago. It's these drivers that are designed for Adobe's Products; I feel the drivers you are using, that are meant for video games / general computing, is the source of your issues. Plus Windows 11 is problem all by itself. (Microsoft is keeping with tradition of every-other-operating-system to be "Good."
Windows XP --> Windows Vista (Sucked) --> Windows 7 -- Windows 8/8.1 (Sucked) --> Windows 10 --> Windows 11 (Sucks so far...)
See a pattern?
At this point, I'd wait a few weeks. Apple is about to make M5 announcements for new products. I will say, start saving, and you will be forced to purchase the stupidly over-priced Apple Studio Display if you want the full "Macintosh Experience." Yes, Apple does this on purpose, as they changed the signal coming out of the HDMI port. With the new Studio Display, you are using the Thunderbolt Port for Video.
Anyway, I don't think we will see a M5 Mac Studio in November, likely it will be April / May of 2026 with the M5 chip. Apple tends to release MacBook Pros with the newest chips in the latter part of the given year. So if you can hold out until early next year, I would. No sense in blowing around $5000 not to get the latest-and-greatest. That said, I do see rumors that Apple might be updating their Displays, and the Studio Display might be refreshed. Hold off on buying anything until around Veteran's Day on the 11th.
As far as feedback from others, there is A LOT of Macintosh hate, and practically everyone will tell you that it's a waste of time and money. I'm not one of those people. Buy whatever you like. One of the things that Adobe takes advantage of, is that there are only a certain number of Video Cards / Hardware that Apple uses, so it's easier to make things compatible when all you have to worry about is around 7-10 different Video Cards, vs the hundreds for the Windows side.
Of course, Adobe Photoshop is Adobe Photoshop and LR is LR. As time goes on, it's becoming more and more bloated with all of this AI Crap that I'm not 100% sure you will eliminate lag entirely; I've seen Photoshop act stupid on both sides of the fence.