It turns out there is a way and since it surprised even the AbapGit authors, it is not as well-known as it should be. The only place I could find it mentioned is this stackoverflow answer with 5 upvotes, the standard for SAP-related questions.
It is actually very simple to do:
Have the transaction you want to debug running
In the folder C:\Windows\SysWOW64\F12 (or windows\system32 for 32bit systems), you will find a utility called IEChooser.exe
Open it and select the window you want to attach to
4. You're done.
Now you can browse the page contents:
I hope this will save you some work. Please share your experiences or problems in the comments. And most importantly, if you know any secret tricks yourself, let others know!