How do I use an alert in a modal and make it work with keyboard?

[mastoj]

I'm using a sheet to show some data, in that data there is an action that should have an alert connected to it. To get the alert to show at all I removed <AlertDialogPortal> from the AlertDialogContent, my main problem now is that when I hit esc both the alert and sheet is closing and I'm also stuck in some weird state where I can't interact with the site without a refresh.