Why does my graphics driver break?

I suppose it is up to you.

You technically can “add” Plasma .. but there are more than a few reasons why its not generally recommended to mix multiple DEs (a DE and a WM or 2 much more acceptable).

Of course there is whatever data and/or your personal time to consider as well.

OH. And another thing. You might be defaulting to X11 on XFCE. While Plasma and Gnome and others would be defaulting to Wayland. Likewise the live environment, while Plasma, also defaults to X11 last I checked. So this might make difference, especially given the extremely old nature of the GPU.

So aside from Plymouth (link above) you might want to make sure to have plasma-x11-session installed so that can be selected at the login page (SDDM) if needed. Or just keep it in mind.

And in case it becomes useful here is a guide that can help drop to TTY, boot without GFX, use chroot, etc;