How I fixed it:
Start GParted in the application menu.
Then delete all partitions of the drive you were originally installing to (this drive will be wiped). Apply.
Then go to device → create a partition table, select GPT. Apply.
Now when you use CachyOS installer and select this drive, it no longer gives the error during install.
Edit: might be useful if I put this here (it was automatically generated) https://termbin.com/4u3x