I've seen this asked about as I was looking for a solution myself, while the steam overlay is on the game seems to lose all visuals when using Alt+Enter to make it windowed. I simply tried disabling the steam overlay while testing a bunch of things to get it working and it seems to enter windowed mode totally fine. I don't know why it does that but it just works for me, and I hope it works for you too.

Edit: You can easily disable it for specifically this game by right clicking "The Yawhg" in your library, going to properties and clicking "Disable steam overlay on this game" or something similar to that.