How do I get around the verified bug in Windows 1903 and launch the VSCode integrated terminal?

I just did a fresh install of Windows 10 Pro version 1903 build 18362.116 and Visual Studio Code. Now the integrated terminal only launches externally.

Pressing Ctrl + ~ results in this.

What am I missing? How do I get it to open integrated again?

After working with VSCode team it is a verified bug. See the Github issue here. I posted the workaround as an answer here.

好，在VSCode repo问题中解决了这一问题。

现在，在修复之前，请在用户设置中关闭ConPTY集成。