Oprindeligt skrevet af java.lang.SkillException:

- I have a feeling the problem has something to do with the reference mesh being concave, -

That's correct. StudioMDL's triangulation can sometimes mess up with concave polygons, so you should cut the concave polygon into multiple convex polygons instead.

For example, picture that your polygon's edge is a road, and place an imaginary car on it.
Can the car complete an entire lap around the polygon while only turning clockwise (or only counter-clockwise)? If so, it's convex, which is good.
But if it has to turn the other way at some point, the polygon is concave, and you have to split it (at the point where the car turns the second way) to try making multiple convex polygons instead.

In other words, the outside edge must always curve "inward", never "outward". If it curves "outward", you have to cut it where it starts curving out (even if this is for every vertex along a curve).


TL;DR: In this example image (with a cherry pin instead of a door), you'd have to cut at the blue lines in https://imgur.com/LBcHTXE ...or alternatively just use Blender's "Triangulate Faces" option to turn the polygon entirely into triangles.