The problem was this missing line in my index.html:
<link href="https://api.mapbox.com/mapbox-gl-js/v2.6.1/mapbox-gl.css" rel="stylesheet">
The documentation tells you nothing about it, had to find it in the source code of this example: https://github.com/visgl/react-map-gl/tree/7.0-release/examples/controls
CLICK HERE to find out more related problems solutions.