The author of TheCoffeeMachine gives a detailed walkthrough of how to add maps to a hugo site using uMap, which allows you to create maps based on OpenStreetMap (OSM) layers, and a shortcode that allows you to embed them. This is a belated TIL as I've already used this a few times, most recently for my post on the Tri-Country Train.
A rough overview of the steps involves signing up for and using uMap. This will allow you to create a map, set markers, and make the adjustments you need on top of existing OSM layer. From here using hanzei's hugo-component-osm theme you can embed the map into the hugo site using a shortcode. This includes passing in any parameters that uMap supports like coordinates to center the map and the zoom level.
I recommend reading the linked post for detailed steps. I needed to take this further and include a class within the iframe that's rendered on the page to able to style it with CSS. Unable to find a way to do this I forked and made my own addition. The pull request was created to the original repository if hanzei would like to include the changes. Here's a sample of what it looks like:
{< openstreetmap mapName="usa-train-lines-using-stadler-multiple-units_1100675" coordX="40.0731" coordY="-74.8924" scale="10" class="alignright" >}}