Here's a bunch of examples showing how to embed OpenStreetMap on your website, and various other tricks with markers etc layered on top of the basemap. Typically this is done in javascript using web mapping libraries with simple techniques which can work on any website. For an overiew of how these and other technologies work in and around OpenStreetMap see my OpenTech presentation
Note that some of the examples are a bit old and broken. I need to re-test them and tidy them up a bit, and describe what they mean