IMO it's a misleading premise to begin with "Start with a website, not a mobile". One should try to stay true to target audience. If your product justifies "Mobile first" approach, mobile app is what one should start with.

Few years ago, I started with a website prototype with an end goal of porting UI elements and business logic to mobile. With some exception of reusing business logic, most UI elements required redo.