We are in a very similar position.. I am also on a start up and we have 1-2 people actively using SolidWorks with another 2-3 non CAD people occasionally accessing the files. We also opted for implementing a PDM solution right from day one.
We use a solution called DataDesignManager or DDM, by CSI Ltd. I think is a great solution packed with all the major features that you would like from a PDM, at a very affordable price (depending on number of seats and license type, but we are talking hundreds/low thousands instead of high/tens of thousands). Their maintenance cost is also the cheapest I have ever seen. The solution is CAD independent and can support many platforms.
I believe this system will cover all the reasons that you listed on your post, plus more. At the moment, we are using it for version control, formal release of drawings, and controlling our CAD workflows. We use it only for CAD files, as we use a different system for our document management requirements, but you can also use it for Office files and other types, giving you effectively a document management system, if you like. It offers remote access and multi-site hosting, if you need that. I would encourage you to check it out.
Be warned of the following… a couple of years ago, Dassault Systems decided to change their SolidWorks Partner Program in line with their strategy to grow their own 3DEXPERIENCE platform, basically limiting the companies on it. DDM was affected by this, meaning that after 2027, they would not be able to support SolidWorks. There has been a large number of discussions and post about this, some on this forum if you are interested (need to find the link to it). Many people say that this has been a push from SW to force everyone to their own cloud based system (3DExperience). We had a look at it, and it is very basic and did not suit our purpose. This, plus some not very flattering comments about it on this forum, made us not go down that route.
We decided to stick with DDM due to the functionality, service, and value that they offered to us. In addition, they serve some very large companies with thousands of SW seats, making a solution to this more likely (i.e. it is not a small joe.blokes company with little leverage). At worst, we would need to stick with SW2026 for a couple of years before moving to another PDM if nothing is resolved.Nevertheless, it was a gamble. Last I heard, they are very much in line to be reinstated as development partners, but there has not been an official notification about this, so please do your own research about this.