The teams at Google have put in a lot of thought in regards to accessibility for Material Design and its various implementations built mostly in-house (such as Material Design Lite and Angular Material, the latter of which I worked on for almost a year).
That said, there are still color contrast issues with the main themes that could impact low vision users. I know it was reported to Google's design team internally, but the websites built with Material Design still have contrast issues (as one example). That's why it's so important to choose an accessible color palette early on–once your brand guide has shipped with certain colors, it's really difficult to change them.