There are quite a few Material design UI frameworks for ReactJS. I would suggest you to use react-mdl since you're looking for something lightweight.

Material UI is also an option, it is feature rich and has the most refined implementation of Material Design but comes with some extra weight for all that goodness.

You can take a look at this story for some more options.

I've used both material-ui react and material angular. I also faced the speed issue with material-ui react when I first used it. What you'd be doing wrong is not running webpack in production environment. Development version is usually slow and also shows warnings on console..

Add this line to your webpack.config.js

plugins: [
  new webpack.DefinePlugin({
    'process.env': {
      'NODE_ENV': JSON.stringify('production')
    }
  })
],

Also read about how to optimize your react app here.. moduscreate.com/optimizing-react-es6-webpack-prod…