SEO
Test your CSR with 'fetch as bingbot', 'fetch as googlebot' .
Yandex (duckduckgo) currently doesn't run javascript.
Performance
It depends. In some countries you might only get a 2G connection. Camping sites have terrible flaky wifi. Both download size and latency becomes an issue. CSR takes time to load and run JS, but once running, an CSR app can react snappier (optimistic UI).
If the requirement is: offline-first app, then CSR is the only option.
Accessibility
CSR/SSR can be made accessible (WCAG 2.0), but it takes effort. Test your pages with Apple VoiceOver and NVDA. A screen reader reads from the browsers DOM tree, so javascript doesn't have to be an obstacle.