I figured it out with some help on StackOverflow; I'll post the answer here as well:
Thanks to user3743222 I figured it out. It turns out (logically actually) that you can't subscribe to a plain observable. It needs to be constructed in a way such that it has values to observe. I changed the initialization for otherObservable in my unit tests from new Rx.Observable() to Rx.Observable.just(1), and now it can be subscribed to. I'm a little confused though why it didn't throw the same error when I passed both an onNext and onError handler. But I'll take it.