Hey everyone, need some C# design help.

I'm refactoring our current design for how we download static data. It's a mess of deep class hierarchy and callback hell and I want to convert it to a more elegant straightforward design that resembles Linq/Reactive programming.

Here are two examples of the steps we take to download static data (JSON files):

Example one, downloading main manifest:

• Attempt to load manifest file from local
  • If fail, download from the internet
    • If fail to download, bail with error
  • If success, deserialize manifest file
    • For each file in the manifest, attempt to load from local
      • If success, deserialize the file into the appropriate class
      • If fail, download from the internet
        • If download succeeds, return string data
        • If download fails, bail with error
    • When all files are deserialized, get the resulting list, and store it in memory

Example two, download a specific file:

• Ask manifest manager to get a specific version of the file
  • If manifest manager has it store locally, return
  • If manifest manager doesn't have it stored locally, attempt to download it
    • If download succeeds, deserialize and return
    • If download fails, bail with error

Currently, every step is a method in... some class in the hierarchy, and subclasses override some steps to accommodate some special casing (e.g. don't use manifest manager, don't attempt to donwload, etc).

Another thing to keep in mind is that, almost every step uses callbacks due to the nature of downloading stuff from the internet. And so, we ended up having to even override the callbacks in... some classes in the hierarchy, which resulted in a complete, unreadable mess.

My first attempt to refactor this was to use either the Decorator or Chain of Responsibility design pattern. The thing is, some steps require CoR and some don't. Moreover, some (if not all) steps use callbacks, which doesn't feel like the patterns where design to use. And finally, I have no idea how I would translate "take the downloaded manifest, and yield each file separately, and when everything ends, continue the chain". This last point actually made me think about Reactive Programming.

        var callbackReader = new CallbackGameDataReader<ManifestVO<ManifestFileVO>>();
        var manifestDeserializer = new DeserializeGameDataReader<ManifestVO<ManifestFileVO>>(callbackReader);
        var wwwReader = new WwwGameDataReader(remoteEnvironmentPath, manifestDeserializer);
        var streamingReader = new StreamingAssetsGameDataReader(streamingAssetsManifestPath, manifestDeserializer, wwwReader);
        streamingReader.Read(dataId, o =>
        {
            ManifestVO<ManifestFileVO> localManifest = (ManifestVO<ManifestFileVO>)o;
            localParsedData = localManifest.files;
            localKeyedData = CreateKeyedData(localParsedData);
            parsedData = localManifest.files;
            keyedData = CreateKeyedData();
            ContinueInit();
        });

This is the closest I got with a mix of Decorator and CoR classes, but it does not include the last part where we split and download each file individually.

My goal is to end up with something like:

List<ManifestFile> dataFiles = new DataReader()
    .ReadLocal() // How do we fallback to WWW?
    .Deserialize<Manifest>()
    .Split(manifest => manifest.files)
        .ReadLocal()
        .Deserialize<ManifestFile>()
        .Merge() // ???
    .ToList();

I'd love to hear your thoughts. And recommended material to read would be great.

Thanks!

(P.S. I'd rather avoid using an existing library because the project is in Unity and I'm not 100% if it will be supported due to its use of a modified Mono implementation. Plus, I'd like to learn how to do this on my own ;) )