Why you should not use your app state as a remote data cache

Why you should not use your app state as a data cache

The problem

As a front-end developer and what some call an "architect", I've worked in very big projects, so I spent lot of time of last years trying to achieve a fully modular system in which the different front-end teams could reuse elements across many applications .

For simple components this is a relativelly easy job, but, when the elements are connected to data origins or to a global state (what I usually call a "module") , then the thing changes, but why?...

In most cases, despite the fact of following recommended good practices and patterns (very extended thanks to great tools and libraries like React , Redux , Reselect , etc.) the elements still were not 100% reusable. Some parts of its logic remained "coupled" to an upper level, which made almost impossible that ones could work without the others in a new and completely isolated environment.

The causes

I realized that usually it was due to the fact that these elements were delegating part of their responsibilities to another one without even realizing it. The other element normally is at charge of making a preliminary initialization, recovering certain data and setting it into the "state" (a very usual example of this is the case where the user data are retrieved during the login phase, then saved into the state, and then accessed afterwards by a lot of elements which are presupposing that the data are always there, ready to be used)

Most of times this is made simply because we want to optimize, save resources avoiding multiple calls to the server and unnecessary extra computations, but, following this pattern, we are making our elements completely dependant . Those elements can not be instantiated without the other ones, and they probably will require an specific load order to work properly. Which is worse, the first element is probably preparing or formatting the data in an specific way to which the rest of elements can be highly coupled. These "hidden" dependencies can easily propagate by the entire system without having notice, and, if you are not very careful, all the system becomes an indivisible and "monolithic" great piece of software. You could end working in an scenario in which hundred of invisible dependencies make almost impossible to remove, modify, or replace some elements (guess which ones) without tons of pain, that is the time when you can rename your project to "Domino blocks play".

At the heart of the matter, the problem is to use the global state as a cache for the data , and, as it is "global", an orchestrator is needed to be at charge about when to retrieve the data, when to invalidate caches and retrieve the data again, etc. If there is no a clear "orchestrator", we usually delegate this responsibilities to the element that seems to be the responsible one (as in the login example) , but, doing this, we are making dependant all the other elements needing some portion of the same data.

The solution

So, maybe the solution can be to make every single element responsible of requesting always the data it needs (connecting always them to the providers they want to read) , and doing it in the most granular way possible , requesting only the data they want, and in the format they expect (using specific selectors) . Then, those selectors could be reused across many elements requiring the same data.

This solution simply is at charge of providing cache and memoization in order to avoid unnecesary resources comsumption , and to abstract the elements about the fact of from where are they reading the data. They don't need to know about the existance of an "API", or a "State", or "localStorage", or whatever. They simply need the data, it is the "data layer" which should be the one responsible of knowing about where the data are being retrieved or sended.

In this way, each element has well defined dependencies , and you can move them from one project to another (or from one part of your project to another) without problem. The data will be requested and processed only when necessary, and only once (until the data decides that one cache has to be cleaned, then all of the elements connected to that data are informed about, so they can request it again).

Concepts

As a summary, main targets of this project are:

• Separate global state from data cache.
• Force elements to always request for the data they need (but avoiding a negative performance impact due to usage of an internal cache).
• Make dependencies with the data clearly identifiable and traceable.
• Provide selectors allowing to combine data from different data origins or other selectors, keeping the same interface and principles.
• Inform elements when the cache of a provider is invalidated, so impacted elements can request the data again.
• Unify the interfaces of different data origins, in order to isolate the elements about the knowledge from where the data are being retrieved.
• Provide simple methods to handle loading and error states.

I hope that, if this library does not result useful, at least these principles do, because this is the really important part of the project, more than the code itself. At the end, all patterns described here can be implemented using combinations of other tools. This project simply tries to facilitate the process.