Extendable way to solve data persistence management problem

Our Lohika team decided to take a risk and invest in one of our current Lab projects by using Google’s Alpha version 1.0 architecture components for Android. Our team is always trying to be ahead of the curve! Here are the release notes for those who are interested: https://developer.android.com/jetpack/docs/release-notes

While the architecture components are a toolset, we mostly leveraged two things: Room to handle everything related to DB and LiveData to expose simple lifecycle-aware observables to the UI layer.

General problem
Data persistence is a key in Android applications. Nobody likes it when after loading data in your favorite application and jumping back and forth around a couple of other applications you see the data loading in your favorite application again. Things can get worse if you open this application while the device is in offline mode – airplane mode or just temporary connectivity problems. At the same time, data persistence should be applied very carefully, because if we’ve persisted some information in the local storage, it does not guarantee that this information is still relevant. The process of persisting data should be implemented in as managed and controlled way as possible. This illustrates a common problem for a huge variety of applications – like social, messaging, financial or e-commerce and not limited to – data persistence management.

Proposed and implemented solution
Here is an example of a “Single Source of Truth” pattern proposed by Google in their guide for Architecture Components visual explanation:

image1

Figure 1

MVVM (Model-View-ViewModel) in this example (Figure 1) is implemented by exposing LiveData models to UI layer thru Repository, and Repository applies Single Source of Truth logic illustrated on Figure 2 for Room DB (Disk) and Retrofit API (Fetch data) objects.

image1

Figure 2

Note, that in Figure 1 both Room DB and Retrofit API objects expose LiveData observables to the Repository and then data flows up to the UI layer.

By default, Google proposes to use LiveData streams for all layers – model (usually represented with the repository containing API and DB), domain (ViewModel) and UI (views, activities and fragments). While LiveData is a form of observable data model with really simple API – meaning there is only one method to implement in LiveData observer – onChanged(), it has nothing to do with data manipulation in complex real-world tasks, when you need to access several APIs, filter the results, merge these results in order to produce output that really has business value. Wide known tool for that in today’s Android development in RxJava. RxJava has a great community, with a lot of tools, and it is also a known topic – how to cover RxJava code with unit tests. So we’ve decided to come up with a combined solution using LiveData for UI layer and RxJava for the model layer. Figure 3 illustrates how we’ve managed to leverage from RxJava while applying MVVM with Single Source of Truth.

image1

Figure 3

The main idea of the implemented solution is to use the power of RxJava to manipulate data and use the power of the LiveData to get rid of lifecycle management in ViewModel and UI layers. This is solved by applying LiveDataReactiveStreams library also from Google – https://developer.android.com/reference/android/arch/lifecycle/LiveDataReactiveStreams

Solution Flexibility and Extendability
Figure 2 describes the general approach to dispatching data to the ViewModel and UI layer. Real applications often require various approaches to obtain data and have different requirements of when to display the data on the screen. We introduced a few variants to provide cached data to the UI as quickly as possible dispatching it from persistence storage, before performing fetch from the network. This allows users to use the application while up-to-date information is received from the server. Figure 4 shows the flow.

image1

Figure 4

Figure 5 describes another frequently used pattern to provide cached data to the UI layer without proceeding to backend API calls.

image1

Figure 5

For example when the application needs to update some data in the background and there is no need to populate this data into the UI layer, for instance, if the application uses some kind of huge model which consists of numerous entries, and data update is triggered with Push notification, or some specific action or condition, it would be helpful to use “fetch from server and store to cache without dispatching the data to observers” pattern which is illustrated in Figure 6.

image1

Figure 6

This solution allows developers to implement Single Source of Truth pattern for Repository in any manner which would fit requirements, for example, we may want to check if data in cache is still relevant (due to time limits, or any other conditions) before returning it to UI layer, or perform some post validation and processing of received data from server(or any other source), etc.

Lessons learned
As usual when something is in “Alpha X” version, architecture components implementation change significantly during the project implementation and we encountered significant issues. Here is the thing – in alpha version, LiveDataReactiveStreams#fromPublisher(Publisher publisher) method created a new LiveData object and subscribed to the received publisher. But the behavior of the method changed for release 1.0 Architecture Components version – compared to “alpha 1.0.8” – now LiveDataReactiveStreams introduces inner class PublisherLiveData which is re-subscribing to the RxJava data-source after each transition from Inactive to the Active state of the corresponding lifecycle owner (Activity or Fragment). Android activity lifecycle just for reference in Figure 7.

image1

Figure 7

And this still worked for any data that we usually get from model layer, but was totally broken when we post or put something to the model layer, because of constant re-subscriptions triggered by onInactive -> onActive transitions.
One possible workaround in this case – to have hot observables on the model layer with subjects or publishers, but this requires significant changes in the way we access data from data sources such as API or DB – since Retrofit and Room doesn’t work with publishers out-of-the-box.
We took the approach to extend LiveDataReactiveStreams library: by design LiveDataReactiveStreams.class has 2 inner classes: one converts LiveData to RxJava and another one converts RxJava back to LiveData. We have modified the latter one, it is covered in the code snippet below, so it makes resubscription behavior managed and controlled.

private static class ManagedPublisherLiveData<T> extends LiveData<T> {
    final AtomicReference<ManagedPublisherLiveData.LiveDataSubscriber> subscriber;
    private final Publisher publisher;
    private final boolean resubscribeOnActiveStateChanged;
 
    ManagedPublisherLiveData(@NonNull final Publisher publisher, boolean
            resubscribeOnActiveStateChanged) {
        this.publisher = publisher;
        this.resubscribeOnActiveStateChanged = resubscribeOnActiveStateChanged;
        this.subscriber = new AtomicReference<>();
        if (!resubscribeOnActiveStateChanged) {
            doOnActive();
        }
    }
 
    /**
     * If resubscribeOnActiveStateChanged is disabled, adds the LifecycleObserver to perform
     * unsubscription only after LifecycleOwner will get Event.ON_DESTROY lifecycle event
     * <p>
     * Calls super
     */
    @Override
    public void observe(@NonNull LifecycleOwner owner, @NonNull Observer<T> observer) {
        if (!resubscribeOnActiveStateChanged) {
            owner.getLifecycle().addObserver((GenericLifecycleObserver) (source, event) -> {
                if (event.equals(Lifecycle.Event.ON_DESTROY)) {
                    Timber.d(ManagedLiveDataReactiveStreams.class.getSimpleName() + ": " +
                            "perform doOnActive for resubscribeOnActiveStateChanged disabled");
                    doOnInactive();
                }
            });
        }
        super.observe(owner, observer);
    }
 
    /**
     * Performs new subscription if resubscribeOnActiveStateChanged is enabled
     * Does nothing otherwise
     * <p>
     * Calls super
     */
    @Override
    protected void onActive() {
        super.onActive();
        if (resubscribeOnActiveStateChanged) {
            doOnActive();
        }
    }
 
    /**
     * Performs unsubscription if resubscribeOnActiveStateChanged is enabled
     * Does nothing otherwise
     * <p>
     * Calls super
     */
    @Override
    protected void onInactive() {
        super.onInactive();
        if (resubscribeOnActiveStateChanged) {
            doOnInactive();
        }
    }

Client code can now choose whether it needs resubscription for Stopped -> Started state transition or not. This essentially solved the problem, because re-subscription is only needed for a content type data (data for lists, texts etc.) and when doing the server update – it is just one-time action and is not repeated after every UI or component change.
Also, we got some sort of modularity and a weak coupling out-of-the-box. Because we got model and UI data streams of different nature and, since they are absolutely DIY APIs, mixing them in a coupled code would bring a lot of problems. In our case, we got full separation between model and UI layers – no Rx* imports in UI layer and no LiveData* imports in the model layer.

Summary
At the end of the day, we can conclude that our risky decision to use the alpha version was worth it – the described approach allowed us to solve complex data persistence management problems in a very extendable way. We saved a lot of time and effort while we handled all data manipulation with RxJava. So the model layer was efficient, easy to onboard for newcomers and very testable. LiveData in its turn simplified UI layer to a great extent – as a consequence of its simple API and automatic lifecycle management. Covering it with unit tests is also quite easy.
So, if you are considering using Android Architecture Components in your projects, now you have the idea of how to achieve flexible integration with the full power of RxJava – you can get really extendable, testable and easy to maintain applications.