Grégory Lureau

An audio visualization with Jetpack Compose

2021-09-03T00:00:00+02:00

In this article, you’ll learn some bits about Jetpack Compose, Android audio visualizer and how to draw a nice animation based on the audio input. You can download the repository here to check the final result.

As Compose is not stable yet, be aware that stuff could evolve. (I based the article on this project, check the exact versions there.)

For the audio data, we’ll use the android audiofx Visualizer as it provides directly FFT and wave form. No worry, we don’t need to understand how it’s computed, but just as a quick introduction:

FFT shows the frequencies used, and it’s generally a lot of basses and way less high frequencies, very unbalanced for a shiny animation
Wave form is the shape of its graph as a function of time, independent of its time and mag… well it’s a signal way more balanced and dynamic, perfect for our animation.

To use Visualizer, we need 2 permissions:

<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

And we’ll also need to request the runtime permission for RECORD_AUDIO:

if (ContextCompat.checkSelfPermission(activity, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
    ActivityCompat.requestPermissions(activity, arrayOf(Manifest.permission.RECORD_AUDIO), 42)
}

(42 is the request id that you’ll need to implement properly the permission, not the topic of this article so let’s hack this)

Now let’s implement a basic code to play a mp3 file from the asset folder

private var player: MediaPlayer? = null
private fun play() {
    val afd = assets.openFd("mymusic.mp3")
    player = MediaPlayer().apply {
        setDataSource(afd.fileDescriptor, afd.startOffset, afd.length)
        prepare()
        start()
    }
}

private fun stop() {
    player?.stop()
    player?.release()
    player = null
}

And our first JetpackCompose button in the UI to play/pause the player. For now, let’s put that directly in the MainActivity, inside the default setContent {} (created automatically when creating a new Compose project with an empty Compose activity). If you’re not clear on the Jetpack Compose usage, it could be the right time to pause the watch some intro first.

    setContent {
        val (isPlaying, setPlaying) = remember { mutableStateOf(false) }
        EqualizerTheme {
            // A surface container using the 'background' color from the theme
            Surface(color = MaterialTheme.colors.background) {
                Button(onClick = { setPlaying(!isPlaying) }) { 
                    Text(if (isPlaying) "pause" else "stop")
                }
            }
            if (isPlaying) {
                play()
            } else {
                stop()
            }
        }

We’re good to instantiate the Visualizer now.

visualizer = Visualizer(audioSessionId).apply {
    enabled = false // All configuration have to be done in a disabled state
    captureSize = Visualizer.getCaptureSizeRange()[0] // Minimum sampling
    setDataCaptureListener(
        object : Visualizer.OnDataCaptureListener {
            override fun onFftDataCapture(visualizer: Visualizer, fft: ByteArray, samplingRate: Int) {
            }
            override fun onWaveFormDataCapture(visualizer: Visualizer, waveform: ByteArray, samplingRate: Int) {
                process(waveform)
            }
        },
        Visualizer.getMaxCaptureRate(), true, true)
    enabled = true // Configuration is done, can enable now...
}

The audioSessionId comes from the player.audioSessionId, it allows to select a specific audio stream, so that you only get the data from your stream and avoid notification sound impact. Also you could use the value ‘0’ to get the result of all mixed audio streams.
The enabled=false is a safe measure if you want to change dynamically capture settings, as all settings operation have to be done in a disabled state (or else it throws an exception).
The capture size is specific to the hardware (ex: on my current device the range is [128-1024]), here we don’t need a lot of data points to have a funky animation yet, so we start low to reduce memory footprint.
Eventually the callback gives us 2 streams, and we’ll only take care of waveform as previously mentionned.

Since the capture size is specific to the device (using a value out of the range will crash), you may want to change the capture size to match your animation sampling size. As we’re starting with a basic equalizer composed of 32 columns, we only need 32 data point, so we can re-sample the Visualizer output :

val resolution = 32
val processed = IntArray(resolution)
val captureSize = Visualizer.getCaptureSizeRange()[0] // Same value than in the Visualizer setup
val groupSize = captureSize / resolution
for (i in 0 until resolution) {
    processed[i] = data.map { abs(it.toInt()) }
        .subList(i * groupSize, min((i + 1) * groupSize, data.size))
        .average().toInt()
}
// processed has the re-sampled data now.

This re-sampling is super basic, but all allowed captures values on the Visualizer should be a power of 2, so if we also take a power of 2 for the resolution we’re fine.

On the github project, I use my own data class named VisualizerData to store the processing result as it will be handy later, but you can consider a simple IntArray for now.

Now let’s draw the equalizer!

Requirements:

takes all the space available
draw a row of 32 bars with a little padding between each bar
animate the height of bars so the animation is smooth

    @Composable
    fun BarEqualizer(
        modifier: Modifier,
        visualizationData: VisualizerData,
    ) {
        var size by remember { mutableStateOf(IntSize.Zero) }
        Row(modifier.onSizeChanged { size = it }) {
            val widthDp = size.getWidthDp()
            val heightDp = size.getHeightDp()
            val padding = 1.dp
            val barWidthDp = widthDp / visualizationData.resolution

            visualizationData.bytes.forEachIndexed { index, data ->
                val height by animateDpAsState(targetValue = heightDp * data / 128f)
                Box(
                    Modifier
                        .width(barWidthDp)
                        .height(height)
                        .padding(start = if (index == 0) 0.dp else padding)
                        .background(MaterialTheme.colors.primaryVariant)
                        .align(Alignment.Bottom)
                )
            }
        }
    }

    @Composable
    fun IntSize.getWidthDp(): Dp = LocalDensity.current.run { width.toDp() }

    @Composable
    fun IntSize.getHeightDp(): Dp = LocalDensity.current.run { height.toDp() }

Yes maths are a bit wrong, the 1st bar is 1dp too large, just for sake of simplicity. But the intersting part is actually the animation, that is a oneliner without any specific values here. That’s right, animateDbAsState takes the new wanted height and will smooth the transition to the new desired height, so that we don’t need to care too much. Indeed you could want to use the full animation API to setup your own AnimationSpec and reduce the sampling frequency.

Let’s plug into into our MainActivity: we need to define another state based on the data, and pass it to the new BarEqualizer

// MainActivity

    setContent {
    val visualizerData = remember { mutableStateOf(VisualizerData()) }
    val (isPlaying, setPlaying) = remember { mutableStateOf(false) }
    EqualizerTheme {
        // A surface container using the 'background' color from the theme
        Surface(color = MaterialTheme.colors.background) {
            Content(isPlaying, setPlaying, visualizerData)
        }
        [...]

// Content

@Composable
fun Content(
    isPlaying: Boolean,
    setPlaying: (Boolean) -> Unit,
    visualizerData: MutableState<VisualizerData>
) {
    Column {
        Button(onClick = {
            setPlaying(!isPlaying)
        }) {
            Text(if (isPlaying) "stop" else "play")
        }
        BarEqualizer(
            Modifier
                .fillMaxWidth()
                .height(100.dp)
                .background(Color(0x40000000)),
            visualizerData.value
        )
    }
}

That’s it folks! Now you can add some fancy colors and add more maths to make it pretty. Let’s have some fun

Bonus : circular equalizer + cubic bezier curves

Ktlint: a great 1st experience

2020-05-26T00:00:00+02:00

This is some feedback with the first ktlint custom rules I wrote the past week. In this migration I had to deal with Json parsing issues related to minifications and bad Moshi setup, and I fixed them once and for all with some rules. Here is my story.

This month we’ve decided to re-evaluate the usage of the Kotshi library. When we picked this library several months ago, Moshi was only good for java classes and wasn’t handling kotlin nullability, default value, init blocks and more. But Moshi evolved a lot (v1.9.2 as I wrote), and provides now a support for Kotlin, with 2 options: reflection (moshi-kotlin) or code generation (moshi-kotlin-codegen). Eventually we opted-in for the codegen version (I don’t want to debate on this choice here), for 2 main reasons: the “if it build if runs” (ala Dagger) and the minimal setup requirement (adapters auto-detected). Unfortunately the 1st reason was not true.

Codegen is not always safe

Moshi, where is my @JsonClass?

Here is the example from the Moshi documentation :

@JsonClass(generateAdapter = true)
data class BlackjackHand(
    val hidden_card: Card,
    val visible_cards: List<Card>
)

This looks fine right? Let me implement Card as naively as possible

data class Card(rank: Int, suit: Int)

I purposefully forgot the @JsonClass, as it’s something easy to forgot. Moshi-kotlin-codegen builds an adapter for the BlackjackHand that asks to Moshi an adapter for Card. But it doesn’t check if Card is a kotlin class and if an adapter has also been generated.

Unfortunately this is crashing at runtime because Moshi consider (righfully) you should provide an adapter for Card.

Indeed the fix is straighforward…

@JsonClass(generateAdapter = true)
data class Card(rank: Int, suit: Int)

but I hope you caught the error before releasing in production. Personnally I rarely use the buildType release when developing…

Moshi, where is my enum class?

Now let’s replace the suit by an enum class, translated Java->Kotlin from the Moshi documentation:

@JsonClass(generateAdapter = true)
enum class Suit {
  CLUBS, DIAMONDS, HEARTS, SPADES;
}
@JsonClass(generateAdapter = true)
data class Card(rank: Int, suit: Suit)

Nop, you can NOT use @JsonClass on an enum class or you’ll get an error like:

error: @JsonClass with 'generateAdapter = "true"' can't be applied to com.glureau.moshilint.Suit:
    code gen for enums is not supported or necessary

Ok great, it’s not necessary as it’s a built-in adapter, so I just have to define my enum like this

enum class Suit {
  CLUBS, DIAMONDS, HEARTS, SPADES;
}
@JsonClass(generateAdapter = true)
data class Card(rank: Int, suit: Suit)

Let’s run this code now…

java.lang.AssertionError: Missing field in b.a.a.b.a
    [...]
Caused by: java.lang.NoSuchFieldException: DIAMONDS
    [...]

Arg, so the generated code looks proguard-safe, and it is on generated adapter, but using enum class like this is NOT proguard-safe.

Here you can have a quick fix with @Keep (on each field or in the class).

@Keep
enum class Suit {
  CLUBS, DIAMONDS, HEARTS, SPADES;
}

EDIT: Manoel Aranda Neto suggested to use a rule to your app proguard file instead (-keepclassmembers enum * { *; }), and it works great but will keep too much stuff (probably not a problem for a standard Android app though). In the meantime I found a comment in the proguard rules from Moshi, looks like we’re supposed to use @JsonClass(generatedAdapter = false)? (waiting for confirmation…)

What’s the problem?

This article is NOT a rant against Moshi.

The problem is when you have more than 50 DTOs to maintain and other developers adding more functionnalities. Generally we only enable proguard on release build (because of the big impact on build time + the fact it’s harder to debug obfuscated code) so 95% of the development time we don’t experience this kind of issues. And sometimes the 5% are not enough to catch the errors during development.

Some options to fix this problem:

Define some best practices, like @Keep on ell your enums. Good, but you can still forget 1 class and experience crash in production.
Add tests to ensure you can parse correctly all potential json files in release build, but it’s very annoying to write, and maybe 1 test will not be written and the missing annotation released in production.
disable proguard (in a part of your project or completely), it avoids issue for sure, but it also avoids optimization pass, reduction of the apk size, obfuscation… A custom proguard rule have to be maintain when a new module is created, and since it’s not something visible when you develop, it’s just another point of failure.

Could be great if it could be checked at compile time right? Let’s imagine a moment if the configuration issue was able to block a bad PR, and this kind of issue fixed once and for all…

Lint/Ktlint

Benefits

Some of you are probably aware of this tool and using it for standard Java/Kotlin style guideline.

The benefits of static code analysis (with a linter):

it enforces good practices
can avoid a bad PR to be merged (using Danger for example)
doesn’t require compilation, so super quick to run
no impact while developing (same build time)
error is localized so errors are easy to fix (in AndroidStudio, or in Github if you use Danger)
you can even implement an auto-fix

But the real power comes from the custom rules!

Previously we were relying on our QA team to ensure all the DTOs where properly setup. Custom rules offer a way to avoid errors almost for free!

How to setup a custom rule

There is some good tutorials on the linter and how to configure them, and as I don’t have enough experience I’ll just share what I know.

Creation of a custom rules takes 4 steps:

create a java module (for example ktlint-rules)
create a class that extends RuleSetProvider
create a class that extends Rule
create a file for your RuleSetProvider to be detected automatically

// File: src/main/resources/META-INF/services/com.pinterest.ktlint.core.RuleSetProvider
com.my.package.MyRuleSetProvider

// src/main/java/com/my/package/MyRuleSetProvider.kt
class MyRuleSetProvider : RuleSetProvider {
    override fun get() = RuleSet("myrules", CustomRule())
}

// src/main/java/com/my/package/CustomRule.kt
class CustomRule : Rule("my-rules") {
    override fun visit(node: ASTNode, autoCorrect: Boolean, emit: (offset: Int, errorMessage: String, canBeAutoCorrected: Boolean) -> Unit) {
        //...
    }
}

This is so simple! I’m sad to learn that only now, after several years of Java/Kotlin…

How to write a custom rule

As a team, we’ve defined some simple rules to apply everywhere:

all dto classes should be suffixed with Dto
all dto classes should be data class or enum class
all data class dto should define a @JsonClass(generateAdapter = true)
all enum class dto should define a @Keep

and several others (about immutability or ensuring Retrofit services always use Dto classes), but let’s see how we could implement the 2 last ones for this article.

override fun visit(node: ASTNode, autoCorrect: Boolean, emit: (offset: Int, errorMessage: String, canBeAutoCorrected: Boolean) -> Unit) {
    if (node.elementType == KtStubElementTypes.CLASS) {
        val klass = node.psi as KtClass
        if (klass.name?.endsWith(DTO_SUFFIX, ignoreCase = true) == true) {
            if (klass.isData()) {
                var foundAnnotation = false
                var correctArguments = false
                klass.annotationEntries.forEach { annotation ->
                    foundAnnotation = annotation.shortName.toString() == "JsonClass"
                    if (foundAnnotation) {
                        correctArguments = annotation.valueArgumentList?.arguments?.any { it.text == "generateAdapter = true" } == true
                    }
                    return@forEach // Stop at the first @JsonClass, multiple annotations is not supported.
                }
                if (!foundAnnotation) {
                    emit(klass.startOffset, "Dto class '${klass.name}' should uses the @JsonClass annotation", true)
                }
                if (foundAnnotation && !correctArguments) {
                    emit(klass.startOffset, "@JsonClass should define the argument 'generateAdapter = true'", false)
                }
            } else if (klass.isEnum()) {
                if (klass.annotationEntries.none { it.shortName.toString() == "Keep" }) {
                    emit(klass.startOffset, "Enum should use a @Keep on top of the class to prevent field minification.", true)
                }
            } else {
                emit(klass.startOffset, "Class with suffix Dto should be 'data class' or 'enum class'", false)
            }
        }
    }
}

This code is pretty naive and could be improved a lot. But just with this snippet of code, you ensure that all your classes that ends with “Dto” are proguard-safe.

How to test it

Ktlint offers a pretty simple API, let’s look at a basic example

@Test
fun `signal missing @JsonClass annotation`() {
    DtoRule().lint(
        """data class MyDto(val foo: Int)"""
    ) `should contain same` listOf(LintError(1, 1, "my-rules", "Dto class 'MyDto' should uses the @JsonClass annotation"))
}

That’s it!

You can check the rule, the line/column (relative to the file, starting at 1) where the error should point, and the message to display.

How to implement auto-fix

KtLint provides 2 methods, ktlintCheck and ktlintFormat. The 1st one display the error and if the error is auto-fixable or not. The second one apply the changes. Sometimes it make sense to propose an autofix, sometimes it’s not relevant and the developer have to take the choice itself.

The 2nd parameter of the visit method is a boolean named autoCorrect, and it’s true when we use ktlintFormat, let’s say we want to implement the autofix for the missing @JsonClass, a simple implementation could be

override fun visit(node: ASTNode, autoCorrect: Boolean, ...) {
    [...]
    if (!foundAnnotation) {
        emit(klass.startOffset, "Dto class '${klass.name}' should uses the @JsonClass annotation", true)
        if (autoCorrect) correctJsonClassAnnotation(node)
    }
}

fun correctJsonClassAnnotation(node: ASTNode) {
    (node as TreeElement).rawInsertBeforeMe(LeafPsiElement(REGULAR_STRING_PART, "@com.squareup.moshi.JsonClass(generateAdapter = true)\n"))
}

And now you can run and this will fix all missing @JsonClass!

How to run it

To setup your project:

// File build.gradle
buildscript {
    ext.kotlin_version = "1.3.71"
    repositories {
        google()
        maven { url "https://plugins.gradle.org/m2/" }
        jcenter()
    }
    dependencies {
        classpath "com.android.tools.build:gradle:3.6.3"
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
        classpath "org.jlleitschuh.gradle:ktlint-gradle:9.2.1"
    }
}

// File app/build.gradle
apply plugin: "org.jlleitschuh.gradle.ktlint"
ktlint {
    version = "0.36.0"
    android = true
    ignoreFailures = true
    reporters {
        reporter(ReporterType.CHECKSTYLE)
    }
}
// [...]
dependencies {
    // [...]
    ktlintRuleset(project(":ktlint-rules"))
}

You can know use the command line ./gradlew ktlintCheck, or create a Run configuration in AndroidStudio: ‘Add new configuration’ > ‘Gradle’ > Gradle project: MyProject:app & Tasks: ktlintCheck

When used from AndroidStudio, you can click on the build logs to open directly at the location of the report.

Conclusion

I had some difficulties to find articles on that matter, and I find the documentation and the API hard to use, but the result is here. I’m now able to detect all issues when opening my PR, without even compiling the project, big win. I’m also way more confident in how we parse our Json and how proguard will impact us.

I hope this will motivate you to try to write your 1st rule. If you’ve already wrote some, I’m interested in any documention on the topic, please leave a message.

If you are interested by the full Moshi rules we’re using at Betclic, please contact me via Disqus or Twitter.

Should I use Jetpack ViewModel?

2020-05-15T00:00:00+02:00

Adding a new library to a production project should always be done with extra care.

Jetpack introduced some months ago a ViewModel library with a class of the same name. This new class “store and manage UI-related data in a lifecycle conscious way”. It’s not really limiting to the MVVM pattern actually and can be inherited by your presenters, controllers,…

In this post, I presume you know the basics of ViewModel (if you don’t, there are many good resources out there) and we’ll try to go further and understand the implications of adding this library into a project.

Lifecycles

One of the sexiest part of ViewModel, from my point of view, was the idea of ‘a simpler lifecycle’. Android lifecycles are quite complex, so this is quite a big deal for simplifying our code and spend more time on feature than on fixing lifecycle-related issues.

Let’s look at this super simple lifecycle!

This look sexy! All is handled in the body of the ViewModel, I don’t even need to remember lifecycle methods anymore…

And it handles re-creation! And I even have a method to clean stuff if needed!

Unfortunately you’ll always need to have lifecycles to know if you’re drawn, if you’re re-resumed, or re-started to play an animation at the right moment…

But first let’s take a bigger picture with this version:

(picture from The Android Lifecycle cheat sheet by Jose Alcérreca)

Let’s point out some details from this picture:

ViewModel is not present when the Activity is created
ViewModel is present when the Activity is re-created with onCreate(Bundle)
ViewModel is always present in onDestroy of the Activity (not very symetrical to the onCreate)

And for the Fragment version

ViewModel is not present when the Fragment is attached or created.
ViewModel is present when it’s re-attached or re-created (so in onAttach you cannot know if it exists or not)
ViewModel may be present in onDestroy of the Fragment (you don’t really know)

The Kotlin extensions to retrieve a ViewModel use the lazy mechanism, so you think you deal with a non-null variable in your code but actually you deal with a “it depends” variable if it’s used inside the onAttach or onCreate methods. And the “it depends” is directly translatable by “maybe it will crash your app” since it’s generally used from the main thread.

Adds to that Fragment can be retained (onDestroy/onCreate no more called when device rotate), that there is unfrequent lifecycle methods like Activity#onRestart, Activity#onNewIntent, … or that flags on the Intent actually change the Activity method calls. Lifecycles are complex.

This is not really simplifying the lifecycles but adding a third one to the mix. New developers will still need to understand the Activity/Fragment lifecycles + the fact they can’t use their ViewModel in all lifecycle methods.

That’s said, if you limit your code in Activity / Fragment to the onCreateView / onStart / onResume and the symmetrical onPause / onStop / onDestroyView, you should be safe.

Dependency injection

One of the big issue with Android and DI is the fact that the construction of Activities and Fragments are done by the system. As such, it’s impossible to inject our dependencies directly in the constructor, and we have to rely on field injection:

requires reflection (little impact on performance),
requires internal/public visibility (bad encapsulation),
coupling with the DI library,
hide the dependencies, you can create a Fragment without knowing about its dependencies, or add yet another injected field without thinking “Am I not creating a class with too many dependencies?” (bad architecture),
nullability/immutability issues (cognitive load -> adding lazy { } to class members to defer the work -> degrading performances due to additional useless inner classes)

If after that you’ve still some doubts, google “field injection”.

Example of code where you’ll experiment a crash (NullPointerException) because using field injection:

class MyActivity: Activity() {
    @Inject lateinit var myService: Service

    private val url = myService.getUrl() // throws NullPointerException

    fun onCreate() {
        // Injecting your dependencies
    }
}

Well, it’s almost the same story for Jetpack ViewModels by default. ViewModels are created by a Factory, that is by default provided by Android, so you can start immediately, and get system injections like the application Context or the SavedStateHandle. You can still use field injection but you’ll have the previous issues…

Thanksfully you can plug your Dagger dependencies to the ViewModelProvider! Right?

So if you use Dagger for example, you’ll have to do some “black magic” to plug your dependencies in the ViewModel. Stuff like handling maps of AbstractViewModelFactory factories. Yep, factories of factories. Like if DI was already so easy that we feel the urge to add yet an additional layer of complexity.

A simple example of black magic from the android blueprints itself. Indeed if you take some time you’ll understand all the implications of this code, but as it’s not provided by the library, you’ll have to maintain it at some point…

One of the benefit of the Minimalist Dagger (and the multi-modules article) approach I presented before was that most of the modules don’t use kapt while using Dagger 2, and don’t even need to have a @Module. So you’re reducing a lot the coupling with the DI library. If you “upgrade” to Jetpack ViewModel and use the code provided in the architecture blueprint, you now need a bind method for each ViewModel, like in the blueprint example :

@Module
abstract class StatisticsModule {
    @Binds
    @IntoMap
    @ViewModelKey(StatisticsViewModel::class)
    abstract fun bindViewModel(viewmodel: StatisticsViewModel): ViewModel
}

And now all your modules knows about Dagger (@Module, @Binds, @IntoMap + your custom annotation) and you run kapt in each one of them. For the build time I’m not sure, for the dependency to Dagger I presume it’s “ok-ish” (at best) as Android will invest more on this lib, but for all the developers thinking Dagger is too much verbose…

Did I tell you this black magic code was actually the simplest case?

What if you want to use a SavedStateHandle or inject the App context… or both? This approach is not enough, so you have 2 options: make more black magic DI code, or just make a big switch cases with all the constructors of all your ViewModels.

@Suppress("UNCHECKED_CAST")
class ViewModelFactory constructor(
    owner: SavedStateRegistryOwner,
    defaultArgs: Bundle? = null,
    private val tasksRepository: TasksRepository
    /* Plug all your module dependencies here */
) : AbstractSavedStateViewModelFactory(owner, defaultArgs) {

    override fun <T : ViewModel> create(
            key: String,
            modelClass: Class<T>,
            handle: SavedStateHandle
    ) = with(modelClass) {
        when {
            isAssignableFrom(StatisticsViewModel::class.java) -> StatisticsViewModel(tasksRepository)
            isAssignableFrom(TaskDetailViewModel::class.java) -> TaskDetailViewModel(tasksRepository)
            isAssignableFrom(AddEditTaskViewModel::class.java) -> AddEditTaskViewModel(tasksRepository)
            isAssignableFrom(TasksViewModel::class.java) -> TasksViewModel(tasksRepository, handle)
            else -> throw IllegalArgumentException("Unknown ViewModel class: ${modelClass.name}")
        }
    } as T
}

Indeed the blueprints is not super scalable (maybe it’s not even intended to be scalable), but it means than before Jetpack, to add a ViewModel, you created a new class, and referenced the class in the Fragment, so modifying 2 files. Now you also need to create a dedicated module, and update your big switch case.

What if you want to add another dependency to your ViewModel? Again need to edit the “DI file”, add yet another field in the constructor of your ViewModelFactory… Does it go over your code-quality metrics now?

Also let’s say you’ve a DefaultViewModel that can be instantiated, but also 2 other subclasses with slightly different behaviors. Now the big switch case needs to be ordered to avoid a bad match, and documented because it’s not that obvious that order matters here…

Yes there are other approaches to the problem. But no default approach, or no viewmodel-dagger2 lib that does the work for you yet (this when is pure boilerplate and could be automated), so you’ll have to find a good approach for you before anything else. I’m confident Android engineers will deliver a solution at some point, but when? And with what limitations?

Ok… let’s say we have copy-pasted some files from github or StackOverflow to make Dagger happy, now we’re good on DI right?

Analysis at compile-time

Well actually, before Jetpack ViewModel, Dagger2 was checking all dependencies and was ensuring everything was provided. It means than if it builds, you don’t have dependency issue. Is it still true?

No. As you can see in the when, the ViewModelProvider.Factory signature is based on a Class:

<T extends ViewModel> T create(@NonNull Class<T> modelClass);

So you don’t have the choice than crash at runtime when the class given in parameters is not mapped in your code. Also, as we’ve seen in the Lifecycles part, you cannot create the ViewModel out of a specific lifecycle without crashing, so clearly the dependency injection with ViewModel is no more checked at compile-time. To get back the analysis at compile-time, you can indeed add another annotation processor library, costing more build time…

Communication between views

Using ViewModel allow to easily interact between Activity and Fragment. Let’s say for example there is a ViewModel at the Activity level and a Fragment want to notify of some changes, dead simple (with fragment-ktx 1.1.0+):

class MyViewModel {
    fun getViewState(): MyViewState { ... }
    fun updateStuff() { ... }
}
class MyActivity: Activity() {
    private val myViewModel by viewModels<MyViewModel>()
    override fun onResume() {
        updateView(myViewModel.getViewState())
    }
}
class MyFragment: Fragment() {
    private val myViewModel by activityViewModels<MyViewModel>()
    fun onClick() {
        myViewModel.updateStuff()
    }
}

This is really cool, there will be only one instance of the MyViewModel in this case. Before you were limited to:

inject in the Activity with your DI, then manually injects your fragments when they are attached. This leaks the dependencies of your Fragments in the Activity, so limiting reusability.
create a scoped Component by Activity, adding some complexity in your DI.

Actually for Dagger users, that’s like there were 2 new scopes “ActivityScope” / “FragmentScope” and a dynamic Component for each activity started, so you don’t have to write them!

As a corollary, a custom view can use a ViewModel from its Activity or Fragment, but not from the view itself. You can still create a MyViewModel with Dagger and inject it in the view, but it will not use the ViewModel mechanism, so no SavedStateHandle or onCleared(), so MyViewModel doesn’t need to extend ViewModel.

Same for a ViewModel that needs another ViewModel, there is no solution as far as I know.

Note that the scopes are hidden, so if a fragment uses by viewModel<MyViewModel>() and the activity uses by viewModel<MyViewModel>(), both will have a distinct MyViewModel instance. Fragment should have used by activityViewModel<MyViewModel>() to share the same instance. This kind of setup error will probably occurs at some point.

Communication between ViewModels

Now let’s say there is an Activity with a “master” ViewModel, and 2 Fragments, each one having its own ViewModel. The master requires the validation of the 2 sub ViewModels. How can we plug that?

We can declare the 2 Fragment’s ViewModels on the (invisible) “Fragment scope”, and the Activity listen onAttach -> Nop, the master ViewModel needs to know about the 2 others but the Activity itself doesn’t need to.
We can declare the 3 ViewModels as scoped to the Activity, but then the Activity needs to plug them together.
We can declare the 2 Fragment’s ViewModels on the (invisible) “Fragment scope”, and depends on the master ViewModel like this:

class FragmentA : Fragment() {
    val masterViewModel by activityViewModels<MasterViewModel>()
    val viewModel by viewModels<AViewModel>()
    fun onResume() {
        masterViewModel.addSlave(viewModel)
        // or
        viewModel.onChange { masterViewModel.notifyAChanged() }
        // or anything doing the glue between both ViewModels
    }
}

Whatever the solution that fits your need, there is still a bit of glue. I’d love a solution to ask a ViewModel (if present) from another one, so that I don’t need to change the Activity/Fragment classes anymore. Please share in comments if you know a solution!

By the way, Activity has generally a longer scope than Fragment, so passing the ViewModel of the Fragment to an Activity ViewModel will probably lead to a memory leak.

Provides data to ViewModel

A major benefits with SavedStateHandle is the possibility to get the Activity’s intent extras, or the Fragment’s arguments from the ViewModel. When it’s pluggued (black magic DI), you’ll have a very pleasant experience:

class MyViewModel(savedStateHandle: SavedStateHandle): ViewModel() {
    companion object {
        private const val ARG_KEY = "ARG_KEY"
        fun bundle(userId: String) = bundleOf(ARG_KEY to userId)
    }
    private val userId: String? = savedStateHandle.get<String>(ARG_KEY)
    init { println(userId) }
}

class MyFragment: Fragment() {
    companion object {
        fun newInstance(userId: String) = MyFragment()
            .apply { arguments = MyViewModel.bundle(userId)}
    }
}

A big win here, you can add parameters to the bundle without modifying your view anymore.

Conclusion

This is indeed completely up to you!

But just to sum up my feelings about it:

Topic	without ViewModel	Jetpack ViewModel
Lifecycle complexity	😔	😔
DI complexity	👌	😔
Checks DI at compile-time	😍	😔
Sharing ViewModel	😔	😍
Communication between ViewModel	😐	😐
Handle saved states	😔	😍
Passing args to ViewModel	😐	😍

There are serious drawbacks to go with Jetpack ViewModel, but it’s also helping to clean and reduce your glue code. In my main project, we’ve chosen to go further with ViewModel, but first we worked on some approaches to mitigate the drawbacks. Stay tuned for the next articles on that matter.

RxJava2 subscribeOn: How to use it

2020-05-02T00:00:00+02:00

As we saw in the quizz about subscribeOn, using subscribeOn can produce unexpected and subtle effects.

subscribeOn pitfalls

Observable.just("item")
    .subscribeOn(Schedulers.computation())
    .subscribeOn(Schedulers.io())
    .subscribe { doSomething() }

Will doSomething() be executed on the IO or the Computation scheduler?

This convoluted but simple code is clearly not obvious for every developer.

Actually it’ll be the Computation thread, basically because it’s the last subscribeOn in the reverse order.

If this example looks silly, let’s look at a more concrete and subtle example

class MyRepository {
    val api = //...
    fun getDataFromNetwork() = api.getData().subscribeOn(Schedulers.io())
}

class MyViewModel {
    val repository = MyRepository()
    fun getViewState() = repository.getDataFromNetwork()
            .subscribeOn(Schedulers.computation())
            .map { data -> computeViewState(data) }
    fun computeViewState(data: ...) = /* Something that should run on a computation thread */
}

class MyFragment {
    val viewModel = MyViewModel()
    fun onViewCreated() {
        viewModel.getViewState()
            .subscribe { viewState -> updateView(viewState) }
    }
}

Just trying to keep things simple in this example, there is many issues here, but let’s focus on the ViewModel.

When you look the ViewModel, you’ll have the feeling that you computeViewState method will be executed on Computation. At least, it’s the intent perceived most of the time, and it’s normal since your read it. Unfortunately, the computeViewState is actually done in the IO thread as we learned before, but it’s not even visible on the ViewModel class!

Another way to ends with this issue, let’s say your repositories never defined the subscribeOn before. Then your code was running on a Computation thread as expected. But due to parallel network connection limited by Computation (bounded thread pool), you decide some months later to move all network calls to IO thread by adding the subscribeOn(Schedulers.io()). Unfortunately, one of your ViewModel was requiring to run on computation and you just break the production code of a non-modified file!

I’ve experienced these issues myself, it’s not science-fiction, and maybe your code also have this problem? So how can we simply avoid that?

Rule 1: Use only 1 subscribeOn for each stream, and only when you create the stream.

You don’t want your readers (including yourself) asking about which subscribeOn is really used or navigating in many files to understand the threading logic.

For example, if you use Retrofit for a network call, you can ensure the network call is always done on the IO scheduler:

Retrofit.Builder()
    .addCallAdapterFactory(RxJava2CallAdapterFactory.createWithScheduler(Schedulers.io()))

If you create a stream for a non-RX library:

Observable
    .create { emitter ->
        externalLib.doSomething(listener = { data ->
            emitter.onNext(data)
            emitter.onComplete()
        })
    }
    .subscribeOn(Schedulers.computation())

Defining the subscribeOn at the very beginning of your stream ensure it cannot be changed by anyone else. As most of the business / IO code you write should not be executed on the main thread to avoid freezing your UI, I’d advise to always define a subscribeOn when creating a stream.

I even think that only the wrapper classes that interacts with something external to your code (external libs / Android SDK) should define the threading.

Rule 2: Use the 3rd parameter of timer & interval methods instead of subscribeOn()

Another article explains what are the differences and why you should use this parameter.

Should you explicit the Schedulers.computation() if it’s already the default value in RxJava?

“It depends of your team’s knowledge on Rx.” It could be ok to think with a team of experts you don’t need to be explicit, but what if in some months you hire a more junior (at least on RX) developer?

Being explicit is a bit of verbosity to ensure you clearly expressed the intent. As threading and asynchronism are particularly difficult concepts, I’d strongly advise to being explicit instead of relying on the level of knowledge of your current team.

Rule 3: Use observeOn instead

Use observeOn instead of subscribeOn when you want to run a part of your code in a specific thread pool. observeOn will ensure all the code below this line will run on the given scheduler, so it’s what you expect when reading AND it cannot be override by someone else. The new code:

class MyViewModel {
    val repository = MyRepository()
    fun getViewState() = repository.getDataFromNetwork()
            .observeOn(Schedulers.computation())
            .map { data -> computeViewState(data) }
    fun computeViewState(data: ...) = /* Something that should run on a computation thread */
}

Here the intent reflects what’s happening, and a change in the repository will have no impact on the ViewModel implementation. This approach ensures your encapsulation is well-decoupled and only modified files will have a new behaviour. As an example, if you decide to add a cache mechanism in your Repository:

class MyRepository {
    val api = //...
    val cache: Data? = null
    fun getDataFromNetwork(): Single = 
        if (cache == null) 
            api.getData()
               .subscribeOn(Schedulers.io())
               .doOnNext {data -> cache = data}
        else 
            Single.just(cache)
}

Here you avoid a network call, and as such you don’t really need to subscribe on IO scheduler, because you’re not performing IO operations. Adding a subscribeOn here will just adds a thread swap and not bring any value.

Thanks to the observeOn usage, the ViewModel will NOT be impacted by this change.

Rule 4: Don’t rely on other classes to define your threading.

Rule 3 examples implies directly Rule 4.

class MyRepository {
    val api = //...
    fun getDataFromNetwork() = api.getData().subscribeOn(Schedulers.io())
}

class MyViewModel {
    val repository = MyRepository()
    fun getViewState() = repository.getDataFromNetwork()
            .observeOn(Schedulers.io()) // Not required, should I remove that line?
            .map { data -> storeViewStateOnDisk(data) }
    fun storeViewStateOnDisk(data: ...) = /* Something that should run on an IO thread */
}

Here the observeOn in ViewModel is not necessary and can even be considered as redundant, because the stream already provides the item on IO scheduler.

Again you have to think for the future modifications of the code. Here you have the 2 classes visible in a couple of lines, this is not true in a real project. In some months, adding a cache in the repository will change the threading of the ViewModel, if you don’t protect yourself with an observeOn. Something as simple as writting in a SharedPref is doing an IO operation, so let’s say that the subscribe is done from the main thread (as often) and get data instantly from the repository due to cache, then your app will actually do an IO operation on the main thread (when there is some cache and if you removed the observeOn).

This problem is quite subtle to detect, you didn’t change the ViewModel in question, the ANR is probably not reproducible since you’ve to emulate an overload of the sdcard IO, and reproducible only when you hit the cache. You’re good to lost hours and even days of investigation on this kind of issues.

Just protect yourself, use an observeOn() before any operation requiring a specific scheduler.

Ok, but changing thread has cost right?

Yes, this solution will have a slight impact on your performance, it’s a trade off.

Cons of using observeOn:

You can lost some micro-seconds when switching thread for no reasons.

Pros of using observeOn:

You avoid potential ANR for your clients,
You saved yourself hours of debugging,
You are confident on the behaviour of xyour code,
Your code is future-proof.

If the UI is not lagging and user see the data 0.000001 seconds later, they will never notice it.

Pretty sure there can be other approaches or guidelines on this matter, please share yours!

RxJava2 puzzle: Subscription threading

2020-05-01T00:00:00+02:00

Some months ago during a code review, I asked a develop to avoid doing IO calls on the main thread (creating laggy UI), by adding a subscribeOn(IO). The code was globally similar to that simplified version:

var count = 0
fun doSomething() {
    Observable.fromCallable { retrieveDataFromStorage() }
        .subscribeOn(Schedulers.io()) // Added after the review
        .subscribe(::handleCount)

    println("$count")
    count++
}
fun retrieveDataFromStorage() = count // Simplified

handleCount implementation is not relevant here, it’s just to give some meaning to the code.

Can you spot the error?

Scroll down for the explanation

Explanation?

Actually the above code has a random behaviour.

Without the subscribeOn, the value received was always 0.

With the subscribeOn, the value received was a random between 1 or 0.

So the original developer experimented and decided to replace fromCallable by just, and the bug disappeared!

var count = 0
fun doSomething() {
    Observable.just(retrieveDataFromStorage())
        .subscribeOn(Schedulers.io())
        .subscribe(::handleCount)

    println("$count")
    count++
}
fun retrieveDataFromStorage() = count

And here comes the real problem 😅

Can you spot the error now?

Explanations!

With the fromCallable, the Callable { retrieveDataFromStorage() } was resolved on the subscription. So the count value is interpreted by an IO thread, sometimes before, sometimes after the increment. Before or after depends of the time of execution of the println()and the time requires to change from main to IO thread.

When a lambda is used in a RX stream, with ::pointerFunction or with { function() } you have to think about the threading.

Here by using just() without the lambda, the method retrieveDataFromStorage is executed immediately (counter=0) in the main thread and we pass the result of this execution to the just operator instead of passing a lambda to execute retrieveDataFromStorage later.

And this code is way worse than the original version, because now we had a line of code that tell “don’t worry, you’re subscribing in the IO thread”. Ok, but the subscription code does nothing but pushing the value given in parameter, it’s not executing the lambda anymore.

How to fix this?

This depends a lot of the context, but I cannot omit this point of the article.

This could be a valid approach for example:

fun doSomething() {
    Observable.just(retrieveDataFromStorage())
        .subscribeOn(Schedulers.io())
        .subscribe {
            handleCount()
            count++
        }
    println("$count")
}

But then the count is maybe not incremented at the end of the method, again depending of the execution time of println vs subscribeOn.

Another simple approach, move all the code in the subscribe, so that the related logic is executed synchronously:

fun doSomething() {
    Observable.just(retrieveDataFromStorage())
        .subscribeOn(Schedulers.io())
        .subscribe {
            handleCount()
            count++
            println("$count")
        }
}

Most of the time it’s because the original code is not implemented in a thread-safe approach like in this example, and those proposals doesn’t fix entirely the problem.

Some tips that can help you in this kind of cases:

Stay in a stream. Pure reactive programming doesn’t allow side effect (like count++), all the code is in the functions, and the immutable data transit from one method to another. Pure reactive can be hard to implement and not a pragmatic solution in my limited experience. But if you can architecture your code so that you’re just adding operators with no side-effects from the source of truth to your view, you’re probably safe.
Debug with logs. Seems silly but actually debugging with breakpoints will probably make your issue disappear. Also keep in mind that printing on std ouput is not free, so adding log can also make the bug disappear.
Learn thread-safe patterns. This is a very old topic and there is many ways to deal with thread-safety, but it’s good to understand that most of the code we write is NOT thread-safe and could lead to random issues. Just an example with an increment on a variable:

    var count = 0
    fun main() {
        val threads = (0..99).map {
            Thread(Runnable {
                (0..99).forEach {
                    incrementCount()
                }
            }).apply {
                start()
            }
        }
        threads.forEach { it.join() }
        println("count=$count")
    }
    fun incrementCount() { // NOT thread-safe
    	count++
    }

This code prints a value between 100 (theoretically) and 100000, because the operator ‘++’ do 3 things, it reads the value then increment, then store the value in the “shared” variable. So 2 threads could read the value at the same time (ex: 3), increment to get the same value (ex: 4), and store twice this value, instead of storing 5. Eventually

Keep one source of truth. It’s not really a solution for the original example but we had 2 sources, the RAM (with the cached data) and the IO (via a SharedPreferences). Due to main of your code being not thread-safe, you could end up with different values in different ViewModel reading the same data on disk.

RxJava2 puzzle: Do you understand subscribeOn?

2020-05-01T00:00:00+02:00

In this post I’ll display the response in white just after the questions, take time to challenge yourself before selecting the text to see the response ;)

By default I’ll consider the code executed from the main thread.

Just as a remember, the basic use case:

Observable.just("item")
    .subscribeOn(Schedulers.io())
    .subscribe { doSomething() }

Here the doSometing method is called on one of the thread of the Schedulers.io().

Puzzle 1 - subscribeOn + observeOn

Observable.just("item")
    .observeOn(Schedulers.io())
    .subscribeOn(Schedulers.computation())
    .subscribe { doSomething() }

A - Main thread

B - Computation

C - IO

D - It’s not executed

IO - the subscribeOn operator impacts the start of the stream, until an observeOn changes the thread. The order of the subscribeOn is irrelevant.

Puzzle 2 - 2 subscribeOn

Observable.just("item")
    .subscribeOn(Schedulers.computation())
    .subscribeOn(Schedulers.io())
    .subscribe { doSomething() }

A - Main thread

B - Computation

C - IO

D - It’s not executed

Computation - the subscribeOn operator only consider the very first subscribeOn and ignore the other ones, so order matters.

Puzzle 3 - subscribeOn + 2 doOnSubscribe

doOnSubscribe offers the opportunity to react when the subscription occurs… but on which order is it executed?

Observable.just("item")
    .doOnSubscribe { foo() }
    .doOnSubscribe { bar() }
    .subscribeOn(Schedulers.computation())
    .subscribe()

A - foo() then bar()

B - bar() then foo()

C - only foo() is executed

D - It’s not executed

foo() then bar() - Same order than when you read the stream, convenient... right?

Puzzle 4 - 2 subscribeOn + 2 doOnSubscribe

What if we add another subscribeOn

Observable.just("item")
    .doOnSubscribe { foo() }
    .subscribeOn(Schedulers.computation())
    .doOnSubscribe { bar() }
    .subscribeOn(Schedulers.computation())
    .subscribe()

A - foo() then bar()

B - bar() then foo()

C - only foo() is executed

D - only bar() is executed

bar() then foo() - Opposite order than the previous method!

Feel lost? Let's do one more step before the explication

Puzzle 5 - 3 subscribeOn + 2 doOnSubscribe

Let’s add another subscribeOn, and think about the threading on which foo() and bar() are executed.

Observable.just("item")
    .subscribeOn(Schedulers.single())
    .doOnSubscribe { foo() }
    .subscribeOn(Schedulers.computation())
    .doOnSubscribe { bar() }
    .subscribeOn(Schedulers.io())
    .subscribe()

A - foo() on Single then bar() on Computation

B - bar() on IO then foo() on Computation

C - bar() on Computation then foo() on Single

D - I give up

B - bar() on IO then foo() on Computation

You can think the subscription stream from bottom to top, and the doOnSubscribe like a add to a runnable list. Each time you reaches a subscribeOn, you execute the runnables and after that change the thread.

The execution of the subscription starts from the bottom and go up. 1st it reaches a subscribeOn so it switch thread, then it reaches doOnSubscribe { bar() } so it adds bar() in a sort of list. Then it reaches another subscribeOn, and here before changing thread it execute its list (so bar is executed). Once the executable list is empty, it will change again from threads, and eventually execute foo()

Puzzle 5 - subscribeOn + timer

Ok, too much lines, let’s restart from simple:

Observable.timer(10, TimeUnit.MILLISECONDS)
    .subscribeOn(Schedulers.io())
    .subscribe { doSomething() }

A - doSomething is executed on IO

B - doSomething is executed on Single

C - doSomething is executed on Computation

D - doSomething is executed on main

C - doSomething is executed on Computation

Some methods, like timer or interval, takes a 3rd parameter for the scheduler. If you don't define it, the method has a default for you, here it's Computation.

Puzzle 6 - subscribeOn + Subject

Add some Subject now?

val subject = BehaviorSubject.create<String>()

Observable.just(0)
    .observeOn(Schedulers.io())
    .subscribe { subject.onNext("any") }

Thread.sleep(10)

subject.subscribe { doSomething(it) }

A - doSomething is executed on main

B - doSomething is executed on Computation

C - doSomething is not executed

D - doSomething is executed on io

A or D - it depends!

I've seen this pattern multiple times. The logic behind was something like

<< if all my network calls are done on IO, then I can publish the result in a Subject and observers of the subject will receive items on IO too >>

Unfortunetaly, it's not that simple when the subject has a "memory" like Behaviour or Replay subjects. When you subscribe, if the data is already in memory, you'll receive the data on the main thread, if there was no data yet then you'll receive the data on the IO thread, preserving the thread. So you can use this approach with Subject without memory but it's not really safe as you can see.

Right now on my computer, if I change the sleep to 0ms, "any" is received on IO, if I change the sleep to 1ms, "any" is received on main.

If you’ve experienced some difficulties on these questions, I’ll soon write an article on my best practices to avoid these pitfalls.

RxJava2 puzzle: Scheduler as 3rd parameter

2020-04-29T00:00:00+02:00

On which thread doSomething() will be executed for these 3 cases?

    Completable.timer(10, TimeUnit.MILLISECONDS)
        .subscribeOn(Schedulers.io())
        .subscribe { doSomething() }

    Completable.timer(10, TimeUnit.MILLISECONDS)
        .observeOn(Schedulers.io())
        .subscribe { doSomething() }

    Completable.timer(10, TimeUnit.MILLISECONDS, Schedulers.io())
        .subscribe { doSomething() }

Yes there is a trap, response is at the end of the article to avoid spoiler.

This example use Completable as it’s the simplest form, but the behaviour is the same for Observable, Single, Maybe and Flowable.

RxJava2 Schedulers

Scheduler implementations in this post can be simplified to a group of threads. Schedulers.io() for example will create a new thread each time it’s needed (no limit on the count), while Schedulers.single() re-use always the same Thread.

RxJava2 observables subscription

First we need to know some things about how works the subscription.

All classes that extends Completable / Observable / etc have to implement a subscribeActual method. This method is called when the stream is actually subscribed.

val stream = Completable.just(1) 
// subscribeActual is not called yet at this point
stream.subscribe() // This code runs subscribeActual.

The subscribeActual is called from the thread that calls subscribe(), so in an Android view, it means the code inside this method actually runs on the Main thread. This is perfectly fine indeed, as the scheduling is not blocking.

Also if you want this code to be run on another thread, you can use subscribeOn().

How is scheduling implemented?

If we open the RxJava2’s CompletableTimer, we can notice that the scheduling code is done in the subscribeActual.

@Override
protected void subscribeActual(final CompletableObserver observer) {
    TimerDisposable parent = new TimerDisposable(observer);
    observer.onSubscribe(parent);
    parent.setFuture(scheduler.scheduleDirect(parent, delay, unit));
}

Behind the scene, RxJava2 is using ScheduledExecutorService#schedule. The internal implementation is provided directly by Java and it handles the scheduling via an Executors.

Thanksfully, the low-level implementation is not very relevant, but we can note that the RxJava2 code is not actively waiting but only delegating the work, so the thread calling scheduler.scheduleDirect() does not matter here. What we want to know is on which thread the Future result will be sent.

The 3rd parameter

The 3rd paramter of the timer like methods is the scheduler. By default this scheduler is Schedulers.computation(). The method documentation says about this param:

  scheduler - the scheduler where to emit the complete event

This means the complete event, and so the lambda containing our code, will be called on one of the thread of the given scheduler.

Completable.timer(10, TimeUnit.MILLISECONDS)
    .subscribe { /* this code actually runs on a Computation thread */ }

The `subscribeOn` case

Let’s have a look again at the subscribeOn version:

Completable.timer(10, TimeUnit.MILLISECONDS)
    .subscribeOn(Schedulers.io())
    .subscribe { doSomething }

With the previous knowledge, we can deduce that:

due to the subscribeOn, the scheduling code will be done in an IO thread
then the result will be given to one of the Computation thread
so the doSomething() actually runs on one of the computation thread.

Clearly this is dangerous code: the perceived intent is that this code should run on the IO scheduler but it’s not.

The `observeOn` case

The observeOn operator changes the thread of the underneath stream. In this case, this means that the completed event sent from the timer will change thread during the observeOn, to a IO thread.

Completable.timer(10, TimeUnit.MILLISECONDS)
    .observeOn(Schedulers.io())
    .subscribe { doSomething() }

As a result:

the timer emits the complete event on a Computation thread,
the observeOn change from Computation to an IO thread (performance cost),
the doSomething() actually runs on one of the IO thread.

This is better as the execution reflects the perceived intent.

The `3rd parameter` case

Completable.timer(10, TimeUnit.MILLISECONDS, Schedulers.io())
    .subscribe { doSomething() }

Eventually if we only use the 3rd parameter:

the timer emits on an IO thread
the doSomething() actually runs on one of the IO thread.

Conclusion

subscribeOn will execute the code on a Computation thread, not the expected behaviour. subscribeOn is generally an operator that is badly interpreted by many developers, more article to come on that topic!

observeOn will execute the code on the IO thread as exepected, but will add an unnecessary change of thread.

Using the 3rd parameter do what you expect and have the minimal performance cost.

Want to test yourself?

@Test
fun the3rdParameter() {
    Completable.timer(10, TimeUnit.MILLISECONDS)
        .subscribeOn(Schedulers.io())
        .subscribe { println("subscribeOn: " + Thread.currentThread()) }

    Completable.timer(10, TimeUnit.MILLISECONDS)
        .observeOn(Schedulers.io())
        .subscribe { println("observeOn: " + Thread.currentThread()) }

    Completable.timer(10, TimeUnit.MILLISECONDS, Schedulers.io())
        .subscribe { println("timer: " + Thread.currentThread()) }

    Thread.sleep(100)
}
// subscribeOn: Thread[RxComputationThreadPool-1,5,main]
// observeOn: Thread[RxCachedThreadScheduler-2,5,main]
// timer: Thread[RxCachedThreadScheduler-1,5,main]

Minimalist Dagger for multi-modules

2020-04-28T00:00:00+02:00

This article is the following of the Minimalist Dagger for monolithic app, I’ll start from there.

In the previous article, we’ve defined the minimum set of tools to use DI advantages with the minimum setup. This setup was enough for a monolithic application, but at some points, some of your apps will need to modularize.

This approach aims to keep the simplicity of the 1st article in a multi-module app, while limiting the build cost due to code generation.

There are tons of great modularization articles, so to resume, a module is a unit of code functionnally and physically distinct. They have nothing in common with a Dagger @Module, you can have multiple @Modules or none in your “compilation” module. (All references to a module in this article refers to the compilation module.) Generally each one of these units will have distinct dependencies (to libraries or/and to other modules), so our first need is to describe what we need to have to work. Inside the module, you’ll also need to inject some fields like some controller in your views. Eventually if this module is also used by some other modules, you could have to expose that… But it’s tedious and this article aims to provide a concrete solution to implement DI in your app, not in an open-source library, so let’s keep it super simple.

The pattern

Let’s say we have a Login module, we’ll add 1 file in the module to define these lines:

1 - The InjectorProvider is just an interface to be implemented by the Application itself, to expose the injector.

interface LoginInjectorProvider { // Will be implemented by the Application itself
	fun loginInjector(): LoginInjector
}

2 - The injector interface lists all the classes where you want to inject fields (generally your activities and fragments).

interface LoginInjector {
	fun inject(loginFragment: LoginFragment)
}

3 - Potentially some extension functions to make the injection easier. For example:

fun Activity.loginInjector() = (this.application as LoginInjectorProvider).loginInjector()
fun Fragment.loginInjector() = (this.activity?.application as LoginInjectorProvider?)?.loginInjector()
	?: error("Cannot inject without a proper reference to the application")
fun View.loginInjector() = (this.context.applicationContext as LoginInjectorProvider).loginInjector()
fun Service.loginInjector() = (this.applicationContext as LoginInjectorProvider).loginInjector()

You don’t need to define them all, just pick what you want. As you can see, the idea here is to get the application context, cast it as our InjectorProvider, and get the injector from there. The cast is not safe, so if you’ve just added a new module and forgot to setup the Application, your app will crash at runtime with a ClassCastException. Thanksfully we’re not creating one module by day, so this should be fine.

So this is working because our app now implement the InjectorProvider:

class MyApplication: Application(), LoginInjectorProvider, FooInjectorProvider, BarInjectorProvider {
    val component = DaggerAppComponent.create()
    override fun loginInjector() = component
    override fun fooInjector() = component
    override fun barInjector() = component
}

This way you can re-use any modules in any application, if the application implements the given InjectorProvider. As you can see, the Application file will increase from a couple of lines for each module. That’s not elegant, but as it’s pure kotlin interface implementation, if your setup is incomplete it’ll not compile.

Last piece of this pattern, you need to update your AppComponent to give the ability to provide this new module:

@Singleton
@Component
interface AppComponent: LoginInjector, FooInjector, BarInjector {
}

Here too, the main component will grow a bit for each new modules. If you’ve hundreds of them, you could totally group them by meta features for example, it’s pure interface Kotlin, you can be creative.

Injecting

No changes when you want to inject via constructor.

class Car @Inject constructor(val steeringWheel: SteeringWheel) {
}

But if for example you want to inject fields in a Fragment from the Login module, you’ll have to pick the module injector like this:

class LoginFragment : Fragment(R.layout.fragment_login) {
    @Inject
    protected lateinit var viewModel: LoginViewModel
 
    override fun onCreate(savedInstanceState: Bundle?) {
        loginInjector().inject(this)
        // ...
    }
    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        // Use viewModel to setup your view
    }
}

Build time

In most of your modules you don’t need to have any Dagger @Module, nor @Component. The big advantage with this approach is that you don’t have to use kapt on these modules!

Actually the code generation happens only at the app level, when building the AppComponent, where it’s always required with Dagger 2 anyway. There, due to our usage of interface inheritance, the AppComponent is actually declaring the inject methods for all the modules.

Our code:

// In Login module
interface LoginInjector {
	fun inject(loginFragment: LoginFragment)
}

// In Foo module
interface FooInjector { 
	fun inject(fooFragment: FooFragment)
	fun inject(fooActivity: FooActivity)
}

// In the app module
@Singleton
@Component
interface AppComponent: LoginInjector, FooInjector {
}

Will produce an interface similar to this manual implementation:

@Singleton
@Component
interface AppComponent: LoginInjector, FooInjector {
	fun inject(loginFragment: LoginFragment)
	fun inject(fooFragment: FooFragment)
	fun inject(fooActivity: FooActivity)
}

Dagger2 see this inherited method and will generate the code for them.

So you can keep your code & dependencies grouped in each well-defined module, and it will result in a single component for all your application.

Pros & Cons

Pros&Cons are similar to the mono-module implementation, but some differences:

Pros:

Build time avoided on the Dagger-free submodules (good for unit-tests especially if you don’t use Dagger in them, like me).
Interface inheritance is way easier to understand than Dagger subcomponents.
Dagger-free modules, this is exactly what Dependency Injection should look like*.
Dagger code is in less files, so changing the DI library is way more affordable than when using extensively Dagger2 subcomponents.

Cons:

build time still paid when building the app (unless you want to try Dagger2 reflect, you have to pay code generation price anyway)

* Imagine a library based on Dagger that requires you to provide some stuff through Dagger mechanism, or ask you to add a @Modules in your main component. It’s a main architectural problem because now you cannot change your DI library when you want to, and you coupled a library to another one (Dagger). Fortunately all well-defined libraries avoid that (often by using static entry point or manual instanciation).

With this approach your modules can be Dagger-free, while still using Dagger 2 to build your dependency graph.

Showcase project

glureau/MinimalistDaggerMultiModules

FAQ

Post a comment and I’ll try to provide a proper response in this article.

Minimalist Dagger

2020-04-27T00:00:00+02:00

Dagger 2 is often considered as too complex for reasons, this article will try to propose a really simple approach limiting complexity and easing the learning curve for newcomers.

I purposefully skip a lot of details, because I consider most of the details have a bad complexity/gain ratio, but we’ll go further on this point on next articles.

Annotations

JSR-330

Developed with Java, this set of annotations define how to declare DI and contains:

@Inject : Applied to a constructor or a field, it indicates this dependency will be provided eventually.

@Singleton : Applied to a class, when you want to have only one instance of it.

This de-facto library only declares annotations, it doesn’t need apt/kapt so it doesn’t have any build time cost and is very lightweight. You can use these annotations with other libraries than Dagger, so having them in your code doesn’t mean you are coupled to Dagger. So you’ll be able to change to another JSR-330 compatible DI library (eg. Kodein) when you want to, without modifying all your classes.

Dagger2

For the basic needs we’ll only use:

@Component : to define a component that will contain every dependencies

@Module + @Provides : to define how to create instances that don’t belong to our code. A classic example is for creating an okhttp instance.

Implementation strategy

Scopes and multi components are complex to deal with, and can lead to subtle errors. We want to be efficient in what we develop, focus on business logic, not on this kind of errors, so no need to have that complex memory management. That’s it.

If you need an instance of an object and you don’t care about sharing, just use @Inject on the class constructor you want to inject and where you need the dependency.

class SteeringWheel @Inject constructor() {
    // Use @Inject on an empty constructor will define 
    // to your DI library how to create the class.
}

class Car @Inject constructor(val steeringWheel: SteeringWheel) {
    // When creating a car, your DI library will create a new SteeringWheel.
}

If you need to keep an object in memory for the entire application’s lifetime, you just have to add @Singleton on the shared class:

@Singleton
class World @Inject constructor() {
    val createdCityCount = AtomicInt(0)
    // Only one instance will be created (more on that later).
}

class City @Inject constructor(val world: World) {
    // When creating multiple cities, all cities will be created with the same world instance.
    init {
        world.createdCityCount.increment()
    }
}

Implementation

First you need to define a Component for the application, to access your dependencies.

@Singleton
@Component
interface AppComponent {
    // List all the classes where you want to inject fields (not required when injecting via constructor)
    // Can also add accessor to singleton objects if required.
}

And a way to access the component from everywhere when injecting fields:

class MyApplication : Application() { // Update your manifest accordingly if you created this new Application class
    val component = DaggerAppComponent.create()
}

val Fragment.injector: AppComponent
    get() = (requireActivity().application as MyApplication).component

You’re good to go!

Want to try on a showcase project? glureau/MinimalistDagger

An example

Let’s define a Singleton:

@Singleton
class NotificationManager @Inject constructor() {
    val count: Int = 0
}

An unscoped ViewModel:

class DashboardViewModel @Inject constructor() {
    val text: String = "Dashboard"
}

Now when using a fragment, you can do :

class DashboardFragment : Fragment(R.layout.fragment_dashboard) {
    
    @Inject
    protected lateinit var notification: NotificationManager
    @Inject
    protected lateinit var viewModel: DashboardViewModel
    
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        injector.inject(this)
    }

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        val textView: TextView = view.findViewById(R.id.text_dashboard)
        textView.text = viewModel.text + " (notif=" + notification.count + ")"
    }
}

Things to keep in mind

Singletons are not Java static classes nor Kotlin objects. They are unique only in our Dagger component, so using @Singleton and using a static variable are fundamentally different.

Singletons are lazily created, it means you are delaying their initialization until the first time they are needed. If you define a singleton with a behaviour in the constructor but no one refers to it, this class will never be created, and the code in construtor will never run. A simple workaround can be to add a line in your application to trigger the lazy resolution.

class MyApplication : Application() {
    val component = DaggerAppComponent.create()
    init {
        component.getMySingleton()
    }
}

Once instantiated, a Singleton will never be released, unless the app is terminated (crash, process killed manually or by the OS).

If the process crash (low memory -> Android decides to kill your process), the data in Singletons (or even Java static classes/kotlin objects) are lost, BUT the backstack (Activities/Fragments) are saved automatically by the OS. So if you do nothing, when the app restore a screen, it will start with a new Singleton. As a consequence, you should not rely on the fact that you came from another screen that has passed the data to a singleton to retrieve it. In this case, a few options:

Store the states you want to reload (onSaveInstanceState / SharedPreferences / Database / SavedStateHandle / …)
Detect the restoration and reload your application to the default screen (startActivity with a flag to clear all other activies, and restart from a clean state). The latter is great for little teams or projects that don’t want to spend too much time on data migration and tests. The first option will provide a better user experience in those cases.

Using @Singleton or using nothing means there are 2 kind of scope, the App scope (@Singleton), and the unscoped. So even if they are not custom scope, it’s important to understand the difference.

Pros & Cons

Pros:

Easy to understand for newcomers, no time spent trying to understand how the Dagger class binding is working or where I should write my modules and sub-components.
No boilerplate, so it’s super easy to maintain (actually it’s free on our project).
A big improvement for the team moral, no more time spent trying to understand generated code.

Cons:

RAM usage: you’ll keep Singleton annotated classes probably longer than what is strictly required. As a team of 7 Android developers at Betclic, we are working on a 150k LoC sport betting application installed on 400k+ devices displaying thousands of matches, animating betting odds updates in realtime. We have a 99.9% crash free rate, and the 0.1% of the crashes are not related to memory issues, so I don’t think it matters that much.
No scoping: you’ll have to clean the data in your Singletons when it’s not used anymore, instead of just dropping a sub-component. (example in FAQ)
Singleton or manual injection when you need to share some data or ViewModels: if using Singleton, they could be poorly implemented and hold some data related to a specific page. Good practices need to be in place to avoid that.

FAQ

Post a comment and I’ll try to provide a proper response in this article.

Doing that with DI is complex, but you can easily have a @Singleton class that will handle that, for example :

@Singleton class UserManager @Inject() {
    var user: User? = null
    fun login(u: User) { user = u }
    fun logout() { user = null }
}

The main implication is that you need to know when you do or don’t use it anymore.

I have multiple instances of the same class that I want to inject.

It can happen in some projects, like for example a couple of okhttp client instances with distinct setup.

If you encounter this specific case, you will have to define a Module to provide instances, and on the provide method add a simple @Named(“some_name”). Now when you need to inject one specific instance, you’ll simply add the @Named annotation on the field:

class MyClass @Inject(
    @Named("public") private val httpClient: OkHttpClient
)

Can I really keep this minimalism in a multi-module architecture?

Sure you can!

Showcase project

glureau/MinimalistDagger

Thanks to Matthieu Coisne for correcting my bad english.

Discovered this nice video while writing this article, great content!

TDD failures?

2020-04-19T00:00:00+02:00

In 10 years of being a developer, I attended multiple “TDD” dojos. They were all great, trying to push some good practices like:

Tests first
Red/Green/Refactor
100% code coverage
self-documented

But unfortunately, when I spoke to the attendes or even the organizer some days or weeks after that, I discovered that no-one was using TDD in its day-to-day job!

Why?

Personally, there were some reasons… or feelings to be right:

Bad reputation “TDD is hard to do right”, “if you don’t do TDD right, it will only increase your dev time”
I don’t always know what the system is supposed to do, so I can’t write tests firsts.
When I add unit-tests before I refactor, I have to rewrite most of the tests anyway, so they are not really safer if wrote first.
100% code coverage looks bad. When I have a data class, or a wrapper, the code is so dummy than the tests are just verbosity. (We don’t consider test coverage based on which parts is more likely to have bugs or be refactored, we just want coverage everywhere.)
Each time I had to change some piece of code that has good coverage, the self-documentation was unclear, because it was more about what is going on, then why is this required in the first place.
Test code is a copy-cat of the code under test! When using mocks and testing every single lines, tests are just another way to define the code. It’s fine in the way it push against regressions, but it also push against changes. Maybe it’s because I don’t follow Open/Closed principles properly, but it can be hard to know by advance how the code will evolve. Eventually when a lot of the code have to be re-written, the unit tests are just dropped (keeping just the test names to provide a close case coverage).
Don’t open your private methods with @VisibleForTesting. You’re breaking the encapsulation to test something that anyway, will never be called this way. (Frankly I’d be better off without this annotation, in my opinion, it’s breaking one of the more important concept of OOP.)

Finally I discovered this talk, thanks to Twitter, and it explained to me why it feels wrong. Spoiler: I’m doing unit test wrong, the dojos were not focused on the more important pieces of TDD.

https://www.youtube.com/watch?v=EZ05e7EMOLM

Is TDD Dead? (David Heinemeier Hannson / Martin Fowler / Kent Beck) https://martinfowler.com/articles/is-tdd-dead/ https://www.youtube.com/watch?v=z9quxZsLcfo https://www.youtube.com/watch?v=JoTB2mcjU7w https://www.youtube.com/watch?v=YNw4baDz6WA https://www.youtube.com/watch?v=dGtasFJnUxI

The things I’ll try to remember and apply for now on:

“test first” is just a way to do TDD, it applies well to specific case, but it’s clearly not the only way to go. The goal is to have confidence in our code by delivering a test suite, doesn’t matter how we get there.
Apply unit test to a higher level, a unit test can imply multiple methods/classes, so refactoring can effectively become easier.
Better unit tests means less unit tests (at least on my limited experience)
case coverage != code coverage:
- Defines UT from business spec, no more no less.
- Cases defined by the busincess/customer should always work. If some parts of the code can be out of the defined cases: check with business to add a new test acceptance, remove the code, or let it untested.
- Don’t get obsessed with code coverage.
Architecture should be very cautious about adding indirection/complexity for the sake of testability.
Indirection (to be able to test, or induced by a “well-designed architecture”) add complexity and psychological load, making anything harder for the next developer to read/understand/fix/change/improve. So adding tests can actually makes everything worth…
A mock that returns a mock that returns a mock…: your class under test relies on other class hierarchy, it’s not good encapsulation and is a smell.
“Unit” is not defined on purpose. Each team is responsible to define the test style for which is best for the project and the team.
(personnal deduction) The psychology of each developer impacts how the code is architectured and if each methodology is applicable or not. Forcing in some ways is not the right approach, but sharing some good practices in the team is indeed critical (for the project and for each developer).
coupling directly opposed to cohesion? Nice thought from DHH, need to go deeper on this topic.
“Hard to test” often means “bad design”
Increase Test fidelity (use case coverage) -> Increase cost of the feature (time to test, maintenance) so should be evaluated cautiously.

As always in developping, all is about trade-off, it’s a difficult balance to adjust all the time.

There is no silver bullet, try new approaches and pick what fit best for you.

Grégory Lureau

An audio visualization with Jetpack Compose

Ktlint: a great 1st experience

Codegen is not always safe

Moshi, where is my @JsonClass?

Moshi, where is my enum class?

What’s the problem?

Lint/Ktlint

Benefits

How to setup a custom rule

How to write a custom rule

How to test it

How to implement auto-fix

How to run it

Conclusion

Should I use Jetpack ViewModel?

Lifecycles

Dependency injection

Analysis at compile-time

Communication between views

Communication between ViewModels

Provides data to ViewModel

Conclusion

RxJava2 subscribeOn: How to use it

subscribeOn pitfalls

Rule 1: Use only 1 subscribeOn for each stream, and only when you create the stream.

Rule 2: Use the 3rd parameter of timer & interval methods instead of subscribeOn()

Rule 3: Use observeOn instead

Rule 4: Don’t rely on other classes to define your threading.

Ok, but changing thread has cost right?

RxJava2 puzzle: Subscription threading

Explanation?

Explanations!

How to fix this?

RxJava2 puzzle: Do you understand subscribeOn?

Puzzle 1 - subscribeOn + observeOn

Puzzle 2 - 2 subscribeOn

Puzzle 3 - subscribeOn + 2 doOnSubscribe

Puzzle 4 - 2 subscribeOn + 2 doOnSubscribe

Puzzle 5 - 3 subscribeOn + 2 doOnSubscribe

Puzzle 5 - subscribeOn + timer

Puzzle 6 - subscribeOn + Subject

RxJava2 puzzle: Scheduler as 3rd parameter

This example use Completable as it’s the simplest form, but the behaviour is the same for Observable, Single, Maybe and Flowable.

RxJava2 Schedulers

RxJava2 observables subscription

How is scheduling implemented?

The 3rd parameter

The subscribeOn case

The observeOn case

The 3rd parameter case

Conclusion

Want to test yourself?

Minimalist Dagger for multi-modules

The pattern

Injecting

Build time

Pros & Cons

Pros:

Cons:

Showcase project

FAQ

Minimalist Dagger

Annotations

JSR-330

Dagger2

Implementation strategy

Implementation

An example

Things to keep in mind

Pros & Cons

FAQ

I want to keep scoped data, like user related information when logged in, and clean that when the user is logged off.

I have multiple instances of the same class that I want to inject.

Can I really keep this minimalism in a multi-module architecture?

Showcase project

Related articles & videos

TDD failures?

The `subscribeOn` case

The `observeOn` case

The `3rd parameter` case