How do thread priorities affect your Android app?
Introduction
Threads are essential for responsive UI applications. When programming in Android, we make sure that any kind of work that could cause the slightest lagging is scheduled to a separate thread, other than the one responsible for the UI updates.
And even though there are various high level constructs available for the developer’s convenience, how threading works at a very low level leaks from all these abstractions nonetheless.
This is sometimes evident in subtle (but quite convincing) assumptions someone tends to make in a multithreaded context. For instance, the supposition that if the work in the background thread is very light it can finish earlier than onCreate() (which we have measured and we know will take much longer).
At Booking.com, performance and scalability are of the utmost importance, so we always try to apply the best possible solution for a specific problem and have many tricks up our sleeves. We have a huge code base and we have more or less used all the possible ways for achieving concurrency, ranging from raw threads and AsyncTask to coroutines.
As frameworks hide more and more details from the developers so they can focus on the actual task, it’s always valuable to have a deeper understanding of what’s actually going on in the system and to be able to reason effectively about the code flow — especially when things break (which usually happens in production, as Murphy’s law dictates).
This article gives a gentle introduction to the topic and provides some insight of what’s actually happening when we use threads in our Android applications.
Java and Concurrency
The JVM specification doesn’t really indicate how Java threads are created. It only mandates thread semantics and behaviour. Today’s JVM implementations keep a mapping between a Java thread and a native OS thread, deferring the actual management of the threads to the underlying OS. For Android, the host OS is a specific version of Linux which has been properly tailored to support Android’s characteristic requirements in terms of security, network connectivity, performance, limited resources etc.
What is a thread?
Although developers conceptually distinguish among threads and processes, there’s no such distinction for Linux. At the kernel level, a thread is just a lightweight process (lwp) which serves the purpose of being a schedulable entity with resources allocated to it, so that it can finish its task.
A set of these lightweight processes form a thread group which implements the concurrency aspects and also acts as a single unit in regards to system calls as the programmer would expect.
The first lwp is the group’s leader and its PID is shared among the rest of the threads in the group, thereby adhering to the requirement that all threads share the same PID (the getPid() returns the tgid).
Linux starts a “thread” by using clone(2), which actually creates a new child process but with specific flags indicating that the parent and child process share resources (such as virtual address space, or file descriptors) as well as priority, but have a separate stack.
How do threads get the CPU?
Since the number of CPUs on a system is limited, threads have to compete with each other for CPU time. This contention can have a visible and undesirable effect on the user experience. So what decides which thread gets the CPU next? Again, this is not part of the Java specification but deferred to the underlying OS. (From here on we use the terms thread and process interchangeably.)
The basic idea is that the available CPU time is divided into slices (quantums) and each process gets a slice for its task. The process runs until that quantum expires, or has to yield the CPU due to blocking I/O or some interrupt, at which point the process is preempted and the CPU is assigned to another process.
Depending on how big the slice is, we have three types of scheduling:
SCHED_FIFO: A process uses the CPU for as long as is required (infinite slice) to complete its task and only a process with higher priority can preempt it. This is how high performance threads are scheduled in Android (e.g. audio)
SCHED_RR: A process can use the CPU for a specific time duration (finite slice) and the scheduler ensures that all tasks of the same priority are run
SCHED_OTHER: Conventional timesharing implemented by the Completely Fair Scheduler (CFS) which is the default scheduler for Android OS.
So the Linux scheduler, based on the scheduling algorithm and the thread priority (among other things) determines the chances each thread has to use the CPU next. In this mix the only factor that a programmer can use to influence how the system treats the threads that his/her application spawns is by changing/adjusting the thread priorities. Not all threads are equally important for the Android developer and the most important of them all is the UI thread.
Android UI thread
We typically follow a construct such as this in Android (simplified a bit for clarity):
@Overrideprotected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); runOnAThread(heavyTask); // rest of code omitted for clarity}
Starting new threads in the background for heavier tasks can lead to competition with the UI thread for usage of the CPU and hence poor responsiveness, which is the opposite of why we delegate work to background threads. Since new threads are spawned from the main thread and the spawned threads inherit the priority of the spawning thread (see previous section), the UI thread and the background thread can compete equally for the CPU time which is not what we had in mind to begin with.
There are two APIs available that allow the changing of priorities. The java.lang.Thread API is associated with the Java thread while the android.os.Process is about the native thread. These two APIs are not in sync, i.e. they can report different information about priorities, ids, etc. Specifically:
- The getPriority() of the Thread API returns the cached/java level priority but the native Process.getThreadPriority() returns the priority of the pthread (not in sync)
- Process.getThreadPriority() actually returns the nice value. It is pthread_priority = 20 + Process.getThreadPriority()
The two APIs are similar and are easy to confuse. A common source of confusion is using the Thread.getId() which is just a sequential number instead of Process.myTid(), which returns the actual pid of the running native thread (that you can see with ps in the system).
When the main application thread starts, it has been set with higher priority (lower nice value) at the linux level, but it has been set as a normal priority thread at the java.lang.Thread level. So the two APIs can return different values on inspection.
This also has the implication that all threads forked from the main thread start as normal priority threads due to the fact that the java thread native API sets the priority based on the Java peer thread priority.
There are various approaches that convenient abstractions we use take on this topic, from leaving it to the client code to configure the priorities if required via system properties (RxJava) or via constructor parameters (HandlerThread) to explicitly setting the priority (AsyncTask) without allowing configuration.
So far, this clears out how the UI thread in our application gets a better chance to run than our other threads as new threads are spawned and degradation due to contention is avoided. This is due to having the UI thread with lower priority/nice value than all forked threads so that the UI thread always has a chance of more CPU time. This is also the reason that a very light background thread we forked is not guaranteed at all to finish earlier than a slighter slower update of the UI.
But is this all we need to know? What happens with other threads of equal priorities currently running but not related to the UI? Which CPUs can the threads use? Does it matter?
Process segmentation and resource allocation
Android uses Control Groups (CGroups) to provide the threading behaviour that developers expect. CGroups is a linux construct, it offers a unified method of managing and restricting processes by setting explicit resource limits for memory, # of CPUs, and specific devices. With CGroups we can also control if a group can use more CPU or I/O than other groups. This will be familiar for anyone who’s worked with Docker since Docker is also a technology that was built based on CGroups.
The essential CGroups for Android are:
SP_BACKGROUND [ “bg” ]SP_FOREGROUND [ “fg” ]SP_TOP_APP [“ta”]
In a nutshell, processes are assigned to groups depending on their lifecycle and interactivity with the user. Each group is configured differently in terms of how much resources and time is available to the group’s associated processes. For example, processes assigned to the background CGroup get about 5% of the total execution time of the device when the system starts to be under contention. Of course if there are available resources (e.g. the system is basically idle) the threads can go above that threshold so as to have the maximum utilization of the system.
We can do a quick check of the placement of groups per CPU (cpuset) as follows:
Check how many available cores
$ adb shell cat /dev/cpuset/cpus0–3 (all available 4 cores)
Which cores are available per group (cores numbered 0–3)?
SP_BACKGROUND$ adb shell cat /dev/cpuset/background/cpus0SP_FOREGROUND$ adb shell cat /dev/cpuset/foreground/cpus0–2SP_TOP_APP$ adb shell cat /dev/cpuset/top-app/cpus0–3
For the particular example we can see that a process running as top-app has all four processors in the system available for it, while only core-0 is available to the background group.
As the user changes between applications, the app moves between cpusets and groups. An app that is currently in use and visible, is assigned to the top-app (“ta”) cpuset and as the user switches to another app or something else, it moves to the background cpuset. This means that all the policies for that newly assigned group are applied.
You can verify it yourself by checking your application using adb.
ps is a linux utility that you can use for this purpose. You can check the exact usage for the version you have with
adb shell ps --helpThe flags of interest are: -t for showing the threads, -p for showing the priority, -P for the scheduling policy and -c for showing the assigned CPU.
You can find you application’s process id by either
adb shell pidof $applicationId or
adb shell ps | grep $applicationIdThen you can check all the threads associated with your application’s PID as follows (the application is in the foreground and you interact with it):
adb shell ps -t -p -P -c $(adb shell pidof com.booking) | head -n 10USER PID PPID VSIZE RSS CPU PRIO NICE RTPRI SCHED PCY WCHAN PC NAMEu0_a335 4830 1767 1687808 287000 1 10 -10 0 0 ta SyS_epoll_ 00000000 S com.bookingu0_a335 4835 4830 1687808 287000 1 29 9 0 0 ta futex_wait 00000000 S Jit thread poolu0_a335 4836 4830 1687808 287000 0 20 0 0 0 ta do_sigtime 00000000 S Signal Catcheru0_a335 4838 4830 1687808 287000 0 20 0 0 0 ta futex_wait 00000000 S ReferenceQueueDu0_a335 4839 4830 1687808 287000 0 20 0 0 0 ta futex_wait 00000000 S FinalizerDaemonu0_a335 4840 4830 1687808 287000 0 20 0 0 0 ta futex_wait 00000000 S FinalizerWatchdu0_a335 4841 4830 1687808 287000 1 20 0 0 0 ta futex_wait 00000000 S HeapTaskDaemonu0_a335 4842 4830 1687808 287000 2 20 0 0 0 ta binder_thr 00000000 S Binder:4830_1u0_a335 4843 4830 1687808 287000 3 20 0 0 0 ta binder_thr 00000000 S Binder:4830_2u0_a335 4847 4830 1687808 287000 1 10 -10 0 0 ta futex_wait 00000000 S Profile Saver
We can see that the threads are assigned to the top-app group (column ta) and are assigned to various CPUs among 0–3. We can also see that each thread is a separate lwp all having the same parent pid which is the process that forked them. Notice how the main application process has lower nice value than the rest that have normal priority.
Another way to poke at the CPUs is:
adb shell cat /proc/$(adb shell pidof com.booking)/status | grep CpusCpus_allowed: f //hexadecimal mask of CPUs on which this process may runCpus_allowed_list: 0–3 // in “list format”
As soon as you put the application in the background and run the same command we see:
adb shell ps -t -p -P -c $(adb shell pidof com.booking) | head -n 10USER PID PPID VSIZE RSS CPU PRIO NICE RTPRI SCHED PCY WCHAN PC NAMEu0_a335 4830 1767 1632412 283908 0 20 0 0 0 bg SyS_epoll_ 00000000 S com.bookingu0_a335 4835 4830 1632412 283908 0 29 9 0 0 bg futex_wait 00000000 S Jit thread poolu0_a335 4836 4830 1632412 283908 0 20 0 0 0 bg do_sigtime 00000000 S Signal Catcheru0_a335 4838 4830 1632412 283908 0 20 0 0 0 bg futex_wait 00000000 S ReferenceQueueDu0_a335 4839 4830 1632412 283908 0 20 0 0 0 bg futex_wait 00000000 S FinalizerDaemonu0_a335 4840 4830 1632412 283908 0 20 0 0 0 bg futex_wait 00000000 S FinalizerWatchdu0_a335 4841 4830 1632412 283908 0 20 0 0 0 bg futex_wait 00000000 S HeapTaskDaemonu0_a335 4842 4830 1632412 283908 0 20 0 0 0 bg binder_thr 00000000 S Binder:4830_1u0_a335 4843 4830 1632412 283908 0 20 0 0 0 bg binder_thr 00000000 S Binder:4830_2u0_a335 4847 4830 1632412 283908 0 10 -10 0 0 bg futex_wait 00000000 S Profile Saver
The threads didn’t just switch to the background (bg) group, they also switched to the CPU #0 (the one assigned specifically for that group).
Additionally the main application thread now changed to a normal priority.
adb shell cat /proc/$(adb shell pidof com.booking)/status | grep CpusCpus_allowed: 1 //hexadecimal mask of CPUs on which this process may runCpus_allowed_list: 0 // in “list format”
Another useful tool to monitor your threads is top with the parameter -H to show threads. We can see that it displays the percent of CPU time used, as well as the priority and nice value:
adb shell top -H | head -n 5User 6%, System 22%, IOW 0%, IRQ 0%User 5 + Nice 0 + Sys 18 + Idle 57 + IOW 0 + IRQ 0 + SIRQ 0 = 80PID TID USER PR NI CPU% S VSS RSS PCY Thread Proc
So by running adb shell top -H | grep $(adb shell pidof package_name) we can see the percent of CPU utilization and priority for each of our threads.
Conclusion
By using specific configurations for control groups, actively monitoring the system and moving processes among the groups, Android, on top of the Linux scheduler, is able to provide the guarantees of performance of the UI thread among all the various other threads running at the same time in the system. The UI thread within our application must have a higher priority than all the other threads in order to have good UI responsiveness. Although this is usually taken care of for us, in the case of using threads ourselves we need to make sure we adjust the priority properly.
I hope that you’ve learned something new and interesting, and now have a better understanding of what’s exactly going on in the system as threads are spawned by your application.
