Rendering on Android

So, you're using Android UI, and you're curious as to how your code turns into pixels being turned on. Read on and let's go on an adventure!

Overview

The basic model for Android UI is to have windows with views draw into surfaces that can be composed onto a display. That all sounds pretty abstract, so let's talk about what the major systems do and what the flow is for rendering, picking some particular paths to keep it simple. There are mutiple points of indirection and seams across the framework to change or override behavior, but I'm going to keep this mostly focused on a 'happy path' rendering on a phone.

Initialization

When an app is brought into existance, its main Activity is created and initialized, and that's where our story begins.

In Activity.java, the attach method creates a new PhoneWinindow instance and sets it as its mWindow field, grabs the current thread as mUiThread, calls context.getSystemService(Context.WINDOW_SERVICE) to set the window manager on the window (which creates a WindowManagerImpl, and then takes that wrapper as its mWindowManager.

OK, so we have an activity wired up to a window. What next?

Drawing

Unlike a game engine where you might be constantly trying to refresh the display, Android UI is more event-based (although it of course supports things like scrolling and animations where smooth rendering is critical).

In a View, a call to invalidate will cause onDraw to be called at some point in the future. You might trigger this for example by changing some property of the view, like the background color or the text being displayed.

How is the invalidation handled, when is drawing called, and what happens with that drawing code?

This goes into internalInvalidate, which can invalidate the cache (the explicit invalidate call does this), which sets some flags and propagates a 'damage' rectangle area up to its parent with invalidateChild.

These calls eventually land on the invalidate method of ViewRootImpl if they do in fact result in visible changes (based on dirty rectangles and other conditions like first display or display state). From here, a call to scheduleTraversals will reach reach out to the Choreographer on the thread and hook into CALLBACK_TRAVERSAL.

The synchronization of drawing happens via Choreographer. If you look up the doFrame source, there is per-frame bookkeeping with clocks and jank detection, then a series of callbacks for input, animation (and inset animations), layout/draw traversal and post-traversal ("commit") operations.

The callback for traversals eventually lands on ViewRootImpl's performTraversals, which is about a 1K lines of code.

Finally, performDraw circa line 4093! That calls draw and then drawSoftware or mAttachInfo.mThreadedRenderer.draw.

Lower Layers

Choreographer receives time pulses from lower layers. Typically, SurfaceFlinger ends up owning the other side of the rendering layer (with a Surface that acts as a swapchain and some metadata) and will pulse it to do work, for example on a display vsync. Surfaces have their release/acquire operations and metadata operations can flow as transactions.

When Choreographer or a dependency like a view or animation see there's work to be done, the call to postVsyncCallback on Choreographer will be made, which eventually schedules a frame if needed.

SurfaceFlinger can also try and get sophisticated with vsyncs, and might try to turn vsync on for a bit to synchronize its own timer, then turn vsync off and go off that timer if it's steady enough.

When SurfaceFlinger decides it's time to composite a display frame, it looks for any new layers that are ready for presentation, and asks the hardware composer whether it wants to do the composition itself, or whether SurfaceFlinger should compose things for it (or some combination perhaps) and then hand off the finished frame.

The layer metadata along with surface information allows building a fair bit of the sprite/layer functionality we discussed earlier (the AOSP documentation often refers to these as overlays), along with what composers should support, including multiple overlays, larger-than-display overlays, blending, and a variety of other fixes.

You can see the interface that the surface flinger uses to talk to the composer and displays in IComposerClient.aidl.

Other Components

These are some other components of note that we didn't go into as much detail.

Within the framework:

Resources

Happy rendering!

Tags:  androidgraphics

Home