So, you're using Android UI, and you're curious as to how your code turns into pixels being turned on. Read on and let's go on an adventure!
The basic model for Android UI is to have windows with views draw into surfaces that can be composed onto a display. That all sounds pretty abstract, so let's talk about what the major systems do and what the flow is for rendering, picking some particular paths to keep it simple. There are mutiple points of indirection and seams across the framework to change or override behavior, but I'm going to keep this mostly focused on a 'happy path' rendering on a phone.
When an app is brought into existance, its main Activity is created and initialized, and that's where our story begins.
In Activity.java, the attach
method creates a new PhoneWinindow instance and sets it as its mWindow
field, grabs the current thread as mUiThread
, calls context.getSystemService(Context.WINDOW_SERVICE)
to set the window manager on the window (which creates a WindowManagerImpl, and then takes that wrapper as its mWindowManager
.
OK, so we have an activity wired up to a window. What next?
Unlike a game engine where you might be constantly trying to refresh the display, Android UI is more event-based (although it of course supports things like scrolling and animations where smooth rendering is critical).
In a View, a call to invalidate will cause onDraw
to be called at some point in the future. You might trigger this for example by changing some property of the view, like the background color or the text being displayed.
How is the invalidation handled, when is drawing called, and what happens with that drawing code?
This goes into internalInvalidate
, which can invalidate the cache (the explicit invalidate
call does this), which sets some flags and propagates a 'damage' rectangle area up to its parent with invalidateChild
.
These calls eventually land on the invalidate
method of ViewRootImpl if they do in fact result in visible changes (based on dirty rectangles and other conditions like first display or display state). From here, a call to scheduleTraversals will reach reach out to the Choreographer on the thread and hook into CALLBACK_TRAVERSAL
.
The synchronization of drawing happens via Choreographer
. If you look up the doFrame source, there is per-frame bookkeeping with clocks and jank detection, then a series of callbacks for input, animation (and inset animations), layout/draw traversal and post-traversal ("commit") operations.
The callback for traversals eventually lands on ViewRootImpl
's performTraversals, which is about a 1K lines of code.
Finally, performDraw
circa line 4093! That calls draw
and then drawSoftware
or mAttachInfo.mThreadedRenderer.draw
.
Choreographer receives time pulses from lower layers. Typically, SurfaceFlinger ends up owning the other side of the rendering layer (with a Surface that acts as a swapchain and some metadata) and will pulse it to do work, for example on a display vsync. Surfaces have their release/acquire operations and metadata operations can flow as transactions.
When Choreographer or a dependency like a view or animation see there's work to be done, the call to postVsyncCallback
on Choreographer will be made, which eventually schedules a frame if needed.
SurfaceFlinger can also try and get sophisticated with vsyncs, and might try to turn vsync on for a bit to synchronize its own timer, then turn vsync off and go off that timer if it's steady enough.
When SurfaceFlinger decides it's time to composite a display frame, it looks for any new layers that are ready for presentation, and asks the hardware composer whether it wants to do the composition itself, or whether SurfaceFlinger should compose things for it (or some combination perhaps) and then hand off the finished frame.
The layer metadata along with surface information allows building a fair bit of the sprite/layer functionality we discussed earlier (the AOSP documentation often refers to these as overlays), along with what composers should support, including multiple overlays, larger-than-display overlays, blending, and a variety of other fixes.
You can see the interface that the surface flinger uses to talk to the composer and displays in IComposerClient.aidl.
These are some other components of note that we didn't go into as much detail.
Within the framework:
Happy rendering!