Beruflich Dokumente
Kultur Dokumente
This post attempts to be a brief and simple introduction to the Linux graphics stack, and as
such, it has an introductory nature. I will focus on giving enough context to understand the role
that Mesa and 3D drivers in general play in the stack and leave it to follow up posts to dive
deeper into the guts of Mesa in general and the Intel DRI driver specifically.
A bit of history
In order to understand some of the particularities of the current graphics stack it is important
to understand how it had to adapt to new challenges throughout the years.
You see, nowadays things are significantly more complex than they used to be, but in the early
times there was only a single piece of software that had direct access to the graphics hardware:
the X server. This approach made the graphics stack simpler because it didn’t need to
synchronize access to the graphics hardware between multiple clients.
In these early days applications would do all their drawing indirectly, through the X server. By
using Xlib they would send rendering commands over the X11 protocol that the X server would
receive, process and translate to actual hardware commands on the other side of a socket.
Notice that this “translation” is the job of a driver: it takes a bunch of hardware agnostic
rendering commands as its input and translates them into hardware commands as expected by
the targeted GPU.
Since the X server was the only piece of software that could talk to the graphics hardware by
design, these drivers were written specifically for it, became modules of the X server itself and
an integral part of its architecture. These userspace drivers are called DDX drivers in X server
argot and their role in the graphics stack is to support 2D operations as exported by Xlib and
required by the X server implementation.
3D graphics
The above covers 2D graphics as that is what the X server used to be all about. However, the
arrival of 3D graphics hardware changed the scenario significantly, as we will see now.
Another important part of DRI is the Direct Rendering Manager (DRM). This is the kernel side of
the DRI architecture. Here, the kernel handles sensitive aspects like hardware locking, access
synchronization, video memory and more. DRM also provides userspace with an API that it can
use to submit commands and data in a format that is adequate for modern GPUs, which
effectively allows userspace to communicate with the graphics hardware.
Notice that many of these things have to be done specifically for the target hardware so there
are different DRM drivers for each GPU. In my Ubuntu system the DRM module for my Intel
GPU is provided via the libdrm-intel1:amd64 package.
Enter Mesa
Mesa is a free software implementation of the OpenGL specification, and as such, it provides
a libGL.so, which OpenGL based programs can use to output 3D graphics in Linux. Mesa can
provide accelerated 3D graphics by taking advantage of the DRI architecture to gain direct
access to the underlying graphics hardware in its implementation of the OpenGL API.
When our 3D application runs in an X11 environment it will output its graphics to a surface
(window) allocated by the X server. Notice, however, that with DRI this will happen without
intervention of the X server, so naturally there is some synchronization to do between the two,
since the X server still owns the window Mesa is rendering to and is the one in charge of
displaying its contents on the screen. This synchronization between the OpenGL application and
the X server is part of DRI. Mesa’s implementation of GLX (the extension of the OpenGL
specification that addresses the X11 platform) uses DRI to talk to the X server and accomplish
this.
Mesa also has to use DRM for many things. Communication with the graphics hardware
happens by sending commands (for example “draw a triangle”) and data (for example the
vertex coordinates of the triangle, their color attributes, normals, etc). This process usually
involves allocating a bunch of buffers in the graphics hardware where all these commands and
data are copied so that the GPU can access them and do its work. This is enabled by the DRM
driver, which is the one piece that takes care of managing video memory and which offers APIs
to userspace (Mesa in this case) to do this for the specific target hardware. DRM is also required
whenever we need to allocate and manage video memory in Mesa, so things like creating
textures, uploading data to textures, allocating color, depth or stencil buffers, etc all require to
use the DRM APIs for the target hardware.
Here, hierarchical means that each upper-layer protocol is supported by two or more lower-
level protocols.
Internet Layer
IP Protocol: IP protocol is used in this layer, and it is the most significant part of the entire
TCP/IP suite.
ARP Protocol
ICMP Protocol
Transport Layer
The transport layer is responsible for the reliability, flow control, and correction of data which is
being sent over the network.
The two protocols used in the transport layer are User Datagram protocol and Transmission
control protocol.
Application Layer
o HTTP: HTTP stands for Hypertext transfer protocol. This protocol allows us to access the
data over the world wide web. It transfers the data in the form of plain text, audio,
video. It is known as a Hypertext transfer protocol as it has the efficiency to use in a
hypertext environment where there are rapid jumps from one document to another.
o SNMP: SNMP stands for Simple Network Management Protocol. It is a framework used
for managing the devices on the internet by using the TCP/IP protocol suite.
o SMTP: SMTP stands for Simple mail transfer protocol. The TCP/IP protocol that supports
the e-mail is known as a Simple mail transfer protocol. This protocol is used to send the
data to another e-mail address.
o DNS: DNS stands for Domain Name System. An IP address is used to identify the
connection of a host to the internet uniquely. But, people prefer to use the names
instead of addresses. Therefore, the system that maps the name to the address is
known as Domain Name System.
o TELNET: It is an abbreviation for Terminal Network. It establishes the connection
between the local computer and remote computer in such a way that the local terminal
appears to be a terminal at the remote system.
o FTP: FTP stands for File Transfer Protocol. FTP is a standard internet protocol used for
transmitting the files from one computer to another computer.
WI-FI and 802.11
The 802.11 standard is defined through several specifications of WLANs. It defines an over-the-
air interface between a wireless client and a base station or between two wireless clients.
There are several specifications in the 802.11 family −
• 802.11 − This pertains to wireless LANs and provides 1 - or 2-Mbps transmission in the
2.4-GHz band using either frequency-hopping spread spectrum (FHSS) or direct-
sequence spread spectrum (DSSS).
• 802.11a − This is an extension to 802.11 that pertains to wireless LANs and goes as fast
as 54 Mbps in the 5-GHz band. 802.11a employs the orthogonal frequency division
multiplexing (OFDM) encoding scheme as opposed to either FHSS or DSSS.
• 802.11b − The 802.11 high rate WiFi is an extension to 802.11 that pertains to wireless
LANs and yields a connection as fast as 11 Mbps transmission (with a fallback to 5.5, 2,
and 1 Mbps depending on strength of signal) in the 2.4-GHz band. The 802.11b
specification uses only DSSS. Note that 802.11b was actually an amendment to the
original 802.11 standard added in 1999 to permit wireless functionality to be analogous
to hard-wired Ethernet connections.
• 802.11g − This pertains to wireless LANs and provides 20+ Mbps in the 2.4-GHz band.
Here is the technical comparison between the three major WiFi standards.
Basic frame format which is required for all MAC implementation is defined in IEEE 802.3
standard. Though several optional formats are being used to extend the protocol’s basic
capability.
Ethernet frame starts with Preamble and SFD, both works at the physical layer. Ethernet header
contains both Source and Destination MAC address, after which the payload of the frame is
present. The last field is CRC which is used to detect the error. Now, let’s study each field of
basic frame format.
Ethernet (IEEE 802.3) Frame Format
• PREAMBLE – Ethernet frame starts with 7-Bytes Preamble. This is a pattern of alternative
0’s and 1’s which indicates starting of the frame and allow sender and receiver to
establish bit synchronization. Initially, PRE (Preamble) was introduced to allow for the loss
of a few bits due to signal delays. But today’s high-speed Ethernet don’t need Preamble to
protect the frame bits.
PRE (Preamble) indicates the receiver that frame is coming and allow the receiver to lock
onto the data stream before the actual frame begins.
• Start of frame delimiter (SFD) – This is a 1-Byte field which is always set to 10101011. SFD
indicates that upcoming bits are starting of the frame, which is the destination address.
Sometimes SFD is considered the part of PRE, this is the reason Preamble is described as 8
Bytes in many places. The SFD warns station or stations that this is the last chance for
synchronization.
• Destination Address – This is 6-Byte field which contains the MAC address of machine for
which data is destined.
• Source Address – This is a 6-Byte field which contains the MAC address of source
machine. As Source Address is always an individual address (Unicast), the least significant
bit of first byte is always 0.
• Length – Length is a 2-Byte field, which indicates the length of entire Ethernet frame. This
16-bit field can hold the length value between 0 to 65534, but length cannot be larger
than 1500 because of some own limitations of Ethernet.
• Data – This is the place where actual data is inserted, also known as Payload. Both IP
header and data will be inserted here if Internet Protocol is used over Ethernet. The
maximum data present may be as long as 1500 Bytes. In case data length is less than
minimum length i.e. 46 bytes, then padding 0’s is added to meet the minimum possible
length.
• Cyclic Redundancy Check (CRC) – CRC is 4 Byte field. This field contains a 32-bits hash
code of data, which is generated over the Destination Address, Source Address, Length,
and Data field. If the checksum computed by destination is not the same as sent checksum
value, data received is corrupted.
Note – Size of frame of Ethernet IEEE 802.3 varies 64 bytes to 1518 bytes including data length
(46 to 1500 bytes).
Standard IEEE 802.3 basic frame format is discussed above in detail. Now let’s see the extended
Ethernet frame header, using which we can get Payload even larger than 1500 Bytes.
Bluetooth has several protocol layers which form the Blue‐tooth protocol stack.
Controller and host stacks are the two main stacks in Bluetooth, where the controller stack
contains the radio interface part, and the host stack deals with the high level data.
Controller stack functionality includes (not limited to): CRC generation and verification, AES
encryption and preamble, access address, and air protocol framing.
If the controller stack is separated physically then an HCI layer (Host Controller Interface) will
handle the communication between the host stack and controller stack. Mostly, it will be using
USB or UART (using AT commands).
You can find in the market 2 types of Bluetooth solutions: 1- SoC: A single IC runs the
application, the host, and the controller.
2- Dual IC over HCI: One IC runs the application and the host and interface using HCI with a
second IC contains the controller (physical layer).
Bluetooth Protocol
Stack. Image courtesy of THE BLUETOOTH REVOLUTION-Michael Trieu
The well known nrf51822 IC is an example of SoC type, and HC-05 is an example of dual IC over
HCI type.
finally, several Bluetooth stacks are there: BlueZ for Linux, BlueDroid for Android and Apple has
its own one.
Android System Architecture
The Android software stack generally consists of a Linux kernel and a collection of C/C++
libraries that is exposed through an application framework that provides services, and
management of the applications and run time.
Linux Kernel
Android was created on the open source kernel of Linux. One main reason for choosing this
kernel was that it provided proven core features on which to develop the Android operating
system. The features of Linux kernel are:
1. Security:
The Linux kernel handles the security between the application and the system.
2. Memory Management:
It efficiently handles the memory management thereby providing the freedom to develop
our apps.
3. Process Management:
It manages the process well, allocates resources to processes whenever they need them.
4. Network Stack:
5. Driver Model:
It ensures that the application works. Hardware manufacturers can build their drivers into
the Linux build.
Libraries:
Running on the top of the kernel, the Android framework was developed with various features.
It consists of various C/C++ core libraries with numerous of open source tools. Some of these
are:
1. The Android runtime:
The Android runtime consist of core libraries of Java and ART(the Android RunTime). Older
versions of Android (4.x and earlier) had Dalvik runtime.
3. WebKit:
This open source web browser engine provides all the functionality to display web content
and to simplify page loading.
4. Media frameworks:
These libraries allow you to play and record audio and video.
Android Runtime:
It is the third section of the architecture. It provides one of the key components which is called
Dalvik Virtual Machine. It acts like Java Virtual Machine which is designed specially for Android.
Android uses it’s own custom VM designed to ensure that multiple instances run efficiently on a
single device.
The Delvik VM uses the device’s underlying Linux kernel to handle low-level
functionality,including security,
threading and memory management.
Application Framework
The Android team has built on a known set proven libraries, built in the background, and all of it
these is exposed through Android interfaces. These interfaces warp up all the various libraries
and make them useful for the Developer. They don’t have to build any of the functionality
provided by the android. Some of these interfaces include:
1. Activity Manager:
2. Telephony Manager:
3. View System:
4. Location manager:
Android is an open source, Linux-based software stack created for a wide array of devices and
form factors. The following diagram shows the major components of the Android platform.
Figure 1. The Android software stack.
The Linux Kernel
The foundation of the Android platform is the Linux kernel. For example, the Android Runtime
(ART) relies on the Linux kernel for underlying functionalities such as threading and low-level
memory management.
Using a Linux kernel allows Android to take advantage of key security features and allows
device manufacturers to develop hardware drivers for a well-known kernel.
The hardware abstraction layer (HAL) provides standard interfaces that expose device hardware
capabilities to the higher-level Java API framework. The HAL consists of multiple library
modules, each of which implements an interface for a specific type of hardware component,
such as the camera or bluetooth module. When a framework API makes a call to access device
hardware, the Android system loads the library module for that hardware component.
Android Runtime
For devices running Android version 5.0 (API level 21) or higher, each app runs in its own
process and with its own instance of the Android Runtime (ART). ART is written to run multiple
virtual machines on low-memory devices by executing DEX files, a bytecode format designed
specially for Android that's optimized for minimal memory footprint. Build toolchains, such
as Jack, compile Java sources into DEX bytecode, which can run on the Android platform.
Prior to Android version 5.0 (API level 21), Dalvik was the Android runtime. If your app runs well
on ART, then it should work on Dalvik as well, but the reverse may not be true.
Android also includes a set of core runtime libraries that provide most of the functionality of
the Java programming language, including some Java 8 language features, that the Java API
framework uses.
Many core Android system components and services, such as ART and HAL, are built from
native code that require native libraries written in C and C++. The Android platform provides
Java framework APIs to expose the functionality of some of these native libraries to apps. For
example, you can access OpenGL ES through the Android framework’s Java OpenGL API to add
support for drawing and manipulating 2D and 3D graphics in your app.
If you are developing an app that requires C or C++ code, you can use the Android NDK to
access some of these native platform libraries directly from your native code.
The entire feature-set of the Android OS is available to you through APIs written in the Java
language. These APIs form the building blocks you need to create Android apps by simplifying
the reuse of core, modular system components and services, which include the following:
• A rich and extensible View System you can use to build an app’s UI, including lists, grids, text
boxes, buttons, and even an embeddable web browser
• A Resource Manager, providing access to non-code resources such as localized strings, graphics,
and layout files
• A Notification Manager that enables all apps to display custom alerts in the status bar
• An Activity Manager that manages the lifecycle of apps and provides a common navigation back
stack
• Content Providers that enable apps to access data from other apps, such as the Contacts app,
or to share their own data
Developers have full access to the same framework APIs that Android system apps use.
System Apps
Android comes with a set of core apps for email, SMS messaging, calendars, internet browsing,
contacts, and more. Apps included with the platform have no special status among the apps the
user chooses to install. So a third-party app can become the user's default web browser, SMS
messenger, or even the default keyboard (some exceptions apply, such as the system's Settings
app).
The system apps function both as apps for users and to provide key capabilities that developers
can access from their own app. For example, if your app would like to deliver an SMS message,
you don't need to build that functionality yourself—you can instead invoke whichever SMS app
is already installed to deliver a message to the recipient you specify.