These are confidential sessionsplease refrain from streaming, blogging, or taking pictures

Advances in OpenGL ES 3.0

Filip Iliescu
Graphics and Media Evangelist
filiescu@apple.com

These are confidential sessionsplease refrain from streaming, blogging, or taking pictures

Apple A7 Processor

Apple A7 Processor

OpenGL ES 3.0

Apple A7 Processor

OpenGL ES 3.0

Xcode 5

Agenda
Highlights of the Apple A7 GPU
Moving from ES2 to ES3
Deeper dive into ES3
Tuning using Xcode 5 OpenGL ES Debugger

New Shader Profiler for A7 GPU

Apple A7 GPU

Overview
Tile Based Deferred Renderer (TBDR)
Up to 2x graphics performance

Compared to A6 (iphone 5)

Fully native OpenGL ES 3.0, 2.0

Shader-based pipeline
ES1.1 backwards compatibility

2D Texture arrays
Program Performance View
Deferred Lighting / Shading

Render to Texture

More Texture Units

Instanced Rendering

Sampler Objects
GLSL ES 1.00

Vertex Array Objects

Multiple Render Targets

Primitive Restart

Immutable Texture Storage

Uniform Buffer Objects

sRGB Textures

PVRTC
ETC2/EAC

Vertex Texture Fetch

1 & 2 component textures
Per-Line Shader Performance Metrics

Render to Mipmap level

Half-Float Color Buffers

Fast texture copy

Seamless cubemap filtering

GLSL ES 3.00
Vertex Buffer Objects
Extended Indices

Separate Shader Objects

Framebuffer_Fetch

Map Buffer Range

MSAA Render to Texture

Occlusion Query

Non Power of Two Textures

3D Textures

Transform feedback

MSAA Render to Texture

Pixel Buffer Objects

OpenGL ES Limits
Apple A7

SGX 554

Apple A7 with
ES2 context

Max Texture Image Units

16

Max Combined Texture Image Units

32

Max Vertex Texture Image Units

16

Max Vertex Uniform Vectors

512

128

128

Max Fragment Uniform Vectors

224

64

64

Max Varying Vectors

15

Max Color Attachments

NA

NA

4096 x 4096

4096 x 4096

4096 x 4096

OpenGL ES Attributes

Max Texture & Renderbuffer Size

Key Differences From A6

Performance
No penalty for dependent texture reads
Higher penalty for frame buffer loads and stores

Precision
lowp shader values promoted to 16-bit (mediump)
All FP shader calculations performed with scalar processor

Limits
Apps with ES2 context get ES2 limits
Apps with ES3 context get ES3 limits

Moving from ES2 to ES3

The Big Picture

Core
ES2 Compatible with ES3 API
ES2 is Subset of ES3

Extensions (3 cases)
Some have moved into the ES3 core as-is
Some move into ES3 core with semantic changes
Some extensions in ES2 are still extensions in ES3

Case #1
ES2 extensions promoted directly to ES3 core
These work identically in ES2 and ES3
Just remove EXT, APPLE, OES API suffixes
OES_depth24
OES_element_index_uint
OES_fbo_render_mipmap
OES_rgb8_rgba8
OES_texture_half_float_linear
OES_vertex_array_object
EXT_blend_minmax

EXT_draw_instanced
EXT_instanced_arrays
EXT_map_buffer_range
EXT_occlusion_query_boolean
EXT_texture_storage
APPLE_sync
APPLE_texture_max_level

Case #1 Examples
EXT_texture_storage
ES2: glTexStorage2DEXT(GL_TEXTURE_2D, 1, GL_RGBA8_OES, width, height);
ES3: glTexStorage2D
(GL_TEXTURE_2D, 1, GL_RGBA8,
width, height);
!

EXT_map_buffer_range
ES2: glMapBufferRangeEXT(GL_ARRAY_BUFFER, offset, length, GL_MAP_WRITE_BIT_EXT |
GL_MAP_FLUSH_EXPLICIT_BIT_EXT | GL_MAP_UNSYNCHRONIZED_BIT_EXT );
ES3: glMapBufferRange
(GL_ARRAY_BUFFER, offset, length, GL_MAP_WRITE_BIT
|
GL_MAP_FLUSH_EXPLICIT_BIT
| GL_MAP_UNSYNCHRONIZED_BIT
);

Case #2
ES2 extensions promoted with API changes
OES_mapbuffer
EXT_discard_framebuffer
APPLE_framebuffer_multisample
OES_depth_texture
OES_packed_depth_stencil
OES_texture_float
OES_texture_half_float
EXT_texture_rg
EXT_sRGB

Case #2 Examples
OES_mapbuffer
ES2: map =glMapBufferOES (GL_ARRAY_BUFFER,GL_WRITE_ONLY_OES);
ES3: map =glMapBufferRange(GL_ARRAY_BUFFER, 0, size, GL_MAP_WRITE_BIT);
!

EXT_discard_framebuffer
ES2: glDiscardFramebufferEXT(GL_FRAMEBUFFER, count, attachments);
ES3: glInvalidateFramebuffer(GL_FRAMEBUFFER, count, attachments);
!

APPLE_framebuffer_multisample
ES2: glResolveMultisampleFramebufferAPPLE();
ES3: glBlitFramebuffer(0,0,w,h, 0,0,w,h, GL_COLOR_BUFFER_BIT, GL_NEAREST);

Case #3
Some extensions in ES2 are still extensions in ES3
Check GL_EXTENSIONS
APPLE_copy_texture_levels
APPLE_rgb_422
APPLE_texture_format_BGRA_8888
EXT_color_buffer_half_float
EXT_debug_label
EXT_debug_marker

!
!
!

EXT_read_format_bgra
EXT_separate_shader_objects
EXT_shader_framebuffer_fetch
EXT_texture_filter_anisotropic
IMG_read_format
IMG_texture_compression_pvrtc

GLSL ES
#version 100 es
Fully supported
ES2-style shaders are compatible with both ES2 and ES3
Version 100 assumed if no #version specified

#version 300 es
Many language additions and changes
Similar to desktop GLSL 330

Video: Migrating to OpenGL Core Profile

Adoption Strategy
Now
Test your ES2 based games on iPhone 5s
Especially, correct any logical buffer loads/stores

Next
Support both ES2 and ES3
Try for an ES3 context, fall back to ES2 if not available
Handle extension APIs conditionally at runtime

Some games: Go ES3 only

Games requiring ES3 features, deferred shading, etc.

Deeper into OpenGL ES 3.0

2D Texture arrays
Program Performance View
Deferred Lighting / Shading

Render to Texture

3D Textures

Instanced Rendering

Sampler Objects
GLSL ES 1.00

Vertex Array Objects

Multiple Render Targets

Primitive Restart

Immutable Texture Storage

Uniform Buffer Objects

sRGB Textures

PVRTC
ETC2/EAC Compressed Textures

New Buffer Formats

1 & 2 component textures
Per-Line Shader Performance Metrics

Render to Mipmap level

Half-Float Color Buffers

Fast texture copy

Seamless cubemap filtering

GLSL ES 3.00
Vertex Buffer Objects
Extended Indices

Separate Shader Objects

Framebuffer fetch

Map Buffer Range

More Texture Units

Occlusion Query

Non Power of Two Textures

NPOT textures

Transform feedback

MSAA Render to Texture

Pixel Buffer Objects

2D Texture arrays
Program Performance View
Deferred Lighting / Shading

Render to Texture

3D Textures

Instanced Rendering

Sampler Objects
GLSL ES 1.00

Vertex Array Objects

Multiple Render Targets

Primitive Restart

Immutable Texture Storage

Uniform Buffer Objects

sRGB Textures

PVRTC
ETC2/EAC Compressed Textures

New Buffer Formats

1 & 2 component textures
Per-Line Shader Performance Metrics

Render to Mipmap level

Half-Float Color Buffers

Fast texture copy

Seamless cubemap filtering

GLSL ES 3.00
Vertex Buffer Objects
Extended Indices

Separate Shader Objects

Framebuffer fetch

Map Buffer Range

More Texture Units

Occlusion Query

Non Power of Two Textures

NPOT textures

Transform feedback

MSAA Render to Texture

Pixel Buffer Objects

Instanced Rendering

Drawing Many (Similar) Objects

Drawing Many Objects

Without instancing
// Draw stars and planet
[self drawStarsAndPlanet];
!

// Draw asteroids
for (x=0; x < NUM_ASTEROIDS; x++)
{
// Set asteroid position, rotation, etc.
glUniformMatrix4fv(asteroidParameters[x]);
!

// Draw one asteroid

glDrawArrays(GL_TRIANGLES, 0, asteroidVertexCount);
}

Instanced Rendering
Faster way to draw many similar objects
Draws the same object many times

All in a single draw call

Each can have different parameters

Positions
Rotations
Texture coordinates
etc.

Instanced Rendering
Two forms
Instanced arrays

All instance parameters stored in an attribute array

Shader instance ID

Instance parameters derived from gl_InstanceID in vertex shader

Both available on all iOS 7 devices

ES3: In the ES3 core
ES2: GL_APPLE_instanced_arrays,

GL_APPLE_draw_instanced

Instance ID
Vertex shader
gl_InstanceID incremented for each instance

0, 1, 2, 3, ... n

You take it from there

Use ID as input to calculation in shader
Use ID for lookup in Uniform Buffer Object (UBO)
Use ID for lookup with Vertex Texture Sampling

Instance ID
Basic example
// Vertex attributes for
glVertexAttribPointer(0,
glVertexAttribPointer(1,
glVertexAttribPointer(2,

one asteroid
..., vertices);
..., normals);
..., colors);

// Uniforms for all

glUniformMatrix4fv(modelViewProjectionMatrix);
!

// All in one draw call

glDrawArraysInstanced(GL_TRIANGLES, 0, NUM_VERTICES, NUM_ASTEROIDS);

Instance ID
Vertex shader - basic example
#version 300 es
in vec4 inPos;
in vec3 inNorm, inColor;
uniform float spacing;
uniform mat4 cameraMVP;
...
void main()
{
vec4 pos = inPos;
!

ivec2 instancePosition = ivec2(gl_InstanceID % 100, gl_InstanceID / 100);

pos.xy += vec2(instancePosition) * spacing;
!

gl_Position = cameraMVP * pos;

}

Demo
Instanced Asteroids

Instanced Rendering
Demo is doing much more
Instance ID
Used to lookup matrix in UBO
Used as seed for spin rate per-asteroid

Uniform Buffer Object

Holds transformation, color data for each instance
Limited size

Transform feedback & rasterize discard

Vertex stage only
Used to populate the UBO at startup with model view matrix, etc...
reduces per vertex calculations to per instance

Multiple Render Targets

Multiple Render Targets

Concept
Render to multiple textures or
renderbuffers from a single
draw call
4 outputs from fragment
shader
Quadruple the channels
Enables deferred lighting/
shading, other effects
Each attachments format can
differ from each other
128 bits per pixel

Fragment
Shader

Deferred Shading
Using Multiple Render Targets

Fragment
Shader

+ Lighting

Geometry Stage

Lighting Stage

Deferred Shading
Using Multiple Render Targets

Fragment
Shader

+ Lighting

Geometry Stage

Lighting Stage

Multiple Render Targets

Setup
// Define 4 color attachments for currently bound framebuffer
GLenum renderbuffers[] = { GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1,
GL_COLOR_ATTACHMENT2, GL_COLOR_ATTACHMENT3 };
!

// Attach textures as output buffers

glFramebufferTexture2D(GL_FRAMEBUFFER,
glFramebufferTexture2D(GL_FRAMEBUFFER,
glFramebufferTexture2D(GL_FRAMEBUFFER,
glFramebufferTexture2D(GL_FRAMEBUFFER,

GL_COLOR_ATTACHMENT0,
GL_COLOR_ATTACHMENT1,
GL_COLOR_ATTACHMENT2,
GL_COLOR_ATTACHMENT3,

// Tell GL to enable buffers to draw into

glDrawBuffers(4, renderbuffers);
!

// Draw
glDrawElements(...);

colorTex, 0);
normalTex, 0);
depthTex, 0);
albedoTex, 0);

Multiple Render Targets

Fragment shader
#version 300 es
!

layout(location
layout(location
layout(location
layout(location

=
=
=
=

0)
1)
2)
3)

out
out
out
out

lowp
lowp
highp
lowp

void main(void)
{
fs_color_output = ...
fs_normal_output = ...
fs_depth_output = ...
fs_albedo_output = ...
}

vec4
vec4
uint
vec4

fs_color_output;
fs_normal_output;
fs_depth_output;
fs_albedo_output;

Multiple Render Targets

For Deferred Shading

Framebuffer Fetch

Framebuffer Fetch
EXT_shader_framebuffer_fetch
Provides current destination color in fragment shader
Syntax

Built-in variable in #version 100 shaders

gl_LastFragData[0]

User-declared in #version 300 es

layout(location = 0) inout lowp vec4 my_destination_name;

Useful for
Programmable blending
Local post-processing effects
Fetching non-color framebuffer data

Framebuffer Fetch
With Multiple Render Targets
Read and write multiple attachments
Read with framebuffer fetch
Write with MRT

Read from one, write to another

All from the same shader

Fragment
Shader

Deferred Shading in One Pass

Using Multiple Render Targets and Framebuffer Fetch

Output
G-buffers

Compute
Lighting

Clean up
and Present

Deferred Shading in One Pass

Using Multiple Render Targets and Framebuffer Fetch

Output
G-buffers

Compute
Lighting

Clean up
and Present

Deferred Shading in One Pass

Using Multiple Render Targets and Framebuffer Fetch

Output
G-buffers

Compute
(and Output)
Lighting

Clean up
and Present

Deferred Shading in One Pass

Using Multiple Render Targets and Framebuffer Fetch

Output
G-buffers

Compute
(and Output)
Lighting

Clean up
and Present

Deferred Shading in One Pass

Using Multiple Render Targets and Framebuffer Fetch

Deferred Shading on A7
Three stages
Multiple Render Targets
Render G-buffer attachments in one pass
formats can vary between attachments

Framebuffer fetch
Render deferred lights in the same pass
Read from all attachments, write to one

Gbuffer becomes per-pixel scratch space

Framebuffer invalidate

To avoid logical buffer stores

OpenGL ES Tools

Demo
Xcode 5 OpenGL ES Debugger

Apple A7 Processor

OpenGL ES 3.0

OpenGL ES Debugger

More Information
Filip Iliescu
Graphics and Media Evangelist
filiescu@apple.com
!

Apple Developer Forums

http://devforums.apple.com/
!

Developer Documentation
http://developer.apple.com/opengles/
!

Migrating to OpenGL Core Profile video

http://developer.apple.com/opengl/