PARALLEL - LINQ

Multi-Core and .NET 4

In the words of developers

> Getting an hour-long computation done in 10

minutes changes how we work.

> .NET 4 has made it practical and cost-effective to

implement parallelism where it may have been

hard to justify in the past.

> I do believe the .NET Framework 4 will change

the way developers think about parallel

programming.

> Why Parral Framework(PFX)

>

In recent times, CPU clock speeds have stagnated and manufacturers have
shifted their focus to increasing core counts. This is problematic for us as
programmers because our standard single-threaded code will not
automatically run faster as a result of those extra cores.

>

Leveraging multiple cores is easy for most server applications, where each
thread can independently handle a separate client request, but is harder on
the desktop because it typically requires that you take your computationally
intensive code and do the following:

>
>
>

Partition it into small chunks.

Execute those chunks in parallel via multithreading.
Collate the results as they become available, in a thread-safe and performant
manner.

PFX Concepts
>

There are two strategies for partitioning work among threads: data parallelism and task parallelism.

PLINQ
>

Parallel LINQ (or PLINQ, as it is called) offers all benefits of LINQ. In addition to that, it enables parallel
execution of LINQ queries to take advantages of multiple processors of the host machine.

>
>
>
>

AsParallel
The AsParallel method is the doorway to PLINQ. It converts data sequence into a ParallelQuery. The
LINQ engine detects the use of a ParallelQuery as the source in a query and switches to PLINQ
execution automatically. You are likely to use the AsParallel method every time you use PLINQ.

>
>
>
>
>
>
>
>

ArrayList list = new ArrayList() {

"Adams", "Arthur", "Buchanan", "Bush", "Carter", "Cleveland",
"Clinton", "Coolidge", "Eisenhower", "Fillmore", "Ford", "Garfield",
"Grant", "Harding", "Harrison", "Hayes", "Hoover", "Jackson"};
IEnumerable<string> results = list.AsParallel().Cast<string>().Where(p => p.Contains('o'))
.Select(p => p);
foreach (string president in results) {
Console.WriteLine("Match: {0}", president);}

>
>

.
IEnumerable<string> ss = list.AsParallel().AsOrdered().Cast<string>().Where(p =>
p.StartsWith("A")).Select(p => p);

>
>
>
>

foreach (string president in ss)

{
DropDownList1.Items.Add(president);
}

ParallelQuery<T> by Filtering a ParallelQuery

>
>
>
>
>
>
>
>
>
>
>
>
>
>

ArrayList list = new ArrayList();

list.Add("Adams");
list.Add(23);
list.Add("Arthur");
list.Add(DateTime.Now);
list.Add("Buchanan");
list.Add(new string[] { "apple", "orange" });
IEnumerable<string> results = list
.AsParallel()
.OfType<string>()
.Select(p => p);
foreach (string president in results) {
Console.WriteLine("Match: {0}", president);
}

Visual Studio 2010

Tools, programming models and runtimes

Visual
Studio
IDE

Concurrency
Visualizer

Parallel
Parallel LINQ
LINQ
Task Parallel
Library

.NET Framework 4

Concurrency Runtime

ThreadPool
Task Scheduler

Data
Data Structures
Structures

Parallel
Debugger
Tool
Windows

Programming Models

Parallel
Pattern
Library

Data
Data Structures
Structures

Tools

Visual C++ 10
Task Scheduler
Resource Manager

Resource Manager
Operating
System

User Mode Scheduling

Threads
User Mode Scheduling

Key:

Managed
Managed

Agents
Library

Native
Native

Tooling
Tooling

UMS Threads

From LINQ to Objects to PLINQ

An easy change

> LINQ to Objects query:

int[]
int[] output
output == arr
arr
.Select(x
.Select(x =>
=> Foo(x))
Foo(x))
.ToArray();
.ToArray();

> PLINQ query:

int[]
int[] output
output == arr.AsParallel()
arr.AsParallel()
.Select(x
.Select(x =>
=> Foo(x))
Foo(x))
.ToArray();
.ToArray();

Array Mapping
int[]
int[] input
input == ...
...
bool[]
bool[] output
output == input.AsParallel()
input.AsParallel()
.Select(x
.Select(x =>
=> IsPrime(x))
IsPrime(x))
.ToArray();
.ToArray();
input:

1
6
3
8

output:
Select
Select

Thread 1
Select
Select

Thread 2

F
F
T
F

Select
Select

Thread N

Array to array mapping

is simple and efficient.

Sequence Mapping
IEnumerable<int>
IEnumerable<int> input
input == Enumerable.Range(1,100);
Enumerable.Range(1,100);
bool[]
bool[] output
output == input.AsParallel()
input.AsParallel()
.Select(x
.Select(x =>
=> IsPrime(x))
IsPrime(x))
.ToArray();
.ToArray();

Input
Input
Enumerator
Enumerator

Loc
k

Each thread
processes a partition
of inputs and stores
results into a buffer.

Select
Select

Thread 1

Select
Select

Thread 2

Results
1

Results
2

...
Select
Select

Thread N

Results
N

Buffers are combined

into one array.
output:

Asynchronous Mapping

var
var qq == input.AsParallel()
input.AsParallel()
.Select(x
.Select(x =>
=> IsPrime(x));
IsPrime(x));
foreach(var
foreach(var xx in
in q)
q) {{ ...
... }}
Select
Select

Thread 1

Input
Input
Enumerator
Enumerator

Loc
k

In this query, the foreach

loop starts consuming
results as they are getting
computed.

Select
Select

Thread 2

Results
1

Thread N

Output
Output
Enumerator
Enumerator
MoveNext
MoveNext

Results
2

foreach
foreach

Main
Thread

...
Select
Select

Poll
Poll

Results
N

Async Ordered Mapping or Filter

var
var qq == input.AsParallel().AsOrdered()
input.AsParallel().AsOrdered()
.Select(x
.Select(x =>
=> IsPrime(x));
IsPrime(x));
foreach(var
foreach(var xx in
in q)
q) {{ ...
... }}
Op
Op

Thread 1

Input
Input
Enumerator
Enumerator

Loc
k

When ordering is turned

on, PLINQ orders elements
in a reordering buffer
before yielding them to
the foreach loop.

Op
Op

Thread 2

Results
1

Results
2

Poll
Poll

Output
Output
Enumerator
Enumerator

Ordering
Buffer

MoveNext
MoveNext

...
foreach
foreach
Op
Op

Thread N

Results
N

Main
Thread

Aggregation
int
int result
result == input.AsParallel()
input.AsParallel()
.Aggregate(
.Aggregate(
0,
0,
(a,
(a, e)
e) =>
=> aa ++ Foo(e),
Foo(e),
(a1,a2)
(a1,a2) =>
=> a1
a1 ++ a2);
a2);

res1:
Input
Input
Enumerator
Enumerator

Loc
k

Each thread
computes a local
result.

Aggregate
Aggregate

Thread 1

Aggregate
Aggregate

res2:

result:

Thread 2

...
Aggregate
Aggregate

Thread N

resN:

The local results

are combined
into a final result.

Search
int
int result
result ==
input.AsParallel().AsOrdered()
input.AsParallel().AsOrdered()
.Where(x
.Where(x =>
=> IsPrime(x))
IsPrime(x))
.First();
.First();
First
First

Thread 1

Input
Input
Enumerator
Enumerator

Loc
k

First
First

Thread 2

resultFound:
Poll
Poll

...
First
First

Thread N

Set
Set

result:

More complex query

int[]
int[] output
output == input.AsParallel()
input.AsParallel()
.Where(x
.Where(x =>
=> IsPrime(x))
IsPrime(x))
.GroupBy(x
.GroupBy(x =>
=> xx %% 5)
5)
.Select(g
.Select(g =>
=> ProcessGroup(g))
ProcessGroup(g))
.ToArray();
.ToArray();
Where
Where

Input
Input
Enumerator
Enumerator

Loc
k

GroupBy
GroupBy

Groups
1

Select
Select

Thread 1

Results
1

Thread 1
Where
Where
GroupBy
GroupBy

Thread 2

...

Groups
2

Select
Select

Thread 2

...

Results
2

output:

PLINQ PERFORMANCE TIPS

Performance Tip #1:

Avoid memory allocations

> When the delegate allocates memory

> GC and memory allocations can become the

bottleneck

> Then, your algorithm is only as scalable as GC

> Mitigations:
> Reduce memory allocations
> Turn on server GC

Performance Tip #2:

Avoid true and false sharing

> Modern CPUs exploit locality

> Recently accessed memory locations are stored
in a fast cache
> Multiple cores
> Each core has its own cache
> When a memory location is modified, it is
invalidated in all caches
> In fact, the entire cache line is invalidated
> A cache line is usually 64 or 128 bytes

Performance Tip #2:

Avoid True and False Sharing
Thread 1

Cache

6
5 7 3 2

Core 1
Thread 2

Memory:

5 7 3 2

Cache
Cache line

5 7 3 2

Core 2

Invalidate
Thread 3

Cache

5 7 3 2

Core 3
Thread 4

Core 4

Cache

5 7 3 2

If cores continue stomping

on each others caches, most
reads and writes will go to
the main memory!

Performance Tip #3:

Use expensive delegates

> Computationally expensive delegate is the best

case for PLINQ

> Cheap delegate over a long sequence may also
scale, but:
> Overheads reduce the benefit of scaling
> MoveNext and Current virtual method calls on enumerator
> Virtual method calls to execute delegates

> Reading a long input sequence may be limited by the

memory throughput

Performance Tip #4:

Write simple PLINQ queries

> PLINQ can execute all LINQ queries

> Simple queries are easier to reason about
> Break up complex queries so that only the

expensive data-parallel part is in PLINQ:

src.Select(x
src.Select(x =>
=> Foo(x))
Foo(x))
.TakeWhile(x
.TakeWhile(x =>
=> Filter(x))
Filter(x))
.AsParallel()
.AsParallel()
.Select(x
.Select(x =>
=> Bar(x))
Bar(x))
.ToArray();
.ToArray();

Performance Tip #5:

Choose appropriate partitioning

> Partitioning algorithms vary in:

> Overhead
> Load-balancing
> The required input representation

> By default:
> Array, IList<> are partitioned statically
> Other IEnumerable<> types are partitioned on demand

in chunks

> Custom partitioning supported via Partitioner