Sie sind auf Seite 1von 38

clojure’s four elevators

java interop
clojure lisp
http://github.com/stuarthalloway/clojure-presentations
functional
state

stuart halloway
http://thinkrelevance.com

Copyright 2007-2009 Relevance, Inc. This presentation is licensed under a Creative Commons Attribution-
Noncommercial-Share Alike 3.0 United States License. See http://creativecommons.org/licenses/by-nc-sa/3.0/us/

1 2

java new

java new Widget("foo")

1. java interop clojure (new Widget "foo")

clojure sugar (Widget. "red")

3 4
access static members access instance members

java Math.PI java rnd.nextInt()

clojure (. Math PI) clojure (. rnd nextInt)

clojure sugar Math/PI clojure sugar (.nextInt rnd)

5 6

chaining access parenthesis count

java person.getAddress().getZipCode()
java ()()()()
clojure (. (. person getAddress) getZipCode)

clojure ()()()
clojure sugar (.. person getAddress getZipCode)

7 8
atomic data types
type example java equivalent
string
character
regex
"foo"
\f
String
Character
Pattern
example:
refactor apache
#"fo*"
a. p. integer 42 Integer/Long/BigInteger
double 3.14159 Double
a.p. double
boolean
3.14159M
TRUE
BigDecimal
Boolean
commons isBlank
nil nil null
symbol foo, + N/A
keyword :foo, ::foo N/A
9 10

initial implementation - type decls


public class StringUtils {
public class StringUtils {
public static boolean isBlank(String str) {
public isBlank(str) {
int strLen;
if (str == null || (strLen = str.length()) == 0) {
if (str == null || (strLen = str.length()) == 0) {
return true;
return true;
}
}
for (i = 0; i < strLen; i++) {
for (int i = 0; i < strLen; i++) {
if ((Character.isWhitespace(str.charAt(i)) == false)) {
if ((Character.isWhitespace(str.charAt(i)) == false)) {
return false;
return false;
}
}
}
}
return true;
return true;
}
}
}
}

11 12
- class + higher-order function

public isBlank(str) {
public isBlank(str) {
if (str == null || (strLen = str.length()) == 0) {
if (str == null || (strLen = str.length()) == 0) {
return true;
return true;
}
}
for (i = 0; i < strLen; i++) {
every (ch in str) {
if ((Character.isWhitespace(str.charAt(i)) == false)) {
Character.isWhitespace(ch);
return false;
}
}
return true;
}
}
return true;
}

13 14

- corner cases lispify

public isBlank(str) {
every (ch in str) {
(defn blank? [s]
Character.isWhitespace(ch);
(every? #(Character/isWhitespace %) s))
}
}

15 16
clojure is a better
2. lisp
java than java

17 18

what makes lisp different regular code


industry cool
feature clojure
norm kids
conditionals ! ! !
variables
garbage collection
!
!
!
!
!
!
foo.bar(x,y,z);
recursion ! ! !
function type ! !
symbol type ! !
whole language available
everything’s an expression
! !
!
foo.bar x y z
homoiconicity !

http://www.paulgraham.com/diff.html

19 20
special forms outside lisp, special forms
imports look different
scopes may have special semantics unavailable to you
protection prevent reuse
metadata
control flow
anything using a keyword

21 22

in a lisp, special forms all forms created equal


look just like anything else
form syntax example
may have special semantics available to you
can be augmented with macros function list (println "hello")
operator list (+ 1 2)
method call list (.trim " hello ")
import list (require 'mylib)
metadata list (with-meta obj m)
control flow list (when valid? (proceed))
scope list (dosync (alter ...))

23 24
who cares?

25 26

clojure is turning
the tide in a fifty-
game break!
year struggle
against bloat
27 28
early impl:
a snake
is a sequence
of points first point is
head

(defn describe [snake]


(println "head is " (first snake))
(println "tail is" (rest snake)))

rest is tail

Sample Code:
http://github.com/stuarthalloway/programming-clojure
29 30

destructure capture
snake is more than location
first element remainder as a
into head sequence

(defn describe [[head & tail]] (defn create-snake []


(println "head is " head) {:body (list [1 1])
(println "tail is" tail)) :dir [1 0]
:type :snake
:color (Color. 15 160 70)})

destructure remaining
elements into tail

31 32
2. nested destructure losing the game
to pull head and tail from the
:body value

(defn describe [{[head & tail] :body}]


(defn lose? [{[head & tail] :body}]
(println "head is " head)
(includes? tail head))
(println "tail is" tail))

1. destructure map,
looking up the :tail

33 34

data literals
type properties example

singly-linked,
list (1 2 3)

3. functional
insert at front

indexed,
vector [1 2 3]
insert at rear

{:a 100
map key/value
:b 90}

set key #{:a :b}

35 36
some data

higher-order lunch-companions
-> ({:fname "Neal", :lname "Ford"}

functions
{:fname "Stu", :lname "Halloway"}
{:fname "Dan", :lname "North"})

37 38

“getter” function pass fn to fn


fn name arg list (vector) call fn
fn arg

(defn last-name [x] (sort-by data arg


(get x :last-name) first-name
lunch-companions)
-> ({:fname "Dan", :lname "North"}
{:fname "Neal", :lname "Ford"}
body {:fname "Stu", :lname "Halloway"})

39 40
anonymous fn anonymous #()

fn arg fn arg
(sort-by body (sort-by
(fn [n] #(get % :fname)
(get n :fname)) lunch-companions)
lunch-companions)

anonymous fn
anonymous fn

41 42

maps are functions keywords are functions


keyword
map is fn! is fn!

(sort-by (sort-by
#(% :fname) #(:fname %)
lunch-companions) lunch-companions)

43 44
beautiful
real languages
(sort-by :fname lunch-companions)
give a 1-1 ratio of
pseudocode/code

45 46

persistent data structures


immutable
“change” by function application
persistent data maintain performance guarantees

structures
full-fidelity old versions

47 48
persistent example: bit-partitioned tries
linked list
“your” “my”
“your” trie trie
“my” list
list

newt tern rabbit tiger

49 50

32-way tries

log2 n:
too slow! ... ... ... ... ...

... ... ... ... ... ... ... ... ...

51 52
clojure: ‘cause
sequence
log32 n is
library
fast enough!

53 54

first / rest / cons take / drop


(first [1 2 3])
-> 1
(take 2 [1 2 3 4 5])
-> (1 2)
(rest [1 2 3])
-> (2 3)
(drop 2 [1 2 3 4 5])
-> (3 4 5)
(cons "hello" [1 2 3])
-> ("hello" 1 2 3)

55 56
map / filter / reduce sort
(range 10)
-> (0 1 2 3 4 5 6 7 8 9)
(sort [ 1 56 2 23 45 34 6 43])
-> (1 2 6 23 34 43 45 56)
(filter odd? (range 10))
-> (1 3 5 7 9)
(sort > [ 1 56 2 23 45 34 6 43])
-> (56 45 43 34 23 6 2 1)
(map odd? (range 10))
-> (false true false true false true
(sort-by #(.length %)
false true false true)
["the" "quick" "brown" "fox"])
-> ("the" "fox" "quick" "brown")
(reduce + (range 10))
-> 45

57 58

conj / into lazy, infinite sequences


(conj '(1 2 3) :a) (set! *print-length* 5)
-> (:a 1 2 3) -> 5

(into '(1 2 3) '(:a :b :c)) (iterate inc 0)


-> (:c :b :a 1 2 3) -> (0 1 2 3 4 ...)

(conj [1 2 3] :a) (cycle [1 2])


-> [1 2 3 :a] -> (1 2 1 2 1 ...)

(into [1 2 3] [:a :b :c]) (repeat :d)


-> [1 2 3 :a :b :c] -> (:d :d :d :d :d ...)

59 60
interpose predicates
(every? odd? [1 3 5])
(interpose \, ["list" "of" "words"])
-> true
-> ("list" \, "of" \, "words")
(not-every? even? [2 3 4])
(apply str
-> true
(interpose \, ["list" "of" "words"]))
-> "list,of,words"
(not-any? zero? [1 2 3])
-> true
(use 'clojure.contrib.str-utils)
(str-join \, ["list" "of" "words"]))
(some nil? [1 nil 2])
-> "list,of,words"
-> true

61 62

nested ops
Ash zna durbatulûk,
(def jdoe {:name "John Doe",
:address {:zip 27705, ...}}) ash zna gimbatul,
(get-in jdoe [:address :zip])
-> 27705 ash zna thrakatulûk
(assoc-in jdoe [:address :zip] 27514)
-> {:name "John Doe", :address {:zip 27514}} agh burzum-ishi
(update-in jdoe [:address :zip] inc)
-> {:name "John Doe", :address {:zip 27706}} krimpatul.
63 64
where are we?
1. java interop
2. lisp
example:
3. functional refactor apache
does it work? commons
indexOfAny
65 66

indexOfAny behavior indexOfAny impl


// From Apache Commons Lang, http://commons.apache.org/lang/
public static int indexOfAny(String str, char[] searchChars)
StringUtils.indexOfAny(null, *) = -1 {
StringUtils.indexOfAny("", *) = -1 if (isEmpty(str) || ArrayUtils.isEmpty(searchChars)) {
StringUtils.indexOfAny(*, null) = -1 return -1;
StringUtils.indexOfAny(*, []) = -1 }
for (int i = 0; i < str.length(); i++) {
StringUtils.indexOfAny("zzabyycdxx",['z','a']) = 0
char ch = str.charAt(i);
StringUtils.indexOfAny("zzabyycdxx",['b','y']) = 3 for (int j = 0; j < searchChars.length; j++) {
StringUtils.indexOfAny("aba", ['z']) = -1 if (searchChars[j] == ch) {
return i;
}
}
}
return -1;
}

67 68
simplify corner cases - type decls

public static int indexOfAny(String str, char[] searchChars)


indexOfAny(str, searchChars) {
{
when (searchChars)
when (searchChars)
for (i = 0; i < str.length(); i++) {
for (int i = 0; i < str.length(); i++) {
ch = str.charAt(i);
char ch = str.charAt(i);
for (j = 0; j < searchChars.length; j++) {
for (int j = 0; j < searchChars.length; j++) {
if (searchChars[j] == ch) {
if (searchChars[j] == ch) {
return i;
return i;
}
}
}
}
}
}
}
}
}
}

69 70

+ when clause + comprehension

indexOfAny(str, searchChars) {
indexOfAny(str, searchChars) {
when (searchChars)
when (searchChars)
for (i = 0; i < str.length(); i++) {
for ([i, ch] in indexed(str)) {
ch = str.charAt(i);
when searchChars(ch) i;
when searchChars(ch) i;
}
}
}
}
}
}

71 72
lispify!

functional
(defn index-filter [pred coll]
(when pred
is
simpler
(for [[idx elt] (indexed coll) :when (pred elt)] idx)))

73 74

imperative functional
functions 1 1
classes
internal exit points
1
2
0
0
functional
variables
branches
3
4
0
0
is
boolean ops
function calls*
1
6
0
3
more general!
total 18 4

75 76
reusing index-filter
imperative functional

; idxs of heads in stream of coin flips searches strings searches any sequence
(index-filter #{:h}
[:t :t :h :t :h :t :t :t :h :h])
-> (2 4 8 9) matches characters matches any predicate

; Fibonaccis pass 1000 at n=17


returns lazy seq of all
(first returns first match
matches
(index-filter #(> % 1000) (fibo)))
-> 17

77 78

fp reduces incidental
complexity by an 4. concurrency
order of magnitude

79 80
oo is incoherent
4. state ?
identity
???
concurrency identity
???

81 82

terms
clojure 1. value: immutable data in a persistent data
explicit semantic structure
2. identity: series of causally related values
identity value over time
value 3. state: identity at a point in time

identity value

state

83 84
identity types (references)

synchronous/
shared

refs/stm
isolated

-
identity 1:
refs and stm
coordinated
synchronous/
atoms vars
autonomous
asynchronous/
agents -
autonomous

85 86

ref example: chat reading value


identity
(deref messages)
=> ()
(def messages (ref ()))
@messages
=> ()

initial value

87 88
alter updating
apply an...
(alter r update-fn & args)

oldval (defn add-message [msg]


(dosync (alter messages conj msg)))
r (apply update-fn
oldval
args)
...update fn
scope a
newval transaction

89 90

unified update model


update by function application
readers require no coordination
a sane approach
readers never block anybody to local state
writers never block readers
permits coordination,
but does not require it

91 92
unified update model
ref atom agent var

create ref atom agent def


identity 2:
deref deref/@ deref/@ deref/@ deref/@
atoms
alter-
update alter swap! send var-
root

93 94

http://blog.bestinclass.dk/index.php/2009/10/brians-functional-brain/
board is just a value

(defn new-board
"Create a new board with about half the cells set
to :on."
([] (apply new-board dim-board))
([dim-x dim-y]
(for [x (range dim-x)]
(for [y (range dim-y)]
(if (< 50 (rand-int 100)) :on :off))))))

distinct bodies by arity

95 96
update is just a function state is trivial
identity initial value
rules
(defn step (let [stage (atom (new-board))]
"Advance the automation by one step, updating all ...)
cells."
[board]
(doall (defn update-stage
(map (fn [window] "Update the automaton."
(apply #(doall (apply map rules %&))
(doall (map torus-window window))))
[stage]
(torus-window board)))) (swap! stage step))

cursor over previous, me, next apply a fn update fn

97 98

send
caller agent fixed thread

identity 3: (send a fn & args)


pool

agents
(apply fn oldval args)

99 100
state is trivial
identity initial value

(def *logging-agent* (agent nil))

(if log-immediately?
identity 4:
(impl-write! log-impl level msg err)
(send-off *logging-agent*
agent-write! log level msg err))
vars
apply a fn “update” fn

101 102

def forms create vars vars can be rebound

(def greeting "hello") api scope

(defn make-greeting [n] alter-var-root root binding


(str "hello, " n)
set! thread-local, permanent

binding thread-local, dynamic

103 104
system settings var usage

(set! *print-length* 20) *in*, *out*, *err* standard streams


=> 20
*print-length*,
structure printing
primes *print-depth*
=> (2 3 5 7 11 13 17 19 23 29 31 37 41
43 47 53 59 61 67 71 ...) *warn-on-reflection* performance tuning

(set! *print-length* 5) *ns* current namespace


=> 5
*file* file being evaluated
primes
=>(2 3 5 7 11 ...) *command-line-args* guess

105 106

with-... helper macros other def forms


(def bar 10) form usage
-> #'user/bar defonce set root binding once
defvar var plus docstring
(with-ns 'foo (def bar 20))
-> #'foo/bar defunbound no initial binding
defstruct map with slots
user/bar
-> 10 defalias same metadata as original

foo/bar
bind a var defhinted infer type from initial binding
-> 20 for a dynamic defmemo defn + memoize
scope
many of these are in clojure.contrib.def...
107 108
use commute
identity: when update
more options can happen
anytime
109 110

not safe for commute safe!

(defn increment-counter
(defn next-id
"Bump the internal count."
"Get the next available id."
[]
[]
(dosync
(dosync
(alter ids inc))
(alter ids inc)))
nil)

111 112
send
prefer send-off caller agent fixed thread
pool

if agent ops
(send a fn & args)

might block
(apply fn oldval args)

113 114

send-off
cached
thread
caller agent pool

(send a fn & args) use ref-set to set


initial/base state
(apply fn oldval args)

115 116
unified update, revisited
update
mechanism
ref atom agent send-off
pure function
application
alter swap! send
to *agent*
pure function
(commutative)
commute - -
for background
pure function
(blocking)
- - send-off
iteration
setter ref-set reset! -

117 118

monte carlo via (not= agents actors)


ongoing agent
queue more agents actors
(defn background-pi [iter-count] work
(let in-process only oop
[agt (agent {:in-circle 0 :total 0})
continue (atom true)
iter (fn sim [a-val] no copying copying
! (when continue (send-off *agent* sim))
! (run-simulation a-val iter-count))]
(send-off agt iter) no deadlock can deadlock
{:guesser agt :continue atom})

do the no coordination can coordinate


escape hatch
work
119 120
create a
function that checks
every item...

(def validate-message-list
(partial
every?

validation
#(and (:sender %) (:text %))))

(def messages for some criteria


(ref
()
:validator validate-message-list))

and associate fn with updates to a ref

121 122

agent error handling


(def counter (agent 0 :validator integer?))

agents
-> #'user/counter

(send counter (constantly :fail))


-> #<Agent 0>

(agent-errors counter)
-> (#<IllegalStateException
will fail soon
and
transactions
java.lang.IllegalStateException:
Invalid reference state>)

(clear-agent-errors counter)
-> nil list of errors

@counter reset and move on


-> 0

123 124
tying agent to a tx where are we?
1. java interop
2. lisp
(defn add-message-with-backup [msg] 3. functional
(dosync
(let [snapshot (alter messages conj msg)] 4. value/identity/state
(send-off backup-agent (fn [filename]
! ! ! (spit filename snapshot)
! ! ! filename))
snapshot))) does it work?

exactly once if tx succeeds

125 126

a workable approach to state


good values: persistent data structures
good identities: references
mostly
functional?
mostly functional?

usable by mortals?

127 128
calls to calls to calls to
project loc
ref agent atom
1 line in 1000 clojure 7232 3 1 2

creates a clojure-contrib 17032 22 2 12

reference compojure 1966 1 0 0

incanter 6248 1 0 0

129 130

; compojure session management multimethod


(def memory-sessions (ref {})) dispatch

usable by (defmethod read-session :memory


[repository id]
(@memory-sessions id))

mortals? (defmethod write-session :memory


[repository session]
(dosync
(alter memory-sessions
assoc (session :id) session)))

read update

131 132
clojure
cache previous
results values are
; from clojure core
(defn memoize [f] immutable, persistent
(let [mem (atom {})]
(fn [& args] identities are
(if-let [e (find @mem args)] well-specified, consistent
cache hit (val e)
(let [ret (apply f args)] state is
(swap! mem assoc args ret) mostly functional
ret)))))
usable by mortals
cache miss:
call f, add to
cache

133 134

languages that
emphasize time
immutability are management
better at mutation
135 136
prepare to parallelize done

(defn step (defn step


"Advance the automation by one step, updating all "Advance the automation by one step, updating all
cells." cells."
[board] [board]
(doall (doall
(map (fn [window] (pmap (fn [window]
(apply #(doall (apply map rules %&)) (apply #(doall (apply map rules %&))
(doall (map torus-window window)))) (doall (map torus-window window))))
(torus-window board)))) (torus-window board))))

137 138

delay future
(def e (delay (expensive-calculation)))
-> #'demo.delay/e
(def e1 (future (expensive-calculation)))
(delay? e) -> #'demo.future/e1
-> true
(deref e1) first call blocks
(force e) -> :some-result until work
-> :some-result completes on
first call blocks @e1 other thread,
(deref e) until work -> :some-result later calls hit
-> :some-result completes on cache
this thread,
later calls hit
@e
cache
-> :some-result

139 140
cancelling a future

(def e2 (future (expensive-calculation)))


-> #'demo.future/e2

(future-cancel e2)
-> true transients
(future-cancelled? e2)
-> true

(deref e2)
-> java.util.concurrent.CancellationException

141 142

persistent...
build structure
on one thread, (defn vrange
(loop [i 0
(if (< i
[n]
v []]
n)

then release into (recur


v)))
(inc i) (conj v i))

the wild
143 144
...to transient fast!
enter transient world

(defn vrang2 [n] (time (def v (vrange 1000000)))


(loop [i 0 v (transient [])] "Elapsed time: 1130.721 msecs"
(if (< i n)
(recur (inc i) (conj! v i)) (time (def v2 (vrange2 1000000)))
(persistent v)))) "Elapsed time: 82.191 msecs"

use transient updater


return to
persistent world

145 146

transients
usage:
transient
bang updates: assoc! conj! etc.
persistent!
what about
optimization, not coordination
O(1) creation from persistent object
objects?
fast, isolated updates
O(1) conversion back to persistent object

147 148
oo: one identity fits all

clojure: bespoke
thread-safety
encapsulation dispatch

state
identity constructors
code in an off-the-
functions rack world
validation
behavior

149 150

clojure’s four elevators Email:


Office: !
Twitter: !
Facebook:
stu@thinkrelevance.com
919-442-3030
twitter.com/stuarthalloway
stuart.halloway
Github: stuarthalloway
This talk: http://github.com/stuarthalloway/clojure-presentations
java interop Talks: !
Blog:
http://blog.thinkrelevance.com/talks
http://blog.thinkrelevance.com
Book: http://tinyurl.com/clojure

lisp
functional
state

151 152