Sie sind auf Seite 1von 324

Going Go Programming

William Kennedy

Why Go Programming
For the past 20 years I have been writing server based and application software on the Microsoft stack. First in C C!! leveraging the Win"2 #$I and then in C% when .&et first was released. 'ver the past few months I have reali(ed that trying to b)ild scalable code on the Microsoft stack is becoming impossible. Why* +echnology and Cost,, -et.s start with the licensing. -)ckily I was accepted into the /i(spark program. If I didn.t have that I wo)ld be looking at tho)sands of dollars to 0)st get access to the tooling I need. 1eal world applications re2)ire a database. If yo) want to )se 34- 3erver 3tandard the licensing model is not only convol)ted b)t o)trageo)sly e5pensive once yo) fig)re all that o)t. If I am b)ilding something for a client and they need something more than 34- 65press I am doing them an in0)stice. 6ven )sing 34- 3erver in the clo)d is no picnic on pricing. +hen yo) have C$7 and Memory re2)irements. I am sorry b)t trying to r)n a 34- 3erver and II3 on anything less than 8 9ig is impossible. -astly* take this cost for one machine and try to b)ild a scalable architect)re aro)nd it and yo)r costs are thro)gh the roof. I don.t think it is an accident that Facebook* +witter and the like are r)nning on lin)5 based technologies. +he lin)5 based technologies are cheaper* faster and re2)ire m)ch less metal. :o) can b)ild larger systems and architect)res for fractions of the cost. +hese are proven technologies that are driving the biggest websites today. I have already moved away from 34- 3erver to Mongo;/. Mongo;/ has been a real win in terms of development* devops and cost. It has been aro)nd since 200<* has a h)ge comm)nity of people s)pporting it and there are drivers for 0)st abo)t every programming lang)age. In fact* I have been able to do more with Mongo;/ than I co)ld with 34- 3erver. 'h yea* no cost ,, For web development I am now )sing 1)by on 1ails. +his was an easy transition from .&et M=C and I )se 1)byMine as my I;6. 1)byMine is a great tool and it only costs >?00. If yo) are b)ilding 0)st web services then )sing $adrino is a great way to go. &ow the r)b* b)ilding application servers on the lin)5 stack. I didn.t want to )se C C!!* I was done with that. 3omeone s)ggested 1)by b)t I have never felt it was a great lang)age for server development. I looked at 6rlang b)t that wasn.t right. 3o I contin)ed to )se my C% based Windows service I called the +ask3erver. My +ask3erver is really cool. It is a Windows service framework that can load and )nload ;-- based packages at r)ntime. 6ach $l)gin is loaded into its own application domain. I added facilities to abstract threading* internal and e5ternal pl)gin comm)nication* trace logging and anything I co)ld do to make $l)gin development fast and easy. +he +ask3erver has served me well b)t it r)ns on the Microsoft stack. For my 1)by web apps to talk with it I )se 1abbitM4. 6vent)ally I was going to b)ild a 9em that implemented the 3ocket protocol so I co)ld talk to the server directly.

+hen someone told me to look at 9o $rogramming. #fter reading and watching a few videos I was blown away. +his is what I have been searching for. # programming lang)age tailored to server development that wo)ld r)n on the lin)5 stack. I have been able to take all of my C C!! C% e5perience and 2)ickly learn the lang)age. 'bvio)sly the goro)tine channel constr)ct is 2)ite different b)t once yo) get yo)r head wrapped aro)nd it yo) reali(e how m)ch easier it makes conc)rrent programming. #s with any new lang)age my first step is to port my )tility classes and I have already beg)n that work. I have finished my +hread$ool package and now I am working on my +race-og package. 'nce that is done I am moving to Mongo;/ and then will begin to b)ild a server that will manage large amo)nts of data imports for a pro0ect I am working on* ;etails soon to come. +his is the beginning of my 0o)rney in 9o $rogramming and never looking back at the Microsoft stack again. 3orry =MWare b)t I hope I never need to )pgrade yo) again.

Thread Pooling in Go Programming


In my world of server development thread pooling has been the key to b)ilding rob)st code on the Microsoft stack. Microsoft has failed in .&et by giving each $rocess a single thread pool with tho)sands of threads and thinking they co)ld manage the conc)rrency at r)ntime. 6arly on I reali(ed this was never going to work. #t least not for the servers I was developing. When I was b)ilding servers in C C!! )sing the Win"2 #$I* I created a class that abstracted I'C$ to give me thread pools I co)ld post work into. +his has always worked very well beca)se I co)ld define the n)mber of threads in the pool and the conc)rrency level @the n)mber of threads allowed to be active at any given timeA. I ported this code for all of my C% development. If yo) want to learn more abo)t this* I wrote an article years ago @httpB www.the)kwebdesigncompany.com articles iocpCthreadCpooling.phpA. 7sing I'C$ gave me the performance and fle5ibility I needed. /+W* the .&6+ thread pool )ses I'C$ )nderneath. +he idea of the thread pool is fairly simple. Work comes into the server and needs to get processed. Most of this work is asynchrono)s in nat)re b)t it doesn.t have to be. Many times the work is coming off a socket or from an internal ro)tine. +he thread pool 2)e)es )p the work and then a thread from the pool is assigned to perform the work. +he work is processed in the order it was received. +he pool provides a great pattern for performing work efficiently. 3pawning a new thread everytime work needs to be processed can p)t heavy loads on the operating system and ca)se ma0or performance problems. 3o how is the thread pool performance t)nedD :o) need to identify the n)mber of threads each pool sho)ld contain to get the work done the 2)ickest. When all the ro)tines are b)sy processing work* new work stays 2)e)ed. :o) want this beca)se at some point having more ro)tines processing work slow things down. +his can be for a myriad of reasons s)ch as* the n)mber of cores yo) have in yo)r machine to the ability of yo)r database to handle re2)ests. ;)ring testing yo) can find that happy n)mber. I always start with looking at how many cores I have and the type of work being processed. 2

;oes this work get blocked and for how long on average. 'n the Microsoft stack I fo)nd that three active threads per core seemed to yield the best performance for most tasks. I have no idea yet what the n)mbers will be in 9o. :o) can also create different thread pools for the different types of work the server will need to process. /eca)se each thread pool can be config)red* yo) can spend time performance t)ning the server for ma5im)m thro)ghp)t. Eaving this type of command and control to ma5imi(e performance is cr)cial. In 9o we don.t create threads b)t ro)tines. +he ro)tines f)nction like m)ltiCthreaded f)nctions b)t 9o manages the act)al )se of '3 level threading. +o learn more abo)t conc)rrency in 9o check o)t this doc)mentB httpB golang.org doc effectiveFgo.html%conc)rrency. +he packages I have created are called workpool and 0obpool. +hese )se the channel and go ro)tine constr)cts to implement pooling.

Workpool
+his package creates a pool of go ro)tines that are dedicated to processing work posted into the pool. # single 9o ro)tine is )sed to 2)e)e the work. +he 2)e)e ro)tine provides the safe 2)e)ing of work* keeps track of the amo)nt of work in the 2)e)e and reports an error if the 2)e)e is f)ll. $osting work into the 2)e)e is a blocking call. +his is so the caller can verify that the work is 2)e)ed. Co)nts for the n)mber of active worker ro)tines are maintained. Eere is some sample code on how to )se the workpoolB package main import ( "github.com/goinggo/workpool" "bufio" "fmt" "os" "runtime" "strconv" "time" ) type MyWork struct { Name string " he Name of a person" !irth"ear int " he "ear the person was born" # $workpool.Work#ool % func (myWork MyWork) &oWork() { fmt.#rintf("'s ( ')*n"+ myWork.Name+ myWork.!irth"ear) fmt.#rintf(",(') -(')*n"+ myWork. #.,ueue)Work()+ "

myWork. #..ctive-outines()) // /imulate some )elay time./leep(011 $ time.Millisecon)) % func main() { runtime.23M.4#-35/(runtime.Num5#6()) work#ool (7 workpool.New(runtime.Num5#6()+ 811) shut)own (7 false // 9ust for testing+ : ;now go func() { for i (7 1< i = 0111< i>> { work (7 new(MyWork) work.Name 7 "." > strconv.:toa(i) work.!irth"ear 7 i work. # 7 work#ool err (7 work#ool.#ostWork(work) if err ?7 nil { fmt.#rintf("@--3-( 's*n"+ err) time./leep(011 $ time.Millisecon)) % if shut)own 77 true { return % % %() fmt.#rintln("Ait any key to eBit") rea)er (7 bufio.New-ea)er(os./t)in) rea)er.-ea)/tring(C*nC) shut)own 7 true fmt.#rintln("/hutting &own*n") work#ool./hut)own() % If we look at main* we create a thread pool where the n)mber of ro)tines to )se is based on the n)mber of cores we have on the machine. +his means we have a ro)tine for each core. :o) can.t do any more work if each core is b)sy. #gain* performance testing will determine what this n)mber sho)ld be. +he second parameter is the si(e of the 2)e)e. In this case I have made the 2)e)e large eno)gh to handle all the re2)ests coming in. +he MyWork type defines the state I need to perform the work. +he member f)nction ;oWork is re2)ired beca)se it implements an interface re2)ired by the $ostWork call. +o 8

pass any work into the thread pool this method m)st be implement by the type. +he ;oWork method is doing two things. First it is displaying the state of the ob0ect. 3econd it is reporting the n)mber of items in 2)e)e and the active n)mber of 9o 1o)tines. +hese n)mbers can be )sed to determining the health of the thread pool and for performance testing. Finally I have a 9o ro)tine posting work into the work pool inside of a loop. #t the same time this is happening* the work pool is e5ec)ting ;oWork for each ob0ect 2)e)ed. 6vent)ally the 9o ro)tine is done and the work pool keeps on doing its 0ob. If we hit enter at anytime the programming sh)ts down gracef)lly. +he $ostWork method co)ld ret)rn an error in this sample program. +his is beca)se the $ostWork method will g)arantee work is placed in 2)e)e or it will fail. +he only reason for this to fail is if the 2)e)e is f)ll. 3etting the 2)e)e length is an important consideration.

Jobpool
+he 0obpool package is similar to the workpool package e5cept for one implementation detail. +his package maintains two 2)e)es* one for normal processing and one for priority processing. $ending 0obs in the priority 2)e)e always get processed before pending 0obs in the normal 2)e)e. +he )se of two 2)e)es makes 0obpool a bit more comple5 than workpool. If yo) don.t need priority processing* then )sing a workpool is going to be faster and more efficient. Eere is some sample code on how to )se the 0obpoolB package main import ( "github.com/goinggo/Dobpool" "fmt" "time" ) type Work#rovi)er0 struct { Name string % func (this $Work#rovi)er0) -un9ob() { fmt.#rintf("#erform 9ob ( #rovi)er 0 ( /tarte)( 's*n"+ this.Name) time./leep(E $ time./econ)) fmt.#rintf("#erform 9ob ( #rovi)er 0 ( &3N@( 's*n"+ this.Name) % type Work#rovi)erE struct { Name string %

func (this $Work#rovi)erE) -un9ob() { fmt.#rintf("#erform 9ob ( #rovi)er E ( /tarte)( 's*n"+ this.Name) time./leep(F $ time./econ)) fmt.#rintf("#erform 9ob ( #rovi)er E ( &3N@( 's*n"+ this.Name) % func main() { Dob#ool (7 Dobpool.New(E+ 0111) Dob#ool.,ueue9ob(GWork#rovi)er0{"Normal #riority ( 0"%+ false) fmt.#rintf("$$$$$$$H ,W( ') .-( ')*n"+ Dob#ool.,ueue)9obs()+ Dob#ool..ctive-outines()) time./leep(0 $ time./econ)) Dob#ool.,ueue9ob(GWork#rovi)er0{"Normal #riority ( E"%+ false) Dob#ool.,ueue9ob(GWork#rovi)er0{"Normal #riority ( I"%+ false) Dob#ool.,ueue9ob(GWork#rovi)erE{"Aigh #riority ( J"%+ true) fmt.#rintf("$$$$$$$H ,W( ') .-( ')*n"+ Dob#ool.,ueue)9obs()+ Dob#ool..ctive-outines()) time./leep(0F $ time./econ)) Dob#ool./hut)own() % In this sample code we create two worker type str)cts. It.s best to think that each worker is some independent 0ob in the system. In main we create a 0ob pool with 2 0ob ro)tines and s)pport for ?000 pending 0obs. First we create " different Work$rovider? ob0ects and post them into the 2)e)e* setting the priority flag to false. &e5t we create a Work$rovider2 ob0ect and post that into the 2)e)e* setting the priority flag to tr)e. +he first two 0obs that are 2)e)ed will be processed first since the 0ob pool has 2 ro)tines. #s soon as one of those 0obs are completed* the ne5t 0ob is retrieved from the 2)e)e. +he Work$rovider2 0ob will be processed ne5t beca)se it was placed in the priority 2)e)e. +o get a copy of the workpool and 0obpool packages* go to gith)b.com goinggo

#s always I hope this code can help yo) in some small way.

Installing Go, Gocode, GDB and LiteIDE


I have been working in windows for 20 years and know the internals of that operating system very well. I am very new to )sing my Mac and it is always a challenge for me when I need to install software or make config)ration changes. I am getting better. It took me abo)t H ho)rs over two days to get my 9o environment working on my Mac the first time. Eere are the steps that will help with installing 9o on the Mac.

Step ! Do"nload Go
'pen yo)r favorite browser and go to the following websiteB httpsB code.google.com p go downloads list

+his will show yo) all the latest b)ilds for the different operating systems. ;arwin is the name for the Mac '3. ;ownload the package for yo)r '3 version and architect)re. 'nce downloaded go to yo)r ;ownloads folder in Finder and do)ble click on the pkg file to start the installation. +he installation will p)t 9o in /usr/local/go. 'nce the installation is complete yo) will want to check the installation. Note: Moving forward we will be using both Terminal and Finder. It helps to be able to see all the files in Finder. Finder by default will not show you everything that is on your hard drive. To show all files in Finder: 'pen a +erminal session. If yo) don.t know where the +erminal program is go to yo)r #pplications folder in Finder and then 7tilities. Click on +erminal and a bash shell command window will open. 65ec)te the following commands in +erminal. defa)lts write com.apple.finder #pple3how#llFiles +176 killall Finder +he killall Finder will reload the Finder program and now yo) can see all the files.

Step #! $heck Installation


I

'pen a +erminal session and type the following command go version :o) sho)ld see the following if everything installed correctly go version go?.2.? darwin amdH8 &ow type the which command to verify the installation is in /usr/local/go which go :o) sho)ld see that 9o can be fo)nd in /usr/local/go/bin )sr local go bin go &ow 9o is installed b)t we are not ready to start programming 0)st yet. In order to get intellisense when we are )sing -iteI;6 we need 9oCode.

Step %! Set &o'r G(P)T*


:o) need a single place to store and work on yo)r 9o pro0ects. Create a folder called $ro0ects from inside yo)r home folderB cd >E'M6 mkdir $ro0ects &ow set this as yo)r 9'$#+E. 'pen the .bashFprofile file from the >E'M6 folder and add the following items to the end. nano .bashFprofile e5port 9'$#+EJK>E'M6 $ro0ectsK e5port $#+EJ>$#+EB>9'$#+E bin +hen e5it the +erminal #pp and open a new +erminal #pp. Check that the 9o environment now has yo)r new 9'$#+E. go env :o) sho)ld see all the 9o related environment variables incl)ding 9'$#+E. Eere are some of themB 9'#1CEJKamdH8K 9'CE#1JKHK 9'E'3+#1CEJKamdH8K 9'E'3+'3JKdarwinK 9''3JKdarwinK G(P)T*+,-.sers-yo'-Pro/ects,

9'1''+JK )sr local goK 9'+''-;I1JK )sr local go pkg tool darwinFamdH8K

Step 0! Get, B'ild and Install Gocode


9ocode is a program that provides intellisense for the lang)age. Many editors s)pport )sing 9ocode incl)ding -iteI;6. +his is going to help with development immeas)rably. We need to download 9ocode* b)ild the binary and then install the binary in a place where it can be easily fo)nd by -iteI;6 and other editors. 'pen a +erminal session or )se the c)rrent one yo) have open and type the following command. +his ass)mes yo) set)p yo)r 9'$#+E as instr)cted in 3tep ". go get gith)b.com nsf gocode 'nce this is done the so)rce code for 9ocode can be fo)nd in the folder 1*(2E/Project/src/github.com/nsf/gocode and the gocode program will be b)ilt and placed in 1*(2E/Project/bin. +o check that everything is working and yo)r go environment is properly set)p* r)n the which commandB which gocode :o) sho)ld seeB 7sers yo) $ro0ects bin gocode

Step 3! Install GDB


Installing the new version of 9;/ is going to be a )ni2)e e5perience for most. +he other problem is that yo) most likely have a version already installed on yo)r machine. 1)n this command in +erminal gdb CCversion If yo) are like me the following information will be provided 9&7 gdb H.".G0C200G0L?G @#pple version gdbC?I0GA &ow r)n this command in +erminal which gdb +his version of gdb is installed )nder )sr bin gdb

<

3o why is this importantD When we b)ild and install the new version of 9;/ it will be installed )nder /usr/local/bin. 1emember 9o was installed )nder /usr/local/go. I don.t )nderstand why some programs are installed )nder /usr/bin or /usr/local/bin or even why yo) can find different version of the same binary )nder both. When we are done this is e5actly what we will have. 'pen yo)r browser again and go to the following )rlB httpB www.gn).org software gdb download :o) will see the following on the page

Click the link called http://ftp.gnu.org/gnu/gdb and yo) will see a list of files yo) can download

;ownload gdb-7.7.tar.gz from yo)r browser and then find the file in Finder inside the ownloads folder. ;o)ble click on the file in Finder and Finder will )n(ip the file to a new folder called gdb-7.7. 9o back to yo)r +erminal session and navigate to the gdb-7.7 folder in ownloads cd M downloads gdbCI.I :o) need to r)n two commands from +erminal that will b)ild the so)rce code. . config)re make If the make command does not work then install NCode. I had NCode installed on my machine prior to r)nning these commands. NCode installs the compilers that are )sed to b)ild code. #lso* I fo)nd with version I.I the make failed beca)se of some errors. If this is happening*

?0

remove the )n(ipped folder* )n(ip the tar.g( file again and )se this parameter with the config)re call. . config)re CCdisableCwerror 'nce the make command is done we need to install this version of 9;/ to /usr/local/bin. In +erminal r)n the following command s)do make install 'nce this command is finished the new version of 9;/ will almost be ready for )se. 1emember we now have two version of 9;/ installed on the machine. From inside of -iteI;6 this will not be a problem b)t from +erminal it is. 1)n the following command in +erminal again gdb CCversion :o) still get the old version 9&7 gdb H.".G0C200G0L?G @#pple version gdbC?I0GA &ow r)n this command in +erminal which gdb +he old version of gdb is still being )sed )sr bin gdb 3o why is this happeningD /eca)se yo)r $#+E has /usr/bin before /usr/local/bin. We can fi5 this for o)r c)rrent +erminal session by r)nning the following +erminal command. e5port $#+EJ )sr local binB>$#+E +his command will )pdate the $#+E and p)t /usr/local/bin at the front. &ow r)n the 9;/ version command again. gdb CCversion &ow yo) get the new version 9&7 gdb @9;/A I.I &ow r)n this command in +erminal which gdb +his version of gdb is being )sed )sr local bin gdb ??

7nfort)nately this is not permanent. If yo) open a new +erminal window the $#+E variable will go back to the original setting. 3o how do we make /usr/local/bin always come before /usr/bin in the $#+E every time we open a +erminal sessionD :o) will need to modify the paths file )nder the etc folder. cd etc s)do nano paths In my original version of the paths file I added the following entriesB )sr bin bin )sr sbin sbin )sr local bin O)st move )sr local bin to the top and save the fileB )sr local bin )sr bin bin )sr sbin sbin +o see if the change worked* open a new +erminal session and echo the $#+EB echo >$#+E +o get more information abo)t how to )se the 9;/ check o)t this website. :o) can r)n these 9;/ commands from inside of -iteI;6 so this web page is helpf)l. httpB golang.org doc gdb &ow that 9;/ is installed we need to do one more thing* codesign the binary so it can be )sed to deb)g the programs we write. &'+6B $lease review the 9;/ $roblems at the end of the article

Step 4! $odesign GDB


If we don.t codesign the 9;/ e5ec)table -iteI;6 will r)n in deb)g mode b)t it won.t work. +he steps I have provided come from the following websites. httpB so)rceware.org gdb wiki /)ilding'n;arwin httpB iosdevelopertips.com macCos5 codeCsigningCerrorCob0ectCfileCformatC)nrecogni(edC invalidCorC)ns)itable.html

?2

&'+6B Make s)re yo) have the latest version of Ncode installed before yo) contin)e 3a5 $reating a $erti6icate 3tart the Keychain #ccess application /!pplications/"tilities/#e$chain !ccess.app 'pen the Men) option /#e$chain !ccess/%ertificate !ssistant/%reate a %ertificate... In the $reate a $erti6icate dialog bo5 )se the following settings &ameB gdb-cert Identity +ypeB &elf &igned 'oot Certificate +ypeB %ode &igning -et Me 'verride ;efa)ltB %hecked Click $ontin'e several times )ntil yo) get to the &pecif$ a (ocation For The %ertificate screen KeychainB &$stem If yo) can.t store the certificate in the &$stem keychain* create it in the login keychain* then e5ported it. :o) can then imported it into the &$stem keychain. +hen find yo)r new certificate in the list and right click and select )et *nfo. +hen e5pand the Trust item and find the %ode &igning drop down. Change the setting for Code 3igning to !lwa$s Trust. :o) m)st 2)it the Keychain #ccess application in order to )se the certificate so close the program. 3b5 $odesigning GDB "ith ne" certi6icate /efore yo) r)n the codesign command in +erminal yo) need to add an e5port to yo)r +erminal session or yo) will get the following error ob0ect file format )nrecogni(ed* invalid* or )ns)itable 'pen or re)se yo)r e5isting +erminal session and r)n the following e5port command e5port C';63I9&F#--'C#+6JK #pplications Ncode.app Contents ;eveloper +oolchains Ncod e;efa)lt.5ctoolchain )sr bin codesignFallocateK &ow change the directory in yo)r +erminal session to where the 9;/ binary is located and codesign the binary. cd )sr local bin codesign Cs gdbCcert gdb +here sho)ld be no errors and a new +erminal command prompt sho)ld 0)st appear. &ow

?"

everything is ready to go. We have 9o* 9ocode and 9;/ installed. &e5t we need to install -iteI;6.

Step 7! Install LiteIDE


-iteI;6 is an e5cellent I;6 and I love working in it. Installing and getting it going re2)ires 0)st a few steps. 'pen yo)r browser and navigate to httpB so)rceforge.net pro0ects liteide files N2?.?

Chose the Mac version @liteide52?.?.macos5.(ipA. ;ownload the file and then open Finder and navigate to the ;ownloads folder. 7n(ip the file and copy -iteI;6.app into the #pplications folder. 'pen yo)r #pplications folder and do)ble click on -iteI;6 to start it.

Step 8! Test LiteIDE


-et.s create a new program and test o)r installation 3witch to -iteI;6 and find the 9ie" men) option. 3elect the Manage 9'$#+E option at the bottom of the men). +he Manage 9'$#+E dialog bo5 will appear.

?8

:o) need to add the Projects folder yo) created in step " as a c)stom directory for yo)r 9'$#+E. +his is where 9o will look for c)stom packages and the code yo) are writing. +he 9o libraries will always be available. +o learn more abo)t the 9'$#+E go to this web pageB httpB golang.org doc code.html Choose File &ew and perform the following tasks.

?. ;o)ble click on yo)r c)stom 9'$#+E. It will be c)t off in the dialog bo5 b)t it sho)ld show above the system folder -'sr-local-go. +his sho)ld change the -ocation field to the 9'$#+E. 2. 3elect )o+ %ommand Project. ". &ame yo)r new program testFprogram. 8. Click 'K and agree to open main.go. Be6ore yo' b'ild and test the program make s're yo' are 'sing the correct b'ild en:ironment ;dar"in40<local=5

?G

+est if yo) can b)ild and r)n the program. Find the bl)e ,B>, b)tton and select it. If -iteI;6 is config)red correctly the program sho)ld b)ild and r)n. +he /)ild ')tp)t sho)ld look like the image below. :o) sho)ld see the Eello World, print in the /)ild ')tp)t window.

&e5t test if the deb)gger is working. 3elect line < of the program and click on the orange b)tton to add a breakpoint.

&e5t yo) need to change the /)ild config)ration a bit. #dd these b)ild arg)ments. +hey help s)pport the 9;/. +he deb)gging is not perfect b)t this makes it better. +he deb)gger will work witho)t it b)t my e5perience is that the deb)gger can pick )p more information. <gc6lags ,<? <l, @Capital & and -owercase -A +o learn more abo)t deb)gging in 9o click on this linkB httpB golang.org doc gdb

?H

&ow from the ;eb)g men) select 3tart ;eb)gging.

If the deb)gger is working a green arrow will stop at line L. :o) may be asked to enter yo)r password before deb)gging begins. If this is happening yo) are good to go. If things stop working always do)ble check that yo) have the right environment selected @darwinH8ClocalA. I can.t tell yo) how m)ch time I have lost beca)se the environment changed witho)t me reali(ing it.

Possible B'ild Errors


,can-t open output file for writing: a.out. errno/+0 for architecture 123435, 3ol)tion /yB #le0andro 9aviria =ersionB Ncode 8.2* Mac '3 N @?0.I.?A* Mac '3 N @?0.I.2A +his is d)e to the gcc compiler that comes with NCode M8.?. 3ol)tion comes from this #pple disc)ssion threadB httpsB disc)ssions.apple.com thread "80HGIL

Possible GDB Errors


,"nable to find 6ach task port for process-id +7020: 8os/kern9 protection failure 8:179.;n 8please check gdb is codesigned - see taskgated829, ?I

3ol)tion /yB Karl +)hkanen =ersionB Mac '3 N @?0.LA ?. s)do chgrp procmod )sr local bin gdb 2. s)do chmod g!s )sr local bin gdb ". add the legacy switch Cp to taskgated by modifying the file 3ystem -ibrary -a)nch#gents com.apple.taskgated.plist 8. Force kill taskgated process @it will restartA or reboot if necessary 3ol)tion comes from this stackoverflow threadB httpB stackoverflow.com 2)estions ?20G02GI gdbCfailsConCmo)ntainClion #-61+, /e s)re to )se standard .plist mark)p with the file modification. 'therwise '3N won.t start ne5t time yo) reboot. +his happened to me. 3ol)tion to this is to reboot with recovery option @CmdC1A and modifying the file with vi to match the standard. <ersion 7.3.7 still has a bug that pre=ents it from loading the )o runtime integration> 3ol)tion /yB Chris Mc9ee =ersionB Mac '3 N :o) can see the problem when yo) fire )p 9;/ and it does not have the special K-oading 9o 1)ntime 3)pportK message. +here is a patch for thisB httpB so)rcewareCorg.?G08.nI.nabble.com $athC#ddCs)pportCforCmachCoCreaderCtoCbeCawareC ofCdeb)gCgdbCscriptsCtd2"L"I2.html ) ? Freezes !nd %onsumes %P" @hen "sing ,*nfo (ocals, 3ol)tion /yB Chris Mc9ee =ersionB Mac '3 N 9;/ free(es. 'nly CtrlCC seems to bring it back. When )sing an application that )ses gdb MI for a 97I interface gdb is totally )nresponsive. httpsB code.google.com p go iss)es detailDidJHG<L

$oncl'sion
I hope these instr)ctions get yo) )p and r)nning with 9o 2)ickly on yo)r Mac. &ow that yo) can write code )sing -iteI;6 I s)ggest b)ilding some test programs and learning how all the different windows in the I;6 work. +here are a lot of great goodies in there. Check o)t this web pageB httpB goClang.catCv.org books It is the 9o lang)age reso)rces page and contains a lot of great links to information. +he /ook Programming in Go by 2ark S'mmer6ield is o)tstanding and I highly recommend b)ying it. Eere are other links to web pages that will be very helpf)lB ?L

httpB golang.org httpsB pl)s.google.com !golang posts httpB blog.golang.org httpB golang.org doc gdb httpB www.yo)t)be.com )ser gocoding :o) m)st watch these videos on 9o Conc)rrency $atterns httpB www.yo)t)be.com watchDvJ4;;wwe$b;tw httpB www.yo)t)be.com watchDvJfHkdp2I+:Ps

Doc'menting Go $ode With Godoc


#s yo) know if yo) read my blog* I have been b)ilding a set of new )tility packages so I can start developing an application server I need for a new pro0ect. I am brand new to 9o and the Mac '3. &eedless to say it has been one hell of an ed)cation over the past month. /)t I don.t miss Windows or C% at all. I made some progress in my coding and wanted to b)ild doc)mentation for the code. I have been )sing the doc)mentation viewer in -iteI;6 and I was hoping to integrate my doc)mentation in there was well. I was really s)rprised to see see that -iteI;6 already had my packages listed inside of their integrated 9odoc viewer. 3o it then begged the 2)estion. Eow is that workingD #fter some digging aro)nd I fo)nd this local E+M- file. If yo) have -iteI;6 installed yo) can copy the following )rl into yo)r browser. fileB #pplications -iteI;6.app Contents 1eso)rces golangdoc abo)t.html +his is what it shows

Overview
+he integrated 9odoc viewer in -iteI;6 provides an easy way to browse doc)mentation generated by the godoc tool witho)t leaving the editor. ;oc)mentation can be viewed for both the official 9o lang)age as well as c)stom packages. +he remainder of this page describes ways to invoke the 9odoc viewer.

Supported URL Schemes


It is possible to view doc)mentation by directly entering a 71- into the 9odoc viewer.s address bar. When doing this* yo) can specify what type of doc)mentation yo) are looking for by prefi5ing the address with one of the following 71- schemesB

6ind
3earches for packages with a specified string in their name. For e5ampleB findB(ip findBgodoc

?<

list
-ists all packages in a given directory. +he main choices are KpkgK and KcmdK* which can be fo)nd as links in the header of the page. For e5ampleB listBpkg C displays the 9olang packages listBcmd C displays the 9olang commands

pdoc
=iews doc)mentation for a specified package or command. For e5ampleB pdocBfmt pdocBarchive (ip pdocBgofmt pdocBfB hg (m2 go(m2

6ile
=iews a specified E+M-* Markdown* or plainCte5t file. For e5ampleB fileBcB go doc docs.html

Automatic Schemes
For the KfileK and KpdocK schemes* yo) do not need to type the scheme as part of the 71-. For e5ampleB doc code.html src pkg src cmd pkg fmt cmd cgo archive (ip go

File Browser
:o) can open the 9odoc viewer directly from the file browser by right clicking on a file or directory and selecting K=iew 9odoc EereK. +he 9odoc viewer will a)tomatically open the package doc)mentation for the chosen item. When I clicked on my package !rdan&tudios/threadpool from within the -iteI;6 9odoc search tool it )sed the pdoc 71- scheme* pdoc:!rdan&tudios/threadpool. I 2)ickly reasoned that -iteI;6 was )sing the 9'1''+ and 9'$#+E variables to find the doc)mentation. +here is only one problem* I haven.t created any doc)mentation yet. 3o I looked aro)nd in both -'sr-local-go and my own space to find the doc)mentation files and there was nothing. 3o how the heck was this doc)mentation being generated and p)blished on the screenD +hen I fo)nd this doc)ment from the 9o teamB

20

httpB golang.org cmd godoc +he very first line states* K)odoc e1tracts and generates documentation for )o programs., 'k* so this program is being )sed by -iteI;6 b)t howD Where are the files that 9odoc is generating for all this doc)mentationD -'-* boy it is diffic)lt coming from a Windows environment for the past 20 years. #fter reading the doc)mentation a b)nch of times I opened )p a +erminal session and ran the following command. godoc /"sers/bill/&paces/)oPackages/src/!rdan&tudios/threadpool 3)ddenly the doc)mentation appeared on my screen in te5t format. /)t I am seeing E+Minside of -iteI;6D I fo)nd the -html option. godoc -html /"sers/bill/&paces/)oPackages/src/!rdan&tudios/threadpool &ow I prod)ced the same doc)mentation I am seeing inside of -iteI;6. +here are no e5tra files on my machine* -iteI;6 is streaming the o)tp)t of 9odoc directly into the screen. =ery smart way of doing things,, 3o if I can see doc)mentation for the standard 9o packages* then the so)rce code for those packages m)st be on my machine. #fter a bit of looking I fo)nd them inB /usr/local/go/src/pkg It seems they are located inside a folder called pkg )nder src. +his is beca)se the 9o team likes to p)t so)rce code for re)sable libraries within a pro0ect )nder pkg. &ot all developers follow that same convention and yo) have the freedom to choose. I personally don.t follow that convention. #pparently the 9odoc tool has no problems finding the so)rce code files. 9odoc tool is always reading the so)rce code files to prod)ce the latest version of the doc)mentation. 3o in -iteI;6 when yo) )pdate yo)r doc)mentation and save the code file* the 9odoc tool will show the changes immediately. &ow the ne5t problem I have* my doc)mentation looks really bad. +he doc)mentation that I see from the standard library files looks m)ch better. 3o how do I format my doc)mentation properly inside the 9o code filesD I fo)nd this doc)ment from the 9o teamB httpB golang.org doc articles godocFdoc)mentingFgoFcode.html +he introd)ction readsB

Godoc: documenting Go code


The Go pro/ect takes doc'mentation serio'sly. ;oc)mentation is a h)ge part of making software accessible and maintainable. 'f co)rse it m)st be wellCwritten and acc)rate* b)t it also m)st be easy to write and to maintain. Ideally* it sho)ld be co)pled to the code itself so 2?

the doc)mentation evolves along with the code. +he easier it is for programmers to prod)ce good doc)mentation* the better for everyone. +o that end* we have developed the godoc doc)mentation tool. +his article describes godoc.s approach to doc)mentation* and e5plains how yo) can )se o)r conventions and tools to write good doc)mentation for yo)r own pro0ects. 9odoc parses 9o so)rce code C incl)ding comments C and prod)ces doc)mentation as E+Mor plain te5t. +he end res)lt is doc)mentation tightly co)pled with the code it doc)ments. For e5ample* thro)gh godoc.s web interface yo) can navigate from a f)nction.s doc)mentation to its implementation with one click. Coming from the C% world and )sing NM- tags like Qs)mmaryR for that past ?0 years and having to remember to check the Kprod)ce NM- doc)mentation fileK option* this was a dream. 'h yea* no e5tra doc)mentation file. Eowever the rest of the page was lacking. I liked the way the doc)mentation for fmt.$rintf looked so I 2)ickly fo)nd the go so)rce files and st)died what the programmer did. #fter a bit of playing I finally fig)red o)t the " basic r)les yo) need to help the 9odoc tool format the doc)mentation cleanly. Eere is a sample of the doc)mentation I have for my tracelog packageB

+here are " elements in play when writing yo)r doc)mentation. :o) have header sections* standard te5t and highlighted te5t. #t the very top of yo)r code file add the following )sing the comment operation or something similar. 'bvio)sly yo) want to give yo)rself the credit for yo)r work* -'-. // 5opyright E10I .r)an /tu)ios. .ll rights reserve). // 6se of this source co)e is governe) by a !/&Kstyle // license that can be foun) in the L:5@N/@ file.

22

+hen add a block comment operator and we can start. Make s)re the package code statement is e5actly after the closing comment operator. +here can not be no blank lines between the two. +abbing is very important. We are )sing two layers of tabbing. Keep these two layers of tabbing consistent. /$ KH .! #ackage raceLog implements a file base) logging. KH .! he programmer shoul) feel free to tace log as much of the co)e. KH5-LM KH .! New #arameters KH5-LM KH .! he following is a list of parameters for creating a raceLog( KH .! baseMile#ath( he base location to store all log )irectories. KH .! machineName( he name of the machine or system. :nformation is use). KH .! write o/t)out( /et to rue if you want the system to also write. KH5-LM KH .! raceLog Mile Management KH5-LM KH .! @very E minutes raceLog will check each open file for two con)itions( KH5-LM KH .! 0. Aas a message been written to the file within the last E minutes. KH .! E. :s the siNe of the file greater than 01 Meg. $/ package tracelog +he first section of comments will show at the top of o)r doc)mentation 0)st below the 'verview 3ection. #lso the first sentence will appear in 9odoc.s package list. +hen we have a blank line and a string that will become a header as long as the ne5t line is do)ble spaced and has the same indentation. +he final component is the second tabbing indentation. +his will ca)se that te5t to be highlighted with a grey backgro)nd. :o) may need to remove all of yo)r e5isting doc)mentation from yo)r 9o code file and throw it into a te5t editor. +hen p)t it back in to make s)re all the tabs and carriage ret)rns are clean.

7sing 9o;oc.org
2"

If yo) are b)ilding a p)blic package yo) can )se the 9o;oc website to p)blish yo)r doc)mentation. Check o)t the 9o;oc websiteB httpB godoc.org +his website has been set)p to read yo)r code files and display all of yo)r great doc)mentation. 6nter this )rl @gith)b.com goinggo )tilities v? workpoolA into the search bo5 and see the doc)mentation that 9o;oc prod)ces for my workpool packageB

:o) can see the same doc)mentation that is being given to yo) locally is now p)blished on the 9o;oc website with yo)r own re)seable )rlB httpB godoc.org gith)b.com goinggo )tilities v? workpool 3o how can yo) best )se this )rl to provide people yo)r doc)mentationD When yo) create a repository for yo)r package add a 16#;M6.md file. +his is a special KMarkdownK file that s)pports standard te5t* html and a few special operators of its own. 9ith)b has its own e5tensions and yo) can find doc)mentation abo)t Markdown hereB httpsB help.gith)b.com articles gith)bCflavoredCmarkdown If yo) happened to come across my p)blic workpool package in 9ith)b* yo) wo)ld see the followingB

28

+here is my code file* license file and my readme Markdown file. Eere is a typical 16#;M6 Markdown file that I )seB O Workpool K Persion 0.1.1 5opyright E10I .r)an /tu)ios. .ll rights reserve).=br /H 6se of this source co)e is governe) by a !/&Kstyle license that can be foun) in the L:5@N/@ han)le. .r)an /tu)ios=br /H 0EQRI /W 00E / + /uite 0FI=br /H Miami+ ML II08S=br /H billTar)anstu)ios.com=br /H U5lick o Piew &ocumentationV (http(//go)oc.org/github.com/goinggo/utilities/v0/workpool) =iew +he 1aw =ersion EereB httpsB raw.gith)b.com goinggo )tilities master v? workpool 16#;M6.md -ook at the Markdown link at the bottom of the file. +his synta5 creates a link to the doc)mentation. +he te5t in the hard brackets ST* provides the anchor te5t for the link. 3ince 9ith)b always display the 1eadme Markdown file to the )ser if one e5ists* this is what people see when they come to that 9ith)b pageB

2G

&ow people have access to the doc)mentation I write on the web as well. I don.t need to copy and paste the doc)mentation into the 1eadme Markdown file* 0)st provide a link. #ll the doc)mentation is in one place and formatted cleanly and consistently. #s always* I hope this helps yo) in some small way and yo)r doc)mentation draws people to yo)r work.

.nderstanding De6er, Panic and >eco:er


I am b)ilding my +race-og package and it is really important that the package logs any internal e5ceptions and prevents panics from sh)tting down the application. +he +race-og package m)st never be responsible for sh)tting down an application. I also have internal go ro)tines that m)st never terminate )ntil the application is sh)t down gracef)lly. 7nderstanding how to )se ;efer and 1ecover in yo)r application can be a bit tricky at first* especially if yo) are )sed to )sing try catch blocks. +here is a pattern yo) can implement to provide that same type of try catch protection in 9o. /efore I can show yo) this yo) need to learn how ;efer* $anic and 1ecover work. First yo) need to )nderstand the intricacies of the keyword defer. 3tart with this piece of codeB package main import ( "errors" "fmt" ) func main() { est() % func Mimic@rror(key string) error { 2H

return errors.New(fmt./printf("Mimic @rror ( 's"+ key)) % func est() { fmt.#rintf("/tart est*n")

err (7 Mimic@rror("0") defer func() { fmt.#rintf("/tart &efer*n") if err ?7 nil { fmt.#rintf("&efer @rror ( 'v*n"+ err) % %() fmt.#rintf("@n) % +he Mimic6rror f)nction is a test f)nction that will be )sed to sim)late an error. It is following the 9o convention of )sing the error type to ret)rn an indication if something went wrong. In 9o the error type is defined as an interfaceB type error interface { @rror() string % If yo) don.t )nderstand what a 9o interface is at the moment then this might help for now. #ny type that implements the Arror89 f)nction implements this interface and can be )sed as a variable of this type. +he Mimic6rror f)nction is )sing errors.Bew8string9 to create an error type variable. +he errors type can be fo)nd in the errors package. +est f)nction prod)ces the following o)tp)tB /tart est @n) est /tart &efer &efer @rror ( Mimic @rror ( 0 When yo) st)dy the o)tp)t yo) see that the +est f)nction started and ended. +hen right before the +est f)nction terminated for good* the inline defer f)nction was called. +wo interesting things are happening here. First* the defer keyword is deferring the e5ec)tion of the inline f)nction )ntil the +est f)nction ends. 3econd* beca)se 9o s)pports clos)re* the err variable is accessible to the inline f)nction and its message KMimic 6rror B ?K is written to stdo)t. :o) can define a defer f)nction at any time inside yo)r f)nction. If that defer f)nction re2)ires state* as in this case with the err variable* then it m)st e5ist before the defer f)nction is defined. 2I est*n")

&ow change the +est f)nction a bitB func est() { fmt.#rintf("/tart est*n")

err := MimicError("1") )efer func() { fmt.#rintf("/tart &efer*n") if err ?7 nil { fmt.#rintf("&efer @rror ( 'v*n"+ err) % %() err = MimicError("2") fmt.#rintf("@n) % +his time the code is calling the Mimic6rror f)nction a second time after creating the inline defer f)nction. Eere is the o)tp)tB /tart est @n) est /tart &efer &efer @rror ( Mimic @rror ( 2 +he o)tp)t is identical from the first test e5cept for one change. +his time the inline defer f)nction wrote KMimic 6rror B 2K. It appears that the inline defer f)nction has a reference to the err variable. 3o if the state of the err variable changes at any time before the inline defer f)nction is called* yo) will see that val)e. +o verify that the inline defer f)nction is getting a reference to the err variable* change the code to write the address of the err variable in both the +est f)nction and the inline defer f)nction. func est() { fmt.#rintf("/tart est*n") est*n")

err (7 Mimic@rror("0") fmt.Printf("Err Addr: %v\n", )efer func() { fmt.#rintf("/tart &efer*n") if err ?7 nil { fmt.Printf("Err Addr !efer: %v\n", err) fmt.#rintf("&efer @rror ( 'v*n"+ err) % %() 2L err)

err 7 Mimic@rror("E") fmt.#rintf("@n) % #s yo) can see from the o)tp)t below* the inline defer f)nction has the same reference to the err variable. +he address is the same inside of the +est f)nction and inside of the inline defer f)nction. /tart est @rr .))r( "#21"1$%2"" @n) est /tart &efer @rr .))r &efer( "#21"1$%2"" &efer @rror ( Mimic @rror ( E #s long as the defer f)nction is stated before the +est f)nction terminates* the defer f)nction will be e5ec)ted. +his is great* b)t what I want is the ability to always place the defer statement at the beginning of any f)nction. +his way the defer f)nction is g)aranteed to be called every time the f)nction is e5ec)ted and I don.t have to over think where to place the defer statement. 'ccam.s 1a(or applies hereB ,When yo' ha:e t"o competing theories that make e@actly the same predictions, the simpler one is the better,. What I want is an easy pattern that can be d)plicated witho)t re2)iring any tho)ght. +he only problem is that the err variable needs to be defined before the defer statement can be implemented. Fort)nately 9o allows ret)rn variables to have names. &ow change the entire program as followsB package main import ( "errors" "fmt" ) func main() { var err error err 7 est() est*n")

if err ?7 nil { fmt.#rintf("Main @rror( 'v*n"+ err) % % func Mimic@rror(key string) error { return errors.New(fmt./printf("Mimic @rror ( 's"+ key)) %

2<

func est() (err error) { )efer func() { fmt.#rintf("/tart &efer*n") if err ?7 nil { fmt.#rintf("&efer @rror ( 'v*n"+ err) % %() fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") fmt.#rintf("@n) return err % +he +est f)nction now defines a ret)rn variable called err of type error. +his is great beca)se the err variable e5ists immediately and yo) can p)t the defer statement at the very beginning of the f)nction. #lso* the +est f)nction now follows the 9o convention and ret)rns an error type back to the calling ro)tine. When yo) r)n the program yo) get the following o)tp)tB /tart est @n) est /tart &efer &efer @rror ( Mimic @rror ( 0 Main @rror( Mimic @rror ( 0 &ow it is time to talk abo)t the b)iltCin f)nction panic. When any 9o f)nction calls panic the normal flow of the application stops. +he f)nction that calls panic ends immediately and ca)ses a chain reaction of panics )p the call stack. #ll the f)nctions in the same call stack will end* one after the ne5t* like dominos falling down. 6vent)ally the panic reaches the top of the call stack and the application crashes. 'ne good thing is that any e5isting defer f)nctions will be e5ec)ted d)ring this panic se2)ence and they have the ability to stop the crash. -ook at this new +est f)nction that calls the b)ilt in panic f)nction and recovers from the callB func est() (err error) { )efer func() { fmt.#rintf("/tart #anic &efer*n") if r := recover()& r '= ni( ) fmt.Printf("!efer Panic : %v\n", r) * %() est*n")

"0

fmt.#rintf("/tart

est*n")

panic("Mimic Panic") fmt.#rintf("@n) return err % -ook closely at the new inline defer f)nctionB )efer func() { fmt.#rintf("/tart #anic &efer*n") if r := recover()& r '= ni( ) fmt.Printf("!efer Panic : %v\n", r) * %() +he inline defer f)nction is now calling another b)iltCin f)nction reco:er. +he recover f)nction stops the chain reaction from going any farther )p the call stack. It is like swiping a domino away so no more can fall down. +he recover f)nction can only be )sed inside of a defer f)nction. +his is beca)se d)ring the panic chain reaction only defer f)nctions will be e5ec)ted. If the recover f)nction is called and there is no panic occ)rring* the recover f)nction will ret)rn nil. If there is a panic occ)rring* then the panic is stopped and the val)e given to the panic call will be ret)rned. +his time the code is not calling the Mimic6rror f)nction b)t the b)ilt in panic f)nction to sim)late a panic. -ook at the o)tp)t from r)nning the codeB /tart est /tart #anic &efer &efer #anic ( Mimic #anic +he inline defer f)nction capt)res the panic* prints it to the screen and stops it dead in its tracks. #lso notice that K6nd +estK is never displayed. +he f)nction terminated as soon as panic was called. +his is great b)t if there is an error I still want to display that as well. 3omething cool abo)t 9o and the defer keyword is that yo) can have more than one defer f)nction stated at a time. Change the +est f)nction as followsB func est() (err error) { defer f+nc() { fmt.#rintf("/tart #anic &efer*n") if r (7 recover()< r ?7 nil { fmt.#rintf("&efer #anic ( 'v*n"+ r) "? est*n")

% %() defer f+nc() { fmt.#rintf("/tart &efer*n") if err ?7 nil { fmt.#rintf("&efer @rror ( 'v*n"+ err) % %() fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") panic("Mimic #anic") fmt.#rintf("@n) return err % &ow both inline defer f)nctions have been incorporated into beginning of the +est f)nction. First the inline defer f)nction that that recovers from panics and then the inline defer f)nction that prints errors. 'ne thing to note is that 9o will e5ec)te these inline defer f)nctions in the opposite order that are defined @First In C -ast ')tA. 1)n the code and look at the o)tp)tB /tart est /tart @rror &efer @rror /tart #anic &efer #anic Main @rror( &efer ( Mimic @rror ( 0 &efer ( Mimic #anic Mimic @rror ( 0 est*n")

+he +est f)nction starts as e5pected and the call to panic halts the e5ec)tion of the +est f)nction. +his ca)ses the inline defer f)nction that prints errors to get called first. 3ince the +est f)nction called the Mimic6rror f)nction before the panic* the error is printed. +hen the inline defer f)nction that recovers from panics is called and the panic is recovered. There is one problem "ith this code. +he main f)nction had no idea that a panic was averted. #ll the main f)nction knows is that an error occ)rred thanks to the Mimic6rror f)nction call. +his is not good. I want the main f)nction to know abo)t the error that ca)sed the panic. +hat is really the error that m)st be reported. In the inline defer f)nction that handles the panic we need to assign the error that ca)sed the panic to the err variable. func est() (err error) { )efer func() { "2

fmt.#rintf("/tart #anic &efer*n") if r (7 recover()< r ?7 nil { fmt.#rintf("&efer #anic ( 'v*n"+ r) err = error,.-e.(fmt./printf("%v", r)) % %() )efer func() { fmt.#rintf("/tart &efer*n") if err ?7 nil { fmt.#rintf("&efer @rror ( 'v*n"+ err) % %() fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") panic("Mimic #anic") fmt.#rintf("@n) return err % &ow when yo) r)n the code yo) get the following o)tp)tB /tart est /tart @rror &efer @rror /tart #anic &efer #anic Main Error: &efer ( Mimic @rror ( 0 &efer ( Mimic #anic Mimic Panic est*n")

+his time when main f)nction reports the error that ca)sed the panic. 6verything looks good b)t this code is not really scalable. Eaving two inline defer f)nctions is cool b)t not practical. What I need is a single f)nction that can handle both errors and panics. Eere is a revised version of the f)ll program with a new f)nction called FCatch$anicB package main import ( "errors" "fmt" ) ""

func main() { var err error err 7 est()

if err ?7 nil { fmt.#rintf("Main @rror( 'v*n"+ err) % % f+nc 01atc2Panic(err error, f+nction-ame ,tring) ) if r := recover()& r '= ni( ) fmt.Printf("%, : PA-31 !efered : %v\n", f+nction-ame, r) if err '= ni( ) err = error,.-e.(fmt./printf("%v", r)) * * e(,e if err '= ni( ) fmt.Printf("%, : E4454 : %v\n", f+nction-ame, err) * * func Mimic@rror(key string) error { return errors.New(fmt./printf("Mimic @rror ( 's"+ key)) % func est() (err error) { defer 01atc2Panic(err, "6e,t") fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") fmt.#rintf("@n) return err % +he new f)nction FCatch$anic incorporates both the panic recover and error handling. +his time instead of defining an inline defer f)nction the code is )sing an e5ternal f)nction for the defer statement. In this first test with the new FCatch$anic defer f)nction* we need to make s)re we didn.t break o)r error handling. 1)n the code and look at the o)tp)tB /tart est @n) est Main @rror( Mimic @rror ( 0 "8 est*n")

6verything looks good. &ow we need to test a panic. func est() (err error) { defer 01atc2Panic(err, "6e,t") fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") panic("Mimic #anic") fmt.#rintf("@n) return err % 1)n the code and look at the o)tp)tB /tart est estF ( #.N:5 &efere) ( Mimic #anic Main @rror( Mimic @rror ( 0 *o'ston "e ha:e a problem. Main was provided the error from the Mimic6rror f)nction call and not from the panic. What went wrongD func W5atch#anic(err error+ functionName string) { if r (7 recover()< r ?7 nil { fmt.#rintf("'s ( #.N:5 &efere) ( 'v*n"+ functionName+ r) if err ?7 nil { err = error,.-e.(fmt./printf("%v", r)) % % else if err ?7 nil { fmt.#rintf("'s ( @--3- ( 'v*n"+ functionName+ err) % % /eca)se defer is now calling an e5ternal f)nction the code lost all the goodness that came with inline f)nctions and Clos)re. Change the code to print the address of the err variable from inside the +est f)nction andFCatch$anic defer f)nction. func W5atch#anic(err error+ functionName string) { if r (7 recover()< r ?7 nil { fmt.#rintf("'s ( #.N:5 &efere) ( 'v*n"+ functionName+ r) fmt.Printf("Err Addr !efer: %v\n", if err ?7 nil { err) est*n")

"G

err 7 errors.New(fmt./printf("'v"+ r)) % % else if err ?7 nil { fmt.#rintf("'s ( @--3- ( 'v*n"+ functionName+ err) % % func est() (err error) { fmt.Printf("Err Addr: %v\n", err)

)efer W5atch#anic(err+ " estR") fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") panic("Mimic #anic") fmt.#rintf("@n) return err % When yo) r)n the code yo) can see why main did not get the error from the panic. @rr .))r( "#21"1$%1f" /tart est estF ( #.N:5 &efere) ( Mimic #anic @rr .))r &efer( "#21"1$%27" Main @rror( Mimic @rror ( 0 When the +est f)nction passes the err variable to the FCatch$anic defer f)nction it is passing the variable by val)e. In 9o all arg)ments are passed by val)e. 3o the FCatch$anic defer f)nction has its own copy of the err variable. #ny changes to FCatch$anic.s copy remains with FCatch$anic. +o fi5 the pass by val)e problem the code needs to pass the err variable by reference. package main import ( "errors" "fmt" ) func main() { var err error err 7 estMinal() est*n")

if err ?7 nil { "H

fmt.#rintf("Main @rror( 'v*n"+ err) % % func W5atch#anic(err 8error+ functionName string) { if r (7 recover()< r ?7 nil { fmt.#rintf("'s ( #.N:5 &efere) ( 'v*n"+ functionName+ r) if err ?7 nil { 8err 7 errors.New(fmt./printf("'v"+ r)) % % else if err ?7 nil GG 8err ?7 nil { fmt.#rintf("'s ( @--3- ( 'v*n"+ functionName+ 8err) % % func Mimic@rror(key string) error { return errors.New(fmt./printf("Mimic @rror ( 's"+ key)) % func estMinal() (err error) { defer 01atc2Panic( err, "6e,t9ina(") fmt.#rintf("/tart est*n")

err 7 Mimic@rror("0") panic("Mimic #anic") fmt.#rintf("@n) return err % &ow r)n the code and look at the o)tp)tB /tart est estS ( #.N:5 &efere) ( Mimic #anic Main @rror( Mimic #anic +he main f)nction now reports the error that occ)rred beca)se of the panic. If yo) want to capt)re a stack trace as well 0)st make this change to FCatch$anic. 1emember to import Kr)ntimeK. func W5atch#anic(err $error+ functionName string) { if r (7 recover()< r ?7 nil { fmt.#rintf("'s ( #.N:5 &efere) ( 'v*n"+ functionName+ r) :: 1apt+re t2e ,tack trace $+f := make(;<$=te, 1"""") "I est*n")

r+ntime./tack($+f, fa(,e) fmt.Printf("%, : /tack 6race : %,", f+nction-ame, ,tring($+f)) if err ?7 nil { $err 7 errors.New(fmt./printf("'v"+ r)) % % else if err ?7 nil GG $err ?7 nil { fmt.#rintf("'s ( @--3- ( 'v*n"+ functionName+ $err) % % With this pattern yo) can implement 9o ro)tines that can handle errors and trap panic sit)ations. In many cases these conditions 0)st need to be logged or reported )p the call stack to be handled gracef)lly. Eaving a single place to implement this type of code and a simple way to integrate it into each f)nction will red)ce errors and keep yo)r code clean. Eowever I have learned it is best to only )se this pattern to catch panics only. -eave the logging of errors to the application logic. If not then yo) may be logging the errors twice. func W5atch#anic(err $error+ functionName string) { if r (7 recover()< r ?7 nil { fmt.#rintf("'s ( #.N:5 &efere) ( 'v*n"+ functionName+ r) :: 1apt+re t2e ,tack trace $+f := make(;<$=te, 1"""") r+ntime./tack($+f, fa(,e) fmt.Printf("%, : /tack 6race : %,", f+nction-ame, ,tring($+f)) if err ?7 nil { $err 7 errors.New(fmt./printf("'v"+ r)) % % % #s always I hope this can help yo) with yo)r 9o programming.

GoAs time5D'ration Type .nra:elled


I have been str)ggling with )sing the +ime package that comes in the 9o standard library. My str)ggles have come from two pieces of f)nctionality. First* trying to capt)re the n)mber of milliseconds between two different time periods. 3econd* comparing that d)ration in milliseconds against a preCdefined time span. It so)nds like a no brainier b)t like I said* I have been str)ggling. In the +ime package there is a c)stom type called ;)ration and a set of helper constantsB

"L

type &uration intSJ const ( Nanosecon) &uration 7 0 Microsecon) 7 0111 $ Nanosecon) Millisecon) 7 0111 $ Microsecon) /econ) 7 0111 $ Millisecon) Minute 7 S1 $ /econ) Aour 7 S1 $ Minute ) I looked at this maybe ?000 times b)t it didn.t mean anything to me. I 0)st want to compare two time periods* get the d)ration back* compare the d)ration and do something if the right amo)nt of time has elapsed. I co)ld not for the life of me )nderstand how this str)ct)re was going to help me )nravel this mystery. I wrote this +est f)nction b)t it didn.t work. func est() { var waitMiveAun)re)Millisections intSJ 7 F11 starting ime (7 time.Now().6 5() time./leep(01 $ time.Millisecon)) en)ing ime (7 time.Now().6 5() var )uration time.&uration 7 en)ing ime./ub(starting ime) var )uration.s:ntSJ 7 intSJ()uration) if )uration.s:ntSJ H7 waitMiveAun)re)Millisections { fmt.#rintf(" ime @lapse) ( WaitU')V &urationU')V*n"+ waitMiveAun)re)Millisections+ )uration.s:ntSJ) % else { fmt.#rintf(" ime &:& N3 @lapse) ( WaitU')V &urationU')V*n"+ waitMiveAun)re)Millisections+ )uration.s:ntSJ) % % When I ran the +est f)nction and look at the o)tp)t the code thinks that G00 milliseconds have elapsed. ime @lapse) ( WaitUF11V &urationU01REJRQ8V 3o what went wrongD I looked at the ;)ration type and constants again. type &uration intSJ const ( Nanosecon) &uration 7 0 Microsecon) 7 0111 $ Nanosecon) "<

Millisecon) 7 0111 $ Microsecon) /econ) 7 0111 $ Millisecon) Minute 7 S1 $ /econ) Aour 7 S1 $ Minute ) +he basic )nit of time for the ;)ration type is a &anosecond. #h* that is why when casting a ;)ration type that contains ?0 milliseconds to an intH8 I get ?0*000*000. 3o direct casting is not going to work. I need a different strategy and a better )nderstanding of how to )se and convert the ;)ration type. I know it wo)ld be best to )se the ;)ration type natively. +his will minimi(e problems when )sing the type. /ased on the constant val)es* I can create a ;)ration type variable in the following waysB func est() { var )urationWMillisecon)s time.!+ration 7 F11 $ time.Millisecon) var )urationW/econ)s time.!+ration 7 (0EF1 $ 01) $ time.Millisecon) var )urationWMinute time.!+ration 7 E $ time.Minute fmt.#rintf("Milli U'vV*n/econ)s U'vV*nMinute U'vV*n"+ )urationWMillisecon)s+ )urationW/econ)s+ )urationWMinute) % I created " variables of type ;)ration. +hen by )sing the time constants* I am able to create the correct d)ration time span val)es. When I )se the standard library $rintf f)nction and the Uv operator I get the following o)tp)tB Milli UF11msV /econ)s U0E.FsV Minute UEm1sV +his is very cool. +he $rintf f)nction knows how to natively display a ;)ration type. /ased on the val)e in each ;)ration variable* the $rintf f)nction prints the val)e in the proper time period. I am also getting the val)es I e5pected. +he ;)ration type does have member f)nctions that will convert the val)e of the ;)ration variable to a native 9o type* either in intH8 or floatH8 respectivelyB func est() { var )urationW/econ)s time.&uration 7 (0EF1 $ 01) $ time.Millisecon) var )urationWMinute time.&uration 7 E $ time.Minute var floatSJW/econ)s f(oat>? 7 )urationW/econ)s./econd,() var floatSJWMinutes f(oat>? 7 )urationWMinute.Min+te,() 80

fmt.#rintf("/econ)s U'.IfV*nMinutes U'.EfV*n"+ floatSJW/econ)s+ floatSJWMinutes) % I noticed pretty 2)ickly that there is no Milliseconds f)nction. +here is a f)nction for every other time )nit b)t Milliseconds. When I display the 3econds and Min)tes natively I get the following o)tp)t as e5pectedB /econ)s U0E.F11V Minutes UE.11V Eowever I need the Milliseconds* 3o why is the Milliseconds f)nction missingD +he designers of 9o didn.t want to lock me into a single native type for the Milliseconds. +hey wanted me to have options. +he following code converts the val)e of the ;)ration variable to Milliseconds as both an intH8 and floatH8B func est() { var )urationWMillisecon)s time.&uration 7 F11 $ time.Millisecon) var cast o:ntSJ int>? 7 )urationWMillisecon)s.Nanosecon)s() : 1e> var cast oMloatSJ f(oat>? 7 )urationWMillisecon)s./econ)s() 8 1e% fmt.#rintf("&uration U'vV*ncast o:ntSJ U')V*ncast oMloatSJ U'.1fV*n"+ )urationWMillisecon)s+ cast o:ntSJ+ cast oMloatSJ) % If I divide the &anoseconds by ?eH I get the Milliseconds as a intH8. If I m)ltiply the 3econds by ?e" I get the Milliseconds as a floatH8. Eere is the o)tp)tB &uration UF11msV cast o:ntSJ UF11V cast oMloatSJ UF11V If yo) are wondering what ?eH or ?e" represents yo) are not aloneB 0eI 7 01I 7 3ne housan)

0eS 7 01S 7 3ne Million 8?

&ow that I )nderstand what a ;)ration type is and how best to )se and manip)late it* I have my final test e5ample )sing MillisecondsB func est() { var waitMiveAun)re)Millisections time.&uration 7 F11 $ time.Millisecon) starting ime (7 time.Now().6 5() time./leep(S11 $ time.Millisecon)) en)ing ime (7 time.Now().6 5() var )uration time.&uration 7 en)ing ime./ub(starting ime) if )uration H7 waitMiveAun)re)Millisections { fmt.#rintf("Wait 'v*nNative U'vV*nMillisecon)s U')V*n/econ)s U'.IfV*n"+ waitMiveAun)re)Millisections+ )uration+ )uration.Nanosecon)s()/0eS+ )uration./econ)s()) % % I get the following o)tp)tB Wait F11ms Native US10.1Q01SSmsV Millisecon)s US10V /econ)s U1.S10V Instead of comparing native types to determine if the time has elapsed I am comparing two ;)ration types. +his is m)ch cleaner. When displaying the val)es I am )sing the ;)ration type c)stom formatting and converting the val)e of the ;)ration variable to Milliseconds as both an intH8 and floatH8. It took a while b)t event)ally )sing the ;)ration type started to make sense. #s always I hope this helps someone else navigate )sing the ;)ration type in their 9o applications.

Send an email in Go "ith smtp5Send2ail


I wanted to send an email from my +race-og package when a critical e5ception occ)rred. Fort)nately 9o.s standard library has a package called smpt which can be fo)nd inside the net package. When yo) look at the doc)mentation yo) are left wanting. I spent 20 min)tes researching how to )se this package. #fter fighting thro)gh the parameters and b)gs* I came )p with this sample codeB package main import ( 82

"bytes" "errors" "fmt" "net/smtp" "runtime" "strings" "teBt/template" ) func main() { /en)@mail( "smtp.0an)0.com"+ F8R+ "usernameT)omain.com"+ "passwor)"+ UVstring{"meT)omain.com"%+ "testing subDect"+ "=htmlH=bo)yH@Bception 0=/bo)yH=/htmlH@Bception 0") % func W5atch#anic(err $error+ functionName string) { if r (7 recover()< r ?7 nil { fmt.#rintf("'s ( #.N:5 &efere) ( 'v*n"+ functionName+ r) // 5apture the stack trace buf (7 make(UVbyte+ 01111) runtime./tack(buf+ false) fmt.#rintf("'s ( /tack string(buf)) race ( 's"+ functionName+

if err ?7 nil { $err 7 errors.New(fmt./printf("'v"+ r)) % % else if err ?7 nil GG $err ?7 nil { fmt.#rintf("'s ( @--3- ( 'v*n"+ functionName+ $err) // 5apture the stack trace buf (7 make(UVbyte+ 01111) runtime./tack(buf+ false) fmt.#rintf("'s ( /tack string(buf)) % % race ( 's"+ functionName+

func /en)@mail(host string+ port int+ userName string+ passwor) string+ to UVstring+ subDect string+ message string) (err error) { )efer W5atch#anic(Gerr+ "/en)@mail") 8"

parameters (7 Gstruct { Mrom string o string /ubDect string Message string %{ userName+ strings.9oin(UVstring(to)+ "+")+ subDect+ message+ % buffer (7 new(bytes.!uffer) template (7 template.Must(template.New("email emplate").#arse(W@mail/cript ())) template.@Becute(buffer+ parameters) auth (7 smtp.#lain.uth(""+ userName+ passwor)+ host) err 7 smtp./en)Mail( fmt./printf("'s(')"+ host+ port)+ auth+ userName+ to+ buffer.!ytes()) return err % // W@mail/cript returns a template for the email message to be sent func W@mail/cript() (script string) { return XMrom( {{.Mrom%% o( {{. o%% /ubDect( {{./ubDect%% M:M@Kversion( 0.1 5ontentK ype( teBt/html< charset7"6 MK8" {{.Message%%X % +he a)th variable does not have to be recreated on every call. +hat can be created once and re)sed. I added my FCatch$anic f)nction so yo) can see any e5ceptions that are ret)rned while testing the code. If yo) look at the raw so)rce of the email yo) will see how the message parameter worksB

88

-eturnK#ath( &eliveryK&ate( We)+ 0E 9un E10I E1(IJ(FQ K1J11 -eceive)( from mout.perfora.net (mout.perfora.net U4.4.4.4V) by mB.perfora.net (no)e7mBus1) with @/M # (Nemesis) i) 1MY 5nK0PI@#0I for meT)omain.com< We)+ 0E 9un E10I E1(IJ(F8 K1J11 -eceive)( from localhost (cKF1K0JIKI0K0F0.hs)0.fl.comcast.net U4.4.4.4V) by mrelay.perfora.net (no)e7mrusJ) with @/M # (Nemesis) i) 1MhiJ-K0P1-uNJ8otK11M cS< We)+ 0E 9un E10I E1(IJ(F8 K1J11 9rom: +,ername@domain.com 6o: me@domain.com /+$Aect: te,ting ,+$Aect M3MEBver,ion: 1." 1ontentB6=pe: te#t:2tm(& c2ar,et="C69BD" MessageK:)( =1mhiJrK )ataKbloggerKescape)K mrelay.perfora.net7""H &ate( We)+ 0E 9un E10I E1(IJ(FS K1J11 @nvelopeK o( meT)omain.com E2tm(FE$od=FE#ception 1E:$od=FE:2tm(F #s always I hope this sample program saves yo) time and aggravation.

>eading B2L Doc'ments in Go


I was really s)rprised how easy it was to read an NM- doc)ment )sing the encoding 5ml package that comes with the standard library. +he package works by defining str)cts that map the NM- doc)ment. If yo) need more fle5ibility then )se 9)stavo &iemeyer.s 5mlpath package @fo)nd hereA. Eere is the NM- doc)ment we are going to read and deCseriali(eB =strapsH =strap key7"5ompanyName" value7"N@W53" /H =strap key7"6se@mail" value7"true" /H =/strapsH +he first thing we need to do is define the str)cts we will )se to map the doc)mentB type 4ML/trap struct { 4MLName Bml.Name XBml("strap"X ;ey string XBml("key+attr"X Palue string XBml("value+attr"X % type 4ML/traps struct { 4MLName Bml.Name XBml("straps"X 8G

/traps %

UV$4ML/trap XBml("strap"X

+here are two str)cts* one for the entire doc)ment @QstrapsRA and one for each individ)al child node @QstrapRA. If yo) look closely at the str)cts yo) may see something new. 6ach field has a tag associated with it. +hese tags are bo)nd to each individ)al field. 9o.s reflect package allows yo) to access these tags. +hese tag formats are specific to the decoding s)pport inside the encoding 5ml package. +he tags map the nodes and attrib)tes of the NM- doc)ment to the str)ct. +he following code decodes the NM- doc)ment and ret)rns the array of strap nodesB func -ea)/traps(rea)er io.-ea)er) (UV$4ML/trap+ error) { Bml/traps (7 G4ML/traps{% )eco)er (7 Bml.New&eco)er(rea)er) if err (7 )eco)er.&eco)e(Bml/traps)< err ?7 nil { return nil+ err % return Bml/traps./traps+ nil % +he f)nction takes an io.1eader. We will be passing a os.File variable into this method. +he f)nction ret)rns an array of pointers for each strap we read from the file. First we create a NM-3traps variable and get its address. &e5t we create a decoder )sing the 5ml.&ew;ecoder method passing the io.1eader ob0ect. +hen we call ;ecode which reads the file and deCseriali(es the file into the NM-3traps variable. +hen we 0)st ret)rn the array of strap val)es. +he following completes the sample codeB /$ straps.Bml shoul) be locate) in the )efault working )irectory =strapsH =strap key7"5ompanyName" value7"N@W53" /H =strap key7"6se@mail" value7"true" /H =/strapsH $/ package main import ( "enco)ing/Bml" "fmt" "io" "os" "path/filepath" ) 8H

type 4ML/trap struct { 4MLName Bml.Name XBml("strap"X ;ey string XBml("key+attr"X Palue string XBml("value+attr"X % type 4ML/traps struct { 4MLName Bml.Name XBml("straps"X /traps UV$4ML/trap XBml("strap"X % func -ea)/traps(rea)er io.-ea)er) (UV$4ML/trap+ error) { Bml/traps (7 G4ML/traps{% )eco)er (7 Bml.New&eco)er(rea)er) if err (7 )eco)er.&eco)e(Bml/traps)< err ?7 nil { return nil+ err % return Bml/traps./traps+ nil % func main() { var Bml/traps UV$4ML/trap var file $os.Mile )efer func() { if file ?7 nil { file.5lose() % %() // !uil) the location of the straps.Bml file // filepath..bs appen)s the file name to the )efault working )irectly strapsMile#ath+ err (7 filepath..bs("straps.Bml") if err ?7 nil { panic(err.@rror()) % // 3pen the straps.Bml file file+ err 7 os.3pen(strapsMile#ath) if err ?7 nil { panic(err.@rror()) % // -ea) the straps file Bml/traps+ err 7 -ea)/traps(file) 8I

if err ?7 nil { panic(err.@rror()) % // &isplay he first strap fmt.#rintf(";ey( 's Palue( 's"+ Bml/trapsU1V.;ey+ Bml/trapsU1V.Palue) % I hope this sample gets yo) started with reading NM- doc)ments for yo)r 9o applications.

>'nning Go Programs as a Backgro'nd Process


I have been writing Windows services in C C!! and then in C% since ?<<<. &ow that I am writing server based software in 9o for the -in)5 '3 I am completely lost. What is even more fr)strating* is that for the first time the '3 I am developing on @Mac '3NA is not the operating system I will be deploying my code on. +hat will be for another blog post. I want to r)n my code as a backgro)nd process @daemonA on my Mac. My only problem is* I have no idea how that works on the Mac '3. I was l)cky to find an open so)rce pro0ect called service on /itb)cket by ;aniel +heophanes. +his code ta)ght me how to create* install* start and stop daemons on the Mac '3. +he code also s)pports daemons for the -in)5 '3 and Windows. Backgro'nd Processes on the 2ac (S +he Mac '3 has two types of backgro)nd processes* ;aemons and #gents. Eere is a definition for eachB # daemon is a program that r)ns in the backgro)nd as part of the overall system @that is* it is not tied to a partic)lar )serA. # daemon cannot display any 97IV more specifically* it is not allowed to connect to the window server. # web server is the perfect e5ample of a daemon. #n agent is a process that r)ns in the backgro)nd on behalf of a partic)lar )ser. #gents are )sef)l beca)se they can do things that daemons can.t* like reliably access the )ser.s home directory or connect to the window server. For More InformationB httpB developer.apple.com library mac %doc)mentation Mac'3N Concept)al /$3ystem3tart )p Chapters Introd)ction.html -et.s start with how to config)re a daemon in the Mac '3.

8L

If yo) open )p finder yo) will see the following folders. +he -a)nch;aemons folder )nder -ibrary is where we we need to add a la)nchd .plist file. +here is also a -ibrary -a)nch;aemons folder )nder 3ystem for the '3 daemons. +he la)nchd program is the service management framework for starting* stopping and managing daemons* applications* processes* and scripts in the Mac '3. 'nce the kernel starts la)nchd* the program scans several directories incl)ding etc for scripts and the -a)nch#gents and -a)nch;aemons folders in both -ibrary and 3ystem -ibrary. $rograms fo)nd in the -a)nch;aemons directories are r)n as the root )ser. Eere is the version of the la)nchd .plist file with all the basic config)ration we needB
=ZBml version7C0.1C enco)ing7C6 MK8CZH =?&35 "#@ plist #6!L:5 *"K//.pple 5omputer//& & #L:/ 0.1//@N*" *"http(//www.apple.com/& &s/#ropertyListK0.1.)t)*" H =plist version7C0.1CH =)ictH =keyHLabel=/keyH=stringHMy /ervice=/stringH =keyH#rogram.rguments=/keyH =arrayH =stringH/6sers/bill/My/ervice/My/ervice=/stringH =/arrayH =keyHWorking&irectory=/keyH=stringH/6sers/bill/My/ervice=/stringH =keyH/tan)ar)3ut#ath=/keyH=stringH/6sers/bill/My/ervice/My.log=/stringH =keyH;eep.live=/keyH=true/H =keyH&isable)=/keyH=false/H =/)ictH =/plistH

:o) can find all the different options for the .plist file hereB httpsB developer.apple.com library mac doc)mentation ;arwin 1eference Man$ages manG l a)nchd.plist.G.html +he $rogram#rg)ments key is an important tagB =keyH#rogram.rguments=/keyH =arrayH =stringH/6sers/bill/My/ervice/My/ervice=/stringH =/arrayH Eere yo) specify the name of the program to r)n and any other arg)ments to be passed into main. +hese other two tags* Working;irectory and 3tandard')t$ath are real helpf)l tooB =keyHWorking&irectory=/keyH=stringH/6sers/bill/My/ervice=/stri ngH =keyH/tan)ar)3ut#ath=/keyH=stringH/6sers/bill/My/ervice/My.log =/stringH 'nce we have a la)nchd .plist file we can )se a special program called la)nchctl to start o)r program as a backgro)nd process @daemonA. launchctl loa) /Library/Launch&aemons/My/ervice.plist 8<

+he la)nchctl program provides service control and reporting. +he load command is )sed to start a daemon based on the la)nchd .plist file. +o verify that a program is r)nning )se the list commandB launchctl list #:& QJ8 QJS Q01 /tatus K K K Label 1BRffJaQF1IJ01.anonymous.launchctl My /ervice 1BRffJaQJEce11.anonymous.bash

$I; <8H was assigned to the r)nning program* My 3ervice. &ow to stop the program from r)nning iss)e an )nload commandB launchctl unloa) /Library/Launch&aemons/My/ervice.plist launchctl list #:& QJ8 Q01 /tatus K K Label 1BRffJaQF1IJ01.anonymous.launchctl 1BRffJaQJEce11.anonymous.bash

&ow the program has been terminated. +here is some code we need to implement to handle the start and stop re2)ests from the '3 when o)r program is started and terminated. (S Speci6ic Go $oding Ciles :o) can create 9o so)rce code files that are only compiled for the target platform yo).re b)ilding.

In my -iteI;6 pro0ect for 9oing 9o yo) will see five 9o so)rce code files. +hree of these files have the name of an environment we can b)ild the code for* darwin @MacA* lin)5 and windows. 3ince I am b)ilding against the Mac '3* the serviceFlin)5.go and serviceFwindows.go files are ignored by the compiler.

G0

+he compiler recogni(es this naming convention by defa)lt. +his is very cool beca)se each environment needs to do a few things differently and )se different packages. #s in the case of serviceFwindows.go* the following imports are re2)iredB "co)e.google.com/p/winsvc/eventlog" "co)e.google.com/p/winsvc/mgr" "co)e.google.com/p/winsvc/svc" I don.t have these packages installed right now beca)se I don.t plan to r)n the code on windows. It doesn.t affect b)ilding the code beca)se serviceFwindows.go is ignored. +here is another really cool side effect from this* I can re)se types and f)nction names within these files since only one of these files are ever compiled with the program. +his means that any code that )ses this package does not have to be modified when changing environments. 1eally Cool ,, Ser:ice Inter6aces 6ach service m)st implement three interfaces that provide command and control for the service. type /ervice interface { :nstaller 5ontroller -unner % type :nstaller interface { :nstall(config $5onfig) error -emove() error % type 5ontroller interface { /tart() error /top() error % type -unner interface { -un(config $5onfig) error % +he Installer interface provides the logic for installing and )ninstalling the program as a backgro)nd process on the specific '3. +he Controller interface provides logic to start and stop the service from the command line. +he final interface 1)nner is )sed to perform all application logic and r)n the program as a service when re2)ested. Dar"in Implementation 3ince this post is specific to the Mac '3 I will concentrate on the implementation of the G?

serviceFdarwin.go code file. +he Installer interface re2)ires the implementation of two f)nctions* Install and 1emove. #s described above we need to create a la)nchd .plist file for the service. +he best way to accomplish this is to )se the te5t template package. +he FInstall3cript f)nction )ses a m)ltiCline string to create the template for the la)nchd .plist file. func W:nstall/cript() (script string) { return X=ZBml version7C0.1C enco)ing7C6 MK8CZH =?&35 "#@ plist #6!L:5 *"K//.pple 5omputer//& & #L:/ 0.1//@N*" *"http(//www.apple.com/& &s/#ropertyListK0.1.)t)*" H =plist version7C0.1CH =)ictH =keyHLabel=/keyH=stringH)).!i,p(a=-ame**=/stringH =keyH#rogram.rguments=/keyH =arrayH =stringH)).Gorking!irector=**/ )).E#ec+ta$(e-ame**=/stringH =/arrayH =keyHWorking&irectory=/keyH=stringH)).Gorking!irector=**=/stri ngH =keyH/tan)ar)3ut#ath=/keyH=stringH)).HogHocation**/ )).-ame**.log=/stringH =keyH;eep.live=/keyH=true/H =keyH&isable)=/keyH=false/H =/)ictH =/plistHX % What is cool abo)t m)ltiCline strings is that the carriage ret)rn* line feeds and spaces are respected. 3ince this is a template* we need to have variables that will be s)bstit)ted with data. +he WW.variableFnameXX convention is )sed to define those variables. Eere is the implementation of the Install f)nctionB func (service $W&arwinLaunch)/ervice) :nstall(config $5onfig) error { conf#ath (7 service.W2et/erviceMile#ath() W+ err (7 os./tat(conf#ath) if err 77 nil { return fmt.@rrorf(":nit alrea)y eBists( 's"+ conf#ath) % file+ err (7 os.5reate(conf#ath) if err ?7 nil { return err % G2

)efer file.5lose() parameters (7 Gstruct { @BecutableName string Working&irectory string Name string &isplayName string Long&escription string LogLocation string %{ service.W5onfig.@BecutableName+ service.W5onfig.Working&irectory+ service.W5onfig.Name+ service.W5onfig.&isplayName+ service.W5onfig.Long&escription+ service.W5onfig.LogLocation+ % template (7 template.Must(template.New("launch)5onfig").#arse(W:nstall/cri pt())) return template.@Becute(file+ parameters) % +he F9et3erviceFile$ath@A abstracts the location of the config)ration file for each environment implementation. For ;arwin the f)nction looks like thisB func (service $W&arwinLaunch)/ervice) W2et/erviceMile#ath() string { return fmt./printf("/Library/Launch&aemons/'s.plist"+ service.W5onfig.Name) % &ow the code checks if the file already e5ists and if it doesn.t* creates an empty file. &e5t we b)ild a str)ct on the fly and pop)late it with all the parameters that we need for the template 65ec)te f)nction call. &otice the names of the fields match the WW.variableFnameXX variables in the template. +he 65ec)te f)nction will process the template and then write the finished prod)ct to disk )sing the file handle. +he Controller interface re2)ires two f)nctions* 3tart and 3top. In the ;arwin so)rce code file the implementation is simpleB func (service $W&arwinLaunch)/ervice) /tart() error { conf#ath (7 service.W2et/erviceMile#ath() cm) (7 eBec.5omman)("launchctl"+ "loa)"+ conf#ath) return cm).-un() %

G"

func (service $W&arwinLaunch)/ervice) /top() error { conf#ath (7 service.W2et/erviceMile#ath() cm) (7 eBec.5omman)("launchctl"+ "unloa)"+ conf#ath) return cm).-un() % 6ach f)nction e5ec)tes the la)nchctl program the same way as we did above. +his provides a convenient way to start and stop the daemon. +he final interface that needs to be implemented is 1)nner with one f)nction called 1)n. func (service $W&arwinLaunch)/ervice) -un(config $5onfig) error { )efer func() { if r (7 recover()< r ?7 nil { fmt.#rintf("$$$$$$H /@-P:5@ #.N:5( 's*n"+ r) % %() var err error fmt.#rint("$$$$$$H :niting /ervice*n") if config.:nit ?7 nil { err 7 config.:nit() if err ?7 nil { return err % % fmt.#rint("$$$$$$H /tarting /ervice*n") if config./tart ?7 nil { err 7 config./tart() if err ?7 nil { return err % % fmt.#rint("$$$$$$H /ervice /tarte)*n") // 5reate a channel to talk with the 3/ var sig5han 7 make(chan os./ignal+ 0) signal.Notify(sig5han+ os.:nterrupt) // Wait for an event what/ig (7 =Ksig5han

G8

fmt.#rint("$$$$$$H /ervice /hutting &own*n") if config./top ?7 nil { err 7 config./top() if err ?7 nil { return err % % fmt.#rint("$$$$$$H /ervice &own*n") return err % 1)n is called when the program is going to r)n as daemon. It first makes a call to the )sers onInit and on3tart f)nctions. +he )ser is e5pected to perform any initiali(ation* start their ro)tines and then ret)rn control back. &e5t the code creates a channel that will be )sed to comm)nicate with the operating system. +he call to signal.&otify binds the channel to receive operating system events. +he code then starts an endless loop and waits )ntil the operating system notifies the program with an event. +he code is looking for any events that tell it to sh)tdown. 'nce an event to sh)tdown is received* the )ser on3top f)nction is called and the 1)n f)nction ret)rns control back to sh)tdown the program. Ser:ice 2anager 3ervice Manager provides all the boilerplate code so 3ervice can be easily implemented by any program. It implements the Config member f)nction called 1)n. func (config $5onfig) -un() { var service+ err 7 WNew/ervice(config) config.W/ervice 7 service if err ?7 nil { fmt.#rintf("'s unable to start( 's"+ config.&isplayName+ err) return % // #erform a comman) an) then return if len(os..rgs) H 0 { var err error verb (7 os..rgsU0V switch verb { case "install"( err 7 service.:nstall(config) if err ?7 nil { fmt.#rintf("Maile) to install( 's*n"+ err) GG

return % fmt.#rintf("/ervice *"'s*" installe).*n"+ config.&isplayName) return case "remove"( err 7 service.-emove() if err ?7 nil { fmt.#rintf("Maile) to remove( 's*n"+ err) return % fmt.#rintf("/ervice *"'s*" remove).*n"+ config.&isplayName) return case ")ebug"( config./tart(config) fmt.#rintln("/tarting 6p :n &ebug Mo)e") rea)er (7 bufio.New-ea)er(os./t)in) rea)er.-ea)/tring(C*nC) fmt.#rintln("/hutting &own") config./top(config) return case "start"( err 7 service./tart() if err ?7 nil { fmt.#rintf("Maile) to start( 's*n"+ err) return % fmt.#rintf("/ervice *"'s*" starte).*n"+ config.&isplayName) return case "stop"( err 7 service./top() if err ?7 nil { fmt.#rintf("Maile) to stop( 's*n"+ err) return % GH

fmt.#rintf("/ervice *"'s*" stoppe).*n"+ config.&isplayName) return )efault( fmt.#rintf("3ptions for *"'s*"( (install [ remove [ )ebug [ start [ stop)*n"+ os..rgsU1V) return % % // -un the service err 7 service.-un(config) % +he 1)n f)nction starts by creating the service ob0ect based on the config)ration that is provided. +hen it looks to at the command line arg)ments. If there is a command* it is processed and the program terminates. If the command is deb)g* the program is started as if it were r)nning as a service e5cept it does not hook into the operating system. Eitting the QenterR kill will sh)t down the program. If no command line arg)ments are provided* the code attempts to start as a daemon by calling service.1)n. Implementing The Ser:ice +he following code shows an e5ample of )sing the serviceB package main import ( "fmt" "github.com/goinggo/service/v0" "path/filepath" ) func main() { // 5apture the working )irectory working&irectory+ W (7 filepath..bs("") // 5reate a config obDect to start the service config (7 Gservice.5onfig{ @BecutableName( "My/ervice"+ Working&irectory( working&irectory+ Name( "My/ervice"+ &isplayName( "My /ervice"+ Long&escription( "My /ervice provi)es support for..."+ LogLocation( W/traps./trap("baseMile#ath")+ :nit( :nit/ervice+ GI

/tart( /tart/ervice+ /top( /top/ervice+ % // -un any comman) line options or start the service config.-un() % func :nit/ervice() { fmt.#rintf("/ervice :nite)") % func /tart/ervice() { fmt.#rintf("/ervice /tarte)*n") % func /top/ervice() { fmt.#rintf("/ervice /toppe)*n") % +he Init* 3tart and 3top f)nctions m)st ret)rn control back to the config.1)n f)nction. +he code I have has been tested with the Mac '3. +he code for lin)5 is identical e5cept for the script that needs to be created and installed. #lso the implementation for 3tart and 3top )ses different programs. In the near f)t)re I will test the -in)5 portion of the code. +he Window portion re2)ires some refactoring and will not b)ild. If yo) plan to )se Windows start with ;aniel.s code. 'nce yo) b)ild the code* open a +erminal session where the binary has been created and r)n the different commands. ./My/ervice )ebug ./My/ervice install ./My/ervice start ./My/ervice stop #s always I hope the code helps yo) create and r)n yo)r own services.

*o" Packages Work in Go


3ince I started writing code in 9o it has been a mystery to me how best to organi(e my code and )se the package keyword. +he package keyword is similar to )sing a namespace in C%* however the convention is to tie the package name to the directory str)ct)re. 9o has this web page that attempts to e5plain how to write 9o Code. httpB golang.org doc code.html

GL

When I started programming in 9o this was one of the first doc)ments I read. +his went way over my head* mainly beca)se I have been working in =is)al 3t)dio and code is packaged for yo) in 3ol)tion and $ro0ect files. Working o)t of a directory on the file system was a cra(y tho)ght. &ow I love the simplicity of it b)t it has taken 2)ite a while for it all to make sense. KEow to Write 9o CodeK starts o)t with the concept of a Workspace. +hink of this as the root directory for yo)r pro0ect. If yo) were working in =is)al 3t)dio this is where the sol)tion or pro0ect file wo)ld be located. +hen from inside yo)r Workspace yo) need to create a single s)bCdirectory called src. +his is mandatory if yo) want the 9o tools to work properly. From within the src directory yo) have the freedom to organi(e yo)r code the way yo) want. Eowever yo) need to )nderstand the conventions set forth by the 9o team for packages and so)rce code or yo) co)ld be refactoring yo)r code down the line. 'n my machine I created a Workspace called +est and the re2)ired s)bCdirectory called src. +his is the first step in creating yo)r pro0ect.

+hen in -iteI;6 open the +est directory @the WorkspaceA* and create the following s)bC directories and empty 9o so)rce code files. First we create a s)bCdirectory for the application we are creating. +he name of the directory where the main f)nction is located will be the name of the e5ec)table. In o)r case main.go G<

contains the main f)nction and is )nder the myprogram directory. +his means o)r e5ec)table file will be called myprogram. +he other s)bCdirectories inside of src will contain packages for o)r pro0ect. /y convention the name of the directory sho)ld be the name of the package for those so)rce code files that are located in that directory. In o)r case the new packages are called samplepkg and s)bpkg. +he name of the so)rce code files can be anything yo) like.

Create the same package folders and empty 9o so)rce code files to follow along. If yo) don.t add the Workspace folder to the 9'$#+E we will have problems. It took me a bit to reali(e that the C)stom ;irectories window is a +e5t /o5. 3o yo) can edit those folders directly. +he 3ystem 9'$#+E is read only. +he 9o designers have done several things when naming their packages and so)rce code files. #ll the file names and directories are lowercase and they did not )se )nderscores to break words apart in the package directory names. #lso* the package names match the directory names. Code files within a directory belong to a package named after the directory. +ake a look at the 9o so)rce code directory for a few of the standard library packagesB

+he package directories for b)fio and b)iltin are great e5amples of the directory naming convention. +hey co)ld have been called b)fFio and b)iltFin. -ook again at the 9o so)rce code directory and review the names of the so)rce code files.

H0

&otice the )se of the )nderscore in some of the file names. When the file contains test code or is specific to a partic)lar platform* an )nderscore is )sed. +he )s)al convention is to name one of the so)rce code files the same as the package name. In b)fio this convention is followed. Eowever* this is a loosely followed convention.

In the fmt package yo) will notice there is no so)rce code file named fmt.go. I personally like naming my packages and so)rce code files differently. -ast* open the doc.go* format.go* print.go and scan.go files. +hey are all declared to be in the fmt package. -et.s take a look at the code for sample.goB package samplepkg import ( "fmt" ) type /ample struct { Name string % func New(name string) (sample $/ample) { sample 7 G/ample{ Name( name+ % return sample %

H?

func (sample $/ample) #rint() { fmt.#rintf("/ample Name ( 's*n"+ sample.Name) % +he code is )seless b)t it will let )s foc)s on the two important conventions. First* notice the name of the package is the same as the name of the s)bCdirectory. 3econd* there is a f)nction called &ew. +he f)nction &ew is a 9o convention for packages that create a core type or different types for )se by the application developer. -ook at how &ew is defined and implemented in log.go* b)fio.go and cypto.goB (og.go // New creates a new Logger. he out variable sets the // )estination to which log )ata will be written. // he prefiB appears at the beginning of each generate) log line. // he flag argument )efines the logging properties. func New(out io.Writer+ prefiB string+ flag int) $Logger { return GLogger{out( out+ prefiB( prefiB+ flag( flag% % $+fio.go // New-ea)er returns a new -ea)er whose buffer has the )efault siNe. func New-ea)er(r) io.-ea)er) $-ea)er { return New-ea)er/iNe(r)+ )efault!uf/iNe) % cr=pto.go // New returns a new hash.Aash calculating the given hash function. New panics // if the hash function is not linke) into the binary. func (h Aash) New() hash.Aash { if h H 1 GG h = maBAash { f (7 hashesUhV if f ?7 nil { return f() % % panic("crypto( re\ueste) hash function is unavailable") % 3ince each package acts as a namespace* every package can have their own version of &ew. In b)fio.go m)ltiple types can be created* so there is no standalone &ew f)nction. Eere yo) will find f)nctions like &ew1eader and &ewWriter. -ook back at sample.go. In o)r code the core type is 3ample* so o)r &ew f)nction ret)rns a reference to a 3ample type. +hen we added a member f)nction to display the name we provided in &ew.

H2

&ow let.s look at the code for s)b.goB package subpkg import ( "fmt" ) type /ub struct { Name string % func New(name string) (sub $/ub) { sub 7 G/ub{ Name( name+ % return sub % func (sub $/ub) #rint() { fmt.#rintf("/ub Name ( 's*n"+ sub.Name) % +he code is identical e5cept we named o)r core type 3)b. +he package name matches the s)bCdirectory name and &ew ret)rns a reference to a 3)b type. &ow that o)r packages are properly defined and coded we can )se them. -ook at the code for main.goB package main import ( "samplepkg" "samplepkg/subpkg" ) func main() { sample (7 samplepkg.New(" est /ample #ackage") sample.#rint() sub (7 subpkg.New(" est /ub #ackage") sub.#rint() % 3ince o)r 9'$#+E points to the Workspace directory* in my case 7sers bill 3paces +est* o)r import references start from that point. Eere we are referencing both packages based on the directory str)ct)re. H"

&e5t we call the &ew f)nctions for each respective package and create variables of those core types.

&ow b)ild and r)n the program. :o) sho)ld see that an e5ec)table program was created called myprogram. 'nce yo)r program is ready for distrib)tion yo) can r)n the install command.

+he install command will create the bin and pkg folders in yo)r Workspace. &otice the final e5ec)table was placed )nder the bin directory. +he compiled packages were placed )nder the pkg directory. Within that directory a s)bC directory that describes the target architect)re is created and that mirrors the so)rce directories. +hese compiled packages e5ist so the go tool can avoid recompiling the so)rce code )nnecessarily. +he problem with that last statement* fo)nd in the KEow to Write 9o CodeK post* is that these .a files are ignore by the 9o tool when performing f)t)re b)ilds of yo)r code. Witho)t the so)rce code files yo) can.t b)ild yo)r program. I have not fo)nd any doc)mentation to really e5plain how these .a files can be )sed directly to b)ild 9o programs. If anyone can shed some light on this topic it wo)ld be greatly appreciated. #t the end of the day it is best to follow the conventions handled down from the 9o designers. -ooking at the so)rce code they have written provides the best doc)mentation for how to do things. Many of )s are writing code for the comm)nity. If we all follow the same conventions we can be ass)red of compatibility and readability. When in do)bt open finder to )sr local go src pkg and start digging.

H8

#s always I hope this helps yo) )nderstand the 9o programming lang)age a little better.

Singleton Design Pattern in Go


M)ltiCthreaded applications are very complicated* especially when yo)r code is not organi(ed and consistent with how reso)rces are accessed* managed and maintained. If yo) want to minimi(e b)gs yo) need philosophies and r)les to live by. Eere are some of mineB ?. 1eso)rce allocation and deCallocation sho)ld be abstracted and managed within the same type. 2. 1eso)rce thread safeness sho)ld be abstracted and managed within the same type. ". # p)blic interface sho)ld be the only means to accessing shared reso)rces. 8. #ny thread that allocates a reso)rce sho)ld deCallocated the same reso)rce. In 9o we don.t have threads b)t 9o 1o)tines. +he 9o r)ntime abstracts the threading and task swapping of these ro)tines. 1egardless* the same philosophies and r)les apply. 'ne of my favorite design patterns is the 3ingleton. It provides a great implementation when yo) only need one instance of a type and that type manages shared reso)rces. # 3ingleton is a design pattern where the type creates an instance of itself and keeps that reference private. #ccess to the shared reso)rces managed by that reference is abstracted thro)gh a static p)bic interface. +hese static methods also provide thread safeness. +he application )sing the 3ingleton is responsible for initiali(ing and deCinitiali(ing the 3ingleton b)t never has direct access to the internals. It escaped me for some time how to implement a 3ingleton in 9o beca)se 9o is not a traditional ob0ect oriented programming lang)age and there are no static methods. I consider 9o to be a light ob0ect oriented programming lang)age. :es it does have encaps)lation and type member f)nctions b)t it lacks inheritance and therefore traditional polymorphism. In all of the ''$ lang)ages I have ever )sed* I never )sed inheritance )nless I wanted to implement polymorphism. With the way interfaces are implemented in 9o there is no need for inheritance. 9o took the best parts of ''$* left o)t the rest and gave )s a better way to write polymorphic code. In 9o we can implement a 3ingleton by leveraging the scoping and encaps)lation r)les of packages and types. For this post we will e5plore my straps package since it will give )s a real world e5ample. +he straps package provides a mechanism to store config)ration options @strapsA in an NMdoc)ment and read them into memory for )se by the application. +he name strap comes from the early days of config)ring networking e2)ipment. +hose settings were called straps and that name has always st)ck with me. In the Mac'3 we have .plist files* in .&et we have app.config files and in 9o I have straps.5ml files. Eere is a sample straps files for one of my applicationsB =strapsH =?KK Log /ettings KKH =strap key7"baseMile#ath" value7"/6sers/bill/Logs/35K

HG

&ata/erver"H =strap key7"machineName" value7"myKmachine"H =strap key7")ays o;eep" value7"0"H =?KK /erverManager /ettings KKH =strap key7"cpuMultiplier" value7"011"H =/strapsH +he straps package knows how to read this 5ml file and provide access to the val)es via a 3ingleton based p)blic interface. 3ince these val)es only need to be read into memory once a 3ingleton is a great option for this package. Eere is the package and type information for strapsB package straps import ( "enco)ing/Bml" "io" "os" "path/filepath" "strconv" ) . . . ypes -emove)

type straps struct { /trapMap mapUstringVstring // pairs %

he map of strap key value

var W his $straps // . reference to the singleton I am not going to talk abo)t the aspects of reading the NM- doc)ment. If yo) are interested please read this blog post httpB www.goinggo.net 20?" 0H readingC5mlCdoc)mentsCinC go.html. In the code snippet above yo) will see the package name @strapsA* the definition of the private type F3traps and the private package variable F+his. +he F+his variable will contain the reference for the 3ingleton. +he scoping r)les for 9o state that types and f)nctions that start with a capital letter are p)blic and accessible o)tside of the package. +ypes and f)nctions that start with a lowercase letter are private and not accessible o)tside of the package. I name my variables that are defined within the scope of a f)nction with lowercase letters. =ariable names defined o)tside the scope of a f)nction* s)ch as type members and package variables start with a capital letter. +his allows me to look at the code and know instantly where memory for any given variable is being referenced. -)ckily for me 9o allows the )se HH

an )nderscore for variable names and makes them private. /oth the F3traps type and the F+his variable are private and only accessible from within the package. -ook at the -oad f)nction which initiali(es the 3ingleton for )seB func Loa)() { var Bml/traps UV$W4ML/trap var file $os.Mile )efer func() { if file ?7 nil { file.5lose() % %() // Min) the location of the straps.Bml file strapsMile#ath+ err (7 filepath..bs("straps.Bml") // 3pen the straps.Bml file file+ err 7 os.3pen(strapsMile#ath) // We nee) this file so panic if err ?7 nil { panic(err.@rror()) % // -ea) the straps file Bml/traps+ err 7 W-ea)/traps(file) if err ?7 nil { panic(err.@rror()) % :: 1reate a ,trap, o$Aect 062i, = ,trap,) /trapMap: make(map;,tring<,tring), * // /tore the key/value pairs for each strap for W+ strap (7 range Bml/traps { W his./trapMapUstrap.;eyV 7 strap.Palue % % +he -oad f)nction is a p)blic f)nction of the package. #pplications can access this f)nction thro)gh the package name. :o) can see how I )se names that start with a lowercase letter for local variables. #t the bottom of the -oad f)nction a straps ob0ect is created and the reference is set to the F+his variable. #t this point the 3ingleton e5ists and straps is ready to )se.

HI

#ccessing the straps is done with the p)blic f)nction 3trapB func /trap(key string) string { strap+ foun) (7 W his./trapMapUkeyV if foun) 77 false { panic("6nable o Locate ;ey") % return strap % +he p)blic f)nction 3trap )ses the 3ingleton reference to access the shared reso)rce. In this case the map of straps. If the map co)ld change d)ring the lifetime of the application* then a m)te5 or some other synchroni(ation ob0ect wo)ld need to be )sed to protect the map. -)ckily the straps never change once they are loaded. 3ince the reso)rce being managed by straps is 0)st memory there is no need for an 7nload or Close method. If we needed a f)nction to close any reso)rces another p)blic f)nction wo)ld have to be created. If private methods are re2)ired in the 3ingleton package to help organi(e code* I like to )se member f)nctions. 'ne reason is I don.t need to )se the )nderscore when naming these methods. 3ince the type is private I can make the member f)nctions p)blic beca)se they won.t be accessible. I also think the member f)nctions help make the code more readable. /y looking to see if the f)nction is a member f)nction or not I know if the f)nction is private or part of the p)blic interface. Eere is a e5ample of )sing a member f)nctionB func /ome#ublicMunction() { . W his./ome#rivateMemberMunction("key") . % func (straps $straps) /ome#rivateMemberMunction(key string) { strap+ foun) (7 straps./trapMapUkeyV . % 3ince the f)nction is a member f)nction we need to )se the F+his variable to make the f)nction call. From within the member f)nction I )se the local variable @strapsA and not the F+his variable. +he member f)nction is p)blic b)t the reference is private so only the package can reference the member f)nction. +his is 0)st a convention I established for myself.

HL

Eere is a sample program that )ses the straps packageB package main import ( ".r)an/tu)ios/straps" ) func main() { straps.Loa)() cpu (7 straps./trap("cpuMultiplier") % In main we don.t need to allocate any memory or maintain references. +hro)gh the package name we call -oad to initiali(e the 3ingleton. +hen thro)gh the package name again we access the p)blic interface* in this case the 3trap f)nction. If yo) have the same need to managed shared reso)rces thro)gh a p)blic interface try )sing a 3ingleton. #s always I hope this helps yo) write better and b)g less code.

(b/ect (riented Programming in Go


3omeone asked a 2)estion on the for)m today on how to gain the benefits of inheritance witho)t embedding. It is really important for everyone to think in terms of 9o and not the lang)ages they are leaving behind. I can.t tell yo) m)ch code I removed from my early 9o implementations beca)se it wasn.t necessary. +he lang)age designers have years of e5perience and knowledge. Eindsight is helping to create a lang)age that is fast* lean and really f)n to code in. I consider 9o to be a light ob0ect oriented programming lang)age. :es it does have encaps)lation and type member f)nctions b)t it lacks inheritance and therefore traditional polymorphism. For me* inheritance is )seless )nless yo) want to implement polymorphism. With the way interfaces are implemented in 9o* there is no need for inheritance. 9o took the best parts of ''$* left o)t the rest and gave )s a better way to write polymorphic code. Eere is a 2)ick view of ''$ in 9o. 3tart with these three str)ctsB type .nimal struct { Name string mean bool % type 5at struct { !asics .nimal Meow/trength int %

H<

type &og struct { .nimal !ark/trength int % Eere are three str)cts yo) wo)ld probably see in any ''$ e5ample. We have a base str)ct and two other str)cts that are specific to the base. +he #nimal str)ct)re contains attrib)tes that all animals share and the other two str)cts are specific to cats and dogs. #ll of the member properties are p)blic e5cept for mean. +he mean property in the #nimal str)ct starts with a lowercase letter. In 9o* the case of the first letter for variables* str)cts* properties* f)nctions* etc. determine the access specification. 7se a capital letter and it.s p)blic* )se a lowercase letter and it.s private. I like )sing )nderscore @FA to make things private. In my program I wo)ld have written mean as FMean. 3ince there is no inheritance in 9o* composition is yo)r only choice. +he Cat str)ct has a property called /asics which is of type #nimal. +he ;og str)ct is )sing an )nCnamed str)ct for the #nimal property. It.s )p to yo) to decide which is better for yo) and I will show yo) both implementations. I want to thank Oohn Mc-a)ghlin for his comment abo)t )nCnamed str)cts,, +o create a member f)nction for both Cat and ;og* the synta5 is as followsB func (dog 8!og) MakeNoise() { bark/trength (7 dog.!ark/trength if dog.mean 77 true { bark/trength 7 bark/trength $ F % for bark (7 1< bark = bark/trength< bark>> { fmt.#rintf("!.-; ") % fmt.#rintf("*n") % func (cat 81at) MakeNoise() { meow/trength (7 cat.Meow/trength if cat.Ia,ic,.mean 77 true { meow/trength 7 meow/trength $ F % for meow (7 1< meow = meow/trength< meow>> { fmt.#rintf("M@3W ") % fmt.#rintf("*n") % I0

/efore the name of the f)nction we specify a pointer to the type str)ct. &ow both Cat and ;og have member f)nctions called Make&oise. /oth these member f)nctions do the same thing. 6ach animal speaks in their native tong)e based on their bark or meow strength and if they are mean. 3illy* b)t it shows yo) how to access the referenced ob0ect. When )sing the ;og reference we access the #nimal properties directly. With the Cat reference we )se the named property called /asics. 'ne thing that is missing is the famo)s KthisK pointer. If yo) are really missing KthisK yo) co)ld change the local variable pointer as followsB func (t2i, $&og) MakeNoise() { bark/trength (7 t2i,.!ark/trength if t2i,.mean 77 true { bark/trength 7 bark/trength $ F % for bark (7 1< bark = bark/trength< bark>> { fmt.#rintf("!.-; ") % fmt.#rintf("*n") % 3o far we have covered encaps)lation* composition* access specifications and member f)nctions. #ll that is left is how to create polymorphic behavior. We )se interfaces to create polymorphic behaviorB type .nimal/oun)er interface { MakeNoise() % func Make/omeNoise(animal/oun)er .nimal/oun)er) { animal/oun)er.MakeNoise() % Eere we add an interface and a p)blic method that takes a reference to this interface. #ct)ally the f)nction will take a reference to an ob0ect that implements this interface. #n interface is not a type that can be instantiated. #n interface is a declaration of behavior. +here is a 9o convention of naming interfaces with the KerK s)ffi5 when the interface only contains one method. In 9o* any type str)ct that implements this interface* via a member f)nction* then represents this type. In o)r case both Cat and ;og have implemented the #nimal3o)nder interface and therefore are considered to be of type #nimal3o)nder. I?

+his means that ob0ects of both Cat and ;og can be passed as parameters to the Make3ome&oise f)nction. +he Make3ome&oise f)nction implements polymorphic behavior thro)gh the #nimal3o)nder interface. If yo) wanted to red)ce the d)plication of code in the Make&oise member f)nctions of Cat and ;og* yo) co)ld create a member f)nction in #nimal to handle itB func (animal $.nimal) #erformNoise(strength int+ soun) string) { if animal.mean 77 true { strength 7 strength $ F % for voice (7 1< voice = strength< voice>> { fmt.#rintf("'s "+ soun)) % fmt.#rintf("*n") % func ()og $&og) MakeNoise() { )og.#erformNoise()og.!ark/trength+ "!.-;") % func (cat $5at) MakeNoise() { cat.!asics.#erformNoise(cat.Meow/trength+ "M@3W") % &ow the #nimal type has a member f)nction with the b)siness logic for making noise. +he b)siness logic stays within the scope of the ob0ects it belongs to. +he other cool benefit is we don.t need to pass the mean val)e in as a parameter beca)se it already belongs to the #nimal type. Eere is the complete working sample programB package main import ( "fmt" ) type .nimal struct { Name string mean bool % type .nimal/oun)er interface { MakeNoise() %

I2

type &og struct { .nimal !ark/trength int % type 5at struct { !asics .nimal Meow/trength int % func main() { my&og (7 G&og{ .nimal{ "-over"+ // Name false+ // mean %+ E+ // !ark/trength % my5at (7 G5at{ !asics( .nimal{ Name( "9ulius"+ mean( true+ %+ Meow/trength( I+ % Make/omeNoise(my&og) Make/omeNoise(my5at) % func (animal $.nimal) #erformNoise(strength int+ soun) string) { if animal.mean 77 true { strength 7 strength $ F % for voice (7 1< voice = strength< voice>> { fmt.#rintf("'s "+ soun)) % fmt.#rintf("*n") % func ()og $&og) MakeNoise() { )og.#erformNoise()og.!ark/trength+ "!.-;") % func (cat $5at) MakeNoise() { cat.!asics.#erformNoise(cat.Meow/trength+ "M@3W") % I"

func Make/omeNoise(animal/oun)er .nimal/oun)er) { animal/oun)er.MakeNoise() % Eere is the o)tp)tB !.-; !.-; M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W M@3W 3omeone posted an e5ample on the board abo)t embedding an interface inside of a str)ct. Eere is an e5ampleB package main import ( "fmt" ) type Aorn/oun)er interface { /oun)Aorn() % type Pehicle struct { List UEVAorn/oun)er % type 5ar struct { /oun) string % type !ike struct { /oun) string % func main() { vehicle (7 new(Pehicle) vehicle.ListU1V 7 G5ar{"!@@#"% vehicle.ListU0V 7 G!ike{"-:N2"% for W+ horn/oun)er (7 range vehicle.List { horn/oun)er./oun)Aorn() % % func (car $5ar) /oun)Aorn() { fmt.#rintln(car./oun)) % func (bike $!ike) /oun)Aorn() { I8

fmt.#rintln(bike./oun)) % func #ressAorn(horn/oun)er Aorn/oun)er) { horn/oun)er./oun)Aorn() % In this e5ample the =ehicle str)ct maintains a list of ob0ects that implement the Eorn3o)nder interface. In main we create a new vehicle and assign a Car and /ike ob0ect to the list. +his assignment is possible beca)se Car and /ike both implement the interface. +hen )sing a simple loop* we )se the interface to so)nd the horn. 6verything yo) need to implement ''$ in yo)r application is there in 9o. #s I said before* 9o took the best parts of ''$* left o)t the rest and gave )s a better way to write polymorphic code. +o learn more on related topics check o)t these postsB httpB www.goinggo.net 20?" 0I howCpackagesCworkCinCgoClang)age.html httpB www.goinggo.net 20?" 0I singletonCdesignCpatternCinCgo.html I hope this small e5ample helps yo) in yo)r f)t)re 9o programming.

.nderstanding Type in Go
When I was coding in C C!! it was imperative to )nderstand type. If yo) didn.t* yo) wo)ld get into a lot of tro)ble with both the compiler and r)nning yo)r code. 1egardless of the lang)age* type to)ches every aspect of programming synta5. # good )nderstand of types and pointers is critical to good programming. +his post will foc)s on type. +ake these bytes of memory for startersB FF68 FF6" FF62 FF6? 00000000 ??00?0?? 0??00?0? 0000?0?0 What is the val)e of the byte at address FF6?D If yo) try to answer the 2)estion yo) will be wrong. Why* beca)se I have not told yo) what that byte represents. I have not given yo) the type information. What if I say that same byte represents a n)mberD :o)r answer wo)ld probably be ?0 and again yo) wo)ld be wrong. Why* beca)se yo) are ass)ming that when I said it was a n)mber I meant a base ?0 n)mber. -+m$er Ia,e,: .ll numbering systems have a base that they function within. /ince you were a baby you were taught to count in base 01. his may be )ue to the fact that most of us have 01 fingers an) 01 toes. .lso+ it seems natural

IG

to perform math in base 01. !ase )efines the number of symbols a numbering system contains. :n base 01 there are 01 )istinct symbols we use to represent the infinite number of things we can count. :n base 01 the symbols are 1+ 0+ E+ I+ J+ F+ S+ R+ 8+ Q. 3nce we reach the symbol Q we nee) to grow the length of the number. .s an eBample+ 01+ 011 an) 0111. here are two other bases we use all the time in computing. !ase E or binary numbers+ such as the bits represente) in the )iagram above. !ase 0S or heBa)ecimal numbers+ such as the a))resses represente) in the )iagram above. :n a binary numbering system (base E)+ there are only E symbols an) those symbols are 1 an) 0. :n a heBa)ecimal number system (base 0S)+ there are 0S symbols an) those symbols are 1+ 0+ E+ I+ J+ F+ S+ R+ 8+ Q+ .+ !+ 5+ &+ @+ M. :f there were apples sitting on a table+ those apples coul) be represente) in any numbering system. We coul) say there are( :n !ase E( 01101110 apples :n !ase 01( 0JF apples :n !ase 0S( Q0 apples .ll of those answers are correct when given the correct base. Notice the number of symbols re\uire) in each numbering system to represent those apples. he larger the base+ the more efficient the numbering system. 6sing base 0S for computer a))resses+ :# a))resses an) color co)es makes a lot of sense now. Look at the number for the A ML color white in all three bases( :n !ase E( 0000 0000 0000 0000 0000 0000 (EJ characters) IH

:n !ase 01( 0SRRRE0F (01 characters) :n !ase 0S( MMMMMM (S characters) Which numbering system woul) you have chosen to represent colorsZ &ow if I tell yo) the byte at address FF6? represents a base ?0 n)mber* yo)r answer of ?0 is correct. +ype provides two pieces of information that both the compiler and yo) need to perform the same e5ercise we 0)st went thro)gh. ?. +he amo)nt of memory* in bytes* to look at 2. +he representation of those bytes +he 9o lang)age provides these basic n)meric typesB Cn,igned 3nteger, uint8+ uint0S+ uintIE+ uintSJ /igned 3nteger, int8+ int0S+ intIE+ intSJ 4ea( -+m$er, floatIE+ floatSJ Predec(ared 3nteger, uint+ int+ uintptr +he names for these keywords provide both pieces of the type information. +he )intL contains a base ?0 n)mber )sing one byte of memory. +he val)e can be between 0 to 2GG. +he int"2 contains a base ?0 n)mber )sing 8 bytes of memory. +he val)e can be between C2?8I8L"H8L to 2?8I8L"H8I. +he predeclared integers get mapped based on the architect)re yo) are b)ilding the code against. 'n a H8 bit '3* int will map to intH8 and on a "2 bit '3* it will be mapped to int"2. 6verything that is stored in memory comes back to one of these n)meric types. 3trings in 9o are 0)st a series of )intL types* with r)les aro)nd stringing those bytes together and identifying end of string positions. # pointer in 9o is of type )intptr. #gain* based on the '3 architect)re this will be a )int"2 or )intH8. It is good that 9o created a special type for pointers. In the old days many C programmers wo)ld write code that ass)med a pointer val)e wo)ld always fit inside an )nsigned int. With )pgrades to the lang)age and architect)res over time* that event)ally was no longer tr)e. # lot of code broke beca)se addresses became larger than the predeclared )nsigned int.

II

3tr)ct types are 0)st combinations of know types that also event)ally resolve to a n)meric type. type @Bample struct{ !oolPalue bool :ntPalue int0S MloatPalue floatIE % +his str)ct)re represents a comple5 type. It represents I bytes with three different n)meric representations. +he bool is one byte* the int?H is 2 bytes and the float"2 adds 8 more bytes. Eowever* L bytes are act)ally allocated in memory for this str)ct. #ll memory is allocated on an alignment bo)ndary to minimi(e memory defragmentation. +o determine the alignment bo)ndary 9o is )sing for yo)r architect)re* yo) can r)n the )nsafe.#lignof f)nction. +he alignment bo)ndary in 9o for the H8bit ;arwin platform is L bytes. 3o when 9o determines the memory allocation for o)r str)cts* it will pad bytes to make s)re the final memory footprint is a m)ltiple of L. +he compiler will determine where to add the padding. If yo) want to learn more abo)t str)ct)re member alignment and padding check o)t this linkB httpB www.geeksforgeeks.org str)ct)reCmemberCalignmentCpaddingCandCdataCpacking +his program shows the padding that 9o inserted into the memory footprint for the 65ample type str)ctB package main import ( "fmt" "unsafe" ) type @Bample struct { !oolPalue bool :ntPalue int0S MloatPalue floatIE % func main() { eBample (7 G@Bample{ !oolPalue( true+ :ntPalue( 01+ MloatPalue( I.0J0FQE+ % eBampleNeBt (7 G@Bample{ !oolPalue( true+ :ntPalue( 01+ IL

MloatPalue( I.0J0FQE+ % alignment!oun)ary (7 unsafe..lignof(eBample) siNe!ool (7 unsafe./iNeof(eBample.!oolPalue) offset!ool (7 unsafe.3ffsetof(eBample.!oolPalue) siNe:nt (7 unsafe./iNeof(eBample.:ntPalue) offset:nt (7 unsafe.3ffsetof(eBample.:ntPalue) siNeMloat (7 unsafe./iNeof(eBample.MloatPalue) offsetMloat (7 unsafe.3ffsetof(eBample.MloatPalue) siNe!oolNeBt (7 unsafe./iNeof(eBampleNeBt.!oolPalue) offset!oolNeBt (7 unsafe.3ffsetof(eBampleNeBt.!oolPalue) fmt.#rintf(".lignment !oun)ary( ')*n"+ alignment!oun)ary) fmt.#rintf("!oolPalue 7 /iNe( ') 3ffset( ') .))r( 'v*n"+ siNe!ool+ offset!ool+ GeBample.!oolPalue) fmt.#rintf(":ntPalue 7 /iNe( ') 3ffset( ') .))r( 'v*n"+ siNe:nt+ offset:nt+ GeBample.:ntPalue) fmt.#rintf("MloatPalue 7 /iNe( ') 3ffset( ') .))r( 'v*n"+ siNeMloat+ offsetMloat+ GeBample.MloatPalue) fmt.#rintf("NeBt 7 /iNe( ') 3ffset( ') .))r( 'v*n"+ siNe!oolNeBt+ offset!oolNeBt+ GeBampleNeBt.!oolPalue) % Eere is the o)tp)tB .lignment !oun)ary( 8 !oolPalue 7 /iNe( 0 :ntPalue 7 /iNe( E MloatPalue 7 /iNe( J NeBt 7 /iNe( 0 3ffset( 3ffset( 3ffset( 3ffset( 1 E J 1 .))r( .))r( .))r( .))r( 1BE010Fb108 1BE010Fb10a 1BE010Fb10c 1BE010Fb1E1

+he alignment bo)ndary for the type str)ct is L bytes as e5pected. +he si(e val)e shows how m)ch memory* for that field* will be read and written to. #s e5pected* the si(e is inline with the type information. +he offset val)e shows how many bytes into the memory footprint we will find the start of that field. +he address is where the start of each field* within the memory footprint* can be fo)nd. We can see that 9o is padding ? byte between the /ool=al)e and Int=al)e fields. +he offset I<

val)e and the difference between the two addresses is 2 bytes. :o) can also see that the ne5t allocation of memory is starting 8 bytes away from the last field in the str)ct. -et.s prove the L byte alignment r)le by only keeping the ? byte bool field in the str)ctB package main import ( "fmt" "unsafe" ) type @Bample struct { !oolPalue bool % func main() { eBample (7 G@Bample{ !oolPalue( true+ % eBampleNeBt (7 G@Bample{ !oolPalue( true+ % alignment!oun)ary (7 unsafe..lignof(eBample) siNe!ool (7 unsafe./iNeof(eBample.!oolPalue) offset!ool (7 unsafe.3ffsetof(eBample.!oolPalue) siNe!oolNeBt (7 unsafe./iNeof(eBampleNeBt.!oolPalue) offset!oolNeBt (7 unsafe.3ffsetof(eBampleNeBt.!oolPalue) fmt.#rintf(".lignment !oun)ary( ')*n"+ alignment!oun)ary) fmt.#rintf("!oolPalue 7 /iNe( ') 3ffset( ') .))r( 'v*n"+ siNe!ool+ offset!ool+ GeBample.!oolPalue) fmt.#rintf("NeBt 7 /iNe( ') 3ffset( ') .))r( 'v*n"+ siNe!oolNeBt+ offset!oolNeBt+ GeBampleNeBt.!oolPalue) % #nd the o)tp)tB .lignment !oun)ary( 8 !oolPalue 7 /iNe( 0 3ffset( 1 .))r( 1BE010Fb108 NeBt 7 /iNe( 0 3ffset( 1 .))r( 1BE010Fb1E1 3)btract the two addresses and yo) will see there is an L byte difference between the two type str)ct allocations. #lso* the ne5t allocation of memory is starting at the same address from the first e5ample. 9o is padding I bytes to the str)ct to maintain the alignment bo)ndary. L0

1egardless of the padding* the val)e of si(e tr)ly represents the amo)nt of memory we can read and write to for each field. We can only manip)late memory when we are working with a n)meric type and the assignment operator @JA is how we do it. +o make life easier for )s* 9o has created some comple5 types that s)pport the assignment operator directly. 3ome of these types are strings* arrays and slices. +o see a complete list of these types check o)t this doc)ment httpB golang.org ref spec%+ypes. +hese comple5 types abstract the manip)lation of the )nderlining n)meric types that can be fo)nd in each implementation. In doing so* these comple5 types can be )sed like n)meric types that directly read and write to memory. 9o is a type safe lang)age. +his means that the compiler will always enforce like types on each side of an assignment operator. +his is really important beca)se it prevents )s from reading or writing to memory incorrectly. Imagine if we co)ld do the following. If yo) try to compile this code yo) will get an error. type @Bample struct{ !oolPalue bool :ntPalue int0S MloatPalue floatIE % eBample (7 G@Bample{ !oolPalue( true+ :ntPalue( 01+ MloatPalue( I.0J0FQE+ % var pointer $intIE pointer 7 $intIE(GeBample.:ntPalue) $pointer 7 E1 What I am trying to do is get the memory address of the 2 byte Int=al)e field and store it in a pointer of type int"2. +hen I am trying to )se the pointer to write a 8 byte integer into that memory address. If I was able to )se that pointer* I wo)ld be violating the type r)les for the Int=al)e field and corr)pting memory along the way. FF6L FF6I FF6H FF6G FF68 FF6" FF62 FF6? 0 0 0 ".?8 0 ?0 0 tr)e pointer FF6" FF6L FF6I FF6H FF6G FF68 FF6" FF62 FF6? 0 0 0 0 0 20 0 tr)e

L?

/ased on the memory footprint above* the pointer wo)ld be writing the val)e of 20 across the 8 bytes between FF6" and FF6H. +he val)e of Int=al)e wo)ld be 20 as e5pected b)t the val)e of Float=al)e wo)ld now be 0. Imagine if writing those bytes went o)tside the memory allocation for this str)ct and started to corr)pt memory in other areas of the application. +he b)gs that wo)ld follow wo)ld appear random and )npredictable. +he 9o compiler will always make s)re assigning memory and casting is safe. In this casting e5ample the compiler is going to complainB package main import ( "fmt" ) // 5reate a new type type intIE@Bt intIE func main() { // 5ast the number 01 to a value of type 9ill var Dill intIE@Bt 7 01 // .ssign the value of Dill to Dack // $$ cannot use Dill (type intIE@Bt) as type intIE in assignment $$ var Dack intIE 7 Dill // .ssign the value of Dill to Dack by casting // $$ the compiler is happy $$ var Dack intIE 7 intIE(Dill) fmt.#rintf("')*n"+ Dack) % First we create a new type in the system called int"265t and tell the compiler this new type represents a single int"2. &e5t we create a new variable called 0ill and assign the val)e of ?0. +he compiler allows the val)e to be assigned beca)se the n)meric type is on the right side of the assignment operator. +he compiler knows the assignment is safe. &ow we try to create a second variable called 0ack of type int"2 and assign the 0ill variable. Eere the compiler throws an errorB Kcannot )se 0ill @type int"265tA as type int"2 in assignmentK +he compiler respects that 0ill is of type int"265t and makes no ass)mptions abo)t the safeness of the assignment. &ow we )se casting and the compiler allows the assignment and the val)e prints as e5pected. When we perform the cast the compiler checks the safeness of the assignment. In o)r case it L2

identifies the val)es are of the same type and allows the assignment. +his may seem like basic st)ff to some of yo) b)t it is the fo)ndation for working in any programming lang)age. 6ven if type is abstracted yo) are manip)lating memory and sho)ld )nderstand what yo) are doing. With this fo)ndation we can talk abo)t pointers in 9o and passing parameters to f)nctions ne5t. #s always* I hope this post helps shed some light on things yo) may not have know.

.nderstanding Pointers and 2emory )llocation


In the doc)mentation provided by the 9o lang)age team yo) will find great information on pointers and memory allocation. Eere is a link to that doc)mentationB httpB golang.org doc fa2%$ointers We need to start with the )nderstanding that all variables contain a val)e. /ased on the type that variable represents will determine how we can )se it to manip)late the memory it contains. 1ead this post to learn moreB 7nderstanding +ype In 9o In 9o we can create variables that contain the Kval)e ofK the ob0ect itself or an address to the ob0ect. When the Kval)e ofK the variable is an address* the variable is considered a pointer. In the diagram below we have a variable called my=ariable. +he Kval)e ofK my=ariable is the address to an ob0ect that was allocated of the same type. my=ariable is considered a pointer variable.

In the ne5t diagram the Kval)e ofK my=ariable is the ob0ect itself* not a reference to the ob0ect.

L"

+o access properties of an ob0ect we )se a selector operator. +he selector operator allows )s to access a specific field in the ob0ect. +he synta5 is always 'b0ect.Field&ame* where the period @.A is the selector operator. In the C programming lang)age we need to )se different selector operators depending on the type of variable we are )sing. If the Kval)e ofK the variable is the ob0ect* we )se a period @.A. If the Kval)e ofK the variable is an address* we )se an arrow @CRA. 'ne really nice thing abo)t 9o is that yo) don.t need to worry abo)t what type of selector operator to )se. In 9o we only )se the period @.A regardless if the variable is the ob0ect or a pointer. +he compiler takes care of the )nderlying details to access the ob0ect. 3o why is all of this importantD It becomes important when we start )sing f)nctions to abstract and break )p logic. 6vent)ally yo) need to pass variables to these f)nctions and yo) need to know what yo) are passing. In 9o* variables are passed to a f)nction by val)e. +hat means the Kval)e ofK each variable that is specified is copied onto the stack for access by that f)nction. In this e5ample we call a f)nction that is s)pposed to change the val)e of an ob0ect that is allocated in main. package main import ( "fmt" "unsafe" ) type My ype struct { Palue0 int PalueE string % func main() { // .llocate an obDect of type My ype my3bDect (7 My ype{01+ "!ill"% // 5reate a pointer to the memory for my3bDect // Mor &isplay #urposes pointer (7 unsafe.#ointer(Gmy3bDect) // &isplay the a))ress an) values fmt.#rintf(".))r( 'v Palue0 ( ') PalueE( 's*n"+ pointer+ my3bDect.Palue0+ my3bDect.PalueE) // 5hange the values of my3bDect 5hangeMy3bDect(my3bDect) // &isplay the a))ress an) values fmt.#rintf(".))r( 'v Palue0 ( ') PalueE( 's*n"+ L8

pointer+ my3bDect.Palue0+ my3bDect.PalueE) % func 5hangeMy3bDect(my3bDect My ype) { // 5hange the values of my3bDect my3bDect.Palue0 7 E1 my3bDect.PalueE 7 "9ill" // 5reate a pointer to the memory for my3bDect pointer (7 unsafe.#ointer(Gmy3bDect) // &isplay the a))ress an) values fmt.#rintf(".))r( 'v Palue0 ( ') PalueE( 's*n"+ pointer+ my3bDect.Palue0+ my3bDect.PalueE) % Eere is the o)tp)t of the programB .))r( 1BE010bc111 Palue0 ( 01 PalueE( !ill Addr: "#21"1$c"?" Ja(+e1 : 2" Ja(+e2: Ki(( .))r( 1BE010bc111 Palue0 ( 01 PalueE( !ill 3o what went wrongD +he changes the f)nction made to main.s my'b0ect did not stay changed after the f)nction call. +he Kval)e ofK the my'b0ect variable in main does not contain a reference to the ob0ect* it is not a pointer. +he Kval)e ofK the my'b0ect variable in main is the ob0ect. When we pass the Kval)e ofK the my'b0ect variable in main to the f)nction* a copy of the ob0ect is placed on the stack. +he f)nction is altering its own version of the ob0ect. 'nce the f)nction terminates* the stack is popped* and the copy is technically gone. +he Kval)e ofK the my'b0ect variable in main is never to)ched. +o fi5 this we can allocate the memory in a way to get a reference back. +hen the Kval)e ofK the my'b0ect variable in main will be the address to the new ob0ect* a pointer variable. +hen we can change the f)nction to accept the Kval)e ofK an address to the ob0ect. package main import ( "fmt" "unsafe" ) type My ype struct { Palue0 int PalueE string % func main() { LG

// .llocate an obDect of type My ype my3bDect (7 M=6=pe{01+ "!ill"% // 5reate a pointer to the memory for my3bDect // Mor &isplay #urposes pointer (7 unsafe.#ointer(m=5$Aect) // &isplay the a))ress an) values fmt.#rintf(".))r( 'v Palue0 ( ') PalueE( 's*n"+ pointer+ my3bDect.Palue0+ my3bDect.PalueE) // 5hange the values of my3bDect 5hangeMy3bDect(my3bDect) // &isplay the a))ress an) values fmt.#rintf(".))r( 'v Palue0 ( ') PalueE( 's*n"+ pointer+ my3bDect.Palue0+ my3bDect.PalueE) % func 5hangeMy3bDect(my3bDect 8M=6=pe) { // 5hange the values of my3bDect my3bDect.Palue0 7 E1 my3bDect.PalueE 7 "9ill" // 5reate a pointer to the memory for my3bDect pointer (7 unsafe.#ointer(m=5$Aect) // &isplay the a))ress an) values fmt.#rintf(".))r( 'v Palue0 ( ') PalueE( 's*n"+ pointer+ my3bDect.Palue0+ my3bDect.PalueE) % When we )se the ampersand @YA operator to allocate the ob0ect* a reference is ret)rned. +hat means the Kval)e ofK the my'b0ect variable in main is now a pointer variable* whose val)e is the address to the newly allocated ob0ect. When we pass the Kval)e ofK the my'b0ect variable in main to the f)nction* the f)nctions my'b0ect variable now contains the address to the ob0ect* not a copy. We now have two pointers pointing to the same ob0ect. +he my'b0ect variable in main and the my'b0ect variable in the f)nction. If we r)n the program again* the f)nction is now working the way we wanted it to. It changes the val)e of the ob0ect allocated in main. .))r( 1BE010bc111 Palue0 ( 01 PalueE( !ill Addr: "#21"1$c""" Ja(+e1 : 2" Ja(+e2: Ki(( .))r( 1BE010bc111 Palue0 ( E1 PalueE( 9ill LH

;)ring the f)nction call the ob0ect is no longer being copied on the stack* the address to the ob0ect is being copied. +he f)nction is now referencing the same ob0ect* via the local pointer variable* and changing the val)es. +he 9o doc)ment titled K6ffective 9oK has a great section abo)t memory allocations* which incl)des how arrays* slices and maps workB httpB golang.org doc effectiveFgo.html%allocationFnew -et.s talk abo)t the keywords new and make. +he new keyword is )sed to allocate ob0ects of a specified type in memory. +he memory allocation is (eroed o)t. +he memory can.t be f)rther initiali(ed on the call to new. In other words* yo) can.t specify specific val)es for properties of the specified type when )sing new. If yo) want to specify val)es at the time yo) allocate the ob0ect* )se a composite literal. +hey come in two flavors* with or witho)t specifying the field names. // .llocate an obDect of type My ype // Palues must be in the correct or)er my3bDect (7 GMy ype{01+ "!ill"% // .llocate an obDect of type My ype // 6se labeling to specify the values my3bDect (7 GMy ype{ Palue0( 01+ PalueE( "!ill"+ % +he make keyword is )sed to allocate and initiali(e slices* maps and channels only. Make does not ret)rn a reference* it ret)rns the Kval)e ofK a data str)ct)re that is created and initiali(ed to manip)late the new slice* map or channel. +his data str)ct)re contains references to other data str)ct)res that are )sed to manip)late the slice* map or channel. Eow does passing a map by val)e to a f)nction work. -ook at this sample codeB package main import ( "fmt" "unsafe" ) type My ype struct { Palue0 int PalueE string % func main() { LI

myMap (7 make(mapUstringVstring) myMapU"!ill"V 7 "9ill" pointer (7 unsafe.#ointer(GmyMap) fmt.#rintf(".))r( 'v Palue ( 's*n"+ pointer+ myMapU"!ill"V) 5hangeMyMap(myMap) fmt.#rintf(".))r( 'v Palue ( 's*n"+ pointer+ myMapU"!ill"V) 5hangeMyMap.))r(GmyMap) fmt.#rintf(".))r( 'v Palue ( 's*n"+ pointer+ myMapU"!ill"V) % func 5hangeMyMap(myMap mapUstringVstring) { myMapU"!ill"V 7 "9oan" pointer (7 unsafe.#ointer(GmyMap) fmt.#rintf(".))r( 'v Palue ( 's*n"+ pointer+ myMapU"!ill"V) % // &onCt &o his+ 9ust Mor 6se :n his .rticle func 5hangeMyMap.))r(myMap#ointer $mapUstringVstring) { ($myMap#ointer)U"!ill"V 7 "9enny" pointer (7 unsafe.#ointer(myMap#ointer) fmt.#rintf(".))r( 'v Palue ( 's*n"+ pointer+ ($myMap#ointer)U"!ill"V) % Eere is the o)tp)t for the programB .))r( Addr: .))r( .))r( .))r( 1BE010Fb108 "#21"1L$"2D 1BE010Fb108 1BE010Fb108 1BE010Fb108 Palue Ja(+e Palue Palue Palue ( : ( ( ( 9ill Koan 9oan 9enny 9enny

We make a map and add a single key called K/illK assigning the val)e of KOillK. +hen we pass the val)e of the map variable to the ChangeMyMap f)nction. 1emember the myMap variable is not a pointer so the Kval)e ofK myMap* which is a data str)ct)re* is copied onto the stack d)ring the f)nction call. /eca)se the Kval)e ofK myMap is a data str)ct)re that contains references to the internals of the map* the f)nction can )se its copy of the data str)ct)re to make changes to the map that will be seen by main after the f)nction call. If yo) look at the o)tp)t yo) can see that when we pass the map by val)e* the f)nction has its LL

own copy of the map data str)ct)re. :o) can see changes made to the map are reflected after the f)nction call. In main we display the val)e of the map key K/illK after the f)nction call and it has changed. It is )nnecessary b)t the ChangeMyMap#ddr f)nction shows how yo) co)ld pass and )se a reference to the myMap variable in main. #gain the 9o team has made s)re passing the Kval)e ofK a map variable can be performed witho)t problems. &otice how we need to dereference the myMap$ointer variable when we want to access the map. +his is beca)se the 9o compiler will not allow )s to access the map thro)gh a pointer variable directly. ;ereferencing a pointer variable is e2)ivalent to having a variable whose val)e is the ob0ect. I have taken the time to write this post beca)se sometimes it can be conf)sing as to what the Kval)e ofK yo)r variable contains. If the Kval)e ofK yo)r variable is a large ob0ect* and yo) pass the Kval)e ofK that variable to a f)nction* yo) will be making a large copy of that variable on the stack. :o) want to make s)re yo).re passing addresses to yo)r f)nctions )nless yo) have a very special )se case. Maps* slices and channels are different. :o) can pass these variables by val)e witho)t any concern. When we pass a map variable to a f)nction* we are copying a data str)ct)re not the entire map. I recommend that yo) read the 6ffective 9o doc)mentation. I have read that doc)ment several times since I started programming in 9o. #s I gain more e5perience I always go back and read this doc)ment again. I always pick )p on something new that didn.t make sense to me before.

G'sta:oAs IEEE<730 Brain Teaser


/ack in O)ne* 9)stavo &iemeyer posted the following 2)estion on his -abi5.org blogB Assume uf is an unsigned integer with 64 bits that holds the I binary floating point number of that si$e. %ow &an you tell if uf represents an integer number' I can.t talk for yo)* b)t I write b)siness applications. I 0)st don.t have the backgro)nd to 2)ickly knock o)t an answer for a 2)estion like this. Fort)nately 9)stavo posted the logic for the answer. I tho)ght it wo)ld be f)n to better )nderstand the 2)estion and break down his answer. I am going to work with a "2 bit n)mber 0)st to make everything easier. Eow does the I666CIG8 standard store a floating point n)mber in a binary formatD +o start I fo)nd these two pagesB httpB steve.hollasch.net cginde5 coding ieeefloat.html httpB class.ece.iastate.ed) ar)n Cpr62L?FF0G ieeeIG8 ieG.html +he I666CIG8 specification represents a floating point n)mber in base 2 scientific notation )sing a special binary format. If yo) don.t know what I mean by base 2 then look at my post on 7nderstanding +ype In 9o @httpB www.goinggo.net 20?" 0I )nderstandingCtypeCinC go.htmlA. L< !"#4 representation for a

3cientific notation is an efficient way of writing very large or small n)mbers. It works by )sing a decimal format with a m)ltiplier. Eere are some e5amplesB /ase ?0 &)mber I00 8*<00*000*000 G"H2.H" C0.00"8G 0.0LG 3cientific &otation Ie!2 8.<e!< G."H2H"e!" ".8GeC" ?."HeC8 Calc)lation I Z ?02 8.< Z ?0< Coefficient /ase 65ponent Mantissa I 8.< ?0 ?0 ?0 ?0 2 2 < " C" C8 0 .< ."H2H" .8G ."H

G."H2H" Z ?0" G."H2H" ".8G Z ?0C" ?."H Z 2C8 ".8G ?."H

In normal scientific notation form there is always 0)st one digit on the left side of the decimal point. For base ?0 n)mbers that digit m)st be between ? thro)gh < and for base 2 n)mbers that digit can only be ?. +he entire n)mber in scientific notation is called the Coefficient. +he Mantissa is all the n)mbers to the right of the decimal point. +hese terms are important so take the time to st)dy and )nderstand the chart above. Eow we move the decimal point to that first position determines the val)e of the 65ponent. If we have to move the decimal point to the left* the 65ponent is a positive n)mber* to the right* it is a negative n)mber. -ook at the chart above and see the 65ponent val)e for each e5ample. +he /ase and the 65ponent work together in the notation. +he e5ponent determines the K$ower 'fK calc)lation we need to perform on the base. In the first e5ample the n)mber I is m)ltiplied by ?0 @+he /aseA to the power of 2 @+he 65ponentA to get back to the original base ?0 n)mber I00. We moved the decimal point to the left two places to convert I00 to I.00* which made the 65ponent !2 and created the notation of Ie!2. +he I666CIG8 standard does not store base ?0 scientific notation n)mbers b)t base 2 scientific notation n)mbers. +he last e5ample in the chart above represents the base ?0 n)mber 0.0LG in base 2 scientific notation. -et.s learn how that notation is calc)lated. /ase ?0 &)mber 3cientific &otation Calc)lation Coefficient /ase 65ponent Mantissa 0.0LG ?."HeC8 ?."H Z 2C8 ?."H 2 C8 ."H

We need to divide the base ?0 n)mber @0.0LGA by some power of two so we get a ? ! Fraction val)e. What do I mean by a ? ! Fraction val)eD We need a n)mber that looks like the Coefficient in the e5ample* ? ! ."H. +he I666CIG8 standard re2)ires that we have a K?.K in the Coefficient. +his allows )s to only have to store the Mantissa and give )s an e5tra bit of precision. If we )se br)te force yo) will see when we finally get the ? ! Fraction val)e for 0.0LGB <0

0.0LG 0.0LG 0.0LG 0.0LG

2C? J 0.?I 2C2 J 0."8 2C" J 0.HL 2C8 J ?."H ZZ We fo)nd the ? ! Fraction

#n e5ponent of C8 gives )s the ? ! Fraction val)e we need. &ow we have everything we need to store the base ?0 n)mber 0.0LG in I666CIG8 format. -et.s look at how the bits are laid o)t in the I666CIG8 format. Precision 3ingle @"2 /itsA ;o)ble @H8 /itsA Sign ? S"?T E@ponent Craction Bits L S"0C2"T 2" S22C00T G2 SG?C00T Bias ?2I ?02"

? SH"T ?? SH2CG2T

+he bits are broken into three parts. +here is a bit reserved for the sign* bits for the e5ponent and bits that are called fraction bits. +he fractions bits are where we store the mantissa as a binary fraction. If we store the val)e of 0.0LG )sing 3ingle $recision @a "2 bit n)mberA* the bit pattern in I666CIG8 wo)ld look like thisB Sign E@ponent ; #%= 0 0??? ?0?? Craction Bits ;5%4= 0?0 ???0 000? 0?00 0??? ?0??

+he 3ign bit* the leftmost bit* determines if the n)mber is positive or negative. If the bit is set to ? then the n)mber is negative else it is positive. +he ne5t L bits represent the 65ponent. In o)r e5ample* the base ?0 n)mber 0.0LG is converted to ?."H Z 2C8 )sing base 2 scientific notation. +herefore the val)e of the e5ponent is C8. In order to be able to represent negative n)mbers* there is a /ias val)e. +he /ias val)e for o)r "2 bit representation is ?2I. +o represent the n)mber C8* we need to find the n)mber* that when s)btract against the /ias* gives )s C8. In o)r case that n)mber is ?2". If yo) look at the bit pattern for the 65ponent yo) will see it represents the n)mber ?2" in binary. +he remaining 2" bits are the Fraction bits. +o calc)late the bit pattern for the fraction bits* we need to calc)late and s)m binary fractions )ntil we get the val)e of the Mantissa* or a val)e that is as close as possible. 1emember* we only need to store the Mantissa beca)se we always ass)me that the K?.K val)e e5ists. +o )nderstand how binary fractions are calc)lated* look at the following chart. 6ach bit position from left to right represents a fractional val)eB Binary Craction Decimal Po"er 0.? ?2 0.G 2C?

<?

0.0? 0.00?

?8 ?L

0.2G 0.?2G

2C2 2C"

We need to set the correct n)mber of fraction bits that add )p to or gets )s close eno)gh to the mantissa. +his is why we can sometimes lose precision. 0 0 Bit 2 8 G H ?? ?" ?I ?L ?< 9al'e 8 ?H "2 H8 208L L?<2 ?"?0I2 2H2?88 G282LL 0 000 0 00 0 Decimal 0.2G 0.0H2G 0.0"?2G 0.0?GH2G 0.0008LL2L?2G 0.000?220I0"?2G 0 J @0."HA Total 0.2G 0."?2G 0."8"IG 0."G<"IG 0."G<LH"2L?2G 0."G<<LG"G?GH2G 0."G<<<2<L0<GI0" 0."G<<<HI<GHG8" 0."G<<<LI0"002<" 0."G<<<<HGHHII2G 0."G<<<<L<G0<GL" 0."H00000?8"0G?2

Craction ?8 ? ?H ? "2 ? H8 ? 208L ? L?<2

? ?"?0I2 0.00000IH2<"<8G" ? 2H2?88 0.00000"L?8H<I2I ? G282LL 0.00000?<0I"8LH"

20 ?08LGIH ? ?08LGIH 0.000000<G"HI8"2 22 8?<8"08 ? 8?<8"08 0.0000002"L8?LGL 2" L"LLH0L ? L"LLH0L 0.000000??<20<2<

:o) can see that setting these ?2 bits get )s to the val)e of 0."H pl)s some e5tra fractions. -et.s s)m )p what we now know abo)t the I666CIG8 formatB ?. #ny base ?0 n)mber to be stored is converted to base 2 scientific notation. 2. +he base 2 scientific notation val)e we )se m)st follow the ? ! Fraction format. ". +here are three distinct sections in the format. 8. +he 3ign bit determines if the n)mber is positive or negative. G. +he 65ponent bits represent a n)mber that needs to be s)btracted against the /ias. H. +he Fraction bits represent the Mantissa )sing binary fraction s)mmation. -et.s prove that o)r analysis is correct abo)t the I666CIG8 format. We sho)ld be able to store the n)mber 0.LG and display bit patterns and val)es for each section that match everything we have seen. +he following code stores the n)mber 0.0LG and displays the I666CIG8 binary representationB

<2

package main import ( "fmt" "math" ) func main() { var number floatIE 7 1.18F fmt.#rintf("/tarting Number( 'f*n*n"+ number) // MloatIEbits returns the :@@@ RFJ binary representation bits (7 math.MloatIEbits(number) binary (7 fmt./printf("'.IEb"+ bits) fmt.#rintf("!it #attern( 's [ 's 's [ 's 's 's 's 's 's*n*n"+ binaryU1(0V+ binaryU0(FV+ binaryUF(QV+ binaryUQ(0EV+ binaryU0E(0SV+ binaryU0S(E1V+ binaryUE1(EJV+ binaryUEJ(E8V+ binaryUE8(IEV) bias (7 0ER sign (7 bits G (0 == I0) eBponent-aw (7 int(bits HH EI) eBponent (7 eBponent-aw K bias var mantissa floatSJ for in)eB+ bit (7 range binaryUQ(IEV { if bit 77 JQ { position (7 in)eB > 0 bitPalue (7 math.#ow(E+ floatSJ(position)) fractional (7 0 / bitPalue mantissa 7 mantissa > fractional % % value (7 (0 > mantissa) $ math.#ow(E+ floatSJ(eBponent)) fmt.#rintf("/ign( ') @Bponent( ') (')) Mantissa( 'f Palue( 'f*n*n"+ sign+ eBponent-aw+ eBponent+ mantissa+ value) %

<"

When we r)n the program we get the following o)tp)tB /tarting Number( 1.18F111 !it #attern( 1 [ 1000 0100 [ 101 0001 1110 1011 1000 0100 /ign( 1 @Bponent( 0EI (KJ) Mantissa( 1.IS1111 Palue( 1.18F111 If yo) compare the displayed bit pattern with o)r e5ample above* yo) will see that it matches. 6verything we learned abo)t I666CIG8 is tr)e. &ow we sho)ld be able to answer 9)stavo.s 2)estion. Eow can we tell if the val)e being stored is an integerD Eere is a f)nction* thanks to 9)stavo.s code* that tests if the I666CIG8 stored val)e is an integerB func :s:nt(bits uintIE+ bias int) { eBponent (7 int(bits HH EI) K bias K EI coefficient (7 (bits G ((0 == EI) K 0)) [ (0 == EI) int est (7 (coefficient G (0 == uintIE(KeBponent) K 0)) fmt.#rintf("*n@Bponent( ') 5oefficient( ') :nt est( ')*n"+ eBponent+ coefficient+ int est) if eBponent = KEI { fmt.#rintf("N3 :N @2@-*n") return % if eBponent = 1 GG int est ?7 1 { fmt.#rintf("N3 :N @2@-*n") return % fmt.#rintf(":N @2@-*n") % 3o how does this f)nction workD -et.s start with the first condition which tests if the 65ponent is less than C2". If we )se the n)mber ? as o)r test n)mber* the e5ponent will be ?2I which is the same as the /ias. +his means when we s)btract the 65ponent against the /ias we will get (ero. /tarting Number( 1."""""" !it #attern( 1 [ 1000 0000 [ 111 1111 1111 1111 1111 1111 /ign( 1 @Bponent( 127 (") Mantissa( 1.111111 Palue( 0.111111

<8

@Bponent( B2% 5oefficient( 8I88S18 :nt est( 1 :N @2@+he test f)nction adds an e5tra s)btraction of 2"* which represents the starting bit position for the 65ponent in the I666CIG8 format. +hat is why yo) see C2" for the 65ponent val)e coming from the test f)nction. Precision 3ingle @"2 /itsA Sign E@ponent Craction Bits 2" S22C00T Bias ?2I

? S"?T L S"0C#%T

+his s)btraction is re2)ired to help with the second test. 3o any val)e that is less than C2" m)st be less than one @?A and therefore not an integer. +o )nderstand how the second test works* let.s )se an integer val)e. +his time we will set the n)mber to 2"8G2" in the code and r)n the program again. /tarting Number( EIJFEI.111111 !it #attern( 1 [ 0110 1111 [ 001 1010 1111 1001 0011 1111 /ign( 1 @Bponent( 0JJ (0R) Mantissa( 1.R8QES8 Palue( EIJFEI.111111 @Bponent( KS 5oefficient( 0F11QJRE :nt est( 1 :N @2@+he second test looks for two conditions to identify if the n)mber is not an integer. +his re2)ires the )se of bitwise mathematics. -et.s look at the math we are performing in the f)nctionB eBponent (7 int(bits HH EI) K bias K EI coefficient (7 (bits G ((0 == EI) K 0)) [ (0 == EI) int est (7 coefficient G ((0 == uintIE(KeBponent)) K 0) +he coefficient calc)lation is adding the ? ! to the Mantissa so we have the base 2 Coefficient val)e. When we look at the first part of the coefficient calc)lation we see the following bit patternsB coefficient (7 ($it, ((1 EE 2%) B 1)) [ (0 == EI)

!its( "1""1""""00110101111100100111111 (0 == EI) K 0( 11111111100000000000000000000000 bits G ((0 == EI) K 0)( """""""""00110101111100100111111 +he first part of the coefficient calc)lation removes the bits for the 3ign and 65ponent from the entire I666CIG8 bit pattern.

<G

+he second part of the coefficient calc)lation adds the K? !K into the binary bit patternB coefficient (7 (bits G ((0 == EI) K 0)) [ (0 == EI) bits G ((0 == EI) K 0)( 11111111"00110101111100100111111 (0 == EI)( 11111111111111111111111111111111 coefficient( 11111111100110101111100100111111 &ow that the coefficient bit pattern is set* we can calc)late the int+est val)eB eBponent (7 int(bits HH EI) K bias K EI int est (7 (fraction G ((0 == uintIE(KeBponent)) K 0)) eBponent( (0JJ K 0ER K EI) 7 KS 0 == uintIE(KeBponent)( 111111 (0 == uintIE(KeBponent)) K 0( 000000 coefficient( 11111111000110101111100100"""""" 0 == uintIE(KeBponent)) K 0( 11111111111111111111111111111111 int est( 11111111111111111111111111"""""" +he val)e of the e5ponent we calc)late in the test f)nction is )sed to determine the n)mber of bits we will compare against the Coefficient. In this case the e5ponent val)e is CH. +hat is calc)lated by s)btracting the stored 65ponent val)e @?88A against the /ias @?2IA and then against the starting bit position for the 65ponent @2"A. +his gives )s a bit pattern of H ones @?.sA. +he final operation takes those H bits and #&;.s them against the rightmost H bits of the Coefficient to get the int+est val)e. +he second test is looking for an e5ponent val)e that is less than (ero @0A and an int+est val)e that is &'+ (ero @0A. +his wo)ld indicate the n)mber being stored is not an integer. In o)r e5ample with 2"8G2"* the 65ponent is less than (ero @0A* b)t the val)e of int+est is (ero @0A. We have an integer. I have incl)ded the sample code in the 9o playgro)nd so yo) can play with it. httpB play.golang.org p "5ra)d8"pi If it wasn.t for 9)stavo.s code I co)ld never have identified the sol)tion. Eere is a link to his sol)tionB httpB ba(aar.la)nchpad.net Mniemeyer strepr tr)nk view H strepr.go%-?H0 Eere is a copy of the code that yo) can copy and pasteB package main import ( "fmt" "math" ) <H

func main() { var number floatIE 7 EIJFEI fmt.#rintf("/tarting Number( 'f*n*n"+ number) // MloatIEbits returns the :@@@ RFJ binary representation bits (7 math.MloatIEbits(number) binary (7 fmt./printf("'.IEb"+ bits) fmt.#rintf("!it #attern( 's [ 's 's [ 's 's 's 's 's 's*n*n"+ binaryU1(0V+ binaryU0(FV+ binaryUF(QV+ binaryUQ(0EV+ binaryU0E(0SV+ binaryU0S(E1V+ binaryUE1(EJV+ binaryUEJ(E8V+ binaryUE8(IEV) bias (7 0ER sign (7 bits G (0 == I0) eBponent-aw (7 int(bits HH EI) eBponent (7 eBponent-aw K bias var mantissa floatSJ for in)eB+ bit (7 range binaryUQ(IEV { if bit 77 JQ { position (7 in)eB > 0 bitPalue (7 math.#ow(E+ floatSJ(position)) fractional (7 0 / bitPalue mantissa 7 mantissa > fractional % % value (7 (0 > mantissa) $ math.#ow(E+ floatSJ(eBponent)) fmt.#rintf("/ign( ') @Bponent( ') (')) Mantissa( 'f Palue( 'f*n*n"+ sign+ eBponent-aw+ eBponent+ mantissa+ value) :s:nt(bits+ bias) % func :s:nt(bits uintIE+ bias int) { eBponent (7 int(bitsHHEI) K bias K EI coefficient (7 (bits G ((0 == EI) K 0)) [ (0 == EI) int est (7 coefficient G ((0 == uintIE(KeBponent)) K 0) <I

/how!its(bits+ bias+ eBponent) fmt.#rintf("*n@Bp( ') Mrac( ') :nt est( ')*n"+ eBponent+ coefficient+ int est) if eBponent = KEI { fmt.#rintf("N3 :N @2@-*n") return % if eBponent = 1 GG int est ?7 1 { fmt.#rintf("N3 :N @2@-*n") return % fmt.#rintf(":N @2@-*n") % func /how!its(bits uintIE+ bias int+ eBponent int) { value (7 (0 == EI) K 0 valueE (7 (bits G ((0 == EI) K 0)) valueI (7 (0 == EI) coefficient (7 (bits G ((0 == EI) K 0)) [ (0 == EI) fmt.#rintf("!its(*t*t*t'.IEb*n"+ bits) fmt.#rintf("(0 == EI) K 0(*t*t'.IEb*n"+ value) fmt.#rintf("bits G ((0 == EI) K 0)(*t*t'.IEb*n*n"+ valueE) fmt.#rintf("bits G ((0 == EI) K 0)(*t*t'.IEb*n"+ valueE) fmt.#rintf("(0 == EI)(*t*t*t'.IEb*n"+ valueI) fmt.#rintf("coefficient(*t*t*t'.IEb*n*n"+ coefficient) valueF (7 0 == uintIE(KeBponent) valueS (7 (0 == uintIE(KeBponent)) K 0 in est (7 (coefficient G ((0 == uintIE(KeBponent)) K 0)) fmt.#rintf("0 == uintIE(KeBponent)(*t*t'.IEb*n"+ valueF) fmt.#rintf("(0 == uintIE(KeBponent)) K 0(*t'.IEb*n*n"+ valueS) fmt.#rintf("coefficient(*t*t*t'.IEb*n"+ coefficient) fmt.#rintf("(0 == uintIE(KeBponent)) K 0(*t'.IEb*n"+ valueS) fmt.#rintf("int est(*t*t*t'.IEb*n"+ in est) % I want to thank 9)stavo for posting the 2)estion and giving me something to really fight thro)gh to )nderstand. <L

.sing Time, TimeDones and Location in Go


I ran into a problem today. I was b)ilding code to cons)me &'##.s tide station NMdoc)ment and 2)ickly reali(ed I was in tro)ble. Eere is a small piece of that NM- doc)mentB EtimeMoneFH/6:H!6E:timeMoneF =itemH EdateF2"1%:"1:"1E:dateF =)ayH ue=/)ayH EtimeF"2:"> AME:timeF =pre)ictionsWinWftH0Q.R=/pre)ictionsWinWftH =pre)ictionsWinWcmHS11=/pre)ictionsWinWcmH =highlowHA=/highlowH =/itemH If yo) notice the time(one tag* it states the time is in -ocal 3tandard +ime -ocal ;aylight +ime. +his is a real problem beca)se I need to store this data in 7+C. Witho)t a proper time(one I am lost. #fter scratching my head for a bit my b)siness partner showed me two #$I.s that take a latit)de and longit)de position and ret)rn time(one information. -)ckily for me I have a latit)de and longit)de position for each tide station. If yo) open this web page yo) can read the doc)mentation for 9oogle.s +ime(one #$IB httpsB developers.google.com maps doc)mentation time(one +he #$I is fairly simple. It re2)ires a location* timestamps and a flag to identify if the re2)esting application is )sing a sensor* like a 9$3 device* to determine the location. Eere is a sample call to the 9oogle #$I and responseB https(//maps.googleapis.com/maps/api/timeNone/DsonZ location7I8.8FS8E+KQE.QQ0R0JGsensor7falseGtimestamp70II0RSS111 { ")st3ffset" ( IS11.1+ "raw3ffset" ( KE0S11.1+ "status" ( "3;"+ "timeYone:)" ( ".merica/5hicago"+ "timeYoneName" ( "5entral &aylight % +here is a limit of 2*G00 calls a day. For my initial load of the tide stations* I knew I was going to hit that limit and I didn.t want to wait several days to load all the data. 3o my b)siness partner fo)nd the time(one #$I from 9eo&ames. If yo) open this web page yo) can read the doc)mentation for 9eo&ames.s +ime(one #$IB httpB www.geonames.org e5port webCservices.html%time(one +he #$I re2)ires a free acco)nt which is real 2)ick to set)p. 'nce yo) activate yo)r acco)nt

ime"

<<

yo) need to find the acco)nt page and activate yo)r )sername for )se with the #$I. Eere is a sample call to the 9eo&ames #$I and responseB http(//api.geonames.org/timeNone9/3NZ lat7JR.10Glng701.EGusername7)emo { "time"("E10IK18K1Q 11(FJ"+ "countryName"(".ustria"+ "sunset"("E10IK18K1Q E1(J1"+ "raw3ffset"(0+ ")st3ffset"(E+ "country5o)e"(". "+ "gmt3ffset"(0+ "lng"(01.E+ "sunrise"("E10IK18K1Q 1S(1R"+ "timeNone:)"("@urope/Pienna"+ "lat"(JR.10 % +his #$I ret)rns a bit more information. +here is no limit to the n)mber of calls yo) can make b)t the response times are not g)aranteed. I )sed it for several tho)sand calls and had no problems. 3o now we have two different web calls we can )se to get the time(one information. -et.s look at how we can )se 9o to make the 9oogle web call and get an ob0ect back that we can )se in o)r program. First* we need to define a new type that can contain the information we will get back from the #$I. package main import ( "enco)ing/Dson" "fmt" "io/ioutil" "net/http" "time" ) const ( W2332L@W6-: string 7 "https(//maps.googleapis.com/maps/api/timeNone/DsonZlocation7 'f+'fGtimestamp7')Gsensor7false" ) type 2oogle imeNone struct { &st3ffset floatSJ Xbson(")st3ffset"X -aw3ffset floatSJ Xbson("raw3ffset"X ?00

/tatus string imeNone:) string imeNoneName string %

Xbson("status"X Xbson("timeYone:)"X Xbson("timeYoneName"X

9o has awesome s)pport for O3'& and NM-. If yo) look at the 9oogle+ime(one str)ct yo) will see that each field contains a KtagK. # tag is e5tra data attached to each field that can later be retrieved by o)r program )sing reflection. +o learn more abo)t tags read this doc)mentB httpB golang.org pkg reflect %3tr)ct+ag +he encoding 0son package has defined a set of tags it looks for to help with marshaling and )nmarshaling O3'& data. +o learn more abo)t the O3'& s)pport in 9o read these doc)mentsB httpB golang.org doc articles 0sonFandFgo.html httpB golang.org pkg encoding 0son If yo) make the field names in yo)r str)ct the same as the field names in the O3'& doc)ment* yo) don.t need to )se the tags. I didn.t do that so the tags are there to tell the 7nmarshal f)nction how to map the data. -et.s look at a f)nction that can make the #$I call to 9oogle and )nmarshal the O3'& doc)ment to o)r new typeB func -etrieve2oogle imeNone(latitu)e floatSJ+ longitu)e floatSJ) (google imeNone $2oogle imeNone+ err error) { )efer func() { if r (7 recover()< r ?7 nil { err 7 fmt.@rrorf("'v"+ r) % %() uri (7 fmt./printf(W2332L@W6-:+ latitu)e+ longitu)e+ time.Now().6 5().6niB()) resp+ err (7 http.2et(uri) )efer func() { if resp ?7 nil { resp.!o)y.5lose() % %() if err ?7 nil { return nil+ err % // 5onvert the response to a byte array var raw&ocument UVbyte raw&ocument+ err 7 ioutil.-ea).ll(resp.!o)y) ?0?

if err ?7 nil { return nil+ err % // 6nmarshal the response to a 2oogle imeNone obDect err 7 Dson.6nmarshal(raw&ocument+ Ggoogle imeNone) if err ?7 nil { return nil+ err % if google imeNone./tatus ?7 "3;" { return nil+ fmt.@rrorf("@rror ( 2oogle /tatus ( 's"+ google imeNone./tatus) % if len(google imeNone. imeNone:)) 77 1 { return nil+ fmt.@rrorf("@rror ( No imeNone :) #rovi)e)") % return google imeNone+ err % +he web call and error handling is fairly boilerplate so let.s 0)st talk briefly abo)t the 7nmarshal call. var raw&ocument UVbyte raw&ocument+ err 7 ioutil.-ea).ll(resp.!o)y) err 7 Dson.6nmarshal(raw&ocument+ Ggoogle imeNone) When the web call ret)rns* we take the response and store it in a byte array. +hen we call the 0son 7nmarshal f)nction* passing the byte array and a reference to o)r ret)rn type pointer variable. +he 7nmarshal call creates an ob0ect of type 9oogle+ime(one* e5tracts and copies the data from the ret)rned O3'& doc)ment and sets the val)e of o)r pointer variable. It.s really brilliant. If any fields can.t be mapped they are simply ignored. +he 7nmarshal call will ret)rn an error if there are casting iss)es. 3o this is great* we can get the time(one data and )nmarshal it to an ob0ect with three lines of code. &ow the only problem is* how the heck do we )se the time(oneid to set o)r locationD Eere is the problem again. We have to take the local time from the feed doc)ment* apply the time(one information and then convert everything 7+C. -et.s look at the feed doc)ment againB EtimeMoneFH/6:H!6E:timeMoneF =itemH EdateF2"1%:"1:"1E:dateF ?02

=)ayH ue=/)ayH EtimeF"2:"> AME:timeF =pre)ictionsWinWftH0Q.R=/pre)ictionsWinWftH =pre)ictionsWinWcmHS11=/pre)ictionsWinWcmH =highlowHA=/highlowH =/itemH #ss)ming we have e5tracted the data from this doc)ment* how can we )se the time(oneid to get )s o)t of this 0amD -ook at the code I wrote in the main f)nction. It )ses the time.-oad-ocation f)nction and the time(one id we get from the #$I call to solve the problemB func main() { // 5all to get the timeNone for this lat an) lng position google imeNone+ err (7 -etrieve2oogle imeNone(I8.8FS8E+ KQE.QQ0R0J) // #reten) this is the )ate an) time we eBtracte) year (7 E10I month (7 0 )ay (7 0 hour (7 E minute (7 S /: 1apt+re t2e (ocation $a,ed on t2e timeMone id from Noog(e (ocation, err := time.HoadHocation(goog(e6imeMone.6imeMone3d) if err ?7 nil { fmt.#rintf("@--3- ( 's"+ err) return % // 5apture the local an) 6 5 time base) on timeNone local ime (7 time.&ate(year+ time.Month(month)+ )ay+ hour+ minute+ 1+ 1+ (ocation) utc ime (7 local ime.6 5() // &isplay the results fmt.#rintf(" imeNone(*t's*n"+ google imeNone. imeNone:)) fmt.#rintf("Local ime( 'v*n"+ local ime) fmt.#rintf("6 5 ime( 'v*n"+ utc ime) % Eere is the o)tp)tB imeNone( .merica/5hicago Local ime( E10IK10K10 1E(1S(11 K1S11 5/ ime( E10IK10K10 18(1S(11 >1111 6 5

?0"

6verything worked like a champ. ')r local+ime variable is set to C3+ or Central 3tandard +ime* which is where Chicago is located. +he 9oogle #$I provided the correct time(one for the latit)de and longit)de beca)se that location falls within Misso)ri. httpsB maps.google.com mapsD2J"<.2"22G"*C<2.<<?I?8Y(JH +he last 2)estion we have to ask is how did the -oad-ocation f)nction take that time(one id string and make this work. +he time(one id contains both a co)ntry and city @#merica ChicagoA. +here m)st be tho)sands of these time(one ids. If we take a look at the time package doc)mentation for -oad-ocation* we will find the answerB httpB golang.org pkg time %-oad-ocation Eere is the doc)mentation for -oad-ocationB Loa)Location returns the Location with the given name. :f the name is "" or "6 5"+ Loa)Location returns 6 5. :f the name is "Local"+ Loa)Location returns Local. 3therwise+ the name is taken to be a location name correspon)ing to a file in the :.N. ime Yone )atabase+ such as ".merica/NewW"ork". 62e time Mone data$a,e needed $= HoadHocation ma= not $e pre,ent on a(( ,=,tem,, e,pecia((= nonBCni# ,=,tem,. HoadHocation (ook, in t2e director= or +ncompre,,ed Mip fi(e named $= t2e O5-E3-95 environment varia$(e, if an=, t2en (ook, in kno.n in,ta((ation (ocation, on Cni# ,=,tem,, and fina((= (ook, in PN54556:(i$:time:Moneinfo.Mip. If yo) read the last paragraph yo) will see that the -oad-ocation f)nction is reading a database file to get the information. I didn.t download any database* nor did I set an environment variable called P'&6I&F'. +he only answer is that this (oneinfo.(ip file e5ists in 9'1''+. -et.s take a lookB

3)re eno)gh there is a (oneinfo.(ip file located in the lib time directory where 9o was installed. =ery Cool ,, +here yo) have it. &ow yo) know how to )se the time.-oad-ocation f)nction to help make s)re yo)r time val)es are always in the correct time(one. If yo) have a latit)de and longit)de* yo) can )se either #$I to get that time(one id.

?08

I have added a new package called time(one to the 9oing9o repository in 9ith)b if yo) want a re)sable copy of the code with both #$I calls. Eere is the entire working sample programB package main import ( "enco)ing/Dson" "fmt" "io/ioutil" "net/http" "time" ) const ( W2332L@W6-: string 7 "https(//maps.googleapis.com/maps/api/timeNone/DsonZlocation7 'f+'fGtimestamp7')Gsensor7false" ) type 2oogle imeNone struct { &st3ffset floatSJ Xbson(")st3ffset"X -aw3ffset floatSJ Xbson("raw3ffset"X /tatus string Xbson("status"X imeNone:) string Xbson("timeYone:)"X imeNoneName string Xbson("timeYoneName"X % func main() { // 5all to get the timeNone for this lat an) lng position google imeNone+ err (7 -etrieve2oogle imeNone(I8.8FS8E+ KQE.QQ0R0J) // #reten) this is the )ate an) time we eBtracte) year (7 E10I month (7 0 )ay (7 0 hour (7 E minute (7 S // 5apture the location base) on the timeNone i) from 2oogle location+ err (7 time.Loa)Location(google imeNone. imeNone:)) if err ?7 nil { fmt.#rintf("@--3- ( 's"+ err) return % // 5apture the local an) 6 5 time base) on timeNone local ime (7 time.&ate(year+ time.Month(month)+ )ay+ hour+ ?0G

minute+ 1+ 1+ location) utc ime (7 local ime.6 5() // &isplay the results fmt.#rintf(" imeNone(*t's*n"+ google imeNone. imeNone:)) fmt.#rintf("Local ime( 'v*n"+ local ime) fmt.#rintf("6 5 ime( 'v*n"+ utc ime) % func -etrieve2oogle imeNone(latitu)e floatSJ+ longitu)e floatSJ) (google imeNone $2oogle imeNone+ err error) { )efer func() { if r (7 recover()< r ?7 nil { err 7 fmt.@rrorf("'v"+ r) % %() uri (7 fmt./printf(W2332L@W6-:+ latitu)e+ longitu)e+ time.Now().6 5().6niB()) resp+ err (7 http.2et(uri) )efer func() { if resp ?7 nil { resp.!o)y.5lose() % %() if err ?7 nil { return nil+ err % // 5onvert the response to a byte array var raw&ocument UVbyte raw&ocument+ err 7 ioutil.-ea).ll(resp.!o)y) if err ?7 nil { return nil+ err % // 6nmarshal the response to a 2oogle imeNone obDect err 7 Dson.6nmarshal(raw&ocument+ Ggoogle imeNone) if err ?7 nil { return nil+ err % if google imeNone./tatus ?7 "3;" { return nil+ fmt.@rrorf("@rror ( 2oogle /tatus ( 's"+ ?0H

google imeNone./tatus) % if len(google imeNone. imeNone:)) 77 1 { return nil+ fmt.@rrorf("@rror ( No imeNone :) #rovi)e)") % return google imeNone+ err %

.nderstanding Slices in Go Programming


3ince I started programming in 9o the concept and )se of slices has been conf)sing. +his is something completely new to me. +hey look like an array* and feel like an array* b)t they are m)ch more than an array. I am constantly reading how slices are )sed 2)ite a bit by 9o programmers and I think it is finally time for me to )nderstand what slices are all abo)t. +here is a great blog post written by #ndrew 9errand abo)t slicesB httpB blog.golang.org goCslicesC)sageCandCinternals +here is no reason to repeat everything #ndrew has written so please read his post before contin)ing. -et.s 0)st cover the internals of a slice.

+he pict)re above represents the internal str)ct)re of a slice. When yo) allocate a slice this data str)ct)re along with an )nderlying array is created. +he val)e of yo)r slice variable will be this data str)ct)re. When yo) pass a slice to a f)nction* a copy of this data str)ct)re is created on the stack. We can create a slice in two waysB Eere we )se the keyword make, passing the type of data we are storing* the initial length of the slice and the capacity of the )nderlying array. my/lice (7 make(UVstring+ F+ 8) my/liceU1V 7 ".pple" my/liceU0V 7 "3range" ?0I

my/liceUEV 7 "!anana" my/liceUIV 7 "2rape" my/liceUJV 7 "#lum" // "ou )onCt nee) to inclu)e the capacity. Length an) 5apacity will be the same my/lice (7 make(UVstring+ F) :o) can also )se a slice literal. In this case the length and capacity will be the same. &otice no val)e is provided inside the hard brackets ST. If yo) add a val)e yo) will have an array. If yo) don.t add a val)e yo) will have a slice. my/lice (7 UVstring{".pple"+ "3range"+ "!anana"+ "2rape"+ "#lum"% :o) can.t e5tend the capacity of a slice once it is created. +he only way to change the capacity is to create a new slice and perform a copy. #ndrew provides a great sample f)nction that shows an efficient way to check the remaining capacity for room and only if necessary* perform a copy. +he length of a slice identifies the n)mber of elements of the )nderlying array we are )sing from the starting inde5 position. +he capacity identifies the n)mber of elements we have available for )se. We can create a new slice from the original sliceB new/lice (7 my/liceUE(JV

+he val)e of the new slice.s pointer variable is associated with inde5 positions 2 and " of the initial )nderlying array. #s far as this new slice is concerned* we now have a )nderlying array of " elements and we are only )sing 2 of those " elements. +his new slice has no knowledge of the first two elements from the initial )nderlying array and never will. When performing a slice operation the first parameter specifies the starting inde5 from the slices pointer variable position. In o)r case we said inde5 2 which is " elements inside the initial )nderlying array we are taking the slice from. +he second parameter is the last inde5 position pl)s one @!?A. In o)r case we said inde5 8 which will incl)de all inde5es between inde5 2 @the starting positionA and inde5 " @the final positionA. We don.t always need to incl)de a starting or ending inde5 position when performing a slice operationB ?0L

new/liceE 7 new/liceU(cap(new/lice)V

In this e5ample we )se the new slice we created before to create a third slice. We don.t provide a starting inde5 position b)t we do specify the last inde5 position. ')r latest slice has the same starting position and capacity b)t the length has changed. /y specifying the last inde5 position as the capacity si(e* this length of this slice )ses all remaining elements from the )nderlying array. &ow let.s r)n some code to prove this data str)ct)re act)ally e5ists and slices work as advertised. I have created a f)nction that will inspect the memory associated with any sliceB func :nspect/lice(slice UVstring) { // 5apture the a))ress to the slice structure a))ress (7 unsafe.#ointer(Gslice) // 5apture the a))ress where the length an) cap siNe is store) len.))r (7 uintptr(a))ress) > uintptr(8) cap.))r (7 uintptr(a))ress) > uintptr(0S) // 5reate pointers to the length an) cap siNe len#tr (7 ($int)(unsafe.#ointer(len.))r)) cap#tr (7 ($int)(unsafe.#ointer(cap.))r)) // 5reate a pointer to the un)erlying array a))#tr (7 ($U8Vstring)(unsafe.#ointer($($uintptr) (a))ress))) fmt.#rintf("/lice .))rU'pV Len .))rU1B'BV 5ap .))rU1B 'BV*n"+ a))ress+ len.))r+ cap.))r) fmt.#rintf("/lice LengthU')V 5apU')V*n"+ $len#tr+ $cap#tr) for in)eB (7 1< in)eB = $len#tr< in)eB>> { ?0<

fmt.#rintf("U')V 'p 's*n"+ in)eB+ G($a))#tr)Uin)eBV+ ($a))#tr)Uin)eBV) % fmt.#rintf("*n*n") % +his f)nction is performing a b)nch of pointer manip)lations so we can inspect the memory and val)es of a slice.s data str)ct)re and )nderlying array. We will break it down* b)t first let.s create a slice and r)n it thro)gh the inspect f)nctionB package main import ( "fmt" "unsafe" ) func main() { org/lice (7 org/liceU1V org/liceU0V org/liceUEV org/liceUIV org/liceUJV % Eere is the o)tp)t of the programB /lice .))rU1BE010be111V Len .))rU1BE010be118V 5ap .))rU1BE010be101V /lice LengthUFV 5apU8V U1V 1BE010b)111 .pple U0V 1BE010b)101 3range UEV 1BE010b)1E1 !anana UIV 1BE010b)1I1 2rape UJV 1BE010b)1J1 #lum It appears the slice.s data str)ct)re really does e5ist as described by #ndrew. +he Inspect3lice f)nction first displays the address of the slice.s data str)ct)re and the address positions where the length and capacity val)es sho)ld be. +hen by creating int pointers )sing those addresses* we display the val)es for length and capacity. -ast we create a pointer to the )nderlying array. 7sing the pointer* we iterate thro)gh the )nderlying array displaying the inde5 position* the starting address of the element and the val)e. make(UVstring+ F+ 8) 7 ".pple" 7 "3range" 7 "!anana" 7 "2rape" 7 "#lum"

:nspect/lice(org/lice)

??0

-et.s break down the Inspect3lice f)nction to )nderstand how it worksB // 5apture the a))ress to the slice structure a))ress (7 unsafe.#ointer(Gslice) // 5apture the a))ress where the length an) cap siNe is store) len.))r (7 uintptr(a))ress) > uintptr(8) cap.))r (7 uintptr(a))ress) > uintptr(0S) )nsafe.$ointer is a special type that is mapped to an )intptr type. /eca)se we need to perform pointer arithmetic* we need to work with generic pointers. +he first line of code casts the address of the slice.s data str)ct)re to an )nsafe.$ointer. +hen we create two generic pointers that point L and ?H bytes into the slice.s data str)ct)re respectively. +he following diagram shows each pointer variable* the val)e of the variable and the val)e that the pointer points toB address len#ddr cap#ddr 052?0?be000 052?0?be00L 052?0?be0?0 052?0?bd000 G L With o)r pointers in hand* we can now create properly typed pointers so we can display the val)es. Eere we create two integer pointers that can be )sed to display the length and capacity val)es from the slice.s data str)ct)re. // 5reate pointers to the length an) cap siNe len#tr (7 ($int)(unsafe.#ointer(len.))r)) cap#tr (7 ($int)(unsafe.#ointer(cap.))r)) We now need a pointer of type SLTstring* which is the type of )nderlying array. // 5reate a pointer to the un)erlying array a))#tr (7 ($U8Vstring)(unsafe.#ointer($($uintptr)(a))ress))) +here is a lot going on in this one statement so let.s break it downB ;E'intptr=;address= B 052?0?be000 +his code takes the starting address of the slice.s data str)ct)re and casts it as a generic pointer. E @Z)intptrA@addressA B 052?0?bd000 +hen we get the val)e that the pointer is pointing to* which is the starting address of the )nderlying array. 'nsa6e5Pointer;Z@Z)intptrA@addressA= +hen we cast the starting address of the )nderlying array to an )nsafe.$ointer type. We need a pointer of this type to perform the final cast. ;EF8Gstring=@)nsafe.$ointer@Z@Z)intptrA@addressAAA

???

Finally we cast the )nsafe.$ointer to a pointer of the proper type. +he remaining code )ses the proper pointers to display the o)tp)tB fmt.#rintf("/lice .))rU'pV Len .))rU1B'BV 5ap .))rU1B'BV*n"+ a))ress+ len.))r+ cap.))r) fmt.#rintf("/lice LengthU')V 5apU')V*n"+ $len#tr+ $cap#tr) for in)eB (7 1< in)eB = $len#tr< in)eB>> { fmt.#rintf("U')V 'p 's*n"+ in)eB+ G($a))#tr)Uin)eBV+ ($a))#tr)Uin)eBV) % &ow let.s p)t the entire program together and create some slices. We will inspect each slice and make s)re everything we know abo)t slices is tr)eB package main import ( "fmt" "unsafe" ) func main() { org/lice (7 org/liceU1V org/liceU0V org/liceUEV org/liceUIV org/liceUJV make(UVstring+ F+ 8) 7 ".pple" 7 "3range" 7 "!anana" 7 "2rape" 7 "#lum"

:nspect/lice(org/lice) sliceE (7 org/liceUE(JV :nspect/lice(sliceE) sliceI (7 sliceEU0(cap(sliceE)V :nspect/lice(sliceI) sliceIU1V 7 "5A.N2@&" :nspect/lice(sliceI) :nspect/lice(sliceE) %

??2

func :nspect/lice(slice UVstring) { // 5apture the a))ress to the slice structure a))ress (7 unsafe.#ointer(Gslice) // 5apture the a))ress where the length an) cap siNe is store) len.))r (7 uintptr(a))ress) > uintptr(8) cap.))r (7 uintptr(a))ress) > uintptr(0S) // 5reate pointers to the length an) cap siNe len#tr (7 ($int)(unsafe.#ointer(len.))r)) cap#tr (7 ($int)(unsafe.#ointer(cap.))r)) // 5reate a pointer to the un)erlying array a))#tr (7 ($U8Vstring)(unsafe.#ointer($($uintptr) (a))ress))) fmt.#rintf("/lice .))rU'pV Len .))rU1B'BV 5ap .))rU1B 'BV*n"+ a))ress+ len.))r+ cap.))r) fmt.#rintf("/lice LengthU')V 5apU')V*n"+ $len#tr+ $cap#tr) for in)eB (7 1< in)eB = $len#tr< in)eB>> { fmt.#rintf("U')V 'p 's*n"+ in)eB+ G($a))#tr)Uin)eBV+ ($a))#tr)Uin)eBV) % fmt.#rintf("*n*n") % Eere is the code and o)tp)t for each sliceB Eere we create the initial slice with a length of G elements and a capacity of L elements. 5o)e( org/lice (7 org/liceU1V org/liceU0V org/liceUEV org/liceUIV org/liceUJV make(UVstring+ F+ 8) 7 ".pple" 7 "3range" 7 "!anana" 7 "2rape" 7 "#lum"

3utput( /lice .))rU1BE010be111V Len .))rU1BE010be118V 5ap ??"

.))rU1BE010be101V /lice LengthUFV 5apU8V U1V 1BE010b)111 .pple U0V 1BE010b)101 3range UEV 1BE010b)1E1 !anana UIV 1BE010b)1I1 2rape UJV 1BE010b)1J1 #lum +he o)tp)t is as e5pected. # length of G* capacity of L and the )nderlying array contains o)r val)es. &e5t we take a slice from the original slice. We ask for 2 elements between inde5es 2 and ". 5o)e( sliceE (7 org/liceUE(JV :nspect/lice(sliceE) 3utput( /lice .))rU1BE010be1S1V Len .))rU1BE010be1S8V 5ap .))rU1BE010be1R1V /lice LengthUEV 5apUSV U1V 1BE010b)1E1 !anana U0V 1BE010b)1I1 2rape In the o)tp)t yo) can see we have a slice with a length of 2 and a capacity of H. /eca)se this new slice is starting " elements into the )nderlying array for the original slice* there is a capacity of H elements. +he capacity incl)des all possible elements that can be accessed by the new slice. Inde5 0 of the new slice maps to inde5 2 of the original slice. +hey both have the same address of 052?0?bd020. +his time we ask for a slice starting from inde5 position ? )p to the last element of slice2. 5o)e( sliceI (7 sliceEU0(cap(sliceE)V :nspect/lice(sliceI) 3utput( /lice .))rU1BE010be1a1V Len .))rU1BE010be1a8V 5ap .))rU1BE010be1b1V /lice LengthUFV 5apUFV U1V 1BE010b)1I1 2rape U0V 1BE010b)1J1 #lum UEV 1BE010b)1F1 UIV 1BE010b)1S1 UJV 1BE010b)1R1 #s e5pected the length and the capacity are both G. When we display all the val)es of the slice yo) see the last three elements don.t have a val)e. +he slice initiali(ed all the elements when the )nderlying array was created. #lso inde5 0 of this slice maps to inde5 ? of slice 2 and inde5 " of the original slice. +hey all have the same address of 052?0?bd0"0.

??8

+he final code changes the val)e of the first element* inde5 0 in slice" to the val)e CE#&96;. +hen we display the val)es for slice" and slice2. sliceIU1V 7 "5A.N2@&" :nspect/lice(sliceI) :nspect/lice(sliceE) /lice .))rU1BE010be1e1V Len .))rU1BE010be1e8V 5ap .))rU1BE010be1f1V /lice LengthUFV 5apUFV U1V 1BE010b)1I1 5A.N2@& U0V 1BE010b)1J1 #lum UEV 1BE010b)1F1 UIV 1BE010b)1S1 UJV 1BE010b)1R1 /lice .))rU1BE010be0E1V Len .))rU1BE010be0E8V 5ap .))rU1BE010be0I1V /lice LengthUEV 5apUSV U1V 1BE010b)1E1 !anana U0V 1BE010b)1I1 5A.N2@& &otice that both slices show the changed val)e in their respect inde5es. +his proves all the slices are )sing the same )nderlying array. +he Inspect3lice f)nction proves that each slice contains its own data str)ct)re with a pointer to an )nderlying array* a length for the slice and a capacity. +ake some time to create more slices and )se the Inspect3lice f)nction to validate yo)r ass)mptions.

.sing $ Dynamic Libraries In Go Programs


My son and I were having f)n last weekend b)ilding a console based game in 9o. I was recreating a game from my yo)th* back when I was programming on a Kaypro II.

I loved this comp)ter. I wo)ld write games in /#3IC on it all day and night. ;id I mention it was portable. +he keyboard wo)ld strap in and yo) co)ld carry it aro)nd. -'-. /)t I digress* back to my 9o program. I fig)red o)t a way to )se the =+?00 escape character ??G

codes to draw o)t a simple screen and started programming some of the logic. +hen something horrible happened and I had a ma0or flashback. I co)ld not get inp)t from stdin witho)t hitting the enter key. #hhhhh I spent all weekend reading )p on how to make this happen. I even fo)nd two 9o libraries that had s)pport for this b)t they didn.t work. I reali(ed that if I was going to make this happen I needed to b)ild the f)nctionality in C and link that to my 9o program. #fter a 8 ho)r coding session at the local Irish p)b* I fig)red it o)t. I wo)ld like to thank 9)inness for the inspiration and enco)ragement I needed. 7nderstand that for the past ?0 years I have been writing windows services in C%. For ?0 years before that I was writing C C!! b)t on the Microsoft stack. 6verything I was readingB gcc* gco* static and shared libraries on the Mac and -in)5* etc* was foreign to me. I had a lot to learn and still do. #fter all my research it became clear I needed to )se the nc)rses dynamic library. I decided to write a simple program in C )sing the library. If I co)ld make it work in a compiled C program* I was s)re I co)ld get it to work in 9o.

+he nc)rses library on the Mac is located in )sr lib. Eere is a link to the doc)mentationB httpsB developer.apple.com library mac doc)mentation ;arwin 1eference Man$ages man" n c)rses."5.html Eere is the C header file code for the test programB

test5h
int 2et5haracter()< voi) :nit;eyboar)()< voi) 5lose;eyboar)()< #nd now the code for the C so)rce fileB

test5c
Oinclu)e =curses.hH Oinclu)e =st)io.hH Oinclu)e "test.h" int main() { :nit;eyboar)()< printf("*n@nter( ")< refresh()< ??H

for (<<) { int r 7 2et5haracter()< printf("'c"+ r)< refresh()< if (r 77 C\C) { break< % % 5lose;eyboar)()< return 1< % voi) :nit;eyboar)() { initscr()< noecho()< cbreak()< keypa)(st)scr+ -6@)< refresh()< % int 2et5haracter() { return getch()< % voi) 5lose;eyboar)() { en)win()< % &ow the hard part. Eow do I b)ild this program )sing the gcc compilerD I want to make s)re I am )sing the same compiler that 9o is )sing. I also want to make s)re I am )sing the bare minim)m parameters and flags. #fter abo)t an ho)r of researching* I came )p with this makefile. I told yo)* I have never done this before.

make6ile
buil)( rm Kf test gcc Kc test.c gcc Klncurses Kr/usr/lib Ko test test.o rm Kf $.o When yo) r)n the make command it will look for a file called makefile in the local directory and e5ec)te it. /e aware that each command is one @?A +#/ to the right. If yo) )se spaces yo) will have problems. 'bvio)sly yo) can r)n these command man)ally as well. I b)ilt this

??I

makefile for convenience. -et.s break down the gcc compiler calls from the makefileB +his call to gcc is creating an ob0ect file called test.o from the so)rce file test.c. +he < c parameter tells gcc to 0)st compile the so)rce file and create an ob0ect file called test.o gcc Kc test.c +his second call to gcc links the test5o ob0ect file together with the shared dynamic library libnc'rses5dylib to create the test e5ec)table file. +he <l @min)s lowercase -A parameter is telling gcc to link the libnc)rses.dylib file and the <r @min)s lowercase 1A parameter is telling gcc where it can find the library. +he <o @min)s lowercase 'A parameter tells gcc to create an e5ec)table o)tp)t file called test and finally we tell gcc to incl)de test.o in the link operation. gcc Klncurses Kr/usr/lib Ko test test.o 'nce these two gcc commands r)n we have a working version of the program called test. :o) m)st r)n this program from a terminal window for it to work. +o e5ec)te the program type 5-test from the terminal windowB

Eere I started typing letters which are being displayed by the call to printf inside the for loop. #s soon as I hit the letter .2.* the program terminates. I now have a working version of a program that )ses the nc)rses dynamic library I want to )se in my 9o program. &ow I need to find a way to wrap these calls into a dynamic library that I can link to from 9o. I was very fort)nate to find these web pages which brilliantly show everything we need to know to create shared and dynamic librariesB httpB www.adpCgmbh.ch cpp gcc createFlib.html httpB stackoverflow.com 2)estions "G"2GL< howCtoCb)ildCaCdylibCfromCseveralCoCinCmacCosC 5C)singCgcc -et.s work together on making all this work in 9o. 3tart with setting )p a Workspace for o)r new pro0ectB

??L

I have created a folder called Keyboard and two s)bCfolders called ;y-ib and +est#pp. Inside the ;y-ib folder we have o)r C based dynamic library code with a makefile. Inside the +est#pp folder we have a single go so)rce code file to test o)r 9o integration with the new dynamic library. Eere is the C header file for the dynamic library. It is identical to the C header file I )sed in the test application.

keyboard5h
int 2et5haracter()< voi) :nit;eyboar)()< voi) 5lose;eyboar)()< Eere is the C so)rce file that implements those f)nctions. #gain it is identical to the C so)rce file from the test application witho)t the main f)nction. We are b)ilding a library so we don.t want main.

keyboard5c
Oinclu)e =curses.hH Oinclu)e "keyboar).h" voi) :nit;eyboar)() { initscr()< noecho()< cbreak()< keypa)(st)scr+ -6@)< refresh()< % int 2et5haracter() { return getch()< % voi) 5lose;eyboar)() { en)win()< % Eere is the makefile for creating the dynamic libraryB

??<

make6ile
)ynamic( rm Kf libkeyboar).)ylib rm Kf ../ est.pp/libkeyboar).)ylib gcc Kc Kf#:5 keyboar).c gcc K)ynamiclib Klncurses Kr/usr/lib Ko libkeyboar).)ylib keyboar).o rm Kf keyboar).o cp libkeyboar).)ylib ../ est.pp/libkeyboar).)ylib share)( rm Kf libkeyboar).so rm Kf ../ est.pp/libkeyboar).so gcc Kc Kf#:5 keyboar).c gcc Kshare) KW0 Klncurses Kr/usr/lib Ksoname+libkeyboar).so Ko libkeyboar).so keyboar).o rm Kf keyboar).o cp libkeyboar).so ../ est.pp/libkeyboar).so With this make file yo) can b)ild either a dynamic library or shared library. If yo) 0)st r)n the make command witho)t any parameters* it will e5ec)te the dynamic set of commands. +o create the shared library r)n make passing .shared. @witho)t 2)otesA as a parameter. +he important flag to notice is <6PI$. +his flag tells gcc to create position independent code which is necessary for shared libraries. We did not incl)de this flag when we b)ilt the e5ec)table program. We are going to )se the dynamic library moving forward. Mainly beca)se on the Mac this is the most common format. #lso* if we clean o)r 9o pro0ect later in -iteI;6* it won.t remove the file along with the binary. -iteI;6 will remove shared libraries on the call to clean. -et.s create the dynamic library by r)nning the make fileB

We call the make command and it r)ns the dynamic section of the make file s)ccessf)lly. 'nce this is done we now have o)r new dynamic library.

?20

&ow we have a new file in both the ;y-ib and +est#pp folders called libkeyboard.dylib. 'ne thing I forgot to mention is that o)r dynamic and shared libraries m)st start with the letters lib. +his is mandatory for things to work correctly later. #lso the library will need to be in the working folder for the program to load it when we r)n the program. -et.s look at the 9o so)rce code file for o)r test applicationB package main /$ Ocgo 5ML.2/( K:../&yLib Ocgo L&ML.2/( KL. Klkeyboar) Oinclu)e =keyboar).hH $/ import "5" import ( "fmt" ) func main() { 5.:nit;eyboar)() fmt.#rintf("*n@nter( ") for { r (7 5.2et5haracter() fmt.#rintf("'c"+ r) if r 77 C\C { break % % 5.5lose;eyboar)() % +he 9o team has p)t together these two doc)ments that e5plain how 9o can incorporate C code directly or )se libraries like we are doing. It is really important to read these doc)ments to better )nderstand this 9o codeB

?2?

httpB golang.org cmd cgo httpB golang.org doc articles cFgoFcgo.html If yo) are interested binding into C!! libraries* then 3WI9 @3implified Wrapper and Interface 9eneratorA is something yo) need to look atB httpB www.swig.org httpB www.swig.org ;oc2.0 9o.html We will leave 3WI9 for another day. For now let.s break down the 9o so)rce code. package main /$ Ocgo 5ML.2/( K:../&yLib Ocgo L&ML.2/( KL. Klkeyboar) Oinclu)e =keyboar).hH $/ import "5" In order to provide the compiler and linker the parameters it needs* we )se these special cgo commands. +hey are always provided inside a set of comments and m)st be on top of the import KCK statement. If there is a gap between the closing comment and the import command yo) will get compiler errors. Eere we are providing the 9o b)ild process flags for the compiling and linking of o)r program. CF-#93 provides parameters to the compiler. We are telling the compiler it can find o)r header files in the 3hared-ib folder. -;F-#93 provide parameters to the linker. We are providing the linker two parameters* C- @min)s capital -A which tells the linker where it can find o)r dynamic library and Cl @min)s lowercase -A which tells the linker the name of o)r library. &otice when we specify the name of o)r library it does not incl)de the lib prefi5 or the e5tension. It is e5pected that the library name starts with lib and ends in either .dylib or the .so e5tensions. -ast we tell 9o to import the se)doCpackage KCK. +his se)doCpackage provides all the 9o level s)pport we need to access o)r library. &one of this is possible witho)t it. -ook at how we call into the each of o)r f)nctions from the libraryB 5.:nit;eyboar)() r (7 5.2et5haracter() 5.5lose;eyboar)() +hanks to the se)doCpackage KCK we have f)nction wrappers for each f)nction from the header file. +hese wrappers handle the marshaling of data in and o)t of o)r f)nctions. &otice how we can )se a native 9o type and synta5 to get the character that is entered into the keyboard.

?22

&ow we can b)ild the test application and r)n it from a terminal sessionB

#wesome. Working like a champ. &ow my son and I can contin)e b)ilding o)r game and get the keyboard action we need to make the game really f)n. It has taken me 2)ite a few n)mber of ho)rs to get a handle on all of this. +here is still a lot to learn and s)pport for this will only get better. #t some point I will look at 3WI9 to incorporate C!! ob0ect oriented libraries. For now* being able to bring in and leverage C libraries is awesome. If yo) want to see and access the code* I have p)t it )p in the 9oing9o gith)b repository )nder Keyboard. Eave F)n ,, 1ead $art IIB 7sing C9' with $kgCConfig #nd C)stom ;ynamic -ibrary -ocations

.sing $G( "ith Pkg<$on6ig )nd $'stom Dynamic Library Locations


6arlier in the month I wrote a post abo)t )sing C ;ynamic -ibraries in 9o $rograms. +he article b)ilt a dynamic library in C and created a 9o program that )sed it. +he program worked b)t only if the dynamic library was in the same folder as the program. +his constraint does not allow for the )se of the go get command to download* b)ild and install a working version of the program. I did not want to have any re2)irements to preC install dependencies or r)n e5tra scripts or commands after the call to go get. +he 9o tool was not going to copy the dynamic library into the bin folder and therefore I wo)ld not be able to r)n the program once the go get command was complete. +his was simply )nacceptable and there had to be a way to make this work. +he sol)tion to this problem was twofold. First* I needed to )se a package config)ration file to specify the compiler and linker options to C9'. 3econd* I needed to set an environment variable so the operating system co)ld find the dynamic library witho)t needing to copy it to the bin folder. If yo) look* yo) will see that some of the standard libraries provide a package config)ration @.pcA file. # special program called pkg<con6ig is )sed by the b)ild tools* s)ch as gcc* to retrieve information from these files.

?2"

If we look in the standard locations for header files* -'sr-lib or -'sr-local-lib, yo) will find a folder called pkgcon6ig. $ackage config)ration files that e5ist in these locations can be fo)nd by the pkg<con6ig program by defa)lt.

-ook at the libcrypto5pc file and yo) can see the format and how it provides compiler and linker information. +his partic)lar file is nice to look at beca)se it incl)des the bare minim)m format and parameters that are re2)ired. +o learn more abo)t these files read this web pageB www.freedesktop.org wiki 3oftware pkgC config +he prefi5 variable at the top of the file is very important. +his variable specifies the base folder location where the library and incl)de files are installed. 3omething very important to note is that yo) canAt 'se an en:ironment :ariable to help speci6y a path location. If yo) do* yo) will have problems with the b)ild tools locating any of the files it needs. +he environment variables end )p being provided to the b)ild tools as a literal string. 1emember this for later beca)se it is important. 1)n the pkg<con6ig program from a +erminal session )sing these parametersB pkgKconfig KKcflags KKlibs libcrypto +hese parameters ask the pkg<con6ig program to show the compiler and linker settings specified in the .pc file called libcrypto. +his is what sho)ld be ret)rnedB B(crypto B(N

?28

-et.s look at one of the package config)ration files from ImageMagick that I downloaded and installed )nder -'sr-local for a pro0ect I am working onB

+his file is a bit more comple5. :o) will notice it specifies that the MagickCode library is also re2)ired and specifies more flags s)ch as environmental variables. When I r)n the pkg<con6ig program on this file I get the following information backB pkgKconfig KKcflags KKlibs MagickWan) Bfopenmp B!M.2:5;53-@WA&-:W@N.!L@71 B!M.2:5;53-@W,6.N 6MW&@# A70S B3/usr/local/inclu)e/:mageMagickKS BH/usr/local/lib B(MagickWan)KS.,0S B(Magick5oreKS.,0S :o) can see that the path locations for the header and library files are f)lly 2)alified paths. #ll of the other flags defined in the package config)ration file are also provided. &ow that we know a bit abo)t package config)ration files and how to )se the pkg<con6ig tool* let.s take a look at the changes I made to the pro0ect for the C ;ynamic -ibraries in 9o $rograms post. +his pro0ect is now )sing a package config)ration file and new cgo parameters. /efore we begin I m)st apologi(e. +he dynamic library that I b)ilt for this pro0ect will only b)ild on the Mac. 1ead the post I 0)st mentioned to )nderstand why. # preCb)ilt version of the dynamic library already e5ists in version control. If yo) are not working on a Mac* the pro0ect will not b)ild properly* however all the ideas* settings and constr)cts still apply. 'pen a +erminal window and r)n the following commandsB c) ]A3M@ eBport 23#. A7]A3M@/keyboar) eBport #;2W53NM:2W#. A7]23#. A/src/github.com/goinggo/keyboar)/pkgcon ?2G

fig eBport &"L&WL:!-.-"W#. A7]23#. A/src/github.com/goinggo/keyboar)/&yLi b go get github.com/goinggo/keyboar) #fter yo) r)n these commands* yo) will have all the code from the 9oing9o keyboard repository downloaded )nder a s)bfolder called keyboard inside yo)r home directory.

:o) will notice the 9o tool was able to download* b)ild and install the keyboard program. 6ven tho)gh the header file and dynamic library was not located in the defa)lt -'sr or -'sr-local folders. In the bin folder we have the single e5ec)table program witho)t the dynamic library. +he dynamic library is only located in the ;y-ib folder. +here is a new folder in the pro0ect now called pkgconfig. +his folder contains the package config)ration file that makes this all possible. +he main.go so)rce code file has been changed to take advantage of the new package config)ration file. If we immediately switch to the bin folder and r)n the program* we will see that it works. c) ]23#. A/bin ./keyboar)

?2H

When yo) start the program* it immediately asks yo) to enter some keys. +ype a few letters and then hit the letter 2 to 2)it the program. +his is only possible if the '3 can find all the dynamic libraries this program is dependent on. -et.s take a look at the code changes that make this possible. -ook at the main.go so)rce code file to see how we reference the new package config)ration file. +his is the original code from the first post. In this version I specified the compiler and linker flags directly. +he location of the header and dynamic library are referenced with a relative path. package main /$ Ocgo 5ML.2/( K:../&yLib Ocgo L&ML.2/( KL. Klkeyboar) Oinclu)e =keyboar).hH $/ import "5" +his is the new code. Eere I tell C9' to )se the pkg<con6ig program to find the compiler and linker flags. +he name of the package config)ration file is specified at the end. package main /$ Qcgo pkgBconfig: BBdefineBvaria$(e=prefi#=. NoingNoRe=$oard Oinclu)e =keyboar).hH $/ import "5" &otice the )se of the pkg<con6ig program option <<de6ine<:ariable. +his option is the trick to making everything work. -et.s get back to that in a moment. 1)n the pkg<con6ig program against o)r new package config)ration fileB pkgKconfig KKcflags KKlibs 2oing2o;eyboar) K:PN5PA6S/src/github.com/goinggo/keyboar)/&yLib KLPN5PA6S/src/github.com/goinggo/keyboar)/&yLib Klkeyboar)

?2I

If yo) look closely at the o)tp)t from the call* yo) will see something that I told yo) was wrong. +he >9'$#+E environment variable is being provided. 'pen the package config file which is located in the pkgconfig folder and yo) will see the pkg<con6ig program doesn.t lie. 1ight there at the top I am setting the prefi5 variable to a path )sing >9'$#+E. 3o why is everything workingD &ow r)n the command again )sing the same <<de6ine<:ariable option we are )sing in main.goB pkgKconfig KKcflags KKlibs 2oing2o;eyboar) BBdefineB varia$(e=prefi#=. K:./&yLib KL./&yLib Klkeyboar) ;o yo) see the differenceD In the first call to the pkg<con6ig program we get back paths that have the literal >9'$#+E string beca)se that is how the prefi5 variable is set. In the second call we override the val)e of the prefi5 variable to the c)rrent directory. What we get back is e5actly what we need. 1emember this environment variable that we set prior to )sing the 9o toolD #;2W53NM:2W#. A7]23#. A/src/github.com/goinggo/keyboar)/pkgcon fig +he $K9FC'&FI9F$#+E environment variable tells the pkg<con6ig program where it can find package config)ration files that are not located in any of the defa)lt locations. +his is how the pkg<con6ig program is able to find o)r 9oing9oKeyboard.pc file. +he last mystery to e5plain is how the '3 can find the dynamic library when we r)n the program. 1emember this environment variable that we set prior to )sing the 9o toolD eBport &"L&WL:!-.-"W#. A7]23#. A/src/github.com/goinggo/keyboar)/&yLi b +he ;:-;F-I/1#1:F$#+E environment variable tells the '3 where else it can look for dynamic libraries.

?2L

Installing yo)r dynamic libraries in the -'sr-local folder keeps things simple. #ll of the b)ild tools are config)red to look in this folder by defa)lt. Eowever* )sing the defa)lt locations for yo)r c)stom or third party libraries re2)ire e5tra steps of installation prior to r)nning the 9o tools. /y )sing a package config)ration file and passing the pkg<con6ig program the options it needs* 9o with C9' can deploy b)ilds that will install and be ready to r)n instantly. 3omething else I didn.t mention is that yo) can )se this techni2)e to install "rd party libraries that yo) may be trying o)t in a temp location. +his makes it real easy to remove the library if yo) decide yo) don.t want to )se it. If yo) want to play with the code or concepts on a Windows or 7b)nt) machine* read C ;ynamic -ibraries in 9o $rograms to learn how to b)ild yo)r own dynamic libraries that yo) can e5periment with.

$ollections (6 .nkno"n Length in Go


If yo) are coming to 9o after )sing a programming lang)age like C% or Oava* the first thing yo) will discover is that there are no traditional collection types like -ist and ;ictionary. +hat really threw me off for months. I fo)nd a package called container list and gravitated to )sing it for almost everything. 3omething in the back of my head kept nagging me. It didn.t make any sense that the lang)age designers wo)ld not directly s)pport managing a collection of )nknown length. 6veryone talks abo)t how slices are widely )sed in the lang)age and here I am only )sing them when I have a well defined capacity or they are ret)rned to me by some f)nction. 3omething is wrong,, 3o I wrote an article earlier in the month that took the covers off of slices in a hope that I wo)ld find some magic that I was missing. I now know how slices work b)t at the end of the day* I still had an array that wo)ld have to grow. I was ta)ght in school that linked lists were more efficient and gave yo) a better way to store large collections of data. 6specially when the n)mber of items yo) need to collect is )nknown. It made sense to me. When I tho)ght abo)t )sing an empty slice* I had this very W1'&9 pict)re in my headB

?2<

I kept thinking how 9o wo)ld be creating a lot of new slice ob0ects and lots of other memory allocations for the )nderlying array with val)es constantly being copied. +hen the garbage collector wo)ld be overworked beca)se of all these little ob0ects being created and discarded. I co)ld not imagine having to do this potentially tho)sands of times. +here had to be a better way or efficiencies that I was not aware of. #fter researching and asking a lot of 2)estions* I came to the concl)sion that in most practical cases )sing a slice is better than )sing a linked list. +his is why the lang)age designers have spent the time making slices work as efficiently as possible and didn.t introd)ce collection types into the lang)age. We can talk abo)t edge cases and performance n)mbers all day long b)t 9o wants yo) to )se slices and therefore it sho)ld be o)r first choice )ntil the code tells )s otherwise. 7nderstand that slices are like the game of chess* easy to learn b)t takes a lifetime to master. +here are gotchas and things yo) need to be aware of* beca)se the )nderlying array can be shared. &ow is a good time to read my post* 7nderstanding 3lices in 9o $rogramming* before contin)ing. +he rest of this post will e5plain how to )se a slice when dealing with an )nknown capacity and what is happening )nderneath. Eere is an e5ample of how to )se an empty slice to manage a collection of )nknown lengthB package main import ( "fmt" "math/ran)" "time" ?"0

) type -ecor) struct { :) int Name string 5olor string % func main() { // LetCs keep things unknown ran)om (7 ran).New(ran).New/ource(time.Now().6niB())) // 5reate a large slice preten)ing we retrieve) )ata // from a )atabase )ata (7 make(UV-ecor)+ 0111) // 5reate the )ata set for recor) (7 1< recor) = 0111< recor)>> { pick (7 ran)om.:ntn(01) color (7 "-e)" if pick 77 E { color 7 "!lue" % )ataUrecor)V 7 -ecor){ :)( recor)+ Name( fmt./printf("-ec( ')"+ recor))+ 5olor( color+ % % // /plit the recor)s by color re) (7 UV$-ecor){% blue (7 UV$-ecor){% for W+ recor) (7 range )ata { if recor).5olor 77 "-e)" { re) 7 appen)(re)+ Grecor)) % else { blue 7 appen)(blue+ Grecor)) % % // &isplay the counts fmt.#rintf("-e)U')V !lueU')V*n"+ len(re))+ len(blue)) % When we r)n this program we will get different co)nts for the red and bl)e slices thanks to the randomi(er. We don.t know what capacity we need for the red or bl)e slices ahead of time. +his is a typical sit)ation for me. ?"?

-et.s break down the more important pieces of the codeB +hese two lines of code create an empty slice. re) (7 UV$-ecor){% blue (7 UV$-ecor){% +his synta5 will also create an empty slice. re) (7 make(UV$-ecor)+ 1) blue (7 make(UV$-ecor)+ 1) In both cases we have a slice whose length and capacity is 0. +o add items to the slice we )se the b)ilt in f)nction called appendB re) 7 appen)(re)+ Grecor)) blue 7 appen)(blue+ Grecor)) +he append f)nction is really cool and does a b)nch of st)ff for )s. Kevin 9illette wrote this in the gro)p disc)ssion I startedB @httpsB gro)ps.google.com for)m %,topic golangCn)ts nN:)MNGGbHcA In terms of (o spe&ifi&s) append doubles &apa&ity ea&h reallo&ation for the first few thousand elements) and then progresses at a rate of *+.,# &apa&ity growth after that. I am not an academic b)t I see the )se of tilde @MA 2)ite a bit. For those of yo) who don.t know what that means* it means appro5imately. 3o the append f)nction will increase the capacity of the )nderlying array to make room for f)t)re growth. 6vent)ally append will grow capacity appro5imately by a factor of ?.2G or 2GU. -et.s prove that append is growing capacity to make things more efficientB package main import ( "fmt" "reflect" "unsafe" ) func main() { )ata (7 UVstring{% for recor) (7 1< recor) = 01F1< recor)>> { )ata 7 appen)()ata+ fmt./printf("-ec( ')"+ recor))) if recor) = 01 [[ recor) 77 EFS [[ recor) 77 F0E [[ recor) 77 01EJ { sliceAea)er (7 ($reflect./liceAea)er) ?"2

((unsafe.#ointer(G)ata))) fmt.#rintf(":n)eBU')V LenU')V 5apU')V*n"+ recor)+ sliceAea)er.Len+ sliceAea)er.5ap) % % % Eere is the o)tp)tB :n)eBU1V LenU0V 5apU0V :n)eBU0V LenUEV 5apUEV 3nde#;2< Hen;%< 1ap;?< 5apacity :n)eBUIV LenUJV 5apUJV 3nde#;?< Hen;L< 1ap;D< 5apacity :n)eBUFV LenUSV 5apU8V :n)eBUSV LenURV 5apU8V :n)eBURV LenU8V 5apU8V 3nde#;D< Hen;T< 1ap;1>< 5apacity :n)eBUQV LenU01V 5apU0SV 3nde#;2L>< Hen;2L7< 1ap;L12< 5apacity 3nde#;L12< Hen;L1%< 1ap;1"2?< 5apacity 3nde#;1"2?< Hen;1"2L< 1ap;12D"< factor of 0.EF

K -an 3ut 3f -oom+ &ouble K -an 3ut 3f -oom+ &ouble

K -an 3ut 3f -oom+ &ouble K -an 3ut 3f -oom+ &ouble K -an 3ut 3f -oom+ &ouble K -an 3ut 3f -oom+ 2row by a

If we look at the capacity val)es we can see that Kevin is absol)tely correct. +he capacity is growing e5actly as he stated. Within the first ?k of elements* capacity is do)bled. +hen capacity is being grown by a factor of ?.2G or 2GU. +his means that )sing a slice in this way will yield the performance we need for most sit)ations and memory won.t be an iss)e. 'riginally I tho)ght that a new slice ob0ect wo)ld be created for each call to append b)t this isn.t the case. # copy of red is being copied on the stack when we make the call to append. +hen when append ret)rns* another copy is made )sing the e5isting memory we already had. red 7 appen)(red+ Grecor)) In this case the garbage collector is not involved so we have no iss)es with performance or memory at all. My C% and reference type mentality strikes me down again. Eold on to yo)r seat beca)se there are changes coming to slices in the ne5t release. ;ominik Eonnef has started a blog that e5plains* in plain english @+hank :o)A* what is being worked on in 9o tip. +hese are things coming in the ne5t release. Eere is a link to his awesome blog and the section on slices. 1ead it* it is really awesome. ?""

httpB dominik.honnef.co goCtip httpB dominik.honnef.co goCtip 20?"C0LC2" %slicing +here is so m)ch more yo) can do with slices. 3o m)ch so that yo) co)ld write an entire book on the s)b0ect. -ike I said earlier* slices are like chess* easy to learn b)t takes a lifetime to master. If yo) are coming from other lang)ages like C% and Oava then embrace the slice and )se it. It is the 9o way.

(rganiDing $ode to S'pport Go Get


For those of yo) who are like me* trying to learn the Mac and -in)5 operating systems* 9olang programming and deployment constr)cts all at the same time* I feel yo)r pain. I have been b)ilding a 9o application for a co)ple of months on my Mac and it was time to deploy the code on a local 7b)nt) server. I was having a really to)gh time and it was t)rning into a disaster. -ike always* I kept telling myself* I m)st be doing something wrong. Well* I was* bigCtime. -et me p)t it this way. #fter spending 2 days reorgani(ing repositories and code* I finally fig)red o)t how the 9o tool works. &ow I can )pdate my development environment* deploy* b)ild* install and )pdate my 9o applications on any machine with one simple call to go get. I am s)re there are several ways yo) can organi(e code that will work with the 9o tool. What I am going to present is working for me and I wish I knew all of this when I started. +here are a ?000 ways to skin a cat and we all have o)r favorite way* this one is now mine. 6verything we do in 9o sho)ld be based on what the 9o tool can do. 9oing off the reservation will get yo) in tro)ble* I am living proof. 'ne special command we are going to foc)s on is get. +he get command will download code* b)ild and install packages and prod)ce an e5ec)table binary if yo) are b)ilding a program. It is very smart and can read yo)r code to find dependences that it will download* b)ilt and install as well. +hat is* if everything is str)ct)red and set correctly. +his doc)ment e5plains all the things the 9o tool can doB httpB golang.org cmd go /)t I want to concentrate on this sectionB httpB golang.org cmd go %hdrC1emoteFimportFpathFsynta5 6ven as I read this doc)ment now I have a hard time )nderstanding what it is trying to tell me. I )se 9ith)b b)t yo) don.t have to. +he 9o tool s)pports 9ith)b* Merc)rial* 3)bversion and /a(aar o)t of the bo5. If yo) are not )sing any of these systems for version control* there are ways to give the 9o tool the information it needs to s)pport yo)r version control system. When I say the 9o tool s)pports these version control systems it.s a bit vag)e right now* so let.s 0)mp into this. 6verything starts with a repository* so let.s look at the ones I have for my ?"8

pro0ect. I have two acco)nts in 9ith)b. 'ne is called goinggo which contains all my shared and re)sable repositories. +he other is )rdanSt'dios which contains my private repositories. Eere is the goinggo repositoryB

Choosing a name for yo)r repository is really important. +he name of the repository is going to be )sed later to reference the code. Eaving a good name makes things easier for everyone. :o) can see I have 2 of my repositories listed. /oth of these repositories contain re)sable code and are referenced in my pro0ect. When I first p)t my repository together for the 9oing9o.net website* I p)ll all these pro0ects )nder a single repository. +his ended )p being a very bad idea. It is best to organi(e yo)r code in different repositories that may or may not be needed depending on the pro0ect. #s an e5ample* I will never need the newsearch code for any pro0ect I am b)ilding. +hat was an application I b)ilt for an article I wrote. When I had that code in the same repository as the )tilities code* and I reference 0)st one code file from )tilities* the 9o tool still p)lled down everything in that repository. &ot 9ood. -et.s look at the )tilities repositoryB

?"G

:o) will notice the repository has a folder with a ma0or version n)mber. +here is a good amo)nt of debate and research going on in the 9o comm)nity abo)t package management and versioning. +his is a complicated problem with many twists and t)rns. +here is a doc)ment that is being worked on by several people who are doing the research and leading the disc)ssion. Eere is that doc)mentB httpsB docs.google.com doc)ment d ?FIO+1;Hd;4vyCfyim8KOe52LPrK7$v7d"9M3#Lc wL#8 edit%headingJh.w82)a2l"id)f 3ince we can.t wait for the final o)tcome to this debate* I have chosen to str)ct)re my repository with a ma0or version n)mber and with the idea that f)t)re versions of the 9o tool will provide better version control system s)pport. I also imagine that we will event)ally have a standard package manager that will provide all the s)pport we need to make this manageable and fle5ible. +here is another reason for )sing a ma0or version n)mber. I don.t want )sers to worry abo)t making changes to the references of this package every time I have a new minor or patch release. 3ince minor versions and patches are not allowed to break e5isting interfaces* this reference to 0)st the ma0or version is eno)gh. 'bvio)sly I m)st comply with this r)le and give a personal g)arantee. If I want to work on version 2* which will not be compatible with version ?* I can create a new v2 folder and not affect those who rely on the code. +his also gives others confidence that they can )se and rely on my package* which is very important. Many feel that tr)sting others blindly to not break compatibility is too risky. I can )nderstand that position. 'ne option to mitigate that risk is to take a copy of the packages yo) want to )se and repo it yo)rself. /eing able to do this depends on the license for the code so check that first.

?"H

:o) will also notice I have two branches on this code. # develop branch and a master branch. +hanks to the 9it Flow tool this is really easy to set)p. +his allows me to work in the develop branch* making changes to the code witho)t affecting those who are )sing the master branch. $eople can )se the develop branch if they wish and start testing new feat)res and b)g fi5es. I can also get feedback before any final release. 'nce I feel the new version is stable* I can merge and p)sh the final changes into master and tell the world the release is ready. It is good practice to tag or label yo)r branches* especially in master after a release of code. +his will allow access to previo)s releases if it is necessary. In the case of 9ith)b* the 9o tool is going to download and )se master. If yo) are b)ilding a package for the masses* branching and versioning is something yo) want to work o)t in the beginning. -et.s look at the pro0ect code that is sitting in the 9ith)b repository )nder my private acco)ntB

:o) can see this repository is )nder my )rdanSt'dios acco)nt and I have a single so)rce code file called main.go. :o) can also see some of the internal package folders that make )p the pro0ect. -et.s look inside the mongo folder and view the mongo.go code file that is in there.

?"I

+his single code file in the mongo folder provides connection s)pport for accessing my Mongo;/ database.

What is important to see is the import references. &otice that the references to the goinggo )tilities* mongo driver and even the internal helper package is all done with a f)ll )rl 2)alifier. +his is going to allow the 9o tool to download these packages for )s when we are ready to b)ild and install o)r program. For all this to work* in both o)r development and prod)ction environments* the code m)st reside in physical folders that match these )rl paths.

+he $ro0ects and $)blic$ackages folders in my development environment are located )nder a parent folder called 9o. 7nder each of these folders I have a special 9o folder called src. In order for the 9o tool to find the so)rce code for the packages we import* the code m)st

?"L

reside )nderneath a folder called src. In my development environment* the 9'$#+E contains these two folderB >E'M6 3paces 9o $ro0ects >E'M6 3paces 9o $)blic$ackages &otice the directories we add to the 9'$#+E variable point to the src folders. +he 9o tool ass)mes there is a src folder immediately following each folder listed in the 9'$#+E. If yo) look at the folders )nderneath src yo) will see the entire )rl is laid o)t as s)bCfolders and then the code folders and files followB

In my development environment I like having a folder called $ro0ects and $)blic$ackages. Code in the $)blic$ackages folder is 0)st that* p)blic code that I don.t own and I am )sing in my pro0ects. I co)ld keep a single 9'$#+E and p)t the p)blic code and my pro0ect code )nder a single folder. +here is nothing wrong with that. I 0)st like separating the code that I own from the code I don.t. +o b)ild o)t the $)blic$ackages folder yo) m)st man)ally bring down each repository yo)rself )sing to 9o tool. -et.s say yo) wanted to )se the 9oing9o )tilities code in yo)r dev environment. Eere is what yo) do. 'pen a terminal session and r)n these commandsB c) ]A3M@ eBport 23#. A7]A3M@/eBample go env go get github.com/goinggo/utilities ?"<

I always r)n go en: after I set the 9'$#+E to do)ble check that it is properly set. In this e5ample it sho)ld say 9'$#+EJK 7sers bill e5ampleK. When yo) r)n the go get command on the goinggo )tilities repository* yo) will get the following messageB package github.com/goinggo/utilities imports github.com/goinggo/utilities imports github.com/goinggo/utilities( no 2o source files in /6sers/bill/eBample/src/github.com/goinggo/utilities +his is beca)se there is nothing for the 9o tool to b)ild. It.s ok beca)se this package 0)st provides )tility code that will be b)ilt by each individ)al pro0ect. If yo) remember* I pointed o)t how there was a main.go file in the root of my pro0ect code. +his is why. I want to make s)re the 9o tool finds the main so)rce code file to b)ild the application. :o) can specify additional paths in the call to go get if the code yo) want to b)ild is not in the root of the pro0ect. 3omething like thisB go get github.com/goinggo/utilities/workpool When this r)ns yo) will not get any warnings and a static library file for workpool will e5ist in the package folder. #ll of the code for the specified repository will still be downloaded. #dding the e5tra folders to the end of the repository )rl only tell the 9o tool where to start b)ilding the code. When we open the e5ample folder we see the 9o tool created the entire tree str)ct)reB

What is even better is we have a cloned 9ith)b repository. +he image on the right shows the hidden files for the .git folder. Why is this importantD #nytime we want to )pdate the code we can r)n the 9o tool againB

?80

c) ]A3M@ eBport 23#. A7]A3M@/eBample go env go get B+ github.com/goinggo/utilities 7sing the <' option will perform an )pdate to the local repository. 3etting )p the $)blic$ackages folder for yo)r development environment will keep one version of all the packages yo) )se in a single place )nder a single 9'$#+E folder. Minimi(ing the n)mber of 9'$#+E folders in yo)r development environment is always a good thing. :o) can also )pdate any of the p)blic code if yo) need to very 2)ickly. &e5t let.s sim)late a prod)ction b)ild and installation )sing the Mongo 1)les program. +his is going to show yo) the real power of the 9o tool when we str)ct)re o)r repositories and code correctly. /efore we can try this* we need to install the /a(aar program. Mongo 1)les references the labi5.org mgo driver. +he mgo driver is being held in a /a(aar version control system and the 9o tool can not download the code witho)t it. +his is a great e5ample of a pro0ect that is getting code from m)ltiple types of repositories and version control systems. If yo) are r)nning on a Mac or Windows )se these links and follow the instr)ctionsB httpB wiki.ba(aar.canonical.com Mac'3N;ownloads httpB wiki.ba(aar.canonical.com Windows;ownloads If yo) are r)nning on -in)5 0)st r)n aptCgetB su)o aptKget install bNr With /a((ar installed we can )se the 9o tool to download and install the Mongo 1)les program in o)r sim)lated prod)ction environment. ZZ W#I+ ZZ It is important that the 9'/I& environment variable is not set. If this variable is set then the 9o tool will attempt to install the Mongo 1)les program in the location specified by the variable. +his is not what we want. +o clear the variable if it is set* iss)e this callB eBport 23!:N7 &ow r)n these commandsB c) ]A3M@ eBport 23#. A7]A3M@/eBample go env go get github.com/goinggo/mongorules #fter the 9o tool is done we have a b)ild and installed program that is ready to go. -ook at the directory str)ct)re after the callB

?8?

Eow cool is this ,, With a single call to go get all the code* incl)ding the packages the code depends on is downloaded* b)ilt and then installed. If yo) look at the bin folder yo) will see the e5ec)table binary for the Mongo 1)les program that was b)ilt. Inside the pkg folders we have the static library files that were prod)ced when go get performed the b)ild. In the src folder yo) can see all the code* incl)ding the code from the labi5.org website that was downloaded. +he 9o tool looked at all the p)blic references in the code and downloaded and b)ilt everything it needed to create a final e5ec)table.

?82

What is also really nice is that everything works within a single directory from o)r 9'$#+E. 9'$#+EJ>E'M6 e5ample If yo) want to learn more abo)t this program check o)t this article I wrote for 3afari /ooks 'nline. It talks abo)t how yo) can )se Mongo;/ and 9o to analy(e data. httpB www.goinggo.net 20?" 0I analy(eCdataCwithCmongodbCandCgo.html #ll of this knowledge came to light when I needed to b)ild* deploy and install my program on a different '3 and machine. I really str)ggled to create a b)ild package that wo)ld install all these dependencies and p)t everything in the right place. -ittle did I know a few days ago that the 9o tool does all this for yo). :o) 0)st need to know how it works. Eopef)lly now* yo) do as well.

Timer >o'tines )nd Grace6'l Sh'tdo"ns In Go


In my ')tcast data server I have several data retrieval 0obs that r)n )sing different go ro)tines. 6ach ro)tine wakes )p on a set interval. +he most comple5 0ob is the downloading of radar images. What makes this comple5 is that there are ?GG radar stations thro)gho)t the 7nited 3tates that take a new pict)re every ?20 seconds. #ll these radar images can be p)t together to create a mosaic. When the go ro)tine wakes )p to p)ll down the new images* it m)st do this as 2)ickly as possible for all ?GG stations. If it doesn.t* the mosaics will be o)t of sync and any overlays across station bo)ndaries will look off.

?8"

+he radar image on the left is for +ampa /ay at 8BG? $M 63+. :o) can see the coverage of that radar station crosses over a large area of the state of Florida. +his radar image act)ally c)ts into several other radar stations incl)ding Miami. +he radar image on the right is for Miami at 8BG" $M 63+. +here is a two min)te difference* or what I call glare* between these two radar images. When we overlay these two images on a map yo) wo)ld not notice any difference. Eowever* if the glare between these images get any greater than a co)ple of min)tes* it can become obvio)s to the naked eye.

?88

+he bl)e colors are radar noise that gets filtered o)t* so we are left with greens* reds and yellows that represent real weather. +hese images were downloaded and cleaned at 8B8H $M 63+. :o) can see they are pretty close and wo)ld overlay well. +he first implementation of the code )sed a single go ro)tine on a ?0 min)te interval. When the go ro)tine woke )p it wo)ld take " to 8 min)tes to download* process* store and write a record to mongo for all ?GG stations. 6ven tho)gh I wo)ld process each region as close together as possible* the glare between the images was too great. +he radar stations already contain a glare of one to two min)tes so adding another one to two min)tes more presented a problem. I always try to )se a single ro)tine if I can for any work that needs to be performed* 0)st to keep things simple. In this case one ro)tine didn.t work. I needed to process m)ltiple stations at the same time and red)ce the amo)nt of glare between the images. #fter adding a work pool to process m)ltiple stations at once* I was able to process all ?GG stations in )nder a min)te. 3o far I have received no complaints from the client team. In this post we are going to concentrate on the timer ro)tine and sh)tdown code. In the ne5t

?8G

post I will show yo) how add a work pool to the sol)tion. I have attempted to provide a complete working code sample. It sho)ld work as a good template for yo)r own implementations. +o download and r)n the code* open a terminal session and iss)e the following commandsB c) ]A3M@ eBport 23#. A7]A3M@/eBample go get github.com/goinggo/timer)esignpattern c) eBample/bin ./timer)esignpattern +he ')tcast data server is a single application that is started and hopef)lly r)ns for long period of time. 'ccasionally these types of applications do have to be sh)t down. It is important that yo) can always sh)t down yo)r application gracef)lly on demand. When I am developing these types of applications* I always make s)re* right from the beginning* I can signal the application to terminate and it does witho)t hanging. +here is nothing worse than an application that yo) need to kill by force. +he sample program creates a single go ro)tine and tells the ro)tine to wake )p every ?G seconds. When the ro)tine wakes )p* it performs ?0 seconds of work. When the work is over* it calc)lates the amo)nt of time it needs to sleep so it can wake )p on that ?G second cycle again. -et.s r)n the application and sh)t it down while it is r)nning. +hen we can learn how it all works. We can sh)tdown the program by hitting the enter key at any time. Eere is the program r)nning and being sh)t down I seconds laterB E10IK1QK1J 08(F8(JF.F1F ( main ( main ( /tarting #rogram E10IK1QK1J 08(F8(JF.F1F ( main ( workmanager./tart+p : /tarted E10IK1QK1J 08(F8(JF.F1F ( main ( workmanager./tartup ( 5omplete) E10IK1QK1J 08(F8(JF.F1F ( Work imer ( WWorkManager.2o-outineWWork imer ( /tarte) E10IK1QK1J 08(LD:?L.L"L ( Work imer ( WWorkManager.2o-outineWWork imer ( :nfo ( Gait 6o 4+n : /econd,;1L< E10IK1QK1J 08(LD:L2.>>> ( main ( workmanager./2+tdo.n : /tarted E10IK1QK1J 08(F8(FE.SSS ( main ( workmanager./hut)own ( :nfo ( /hutting &own E10IK1QK1J 08(F8(FE.SSS ( main ( workmanager./hut)own ( :nfo ( /hutting &own Work imer E10IK1QK1J 08(F8(FE.SSS ( Work imer ( WWorkManager.2o-outineWWork imer ( /hutting &own E10IK1QK1J 08(F8(FE.SSS ( main ( workmanager./hut)own ( 5omplete) E10IK1QK1J 08(F8(FE.SSS ( main ( main ( #rogram 5omplete

?8H

+his is a great first test. #s soon as we tell the program to sh)tdown* it does and gracef)lly. &e5t let.s have the program start it.s work and try to sh)t it downB E10IK1QK1J 0Q(0J(E0.I0E ( main ( main ( /tarting #rogram E10IK1QK1J 0Q(0J(E0.I0E ( main ( workmanager./tart+p : /tarted E10IK1QK1J 0Q(0J(E0.I0E ( main ( workmanager./tartup ( 5omplete) E10IK1QK1J 0Q(0J(E0.I0E ( Work imer ( WWorkManager.2o-outineWWork imer ( /tarte) E10IK1QK1J 0Q(0J(E0.I0I ( Work imer ( WWorkManager.2o-outineWWork imer ( :nfo ( Gait 6o 4+n : /econd,;1L< E10IK1QK1J 0Q(0J(IS.I0I ( Work imer ( WWorkManager.2o-outineWWork imer ( Goke Cp E10IK1QK1J 0Q(0J(IS.I0I ( Work imer ( WWorkManager.2o-outineWWork imer ( /tarte) E10IK1QK1J 0Q(0J(IS.I0I ( Work imer ( WWorkManager.2o-outineWWork imer ( #rocessing :mages Mor /tation ( 1 E10IK1QK1J 0Q(0J(IS.FSJ ( Work imer ( WWorkManager.2o-outineWWork imer ( #rocessing :mages Mor /tation ( 0 E10IK1QK1J 0Q(0J(IS.80F ( Work imer ( WWorkManager.2o-outineWWork imer ( #rocessing :mages Mor /tation ( E E10IK1QK1J 0Q(1?:%7.">L ( Work imer ( WWorkManager.2o-outineWWork imer ( #rocessing :mages Mor /tation ( I E10IK1QK1J 0Q(1?:%7.12T ( main ( workmanager./2+tdo.n : /tarted E10IK1QK1J 0Q(0J(IR.0EQ ( main ( workmanager./hut)own ( :nfo ( /hutting &own E10IK1QK1J 0Q(0J(IR.0EQ ( main ( workmanager./hut)own ( :nfo ( /hutting &own Work imer E10IK1QK1J 0Q(0J(IR.I0F ( Work imer ( WWorkManager.2o-outineWWork imer ( :nfo ( -e\uest o /hut)own E10IK1QK1J 0Q(0J(IR.I0F ( Work imer ( WWorkManager.2o-outineWWork imer ( :nfo ( Wait o -un ( /econ)sU0JV E10IK1QK1J 0Q(0J(IR.I0F ( Work imer ( WWorkManager.2o-outineWWork imer ( /hutting &own E10IK1QK1J 0Q(0J(IR.I0S ( main ( workmanager./hut)own ( 5omplete) E10IK1QK1J 0Q(0J(IR.I0S ( main ( main ( #rogram 5omplete +his time I waited the ?G seconds and let the work begin. #fter it finished processing the forth image* I told the program to sh)tdown. It did so immediately and gracef)lly.

?8I

-et.s look at the core piece of the code that implement the timer ro)tine and the gracef)l sh)tdownB func (this $WWorkManager) 2o-outineWWork imer() { wait (7 :M@-W#@-:3& for { select { case =Kthis./hut)own5hannel( this./hut)own5hannel =K "&own" return case =Ktime..fter(wait)( break % start ime (7 time.Now() this.#erform heWork() en) ime (7 time.Now() )uration (7 en) ime./ub(start ime) wait 7 :M@-W#@-:3& K )uration % % I have removed all the comments and logging to make it easier to read. +his is classic channels at work and the sol)tion is really elegant. 6legant compared to how something like this needs to be implemented in C%. +he 9o1o)tineFWork+imer f)nction r)ns as a 9o 1o)tine and is started with the keyword goB func /tartup() (err error) { W his 7 GWWorkManager{ /hut)own( false+ /hut)own5hannel( make(chan string)+ % go 062i,.No4o+tine0Gork6imer() return err % +he WorkManager is created as a singleton and then the timer ro)tine is started. +here is a single channel for sh)tting down the timer ro)tine and a flag to denote when the system is sh)tting down. +he timer ro)tine r)ns inside an endless for loop so it does not terminate )ntil we ask it to. -et.s look at the channel related code inside the for loopB

?8L

select { case =Kthis./hut)own5hannel( this./hut)own5hannel =K "&own" return case =Ktime..fter(wait)( break % this.#erform heWork() We are )sing a special keyword called select. Eere is 9o doc)mentation on the keyword selectB httpB golang.org ref spec%3electFstatements We are )sing the select statement to keep the timer ro)tine asleep )ntil it is time to perform work or time to sh)t down. +he select p)ts the timer ro)tine to sleep )ntil one of the channels are signaled. 'nly one case will e5ec)te at a time* making the code synchrono)s. +his really helps keep things simple and allows )s to r)n atomic* Kro)tine safeK* operations across the m)ltiple channels cased inside the select. +he select in the timer ro)tine contains two channels* one for sh)tting down the ro)tine and one for performing the work. 3h)tting down the ro)tine is performed by the following codeB func /hut)own() (err error) { W his./hut)own 7 true W his./hut)own5hannel =K "&own" =KWthis./hut)own5hannel close(W his./hut)own5hannel) return err % When it is time to sh)t down* we set the 3h)t;own flag to tr)e and then signal the 3h)t;ownChannel by passing the string K;ownK thro)gh the channel. +hen we wait for a response back from the timer ro)tine. +his comm)nication of data synchroni(es the entire sh)tdown process between the main ro)tine and the timer ro)tine. 1eally nice* simple and elegant. +o wake )p on an interval )sing the select statement* I )se a special f)nction called time.#fter. +his f)nction waits for the specified d)ration to elapse and then ret)rns the c)rrent time on a signaled channel. +his wakes )p the select allowing the $erform+heWork f)nction to be e5ec)ted. 'nce the $erform+heWork f)nction ret)rns* the timer ro)tine is p)t back to sleep by the select statement again* )nless another channel is in the signaled state. -et.s look at the $erform+heWork f)nctionB

?8<

func (this $WWorkManager) #erform heWork() { for count (7 1< count = J1< count>> { if this./hut)own 77 true { return % fmt.#rintf("#rocessing :mages Mor /tation ( ')"+ count) time./leep(time.Millisecon) $ EF1) % % +he f)nction is printing a message to the console window 80 times every 2G0 milliseconds. +his will take ?0 seconds to complete. Within the loop the code is checking the 3h)tdown flag. It is really important for this f)nction to terminate 2)ickly if the program is sh)tting down. We don.t want the admin who is sh)tting the program down to think the program has h)ng. 'nce the f)nction terminates* the timer ro)tine can e5ec)te the select statement again. If the program is sh)tting down* the select will immediately wake )p again to process the signaled 3h)tdown channel. From there the timer ro)tine signals back to the main ro)tine that it is sh)tting down and the program terminates gracef)lly. +his is my timer and gracef)l sh)tdown code pattern that yo) can also )se in yo)r applications. If yo) download the f)ll e5ample from the 9oing9o repository* yo) can see the code in action and a few more goodies. 1ead this post to learn how to implement a work pool to process work across m)ltiple go ro)tines. -ike in the radar image processing I described above. httpB www.goinggo.net 20?" 0< poolCgoCro)tinesCtoCprocessCtask.html

>'nning Go Programs In IronWorker

Introd'ction
Iron.io has a prod)ct called IronWorker which provides a task oriented -in)5 container that yo) can r)n yo)r programs inside. If yo) are not s)re what I mean* think of this as having a temporary -in)5 virt)al machine instantly available for yo)r personal b)t short term )se. IronWorker allows yo) to load yo)r binaries* code files* s)pport files* shells scripts and 0)st abo)t anything else yo) may need to r)n yo)r program in the container. :o) specify a single task to e5ec)te* s)ch as r)nning a shell script or a binary and IronWorker will perform that task when re2)ested. 'nce the task is complete* IronWorker will tear down the container as if it never e5isted. If yo) are developing in Windows or on the Mac and plan to load and r)n preCb)ilt binaries in IronWorker* they m)st be b)ilt for -in)5. If that co)ld be a problem don.t despair* yo) have options. :o) can create an 7b)nt) =M so yo) can b)ild the -in)5 binaries yo) need. +his is what I do. I don.t develop in my 7b)nt) =M* I )se it as a testing and staging area. :o)r second ?G0

option is to b)ild yo)r program inside of the IronWorker container and then e5ec)te it. :o) have a lot of fle5ibility to do what yo) need with IronWorker.

To Install or ?ot to Install .b'nt'


I highly recommend that yo) b)y =MWare F)sion or $arallels and create an 7b)nt) =M. Most of the clo)d based environments yo) may choose to )se for yo)r testing and prod)ction environments will be r)nning -in)5. It helps to stage and test things before )ploading code to these environments. I )se =MWare F)sion and I love the prod)ct. =MWare F)sion will cost aro)nd >H0 73; and 7b)nt) is F166. I r)n 7b)nt) ?2.08 -+3 b)t version ?".08 is now available. httpB www.vmware.com prod)cts f)sion httpB www.)b)nt).com download desktop In this post I will be )sing my Mac for everything. 3o if yo) are r)nning Windows yo) sho)ld be able to follow along. 'nce yo) see how to b)ild yo)r applications inside the IronWorker container* yo) will know how to load and r)n -in)5 binaries. For those who are interested in setting )p or )sing 7b)nt)* read the ne5t section. If yo) want to contin)e in yo)r Windows or Mac environment* go to the Installing 1)by section.

Setting .p &o'r Terminal In .b'nt'


If yo) decided to go ahead and install an 7b)nt) =M or yo) already have one* awesome. Eere are a few things yo) will want to do so yo) can )se IronWorker.

+he first Icon in the -a)ncher bar is the ;ash Eome Icon. 3elect this and perform a search for the +erminal application. :o) will see several terminal applications show )p. 3elect the +erminal program* which will la)nch a +erminal 3ession. &ow back in -a)ncher yo) will see an icon for the +erminal session yo) started. 1ight Click on the +erminal Icon and select -ock +o -a)ncher. 'ne last step* we need to change a config)ration option for +erminalB

?G?

Make s)re the +erminal session is the active program and move yo)r mo)se to the top of the screen. +he men) for +erminal will appear. 7nder 6dit choose $rofile $references. +hen select the +itle and Command tab and check the 1)n command as a login shell. We need this for o)r 1)by programs to work properly in o)r +erminal sessions.

Installing >'by
/efore we can start )sing IronWorker we need the latest version of 1)by installed. IronWorker provides a 1)by based command line tool that allows )s to )pload IronWorker tasks for the different pro0ects we create. Windo" .sers In Windows )se this website to install the latest version of 1)byB httpB r)byinstaller.org 'nce yo) are done skip to the ne5t section called Installing IronWorker. 2ac and Lin'@ .sers 'n the Mac and -in)5 I have fo)nd the best way to install and manage 1)by is with the 1)by =ersion Manager @1=MAB httpsB rvm.io rvm install

?G2

$erform these steps on both yo)r Mac and -in)5 operating systems. 'pen a +erminal session and r)n this commandB *curl KL https(//get.rvm.io [ bash Ks stable KKruby KKautolibs7enable KKautoK)otfiles 7se the backslash when r)nning the command. +his prevents misbehaving if yo) have aliased it with config)ration in yo)r M .c)rlrc file. +hat command sho)ld download the latest version of 1)by and make it the defa)lt version. /efore moving on* close yo)r +erminal session and start a new one. Check that everything is good by r)nning this commandB run rvm list +his sho)ld be the o)tp)tB rvm rubies 7$ rubyKE.1.1KpEJR U B8SWSJ V O 7H K current O 7$ K current GG )efault O $ K )efault +here may be a newer version of 1)by by the time yo) are reading this. /)t as long as the version is greater than or e2)al to 2.0.0 and has the JZ operator in front* yo) will be all set. If yo) are not s)re if yo) have the correct defa)lt version set and want to make absol)tely s)re* r)n this commandB rvm use rubyKE.1.1KpEJR KK)efault O)st make s)re yo) specify the version correctly. O)st for yo)r information* the rvm tool and the different versions of 1)by yo) install get placed in a folder called .rvm inside yo)r >E'M6 folder. It will be a hidden folder. +o see that is does e5ist* r)n the following command in yo)r +erminal sessionB c) ]A3M@ ls K) .rvm +his sho)ld display the name of the directory back in the +erminal session.

Installing IronWorker
With 1)by properly installed we can install the IronWorker tool. +his tool is a 1)by program that allows )s to create tasks by )ploading o)r programs and s)pport files into IronWorker. +o install the tool we m)st install the IronWorker 1)by gem. 9em.s are 1)by packages that get installed )nder the defa)lt version of 1)by. +his is why I stressed we have the right defa)lt version set. ?G"

1)n the following command in the +erminal sessionB gem install ironWworkerWng If everything installs properly* the o)tp)t sho)ld end like thisB R gems installe) #s a final test to make s)re everything is set)p correctly* r)n the IronWorker programB ironWworker If everything is working properly yo) sho)ld get the following o)tp)tB usage( ironWworker 53MM.N& U3# :3N/V 53MM.N&( uploa)+ patch+ \ueue+ retry+ sche)ule+ log+ run+ install+ webhook+ info run ironWworker 53MM.N& KKhelp to get more information about each comman) &ow we have o)r environment set)p to talk with the IronWorker platform. Make s)re yo) do this in all of yo)r environments.

The Test Program


I have b)ilt a test application that we are going to r)n in IronWorker. +o download the code and the IronWorker s)pport files* r)n the following commandsB c) ]A3M@ eBport 23#. A7]A3M@/eBample go get github.com/goinggo/ironworker +his will copy* b)ild and install the code into the e5ample folder )nder >E'M6. +he program has been written to test a few things abo)t the IronWorker -in)5 container environment. -et.s review the code for the program first and test it locally. For IronWorker to be really effective yo) want to b)ild programs that perform a specific task. +he program sho)ld be designed to be started on demand or on a sched)le* r)n and then be terminated. :o) don.t want to b)ild programs that r)n for long periods of time. +he test program r)ns for H0 seconds before it terminates. +here were two things I wanted to know abo)t IronWorker that the program tests. First* I wanted to know how logging worked. 3econd* I wanted to know if the program wo)ld receive '3 signals when I re2)ested the r)nning program to be killed. With all that said let.s review the code we are going to r)n. Eere is the logging code which can be fo)nd in the helper packageB

?G8

package helper import ( "fmt" "runtime" "time" ) func Write/t)out(go-outine string+ functionName string+ message string) { fmt.#rintf("'s ( 's ( 's ( 's*n"+ time.Now().Mormat("E11SK10K1E 0F(1J(1F.111")+ go-outine+ functionName+ message) % func Write/t)outf(go-outine string+ functionName string+ format string+ a ...interface{%) { Write/t)out(go-outine+ functionName+ fmt./printf(format+ a...)) % +here is nothing fancy here* 0)st an abstraction layer so I change o)t the logging mechanism if this doesn.t work. Eere is the controller code that manages the starting and termination of the programB package controller import ( "github.com/goinggo/ironworker/helper" "github.com/goinggo/ironworker/program" "os" "os/signal" ) const ( N.M@/#.5@ 7 "controller" 23W-36 :N@ 7 "main" ) func -un() { helper.Write/t)out("Main"+ "controller.-un"+ "/tarte)") // 5reate a channel to talk with the 3/. sig5han (7 make(chan os./ignal+ 0) // 5reate a channel to let the program tell us it is )one. wait5han (7 make(chan bool) ?GG

// 5reate a channel to shut )own the program early. shut5han (7 make(chan bool) :: Ha+nc2 t2e .ork ro+tine. go program.!oGork(,2+t12an, .ait12an, "6e,t") // .sk the 3/ to notify us about interrupt events. signal.Notify(sig5han+ os.:nterrupt) for { select { case =Ksigchan( helper.Write/t)out("Main"+ "controller.-un"+ "$$$$$$H #rogram !eing ;ille)") // /ignal the program to shut)own an) wait for confirmation. shut5han =K true =Kshut5han helper.Write/t)out("Main"+ "controller.-un"+ "$$$$$$H /hutting &own") return case =Kwaitchan( helper.Write/t)out("Main"+ "controller.-un"+ "$$$$$$H /hutting &own") return % % % I like to sh)tdown my applications gracef)lly* so if I co)ld receive an '3 signal on a kill re2)est* that wo)ld be fantastic. I am not a channel g)r) and I am s)re there are better ways to accomplish this. I welcome any s)ggestions. For now this is what we got. +he f)nction creates three channels. +he first channel is )sed to receive signals from the '3. +he second channel is )sed by the 9o ro)tine that is performing the program logic to signal when it is done. +he third channel is )sed by the controller to signal the 9o ro)tine to terminate early if necessary. +he program.;oWork f)nction is started as a 9o ro)tine and then the controller waits for either the '3 to signal or the r)nning 9o ro)tine to signal it is done. If the '3 signals to terminate* then the controller )ses the sh)tdown channel and waits for the 9o ro)tine to respond. +hen everything sh)ts down gracef)lly. Eere is the code for the 9o ro)tine that is sim)lating the workB package program

?GH

import ( "github.com/goinggo/ironworker/helper" "time" ) func &oWork(shut5han chan bool+ wait5han chan bool+ log;ey string) { helper.Write/t)out("#rogram"+ "program.&oWork"+ "#rogram /tarte)") )efer func() { wait5han =K true %() // #erform work for S1 secon)s for count (7 1< count = EJ1< count>> { select { case =Kshut5han( helper.Write/t)out("#rogram"+ "program.&oWork"+ ":nfo ( 5omplete) ( ;:LL -@,6@/ @&") shut5han =K true return )efault( helper.Write/t)outf("#rogram"+ "program.&oWork"+ ":nfo ( #erforming Work ( ')"+ count) time./leep(time.Millisecon) $ EF1) % % helper.Write/t)out("#rogram"+ "program.&oWork"+ "5omplete)") % +he ;oWork f)nction prints a message to the log every 2G0 milliseconds 280 times. +his gives )s a min)te of work that m)st be performed. #fter each write log call* the f)nction checks if the sh)tdown channel has been signaled. If it has* the f)nction terminates immediately. O)st to have a complete code sample in the post* here is the main f)nctionB package main import ( "github.com/goinggo/ironworker/controller" "github.com/goinggo/ironworker/helper" ) func main() { helper.Write/t)out("Main"+ "main"+ "/tarte)")

?GI

controller.-un() helper.Write/t)out("Main"+ "main"+ "5omplete)") % In yo)r native environment* mine being the Mac* download the code and let 9o tool b)ild and install the application. c) ]A3M@ eBport 23#. A7]A3M@/eBample go get github.com/goinggo/ironworker c) ]A3M@/eBample/bin ./ironworker When yo) r)n the program* let it r)n to completion. :o) sho)ld see the following o)tp)tB E10IK1QK1R 00(JE(J8.R10 E10IK1QK1R 00(JE(J8.R10 E10IK1QK1R 00(JE(J8.R10 /tarte) E10IK1QK1R 00(JE(J8.R10 #erforming Work ( 1 E10IK1QK1R 00(JE(J8.QF0 #erforming Work ( 0 E10IK1QK1R 00(JE(JQ.E1I #erforming Work ( E E10IK1QK1R 00(JE(JQ.JFI #erforming Work ( I E10IK1QK1R 00(JE(JQ.R1J #erforming Work ( J E10IK1QK1R 00(JE(JQ.QFF #erforming Work ( F ... E10IK1QK1R 00(JI(J8.0S0 #erforming Work ( EIR E10IK1QK1R 00(JI(J8.J0E #erforming Work ( EI8 E10IK1QK1R 00(JI(J8.SSE #erforming Work ( EIQ E10IK1QK1R 00(JI(J8.Q0I E10IK1QK1R 00(JI(J8.Q0I /hutting &own E10IK1QK1R 00(JI(J8.Q0I ( Main ( main ( /tarte) ( Main ( controller.-un ( /tarte) ( #rogram ( program.&oWork ( #rogram ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( :nfo ( ( #rogram ( program.&oWork ( 5omplete) ( Main ( controller.-un ( $$$$$$H ( Main ( main ( 5omplete)

+he program started and terminated s)ccessf)lly. 3o the Controller logic is working. +his time let.s kill the program early by hitting QCtrlR C after it startsB E10IK1QK1R 00(JS(I0.8FJ ( Main ( main ( /tarte) E10IK1QK1R 00(JS(I0.8FJ ( Main ( controller.-un ( /tarte) E10IK1QK1R 00(JS(I0.8FJ ( #rogram ( program.&oWork ( #rogram /tarte)

?GL

E10IK1QK1R 00(JS(I0.8FJ ( #rogram ( program.&oWork ( :nfo ( #erforming Work ( 1 E10IK1QK1R 00(JS(IE.01F ( #rogram ( program.&oWork ( :nfo ( #erforming Work ( 0 E10IK1QK1R 00(JS(IE.IFS ( #rogram ( program.&oWork ( :nfo ( #erforming Work ( E E10IK1QK1R 00(JS(IE.S1R ( #rogram ( program.&oWork ( :nfo ( #erforming Work ( I ^5E10IK1QK1R 00(JS(IE.R1S ( Main ( controller.-un ( $$$$$$H 3/ Notification( interrupt ( 1BE E10IK1QK1R 00(JS(IE.R1S ( Main ( controller.-un ( $$$$$$H #rogram !eing ;ille) E10IK1QK1R 00(JS(IE.8FR ( #rogram ( program.&oWork ( :nfo ( 5omplete) ( ;:LL -@,6@/ @& E10IK1QK1R 00(JS(IE.8FR ( Main ( controller.-un ( $$$$$$H /hutting &own E10IK1QK1R 00(JS(IE.8FR ( Main ( main ( 5omplete) #s soon as I hit QCtrlR C the '3 signaled the program with the syscall.3I9I&+ message. +hat ca)sed the Controller to signal the r)nning program to sh)tdown and the program terminated gracef)lly.

$on6ig'ring IronWork
+his is the doc)mentation for )sing IronWorkerB httpB dev.iron.io worker I am going to walk yo) thro)gh the process for the test application we have. -ogin to yo)r Iron.io acco)nt and select $ro0ects from the top men)B

6nter +est into the te5t bo5 and hit the Create &ew $ro0ect b)tton. +hat sho)ld send yo) to the following screenB

?G<

3elect the Key icon which is where yo) will find the Credentials for this pro0ect. We need these credentials to create a special file called iron.0son. +his file will be re2)ired by the IronWorker 1)by program to load o)r tasks into this pro0ect. In o)r +erminal session let.s move to the folder where the IronWorker script files are located. I want to work with the b)ildandr)n scriptsB c) ]A3M@/eBample/src/github.com/goinggo/ironworker/scripts/buil)a n)run ls Kl :o) sho)ld see the following filesB KrwKrKKrKK 0 bill staff QQF /ep 8 E1(0E buil)an)run.sh KrwKrKKrKK 0 bill staff 01S /ep 8 E1(0E buil)an)run.worker KrwKrKKrKK 0 bill staff 80 /ep 8 E1(0E iron.Dson :o) will find a iron.0son file in the folder. 6dit the file and p)t in yo)r credentialsB { "proDectWi)" ( "4444444444444444444444"+ "token" ( "4444444444444444444444" % +he ne5t file we want to look at is the .worker file. Eere is the doc)mentation for .worker filesB httpB dev.iron.io worker reference dotworker ?H0

Eere is what the b'ildandr'n5"orker file looks likeB O )efine the runtime language runtime CbinaryC O eBec is the file that will be eBecute)( eBec Cbuil)an)run.shC +he b)ildandr)n.worker file is telling the IronWorker 1)by program to )pload and e5ec)te the b)ildandr)n.sh file. +his is the only file that will be placed into the IronWork container. Eere is what the b)ildandr)n.sh file looks likeB eBport A3M@WM3L&@-7"]A3M@/5ontainer" eBport 53&@WM3L&@-7"]A3M@WM3L&@-/co)e" eBport #-32-.MWM3L&@-7"]53&@WM3L&@-/src/github.com/goingo/ironworker" if U ? Ke ]53&@WM3L&@-/bin/ironworker V then mk)ir ]A3M@WM3L&@c) ]A3M@WM3L&@curl https(//go.googleco)e.com/files/go0.0.E.linuBK am)SJ.tar.gN Ko p.tar.bNE GG tar Bf p.tar.bNE GG rm p.tar.bNE eBport 23.-5A7"am)SJ" eBport 23!:N7"" eBport 235A.-7"S" eBport 23@4@7"" eBport 23A3/ .-5A7"am)SJ" eBport 23A3/ 3/7"linuB" eBport 233/7"linuB" eBport 23#. A7"]53&@WM3L&@-" eBport 23-.5@7"" eBport 23-33 7"]A3M@WM3L&@-/go" eBport 23 33L&:-7"]A3M@WM3L&@-/go/pkg/tool/linuBWam)SJ" eBport 557"gcc" eBport 23255ML.2/7"Kg K3E Kf#:5 KmSJ Kpthrea)" eBport 523W@N.!L@&7"0" eBport #. A7]23-33 /bin(]#. A go get github.com/goinggo/ironworker Ogit clone https(//username(passwor)Tgithub.com/goinggo/ironworker ]#-32-.MWM3L&@Oc) ]#-32-.MWM3L&@Ogo clean Ki Ogo buil) Ogo install fi

?H?

c) ]53&@WM3L&@-/bin ./ironworker +his script tests to see if the ironworker test application already e5ists. If it doesn.t* it then proceeds to download the latest -in)5 binary package for 9o and b)ilds the program )sing the 9o tool. 'nce the b)ild and install is complete* the script e5ec)tes the program. :o) will notice these lines have been commented o)t in the shell scriptB Ogit clone https(//+,ername:pa,,.ordTgithub.com/goinggo/ironworker ]#-32-.MWM3L&@Oc) ]#-32-.MWM3L&@Ogo clean Ki Ogo buil) Ogo install If yo) have a repository that re2)ires a)thentication yo) can )se this techni2)e. +his calls git clone the same way the 9o tool does* so everything is copied to the right directory str)ct)re. If yo)r code references other libraries* yo) will need to do that man)ally. If yo) r)n the 9o 9et command with the SC5T option* yo) can see all the calls the 9o tool iss)es. O)st copy what yo) need and add it to yo)r shell script. IronWorker does have a copy of the -in)5 binary package for 9o version ?.0.2 already preC config)red in every IronWorker container. +he script installs the latest version to show yo) how that can be accomplished if the version yo) need is not the one installed. +he techni2)e can also be )sed to install other packages that might be re2)ired. If yo) want to b)ild the code every time the task r)ns* yo) co)ld be smarter and r)n go version first. If the right version is not already available then yo) can download the version of 9o yo) needB eBport A3M@WM3L&@-7"]A3M@/5ontainer" eBport 53&@WM3L&@-7"]A3M@WM3L&@-/co)e" eBport #-32-.MWM3L&@-7"]53&@WM3L&@-/src/github.com/goingo/ironworker" go ver,ion F ver.t#t gover,ion=P(Ever.t#t) 2ead ver.t#t O :ronWorker will report go version go0.1.E if ; "Pgover,ion" '= "go ver,ion go1.1.2 (in+#:amd>?" < then mk)ir ]A3M@WM3L&@c) ]A3M@WM3L&@curl https(//go.googleco)e.com/files/go0.0.E.linuBK am)SJ.tar.gN Ko p.tar.bNE GG tar Bf p.tar.bNE GG rm p.tar.bNE eBport 23.-5A7"am)SJ" eBport 23!:N7"" ?H2

eBport eBport eBport eBport eBport eBport eBport eBport eBport eBport eBport eBport eBport fi

235A.-7"S" 23@4@7"" 23A3/ .-5A7"am)SJ" 23A3/ 3/7"linuB" 233/7"linuB" 23#. A7"]53&@WM3L&@-" 23-.5@7"" 23-33 7"]A3M@WM3L&@-/go" 23 33L&:-7"]A3M@WM3L&@-/go/pkg/tool/linuBWam)SJ" 557"gcc" 23255ML.2/7"Kg K3E Kf#:5 KmSJ Kpthrea)" 523W@N.!L@&7"0" #. A7]23-33 /bin(]#. A

go get KB github.com/goinggo/ironworker c) ]53&@WM3L&@-/bin ./ironworker

Loading IronWork With ) Task


We have everything we need to load and r)n o)r first task for the +est pro0ect. In yo)r +erminal session r)n the following commandB c) ]A3M@/eBample/src/github.com/goinggo/ironworker/scripts/buil)a n)run ironWworker uploa) buil)an)run If everything is s)ccessf)l yo) sho)ld see the following o)tp)tB KKKKKKH 5reating client #roDect C estC with i)7CFEEbJcF08a1cQS111Q11111RC KKKKKKH 5reating co)e package Moun) workerfile with path7Cbuil)an)run.workerC &etecte) eBec with path7Cbuil)an)run.shC an) args7C{%C 5o)e package name is Cbuil)an)runC KKKKKKH 6ploa)ing co)e package Cbuil)an)runC 5o)e package uploa)e) with i)7CFEE)0JRbQ0cFI1FI0fSfJeQEC an) revision7C0C 5heck Chttps(//hu).iron.io/t\/proDects/FEEbJcF08a1cQS111Q11111R/co)e /FEE)0JRbQ0cFI1FI0fSfJeQEC for more info 9o back to the Iron.io website and let.s see if o)r new task is there. 3elect $ro0ects again from the main men) and select the Worker b)tton to the right of yo)r +est pro0ect.

?H"

3elect the +asks tab and yo) sho)ld see the b)ildandr)n task we 0)st )ploaded. 3elect the task and yo) sho)ld see the following screenB

+here is a big grey b)tton that says 4)e)e a +ask. -et.s hit that b)tton to r)n o)r task. +his dialog bo5 will pop)p. 7se all the defa)lts and hit the 4)e)e +ask b)tton. +his will 2)e)e the task and then it sho)ld start r)nning. 'nce yo) hit the 2)e)e b)tton the screen sho)ld changeB

?H8

+he task will start in the 2)e)ed state* then r)nning and finally complete. 'nce the task is done* click on the -og link.

:o) will see the shell script did everything perfectly. It downloaded 9o and s)ccessf)lly b)ilt the program. +hen it started e5ec)ting it. -et.s r)n the program again and check two things. First* let.s see if the container is saved and the download of 9o does not have to occ)r again. 3econd* let.s kill the program early and see if it receives any '3 signals. Eit the 4)e)e a +ask b)tton again and after several seconds let.s kill itB

?HG

:o) can see that I killed the task 2? seconds into the r)n. When I look at the logs I am a bit disappointed. First* the task downloaded 9o again. +his means I have a new and clean IronWorker container every time the task r)ns. 3econd* the program did not receive any '3 signals when I iss)ed the kill. It appears the IronWork container is forced to die and the program gets no '3 notifications. It is not the end of the world* 0)st something that is good to know. /ased on this new information it seems we want to load binaries into the IronWorker container when we can. +his way we don.t need to spend time downloading things that can be preCcompiled. Eowever* I was able to )se IronWorker from my Mac environment which is a real pl)s. 'n the other hand* having 9o b)ild and install yo)r program every time co)ld be h)ge. If yo) )pload code changes to yo)r repository yo) don.t need to )pload a new revision of the task. +he 9o tool will p)ll down the latest code* b)ild it and r)n the program. +hen again* that co)ld ca)se problems. #t the end of the day yo) need to decide what is best for yo)r different scenarios. What.s awesome is that IronWorker gives yo) all the fle5ibility yo) need to make it work.

Working With B'ilder Tasks


/)ilder +asks are a hidden gem within IronWorker. # b)ilder task allows yo) to b)ild yo)r code and generate a task that yo) will )se for yo)r application inside of IronWorker. +his is really the best of both worlds beca)se yo) don.t need to download 9o and b)ild the code every time the task r)ns. :o) can do yo)r b)ild once. :o)r application task r)ns immediately beca)se it is always ready to go with the binaries and other s)pport files it needs. 9o back into the scripts folder and find the s)bCfolder called b)ildertask. -et.s 2)ickly r)n thro)gh the files and see how this works. +he task<b'ilder5sh file is the script that knows how to b)ild o)r code. eBport A3M@WM3L&@-7"]A3M@/5ontainer" eBport 53&@WM3L&@-7"]A3M@WM3L&@-/co)e" eBport #-32-.MWM3L&@-7"]53&@WM3L&@-/src/github.com/goingo/ironworker" ?HH

mk)ir ]A3M@WM3L&@c) ]A3M@WM3L&@curl https(//go.googleco)e.com/files/go0.0.E.linuBK am)SJ.tar.gN Ko p.tar.bNE GG tar Bf p.tar.bNE GG rm p.tar.bNE eBport 23.-5A7"am)SJ" eBport 23!:N7"" eBport 235A.-7"S" eBport 23@4@7"" eBport 23A3/ .-5A7"am)SJ" eBport 23A3/ 3/7"linuB" eBport 233/7"linuB" eBport 23#. A7"]53&@WM3L&@-" eBport 23-.5@7"" eBport 23-33 7"]A3M@WM3L&@-/go" eBport 23 33L&:-7"]A3M@WM3L&@-/go/pkg/tool/linuBWam)SJ" eBport 557"gcc" eBport 23255ML.2/7"Kg K3E Kf#:5 KmSJ Kpthrea)" eBport 523W@N.!L@&7"0" eBport #. A7]23-33 /bin(]#. A go get github.com/goinggo/ironworker cd P15!E095H!E4:$in cp iron.orker PS5ME:00$+i(d00:iron.orker +he code downloads 9o and then )ses the 9o tool to b)ild the program. #t the end of the script we copy the binary that the 9o tool b)ilt to the IronWorker staging area. #nything yo) copy to this folder will be placed into o)r new application task. +he task5sh file is the script that is e5ec)ted by the application task. ./ironworker In o)r case we only need to r)n the binary. 1emember the binary is being created by the taskC b)ilder.sh script file. +he task5"orker file performs all the magicB runtime CbinaryC eBec Ctask.shC $+i(d Csh ./taskKbuil)er.shC fi(e CtaskKbuil)er.shC +he worker file tells IronWorker that o)r application task is a binary and to load and r)n the task.sh script. &e5t we have the b'ild command. +his tells IronWorker to perform a remote b)ild by e5e5)ting the taskCb)ilder.sh script. +he file command p)lls the taskCb)ilder.sh file into the b)ilder task so it can e5ec)ted remotely. -et.s navigate to the b)ildertask folder and try all this o)tB

?HI

c) ]A3M@/eBample/src/github.com/goinggo/ironworker/scripts/buil)e rtask We need to edit the iron.0son file with the credentials again. 'nce yo) do that r)n the following commandB ironWworker uploa) task +his time the IronWorker will take a bit longer to r)n. It will be performing a remote b)ild and we m)st wait )ntil it is complete. 'nce everything is done yo) sho)ld see the followingB KKKKKKH 5reating client #roDect C estC with i)7CFEEbJcF08a1cQS111Q11111RC KKKKKKH 5reating co)e package Moun) workerfile with path7Ctask.workerC &etecte) eBec with path7Ctask.shC an) args7C{%C Merging file with path7CtaskKbuil)er.shC an) )est7CC 5o)e package name is CtaskC KKKKKKH 6ploa)ing an) buil)ing co)e package CtaskC 4emote $+i(ding .orker 5o)e package uploa)e) with i)7CFEE)1ca)IcbJSSFIcFe0FcbeC an) revision7C0C 5heck Chttps(//hu).iron.io/t\/proDects/FEEbJcF08a1cQS111Q11111R/co)e /FEE)1ca)IcbJSSFIcFe0FcbeC for more info &ow let.s switch to the Iron.io website and see what we have. 9o back to the +asks tab and yo) sho)ld see two new tasksB

:o) will notice the taskBBb)ilder task has already been e5ec)ted. +his is the remote b)ild that was being performed. -et.s look at the logs. :o) will see that the 9o binary package for -in)5 was downloaded and the pro0ect was also downloaded and b)ilt. -ook at the last two lines in the logB

+his is where we copied the binary to the staging folder FFb)ildFF. We didn.t have any errors or problems coping the final binary. ?HL

&ow we can try r)nning the application task. 3elect task from the +ask list and 2)e)e the 0ob. It sho)ld start right )p and r)n for a min)te. 'nce it is done let.s look at the logB

When yo) look at the log yo) can see the program starts r)nning. +here is no downloading of 9o or any other b)ild work. 7sing a b)ilder task is a great way to have IronWorker stage and b)ild the code for )s. If yo) change the code and need to perform another b)ild yo) m)st r)n the IronWorker program again. +his will create new revisions of both the b)ilder and application tasks. :o) can.t rer)n the /)ilder task man)ally.

IronWork and PaperTrail


-ooking at the logs in IronWork is a real convenience b)t yo) will want to )se a third party system to manage yo)r logging. I love $aper+rail and the IronWorker integration is seamless. 9o to $aper+rail and create a free acco)ntB httpsB papertrailapp.com #fter yo) get yo)r acco)nt and login* go to the ;ashboard.

Find the Create 9ro)p b)tton and create a new gro)pB

Click the 3ave b)tton at the bottom. &ow on the ;ashboard yo) sho)ld have yo)r new gro)pB

?H<

9o to the #cco)nt options.

Create a -og ;estination for yo)r IronWorker +est pro0ect. Click on the Create log destination b)ttonB

Click on the 6dit 3ettings b)ttonB

Make s)re yo) accept logs from )nrecogni(ed systems and select the gro)p yo) 0)st created. +hen hit the 7pdate b)tton. &ow copy the destination )rl and go back to yo)r IronWorker +est pro0ect. 3elect the settings iconB

?I0

+ake the $aper+rail )rl and enter it into the -ogger 71- te5t bo5 )sing )dp as the protocol. Click the 7pdate b)tton and then click on the Worker b)tton again and find the b)ildandr)n task. +he )dp m)st be lowercase or yo)r task will fail. 4)e)e the task one last time and let it r)n to completion. #s the task is r)nning* go back to the $aper+rail website. 9o to the ;ashboard and hit the refresh b)tton on the browserB

:o) sho)ld start seeing events coming into yo)r gro)p. Click on the #ll events drop down on the right and yo) will see the logs in $aper+rail.

$oncl'sion
I 0)st scratched the s)rface with how yo) can )se IronWorker to r)n yo)r applications. +his is a really fle5ible environment with really no restrictions. :o) have access to download and install any packages yo) need* access to the local disk and integration to $aper+rail and a few other logging systems. +ho)gh yo) don.t need a -in)5 =M to )se IronWorker* yo) may want to consider having one so yo) can stage and load yo)r binary programs directly. #gain* yo) have the fle5ibility to )se IronWorker as yo) see fit. I hope yo) try o)t the service. 7se my application or b)ild yo)r own. $ost a comment abo)t yo)r e5perience and anything new yo)r learn. I plan on )sing IronWorker for two pro0ects I am working on and e5pect only great things.

?I?

Slices o6 Slices o6 Slices in Go


I am working on b)ilding code to load polygons for the different Marine Forecast areas in the 7nited 3tates. +hese polygons need to be stored in Mongo;/ and there is a special way that needs to be done. It wo)ld not have been a big deal if it wasn.t for this fact. +here isn.t 0)st one polygon for each area. +here is an e5ternal polygon and then (ero to many interior polygons that need to be stored in relationship. #fter staring at the problem for a bit I reali(ed that I needed to create a slice of Marine Forecast areas* each of which contained a slice of polygons. +o store each polygon ring I needed a slice of geographic coordinates. Finally each coordinate needed to be stored in a two dimensional array of floats. # pict)re is worth a tho)sand wordsB

When the data is stored in Mongo;/ it needs to follow this data patternB

My head is spinning 0)st looking at the diagram and the pict)re. +he diagram depicts how all the slices and ob0ects need to be organi(ed.

?I2

+he pict)re shows how the polygons need to be stored in Mongo;/. +here will be m)ltiple elements )nder coordinates* each with its own set of points. I decided to b)ild a test application to fig)re o)t how to str)ct)re and store the data. +he more I )se slices the more I really love them. I love how I can pass them in and o)t of f)nctions and not concern myself with handling references or how memory is being handled. # slice is a lightware data str)ct)re that can safely be copied in and o)t of f)nctions. I catch myself thinking all the time that I need to pass a reference of the slice so a copy of the data str)ct)re is not made on the stack. +hen I remember* the data str)ct)re is 28 bytes* I am not copying all the data that is abstracted )nderneath it. 1ead these two articles to learn more abo)t slicesB httpB www.goinggo.net 20?" 0L )nderstandingCslicesCinCgoCprogramming.html httpB www.goinggo.net 20?" 0L collectionsCofC)nknownClengthCinCgo.html -et.s look at the data str)ct)re that will hold and store the data for Mongo;/B // #olygon )efines a set of points that complete a ring // aroun) a geographic area type #olygon UVUEVfloatSJ // #olygon-ings )efines a Mongo&! /tructure for storing multiple polygon rings type #olygon-ings struct { ype string Xbson("type"X 5oor)inates UV#olygon Xbson("coor)inates"X % // -epresents a marine station an) its polygons type Marine/tation struct { /tation:) string Xbson("stationWi)"X #olygons #olygon-ings Xbson("polygons"X % +he $olygon type represents a slice of 2 floating point n)mbers. +his will represent each point that makes )p the polygon. +he $olygon1ings str)ct)re takes on the Mongo;/ format re2)ired for storing polygons. If yo) want to )se Mongo;/ to perform geospatial searches against the polygons this is re2)ired. +he Marine3tation str)ct)re sim)lates an individ)al station and the set of polygons associated with the station. +he test code is going to create one station with two polygons. +hen it will display everything. -et.s look at how to create the slice of marine stations and create a single marine station for testingB

?I"

// 5reate an empty slice to store the polygon rings // for the )ifferent marine stations marine/tations (7 UVMarine/tation{% // 5reate a marine station for .MY0EI marine/tation (7 Marine/tation{ /tation:)( ".MY0EI"+ #olygons( #olygon-ings{ ype( "#olygon"+ 5oor)inates( UV#olygon{%+ %+ % +he first line of code creates an empty slice that can hold Marine3tation ob0ects. +hen we create a Marine3tation ob0ect )sing a composite literal. Within the composite literal we have another composite literal to create an ob0ect of type $olygon1ings for the $olygons property. +hen within the creation of the $olygon1ings ob0ect we create an empty slice that can hold $olygon ob0ects for the Coordinates property. +o learn more abo)t composite literals check o)t this doc)mentB httpB golang.org ref spec%CompositeFliterals &ow it is time to add a co)ple of polygons to the stationB // 5reate point0 (7 pointE (7 pointI (7 pointJ (7 pointF (7 the points for the first polygon ring UEVfloatSJ{KRQ.REQ00Q1REQQQQ+ ES.QREQIQ8S11110% UEVfloatSJ{K81.1RQQFIE10QQQQ+ ES.QSQES8QF11110% UEVfloatSJ{K81.181ISERQFQQQQ+ ES.QR1FIIIR0% UEVfloatSJ{K81.1801F18REQQQQ+ ES.QRF11J0QS% UEVfloatSJ{KRQ.REQ00Q1REQQQQ+ ES.QREQIQ8S11110%

// 5reate a polygon for this ring polygon (7 #olygon{point0+ pointE+ pointI+ pointJ+ pointF% // .)) the polygon to the slice of polygon coor)inates marine/tation.#olygons.5oor)inates 7 appen)(marine/tation.#olygons.5oor)inates+ polygon) First we create five points. &otice the first and last point are identical. +his completes the ring. +hen we store all the points into a $olygon ob0ect* )sing a composite literal. -ast* we append the $olygon ob0ect to the slice of $olygons for the marine station. +hen we do it all over again so we have two polygons associated with this marine stationB // 5reate the points for the secon) polygon ring point0 7 UEVfloatSJ{K81.JIR100R08QQQQ+ ER.R8RR0QRER1110% pointE 7 UEVfloatSJ{K81.JIRSEE1SQQQQQ+ ER.R88F0I08J1110% pointI 7 UEVfloatSJ{K81.JI8J0FFESQQQQ+ ER.R88F0I08J1110% pointJ 7 UEVfloatSJ{K81.JIR100R08QQQQ+ ER.R8RR0QRER1110%

?I8

// 5reate a polygon for this ring polygon 7 #olygon{point0+ pointE+ pointI+ pointJ% // .)) the polygon to the slice of polygon coor)inates marine/tation.#olygons.5oor)inates 7 appen)(marine/tation.#olygons.5oor)inates+ polygon) +his second polygon has fo)r points instead of five. +he last thing left to do is add the Marine3tation ob0ect to the slice of stations and display everythingB // .)) the marine station marine/tations 7 appen)(marine/tations+ marine/tation) &isplay(marine/tations) +he display f)nction )ses the keyword range to iterator over all the slicesB func &isplay(marine/tations UVMarine/tation) { for W+ marine/tation (7 range marine/tations { fmt.#rintf("*n/tation( 's*n"+ marine/tation./tation:)) for in)eB+ rings (7 range marine/tation.#olygons.5oor)inates { fmt.#rintf("-ing( ')*n"+ in)eB) for W+ coor)inate (7 range rings { fmt.#rintf("#oint( 'f+'f*n"+ coor)inateU1V+ coor)inateU0V) % % % % +he f)nction takes a slice of Marine3tation ob0ects. 1emember only the slice str)ct)re is being copied on the stack* not all the ob0ects the slice represents. When we iterate thro)gh the slice of Marine3tation ob0ects and all the internal slices that make )p the ob0ect* we get the following res)ltB /tation( .MY0EI -ing( 1 #oint( KRQ.REQ00Q+ES.QREQJ1 #oint( K81.1RQQFI+ES.QSQESQ #oint( K81.181ISI+ES.QR1FII #oint( K81.1801F0+ES.QRF11J #oint( KRQ.REQ00Q+ES.QREQJ1 -ing( 0 #oint( K81.JIR10E+ER.R8RRE1 #oint( K81.JIRSEE+ER.R88F0I #oint( K81.JI8J0S+ER.R88F0I #oint( K81.JIR10E+ER.R8RRE1 ?IG

7sing slices to solve this problem is fast* easy and effective. I have placed a working copy of the test code in the 9o $laygro)ndB httpB play.golang.org p 7:'2EIKggy /)ilding this 2)ick test application has shown me again how )sing slices has very real advantages. +hey will make yo) more prod)ctive and yo)r code perform well. &ot having to worry abo)t memory management and handling references to pass data in and o)t of f)nctions is h)ge. +ake the time to learn how to )se slices in yo)r code* yo) will thank yo)rself later.

Pool Go >o'tines To Process Task (riented Work


'n more than one occasion I have been asked why I )se the Work $ool pattern. Why not 0)st start as many 9o ro)tines as needed at any given time to get the work doneD My answer is always the same. ;epending on the type of work* the comp)ting reso)rces yo) have available and the constraints that e5ist within the platform* blindly throwing 9o ro)tines to perform work co)ld make things slower and h)rt overall system performance and responsiveness. 6very application* system and platform has a breaking point. 1eso)rces are not )nlimited* whether that is memory* C$7* storage* bandwidth* etc. +he ability for o)r applications to red)ce and re)se reso)rces is important. Work pools provide a pattern that can help applications manage reso)rces and provide performance t)ning options. Eere is the pattern behind the work poolB

In the diagram above* the Main 1o)tine posts ?00 tasks into the Work $ool. +he Work $ool 2)e)es each individ)al task and once a 9o ro)tine is available* the task is de2)e)ed* assigned

?IH

and performed. When the task is finished* the 9o ro)tine becomes available again to process more tasks. +he n)mber of 9o ro)tines and the capacity of the 2)e)e can be config)red* which allows for performance t)ning the application. With 9o don.t think in terms of threads b)t in 9o ro)tines. +he 9o r)ntime manages an internal thread pool and sched)les the 9o ro)tines to r)n within that pool. +hread pooling is key to minimi(ing load on the 9o r)ntime and ma5imi(ing performance. When we spawn a 9o ro)tine* the 9o r)ntime will manage and sched)le that 9o ro)tine to r)n on its internal thread pool. &o different than the operating system sched)ling a thread to r)n on an available C$7. We can gain the same benefits o)t of a 9o ro)tine pool as we can with a thread pool. $ossibly even more. I have a simple philosophy when it comes to task oriented work* -ess is More. I always want to know what is the least n)mber of 9o ro)tines* for a partic)lar task* I need that yields the best res)lt. +he best res)lt m)st take into acco)nt not only how fast all the tasks are geting done* b)t also the total impact processing those tasks have on the application* system and platform. :o) also have to look at the impact short term and long term. We might be able to yield very fast processing times in the beginning* when the overall load on the application or system is light. +hen one day the load changes slightly and the config)ration doesn.t work anymore. We may not reali(e that we are crippling a system we are interacting with. We co)ld be p)shing a database or a web server too hard and event)ally* always at the wrong time* the system sh)ts down. # b)rst r)n of ?00 tasks might work great* b)t s)stained over an ho)r might be deadly. # Work $ool is not some magic pi5ie d)st that will solve the worlds comp)ting problems. It is a tool yo) can )se for yo)r task oriented work inside yo)r applications. It provides options and some control on how yo)r application performs. #s things change* yo) have fle5ibility to change with it. -et.s prove the simple case that a Work $ool will process o)r task oriented work faster than 0)st blindly spawning 9o ro)tines. +he test application I b)ilt r)ns a task that grabs a Mongo;/ connection* performs a Find on that Mongo;/ and retrieves the data. +his is something the average b)siness application wo)ld do. +he application will post this task ?00 times into a Work $ool and do this G times to get an average r)ntime. +o download the code* open a +erminal session and r)n the following commandsB eBport 23#. A7]A3M@/eBample go get github.com/goinggo/workpooltest c) ]A3M@/eBample/bin -et.s start with a work pool of ?00 9o ro)tines. +his will sim)late the model of spawning as many ro)tines and as we have tasks. ./workpooltest 011 off +he first arg)ment tells the program to )se ?00 9o ro)tines in the pool and the second parameter t)rns off the detailed logging.

?II

Eere is the res)lt of )sing ?00 9o ro)tines to process ?00 tasks on my MacbookB 5#6U8V -outinesU011V .mount3fWorkU011V MaB-outinesU011V MaB,ueue)UIV 5#6U8V -outinesU011V .mount3fWorkU011V MaB-outinesU011V MaB,ueue)UIV 5#6U8V -outinesU011V .mount3fWorkU011V MaB-outinesU011V MaB,ueue)UIV 5#6U8V -outinesU011V .mount3fWorkU011V MaB-outinesU011V MaB,ueue)UIV 5#6U8V -outinesU011V .mount3fWorkU011V MaB-outinesU011V MaB,ueue)UIV .verageUJ.Q8FQRIV +he o)tp)t tells )s a few things abo)t the r)nB 5#6U8V ( -outinesU011V ( .mount3fWorkU011V ( &urationUJ.FQQRFEV ( took MaB-outinesU011V ( active )uring the run MaB,ueue)UIV ( )uring the run he he he he number number number amount of of of of cores on my machine routines in the work pool tasks to run time in secon)s the run &urationUJ.FQQRFEV &urationUF.RQQ8RJV &urationUF.IEFEEEV &urationUJ.SFERQIV &urationUJ.FFEEEIV

he maB number of routines that were he maB number of tasks waiting in \ueue)

&e5t let.s r)n the program )sing H8 9o ro)tinesB 5#6U8V -outinesUSJV .mount3fWorkU011V MaB-outinesUSJV MaB,ueue)UIFV 5#6U8V -outinesUSJV .mount3fWorkU011V MaB-outinesUSJV MaB,ueue)UIFV 5#6U8V -outinesUSJV .mount3fWorkU011V MaB-outinesUSJV MaB,ueue)UIFV 5#6U8V -outinesUSJV .mount3fWorkU011V MaB-outinesUSJV MaB,ueue)UIFV 5#6U8V -outinesUSJV .mount3fWorkU011V MaB-outinesUSJV MaB,ueue)UIFV .verageUJ.FFSIIFV &ow )sing 28 9o ro)tinesB 5#6U8V -outinesUEJV .mount3fWorkU011V MaB-outinesUEJV MaB,ueue)URFV 5#6U8V -outinesUEJV .mount3fWorkU011V MaB-outinesUEJV MaB,ueue)URFV 5#6U8V -outinesUEJV .mount3fWorkU011V MaB-outinesUEJV MaB,ueue)URFV 5#6U8V -outinesUEJV .mount3fWorkU011V MaB-outinesUEJV MaB,ueue)URFV &urationUJ.FQF8IEV &urationUJ.JI1111V &urationUJ.JRRFJJV &urationUJ.FF1RS8V &urationUJ.FRJISRV &urationUJ.FJQIIQV &urationUJ.J8I001V &urationUJ.FQF08IV &urationUJ.FRQSRSV

?IL

5#6U8V -outinesUEJV .mount3fWorkU011V &urationUJ.SEQQ8QV MaB-outinesUEJV MaB,ueue)URFV .verageUJ.FIS8ERV &ow )sing L 9o ro)tinesB 5#6U8V -outinesU8V .mount3fWorkU011V MaB-outinesU8V MaB,ueue)UQ0V 5#6U8V -outinesU8V .mount3fWorkU011V MaB-outinesU8V MaB,ueue)UQ0V 5#6U8V -outinesU8V .mount3fWorkU011V MaB-outinesU8V MaB,ueue)UQ0V 5#6U8V -outinesU8V .mount3fWorkU011V MaB-outinesU8V MaB,ueue)UQ0V 5#6U8V -outinesU8V .mount3fWorkU011V MaB-outinesU8V MaB,ueue)UQ0V .verageUJ.RRFJSJV -et.s collect the res)lts of the different r)nsB 011 SJ 2? 8 2o 2o No 2o -outines -outines 4o+tine, -outines ( ( : ( J.Q8FQRI J.FFSIIF ?.L%>D27 J.RRFJSJ ( ( _JI1 Millisecon)s Master : U?L" Mi((i,econd, 9a,ter ( _E01 Millisecon)s Master &urationUJ.S0S8JIV &urationUJ.JRRRQSV &urationUJ.8J0JRSV &urationUJ.Q1S1SFV &urationUF.1IF0IQV

+his program seems to r)n the best when we )se " 9o ro)tines per core. +his seems to be a magic n)mber beca)se it always yields pretty good res)lts for the programs I write. If we r)n the program on a machine with more cores* we can increase the 9o ro)tine n)mber and take advantage of the e5tra reso)rces and processing power. +hat.s to say if the Mongo;/ can handle the e5tra load for this partic)lar task. 6ither way we can always ad0)st the si(e and capacity of the Work $ool. We have proved that for this partic)lar task* spawning a 9o ro)tine for each task is not the best performing sol)tion. -et.s look at the code for the Work $ool and see how it worksB +he Work $ool can be fo)nd )nder the following folder if yo) downloaded the codeB c) ]A3M@/eBample/src/github.com/goinggo/workpool #ll the code can be fo)nd in a single 9o so)rce code file called workpool.go. I have removed all the comments and some lines of code to let )s foc)s on the important pieces. &ot all the f)nctions are listed in this post as well. -et.s start with the types that make )p the Work $oolB type Work#ool struct { W/hut)own,ueue5hannel W/hut)ownWork5hannel W/hut)ownWait2roup 0V+e+e12anne( 0Gork12anne( chan string chan struct{% sync.Wait2roup c2an 0Poo(Gork c2an Poo(Gorker ?I<

W,ueue)Work W.ctive-outines W,ueue5apacity %

intIE intIE intIE

type W#oolWork struct { Work Poo(Gorker -esult5hannel chan error % type #oolWorker interface { !oGork(.ork4o+tine int) % +he Work$ool str)ct)re is a p)blic type that represents the Work $ool. +he implementation )ses two channels to r)n the pool. +he WorkChannel is at the heart of the Work $ool. It manages the 2)e)e of work that needs to be processed. #ll of the 9o ro)tines that will be performing the work will wait for a signal on this channel. +he 4)e)eChannel is )sed to manage the posting of work into the WorkChannel 2)e)e. +he 4)e)eChannel provides acknowledgments to the calling ro)tine that the work has or has not been 2)e)ed. It also helps to maintains the 4)e)edWork and 4)e)eCapacity co)nters. +he $oolWork str)ct)re defines the data that is sent into the 4)e)eChannel to process en2)e)ing re2)ests. It contains an interface reference to the )sers $oolWorker ob0ect and a channel to receive a confirmation that the task has been en2)e)ed. +he $oolWorker interface defines a single f)nction called ;oWork that has a parameter that represents an internal id for the 9o ro)tine that is r)nning the task. +his is very helpf)l for logging and other things that yo) may want to implement at a per 9o 1o)tine level. +he $oolWorker interface is the key for accepting and r)nning tasks in the Work $ool. -ook at this sample client implementationB type My ask struct { Name string W# $workpool.Work#ool % func (this $My ask) &oWork(work-outine int) { fmt.#rintf("'s*n"+ this.Name) fmt.#rintf("$$$$$$$H W-( ') ,W( ') .-( ')*n"+ work-outine+ this.W#.,ueue)Work()+ this.W#..ctive-outines()) time./leep(011 $ time.Millisecon)) % ?L0

func main() { runtime.23M.4#-35/(runtime.Num5#6()) work#ool (7 .orkpoo(.-e.(r+ntime.-+m1PC() 8 %, 1"") task (7 GMy ask{ Name( "." > strconv.:toa(i)+ W#( work#ool+ % err (7 .orkPoo(.Po,tGork("main"+ ta,k) ... % I create a type called My+ask that defines the state I need for the work to be performed. +hen I implement a member f)nction for My+ask called ;oWork* which matches the signat)re of the $oolWorker interface. 3ince My+ask implements the $oolWorker interface* ob0ects of type My+ask are now considered ob0ects of type $oolWorker. &ow we can pass an ob0ect of type My+ask into the $ostWork call. +o learn more abo)t interfaces and ob0ect oriented programming in 9o read this blog postB httpB www.goinggo.net 20?" 0I ob0ectCorientedCprogrammingCinCgo.html In main I tell the 9o r)ntime to )se all of the available C$7s and cores on my machine. +hen I create a Work $ool with 28 9o ro)tines. 'n my c)rrent machine I have L cores and as we learned above* three 9o ro)tines per core is a good starting place. +he last parameter tells the Work $ool to create a 2)e)e capacity for ?00 tasks. +hen I create a My+ask ob0ect and post it into the 2)e)e. For logging p)rposes* the first parameter of the $ostWork f)nction is a name yo) can give to the ro)tine making the call. If the err variable is nil after the call* the task has been posted. If not* then most likely yo) have reached 2)e)e capacity and the task co)ld not be posted. -et.s look at the internals of how a Work$ool ob0ect is created and startedB func New(number3f-outines int+ \ueue5apacity intIE) (work#ool $Work#ool) { work#ool 7 GWork#ool{ W/hut)own,ueue5hannel( W/hut)ownWork5hannel( 0V+e+e12anne(: 0Gork12anne(: W+e+e1apacit=), W,ueue)Work( W.ctive-outines( W,ueue5apacity( % make(chan make(chan make(c2an make(c2an string)+ struct{%)+ 0Poo(Gork), Poo(Gorker,

1+ 1+ \ueue5apacity+

?L?

for work-outine (7 1< work-outine = number3f-outines< work-outine>> { .orkPoo(.0/2+tdo.nGaitNro+p.Add(1) go .orkPoo(.0Gork4o+tine(.ork4o+tine) % go work#ool.W,ueue-outine() return work#ool % +he &ew f)nction accepts the n)mber of ro)tines and the 2)e)e capacity as we saw in the above sample client code. +he WorkChannel is a b)ffered channel which is )sed as the 2)e)e for storing the work we need to process. +he 4)e)eChannel is an )nb)ffered channel )sed to synchroni(e access to the WorkChannel b)ffer* g)arantee 2)e)ing and to maintain the co)nters. +o learn more abo)t b)ffered and )nb)ffered channels read this web pageB httpB golang.org doc effectiveFgo.html%channels 'nce the channels are initiali(ed we are able to spawn the 9o ro)tines that will perform the work. First we add ? to the wait gro)p for each 9o ro)tine so we can sh)t it down the pool cleanly it is time. +hen we spawn the 9o ro)tines so we can process work. +he last thing we do is start )p the 4)e)e1o)tine so we can begin to accept work. +o learn how the sh)tdown code and Wait9ro)p works read this web pageB httpB dave.cheney.net 20?" 08 "0 c)rio)sCchannels 3h)tting down the Work $ool is done like thisB func (this $Work#ool) /hut)own(go-outine string) (err error) { t2i,.0/2+tdo.nV+e+e12anne( EB "!o.n" EBt2i,.0/+tdo.nV+e+e12anne( close(this.W,ueue5hannel) close(this.W/hut)own,ueue5hannel) c(o,e(t2i,.0/2+tdo.nGork12anne() t2i,.0/2+tdo.nGaitNro+p.Gait() close(this.WWork5hannel) return err % +he 3h)tdown f)nction brings down the 4)e)e1o)tine first so no more re2)ests can be accepted. +hen the 3h)tdownWorkChannel is closed and the code waits for each 9o ro)tine ?L2

to decrement the Wait9ro)p co)nter. 'nce the last 9o ro)tine calls ;one on the Wait9ro)p* the call to Wait will ret)rn and the Work $ool is sh)tdown. &ow let.s look at the $ostWork and 4)e)e1o)tine f)nctionsB func (this $Work#ool) #ostWork(go-outine string+ work #oolWorker) (err error) { poolWork (7 W#oolWork{work+ make(chan error)% )efer close(poolWork.-esult5hannel) t2i,.0V+e+e12anne( EB poo(Gork err = EBpoo(Gork.4e,+(t12anne( return err % func (this $Work#ool) W,ueue-outine() { for { select { case =Kthis.W/hut)own,ueue5hannel( this.W/hut)own,ueue5hannel =K "&own" return case W+e+e3tem := EBt2i,.0V+e+ec2anne(( if atomic..)):ntIE(Gthis.W,ueue)Work+ 1) 77 this.W,ueue5apacity { W+e+e3tem.4e,+(t12anne( EB fmt.Errorf("62read Poo( At 1apacit=") continue % atomic..)):ntIE(Gthis.W,ueue)Work+ 0) t2i,.0Gork12anne( EB W+e+e3tem.Gork W+e+e3tem.4e,+(t12anne( EB ni( break % % % +he idea behind the $ostWork and 4)e)e1o)tine f)nctions are to seriali(e access to the WorkChannel b)ffer* g)arantee 2)e)ing and to maintain the co)nters. Work is always placed at the end of the WorkChannel b)ffer by the 9o r)ntime when it is sent into the channel. +he highlighted code shows all the comm)nication points. When the 4)e)eChannel is signaled* the 4)e)e1o)tine receives the work. 4)e)e capacity is checked and if there is room* the )ser $oolWorker ob0ect is 2)e)ed into the WorkChannel b)ffer. Finally the calling ro)tine is signaled back that everything is 2)e)ed. ?L"

-ast let.s look at the Work1o)tine f)nctionsB func (this $Work#ool) WWork-outine(work-outine int) { for { select { case =Kthis.Wshut)ownworkchannel( t2i,.0/2+tdo.nGaitNro+p.!one() return ca,e poo(Gorker := EBt2i,.0Gork12anne(: this.W/afely&oWork(work-outine+ poolWorker) break % % % func (this $Work#ool) W/afely&oWork(work-outine int+ poolWorker #oolWorker) { defer 01atc2Panic(ni(, "0Gork4o+tine", ".orkpoo(.GorkPoo(", "/afe(=!oGork") )efer func() { atomic..)):ntIE(Gthis.W.ctive-outines+ K0) %() atomic..)):ntIE(Gthis.W,ueue)Work+ K0) atomic..)):ntIE(Gthis.W.ctive-outines+ 0) poo(Gorker.!oGork(.ork4o+tine) % +he 9o r)ntime takes care of assigning work to a 9o ro)tine in the pool by signaling the WorkChannel for a partic)lar 9o ro)tine that is waiting. When the channel is signaled* the 9o r)ntime passes the work that is at the head of the channel b)ffer. +he channel b)ffer acts as a 2)e)e* FIF'. If all the 9o ro)tines are b)sy* then none will be waiting on the WorkChannel* so all remaining work has to wait. #s soon as a ro)tine completes it work* it ret)rns to wait again on the WorkChannel. If there is work in the channel b)ffer* the 9o r)ntime will signal the 9o ro)tine to wake )p again. +he code )ses the 3afely;o pattern for processing work. #t this point the code is calling into )ser code and co)ld panic. :o) don.t want anything to ca)se the 9o ro)tine to terminate. &otice the )se of the first defer statement. It catches any panics and stops them in their tracks. +he rest of the code safely increments and decrements the co)nters and calls into the )ser ro)tine via the Interface. +o learn more abo)t catch panics read this blog postB

?L8

httpB www.goinggo.net 20?" 0H )nderstandingCdeferCpanicCandCrecover.html +hat.s the heart of the code and how it implements the pattern. +he Work$ool really shows the elegance and grace of channels. With very little code I was able to implement a pool of 9o ro)tines to process work. #dding g)aranteed 2)e)ing and maintain the co)nters was a bree(e. ;ownload the code from the 9oing9o repository on 9ith)b and try it for yo)rself.

Iterating (:er Slices In Go


3lices are )sed everywhere in my code. If I am working with data from Mongo;/* it is stored in a slice. If I need to keep track of a collection of problems after r)nning an operation* it is stored in a slice. If yo) don.t )nderstand how slices work yet or have been avoiding them like I did when I started* read these two posts to learn more. httpB www.goinggo.net 20?" 0L )nderstandingCslicesCinCgoCprogramming.html httpB www.goinggo.net 20?" 0L collectionsCofC)nknownClengthCinCgo.html # 2)estion that I am constantly asking myself when coding is* [;o I want to )se a pointer to this ob0ect or do I want to make a copyD\ +ho)gh 9o can be )sed as a f)nctional programming lang)age* it is an imperative programming lang)age at heart. What.s the differenceD # f)nctional programming lang)age does not allow yo) to change the state of a variable or an ob0ect once it has been created and initiali(ed. +his means variables and ob0ects are imm)table* they can.t be changed. If yo) want to change the state of a variable or an ob0ect* yo) m)st make a copy and initiali(e the copy with the changes. F)nctions are always passed copies and ret)rn val)es are always copies too. In an imperative programming lang)age we can create variables and ob0ects that are m)table* or can be changed. We can pass a pointer for any variable or ob0ect to a f)nction* which in t)rn can change the state as necessary. # f)nctional programming lang)age wants yo) to think in terms of mathematical f)nctions that take inp)t and prod)ce a res)lt. In an imperative programming lang)age we can b)ild similar f)nctions* b)t we can also b)ild f)nctions that perform operations on state that can e5ist anywhere in memory. /eing able to )se a pointer has advantages b)t can also get yo) in tro)ble. 7sing pointers can alleviate memory constraints and possibly improve performance. It can also create synchroni(ation iss)es s)ch as shared access to ob0ects and reso)rces. Find the sol)tion that works best for each individ)al )se case. For yo)r 9o programs I recommend )sing pointers when it is safe and practical. 9o is an imperative programming lang)age so take advantage of that. In 9o everything is pass by val)e and it is really important to remember that. We can pass by val)e the address of an ob0ect or pass by val)e a copy of an ob0ect. When we )se a pointer in 9o it can sometime be conf)sing beca)se 9o handles all the dereferencing for )s. ;on]t get me wrong* its great that 9o does this* b)t sometime yo) can forget what the val)e of yo)r variable act)ally is.

?LG

#t some point in every program* I have the need to iterate over a slice to perform some work. In 9o we )se the keyword range within a for loop constr)ct to iterate over a slice. In the beginning I made some very bad mistakes iterating over slices beca)se I mis)nderstood how the range keyword worked. I will show yo) a nasty b)g I created iterating over a slice that p)((led me for a bit. &ow it is obvio)s to me why the code misbehaved* b)t at the time I was shaking my head. -et.s create some simple ob0ects and place them inside of a slice. +hen we will iterate over the slice and see what happens. package main import ( "fmt" ) type &og struct { Name string .ge int % func main() { Dackie (7 &og{ Name( "9ackie"+ .ge( 0Q+ % fmt.#rintf("9ackie .))r( 'p*n"+ GDackie) sammy (7 &og{ Name( "/ammy"+ .ge( 01+ % fmt.#rintf("/ammy .))r( 'p*n"+ Gsammy) )ogs (7 UV&og{Dackie+ sammy% fmt.#rintln("") for W+ )og (7 range )ogs { fmt.#rintf("Name( 's .ge( ')*n"+ )og.Name+ )og..ge) fmt.#rintf(".))r( 'p*n"+ G)og) fmt.#rintln("") % % +he program creates two dog ob0ects and p)ts them into a slice of dogs. We display the address of each dog ob0ect. +hen we iterate over the slice displaying the name* age and address of each ;og. Eere is the o)tp)t for the programB ?LH

9ackie .))r( 1BE010bc111 /ammy .))r( 1BE010bc1J1 Name( 9ackie .ge( 0Q .))r( "#21"1$c">" Name( /ammy .ge( 01 .))r( "#21"1$c">" 3o why is the address of the dog ob0ect different inside the range loop and why does the same address appear twiceD +his all has to do with the fact that everything is pass by val)e in 9o. In this code e5ample we act)ally create 2 e5tra copies of each ;og ob0ect in memory.

+he initial e5istence of each ;og ob0ect is created with a composite literalB Dackie (7 &og{ Name( "9ackie"+ .ge( 0Q+ % +he first copies of the ob0ects are created when the ob0ects are placed into the sliceB )ogs (7 UV&og{Dackie+ sammy% +he second copies of the ob0ects are created when we iterate over the sliceB )og (7 range )ogs &ow we can see why the address of the dog variable inside range loop is always the same. We are displaying the address of the dog variable* which happens to be a local variable of type ;og and contains a copy of the ;og ob0ect for each inde5 of the slice. With each iteration of the slice* the location of the dog variable is the same. +he val)e of the dog variable is changing. +hat nasty b)g I was talking abo)t earlier had to do with me thinking the address of the dog variable co)ld be )sed as a pointer to each individ)al ;og ob0ect inside the slice. 3omething like thisB

?LI

all&ogs (7 UV$&og{% for W+ )og (7 range )ogs { all&ogs 7 appen)(all&ogs+ G)og) % for W+ )og (7 range all&ogs { fmt.#rintf("Name( 's .ge( ')*n"+ )og.Name+ )og..ge) % I create a new slice that can hold pointers to ;og ob0ects. +hen I range over the slice of dogs storing the address of each ;og ob0ect into the new slice. 'r at least I think I am storing the address of each ;og ob0ect. If I add this code to the program and r)n it* this is the o)tp)tB Name( /ammy .ge( 01 Name( /ammy .ge( 01 I end )p with a slice where every element has the same address. +his address is pointing to a copy of the last ob0ect that we iterated over. :ikes,, If making all these copies is not what yo) want* yo) co)ld )se pointers. Eere is the e5ample program )sing pointersB package main import ( "fmt" ) type &og struct { Name string .ge int % func main() { Dackie (7 &og{ Name( "9ackie"+ .ge( 0Q+ % fmt.#rintf("9ackie .))r( 'p*n"+ Dackie) sammy (7 &og{ Name( "/ammy"+ .ge( 01+ % fmt.#rintf("/ammy .))r( 'p*n*n"+ sammy)

?LL

)ogs (7 UV8&og{Dackie+ sammy% for W+ )og (7 range )ogs { fmt.#rintf("Name( 's .ge( ')*n"+ )og.Name+ )og..ge) fmt.#rintf(".))r( 'p*n*n"+ )og) % % Eere is the o)tp)tB 9ackie .))r( 1BE010bb111 /ammy .))r( 1BE010bb1J1 Name( 9ackie .ge( 0Q .))r( 1BE010bb111 Name( /ammy .ge( 01 .))r( 1BE010bb1J1 +his time we create a slice of pointers to ;og ob0ects. When we iterate over this slice* the val)e of the dog variable is the address of each ;og ob0ect we stored in the slice. Instead of creating two e5tra copies of each ;og ob0ect* we are )sing the same initial ;og ob0ect we created with the composite literal. When the slice is a collection of ;og ob0ects or a collection of pointers to ;og ob0ects* the range loop is the same. for W+ dog (7 range )ogs { fmt.#rintf("Name( 's .ge( ')*n"+ dog.Name+ dog..ge) % 9o handles access to the ;og ob0ect regardless of whether we are )sing a pointer or not. +his is awesome b)t can sometimes lead to a bit of conf)sion. #t least it was for me in the beginning. I can.t tell yo) when yo) sho)ld )se a pointer or when yo) sho)ld )se a copy. O)st remember that 9o is going to pass everything by val)e. +hat incl)des f)nction parameters* ret)rn val)es and when iterating over a slice* map or channel. :es* yo) can also range over a channel. +ake a look at this sample code I altered from a blog post written by 6wen CheslackC$ostavaB httpB ewencp.org blog golangCiterators package main import ( "fmt" ) type &og struct { ?L<

Name string .ge int % type &og5ollection struct { &ata UV$&og % func (this $&og5ollection) :nit() { cloey (7 G&og{"5loey"+ 0% ralph (7 G&og{"-alph"+ F% Dackie (7 G&og{"9ackie"+ 01% bella (7 G&og{"!ella"+ E% Damie (7 G&og{"9amie"+ S% this.&ata 7 UV$&og{cloey+ ralph+ Dackie+ bella+ Damie% % func (this $&og5ollection) 5ollection5hannel() chan $&og { )ata5hannel (7 make(chan $&og+ len(this.&ata)) for W+ )og (7 range this.&ata { )ata5hannel =K )og % close()ata5hannel) return )ata5hannel % func main() { )c (7 &og5ollection{% )c.:nit() for )og (7 range )c.5ollection5hannel() { fmt.#rintf("5hannel Name( 's*n"+ )og.Name) % % If yo) r)n the program yo) will get the following o)tp)tB 5hannel 5hannel 5hannel 5hannel 5hannel Name( Name( Name( Name( Name( 5loey -alph 9ackie !ella 9amie

I really love this sample code beca)se it shows the bea)ty of a closed channel. +he key to making this program work is the fact that a closed channel is always in the signaled state. +hat means any read on the channel will ret)rn immediately. If the channel is empty a defa)lt val)e is ret)rned. +his is what allows the range to iterate over all the data that was passed into ?<0

the channel and complete when the channel is empty. 'nce the channel is empty* the ne5t read on the channel will ret)rn nil. +his ca)ses the loop to terminate. 3lices are great* lightweight and powerf)l. :o) sho)ld be )sing them and gaining the benefits they provide. O)st remember that when yo) are iterating over a slice* yo) are getting a copy of each element of the slice. If that happens to be an ob0ect* yo) are getting a copy of that ob0ect. ;on]t ever )se the address of the local variable in the range loop. +hat is a local variable that contains a copy of the slice element and only has local conte5t. ;on]t make the same mistake I made.

>ec'rsion )nd Tail $alls In Go


This article was written for and published by Gopher Academy

I was looking at a code sample that showed a rec)rsive f)nction in 9o and the writer was very 2)ick to state how 9o does not optimi(e for rec)rsion* even if tail calls are e5plicit. I had no idea what a tail call was and I really wanted to )nderstand what he meant by 9o was not optimi(ed for rec)rsion. I didn.t know rec)rsion co)ld be optimi(ed. For those who don.t know what rec)rsion is* p)t simply* it is when a f)nction calls itself. Why wo)ld we ever write a f)nction that wo)ld call itselfD 1ec)rsion is great for algorithms that perform operations on data that can benefit from )sing a stack* FI-' @First In -ast ')tA. It can be faster than )sing loops and can make yo)r code m)ch simpler. $erforming math operations where the res)lt of a calc)lation is )sed in the ne5t calc)lation is a classic e5ample where rec)rsion shines. #s with all rec)rsion* yo) m)st have an anchor that event)ally ca)ses the f)nction to stop calling itself and ret)rn. If not* yo) have an endless loop that event)ally will ca)se a panic beca)se yo) will r)n o)t of memory. Why wo)ld yo) r)n o)t of memoryD In a traditional C program* stack memory is )sed to handle all the coming and going of f)nction calls. +he stack is preCallocated memory and very fast to )se. -ook at the following diagramB

?<?

+his diagram depicts an e5ample of a typical program stack and what it may look like for any program we write. #s yo) can see the stack in growing with each f)nction call we make. 6very time we call a f)nction from another f)nction* variables* registers and data is p)shed to the stack and it grows. In a C program each thread is allocated with its own fi5ed amo)nt of stack space. +he defa)lt stack si(e can range from ? Meg to L Meg depending on the architect)re. :o) have the ability to change the defa)lt as well. If yo) are writing a program that spawns a very large n)mber of threads* yo) can very 2)ickly start eating )p a ton of memory that yo) probably will never )se. In a 9o program each 9o ro)tine is allocated its own stack space. Eowever* 9o is smarted abo)t allocating space for the ro)tines stack. +he stack for a 9o ro)tine starts o)t at 8k and grows as needed. +he ability of 9o to be able to grow the stack dynamically comes from the concept of split stacks. +o learn more abo)t split stacks and how they work with the gcc compiler read thisB httpB gcc.gn).org wiki 3plit3tacks ?<2

:o) can always look at the code implemented for the 9o r)ntime as wellB httpB golang.org src pkg r)ntime stack.h httpB golang.org src pkg r)ntime stack.c When we )se rec)rsion we need to be aware that the stack is going to grow )ntil we finally hit o)r anchor and begin to shrink the stack back down. When we say that 9o does not optimi(e for rec)rsion* we are talking abo)t the fact that 9o does not attempt to look at o)r rec)rsive f)nctions and find ways to minimi(e stack growth. +his is where tail calls come in. /efore we talk more abo)t tail calls and how they can help optimi(e rec)rsive f)nctions* let.s begin with a simple rec)rsive f)nctionB func 4ec+r,ive(number int) int { if number 77 0 { return number % return number > 4ec+r,ive(numberK0) % func main() { answer (7 4ec+r,ive(J) fmt.#rintf("-ecursive( ')*n"+ answer) % +his 1ec)rsive f)nction takes an integer as a parameter and ret)rns an integer. If the val)e of the n)mber variable is one* then the f)nction ret)rns the val)e o)t. +his if statement contains the anchor and starts the process of )nwinding the stack to complete the work.

?<"

When the val)e of the n)mber variable is not the n)mber one* a rec)rsive call is made. +he f)nction decrements the n)mber variable by one and )ses that val)e as the parameter for the ne5t f)nction call. With each f)nction call the stack grows. 'nce the anchor is hit* each rec)rsive call begins to ret)rn )ntil we get back to main. -et.s look at a view of all the f)nction calls and ret)rn val)es for the programB 3tarting from the left side and from bottom to top we can see the call chain for the program. Main calls 1ec)rsive with a val)e of 8. +hen 1ec)rsive calls itself with a val)e of ". +his contin)es to happen )ntil the val)e of ? is passed into the 1ec)rsive f)nction call. +he f)nction calls itself " times before it reaches the anchor. /y the time the anchor is

?<8

reached* there are " e5tended stack frames* one for each call. +hen the rec)rsion begins to )nwind and the real work begins. 'n the right side and from top to bottom we can see the )nwind operations. 6ach ret)rn operation is now e5ec)ted by taking the parameter and adding it to the ret)rn val)e from the f)nction call. 6vent)ally the last ret)rn is e5ec)ted and we have the final answer which is ?0. +he f)nction performs this operation very 2)ickly and it is one of the benefits of rec)rsion. We don.t need any iterators or inde5 co)nters for looping. +he stack stores the res)lt of each operation and ret)rns it to the previo)s call. #gain* the only drawback is we need to be caref)l of how m)ch memory we are cons)ming. What is a tail call and how can it help optimi(e rec)rsive f)nctionsD Constr)cting a rec)rsive f)nction with a tail call tries to gain the benefits of rec)rsion witho)t the drawbacks of cons)ming large amo)nts of stack memory. Eere is the same rec)rsive f)nction implemented with a tail callB func 6ai(4ec+r,ive(number int+ pro)uct int) int { pro)uct 7 pro)uct > number if number 77 0 { return pro)uct % return 6ai(4ec+r,ive(numberK0+ pro)uct) % func main() { answer (7 6ai(4ec+r,ive(J+ 1) fmt.#rintf("-ecursive( ')*n"+ answer) % Can yo) see the difference in the implementationD It has to do with how we are )sing the stack and calc)lating the res)lt. In this implementation the anchor prod)ces the final res)lt. We don.t re2)ire any ret)rn val)es from the stack e5cept the final ret)rn val)e by the anchor which contains the answer. 3ome compilers are able to see this n)ance and change the )nderlying assembly that is prod)ced to )se one stack frame for all the rec)rsive calls. +he 9o compiler is not able to detect this n)ance yet. +o prove that let.s look at the assembly code that is prod)ced by the 9o compiler for both these f)nctions. +o prod)ce a file with the assembly code* r)n this command from a +erminal sessionB go tool Sg K/ ./main.go H assembly.asm ?<G

+here are three compilers depending on yo)r machine architect)re. HgB #M;H8 #rchitect)reB +his is for modern H8 bit processors regardless if the processor is b)ilt by Intel or #M;. #M; developed the H8 bit e5tension to the 5LH architect)re. LgB 5LH #rchitect)reB +his is for "2 bit processors based on the L0LH architect)re. GgB #1M #rchitect)reB +his is for 1I3C based processors which stands for 1ed)ced Instr)ction 3et Comp)ting. +o learn more abo)t this and other go tool commands look at this pageB httpB golang.org cmd gc I listed the 9o code and the assembly code together. O)st one item of note to help yo). In order for the processor to be able to perform an operation on data* s)ch as adding or comparing two n)mbers* the data m)st e5ist in one of the processor registers. +hink of registers as processor variables. When yo) look at the assembly below it helps to know that #N and /N are general p)rpose registers and )sed all the time. +he 3$ register is the stack pointer and the F$ register is the frame pointer* which also has to do with the stack. &ow let.s look at the codeB 1R func 4ec+r,ive(number int) int { 18 1Q if number 77 0 { 01 00 return number 0E % 0I 0J return number > 4ec+r,ive(numberK0) 0F % KKK prog list "-ecursive" KKK 1111 (./main.go(R) @4 -ecursive>1(/!)+]0SK0S 1110 (./main.go(R) M3P, number>1(M#)+.4 111E (./main.go(R) L35.L/ +]1 111I (./main.go(R) "#@ number>1(M#){int%+]8 111J (./main.go(R) "#@ _anon0>8(M#){int%+]8 111F (./main.go(Q) 5M#, .4+]0 111S (./main.go(Q) 9N@ +Q 111R (./main.go(00) M3P, .4+_anon0>8(M#) """D (.:main.go:11) 4E6 , ?<H

111Q (./main.go(0J) M3P, .4+!4 1101 (./main.go(0J) &@5, +!4 1100 (./main.go(0J) M3P, !4+(/#) ""12 (.:main.go:1?) 1AHH ,4ec+r,iveX"(/I) 110I (./main.go(0J) M3P, 8(/#)+.4 110J (./main.go(0J) M3P, number>1(M#)+!4 110F (./main.go(0J) .&&, .4+!4 110S (./main.go(0J) M3P, !4+_anon0>8(M#) ""17 (.:main.go:1?) 4E6 , If we follow along with the assembly code yo) can see all the places the stack is to)chedB 000?B +he #N register is given the val)e from the stack that was passed in for the n)mber variable. 000GC000HB +he val)e of the n)mber variable is compared with the n)mber ?. If they are not e2)al* then the code 0)mps to line ?8 in the 9o code. 000IC000LB +he anchor is hit and the val)e of the n)mber variable is copied onto the stack and the f)nction ret)rns. 000<C00?0B +he n)mber variable is s)btracted by one. 00??C00?2B +he val)e of the n)mber variable is p)shed onto to the stack and the rec)rsive f)nction call is performed. 00?"C00?GB +he f)nction ret)rns. +he ret)rn val)e is popped from the stack and placed in the #N register. +hen the val)e for the n)mber variable is copied from the stack frame and placed in the /N register. Finally they are added together. 00?HC00?IB +he res)lt of the add is copied onto the stack and the f)nction ret)rns. What the assembly code shows is that we have the rec)rsive call being made and that val)es are being p)shed and popped from the stack as e5pected. +he stack is growing and then being )nwo)nd. &ow let.s generate the assembly code for the rec)rsive f)nction that contains the tail call and see if the 9o compiler optimi(es anything. 0R func 6ai(4ec+r,ive(number int+ pro)uct int) int { 08 0Q pro)uct 7 pro)uct > number E1 E0 if number 77 0 { EE EI return pro)uct EJ % ?<I

EF ES ER %

return 6ai(4ec+r,ive(numberK0+ pro)uct)

KKK prog list " ail-ecursive" KKK 1108 (./main.go(0R) @4 ail-ecursive>1(/!)+]EJKEJ 110Q (./main.go(0R) M3P, number>1(M#)+54 11E1 11E0 11EE 11EI (./main.go(0R) L35.L/ +]1 (./main.go(0R) "#@ number>1(M#){int%+]8 (./main.go(0R) "#@ pro)uct>8(M#){int%+]8 (./main.go(0R) "#@ _anonE>0S(M#){int%+]8

11EJ (./main.go(0Q) M3P, pro)uct>8(M#)+.4 11EF (./main.go(0Q) .&&, 54+.4 11ES (./main.go(E0) 5M#, 54+]0 11ER (./main.go(E0) 9N@ +I1 11E8 (./main.go(EI) M3P, .4+_anonE>0S(M#) ""2T (.:main.go:2%) 4E6 , 11I1 (./main.go(ES) M3P, 54+!4 11I0 (./main.go(ES) &@5, +!4 11IE (./main.go(ES) M3P, !4+(/#) 11II (./main.go(ES) M3P, .4+8(/#) ""%? (.:main.go:2>) 1AHH ,6ai(4ec+r,iveX"(/I) 11IF (./main.go(ES) M3P, 0S(/#)+!4 11IS (./main.go(ES) M3P, !4+_anonE>0S(M#) ""%7 (.:main.go:2>) 4E6 , +here is a bit more assembly code with the +ail1ec)rsive f)nction. Eowever the res)lt is very m)ch the same. In fact* from a performance perspective we have made things a bit worse. &othing has been optimi(ed for the tail call we implemented. We still have all the same stack manip)lation and rec)rsive calls being made. 3o I g)ess it is tr)e that 9o c)rrently does not optimi(e for rec)rsion. +his does not mean we sho)ldn.t )se rec)rsion* 0)st be aware of all the things we learned. If yo) have a problem that co)ld best be solved by rec)rsion b)t are afraid of blowing o)t memory* yo) can always )se a channel. Mind yo) this will be significantly slower b)t it will work. Eere is how yo) co)ld implement the 1ec)rsive f)nction )sing channelsB

?<L

func -ecursive5hannel(number int+ pro)uct int+ result chan int) { pro)uct 7 pro)uct > number if number 77 0 { result =K pro)uct return % go -ecursive5hannel(numberK0+ pro)uct+ result) % func main() { result (7 make(chan int) -ecursive5hannel(J+ 1+ result) answer (7 =Kresult fmt.#rintf("-ecursive( ')*n"+ answer) % It follows along with the tail call implementation. 'nce the anchor is hit it contains the final answer and the answer is placed into the channel. Instead of making a rec)rsive call* we spawn a 9o ro)tine providing the same state we were p)shing onto the stack in the tail call e5ample. +he one difference is we pass an )nb)ffered channel to the 9o ro)tine. 'nly the anchor writes data to the channel and ret)rns witho)t spawning another 9o ro)tine. In main an )nb)ffered channel is created and the 1ec)rsiveChannel f)nction is called with the initial parameters and the channel. +he f)nction ret)rns immediately b)t main does not terminate. +his is beca)se it waits for data to be written to the channel. 'nce the anchor is hit and writes the answer to the channel* main wakes )p with the res)lt and it is printed to the screen. In most cases main will wake before the 9o ro)tine terminates. 1ec)rsion is another tool yo) can )se when writing yo)r 9o programs. For now the 9o compiler will not optimi(e the code for tail calls b)t there is nothing stopping f)t)re version of 9o from doing so. If memory co)ld be a problem yo) can always yo) a channel to minic rec)rsion.

Detecting >ace $onditions With Go


I always find it interesting when I reali(e that something I have been practicing or dealing with for a long time has a name. +his time it happens to be race conditions. +his is something yo) can.t avoid thinking abo)t as soon as yo) have more than one ro)tine sharing any kind of reso)rce. If yo).re not thinking abo)t race conditions in yo)r code* now is the time.

?<<

# race condition is when two or more ro)tines have access to the same reso)rce* s)ch as a variable or data str)ct)re and attempt to read and write to that reso)rce witho)t any regard to the other ro)tines. +his type of code can create the cra(iest and most random b)gs yo) have ever seen. It )s)ally takes a tremendo)s amo)nt of logging and l)ck to find these types of b)gs. 'ver the years I have really perfected my logging skills to help identify these problems when they occ)r. /ack in O)ne with 9o version ?.?* the 9o tooling introd)ced a race detector. +he race detector is code that is b)ilt into yo)r program d)ring the b)ild process. +hen once yo)r program is r)nning* it is able to detect and report any race conditions it finds. It is serio)sly cool and does an incredible 0ob in identifying the code that is the c)lprit. -et.s take a very simple program that contains a race condition and b)ild the code with the race detector. package main import ( "fmt" "sync" ) var Wait sync.Wait2roup var 5ounter int 7 1 func main() { for routine (7 0< routine =7 E< routine>> { Wait..))(0) go -outine(routine) % Wait.Wait() fmt.#rintf("Minal 5ounter( ')*n"+ 5ounter) % func -outine(i) int) { for count (7 1< count = E< count>> { value (7 5ounter value>> 5ounter 7 value % Wait.&one() % +he program looks innocent eno)gh. +he program spawns two ro)tines that each increment the global Co)nter variable twice. When both ro)tines are done r)nning* the program 200

displays the val)e of the global Co)nter variable. When I r)n the program it displays the n)mber 8 which is the correct answer. 3o everything m)st be working correctly* rightD -et.s r)n the code thro)gh the 9o race detector and see what it finds. 'pen a +erminal session where the so)rce code is located and b)ild the code )sing the Crace option. go buil) Krace +hen r)n the programB 777777777777777777 W.-N:N2( &. . -.5@ -ea) by goroutine F( main.-outine() /6sers/bill//paces/ est/src/test/main.go(EQ >1BJJ gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf #revious write by goroutine J( main.-outine() /6sers/bill//paces/ est/src/test/main.go(II >1BSF gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf 2oroutine F (running) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go(0R >1BSS runtime.main() /usr/local/go/src/pkg/runtime/proc.c(08E >1BQ0 2oroutine J (finishe)) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go(0R >1BSS runtime.main() /usr/local/go/src/pkg/runtime/proc.c(08E >1BQ0 777777777777777777 9ina( 1o+nter: ? Moun) 0 )ata race(s) -ooks like the tool detected a race condition with the code. If yo) look below the race condition report* yo) can see the o)tp)t for the program. +he val)e of the global Co)nter variable is 8. +his is the problem with these types of b)gs* the code co)ld work most of the time and then randomly something bad happens. +he race detector is telling )s something bad is l)rking in the trees. +he res)lt of the warning tells )s e5actly where the problem isB -ea) by goroutine F( main.-outine() /6sers/bill//paces/ est/src/test/main.go:2T >1BJJ 20?

gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf va(+e := 1o+nter #revious write by goroutine J( main.-outine() /6sers/bill//paces/ est/src/test/main.go:%% >1BSF gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf 1o+nter = va(+e 2oroutine F (running) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go:17 >1BSS runtime.main() /usr/local/go/src/pkg/runtime/proc.c(08E >1BQ0 go 4o+tine(ro+tine) :o) can see that the race detector has p)lled o)t the two lines of code that is reading and writing to the global Co)nter variable. It also identified the point in the code where the ro)tine was spawned. -et.s make a 2)ick change to the program to ca)se the race condition to raise its )gly headB package main import ( "fmt" "sync" "time" ) var Wait sync.Wait2roup var 5ounter int 7 1 func main() { for routine (7 0< routine =7 E< routine>> { Wait..))(0) go -outine(routine) % Wait.Wait() fmt.#rintf("Minal 5ounter( ')*n"+ 5ounter) % func -outine(i) int) { 202

for count (7 1< count = E< count>> { value (7 5ounter time./(eep(1 8 time.-ano,econd) value>> 5ounter 7 value % Wait.&one() % I have added a billionth of a second pa)se into the loop. I p)t the pa)se right after the ro)tine reads the global Co)nter variable and stores a local copy. -et.s r)n the program and see what the val)e of the global Co)nter variable is with this simple changeB Minal 5ounter( E +his pa)se in the loop has ca)sed the program to fail. +he val)e of the Co)nter variable is now 2 and no longer 8. 3o what happenedD -et.s break down the code and )nderstand why the billionth of a second pa)se revealed the b)g. Witho)t the pa)se the program r)ns as followsB

20"

Witho)t the pa)se the first ro)tine that is spawned r)ns to completion and then the second ro)tine begins to r)n. +his is why the program appears to be r)nning properly. +he code is seriali(ing itself beca)se of how fast it is able to r)n on my machine. -et.s look at how the program r)ns with the pa)seB

208

I didn.t complete the diagram for space b)t it shows eno)gh. +he pa)se is ca)sing a conte5t switch between the two ro)tines that are r)nning. +his time we have a m)ch different story. -et.s look at the code that is being r)n in the diagramB value (7 5ounter time./(eep(1 8 time.-ano,econd) value>> 5ounter 7 value With each iteration of the loop the val)e of the global Co)nter variable is capt)red locally* then the local copy is incremented and finally written back to the global Co)nter variable. If these three lines of code do not r)n immediately* witho)t interr)ption* we begin to have 20G

problems. +he diagram shows how the read of the global Co)nter variable and then the conte5t switch is ca)sing all of the initial problems. In the diagram* before the incremented val)e by 1o)tine ? is written back to the global Co)nter variable* 1o)tine 2 wakes )p and reads the global Co)nter variable. 6ssentially both ro)tines perform the same e5act reads and writes to the global Co)nter variable so we end )p with a final val)e of 2. +o fi5 this problem yo) might think we 0)st need to red)ce the incrementing of the global Co)nter variable from three lines of code to one line of codeB package main import ( "fmt" "sync" "time" ) var Wait sync.Wait2roup var 5ounter int 7 1 func main() { for routine (7 0< routine =7 E< routine>> { Wait..))(0) go -outine(routine) % Wait.Wait() fmt.#rintf("Minal 5ounter( ')*n"+ 5ounter) % func -outine(i) int) { for count (7 1< count = E< count>> { 1o+nter = 1o+nter X 1 time./leep(0 $ time.Nanosecon)) % Wait.&one() % When we r)n this version of the program we get the right answer againB Minal 5ounter( J If we r)n this code thro)gh the race detector o)r problems sho)ld go awayB

20H

go buil) Krace #nd the o)tp)tB 777777777777777777 W.-N:N2( &. . -.5@ Write by goroutine F( main.-outine() /6sers/bill//paces/ est/src/test/main.go(I1 >1BJJ gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf #revious write by goroutine J( main.-outine() /6sers/bill//paces/ est/src/test/main.go(I1 >1BJJ gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf 2oroutine F (running) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go(08 >1BSS runtime.main() /usr/local/go/src/pkg/runtime/proc.c(08E >1BQ0 2oroutine J (running) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go(08 >1BSS runtime.main() /usr/local/go/src/pkg/runtime/proc.c(08E >1BQ0 777777777777777777 Minal 5ounter( J Moun) 0 )ata race(s) We still have a race condition with line "0 of the programB Write by goroutine F( main.-outine() /6sers/bill//paces/ est/src/test/main.go(I1 >1BJJ gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf 1o+nter = 1o+nter X 1 #revious write by goroutine J( main.-outine() /6sers/bill//paces/ est/src/test/main.go(I1 >1BJJ gosche)1() /usr/local/go/src/pkg/runtime/proc.c(0E08 >1BQf 1o+nter = 1o+nter X 1 20I

2oroutine F (running) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go(08 >1BSS runtime.main() /usr/local/go/src/pkg/runtime/proc.c(08E >1BQ0 go 4o+tine(ro+tine) +he program r)ns correctly )sing one line of code to perform the increment. 3o why do we still have a race conditionD ;on.t be deceived by the one line of 9o code we have for incrementing the co)nter. -et.s look at the assembly code generated for that one line of codeB 11SJ (./main.go(I1) M3P, 5ounter>1(/!)+!4 < 5opy the value of 5ounter to !4 11SF (./main.go(I1) :N5, +!4 < :ncrement the value of !4 11SS (./main.go(I1) M3P, !4+5ounter>1(/!) < Move the new value to 5ounter +here are act)ally three lines of assembly code being e5ec)ted to increment the co)nter. +hese three lines of assembly code eerily look like the original 9o code. +here co)ld be a conte5t switch after any of these three lines of assembly code. 6ven tho)gh the program is working now* technically the b)g still e5ists. 6ven tho)gh the e5ample I am )sing is simple* it shows yo) how comple5 finding these b)gs can be. #ny line of assembly code prod)ced by the 9o compiler can be pa)sed for a conte5t switch. ')r 9o code may look like it is safely accessing reso)rces when act)ally the )nderlying assembly code is not safe at all. +o fi5 this program we need to g)arantee that reading and writing to the global Co)nter variable always happens to completion before any other ro)tine can access the variable. Channels are a great way to seriali(e access to reso)rces. In this case I will )se a M)te5 @M)t)al 65cl)sion -ockA. package main import ( "fmt" "sync" "time" ) var Wait sync.Wait2roup var 5ounter int 7 1 var Hock ,=nc.M+te# func main() { for routine (7 0< routine =7 E< routine>> {

20L

Wait..))(0) go -outine(routine) % Wait.Wait() fmt.#rintf("Minal 5ounter( ')*n"+ 5ounter) % func -outine(i) int) { for count (7 1< count = E< count>> { Hock.Hock() value (7 5ounter time./leep(0 $ time.Nanosecon)) value>> 5ounter 7 value Hock.Cn(ock() % Wait.&one() % -et.s b)ild the program with the race detector and see the res)ltB go buil) Krace ./test Minal 5ounter( J +his time we get the right answer and no race condition is identified. +he program is clean. +he M)te5 protects all the code between the -ock and 7nlock* making s)re only one ro)tine can e5ec)te that code at a time. +o learn more abo)t the 9o race detector and to see more e5amples read this postB httpB blog.golang.org raceCdetector It.s not a bad idea to test yo)r programs with the race detector on if yo) are )sing m)ltiple ro)tines. It will save yo) a lot of time and headaches early on in yo)r )nit and 2)ality ass)rance testing. We are l)cky as 9o developers to have s)ch a tool so check it o)t.

$ross $ompile &o'r Go Programs

Introd'ction
In my post abo)t b)ilding and r)nning programs in Iron.Io* I needed to switched over to my 7b)nt) =M to b)ild lin)5 versions of my test programs locally. I love the ability to have 7b)nt) available to me for b)ilding and testing my code. Eowever* if I can stay on the Mac 20<

side it is better. I have wanted to learn how to cross compile my 9o programs for the two platforms I )se* darwin amdH8 and lin)5 amdH8. +his way I co)ld create final b)ilds of my programs and p)blish everything from my Mac '3. #fter a co)ple of ho)rs I am finally able to do this. If yo) don.t have the need to cross compile yo)r code then I recommend yo) stick with the traditional distrib)tion packages and installs for 9o. If this is something yo) need* then it all starts with downloading the c)rrent release of the 9o so)rce code.

Installing 2erc'rial
+he 9o so)rce code is stored in a ;=C3 called Merc)rial and is located on code.google.com. +he first thing yo) need to do is install Merc)rial if yo) don.t already have it. 9o to the download page on the Merc)rial websiteB httpB merc)rial.selenic.com downloads

3ince I am r)nning on a Mac with '3N ?0.L* I downloaded that version. +his gives yo) a real nice installation and when it is done yo) will have the Merc)rial tool hg installed and ready to go.

$loning Go So'rce $ode


httpB golang.org doc install so)rce 'pen )p a +erminal session and go to yo)r >E'M6 directory and clone the c)rrent release of the 9o so)rce codeB c) ]A3M@ hg clone Ku release https(//co)e.google.com/p/go If everything works correctly yo) sho)ld see the following o)tp)t or something similarB warning( co)e.google.com certificate with fingerprint FJ(aR(IJ(IQ(0b(Ea(ec(b8(QE(S8()c(Ia(Ie(fe(Eb()I(Q0(e)(EI(0f not verifie) (check hostfingerprints or web.cacerts config setting) )estination )irectory( go re\uesting all changes a))ing changesets

2?0

a))ing manifests a))ing file changes a))e) 08EFE changesets with SIJQI changes to 8IEF files (>S hea)s) up)ating to branch releaseKbranch.go0.0 IRFF files up)ate)+ 1 files merge)+ 1 files remove)+ 1 files unresolve) 'nce that is done yo) now have a folder called go inside of the >E'M6 directory with the so)rce code )nder src. -ater on when yo) need to )pdate this repository* yo) can go to the go folder and r)n the following commandB hg up)ate /efore yo) b)ild the 9o code be aware that if yo)r c)rrent )ser is not the owner of all the directories and files of the go code* the b)ild will fail with permission iss)es. If this happens yo) can )se the chown program to fi5 the ownership. 5omman)( su)o chown K- userWname(groupWname file/fol)er to change @Bample( su)o chown K- bill(staff bufio.go

B'ilding Go Cor Each Target


&ow yo) need to b)ild the 9o code for the targets yo) need. For now 0)st b)ild darwin amdH8 and lin)5 amdH8. First b)ild the 9o code for the darwin amdH8 target which is for the host machineB c) go/src 233/7)arwin 23.-5A7am)SJ 523W@N.!L@&70 ./make.bash KKnoKclean If everything b)ilds correctly yo) sho)ld see the b)ild end like thisB KKK :nstalle) 2o for )arwin/am)SJ in /6sers/bill/go :nstalle) comman)s in /6sers/bill/go/bin If yo) get the following error on yo)r Mac there is a fi5B O crypto/BF1Q rootW)arwin.go(Q(JI( error( 5oreMoun)ation/5oreMoun)ation.h( No such file or )irectory rootW)arwin.go(01(I0( error( /ecurity//ecurity.h( No such file or )irectory mime/multipart net/mail +his means yo) don.t have the Command -ine +ools for Ncode installed on yo)r machine. 'pen Ncode and go to $references CR ;ownloadsB

2??

Click the Install b)tton for the Command -ine +ools. 'nce that is done try b)ilding the 9o code again. 'nce the b)ild is s)ccessf)l* open the 9o folder. :o) sho)ld see everything yo) need for b)ilding 9o programs on the Mac H8 bit environmentB

:o) have the go* godoc and gofmt tools and all the package related libraries and tools. &e5t yo) need to fi5 yo)r $#+E to point to the bin folder so yo) can start )sing the tools. eBport #. A7]A3M@/go/bin(]#. A :o) may want to set that in .bashrc or .bashFprofile file from inside the >E'M6 directory. With the path set* check that 9o is working. Check the version and the environmentB go ver,ion go version go0.E.0 )arwin/am)SJ go env 23.-5A7"am)SJ" 23!:N7"" 235A.-7"S" 23@4@7"" 23A3/ .-5A7"am)SJ" 23A3/ 3/7")arwin" 233/7"linuB" 23#. A7"" 2?2

23-.5@7"" 23-33 7"/6sers/bill/go" 23 33L&:-7"/6sers/bill/go/pkg/tool/)arwinWam)SJ" 557"gcc" 23255ML.2/7"Kg K3E Kf#:5 KmSJ" 523W@N.!L@&7"0" 6verything looks good. &ow b)ild a version of 9o that will let yo) b)ild lin)5 amdH8 binariesB 233/7(in+# 23.-5A7am)SJ 523W@N.!L@&7" ./make.bash KKnoKclean :o) can.t )se C9' when cross compiling. Make s)re C9'F6&#/-6; is set to 0 If everything b)ilds correctly yo) sho)ld see the b)ild end like thisB KKK :nstalle) 2o for (in+#/am)SJ in /6sers/bill/go :nstalle) comman)s in /6sers/bill/go/bin If yo) look at the 9o folder again yo) sho)ld see some new folders for lin)5 amdH8B

&ow it is time to test if yo) can b)ild 9o programs for both the Mac and -in)5 operating systems. 3et )p a 2)ick 9'$#+E and 9o program. In +erminal r)n the following commandsB c) ]A3M@ mk)ir eBample mk)ir src mk)ir simple eBport 23#. A7]A3M@/eBample c) eBample/src/simple Create a file called main.go inside of the simple folder with this codeB package main import ( 2?"

"fmt" ) func main() { fmt.#rintf("Aello 2ophers*n") % First b)ild the Mac version and make s)re it is a Mac e5ec)table )sing the file commandB go buil) file simple simple( MachK3 SJKbit eBecutable B8SWSJ +he file command tells )s what type of file o)r program is. It is certainly a Mac e5ec)table file. &ow b)ild the code for lin)5 amdH8B e#port N5A41S="amd>?" e#port N55/="(in+#" go buil) file simple /imple( @LM SJKbit L/! eBecutable+ B8SKSJ+ version 0 (/"/P)+ statically linke)+ not strippe) :o) need to change either one or both of the '3 #1CE environment variables to point to the target platform and architect)re. +hen yo) can b)ild the code. #fter the b)ild yo) can see the file command is reporting the program is a lin)5 e5ec)table. #s a reference* here are the different platforms and architect)res yo) can b)ild for cross compilationB ]233/ )arwin )arwin freebs) freebs) linuB linuB linuB netbs) netbs) openbs) openbs) planQ win)ows win)ows ]23.-5A I8S am)SJ I8S am)SJ I8S am)SJ arm I8S am)SJ I8S am)SJ I8S I8S am)SJ KK IE bit Mac3/4 KK SJ bit Mac3/4 KK IE bit LinuB KK SJ bit LinuB KK -:/5 LinuB

KK IE bit Win)ows KK SJ bit Win)ows

2?8

Installing Godoc and 9et


'nce yo) finish b)ilding 9o for the targets yo) want* yo) will want to install godoc and vet for the defa)lt target. +hese tools will get b)ilt and installed in yo)r 9'1''+. go get co)e.google.com/p/go.tools/cm)/go)oc go get co)e.google.com/p/go.tools/cm)/vet

$oncl'sion
+his is the doc)mentation for b)ilding 9o from the so)rceB httpB golang.org doc install so)rce +his is a doc)ment written by ;ave Cheney abo)t cross compilation. Ee has b)ild a script that yo) can download. It makes all of this real simple to performB httpB dave.cheney.net 20?" 0I 0< anCintrod)ctionCtoCcrossCcompilationCwithCgoC?C? Mitchell Eashimoto b)ilt this great tool called go5. +his tool makes it real easy to b)ild yo)r program for all the different targets witho)t the need to man)ally change the 9'#1CE and 9''3 environment variables.

C'nctions and ?aked >et'rns In Go


In 9o val)es that are ret)rned from f)nctions are passed by val)e. 9o gives yo) some nice fle5ibility when it comes to ret)rning val)es from a f)nction. Eere is a simple e5ample of ret)rning two val)es from a f)nctionB package main import ( "fmt" ) func main() { i)+ err (7 -eturn:)() if err ?7 nil { fmt.#rintf("@--3-( 's"+ err) return % fmt.#rintf(":)( ')*n"+ i)) % func -eturn:)() (int+ error) { i) (7 01 return i)+ nil % 2?G

+he f)nction 1et)rnId ret)rns a val)e of type integer and of type error. +his is something very common that is done in 9o. 6rror handling is performed by ret)rning a val)e of type error from yo)r f)nctions and the calling f)nction eval)ating that val)e before contin)ing. If yo) don.t care abo)t the error for some reason after a f)nction call ret)rns* yo) can do something like thisB i)+ W (7 -eturn:)() +his time I )sed an )nderscore to represent the ret)rn val)e for the second ret)rn arg)ment* which was the error. +his is really nice beca)se I don.t need to declare a variable to hold the val)e being passed in* I can simply ignore it. :o) also have the option to name yo)r ret)rn arg)mentsB func -eturn:)() (i) int+ err error) { i) 7 01 return i)+ err % If yo) name yo)r ret)rn arg)ments yo) are creating local variables 0)st like with yo)r f)nction parameters. +his time when I set the id variable* I remove the colon @BA from the short variable declaration and convert it to an assignment operation. +hen in the ret)rn I specify the ret)rn variables as normal. &aming yo)r ret)rn arg)ments is a nice way to doc)ment what yo) are ret)rning. +here is also something else that yo) can do with yo)r named arg)ments* or not doB func -eturn:)() (i) int+ err error) { i) 7 01 ret+rn % +his is what is called a naked ret)rn. I have removed the arg)ments from the ret)rn statement. +he 9o compiler a)tomatically ret)rns the c)rrent val)es in the ret)rn arg)ments local variables. +ho)gh this is really cool yo) need to watch for shadowingB func -eturn:)() (i) int+ err error) { i) 7 01 if i) 77 01 { err (7 fmt.@rrorf(":nvali) :)*n") return % return % If yo) try to compile this yo) will get the following compiler errorB

2?H

err is sha)owe) )uring return +o )nderstand why this error e5ists yo) need to )nderstand what c)rly bracket do inside of a f)nction. 6ach set of c)rly brackets define a new level of scope. +ake this code for e5ampleB func main() { i) (7 01 i) (7 E1 fmt.#rintf(":)( ')*n"+ i)) % If yo) try to compile this code yo) get the following errorB no new variables on left si)e of (7 +his makes sense beca)se yo) are trying to declare the same variable name twice. +he error goes away if we change the code to look like thisB func main() { i) (7 01 { i) (7 E1 fmt.#rintf(":)( ')*n"+ i)) % fmt.#rintf(":)( ')*n"+ i)) % +he c)rly brackets define a new stack frame and therefore a new level of scope. +he variable name can be re)sed inside the new set of c)rly brackets. When the code reaches the closing c)rly bracket that small piece of the stack is popped. -ook again at the code that ca)sed the shadowing errorB func -eturn:)() (i) int+ err error) { i) 7 01 if i) 77 01 ) err := fmt.Errorf("3nva(id 3d\n") return * return % Inside the if statement we are creating a new variable called err. We are not )sing the err variable declared as the f)nction ret)rn arg)ment. +he compiler recogni(es this and prod)ces the error. If the compiler did not report this error* yo) wo)ld never see the error that occ)red inside the if statement. +he ret)rn err variable is what is passed by defa)lt 2?I

&aming yo)r ret)rn arg)ments come in real handy when )sing a defer statementB func -eturn:)() (i) int+ err error) { )efer func() { if i) 77 01 { err 7 fmt.@rrorf(":nvali) :)*n") % %() i) 7 01 return % /eca)se the ret)rn arg)ments are named* yo) can reference them in the defer f)nction. :o) can even change the val)e of the ret)rn arg)ments inside the defer call and the calling f)nction will see the new val)es. +his version will display the error message. :o) need to be aware that the defer statement is eval)ated inline with the rest of the codeB func -eturn:)() (i) int+ err error) { )efer func(i) int) { if i) 77 01 { err 7 fmt.@rrorf(":nvali) :)*n") % %(i)) i) 7 01 return % +his version does not display the error message. +he val)e of id is not ?0 )ntil after the defer statement is eval)ated. 3ometimes it makes sense to )se named ret)rn arg)ments* s)ch when )sing a defer statement at the top of yo)r f)nction. If yo) are passing raw val)es o)t of yo)r f)nction then something like this does not make senseB package main import ( "fmt" ) func main() { ans (7 .))Numbers(01+ 0E) fmt.#rintf(".nswer( ')*n"+ ans) %

2?L

func .))Numbers(a int+ b int) (result int) { return a > b % +he ret)rn arg)ment is named for the #dd&)mbers f)nction b)t never )sed. Instead we ret)rn the answer of the operation directly o)t of the ret)rn. +his shows yo) how yo) can still ret)rn any val)e yo) want even if yo) name the ret)rn arg)ments. I asked the 9o comm)nity for their opinions abo)t )sing named arg)ments and naked ret)rnsB httpsB pl)s.google.com ?0IG"IIG2?G<2I<08"?I0 posts LhM0Ehmy&k2 I got a very good mi5 of opinions that I think everyone sho)ld read. 9o gives yo) a lot of fle5ibility and choice when b)ilding yo)r f)nctions. ;on.t look for a single common practice for everything. +ake each f)nction individ)ally and implement the best constr)ct for that )se case.

2y $hannel Select B'g


I was testing new f)nctionality on a program that is already r)nning in prod)ction when s)ddenly the code behaved very badly. What I saw shocked me and then it became obvio)s why it happened. I also have a race condition 0)st waiting to be a problem. I have tried to provide a simplified version of the code and the two b)gs. package main import ( "fmt" "os" "os/signal" "time" ) var /hut)own bool 7 false func main() { sig5han (7 make(chan os./ignal+ 0) signal.Notify(sig5han+ os.:nterrupt) for { select { case =Ksig5han( /hut)own 7 true continue case =Kfunc() chan struct{% { complete (7 make(chan struct{%) go Launch#rocessor(complete)

2?<

return complete %()( return % % % func Launch#rocessor(complete chan struct{%) { )efer func() { close(complete) %() fmt.#rintf("/tart Work*n") for count (7 1< count = F< count>> { fmt.#rintf("&oing Work*n") time./leep(0 $ time./econ)) if /hut)own 77 true { fmt.#rintf(";ill @arly*n") return % % fmt.#rintf("@n) Work*n") % +he idea behind this code is to r)n a task and terminate. It allows the operating system to re2)est the program to terminate early. I always like sh)tting down the program cleanly when possible. +he sample code creates a channel that is bo)nd to an operating system signal and looks for QctrlR C from the terminal window. If QctrlR C is iss)ed* the 3h)tdown flag is set to tr)e and the program contin)es back into the select statement. +he code also spawns a 9o ro)tine that performs the work. +hat ro)tine checks the 3h)tdown flag to determine if the program needs to terminate early.

B'g ?'mber
+ake a look at this part of the codeB case =Kfunc() chan struct{% { complete (7 make(chan struct{%) go Launch#rocessor(complete) return complete %()( I tho)ght I was being so clever when I wrote this code. I tho)ght it wo)ld be cool to e5ec)te a f)nction on the fly to spawn the 9o ro)tine. It ret)rns a channel that the select waits on to be told the work is complete. When the 9o ro)tine is done it closes the channel and the program

220

terminates. -et.s r)n the programB /tart Work &oing Work &oing Work &oing Work &oing Work &oing Work @n) Work #s e5pected the program starts and spawns the 9o ro)tine. 'nce the 9o ro)tine is complete the program terminates. +his time I will hit QctlrR C while the program is r)nningB /tart Gork &oing Work ^5/tart Gork &oing Work ;ill @arly ;ill @arly When I hit QctrlR C the program la)nched the 9o ro)tine again,, I tho)ght that the f)nction associated with the case wo)ld only be e5ec)ted once. +hen the select wo)ld 0)st wait on the channel moving forward. I had no idea that the f)nction wo)ld be e5ec)ted every time the loop iterated back to the select statement. +o fi5 the code I needed to remove the f)nction o)t of the select statement and spawn the 9o ro)tine o)tside of the loopB func main() { sig5han (7 make(chan os./ignal+ 0) signal.Notify(sig5han+ os.:nterrupt) comp(ete := make(c2an ,tr+ct)*) go Ha+nc2Proce,,or(comp(ete) for { select { case =Ksig5han( /hut)own 7 true continue ca,e EBcomp(ete: ret+rn %

22?

% % &ow when we r)n the program we get a better res)ltB /tart Work &oing Work &oing Work ^5;ill @arly +his time when I hit QctrlR C the program terminate early and doesn.t spawn another 9o ro)tine again.

B'g ?'mber #
+here is a second less obvio)s b)g l)rking in the code as well. +ake a look at these pieces of codeB var /hut)own bool 7 false if what/ig 77 syscall./:2:N /hut)own 7 true % {

if /hut)own 77 true { fmt.#rintf(";ill @arly*n") return % +he code is )sing a package level variable to signal the r)nning 9o ro)tine to sh)t down when QctrlR C is hit. +he code is working every time I hit QctrlR C so why is there a b)gD First let.s r)n the race detector against the codeB go buil) Krace ./test While it is r)nning I hit QctrlR C againB /tart Work &oing Work ^5777777777777777777 W.-N:N2( &. . -.5@ -ea) by goroutine F( main.Launch#rocessor() /6sers/bill//paces/ est/src/test/main.go(JS >1B01b gosche)1() /6sers/bill/go/src/pkg/runtime/proc.c(0E08 >1BQf #revious write by goroutine 0( main.main() 222

/6sers/bill//paces/ est/src/test/main.go(EF >1B0IS runtime.main() /6sers/bill/go/src/pkg/runtime/proc.c(08E >1BQ0 2oroutine F (running) create) at( main.main() /6sers/bill//paces/ est/src/test/main.go(08 >1B8f runtime.main() /6sers/bill/go/src/pkg/runtime/proc.c(08E >1BQ0 2oroutine 0 (running) create) at( Wrt1Wam)SJ() /6sers/bill/go/src/pkg/runtime/asmWam)SJ.s(8R >1B01S 777777777777777777 ;ill @arly Moun) 0 )ata race(s) My )se of the 3h)tdown flag comes )p on the race detector. +his is beca)se I have two 9o ro)tines trying to access the variable in an )nsafe way. My initial reason for not sec)ring access to the variable was practical b)t wrong. I fig)red that since the variable is only )sed to sh)tdown the program when it becomes necessary* I didn.t care abo)t a dirty read. If by chance* within the microsecond of glare there was between writing to the variable and reading the variable* if a dirty read occ)rred* I wo)ld catch it again on the ne5t loop. &o harm done* rightD Why add complicated channel or locking code for something like thisD Well* there is a little thing called the 9o Memory Model. httpB golang.org ref mem +he 9o Memory Model does not g)arantee that the 9o ro)tine reading the 3h)tdown variable will ever see the write by the main ro)tine. It is valid for the write to the 3h)tdown variable by the main ro)tine to never be written back to main memory. +his is beca)se the main ro)tine never reads the 3h)tdown variable. +his is not happening today b)t as the 9o compiler becomes more sophisticated it co)ld decide to eliminate the write to the 3h)tdown variable altogether. +his behavior is allowed by the 9o Memory Model. #lso* we don.t want code that can.t pass the race detector* it is 0)st bad practice* even for practical reasons. Eere is a final version of the code with all b)gs fi5edB package main import ( "fmt" "os" "os/signal" "sync/atomic" 22"

"time" ) var /2+tdo.n int%2 = " func main() { sig5han (7 make(chan os./ignal+ 0) signal.Notify(sig5han+ os.:nterrupt) comp(ete := make(c2an ,tr+ct)*) go Ha+nc2Proce,,or(comp(ete) for { select { case =Ksig5han( atomic./tore3nt%2( /2+tdo.n, 1) continue ca,e EBcomp(ete: ret+rn % % % func Launch#rocessor(complete chan struct{%) { )efer func() { close(complete) %() fmt.#rintf("/tart Work*n") for count (7 1< count = F< count>> { fmt.#rintf("&oing Work*n") time./leep(0 $ time./econ)) if atomic.Hoad3nt%2( /2+tdo.n) == 1 { fmt.#rintf(";ill @arly*n") return % % fmt.#rintf("@n) Work*n") % I prefer to )se an if statement to check if the 3h)tdown flag is set so I can sprinkle that code as needed. +his sol)tion changes the 3h)tdown flag from a boolean val)e to an int"2 and )ses the atomic f)nctions 3tore and -oad. In the main ro)tine if a QctrlR C is detected* the 3h)tdown flag is safely changed from 0 to ?. In the -a)nch$rocessor 9o ro)tine* the val)e of the 3h)tdown flag is compared to ?. If that 228

condition is tr)e the 9o ro)tine ret)rns. It.s ama(ing sometimes how a simple program like this can contain a few gotchas. +hings yo) may have never tho)ght abo)t or reali(ed when yo) started. 6specially when the code always seems to work.

2anage Dependencies With G(DEP Introd'ction


If yo) are )sing "rd party packages* @packages that yo) don.t own or controlA* yo) will want a way to create a reprod)cible b)ild every time yo) b)ild yo)r pro0ects. If yo) )se "rd party packages directly and the package a)thors change things* yo)r pro0ects co)ld break. 6ven if things don.t break* code changes co)ld create inconsistent behavior and b)gs. Keith 1arick.s tool godep is a great step in the right direction for managing "rd party dependencies and creating reprod)cible b)ilds. +he godep tool gives yo) two options for managing dependencies. +he first option creates a dependency file with version control information and then with some godep magic* the code is b)ilt against those versions. :o) can also =endor yo)r "rd party packages inside yo)r pro0ects as well. :o) never need to change a single so)rce code file and everything is accomplished in con0)nction with the go tooling.

Do"nloading Godep
;ownload godep )sing go get and make s)re yo)r >9'$#+E bin directory is in yo)r $#+E. go get github.com/kr/go)ep eBport #. A7]#. A(]23#. A/bin

$reate ) Pro/ect
/)ild yo)r pro0ect )sing the "rd party packages as yo) normally wo)ld. 3ince godep does not re2)ire yo) to change any import paths in the code* .go get. the code yo) need and import those packages directly. +o keep the post simple* I am going to )se an e5isting program called &ews 3earch that )ses one "rd party dependency. eBport 23#. A7]A3M@/eBample go get github.com/goinggo/newssearch

22G

#fter .go get. completes* I have the following code on disk inside the 9'$#+E. +he &ews 3earch program is )sing code from a different 9oing 9o repository* which for this pro0ect is a "rd party package. 3ince .go get. was s)ccessf)l* the code b)ilt and is installed.

Dependency 2anagement
'nce yo) have a pro0ect that can b)ild and install properly* yo) can )se godep to create the 9odeps dependency file. Change to the root location for the pro0ect and r)n the godep sa:e command with the <copy+6alse optionB c) ]23#. A/src/github.com/goinggo/newssearch go)ep save Kcopy7false 'nce the save is complete* godep creates a file called 9odeps. :o) will save this file with yo)r pro0ectB { ":mport#ath"( "github.com/goinggo/newssearch"+ "2oPersion"( "go0.0.E"+ "&eps"( U { ":mport#ath"( "github.com/goinggo/utilities/workpool"+ "-ev"( "ReS0J0)S0bEa0Sae8IQ88Q1RI18f8e1QfR1Ia1)1" % V % +he 9odeps file contains everything godep needs to create a reprod)cible b)ild. +he 9odep file lists each "rd party package and the git commit n)mber for that version of code to )se.

22H

&ow remove the "rd party package from its original location and perform a b)ild )sing the godep toolB go)ep go buil)

If yo) remove the original "rd party code* yo) can.t )se .go b)ild. directly anymore* the imports don.t e5ist. +o b)ild the pro0ect )se .go b)ild. from the godep tool. :o) can also )se .go install. and .go test. as long as yo) r)n those commands thro)gh godep. godep go b)ild godep go install godep go test +o make this work* godep performs a bit of magic. It )ses a working directory and manip)lates the 9'$#+E )nderneath. 1)n the godep path command from inside the pro0ect folderB c) ]23#. A/src/github.com/goinggo/newssearch go)ep path

22I

:o) sho)ld see the following o)tp)tB / var/fol)ers/8\/)Epf)kWBJ\)JWWlSgypvNswJ1111gn/ /go)ep/rev/7e:> 1?1d>1$2a1>aeD%TDDT"7%"DfDe"Tf7"%a"d" If yo) open that folder yo) will see the code for that version. +his code is being )sed to b)ild the pro0ectB

+he godep tool will contin)e to )se the code from this location to perform f)t)re b)ilds if it e5ists. Calling godep go b'ild will download the version of code specified in the 9odeps file if it doesn.t already e5ist. If yo) open any of yo)r so)rce code files yo) will see the imports have not changed. +he way godep works* it doesn.t have to change a thing. +his is one of the really awesome things abo)t godep.

.pdating Dependencies
When it is time to )pdate one of yo)r "rd party libraries 0)st .go get. it. +hen yo) 0)st need to r)n godep sa:e once again to )pdate the 9odeps file. /eca)se the imports paths in the so)rce code files are not changed* godep will find and )pdate the dependencies. I have changed the "rd party package and p)shed it )p to 9itE)bB

&ow I .go get. the code changes and )pdate the 9odeps fileB go get github.com/goinggo/utilities c) ]23#. A/src/github.com/goinggo/newssearch go)ep save If I open the 9odeps file the dependencies have changedB

22L

{ ":mport#ath"( "github.com/goinggo/newssearch"+ "2oPersion"( "go0.0.E"+ "&eps"( U { ":mport#ath"( "github.com/goinggo/utilities/workpool"+ "-ev"( "8ec)10ec1IFeEQQ0FaaS8QRaII8FeeJf8)81cc1F" % V % &ow I )se godep to b)ild the codeB go)ep go buil)

+he godep tool downloaded the new version and b)ilt the code s)ccessf)lly.

9endoring
=endoring is the act of making yo)r own copy of the "rd party packages yo)r pro0ect is )sing. +hose copies are traditionally placed inside each pro0ect and then saved in the pro0ect repository. +he godep tool s)pports =endoring and will place the copies inside the pro0ect that are )sing them. +o =endor code with godep, don.t )se any options with the save command. First clean the workspace and download a new version of the &ews 3earch programB eBport 23#. A7]A3M@/eBample go get github.com/goinggo/newssearch c) ]23#. A/src/github.com/goinggo/newssearch &ow iss)e the godep sa:e command again b)t this time witho)t the copy optionB go)ep save

22<

+his time yo) will have a 9odeps folder with a special workspace s)bfolder and the 9odeps file. #ll the "rd party packages are copied into the workspace folder )nder src. +his is set)p to work with 9'$#+E. =ersion control files are removed and no import paths are changed in any of the so)rce code files. &e5t remove the original code for the "rd party package and perform the b)ildB go)ep go buil) +he b)ild is s)ccessf)l and everything is ready to be p)shed back into the repository. $erforming an )pdate is as simple as downloading the new version of the "rd party package and r)nning godep sa:e again.

$oncl'sion
+he godep tool solves many of the problems that e5ist with creating reprod)cible b)ilds. It is incredibly easy to )se and sits on top of the go tooling. It doesn.t change anything abo)t go* how yo) write go programs or how yo) import "rd party packages. +he only drawback is that godep does not s)pport /a(aar )sing the nonCvendored option. For the p)blic packages yo)r are p)blishing* yo) can incl)de a 9odeps file to provide yo)r Kstable b)ildK config)ration. $ackage )sers can choose to )se it or not. +hat is really cool. /)ild the code with go directly or thro)gh godep. In the end* godep is a tool thatB ?. 3)pports a =endor and &onC=endor sol)tion that provides a reprod)cible b)ild 2. Maintains backwards compatible with all e5isting 9o packages ". $rovides a way to p)blish and access the Kstable b)ildK config)ration of a prod)ct 8. 6asy to )pdate package dependencies when new package versions are available

Write &o'r Go Programs .sing GEdit

2"0

This is a guest post from Tad <izbaras from Atasoft in &outh Florida. +here are a n)mber of editors and I;6s for 9o development. -iteIde* =im* 6macs and 96dit 0)st to name a few. 6ach developer has their own favorite editor for each lang)age they work with. 3ome like f)ll feat)red I;6 environments while others prefer speed over feat)res. My personal favorite editors for 9o development at the moment are =im and 96dit. 96dit comes as part of many -in)5 distros. If yo) )se 7b)nt)* it is part of the operating system. 96dit has some decent feat)res likeB Z 3ynta5 Eighlighting Z 3plit Windows Z Word Wrapping #dvanced feat)res are left to be handled by e5ternal pl)gCins. I prefer pro0ectCless development. +hat means there are no formal pro0ect files and pro0ects are preserved via a workspace bo)nd to a directory str)ct)re. 9o has e5cellent s)pport for pro0ectCless development. When b)ilding and installing yo)r pro0ects* the 9o tooling* in con0)nction with the way 9o packages code* can minimi(e the need for e5ternal scripts and makefiles. 96dit is a decent editor b)t I co)ld not find any good $l)gCins that wo)ld allow me to perform a 9o b)ild right from the editor. +he K65ternal +oolsK $l)gCin has worked for me. I was able to set )p shortc)ts and get Kgo b)ildK to e5ec)te. When yo) click on errors* displayed in the bottom pane of 96dit* the c)rsor 0)mps to e5act error location. When I started programming in 9o* the K65ternal +oolsK $l)gCin worked for me for 2)ite some time. /)t after awhile* I started to wish that Kgo b)ildK wo)ld r)n similar to how -inters ran. With -inters* yo) can r)n a command after the file is saved. 3ince 9o )s)ally takes only few seconds to b)ild* the $l)gCin co)ld e5ec)te a Kgo b)ildK on save and then 0)mp to the error location if there were any. I wrote a 96dit $l)gCin that is developed in $ython. ;epending on the version of $ython yo) have installed* yo) may re2)ire some small ad0)stments. +his is covered in the Known Iss)es section below. 'ps* I forgot to mention... 9o is #wesome. /)t yo) probably already know that.

2eet GoB'ild 6or GEdit %5@


9o/)ild C 96dit " $l)gCin for 9o @golangA development. 9o/)ild $l)gCin version ?.0 for 96dit. $l)gCin attaches to the onHsa:e event in 96dit for 9o so)rce code files only. It does nothing for any other file type. It will r)n Kgo b)ildK after the file is saved. If the c)rrent filename has KFtest.goK in the name* then the $l)gCin will r)n Kgo testK against the c)rrent file.s directory. +he $l)gCin will wait a n)mber of seconds for the b)ild or test to complete. It will timeo)t and 2)it the b)ild or test so 96dit will not free(e.

2"?

Images
$l)gCin capt)res Kgo b)ildK errors and shows them in the 9edit stat)s bar. It also 0)mps to the first error and highlights error line if error is in the c)rrent file.

$l)gCin shows the last s)ccessf)l b)ild.

2"2

$l)gCin shows any Kgo testK fail)res.

2""

Do"nload
I have posted the pl)gin on 9itE)b. $lease send any feedback yo) may have. httpsB gith)b.com tadvi geditCgob)ild

Ino" Iss'es
Eas been tested on 7b)nt) ?".08 and ?".?0. 7b)nt) ?".?0 re2)ires small change in the gob)ild.pl)gin. -ine with python sho)ld be changed to python" like belowB -oaderJpython" +his is beca)se 9edit seems to defa)lt on )sing python " instead of python 2.I on newer versions of -in)5.

.sage
Z 3imply drop files into M .local share gedit pl)gins . Z If this directory does not e5ist C create it. Z 3tart 96dit Z 'pen 6ditC$references* then pl)gCins and check K9o/)ild after saveK pl)gCin.

?otes
Z C)rrent b)ild directory is determined based on the active open so)rce file. Z $l)gCin is designed for fast development on small to mid si(e pro0ects. Z 3o)rce code file is b)ilt with every save. Z +ight iteration of the saveCeditCsaveCedit cycle. Z &ot designed for large 9o pro0ects beca)se compilation will timeo)t if it takes too long. Z &ot designed for 9o )nit tests that take a long time to r)n. Z If yo) work on 9o pro0ects with b)ild times over G seconds* this pl)gCin sho)ld be modified to )se keyboard shortc)t @s)ch as .FG.A instead of onFsave action.

.sing BSLT With Go


I am working on a pro0ect that re2)ires p)lling and processing different NM- feeds from the web and storing the data into Mongo;/ as O3'&. 3ince new feeds come )p everyday* changing the 9o program to process and p)blish new feeds is o)t of the 2)estion. # second constraint is that processing has to work in Iron.io or any other lin)5 clo)d based environment. What I needed was a 9o program that co)ld take an NM- doc)ment and N3-+ stylesheet at r)ntime* transform the NM- into O3'& and then store the O3'& to Mongo;/. I have some specific field names and other re2)irements for the O3'& doc)ment that I need to make s)re e5ist. N3-+ makes this real easy to s)pport. #t first I looked at the different C libraries that e5ist. I fig)red I co)ld integrate a library 2"8

)sing C9' b)t after a few ho)rs I reali(ed this was not going to work. +he libraries I fo)nd were h)ge and comple5. +hen by chance I fo)nd a reference abo)t a program called 5sltproc. +he program e5ists both for the Mac and -in)5 operating systems. In fact* it comes preC installed on the Mac and an aptCget will get yo) a copy of the program on yo)r lin)5 operating system. I have b)ilt a sample program that shows how to )se 5sltproc in yo)r 9o programs. /efore we download the sample code we need to make s)re yo) have 5sltproc installed. If yo) are r)nning on a Mac* 5sltproc sho)ld already e5ist )nder )sr bin which Bsltproc /usr/bin/Bsltproc 'n yo)r lin)5 operating system 0)st r)n aptCget if yo) don.t already have 5sltproc installed su)o aptKget install Bsltproc +he 5sltproc program will be installed in the same place )nder )sr bin. +o make s)re everything is good* r)n the 5sltproc program re2)esting the versionB Bsltproc KKversion Bsltproc was compile) against libBml E1R18+ libBslt 010ES an) libeBslt 80F libBslt 010ES was compile) against libBml E1R18 libeBslt 80F was compile) against libBml E1R18 +o download and try the sample program* open a terminal session and r)n the following commandsB eBport 23#. A7]A3M@/eBample go get github.com/goinggo/Bslt c) ]23#. A/src/github.com/goinggo/Bslt go buil) If yo) want to install the code )nder yo)r normal 9'$#+E* start with the .go get. line. Eere are the files that sho)ld e5ist after the b)ildB main.go )eals.Bml stylesheet.Bslt fee) to 9/3N Bslt KK /ource co)e for test program KK /ample 4ML )ocument from "ipit KK /tylesheet to transform the "ipit 4ML KK est program

-et.s look at a portion of the NM- doc)ment the sample program will transformB =)ealsH =listKitemH

2"G

=yipitWurlHhttp(//yipit.com/business/ron)eausK kickboBing/=/yipitWurlH =en)W)ateHE10JK10KER0S(11(1I=/en)W)ateH =titleHLet a Mormer #ro each "ou a Mew ;icks of the Month...=/titleH =tagsH =listKitemH =url /H =nameHMitness 5lasses=/nameH =slugHfitnessKclasses=/slugH =/listKitemH =/tagsH ... =/listKitemH =/)ealsH

ra)e

+he NM- can be fo)nd in the deals.5ml file. It is an e5tensive NM- doc)ment and too large to show in its entirety. -et.s look at a portion of the N3-+ stylesheetB =ZBml version7"0.1" enco)ing7"6 MK8"ZH =Bsl(stylesheet Bmlns(Bsl7"http(//www.wI.org/0QQQ/4/L/ ransform" Bmlns(str7"http(//eBslt.org/strings" version7"0.1" eBtensionKelementKprefiBes7"str"H =Bsl(output metho)7"teBt" /H =Bsl(template name7"clean eBt"H =Bsl(param name7"p eBt" /H =Bsl(variable name7"cleane)0" select7"str(replace(]p eBt+ CG\uot<C+ CC)" /H =Bsl(variable name7"cleane)E" select7"str(replace(]cleane)0+ C*C+ CC)" /H =Bsl(variable name7"cleane)I" select7"str(replace(]cleane)E+ CGOB.<C+ CC)" /H =Bsl(valueKof select7"]cleane)I" /H =/Bsl(templateH ... =Bsl(template match7"/"H{")eals"( U =Bsl(forKeach select7"root/response/)eals/listKitem"H{ ")eali)"( =Bsl(valueKof select7"i)" /H+ "fee)"( ""ipit"+ ")ateWa))e)"( "=Bsl(valueKof select7")ateWa))e)" /H"+ "en)W)ate"( "=Bsl(valueKof select7"en)W)ate" /H"+ ... "categories"( U=Bsl(forKeach select7"tags/listK item"H"=Bsl(valueKof select7"slug"/H"=Bsl(chooseH=Bsl(when test7"position() ?7 last()"H+=/Bsl(whenH=/Bsl(chooseH=/Bsl(forKeachHV+ ... 2"H

%=Bsl(chooseH=Bsl(when test7"position() ?7 last()"H+ =/Bsl(whenH=/Bsl(chooseH =/Bsl(forKeachH V% =/Bsl(templateH =/Bsl(stylesheetH +his N3-+ can be fo)nd in the stylesheet.5slt file. It is an e5tensive N3-+ stylesheet with templates to help clean)p the NM- data. 3omething really great abo)t 5sltproc is that it already contains a b)nch of great e5tensionsB ./BsltprocW)arwin K)umpeBtensions -egistere) 4/L @Btensions KKKKKKKKKKKKKKKKKKKKKKKKKK -egistere) @Btension Munctions( {http(//eBslt.org/math%lowest {http(//eBslt.org/math%power {http(//eBslt.org/strings%concat {http(//eBslt.org/)atesKan)Ktimes%)ate {http(//eBslt.org/)atesKan)Ktimes%)ayKname {http(//eBslt.org/common%obDectKtype {http(//eBslt.org/math%atan {http(//eBslt.org/strings%enco)eKuri {http(//eBslt.org/strings%)eco)eKuri {http(//eBslt.org/)atesKan)Ktimes%a))K)uration {http(//eBslt.org/)atesKan)Ktimes%)ifference {http(//eBslt.org/)atesKan)Ktimes%leapKyear {http(//eBslt.org/)atesKan)Ktimes%monthKabbreviation {http(//eBslt.org/)ynamic%map {http(//eBslt.org/math%tan {http(//eBslt.org/math%eBp {http(//eBslt.org/)atesKan)Ktimes%)ateKtime {http(//eBslt.org/)atesKan)Ktimes%)ayKinKweek {http(//eBslt.org/)atesKan)Ktimes%secon)KinKminute {http(//eBslt.org/)atesKan)Ktimes%year {http(//icl.com/saBon%evaluate {http(//eBslt.org/math%log {http(//eBslt.org/)atesKan)Ktimes%a)) {http(//eBslt.org/)atesKan)Ktimes%)ayKabbreviation {http(//icl.com/saBon%lineKnumber {http(//eBslt.org/math%constant {http(//eBslt.org/sets%)ifference {http(//eBslt.org/)atesKan)Ktimes%)uration {http(//eBslt.org/)atesKan)Ktimes%minuteKinKhour {http(//icl.com/saBon%eval {http(//eBslt.org/math%min {http(//eBslt.org/math%maB {http(//eBslt.org/math%highest {http(//eBslt.org/math%ran)om {http(//eBslt.org/math%s\rt 2"I

{http(//eBslt.org/math%cos {http(//eBslt.org/sets%hasKsameKno)e {http(//eBslt.org/strings%tokeniNe {http(//eBslt.org/)atesKan)Ktimes%secon)s {http(//eBslt.org/)atesKan)Ktimes%time {http(//eBslt.org/)ynamic%evaluate {http(//eBslt.org/common%no)eKset {http(//eBslt.org/)atesKan)Ktimes%monthKname {http(//eBslt.org/)atesKan)Ktimes%weekKinKyear {http(//eBslt.org/math%acos {http(//eBslt.org/sets%intersection {http(//eBslt.org/sets%lea)ing {http(//eBslt.org/sets%trailing {http(//eBslt.org/strings%replace {http(//eBslt.org/)atesKan)Ktimes%)ayKinKyear {http(//icl.com/saBon%eBpression {http(//eBslt.org/math%abs {http(//eBslt.org/math%sin {http(//eBslt.org/math%asin {http(//eBslt.org/math%atanE {http(//eBslt.org/sets%)istinct {http(//eBslt.org/)atesKan)Ktimes%hourKinK)ay {http(//eBslt.org/)atesKan)Ktimes%sum {http(//eBslt.org/)atesKan)Ktimes%weekKinKmonth {http(//eBslt.org/strings%split {http(//eBslt.org/strings%pa))ing {http(//eBslt.org/strings%align {http(//eBslt.org/)atesKan)Ktimes%)ayKinKmonth {http(//eBslt.org/)atesKan)Ktimes%)ayKofKweekKinKmonth {http(//eBslt.org/)atesKan)Ktimes%monthKinKyear {http(//Bmlsoft.org/4/L /%test -egistere) @Btension @lements( {http(//eBslt.org/common%)ocument {http(//eBslt.org/functions%result {http(//Bmlsoft.org/4/L /%test -egistere) @Btension Mo)ules( http(//eBslt.org/functions http(//icl.com/saBon http(//Bmlsoft.org/4/L / -ook at the stylesheet to see how to access these e5tensions. I am )sing the strings e5tension to help replace characters that are not O3'& compliant. &ow let.s look at the sample code that )ses 5sltproc to process the NM- against the N3-+ stylesheetB package main import ( 2"L

"enco)ing/Dson" "fmt" "os" "os/eBec" ) type )ocument mapUstringVinterface{% func main() { Dson&ata+ err (7 process4slt("stylesheet.Bslt"+ ")eals.Bml") if err ?7 nil { fmt.#rintf("#rocess4slt( 's*n"+ err) os.@Bit(0) % )ocuments (7 struct { &eals UV)ocument XDson(")eals"X %{% err 7 Dson.6nmarshal(Dson&ata+ G)ocuments) if err ?7 nil { fmt.#rintf("6nmarshal( 's*n"+ err) os.@Bit(0) % fmt.#rintf("&eals( ')*n*n"+ len()ocuments.&eals)) for W+ )eal (7 range )ocuments.&eals { fmt.#rintf("&eal:)( ')*n"+ int()ealU")eali)"V. (floatSJ))) fmt.#rintf(" itle( 's*n*n"+ )ealU"title"V.(string)) % % func process4slt(BslMile string+ BmlMile string) (Dson&ata UVbyte+ err error) { cm) (7 eBec.5m){ .rgs( UVstring{"Bsltproc"+ BslMile+ BmlMile%+ @nv( os.@nviron()+ #ath( "Bsltproc"+ % Dson/tring+ err (7 cm).3utput() if err ?7 nil { return Dson&ata+ err % fmt.#rintf("'s*n"+ Dson/tring) Dson&ata 7 UVbyte(Dson/tring) 2"<

return Dson&ata+ err % +he processNslt f)nction )ses an e5ec.Cmd ob0ect to shell o)t and r)n the 5sltproc program. +he key to making this work is the cmd.')tp)t f)nction. +he 5sltproc program will ret)rn the res)lt of the transformation to stdo)t. +his means we only need to write the 5ml and 5slt files to disk before r)nning 5sltproc. We will receive the res)lt from 5sltproc as a string from the cmd.')tp)t call. 'nce the processNslt f)nction has the res)lting O3'& transformation from 5sltproc* the O3'& is displayed on the screen and then converted to a slice of bytes for f)rther processing. In main after the call to the processNslt f)nction* the slice of bytes containing the O3'& transformation is )nmarshalled into a map so it can be cons)med by o)r 9o program and displayed on the screen. In the f)t)re that map can be stored in Mongo;/ via the mgo Mongo;/ driver. +he 5sltproc program can be )ploaded to any clo)d environment that will allow yo) to write the NM- and N3-+ to disk. I have been s)ccessf)l in )sing 5sltproc inside an Iron.io IronWorker container. If yo) have the need to process N3-+ in yo)r 9o programs* give this a try.

.sing The Log Package In Go


-in)5 is )ni2)e to Windows in many ways* and writing programs in -in)5 is no e5ception. +he )se of standard o)t* standard err and n)ll devices is not only a good idea b)t it.s the law. If yo)r programs are going to be logging information* it is best to follow the destination conventions. +his way yo)r programs will work with all of the Mac -in)5 tooling and hosted environments. 9o has a package in the standard library called log and a type called logger. 7sing the log package will give yo) everything yo) need to be a good citi(en. :o) will be able to write to all the standard devices* c)stom files or any destination that s)pport the io.Writer interface. I have provided a really simple sample that will get yo) started with )sing loggerB package main import ( "io" "io/ioutil" "log" "os" ) var ( -.5@ :NM3 $log.Logger $log.Logger

280

W.-N:N2 $log.Logger @--3$log.Logger ) func :nit( traceAan)le io.Writer+ infoAan)le io.Writer+ warningAan)le io.Writer+ errorAan)le io.Writer) { -.5@ 7 log.New(traceAan)le+ " -.5@( "+ log.L)ate[log.Ltime[log.Lshortfile) :NM3 7 log.New(infoAan)le+ ":NM3( "+ log.L)ate[log.Ltime[log.Lshortfile) W.-N:N2 7 log.New(warningAan)le+ "W.-N:N2( "+ log.L)ate[log.Ltime[log.Lshortfile) @--3- 7 log.New(errorAan)le+ "@--3-( "+ log.L)ate[log.Ltime[log.Lshortfile) % func main() { :nit(ioutil.&iscar)+ os./t)out+ os./t)out+ os./t)err) -.5@.#rintln(": have something stan)ar) to say") :NM3.#rintln("/pecial :nformation") W.-N:N2.#rintln(" here is something you nee) to know about") @--3-.#rintln("/omething has faile)") % When yo) r)n this program yo) will get the follow o)tp)tB :NM3( E10I/00/1F 08(00(10 main.go(JJ( /pecial :nformation W.-N:N2( E10I/00/1F 08(00(10 main.go(JF( here is something you nee) to know about @--3-( E10I/00/1F 08(00(10 main.go(JS( /omething has faile) :o) will notice that +1#C6 logging is not being displayed. -et.s look at the code to find o)t why. -ook at the +1#C6 logger piecesB -.5@ $log.Logger

28?

-.5@ 7 log.New(traceSand(e+ " -.5@( "+ log.L)ate[log.Ltime[log.Lshortfile) :nit(io+ti(.!i,card+ os./t)out+ os./t)out+ os./t)err) -.5@.#rintln(": have something stan)ar) to say") +he code creates a package level variable called +1#C6 which is a pointer to a log.-ogger ob0ect. +hen inside the Init f)nction* a new log.-ogger ob0ect is created. +he parameters to the log.&ew f)nction are as followsB func New(out io.Writer+ prefiB string+ flag int) $Logger out( he out variable sets the )estination to which log )ata will be written. prefiB( he prefiB appears at the beginning of each generate) log line. flags( he flag argument )efines the logging properties. Mlags( const ( // !its orCe) together to control whatCs printe). here is no control over the // or)er they appear (the or)er liste) here) or the format they present (as // )escribe) in the comments). . colon appears after these items( // E11Q/10/EI 10(EI(EI.0EI0EI /a/b/c/).go(EI( message L)ate 7 0 == iota // the )ate( E11Q/10/EI Ltime // the time( 10(EI(EI Lmicrosecon)s // microsecon) resolution( 10(EI(EI.0EI0EI. assumes Ltime. Llongfile // full file name an) line number( /a/b/c/).go(EI Lshortfile // final file name element an) line number( ).go(EI. overri)es Llongfile Lst)Mlags 7 L)ate [ Ltime // initial values for the stan)ar) logger ) In this sample program the destination for +1#C6 is io)til.;iscard. +his is a n)ll device where all write calls s)cceed witho)t doing anything. +herefore when yo) write )sing +1#C6* nothing appears in the terminal window. -ook at I&F'B :NM3 $log.Logger -.5@ 7 log.New(infoSand(e+ ":NM3( "+ 282

log.L)ate[log.Ltime[log.Lshortfile) :nit(ioutil.&iscar)+ o,./tdo+t+ os./t)out+ os./t)err) :NM3.#rintln("/pecial :nformation") For I&F' os.3tdo)t is passed into Init for the infoEandle. +his means when yo) write )sing I&F'* the message will appear on the terminal window* via standard o)t. -ast* look at 611'1B @--3- $log.Logger -.5@ 7 log.New(errorSand(e+ ":NM3( "+ log.L)ate[log.Ltime[log.Lshortfile) :nit(ioutil.&iscar)+ os./t)out+ os./t)out+ o,./tderr) :NM3.#rintln("/pecial :nformation") +his time os.3tderr is passed into Init for the errorEandle. +his means when yo) write )sing 611'1* the message will appear on the terminal window* via standard error. Eowever* passing these messages to os.3tderr allows other applications r)nning yo)r program to know an error has occ)rred. 3ince any destination that s)pport the io.Writer interface is accepted* yo) can create and )se filesB fi(e+ err (7 os.3penMile("file.tBt"+ os.3W5-@. @[os.3WW-3NL"[ os.3W.##@N&+ 1SSS) if err ?7 nil { log.Matalln("Maile) to open log file"+ output+ "("+ err) % M"M:L@ 7 log.New(fi(e+ "#-@M:4( "+ log.L)ate[log.Ltime[log.Lshortfile) In the sample code* a file is opened and then passed into the log.&ew call. &ow when yo) )se M:FI-6 to write* the writes go to file.t5t. :o) can also have the logger write to m)ltiple destinations at the same time. fi(e+ err (7 os.3penMile("file.tBt"+ os.3W5-@. @[os.3WW-3NL"[ os.3W.##@N&+ 1SSS) if err ?7 nil { log.Matalln("Maile) to open log file"+ output+ "("+ err) % m+(ti (7 io.MultiWriter(fi(e+ o,./tdo+t) 28"

M"M:L@ 7 log.New(m+(ti+ "#-@M:4( "+ log.L)ate[log.Ltime[log.Lshortfile) Eere writes are going to the file and to standard o)t. &otice the )se of log.Fatalln in the handling of any error with 'penFile. +he log package provides an initial logger that can be config)red as well. Eere is a sample program )sing log with the standard config)rationB package main import ( "log" ) func main() { log.#rintln("Aello Worl)") % Eere is the o)tp)tB E10I/00/1F 08(JE(ES Aello Worl) If yo) want to remove the formatting or change it* yo) can )se the log.3etFlags f)nctionB package main import ( "log" ) func main() { log./etMlags(1) log.#rintln("Aello Worl)") % Eere is the o)tp)tB Aello Worl) &ow all the formatting has been removed. If yo) want to send the o)tp)t to a different destination )se the log.3et')tp)tB package main import ( "io/ioutil" "log" )

288

func main() { log./et3utput(ioutil.&iscar)) log.#rintln("Aello Worl)") % &ow nothing will display on the terminal window. :o) can )se any destination that s)pport the io.Writer interface. /ased on this e5ample I wrote a new logging package for all my programsB go get gith)b.com goinggo tracelog I wish I knew abo)t log and loggers when I started writing 9o programs. 65pect to see a lot more of the log package from me in the f)t)re.

Label Breaks In Go
Eave yo) ever fo)nd yo)rself in this sit)ation. :o) have a case statement inside of a for loop and yo) wo)ld like to break from both the case and for statements in a single callD var err error timeout (7 time..fter(I1 $ time./econ)) sig5han (7 make(chan os./ignal+ 0) signal.Notify(sig5han+ os.:nterrupt) complete (7 make(chan error) go launch#rocessor(complete) for { select { case =Ksig5han( atomic./tore:ntIE(Gshut)ownMlag+ 0) continue case =Ktimeout( os.@Bit(0) case err 7 =Kcomplete( $reak % // !reak the loop $reak % return err Eere I have an endless for loop waiting on three channels )sing a select statement.

28G

+he first case is listening for an operating system Interr)pt event. If the operating system re2)ests the program to sh)tdown* this case will set a package level variable and contin)e back into the loop. +he second case is listening for a timeo)t event. If the programs r)ns for "0 seconds* the timeo)t event will fire and the program will immediately terminate. +he third case is listening for a complete event. If the 9oro)tine that is la)nched prior to entering the loop completes it work* it will notify the code on this channel. In this case we need to break o)t of both the case and the for loop. Fort)nately there isn.t any more logic to process o)tside of the select statement* so the second break statement works. If there were other cases that broke o)t of the select statement and did not re2)ire the loop to terminate* I wo)ld be in tro)ble. +he code wo)ld re2)ire more logic and flags to determine when to break o)t of the loop and when to contin)e iterating. 9o has an answer to this coding delima. :o) can define a label and break to that label. var err error timeout (7 time..fter(I1 $ time./econ)) sig5han (7 make(chan os./ignal+ 0) signal.Notify(sig5han+ os.:nterrupt) complete (7 make(chan error) go launch#rocessor(complete) Hoop: for { select { case =Ksig5han( atomic./tore:ntIE(Gshut)ownMlag+ 0) continue case =Ktimeout( os.@Bit(0) case err 7 =Kcomplete( $reak Hoop % % return err I have changed the code a bit by declaring a label called -oop 0)st above the for statement. +hen in the last case* the break statement is provided the name of that label. +his single call to break will 0)mp the e5ec)tion of the program o)tside of the for loop and to the ne5t line of code. In this case* the ne5t line of code is the call to ret)rn err. :o) can also )se a label with a contin)e statement. +his is a silly e5ample b)t it shows yo) the mechanismB 28H

guestList (7 UVstring{"bill"+ "Dill"+ "Doan"% arrive) (7 UVstring{"sally"+ "Dill"+ "Doan"% 5heckList( for W+ guest (7 range guestList { for W+ person (7 range arrive) { fmt.#rintf("2uestU'sV #ersonU'sV*n"+ guest+ person) if person 77 guest { fmt.#rintf("Let 's :n*n"+ person) continue 5heckList % % % Eere is the o)tp)tB 2uestUbillV 2uestUbillV 2uestUbillV 2uestUDillV 2uestUDillV Let Dill :n 2uestUDoanV 2uestUDoanV 2uestUDoanV Let Doan in #ersonUsallyV #ersonUDillV #ersonUDoanV #ersonUsallyV #ersonUDillV #ersonUsallyV #ersonUDillV #ersonUDoanV

In this e5ample there are two for loops* one nested inside the other. From the nested for loop* the contin)e statement )ses a label to 0)mp back to the o)ter for loop. From the o)tp)t* yo) can see that the o)ter for loop starts its ne5t iteration. 'nce the o)ter for loop is complete* the e5ec)tion of the program contin)es on. If yo) think this is 0)st a fancy goto statement* it really isn.t. +he label being referenced m)st enclose the same for* switch or select statement. #s yo) saw* the contin)e will still begin the ne5t iteration of the for loop. 7sing label breaks and contin)es in these scenario keeps the code clean and precise.

B'ilding ) Weather )pp .sing Go

28I

#t #rdan 3t)dios we have spent the last H months* in o)r spare time and on weekends* b)ilding a cons)mer based mobile application called ')tCast. +he mobile application is tailored towards those who like spending time o)tdoors* whether that be fishing* h)nting or any other type of activity.

+his first release of ')tCast shows the conditions for the b)oy stations and marine forecasts areas within the 7nited 3tates. #ll this information is )pdated every ?0 min)tes and there are map views with traditional grids and search. +he backend processing for b)oy and marine data is b)ilt )sing 9o and r)nning at Iron.I' as a sched)led worker task. +he b)oy processing downloads a te5t file from the &'## website and rips thro)gh it* )pdating Mongo;/ with any changes. +he marine processing is a bit more complicated. +his re2)ires p)lling down m)ltiple web pages from the &'## website and parsing o)t all the te5t. 9o made b)ilding and r)nning these tasks a bree(e. #nother important aspect of ')tCast is real time weather radar for the last G0 min)tes. +his has been very challenging on m)ltiple levels. Mainly beca)se we needed a real good image 28L

library that wo)ld r)n on -in)5 and co)ld be integrated with 9o. We were fort)nate to find ImageMagick.s MagickWand C #$I and their 9o package that provides the C9' bindings @httpsB gith)b.com gographics imagickA. $rocessing images is an intense piece of work. 3ometimes it takes " seconds to clean a single image. With ?GG radar stations that need to be processed every G min)tes* it took )s several refactors to get things working well. +he MagickWand library can only handle processing one image at a time. +his restriction places a lot of stress on getting things done acc)rately within an acceptable amo)nt of time. Eere is a sample of a radar image before and after processingB

+here is another interesting constraint. &'## )pdates the images every ?20 seconds on different time bo)ndaries. If the program can.t download all the images very 2)ickly* the application co)ld have images o)t of sync when they are animated across the map. Many radar images cross each other like in 'rlando* F-. 'n this area of the map we have " radar images overlapping each other. 3ometimes the images are not available. +he program goes o)t to get the image and it doesn.t e5ist. +his creates problems with gaps in the timeline. When this happens* there is an alternate image location the program attempts to )se. If the image is still not available* then the image from the previo)s r)n is )sed. 7nless yo) are looking for it* yo) )s)ally can.t tell. +hen yo) have the iss)e of )pdating 3" storage and Mongo;/ for each image. +his again needs to happen 2)ickly to prevent image overlays from being o)t of sync. #t the end of the day* yo) want to do yo)r best to make s)re that the images for all the radar stations are in sync. +his will provide the best )ser e5perience. 1adar image processing happens in three stages and r)ns on a =M at ;igital 'cean with 2 9ig of memory and 2 Cores. 3o how does 9o help make this all happen every G min)tes all day and all night witho)t failD

Stage ! Do"nload Images


+here are ?GG images that have to be downloaded. +he images range from ?k to 20k in si(e depending on the activity of the weather at that moment. In this stage* the program spawns a 9o ro)tine for each image that needs to be downloaded. +he 9o program consistently does all ?GG downloads in less than one secondB

28<

0F(0F(1E ra)ar.go(JEF( main ( )ownloa):mages ( /tarte) KK /pawn 2o -outines 0F(0F(1E ra)ar.go(JI0( main ( )ownloa):mages ( :nfo ( :mage U0V of U0FFV 0F(0F(1E ra)ar.go(JI0( main ( )ownloa):mages ( :nfo ( :mage U0FFV of U0FFV KK /ample &ownloa) #er :mage WorkerK9.4 ( )ownloa):mage ( N1-/9.4WN1-W1.gif WorkerK9.4 ( )ownloa):mage ( UMri+ 1S &ec E10I E1(0EV WorkerK9.4 ( )ownloa):mage ( Uimage/gifV WorkerK9.4 ( )ownloa):mage ( @nco)ingV WorkerK9.4 ( )ownloa):mage ( UmaBKage7081V WorkerK9.4 ( )ownloa):mage ( 1S &ec E10I E1(08(1E 2M V WorkerK9.4 ( )ownloa):mage ( &ec E10I E1(0F(1E 2M V WorkerK9.4 ( )ownloa):mage ( UkeepKaliveV WorkerK9.4 ( )ownloa):mage ( U.pache/E.E.0F (-e) Aat)V WorkerK9.4 ( )ownloa):mage ( UbytesV WorkerK9.4 ( )ownloa):mage ( K0 WorkerK9.4 ( )ownloa):mage ( S8RI :nfo ( A@.&@- ( 6rl + :nfo ( A@.&@- ( LastKMo)ifie) + :nfo ( A@.&@- ( 5ontentK ype + :nfo ( A@.&@- ( Pary + U.cceptK :nfo ( A@.&@- ( 5acheK5ontrol + :nfo ( A@.&@- ( @Bpires + UMri+ :nfo ( A@.&@- ( &ate + UMri+ 1S :nfo ( A@.&@- ( 5onnection + :nfo ( A@.&@- ( /erver + :nfo ( A@.&@- ( .cceptK-anges + :nfo ( A@.&@- ( 5ontentKLength + :nfo ( A@.&@- ( :mageKLength +

KK .ll :mages 5omplete 0F(0F(1E ra)ar.go(JJF( main ( )ownloa):mages ( 5omplete)

Stage #! Image $lean'p


&ow that all ?GG images have been downloaded* they need to be cleaned )sing the ImageMagick #$I. 7nfort)nately* this can only be done with a single 9o ro)tine. +rying to clean more than one image at a time ca)ses the program to p)ll a lot of memory. It really slows things down and can ca)se the program to be terminated by the '3. I have also seen other very odd behavior. +he program will consistently complete this work in <0 seconds or lessB 0F(0F(1E ra)ar.go(JFI( main ( clean:mages ( /tarte) 0F(0F(1E ra)ar.go(JFR( main ( clean:mages ( :nfo ( :mage U0V of U0FFV KK /ample #rocessing #er :mage

2G0

WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W WorkerK-:W

( ( ( ( ( ( ( ( ( ( ( ( ( (

clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage clean:mage

( ( ( ( ( ( ( ( ( ( ( ( ( (

/tarte) :nfo ( -ea):mage!lob :nfo ( ransparent#aint:mage :nfo ( Wave:mage :nfo ( 5rop :nfo ( -esiNe :nfo ( @\ualiNe:mage :nfo ( 2aussian!lur:mage :nfo ( !rightness5ontrast:mage :nfo ( -eset:terator :nfo ( 2et:mage!lob 5omplete) :nfo ( &efer ( #iBelWan) &estroy :nfo ( &efer ( MagicWan) &estroy

KK .ll :mages 5omplete 0F(0S(E1 ra)ar.go(JRR( main ( clean:mages ( 5omplete)

Stage %! .pload To S% and .pdate 2ongoDB


+he last stage re2)ires )ploading all the cleaned images to 3" storage* removing the old images from 3" and then )pdating Mongo;/ with the new list of available image files. I am )sing the goam( package from 7b)nt) for accessing 3" and the mgo package from 9)stavo &eimeyer to access Mongo;/. O)st like when we download the images* this stage spawns a 9o ro)tine for each image that needs to be )ploaded and recorded. +he 9o program consistently performs this work in one secondB 0F(0S(E1 0F(0S(E1 of U0FFV 0F(0S(E1 of U0FFV 0F(0S(E1 of U0FFV ra)ar.go(J8F( main ( up)ate:mages ( /tarte) ra)ar.go(JQ0( main ( up)ate:mages ( :nfo ( :mage U0V ra)ar.go(JQ0( main ( up)ate:mages ( :nfo ( :mage UEV ra)ar.go(JQ0( main ( up)ate:mages ( :nfo ( :mage UIV

KK /ample #rocessing #er :mage collate:mage ( /tarte) ( /tation:)U-:WV MileNameU6//W"/-:W/E10I0E1SKE10F.gifV collate:mage ( :nfo ( -emove ( MinutesUF1.10V MileNameU6//W"/-:W/E10I0E1SK0QIFV collate:mage ( :nfo ( ;eep ( MinutesUJF.11V MileNameU6//W"/-:W/E10I0E1SK0QJ1.gifV collate:mage ( :nfo ( ;eep ( MinutesUJ1.11V MileNameU6//W"/-:W/E10I0E1SK0QJF.gifV collate:mage ( :nfo ( ;eep ( MinutesUIF.11V MileNameU6//W"/-:W/E10I0E1SK0QJ1.gifV collate:mage ( :nfo ( ;eep ( MinutesUI1.10V MileNameU6//W"/-:W/E10I0E1SK0QJF.gifV collate:mage ( :nfo ( ;eep ( MinutesUEF.1EV

2G?

MileNameU6//W"/-:W/E10I0E1SK0QF1.gifV collate:mage ( :nfo ( ;eep ( MinutesUE1.10V MileNameU6//W"/-:W/E10I0E1SK0QFF.gifV collate:mage ( :nfo ( ;eep ( MinutesU0F.10V MileNameU6//W"/-:W/E10I0E1SKE111.gifV collate:mage ( :nfo ( ;eep ( MinutesU01.11V MileNameU6//W"/-:W/E10I0E1SKE11F.gifV collate:mage ( :nfo ( ;eep ( MinutesUF.10V MileNameU6//W"/-:W/E10I0E1SKE101.gifV collate:mage ( :nfo ( ;eep K New ( MileNameU6//W"/-:W/E10I0E1SKE10F.gifV collate:mage ( 5omplete) store:mageMongo&! ( :nfo ( 6p)ating Mongo store:mageMongo&! ( 5omplete) store:mage/I ( /tarte) ( !ucketU/--K&evV MileNameU6//.;/.#&/E10I0E1SKE10F.gifV store:mage/I ( :nfo ( #utting Mile :nto /I ( MileNameU6//.;/.#&/E10I0E1SKE10F.gifV store:mageMongo&! ( 5omplete) KK .ll :mages 5omplete 0F(0S(E0 ra)ar.go(F1F( main ( up)ate:mages ( 5omplete)

$oncl'sion
+his pro0ect has ta)ght me a lot abo)t 9o. It is e5citing to see how fast the 9o ro)tines can download the images and perform all the 3" and Mongo;/ work. +hanks to C9'* I was able to leverage a powerf)l image processing library and make calls directly from my 9o code. C)rrently we are porting the web service that powers the mobile application to 9o. It is c)rrently written in 1)by. We are )sing the beego package for o)r web framework* the goconvey package for o)r tests and the envconfig package to handle o)r config)ration needs. ')r goal for ')tCast is to provide people the ability to know in advance that the weekend is going to be great. We plan on )sing 9o and Mongo;/ to analy(e o)tdoor condition data with )ser preferences and e5periences to deliver relevant information and forecasting. In the f)t)re* )sers will interact with ')tCast by providing an e5perience review after their o)tdoor activities have ended. C)rrently ')tCast is only available in the #pple #pp 3tore. We will have o)r #ndroid version complete in Oan)ary 20?8.

Sample Web )pplication .sing Beego and 2go

Introd'ction
I am very e5cited abo)t the /eego web framework. I wanted to share with yo) how I )se the framework to b)ild real world web sites and web services. Eere is a pict)re of the sample website the post is going to showcaseB

2G2

+he sample web applicationB ?. Implements a traditional grid view of data calling into Mongo;/ 2. $rovides a modal dialog bo5 to view details )sing a partial view to generate the E+M". Implements a web service that ret)rns a O3'& doc)ment 8. +akes config)ration parameters from the environment )sing envconfig G. Implements tests via goconvey H. -everages my logging package +he code for the sample can be fo)nd in the 9oing9o repository )p on 9ith)bB httpsB gith)b.com goinggo beegoCmgo :o) can bring the code down and r)n it. It )ses a p)blic Mongo;/ database I created at Mongo-ab. :o) will need git and ba(aar installed on yo)r system before r)nning go get. go get github.com/goinggo/beegoKmgo +o 2)ickly r)n or test the web application* )se the scripts located in the (scripts folder.

Web )pplication $ode Str'ct're


-et.s take a look at the pro0ect str)ct)re and the different folders that e5istB controllers locali(e models ro)tes services static test )tilities views (scripts 6ntry point for each Web call. Controllers process the re2)ests. $rovides locali(ation s)pport for different lang)ages and c)lt)res Models are data str)ct)res )sed by the b)siness and service layers Mappings between 71-.s and the controller code that handles those calls. 3ervices provide primitive f)nctions for the different services that e5ist. +hese co)ld be database or web calls that perform a specific f)nction. 1eso)rce files s)ch as scripts* stylesheets and images +ests that can be r)n thro)gh the go test tool. Code that s)pports the web application. /oilerplate and abstraction layers for accessing the database and handling panics. Code related to rendering views 3)pport scripts to help make it easier to b)ild* r)n and test the web application

2G"

$ontrollers, 2odels and Ser:ices


+hese layers make )p the b)lk of the code that implement the web application. +he idea behind the framework is to hide and abstract as m)ch boilerplate code as possible. +his is accomplished by implementing a base controller package and a base services package.

Base $ontroller Package


+he base controller package )ses composition to abstract defa)lt controller behavior re2)ired by all controllersB type ( !ase5ontroller struct { beego.5ontroller services./ervice % ) func (this $!ase5ontroller) #repare() { this.6ser:) 7 this.2et/tring("user:)") if this.6ser:) 77 "" { this.6ser:) 7 this.2et/tring("(user:)") % err (7 this./ervice.#repare() if err ?7 nil { this./erve@rror(err) return % % func (this $!ase5ontroller) Minish() { )efer func() { if this.Mongo/ession ?7 nil { mongo.5lose/ession(this.6ser:)+ this.Mongo/ession) this.Mongo/ession 7 nil % %() % # new type called /aseController is declared with the /eego Controller type and the base 3ervice type embedded directly. +his composes the fields and methods of these types directly into the /aseController type and makes them directly accessible thro)gh an ob0ect of the /aseController type. /eego Controller framework will e5ec)te the $repare and Finish f)nctions on any Controller ob0ect that implements these interfaces. +he $repare f)nction is e5ec)ted prior to the Controller f)nction being called. +hese f)nctions will belong to every Controller type by defa)lt* allowing this boilerplate code to be implemented once.

Ser:ices Package
2G8

+he 3ervice package maintains state and implements boilerplate code re2)ired by all servicesB type ( // /ervices contains common properties /ervice struct { Mongo/ession $mgo./ession 6ser:) string % ) func (this $/ervice) #repare() (err error) { this.Mongo/ession+ err 7 mongo.5opyMonotonic/ession(this.6ser:)) if err ?7 nil { return err % return err % func (this $/ervice) Minish() (err error) { )efer helper.5atch#anic(Gerr+ this.6ser:)+ "/ervice.Minish") if this.Mongo/ession ?7 nil { mongo.5lose/ession(this.6ser:)+ this.Mongo/ession) this.Mongo/ession 7 nil % return err % func (this $/ervice) &!.ction()atabaseName string+ collectionName string+ mongo5all mongo.Mongo5all) (err error) { return mongo.@Becute(this.6ser:)+ this.Mongo/ession+ )atabaseName+ collectionName+ mongo5all) % In the 3ervice type* the Mongo session and the id of the )ser is maintained. +his version of $repare handles creating a Mongo;/ session for )se. Finish closes the session which releases the )nderlying connection back into the pool. +he f)nction ;/#ction provides an abstraction layer for r)nning Mongo;/ commands and 2)eries.

B'oy Ser:ice
+his /)oy 3ervice package implements the calls to Mongo;/. -et.s look at the Find3tation f)nction that is called by the controller methodsB func Min)/tation(service $services./ervice+ station:) string) (buoy/tation $buoyMo)els.!uoy/tation+ err error) { 2GG

)efer helper.5atch#anic(Gerr+ service.6ser:)+ "Min)/tation") \ueryMap (7 bson.M{"stationWi)"( station:)% buoy/tation 7 GbuoyMo)els.!uoy/tation{% err 7 service.&!.ction(5onfig.&atabase+ "buoyWstations"+ func(collection $mgo.5ollection) error { return collection.Min)(\ueryMap).3ne(buoy/tation) %) if err ?7 nil { if strings.5ontains(err.@rror()+ "not foun)") 77 false { return buoy/tation+ err % err 7 nil % return buoy/tation+ err % +he Find3tation f)nction prepares the 2)ery and then )sing the ;/#ction f)nction to e5ec)te the 2)ery against Mongo;/.

Implementing Web $alls


With the base types* boilerplate code and service f)nctionality in place* we can now implement the web calls.

B'oy $ontroller
+he /)oyController type is composed solely from the /aseController. /y composing the /)oyController in this way* it immediately satisfies the $repare and Finish interfaces and contains all the fields of a /eego Controller. +he controller f)nctions are bo)nd to ro)tes. +he ro)tes specify the )rls to the different web calls that the application s)pports. In o)r sample application we have three ro)tesB beego.-outer("/"+ Gcontrollers.!uoy5ontroller{%+ "get(:n)eB") beego.-outer("/buoy/retrievestation"+ Gcontrollers.!uoy5ontroller{%+ "post(-etrieve/tation") beego.-outer("/buoy/station/(station:)"+ Gcontrollers.!uoy5ontroller{%+ "get+post(-etrieve/tation9son") +he ro)te specifies a )rl path* an instance of the controller )sed to handle the call and the name of the method from the controller to )se. # prefi5 of which verb is accepted can be specified as well. +he Inde5 controller method is )sed to deliver the initial html to the browser. +his will incl)de the 0avascript* style sheets and anything else needed to get the web application goingB 2GH

func (this $!uoy5ontroller) :n)eB() { region (7 "2ulf 3f MeBico" buoy/tations+ err (7 buoy/ervice.Min)-egion(Gthis./ervice+ region) if err ?7 nil { this./erve@rror(err) return % this.&ataU"/tations"V 7 buoy/tations this.Layout 7 "share)/basicKlayout.html" this. plNames 7 "buoy/content.html" this.Layout/ections 7 mapUstringVstring{% this.Layout/ectionsU"#ageAea)"V 7 "buoy/pageKhea).html" this.Layout/ectionsU"Aea)er"V 7 "share)/hea)er.html" this.Layout/ectionsU"Mo)al"V 7 "share)/mo)al.html" % # call is made into the service layer to retrieve the list of regions. +hen the slice of stations are passed into the view system. 3ince this is setting )p the initial view of the application* layo)ts and the template are specified. When the controller method ret)rns* the beego framework will generate the html for the response and deliver it to the browser.

+o generate that grid of stations* we need to be able to iterate over the slice of stations. 9o templates s)pport iterating over a slice. Eere we )se the .3tations variable which was passed into the view systemB {{range ]in)eB+ ]val (7 ./tations%% =trH =t)H=a class7")etail" )ata7"{{]val./tation:)%%" href7"O"H{{]val./tation:)%%=/aH=/t)H =t)H{{]val.Name%%=/t)H =t)H{{]val.Loc&esc%%=/t)H =t)H{{]val.5on)ition.&isplayWin)/pee)%%=/t)H =t)H{{]val.5on)ition.Win)&irection%%=/t)H =t)H{{]val.5on)ition.&isplayWin)2ust%%=/t)H =/trH {{en)%% 6ach station id is a link that brings )p a modal dialog bo5 with the details for each station. +he 1etrieve3tation controller method generates the html for the modal dialogB

2GI

func (this $!uoy5ontroller) -etrieve/tation() { params (7 struct { /tation:) string Xform("station:)" vali)("-e\uire)< Min/iNe(J)" error("invali)WstationWi)"X %{% if this.#arse.n)Pali)ate(Gparams) 77 false { return % buoy/tation+ err (7 buoy/ervice.Min)/tation(Gthis./ervice+ params./tation:)) if err ?7 nil { this./erve@rror(err) return % this.&ataU"/tation"V 7 buoy/tation this.Layout 7 "" this. plNames 7 "buoy/pvWstation.html" view+ W (7 this.-en)er/tring() this..DaB-esponse(1+ "/655@//"+ view) % 1etrieve3tation gets the details for the specified station and then )ses the view system to generate the html for the dialog bo5. +he partial view is passed back to the re2)esting a0a5 call and placed into the browser doc)mentB function /how&etail(result) { try { var post&ata 7 {%< post&ataU"station:)"V 7 ](result).attr(C)ataC)< var service 7 new /ervice-esult()< service.get9/3N&ata("/buoy/retrievestation"+ post&ata+ /how&etailW5allback+ /tan)ar)WPali)ation5allback+ /tan)ar)W@rror5allback )< % catch (e) { alert(e)< % % function /how&etailW5allback() { try { ](COsystemKmo)alKtitleC).html("!uoy &etails")< 2GL

](COsystemKmo)alKcontentC).html(this.-esult3bDect)< ]("OsystemMo)al").mo)al(CshowC)< % catch (e) { alert(e)< % % 'nce the call to modal.@.show.A is performed* the following modal diaglog appears.

+he 1etrieve3tationOson f)nction implements a web service call that ret)rns a O3'& doc)mentB func (this $!uoy5ontroller) -etrieve/tation9son() { params (7 struct { /tation:) string Xform("(station:)" vali)("-e\uire)< Min/iNe(J)" error("invali)WstationWi)"X %{% if this.#arse.n)Pali)ate(Gparams) 77 false { return % buoy/tation+ err (7 buoy/ervice.Min)/tation(Gthis./ervice+ params./tation:)) if err ?7 nil { this./erve@rror(err) return % 2G<

this.&ataU"Dson"V 7 Gbuoy/tation this./erve9son() % :o) can see how it calls into the service layer and )ses the O3'& s)pport to ret)rn the response.

Testing The Endpoint


In order to make s)re the application is always working* it needs to have testsB func est/tation(t $testing. ) { r+ W (7 http.New-e\uest("2@ "+ "/station/JE11E"+ nil) w (7 httptest.New-ecor)er() beego.!ee.pp.Aan)lers./erveA #(w+ r) response (7 struct { /tation:) string XDson("stationWi)"X Name string XDson("name"X Loc&esc string XDson("locationW)esc"X 5on)ition struct { ype string XDson("type"X 5oor)inates UVfloatSJ XDson("coor)inates"X % XDson("con)ition"X Location struct { Win)/pee) floatSJ XDson("win)Wspee)Wmilehour"X Win)&irection int XDson("win)W)irectionW)egnorth"X Win)2ust floatSJ XDson("gustWwin)Wspee)Wmilehour"X % XDson("location"X %{% Dson.6nmarshal(w.!o)y.!ytes()+ Gresponse) 5onvey("/ubDect( est /tation @n)point*n"+ t+ func() { 5onvey("/tatus 5o)e /houl) !e E11"+ func() { /o(w.5o)e+ /houl)@\ual+ E11) %) 5onvey(" he -esult /houl) Not !e @mpty"+ func() { /o(w.!o)y.Len()+ /houl)!e2reater han+ 1) %) 5onvey(" here /houl) !e . -esult Mor /tation JE11E"+ func() { /o(response./tation:)+ /houl)@\ual+ "JE11E") %) %) % +his test creates a fake call thro)gh the /eego handler for the specified ro)te. +his is awesome beca)se we don.t need to r)n the web application to test. /y )sing goconvey we can create tests that prod)ce nice o)tp)t that is logical and easy to read.

2H0

Eere is a sample when the test failsB /ubDect( est /tation @n)point

/tatus 5o)e /houl) !e E11 he -esult /houl) Not !e @mpty here /houl) !e . -esult Mor /tation JE11E Mailures( $ /6sers/bill//paces/2o/#roDects/src/github.com/goinggo/beegoK mgo/test/en)points/buoy@n)pointsWtest.go Line IF( @Bpecte)( CE11C .ctual( CJ11C (/houl) be e\ual) $ /6sers/bill//paces/2o/#roDects/src/github.com/goinggo/beegoK mgo/test/en)points/buoy@n)pointsWtest.go Line IR( @Bpecte)( C1C .ctual( CQC (/houl) be e\ual) I assertions thus far KKK M.:L( est/tationK8 (1.1I secon)s)

Eere is a sample when it is s)ccessf)lB /ubDect( est /tation @n)point

/tatus 5o)e /houl) !e E11 he -esult /houl) Not !e @mpty here /houl) !e . -esult Mor /tation JE11E I assertions thus far KKK #.//( est/tationK8 (1.1F secon)s)

$oncl'sion
+ake the time to download the pro0ect and look aro)nd. I have attempted to show yo) the ma0or points of the sample and how things are p)t together. +he /eego framework makes it easy to implement yo)r own ways to abstract and implement boilerplate code* leverage the go testing harness and r)n and deploy the code )sing 9o standard mechanisms.

Three<Inde@ Slices in Go 5#

2H?

With the release of 9o ?.2* slices gained the ability to specify the capacity when performing a slicing operation. +his doesn.t mean we can )se this inde5 to e5tend the capacity of the )nderlying array. It means we can create a new slice whose capacity is restricted. 1estricting the capacity provides a level of protection to the )nderlying array and gives )s more control over append operations. Eere are the release notes and design doc)ment for the feat)re re2)estB httpB tip.golang.org doc go?.2%threeFinde5 httpsB docs.google.com doc)ment d ?9KKdi9:#ghN15C2/Fr36bE/PPg#9K4CyNKC h1K/o0Kk p)b -et.s write some code to e5plore )sing the new capacity inde5. #s with all my slice posts* I am going to )se this Inspect3lice f)nctionB func :nspect/lice(slice UVstring) { // 5apture the a))ress to the slice structure a))ress (7 unsafe.#ointer(Gslice) // 5apture the a))ress where the length an) cap siNe is store) len.))r (7 uintptr(a))ress) > uintptr(8) cap.))r (7 uintptr(a))ress) > uintptr(0S) // 5reate pointers to the length an) cap siNe len#tr (7 ($int)(unsafe.#ointer(len.))r)) cap#tr (7 ($int)(unsafe.#ointer(cap.))r)) // 5reate a pointer to the un)erlying array a))#tr (7 ($U8Vstring)(unsafe.#ointer($($uintptr) (a))ress))) fmt.#rintf("/lice .))rU'pV Len .))rU1B'BV 5ap .))rU1B 'BV*n"+ a))ress+ len.))r+ cap.))r) fmt.#rintf("/lice LengthU')V 5apU')V*n"+ $len#tr+ $cap#tr) for in)eB (7 1< in)eB = $len#tr< in)eB>> { fmt.#rintf("U')V 'p 's*n"+ in)eB+ G($a))#tr)Uin)eBV+ ($a))#tr)Uin)eBV) % fmt.#rintf("*n*n") % 2H2

+o start* let.s create a slice we will )se as o)r so)rceB source (7 UVstring{".pple"+ "3range"+ "#lum"+ "!anana"+ "2rape"% :nspect/lice(source) 3utput( /lice .))rU1BE01EI0111V Len .))rU1BE01EI0118V 5ap .))rU1BE01EI0101V /lice LengthUFV 5apUFV U1V 1BE01E1e0J1 .pple U0V 1BE01E1e0F1 3range ;2< "#21"2"e1>" P(+m UIV 1BE01E1e0R1 !anana UJV 1BE01E1e081 2rape We start with a slice of strings with a length and capacity of G. +his means the )nderlying array has G elements and we have access to the entire array. &e5t* let.s take a traditional slice of the source and inspect the contentsB take3ne (7 sourceUE(IV :nspect/lice(take3ne) 3utput( /lice .))rU1BE01EI01J1V Len .))rU1BE01EI01J8V 5ap .))rU1BE01EI01F1V /lice LengthU0V 5apUIV ;"< "#21"2"e1>" P(+m With this slice operation we only take the third element from the source. :o) can see the first element of the takeCne slice has the same address as the third element of the source slice. +he takeCne slice has a length of one and a capacity of three. +his is beca)se there are three elements left in the )nderlying array that are available for )se. What if we didn.t want the new slice to have access to the remaining capacityD $rior to version ?.2* this was not possible. -et.s take the slice again* b)t this time restrict the capacity to oneB take3ne5ap3ne (7 sourceUE(I(IV position to :nspect/lice(take3ne5ap3ne) // 6se the thir) in)eB // set the capacity

3utput( /lice .))rU1BE01EI01S1V Len .))rU1BE01EI01S8V 5ap .))rU1BE01EI01R1V

2H"

/lice LengthU0V 5apU0V U1V 1BE01E1e0S1 #lum #fter creating the takeCne%apCne slice* the length and capacity are now one. +he takeCne%apCne slice no longer has access to the remaining capacity in the )nderlying array. -ength and capacity is calc)lated )sing this form)laB Mor sliceU i ( D ( k V the Length( D K i 5apacity( k K i If we attempt to set the capacity greater than the )nderlying array* the code will panic. take3ne5apMour (7 sourceUE(I(SV the capacity what is // available. -untime @rror( panic( runtime error( slice boun)s out of range goroutine 0 UrunningV( runtime.panic(1BQa)E1+ 1B0SJQea) /6sers/bill/go/src/pkg/runtime/panic.c(ESS >1BbS main.main() /6sers/bill//paces/ est/src/test/main.go(0F >1BEJf 3o what happens if we append an element to the takeCne%apCne sliceD source (7 UVstring{".pple"+ "3range"+ "#lum"+ "!anana"+ "2rape"% :nspect/lice(source) take3ne5ap3ne (7 sourceUE(I(IV :nspect/lice(take3ne5ap3ne) take3ne5ap3ne 7 appen)(take3ne5ap3ne+ ";iwi") :nspect/lice(take3ne5ap3ne) Eere is the o)tp)tB /lice .))rU1BE01EI0111V Len .))rU1BE01EI0118V 5ap .))rU1BE01EI0101V /lice LengthUFV 5apUFV U1V 1BE01E1e0J1 .pple U0V 1BE01E1e0F1 3range ;2< "#21"2"e1>" P(+m // (S K E) attempts to set // to J. his is greater than

2H8

UIV 1BE01E1e0R1 !anana UJV 1BE01E1e081 2rape KK !efore .ppen) KK /lice .))rU1BE01EI01J1V Len .))rU1BE01EI01J8V 5ap .))rU1BE01EI01F1V /lice LengthU0V 5apU0V ;"< "#21"2"e1>" P(+m KK .fter .ppen) KK /lice .))rU1BE01EI0181V Len .))rU1BE01EI0188V 5ap .))rU1BE01EI01Q1V /lice LengthUEV 5apUEV ;"< "#21"2%1">" P(+m U0V 1BE01EI01R1 ;iwi When we append an element to the takeCne%apCne slice* a new )nderlying array is created for the slice. +his new )nderlying array contains a copy of the elements being referenced from the source and then is e5tended to add the new element. +his is beca)se the capacity of the takeCne%apCne slice was reached and append needed to grow the capacity. &otice how the address changes in the takeCne%apCne slice after the append. Eow is this different from not setting the capacityD source (7 UVstring{".pple"+ "3range"+ "#lum"+ "!anana"+ "2rape"% :nspect/lice(source) take5ne := ,o+rce;2:%< :nspect/lice(take3ne) :: !onYt ,pecif= capacit=

take3ne 7 appen)(take3ne+ ";iwi") :nspect/lice(take3ne) :nspect/lice(source) Eere is the o)tp)tB /lice .))rU1BE01EI0111V Len .))rU1BE01EI0118V 5ap .))rU1BE01EI0101V /lice LengthUFV 5apUFV U1V 1BE01E1e0J1 .pple U0V 1BE01E1e0F1 3range ;2< "#21"2"e1>" P(+m ;%< "#21"2"e17" Ianana UJV 1BE01E1e081 2rape KK !efore .ppen) KK

2HG

/lice .))rU1BE01EI01J1V Len .))rU1BE01EI01J8V 5ap .))rU1BE01EI01F1V /lice LengthU0V 5apUIV ;"< "#21"2"e1>" P(+m KK .fter .ppen) KK /lice .))rU1BE01EI01S1V Len .))rU1BE01EI01S8V 5ap .))rU1BE01EI01R1V /lice LengthUEV 5apUIV ;"< "#21"2"e1>" P(+m ;1< "#21"2"e17" Ri.i /lice .))rU1BE01EI0181V Len .))rU1BE01EI0188V 5ap .))rU1BE01EI01Q1V /lice LengthUFV 5apUFV U1V 1BE01E1e0J1 .pple U0V 1BE01E1e0F1 3range ;2< "#21"2"e1>" P(+m ;%< "#21"2"e17" Ri.i UJV 1BE01E1e081 2rape +his time the append )ses the e5isting capacity and overwrites the val)e at element 8 in the )nderlying array. +his co)ld be a disaster if this was not o)r intent. +his new feat)re of setting the capacity can really help protect )s and o)r data from )nwanted overwrites. +he more we can leverage the b)iltCin f)nctions and r)ntime to handle these types of operations the better. +hese types of b)gs are very diffic)lt to find so this is going to help immeas)rably. Eere are other posts abo)t slicesB 7nderstanding 3lices In 9o $rogramming Collections 'f 7nknown -ength In 9o 3lices 'f 3lices 'f 3lices In 9o Iterating 'ver 3lices In 9o

J'e'e &o'r Way To Scalability

Introd'ction
+he first thing I did when I started programming in 9o was begin porting my Windows )tilities classes and service frameworks over to -in)5. +his is what I did when I moved from C!! to C%. +hank goodness* I soon learned abo)t Iron.I' and the services they offered. +hen it hit me* if I wanted tr)e scalability* I needed to start b)ilding worker tasks that co)ld be 2)e)ed to r)n anywhere at any time. It was not abo)t how many machines I needed* it was abo)t how m)ch comp)te time I needed.

2HH

')tcast Marine Forecast +he freedom that comes with architecting a sol)tion aro)nd web services and worker tasks is refreshing. If I need ?*000 instances of a task to r)n* I can 0)st 2)e)e it )p. I don.t need to worry abo)t capacity* reso)rces* or any other I+ related iss)es. If my service becomes an instant hit overnight* the architect)re is ready* the capacity is available. My mobile weather application ')tcast is a prime e5ample. I c)rrently have a single sched)led task that r)ns in Iron.I' every ?0 min)tes. +his task )pdates marine forecast areas for the 7nited 3tates and downloads and parses 8I2 web pages from the &'## website. We are abo)t to add Canada and event)ally we want to move into 6)rope and #)stralia. #t that point a single sched)led task is not a scalable or red)ndant architect)re for this process. +hanks to the 9o Client from Iron.I'* I can b)ild a task that wakes )p on a sched)le and 2)e)es )p as many marine forecast area worker tasks as needed. I can )se this architect)re to process each marine forecast area independently* in their own worker task* providing incredible scalability and red)ndancy. +he best part* I don.t have to think abo)t hardware or I+ related capacity iss)es.

$reate a Worker Task


/ack in 3eptember I wrote a post abo)t b)ilding and )ploading an Iron.I' worker task )sing 9oB httpB www.goinggo.net 20?" 0< r)nningCgoCprogramsCinCironworker.html +his task sim)lated H0 seconds of work and ran e5periments to )nderstand some of the capabilities of the worker task container. We are going to )se this worker task to demonstrate how to )se the 9o Client to 2)e)e a task. If yo) want to follow along* go ahead and walk thro)gh the post and create the worker task. I am going to ass)me yo) walked thro)gh the post and created the worker called KtaskK as depicted in the image belowB

2HI

Do"nload The Go $lient


;ownload the 9o Client from Iron.I'B go get github.com/ironKio/ironWgo &ow navigate to the e5amples folderB

+he e5amples leverage the #$I that can be fo)nd hereB httpB dev.iron.io worker reference api &ot all the #$I calls are represented in these e5amples* b)t from these e5amples the rest of the #$I can be easily implemented. In this post we are going to foc)s on the task #$I calls. +hese are #$I.s that yo) will most likely be able to leverage in yo)r own programs and architect)res.

J'e'e a Task
'pen )p the 2)e)e e5ample from the e5amples tasks folder. We will walk thro)gh the more important aspects of the code. In order to 2)e)e a task with the 9o client* we need to create this doc)ment which will be posted with the re2)estB { "tasks"( U { "co)eWname"( "MyWorker"+ "timeout" ( S1+

2HL

"payloa)"( "{*"B*"( *"abc*"+ *"y*"( *")ef*"%" % V % In the case of o)r worker task* the payload doc)ment in 9o sho)ld look like thisB var payloa) 7 { "co)eWname" "timeout" ( "payloa)" ( %V%X X{"tasks"(U ( "ta,k"+ 12"+ ""

&ow let.s look at the code that will re2)est o)r task to be 2)e)ed. +he first thing we need to do is set o)r pro0ect id and token. config (7 config.5onfig("ironWworker") config.#roDect:) 7 "yourWproDectWi)" config. oken 7 "yourWtoken" #s described in the post from 3eptember* this information can be fo)nd inside o)r pro0ect config)rationB

&ow we can )se the 9o Client to b)ild the )rl and prepare the payload for the re2)estB url (7 api..ction@n)point(config+ "tasks") post&ata (7 bytes.New!uffer/tring(payloa))

2H<

7sing the )rl ob0ect* we can send the re2)est to Iron.I' and capt)re the responseB resp+ err (7 url.-e\uest("#3/ "+ post&ata) )efer resp.!o)y.5lose() if err ?7 nil { log.#rintln(err) return % bo)y+ err (7 ioutil.-ea).ll(resp.!o)y) if err ?7 nil { log.#rintln(err) return % We want to check the response to make s)re everything was s)ccessf)l. +his is the response we will get backB { "msg"( ",ueue) up"+ "tasks"( U { "i)"( "Jeb0bJR0c))b0IS1SF111101" % V % +o )nmarshal the res)lt* we need these data str)ct)resB type ( ask-esponse struct { Message string XDson("msg"X asks UV ask XDson("tasks"X % ask struct { :) string XDson("i)"X % ) &ow let.s )nmarshal the res)ltsB task-esponse (7 G ask-esponse{% err 7 Dson.6nmarshal(bo)y+ task-esponse) if err ?7 nil { log.#rintf("'v*n"+ err) return % If we want to )se a map instead to red)ce the code base* we can do thisB

2I0

results (7 mapUstringVinterface{%{% err 7 Dson.6nmarshal(bo)y+ Gresults) if err ?7 nil { log.#rintf("'v*n"+ err) return % When we r)n the e5ample code and everything works* we sho)ld see the following o)tp)tB 6rl( https(//workerKawsKusKeastK 0.iron.io(JJI/E/proDects/FEEbJcF08a1cQS111Q11111R/tasks "msg"( ,ueue) up { "i)"( "FEbJRE0RES)QJ01EQS10Ecc8"+ %+ If we navigate to the Iron.I' E7;* we sho)ld see the task was 2)e)ed and completed s)ccessf)llyB

$oncl'sion
+he 9o client is doing a lot of the boilerplate work for )s behind the scenes. We 0)st need to make s)re we have all the config)ration parameters that are re2)ired. 4)e)ing a task is one of the more complicated #$I calls. -ook at the other e5amples to see how to get information for the tasks we 2)e)e and even get the logs. 4)e)ing a task like this gives yo) the fle5ibility to sched)le work on specific intervals or based on events. +here are a lot of )se cases where different types of web re2)ests co)ld leverage 2)e)ing a task. -everaging this type of architect)re provides a nice separation of concerns with scalability and red)ndancy b)ilt in. It keeps o)r web applications foc)sed and optimi(ed for handling )ser re2)ests and p)shes the asynchrono)s and backgro)nd tasks to a clo)d environment designed and architected to handle things at scale. #s ')tcast grows we will contin)e to leverage all the services that Iron.I' and the clo)d has to offer. +here is a lot of data that needs to be downloaded* processing and then delivered to )sers thro)gh the mobile application. /y b)ilding a scalable architect)re today* we can handle what happens tomorrow.

2acro 9ie" o6 2ap Internals In Go

Introd'ction
+here are lots of posts that talk abo)t the internals of slices* b)t when it comes to maps* we are left in the dark. I was wondering why and then I fo)nd the code for maps and it all made sense.

2I?

httpB golang.org src pkg r)ntime hashmap.c #t least for me* this code is complicated. +hat being said* I think we can create a macro view of how maps are str)ct)red and grow. +his sho)ld e5plain why they are )nordered* efficient and fast.

$reating and .sing 2aps


-et.s look at how we can )se a map literal to create a map and store a few val)esB // 5reate an empty map with a key an) value of type string colors (7 mapUstringVstring{% // .)) a few keys/value pairs to the map colorsU".lice!lue"V 7 "OM1M8MM" colorsU"5oral"V 7 "OMMRMF1" colorsU"&ark2ray"V 7 "O.Q.Q.Q" When we add val)es to a map* we always specify a key that is associated with the val)e. +his key is )sed to find this val)e again witho)t the need to iterate thro)gh the entire collectionB fmt.#rintf("Palue( 's"+ colorsU"5oral"V) If we do iterate thro)gh the map* we will not necessarily get the keys back in the same order. In fact* every time yo) r)n the code* the order co)ld changeB colors (7 mapUstringVstring{% colorsU".lice!lue"V 7 "OM1M8MM" colorsU"5oral"V 7 "OMMRMF1" colorsU"&ark2ray"V 7 "O.Q.Q.Q" colorsU"Morest2reen"V 7 "OEE8!EE" colorsU":n)igo"V 7 "OJ!118E" colorsU"Lime"V 7 "O11MM11" colorsU"Navy"V 7 "O111181" colorsU"3rchi)"V 7 "O&.R1&S" colorsU"/almon"V 7 "OM.81RE" for key+ value (7 range colors { fmt.#rintf("'s('s+ "+ key+ value) % 3utput( .lice!lue(OM1M8MM+ &ark2ray(O.Q.Q.Q+ :n)igo(OJ!118E+ 5oral(OMMRMF1+ Morest2reen(OEE8!EE+ Lime(O11MM11+ Navy(O111181+ 3rchi)(O&.R1&S+ /almon(OM.81RE &ow that we know how to create* set key val)e pairs and iterate over a map* we can peek )nder the hood.

2I2

*o" 2aps )re Str'ct'red


Maps in 9o are implemented as a hash table. If yo) need to learn what a hash table is* there are lots of articles and posts abo)t the s)b0ect. +his is the Wikipedia page to get yo) startedB httpB en.wikipedia.org wiki EashFtable +he hash table for a 9o map is str)ct)red as an array of b)ckets. +he n)mber of b)ckets is always e2)al to a power of 2. When a map operation is performed* s)ch as @colorsF,Black,G + ,KLLLLLL,=* a hash key is generated against the key that is specified. In this case the string K/lackK is )sed to generate the hash key. +he low order bits @-'/A of the generated hash key is )sed to select a b)cket.

'nce a b)cket is selected* the key val)e pair needs to be stored* removed or looked )p* depending on the type of operation. If we look inside any b)cket* we will find two data str)ct)res. First* there is an array with the top L high order bits @E'/A from the same hash key that was )sed to select the b)cket. +his array disting)ishes each individ)al key val)e pair stored in the respective b)cket. 3econd* there is an array of bytes that store the key val)e pairs. +he byte array packs all the keys and then all the val)es together for the respective b)cket.

2I"

When we are iterating thro)gh a map* the iterator walks thro)gh the array of b)ckets and then ret)rn the key val)e pairs in the order they are laid o)t in the byte array. +his is why maps are )nsorted collections. +he hash keys determines the walk order of the map beca)se they determine which b)ckets each key val)e pair will end )p in.

2emory and B'cket (:er6lo"


+here is a reason the key val)e pairs are packed like this in a single byte array. If the keys and val)es were stored like key val)e key val)e* padding allocations between each key val)e pair wo)ld be needed to maintain proper alignment bo)ndaries. #n e5ample where this wo)ld apply is with a map that looks like thisB mapUintSJVint8 +he ? byte val)e in this map wo)ld res)lt in I e5tra bytes of padding per key val)e pair. /y packing the key val)e pairs as key key val)e val)e* the padding only has to be appended to the end of the byte array and not in between. 6liminating the padding bytes saves the b)cket and the map a good amo)nt of memory. +o learn more abo)t alignment bo)ndaries* read this postB httpB www.goinggo.net 20?" 0I )nderstandingCtypeCinCgo.html # b)cket is config)red to store only L key val)e pairs. If a ninth key needs to be added to a b)cket that is f)ll* an overflow b)cket is created and reference from inside the respective b)cket.

2I8

*o" 2aps Gro"


#s we contin)e to add or remove key val)e pairs from the map* the efficiency of the map look)ps begin to deteriorate. +he load threshold val)es that determine when to grow the hash table are based on these fo)r factorsB ' overflow ( #ercentage of buckets which have an overflow bucket bytes/entry ( Number of overhea) bytes use) per key/value pair hitprobe ( Number of entries that nee) to be checke) when looking up a key missprobe ( Number of entries that nee) to be checke) when looking up an absent key C)rrently* the code )ses the following load threshold val)esB L()D H.G0 Mo:er6lo" 20.<0 bytes-entry ?0.I< hitprobe 8.2G missprobe H.G0

9rowing the hash table starts with assigning a pointer called the Kold b)cketK pointer to the c)rrent b)cket array. +hen a new b)cket array is allocated to hold twice the n)mber of e5isting b)ckets. +his co)ld res)lt in large allocations* b)t the memory is not initiali(ed so the allocation is fast. 'nce the memory for the new b)cket array is available* the key val)e pairs from the old b)cket array can be moved or Kevac)atedK to the new b)cket array. 6vac)ations happen as key val)e pairs are added or removed from the map. +he key val)e pairs that are together in an old b)cket co)ld be moved to different b)ckets inside the new b)cket array. +he evac)ation algorithm attempts to distrib)te the key val)e pairs evenly across the new b)cket array.

2IG

+his is a very delicate dance beca)se iterators still need to r)n thro)gh the old b)ckets )ntil every old b)cket has been evac)ated. +his also affects how key val)e pairs are ret)rned d)ring iteration operations. # lot of care has been taken to make s)re iterators work as the map grows and e5pands.

$oncl'sion
#s I stated in the beginning* this is 0)st a macro view of how maps are str)ct)red and grow. +he code is written in C and performs a lot of memory and pointer manip)lation to keep things fast* efficient and safe. 'bvio)sly* this implementation can be changed at any time and having this )nderstanding doesn.t affect o)r ability* one way or the other* to )se maps. It does show that if yo) know how many keys yo) need ahead of time* it is best to allocated that space d)ring initiali(ation. It also e5plains why maps are )nsorted collections and why iterators seem random when walking thro)gh maps.

Special Thanks
I wo)ld like to thank 3tephen Mc4)ay and Keith 1andall for their review* inp)t and corrections for the post.

Go Package 2anagement Cor #L 0

Introd'ction
In 'ctober 20?" I sent o)t a call to action to the 9o comm)nity. I wanted to form a gro)p of 9ophers that wo)ld come together and help write a specification and b)ild a working implementation of a package management tool. We are not there yet* b)t the gro)p did accomplish a few thingsB We started a mailing list called 9o package management SgoCpmT where people co)ld disc)ss ideas and get feedback on e5isting and new tools. We wrote a goals doc)ment that o)tlined what we wanted to accomplish along with g)idelines and help for package management tool a)thors. We identified and categori(ed the e5isting set of tools that are available for )se. We developed a set of )ser stories that describe the different scenarios people were facing. 2IH

We started a list of packages that tool developers can )se to test their tools against. We helped some of the tool developers find and fi5 b)gs.

'verall I think the last " months have been prod)ctive and I am pleased with the res)lts.

B'ilding Tools
I have come to appreciate that there isn.t going to be a silver b)llet or a one tool fits all sol)tion. I believe a ma0ority of the )seCcases that have been defined in the goals doc)ment can be solved and b)ilding tools to handle these )seCcases is worth the time and effort. If yo).re thinking abo)t b)ilding a tool* please consider these g)idelines which are o)tlined in the goals doc)mentB Work "ith the Go lang'age as de6ined in the Go spec o6 2arch #8, #L #5 ;on]t implement sol)tions that re2)ire feat)re changes or b)ild tools that change the way 9o works. Pro:ide back"ards compatibility "ith go get and the con:ention o6 'sing 9$S repositories in the import paths5 +he e5isting set of programs* b)ild processes and workflows can]t break. :o) m)st respect the e5isting environments and allow them to contin)e to f)nction. Pre:ent :ersion ske"ing5 ;on]t b)ild into the sol)tion the potential for version skewing to occ)r. 3)ch as re2)iring semantic versioning in the import path )rls. Imports sho)ld not need to be changed to access the latest or different versions of a package. Work on all operating systems and architect'res that Go c'rrently s'pports5 'ne of the great things abo)t 9o is that programs can be b)ilt on all these different operating systems and architect)res. :o)r tool sho)ld not e5cl)de a platform or make )se of a specific operating system constr)ct like symlinks. Interoperate in s'ch a "ay that Ngo getO contin'es to "ork 6or package a'thors "ho do not "ish to participate, or 6or e@isting Go so'rce code that cannot be changed5 )lso, do not 6orce package a'thors to choose bet"een making their code go getable or 'sing the proposed sol'tion5 &o one sho)ld be re2)ired to )se a tool in order to share or import a repository. +he tooling m)st contin)e to work for everyone. #ll these g)idelines are important beca)se they will allow others to try and )se yo)r tools witho)t the need to refactor any e5isting code. +hey also g)arantee that e5isting pro0ects contin)e to b)ild and install with the 9o tooling on all platforms.

$hoices
I wo)ld love to see the comm)nity rally aro)nd a few tools and help improve them. +here are two classes of tools that I think work well with 9o* =endoring and 1evision -ocking. 9endoring takes the "rd party so)rce code that is referenced in yo)r pro0ect and makes a copy of that code inside a new folder within the pro0ect. #ll the code yo)r pro0ect needs is inside the one pro0ect repository. =endoring also provides a performance enhancement on 2II

getting the code beca)se only one )rl call is re2)ired. >e:ision Locking creates a dependency file that references specific commits in the different version control systems the code is located in. O)st like vendoring* the 1- tool is )sed to get* b)ild and install yo)r pro0ect. 'ne advantage is that yo)r pro0ect repository contin)es to only contain the specific pro0ect code. Choosing how to handle package management it is a matter of personal preference. +he 9o team recommends vendoring* which can be fo)nd in the 9o F#4. +hey mention Keith 1arick.s tool goven as an option. Keith has abandonedZZ goven for his other tool godep* which provides both vendoring and revision locking. -- After tal.ing with /eith he has stated that he has not totally abandoned goven) it is 0ust 1finished1. %e &ontinues to maintain the pa&.age and merge bug fi2es when ne&essary.

?e" $all To )ction


For 20?8 I wo)ld like to see the 9o comm)nity play a greater role in helping the package management tool a)thors. +here are several ways I think this can be doneB $articipate in the goCpm gro)p. 9ive advice and help to those who have 2)estions. 3)bmit packages that the tool a)thors can )se to help test their tools. 1eport b)gs and feat)re re2)ests. Contin)e to add )ser stories that are missing in the goals doc)ment. 3)bmit yo)r new tools and ideas to the goCpm gro)p. Work with the e5isting tool a)thors to improve the tools that we have today. I hope to see all of yo) participating in the goCpm mailing list this year. I love 9o and only want to see it improve for everyone.

Decode JS(? Doc'ments In Go

Introd'ction
We are working on a pro0ect where we have to make calls into a web service. Many of the web calls ret)rn very large doc)ments that contain many s)bCdoc)ments. +he worst part is* we )s)ally only need a handf)l of the fields for any given doc)ment and those fields tend to be scattered all over the place. Eere is a sample of a smaller doc)mentB var )ocument string 7 X{ "user5onteBt"( { "conversation5re)entials"( { "session oken"( "1S0JE101W0(RFbfSaJ0IIER))R0ebe8fIfI1cFaJE01aQb00eQIc1E8)Se00a bfcaRff" %+ "vali)"( true+ "is#asswor)@Bpire)"( false+ "cobran):)"( 0111111J+ "channel:)"( K0+

2IL

"locale"( "enW6/"+ "tncPersion"( E+ "application:)"( "0R5!@EEE.JE0S0.IMMJF1@JR5MJ50.11"+ "cobran)5onversation5re)entials"( { "session oken"( "1S0JE101W0(b8)100fefbab8bf0RFIIQ0b1RJffe)fQFR8S0E)SRSe)EbRf1R IbFR8Fb" %+ "preference:nfo"( { "currency5o)e"( "6/&"+ "timeYone"( "#/ "+ ")ateMormat"( "MM/))/yyyy"+ "currencyNotation ype"( { "currencyNotation ype"( "/"M!3L" %+ "numberMormat"( { ")ecimal/eparator"( "."+ "grouping/eparator"( "+"+ "group#attern"( "OOO+OO1.OO" % % %+ "lastLogin ime"( 0IRFS8S8J0+ "login5ount"( EQQ+ "passwor)-ecovere)"( false+ "email.))ress"( "Dohn)oeTemail.com"+ "loginName"( "sptest0"+ "user:)"( 01J8I8S1+ "user ype"( { "user ype:)"( 0+ "user ypeName"( "normalWuser" % %X It is not scalable for )s to create all the str)cts and embedded str)cts to )nmarshal the different O3'& doc)ments )sing 0son.7nmarshal and working directly with a map was o)t of the 2)estion. What we needed was a way to decode these O3'& doc)ments into str)cts that 0)st contained the fields we needed* regardless where those fields lived in the O3'& doc)ment. -)ckily we came a across a package by Mitchell Eashimoto called mapstr)ct)re and we forked it. +his package is able to take a O3'& doc)ment that is already )nmarshaled into a map and decode that into a str)ct. 7nfort)nately* yo) still needed to create all the embedded str)cts if yo) wanted the data at the different levels. 3o I st)died the code and b)ild some f)nctionality on top that allowed )s do what we needed.

DecodePath
+he first f)nction we added is called ;ecode$ath. +his allows )s to specify the fields and s)bCdoc)ments we want from the O3'& doc)ment and store them into the str)cts we need. -et.s start with a small e5ample )sing the O3'& doc)ment aboveB

2I<

type 6ser ype struct { 6ser ype:) int 6ser ypeName string % type 6ser struct { /ession string XDpath("user5onteBt.cobran)5onversation5re)entials.session oke n"X 5obran):) int XDpath("user5onteBt.cobran):)"X 6ser ype 6ser ype XDpath("user ype"X LoginName string XDpath("loginName"X % )oc/cript (7 UVbyte()ocument) )ocMap (7 mapUstringVinterface{%{% Dson.6nmarshal()oc/cript+ G)ocMap) user (7 6ser{% &eco)e#ath()ocMap+ Guser) fmt.#rintf("'Ov"+ user) If we r)n this program we get the following o)tp)tB mapstructure.6ser{ /ession("1S0JE101W0(b8)100fefbab8bf0RFIIQ0b1RJffe)fQFR8S0E)SRS e)EbRf1RIbFR8Fb"+ 5obran):)(0111111J+ 6ser ype(mapstructure.6ser ype{ 6ser ype:)(0+ 6ser ypeName("normalWuser" % LoginName("sptest0" % +he K0pathK tag is )sed to find the map keys and set the val)es into the str)ct. +he 7ser str)ct contains fields from three different layers of the O3'& doc)ment. We only needed to define two str)cts to p)ll the data o)t of the map we needed. We can also map entire str)cts the same way a normal )nmarshal wo)ld work. O)st name the fields in the str)ct to match the field names in the O3'& doc)ment. +he names of the fields in the str)ct don.t need to be in the same case as the fields in the O3'& doc)ment. Eere is a more complicated e5ample )sing an anonymo)s field in o)r str)ctB type NumberMormat struct { &ecimal/eparator string XDpath("user5onteBt.preference:nfo.numberMormat.)ecimal/eparat or"X 2L0

2rouping/eparator string XDpath("user5onteBt.preference:nfo.numberMormat.grouping/epara tor"X 2roup#attern string XDpath("user5onteBt.preference:nfo.numberMormat.group#attern"X % type 6ser struct { LoginName string XDpath("loginName"X NumberMormat % )oc/cript (7 UVbyte()ocument) )ocMap (7 mapUstringVinterface{%{% Dson.6nmarshal()oc/cript+ G)ocMap) user (7 6ser{% &eco)e#ath()ocMap+ Guser) fmt.#rintf("'Ov"+ user) If we r)n this program we get the following o)tp)tB mapstructure.6ser{ LoginName("sptest0" NumberMormat(mapstructure.NumberMormat{ &ecimal/eparator("."+ 2rouping/eparator("+"+ 2roup#attern("OOO+OO1.OO" % % We can also )se an anonymo)s field pointerB type 6ser struct { LoginName string XDpath("loginName"X $NumberMormat % In this case ;ecode$ath will instantiate an ob0ect of that type and perform the decode* b)t only if a mapping can be fo)nd. We now have great control over decoding O3'& doc)ments into str)cts. What happens when the O3'& yo) get back is an array of doc)mentsD

DecodeSlicePath
+here are times when the web api ret)rns an array of O3'& doc)mentsB var )ocument 7 XU{"name"("bill"%+{"name"("lisa"%VX

2L?

In this case we need to decode the slice of maps into a slice of ob0ects. We added another f)nction called ;ecode3lice$ath that does 0)st thatB type Name&oc struct { Name string XDpath("name"X % slice/cript (7 UVbyte()ocument) sliceMap (7 UVmapUstringVinterface{%{% Dson.6nmarshal(slice/cript+ GsliceMap) var myslice UVName&oc &eco)e/lice#ath(sliceMap+ Gmyslice) fmt.#rintf("'Ov"+ myslice) Eere is the o)tp)tB UVmapstructure.Name&oc{ mapstructure.Name&oc{Name("bill"%+ mapstructure.Name&oc{Name("lisa"% % +he f)nction ;ecode3lice$ath creates the slice based on the length of the map and then decodes each O3'& doc)ment* one at a time.

$oncl'sion
If it were not for Mitchell I wo)ld not have been able to get this to work. Eis package is brilliant and handles all the real technical iss)es aro)nd decoding maps into str)cts. +he two f)nctions I have b)ilt on top of mapstr)ct)re provides a nice convenience factor we needed for o)r pro0ect. If yo).re dealing with some of the same iss)e* please try o)t the package.

$onc'rrency, Goro'tines and G(2)BP>($S

Introd'ction
When new people 0oin the 9oCMiami gro)p they always write that they want to learn more abo)t 9o.s conc)rrency model. Conc)rrency seems to be the big b)(( word aro)nd the lang)age. It was for me when I first started hearing abo)t 9o. It was 1ob $ike.s 9o Conc)rrency $atterns video that finally convinced me I needed to learn this lang)age. +o )nderstand how 9o makes writing conc)rrent programs easier and less prone to errors* we first need to )nderstand what a conc)rrent program is and the problems that res)lt from s)ch programs. I will not be talking abo)t C3$ @Comm)nicating 3e2)ential $rocessesA in this post* which is the basis for 9o.s implementation of channels. +his post will foc)s on what a conc)rrent program is* the role that goro)tines play and how the 9'M#N$1'C3 environment variable and r)ntime f)nction affects the behavior of the 9o r)ntime and the programs we write.

2L2

Processes and Threads


When we r)n an application* like the browser I am )sing to write this post* a process is created by the operating system for the application. +he 0ob of the process is to act like a container for all the reso)rces the application )ses and maintains as it r)ns. +hese reso)rces incl)de things like a memory address space* handles to files* devices and threads. # thread is a path of e5ec)tion that is sched)led by the operating system to e5ec)te the applications code against a processor or core. # process starts o)t with one thread* the main thread* and when that thread terminates the process terminates. +his is beca)se the main thread is the origin for the application. +he main thread can then in t)rn la)nch more threads and those threads can la)nch even more threads. 'nce we have more than one thread r)nning in o)r program* we have a conc)rrent program. +he operating system sched)les a thread to r)n on an available processor or core regardless of which process the thread belongs to. 6ach operating system has its own algorithms that make these decisions and it is best for )s to write conc)rrent programs that are not specific to one algorithm or the other. $l)s these algorithms change with every new release of an operating system* so it is dangero)s game to play.

Goro'tines and Parallelism


9oro)tines are f)nctions that we re2)est the 9o r)ntime goro)tine sched)ler to e5ec)te conc)rrently. We can consider that the main f)nction is e5ec)ting on a goro)tine* however the 9o r)ntime does not start that goro)tine. 9oro)tines are considered to be lightweight beca)se they )se little memory and reso)rces pl)s their initial stack si(e is small. $rior to version ?.2 the stack si(e started at 8K and now it starts at LK. +he stack has the ability to grow and shrink as needed. +he operating system sched)les threads to r)n against available processors and the 9o r)ntime sched)les goro)tines to r)n against available threads from the sched)lers thread pool. /y defa)lt the sched)lers thread pool is allocated with only one thread. 6ven with one thread* h)ndreds of tho)sands of goro)tines can be sched)led to r)n conc)rrently. It is not recommended to change the si(e of the sched)lers thread pool* b)t if yo) want to r)n goro)tines in parallel* 9o provides the ability to change the si(e of the sched)lers thread pool via the 9'M#N$1'C3 environment variable or r)ntime f)nction. $arallelism is when two or more threads are e5ec)ting code sim)ltaneo)sly against different processors or cores. We can achieve r)nning goro)tines in parallel as long as we are r)nning on a machine with m)ltiple processors or cores and we add more than one thread to the sched)lers thread pool. If we add more threads to the sched)lers thread pool b)t r)n o)r program on a single C$7 machine* o)r goro)tines will r)n against m)ltiple threads b)t will be r)nning conc)rrently against the single C$7* not in parallel.

$onc'rrency E@ample
-et.s b)ild a small program that shows 9o r)nning goro)tines conc)rrently. In this e5ample we are )sing the defa)lt setting for the sched)lers thread pool which is one threadB package main import ( 2L"

"fmt" "time" ) func main() { fmt.#rintln("/tarting 2o -outines") go func() { for char (7 CaC< char = CaC>ES< char>> { fmt.#rintf("'c "+ char) % %() go func() { for number (7 0< number = ER< number>> { fmt.#rintf("') "+ number) % %() fmt.#rintln("Waiting o Minish") time./leep(0 $ time./econ)) fmt.#rintln("*n erminating #rogram") % +his program la)nches two goro)tines by )sing the keyword go and declaring two anonymo)s f)nctions. +he first goro)tine displays the english alphabet )sing lowercase letters and the second goro)tine displays n)mbers ? thro)gh 2H. When we r)n this program we get the following o)tp)tB /tarting 2o -outines Waiting o Minish a b c ) e f g h i D k l m n o p \ r s t u v w B y N 0 E I J F S R 8 Q 01 00 0E 0I 0J 0F 0S 0R 08 0Q E1 E0 EE EI EJ EF ES erminating #rogram When we look at the o)tp)t we can see that the code was r)n conc)rrently. 'nce the two goro)tines are la)nched* the main goro)tine is p)t to sleep for ? second. We need to do this beca)se once the main goro)tine terminates* the program terminates. We want to give eno)gh time for the two goro)tines to complete their work. We can see that the first goro)tine completes displaying all 2H letters and then the second goro)tine gets a t)rn to display all 2H n)mbers. /eca)se it takes less than a microsecond for the first goro)tine to complete its work* we don.t see the sched)ler interr)pt the first goro)tine before it finishes its work. We can give a reason to the sched)ler to swap the goro)tines by p)tting a sleep into the first goro)tineB package main import ( "fmt" "time" 2L8

) func main() { fmt.#rintln("/tarting 2o -outines") go func() { time./(eep(1 8 time.Micro,econd) for char (7 CaC< char = CaC>ES< char>> { fmt.#rintf("'c "+ char) % %() go func() { for number (7 0< number = ER< number>> { fmt.#rintf("') "+ number) % %() fmt.#rintln("Waiting o Minish") time./leep(0 $ time./econ)) fmt.#rintln("*n erminating #rogram") % +his time we add a microsecond of sleep in the first goro)tine as soon as it starts. +his is eno)gh to ca)se the sched)ler to swap the two goro)tinesB /tarting 2o -outines Waiting o Minish 0 E I J F S R 8 Q 01 00 0E 0I 0J 0F 0S 0R 08 0Q E1 E0 EE EI EJ EF ES a b c ) e f g h i D k l m n o p \ r s t u v w B y N erminating #rogram +his time the n)mbers display first and then the letters. # microsecond of sleep is eno)gh to ca)se the sched)ler to stop r)nning the first goro)tine and let the second goro)tine do its thing.

Parallel E@ample
In o)r past two e5amples the goro)tines were r)nning conc)rrently* b)t not in parallel. -et.s make a change to the code to allow the goro)tines to r)n in parallel. #ll we need to do is change the defa)lt si(e of the sched)lers thread pool to )se two threadsB package main import ( "fmt" "r+ntime" "time" ) func main() {

2LG

r+ntime.N5MAZP451/(2) fmt.#rintln("/tarting 2o -outines") go func() { for char (7 CaC< char = CaC>ES< char>> { fmt.#rintf("'c "+ char) % %() go func() { for number (7 0< number = ER< number>> { fmt.#rintf("') "+ number) % %() fmt.#rintln("Waiting o Minish") time./leep(0 $ time./econ)) fmt.#rintln("*n erminating #rogram") % Eere is the o)tp)t for the programB /tarting 2o -outines Waiting o Minish a b 0 E I J c ) e f F g h S i R D 8 k Q 01 00 0E l m n o p \ 0I r s 0J t 0F u v 0S w 0R B y 08 N 0Q E1 E0 EE EI EJ EF ES erminating #rogram 6very time we r)n the program we are going to get different res)lts. +he sched)ler does not behave e5actly the same for each and every r)n. We can see that the goro)tines are tr)ly r)nning in parallel. /oth goro)tines start r)nning immediately and yo) can see them both competing for standard o)t to display their res)lts.

$oncl'sion
O)st beca)se we can change the si(e of the sched)lers thread pool* doesn.t mean we sho)ld. +here is a reason the 9o team has set the defa)lts to the r)ntime the way they did. 6specially the defa)lt for the sched)lers thread pool. O)st know that arbitrarily adding threads to the sched)lers thread pool and r)nning goro)tines in parallel will not necessarily provide better performance for yo)r programs. #lways profile and benchmark yo)r programs and make s)re the 9o r)ntime config)ration is only changed if absol)tely re2)ired. +he problem with b)ilding conc)rrency into o)r applications is event)ally o)r goro)tines are going to attempt to access the same reso)rces* possibly at the same time. 1ead and write operations against a shared reso)rce m)st always be atomic. In other words reads and writes m)st happen by one goro)tine at a time or else we create race conditions in o)r programs. +o learn more abo)t race conditions read my post. Channels are the way in 9o we write safe and elegant conc)rrent programs that eliminate race conditions and make writing conc)rrent programs f)n again. &ow that we know how

2LH

goro)tines work* are sched)led and can be made to r)n in parallel* channels are the ne5t thing we need to learn.

The ?at're (6 $hannels In Go

Introd'ction
In my last post called Conc)rrency* 9oro)tines and 9'M#N$1'C3* I set the stage for talking abo)t channels. We disc)ssed what conc)rrency was and how goro)tines played a role. With that fo)ndation in hand* we can now )nderstand the nat)re of channels and how they can be )sed to synchroni(e goro)tines to share reso)rces in a safe* less error prone and f)n way.

What )re $hannels


Channels are type safe message 2)e)es that have the intelligence to control the behavior of any goro)tine attempting to read or write to it. # channel acts as a cond)it between two goro)tines and will synchroni(e the e5change of any reso)rce that is passed thro)gh it. It is the channel.s ability to control the goro)tines interaction that creates the synchroni(ation mechanism. When a channel is created with no capacity for its 2)e)e* it is called an )nb)ffered channel. In t)rn* a channel created with capacity for its 2)e)e is called a b)ffered channel. +o )nderstand what the synchroni(ation behavior will be for any goro)tine interacting with a channel* we need to know the type and state of the channel. +he scenarios are a bit different depending on whether we are )sing an )nb)ffered or b)ffered channel* so let.s talk abo)t each one independently.

.nb'66ered $hannels
7nb)ffered channels have no capacity and therefore re2)ire both goro)tines to be ready to make any e5change. When a goro)tine attempts to write a reso)rce to an )nb)ffered channel and there is no goro)tine waiting to receive the reso)rce* the channel will lock the goro)tine and make it wait. When a goro)tine attempts to read from an )nb)ffered channel* and there is no goro)tine waiting to send a reso)rce* the channel will lock the goro)tine and make it wait.

2LI

In the diagram above* we see an e5ample of two goro)tines making an e5change )sing an )nb)ffered channel. In step ?* the two goro)tines approach the channel and then in step 2* the goro)tine on the left sticks his hand into the channel or performs a write. #t this point* that goro)tine is locked in the channel )ntil the e5change is complete. +hen in step "* the goro)tine on the right places his hand into the channel or performs a read. +hat goro)tine is also locked in the channel )ntil the e5change is complete. In step 8 and G the e5change is made and finally in step H* both goro)tines are free to remove their hands and go on their way. 3ynchroni(ation is inherent in the interaction between the write and the read. 'ne can not happen witho)t the other. +he nat)re of an )nb)ffered channel is synchroni(ation.

B'66ered $hannels
/)ffered channels have capacity and therefore can behave a bit differently. When a goro)tine attempts to write a reso)rce to a b)ffered channel and channel.s 2)e)e is f)ll* the channel will lock the goro)tine and make it wait )ntil a b)ffer becomes available. If there is room in the 2)e)e* the write can take place immediately and the goro)tine can move on. When a goro)tine attempts to read from a b)ffered channel and the b)ffered channel.s 2)e)e is empty* the channel will lock the goro)tine and make it wait )ntil a reso)rce has been 2)e)ed.

2LL

In the diagram above* we see an e5ample of two goro)tines adding and removing items from a b)ffered channel independently. In step ?* the goro)tine on the right is removing a reso)rce from the channel or performing a read. In step 2* the goro)tine on the right can remove the reso)rce independent of the goro)tine on the left adding a new reso)rce to the channel. In step "* both goro)tines are adding and removing a reso)rce from the channel at the same time and in step 8 both goro)tines are done. 3ynchroni(ation still occ)rs within the interactions of reads and writes* however when the 2)e)e has b)ffer availability* the writes will not lock. 1eads will not lock when there is something to read from the 2)e)e. Conse2)ently* if the b)ffer is f)ll or if there is nothing to retrieve* a b)ffered channel will behave very m)ch like an )nb)ffered channel.

>elay >ace
If yo) have ever watched a track meet yo) may have seen a relay race. In a relay race there are fo)r athletes who r)n aro)nd the track as fast as they can as a team. +he key to the race is that only one r)nner per team can be r)nning at a time. +he r)nner with the baton is the only one allowed to r)n* and the e5change of the baton from r)nner to r)nner is critical to winning the race. -et.s b)ild a sample program that )ses fo)r goro)tines and a channel to sim)late a relay race. +he goro)tines will be the r)nners in the race and the channel will be )sed to e5changed the baton between each r)nner. +his is a classic e5ample of how reso)rces can be passed between goro)tines and how a channel controls the behavior of the goro)tines that interact with it. package main import ( "fmt" "time" ) func main() { // 5reate an unbuffere) channel baton (7 make(chan int) // Mirst runner to his mark go -unner(baton) // /tart the race baton =K 0 // 2ive the runners time to race time./leep(F11 $ time.Millisecon)) % func -unner(baton chan int) { var new-unner int // Wait to receive the baton

2L<

runner (7 =Kbaton // /tart running aroun) the track fmt.#rintf("-unner ') -unning With !aton*n"+ runner) // New runner to the line if runner ?7 J { new-unner 7 runner > 0 fmt.#rintf("-unner ') o go -unner(baton) %

he Line*n"+ new-unner)

// -unning aroun) the track time./leep(011 $ time.Millisecon)) // :s the race over if runner 77 J { fmt.#rintf("-unner ') Minishe)+ -ace 3ver*n"+ runner) return % // @Bchange the baton for the neBt runner fmt.#rintf("-unner ') @Bchange With -unner ')*n"+ runner+ new-unner) baton =K new-unner % When we r)n the sample program we get the following o)tp)tB -unner -unner -unner -unner -unner -unner -unner -unner -unner -unner -unner 0 E 0 E I E I J I J J -unning With !aton o he Line @Bchange With -unner E -unning With !aton o he Line @Bchange With -unner I -unning With !aton o he Line @Bchange With -unner J -unning With !aton Minishe)+ -ace 3ver

+he program starts o)t creating an )nb)ffered channelB // 5reate an unbuffere) channel baton (7 make(chan int) 7sing an )nb)ffered channel forces the goro)tines to be ready at the same time to make the e5change of the baton. +his need for both goro)tines to be ready creates the synchroni(ation. If we look at the rest of the main f)nction* we see a goro)tine created for the first r)nner in the race and then the baton is handed off to that r)nner. +he baton in this e5ample is an 2<0

integer val)e that is being passed between each r)nner. +he sample is )sing a sleep to let the race complete before main terminates and ends the programB // 5reate an unbuffere) channel baton (7 make(chan int) // Mirst runner to his mark go -unner(baton) // /tart the race baton =K 0 // 2ive the runners time to race time./leep(F11 $ time.Millisecon)) If we 0)st foc)s on the core parts of the 1)nner f)nction* we can see how the baton e5change takes place )ntil the race is over. +he 1)nner f)nction is la)nched as a goro)tine for each r)nner in the race. 6very time a new goro)tine is la)nched* the channel is passed into the goro)tine. +he channel is the cond)it for the e5change* so the c)rrent r)nner and the one waiting to go ne5t need to reference the channelB func -unner(baton chan int) +he first thing each r)nner does is wait for the baton e5change. +hat is sim)lated with the read on the channel. +he read immediately locks the goro)tine )ntil the baton is written to the channel. 'nce the baton is written to the channel* the read will release and the goro)tine will sim)late the ne5t r)nner sprinting down the track. If the fo)rth r)nner is r)nning* no new r)nner will enter the race. If we are still in the middle of the race* a new goro)tine for the ne5t r)nner is la)nched. // Wait to receive the baton runner (7 =Kbaton // New runner to the line if runner ?7 J { new-unner 7 runner > 0 go -unner(baton) % +hen we sleep to sim)late some time it takes for the r)nner to r)n aro)nd the track. If this is the fo)rth r)nner* the goro)tine terminates after the sleep and the race is complete. If not* the baton e5change takes place with the write to the channel. +here is a goro)tine already locked and waiting for this e5change. #s soon as the baton is written to the channel* the e5change is made and the race contin)eB // -unning aroun) the track time./leep(011 $ time.Millisecon)) // :s the race over if runner 77 J { return 2<?

% // @Bchange the baton for the neBt runner baton =K new-unner

$oncl'sion
+he e5ample showcases a real world event* a relay race between r)nners* being implemented in a way that mimics the act)al events. +his is one of the bea)tif)l things abo)t channels. +he code flows in a way that sim)lates how these types of e5changes can happen in the real world. &ow that we have an )nderstanding of the nat)re of )nb)ffered and b)ffered channels* we can look at different conc)rrency patterns we can implement )sing channels. Conc)rrency patterns allow )s to implement more comple5 e5changes between goro)tines that sim)late real world comp)ting problems like semaphores* generators and m)ltiple5ers.

>'nning 2ongoDB J'eries $onc'rrently With Go


If you are attending (opher3on ,4+4 or plan to wat&h the videos on&e they are released) this arti&le will prepare you for the tal. by (ustavo Niemeyer and 5teve Fran&ia. It provides a beginners view for using the (o mgo driver against a Mongo67 database.

Introd'ction
Mongo;/ s)pports many different programming lang)ages thanks to a great set of drivers. 'ne s)ch driver is the Mongo;/ 9o driver which is called mgo. +his driver has been e5ternally developed by 9)stavo &iemeyer from Canonical* and event)ally 3teve Francia* the head of the drivers team at Mongo;/ Inc* took notice and offered s)pport. /oth 9)stavo and 3teve will be talking at 9opherCon 20?8 in #pril abo)t K$ainless ;ata 3torage With Mongo;/ and 9oK. +he talk centers aro)nd the mgo driver and how Mongo;/ and 9o really work well together to b)ild highly scalable and conc)rrent software. Mongo;/ and 9o let )s b)ild scalable software on many different operating systems and architect)res* witho)t the need to install any frameworks or r)ntime environments. 9o programs are native binaries and the 9o tooling is constantly improving to create binaries that r)n as fast as e2)ivalent C programs. +hat wo)ldn.t mean anything if writing code in 9o was complicated and as tedio)s as writing programs in C. +his is where 9o really shines beca)se once yo) get )p to speed* writing programs in 9o is fast and f)n. In this post I am going to show yo) how to write a 9o program )sing the mgo driver to connect and r)n 2)eries conc)rrently against a Mongo;/ database. I will break down the sample code and e5plain a few things that seem to be always be a bit conf)sing to those new to Mongo;/ and 9o.

Sample Program
+he sample program connects to a p)blic Mongo;/ database I have hosted with Mongo-ab. If yo) have 9o and /a(aar installed on yo)r machine* yo) can r)n the program. +he program la)nches ten goro)tines that individ)ally 2)ery all the records from the b)oyFstations

2<2

collection inside the goinggo database. +he records are )nmarshaled into native 9o types and each goro)tine logs the n)mber of doc)ments ret)rnedB ? 2 " 8 G H I L < ? 0 ? ? ? 2 ? " ? 8 ? G ? H ? I ? L ? < 2 0 2 ? 2 2 2 " 2 8 2 G 2 H 2 I 2 L
// his program provi)es a sample application for using Mongo&! with // the mgo )river. package main import ( "labiB.org/vE/mgo" "labiB.org/vE/mgo/bson" "log" "sync" "time" ) const ( Mongo&!Aosts .uth&atabase .uth6serName .uth#asswor) est&atabase 7 7 7 7 7 ")s1IFJE8.mongolab.com(IFJE8" "goinggo" "guest" "welcome" "goinggo"

) type ( // !uoy5on)ition contains information for an in)ivi)ual station. !uoy5on)ition struct { Win)/pee) floatSJ Xbson("win)Wspee)Wmilehour"X Win)&irection int Xbson("win)W)irectionW)egnorth"X Win)2ust floatSJ Xbson("gustWwin)Wspee)Wmilehour"X % // !uoyLocation contains the buoyCs location. !uoyLocation struct { ype string Xbson("type"X 5oor)inates UVfloatSJ Xbson("coor)inates"X % // !uoy/tation contains information for an in)ivi)ual station. !uoy/tation struct { :& bson.3bDect:) Xbson("Wi)+omitempty"X /tation:) string Xbson("stationWi)"X Name string Xbson("name"X Loc&esc string Xbson("locationW)esc"X 5on)ition !uoy5on)ition Xbson("con)ition"X Location !uoyLocation Xbson("location"X %

// main is the entry point for the application. func main() { // We nee) this obDect to establish a session to our Mongo&!. mongo&!&ial:nfo (7 Gmgo.&ial:nfo{ .))rs( UVstring{Mongo&!Aosts%+ imeout( S1 $ time./econ)+ &atabase( .uth&atabase+ 6sername( .uth6serName+ #asswor)( .uth#asswor)+ % // 5reate a session which maintains a pool of socket connections // to our Mongo&!.

2<"

2 < " 0 " ? " 2 " " " 8 " G " H " I " L " < 8 0 8 ? 8 2 8 " 8 8 8 G 8 H 8 I 8 L 8 < G 0 G ? G 2 G "

mongo/ession+ err (7 mgo.&ialWith:nfo(mongo&!&ial:nfo) if err ?7 nil { log.Matalf("5reate/ession( 's*n"+ err) % // -ea)s may not be entirely upKtoK)ate+ but they will always see the // history of changes moving forwar)+ the )ata rea) will be consistent // across se\uential \ueries in the same session+ an) mo)ifications ma)e // within the session will be observe) in following \ueries (rea)KyourKwrites). // http(//go)oc.org/labiB.org/vE/mgoO/ession./etMo)e mongo/ession./etMo)e(mgo.Monotonic+ true) // 5reate a wait group to manage the goroutines. var wait2roup sync.Wait2roup // #erform 01 concurrent \ueries against the )atabase. wait2roup..))(01) for \uery (7 1< \uery = 01< \uery>> { go -un,uery(\uery+ Gwait2roup+ mongo/ession) % // Wait for all the \ueries to complete. wait2roup.Wait() log.#rintln(".ll ,ueries 5omplete)") % // -un,uery is a function that is launche) as a goroutine to perform // the Mongo&! work. func -un,uery(\uery int+ wait2roup $sync.Wait2roup+ mongo/ession $mgo./ession) { // &ecrement the wait group count so the program knows this // has been complete) once the goroutine eBits. )efer wait2roup.&one() \uery. // -e\uest a socket connection from the session to process our

// 5lose the session when the goroutine eBits an) put the connection back // into the pool. session5opy (7 mongo/ession.5opy() )efer session5opy.5lose() // 2et a collection to eBecute the \uery against. collection (7 session5opy.&!( est&atabase).5("buoyWstations") log.#rintf("-un,uery ( ') ( @Becuting*n"+ \uery) // -etrieve the list of stations. var buoy/tations UV!uoy/tation err (7 collection.Min)(nil)..ll(Gbuoy/tations) if err ?7 nil { log.#rintf("-un,uery ( @--3- ( 's*n"+ err) return % log.#rintf("-un,uery ( ') ( 5ountU')V*n"+ \uery+ len(buoy/tations))

2<8

G % 8 G G G H G I G L G < H 0 H ? H 2 H " H 8 H G H H H I H L H < I 0 I ? I 2 I " I 8 I G I H I I I L

2<G

I < L 0 L ? L 2 L " L 8 L G L H L I L L L < < 0 < ? < 2 < " < 8 < G < H < I < L < < view raw 9oMgo3ampleCF)ll.go hosted with by 9itE)b &ow that yo) have seen the entire program* we can break it down. -et.s start with the type str)ct)res that are defined in the beginningB ? type ( // !uoy5on)ition contains information for an in)ivi)ual station. 2 !uoy5on)ition struct { " Win)/pee) floatSJ Xbson("win)Wspee)Wmilehour"X 8 Win)&irection int Xbson("win)W)irectionW)egnorth"X

2<H

G H I L < ? 0 ? ? ? Win)2ust floatSJ Xbson("gustWwin)Wspee)Wmilehour"X 2 % ? // !uoyLocation contains the buoyCs location. " !uoyLocation struct { ? ype string Xbson("type"X 8 5oor)inates UVfloatSJ Xbson("coor)inates"X ? % G // !uoy/tation contains information for an in)ivi)ual station. ? !uoy/tation struct { H :& bson.3bDect:) Xbson("Wi)+omitempty"X ? /tation:) string Xbson("stationWi)"X I Name string Xbson("name"X Loc&esc string Xbson("locationW)esc"X ? 5on)ition !uoy5on)ition Xbson("con)ition"X L Location !uoyLocation Xbson("location"X ? % < ) 2 0 2 ? 2 2 2 " 2 8 view raw 9oMgo3ampleC?.go hosted with by 9itE)b +he str)ct)res represent the data that we are going to retrieve and )nmarshal from o)r 2)ery. /)oy3tation represents the main doc)ment and /)oyCondition and /)oy-ocation are embedded doc)ments. +he mgo driver makes it easy to )se native types that represent the doc)ments stored in o)r collections by )sing tags. With the tags* we can control how the mgo driver )nmarshals the ret)rned doc)ments into o)r native 9o str)ct)res. &ow let.s look at how we connect to a Mongo;/ database )sing mgoB ? // We nee) this obDect to establish a session to our Mongo&!. (7 Gmgo.&ial:nfo{ 2 mongo&!&ial:nfo .))rs( UVstring{Mongo&!Aosts%+ " imeout( S1 $ time./econ)+ 8 &atabase( .uth&atabase+ 6sername( .uth6serName+ G #asswor)( .uth#asswor)+ H
%

2<I

I L < ? 0 ? // 5reate a session which maintains a pool of socket connections ? // to our Mongo&!. ? mongo/ession+ err (7 mgo.&ialWith:nfo(mongo&!&ial:nfo) 2 if err ?7 nil { log.Matalf("5reate/ession( 's*n"+ err) ? % " ? 8 ? G view raw 9oMgo3ampleC2.go hosted with by 9itE)b We start with creating a mgo.;ialInfo ob0ect. We can connect to a single Mongo;/ database or a replica set. Connecting to a replica set can be accomplished by providing m)ltiple addresses in the #ddrs field or with a single address. If we are )sing a single host address to connect to a replice set* the mgo driver will learn abo)t any remaining hosts from the replica set member we connect to. In o)r case we are connecting to a single host. #fter providing the host* we specify the database* )sername and password we need for a)thentication. 'ne thing to note is that the database we a)thenticate against may not necessarily be the database o)r application needs to access. 3ome applications a)thenticate against the admin database and then )se other databases depending on their config)ration. +he mgo driver s)pports these types of config)rations very well. &e5t we )se the mgo.;ialWithInfo method to create a mgo.3ession ob0ect. +he mgo.3ession ob0ect maintains a pool of connections to the Mongo;/ host. We can create m)ltiple sessions with different modes and settings to s)pport different aspects of o)r applications. We can specify if the session is to )se a 3trong or Monotonic mode* and we can set the safe level as well as other settings. +he ne5t line of code sets the mode for the session. +here are three modes that can be set* 3trong* Monotonic and 6vent)al. 6ach mode sets a specific consistency for how reads and writes are performed. For more information on the differences between each mode* check o)t the doc)mentation for the mgo driver. We are )sing Monotonic mode which provides reads that may not entirely be )p to date* b)t the reads will always see the history of changes moving forward. In this mode reads occ)r against secondary members of o)r replica sets )ntil a write happens. 'nce a write happens* the primary member is )sed. +he benefit is some distrib)tion of the reading load can take place against the secondaries when possible. With the session set and ready to go* ne5t we e5ec)te m)ltiple 2)eries conc)rrentlyB ? // 5reate a wait group to manage the goroutines. 2 var wait2roup sync.Wait2roup " // #erform 01 concurrent \ueries against the )atabase. 2<L

8 G H wait2roup..))(01) I for \uery (7 1< \uery = 01< \uery>> { L go -un,uery(\uery+ Gwait2roup+ mongo/ession) < % ? 0 // Wait for all the \ueries to complete. wait2roup.Wait() ? log.#rintln(".ll ,ueries 5omplete)") ? ? 2 view raw 9oMgo3ampleC8.go hosted with by 9itE)b +his code is classic 9o conc)rrency in action. First we create a sync.Wait9ro)p ob0ect so we can keep track of all the goro)tines we are going to la)nch as they complete their work. +hen we immediately set the co)nt of the sync.Wait9ro)p ob0ect to ten and )se a for loop to la)nch ten goro)tines )sing the 1)n4)ery f)nction. +he keyword go is )sed to la)nch a f)nction or method to r)n conc)rrently. +he final line of code calls the Wait method on the sync.Wait9ro)p ob0ect which locks the main goro)tine )ntil everything is done processing. +o learn more abo)t 9o conc)rrency and better )nderstand how this partic)lar piece of code works* check o)t these posts on conc)rrency and channels. &ow let.s look at the 1)n4)ery f)nction and see how to properly )se the mgo.3ession ob0ect to ac2)ire a connection and e5ec)te a 2)eryB ? 2 " 8// has been complete) once the goroutine eBits.
)efer wait2roup.&one() // &ecrement the wait group count so the program knows this

G// -e\uest a socket connection from the session to process our \uery. H// into the pool.
// 5lose the session when the goroutine eBits an) put the connection back session5opy (7 mongo/ession.5opy() I)efer session5opy.5lose()

L < view raw 9oMgo3ampleCG.go hosted with by 9itE)b +he very first thing we do inside of the 1)n4)ery f)nction is to defer the e5ec)tion of the ;one method on the sync.Wait9ro)p ob0ect. +he defer keyword will postpone the e5ec)tion of the ;one method* to take place once the 1)n4)ery f)nction ret)rns. +his will g)arantee that the sync.Wait9ro)p ob0ects co)nt will decrement even if an )nhandled e5ception occ)rs.

2<<

&e5t we make a copy of the session we created in the main goro)tine. 6ach goro)tine needs to create a copy of the session so they each obtain their own socket witho)t seriali(ing their calls with the other goro)tines. #gain* we )se the defer keyword to postpone and g)arantee the e5ec)tion of the Close method on the session once the 1)n4)ery f)nction ret)rns. Closing the session ret)rns the socket back to the main pool* so this is very important. ? 2 " 8 // 2et a collection to eBecute the \uery against. G collection (7 session5opy.&!( est&atabase).5("buoyWstations") H I log.#rintf("-un,uery ( ') ( @Becuting*n"+ \uery) L < // -etrieve the list of stations. var buoy/tations UV!uoy/tation ? err (7 collection.Min)(nil)..ll(Gbuoy/tations) 0 if err ?7 nil { log.#rintf("-un,uery ( @--3- ( 's*n"+ err) ? return ? % ? 2 log.#rintf("-un,uery ( ') ( 5ountU')V*n"+ \uery+ len(buoy/tations)) ? " ? 8 view raw 9oMgo3ampleCH.go hosted with by 9itE)b +o e5ec)te a 2)ery we need a mgo.Collection ob0ect. We can get a mgo.Collection ob0ect thro)gh the mgo.3ession ob0ect by specifying the name of the database and then the collection. 7sing the mgo.Collection ob0ect* we can perform a Find and retrieve all the doc)ments from the collection. +he #ll f)nction will )nmarshal the response into o)r slice of /)oy3tation ob0ects. # slice is a dynamic array in 9o. /e aware that the #ll method will load all the data in memory at once. For large collections it is better to )se the Iter method instead. Finally* we 0)st log the n)mber of /)oy3tation ob0ects that are ret)rned.

$oncl'sion
+he e5ample shows how to )se 9o conc)rrency to la)nch m)ltiple goro)tines that can e5ec)te 2)eries against a Mongo;/ database independently. 'nce a session is established* the mgo driver e5poses all of the Mongo;/ f)nctionality and handles the )nmarshaling of /3'& doc)ments into 9o native types. Mongo;/ can handle a large n)mber of conc)rrent re2)ests when yo) architect yo)r Mongo;/ databases and collections with conc)rrency in mind. 9o and the mgo driver are perfectly aligned to p)sh Mongo;/ to its limits and b)ild software that can take advantage of all the comp)ting power that is available. +he mgo driver can help yo) distrib)te yo)r 2)eries across a Mongo;/ replica set. +he mgo driver gives yo) the ability to create and config)re yo)r sessions and take advantage of Mongo;/.s mode and config)ration options. +he mode yo) )se for yo)r session* how and where the cl)ster and load balancer is set)p* and the type of work being processed by Mongo;/ at the time of those 2)eries* plays an important role in the act)al distrib)tion. "00

+he mgo driver provides a safe way to leverage 9o.s conc)rrency s)pport and yo) have the fle5ibility to e5ec)te 2)eries conc)rrently and in parallel. It is best to take the time to learn a bit abo)t Mongo;/ replica sets and load balancer config)ration. +hen make s)re the load balancer is behaving as e5pected )nder the different types of load yo)r application can prod)ce. &ow is a great time to see what Mongo;/ and 9o can do for yo)r software applications* web services and service platforms. /oth technologies are being battle tested everyday by all types of companies* solving all types of b)siness and comp)ting problems.

Web Corm 9alidation )nd LocaliDation In Go

Introd'ction
#s I improve my knowledge and framework for a 9o based web service I am b)ilding* I contin)e to go back and enhance my /eego 3ample #pp. 3omething I 0)st added recently was providing locali(ed messages for validation errors. I was fort)nate to find &ick 3nyder.s goCi?Ln package. &ick.s package made it easy to s)pport m)ltiple lang)ages for the 9o web service I am writing.

)bstracting go<i 8n
+he goCi?Ln package is simple to )se and yo) can )se it to read files or strings that contain all the messages yo) want to locali(e. It has some nice feat)res incl)ding variable s)bstit)tion and s)pport for handling pl)rals for each individ)al locale. &ick has doc)mentation for his package* so I am going to show yo) how I abstracted and integrated goCi?Ln into the /eego sample app. I decided I didn.t want to )se files to store the messages* b)t create raw string literal variables. +he less I had to worry abo)t managing e5ternal reso)rces the better. With that being said* I b)ilt a simple package that abstracted the s)pport I needed. -)ckily goCi?Ln s)pports passing in a string that can contain the O3'& doc)ment with the message dataB ? 2 " 8 G H I L < ? 0 ? ? ? 2 ? "
// enK6/.go provi)es the localiNe) messages for @nglish in the 6nite) /tates. package localiNe var @nW6/ 7 XU { %+ { %+ { % VX

"i)"( "invali)Wcre)entials"+ "translation"( ":nvali) 5re)entials were supplie)." "i)"( "applicationWerror"+ "translation"( ".n .pplication @rror has occure)." "i)"( "invali)WstationWi)"+ "translation"( ":nvali) /tation :) 3r Missing"

"0?

? 8 ? G ? H ? I view raw locali(ationFmessages.go hosted with by 9itE)b I am 0)st )sing simple messages right now* b)t as yo) can see* the variable 6nF73 is defined and assigned a O3'& doc)ment with the messages I need locali(ed. +he goCi?Ln package also lets yo) define messages like thisB ? 2 U { " "i)"( ")W)ays"+ 8 "translation"( { "one"( "{{.5ount%% )ay"+ G "other"( "{{.5ount%% )ays" H % I % L V < view raw locali(ationFsample.go hosted with by 9itE)b In this sample* the translation has one message for the sing)lar case and one for the pl)ral case. +here is also s)pport for )sing variable s)bstit)tion thanks to template s)pport. Eere is the locali(e package that provides s)pport for the web serviceB ? 2 " 8 G H I L < ? 0 ? ? ? 2 ? " ? 8 ? G ?
// he localiNe package provi)es support for han)ling )ifferent languages // an) cultures. package localiNe import ( "enco)ing/Dson" "github.com/nicksny)er/goKi08n/i08n" "github.com/nicksny)er/goKi08n/i08n/locale" "github.com/nicksny)er/goKi08n/i08n/translation" ) var ( // is the translate function for the specifie) user // locale an) )efault locale specifie) )uring the loa). i08n. ranslateMunc

) // :nit initialiNes the local environment. func :nit()efaultLocale string) error { switch )efaultLocale { case "enK6/"( Loa)9/3N()efaultLocale+ @nW6/) )efault( return fmt.@rrorf("6nsupporte) Locale( 's"+ )efaultLocale) %

"02

H ? I ? L ? < 2 0 2 ? 2 2 2 " 2 8 2 G 2 H 2 I 2 L 2 < " 0 " ? " 2 " " " 8 " G " H " I " L " < 8 0 8

// 3btain the )efault translation function for use. var err error + err 7 New ranslation()efaultLocale) if err ?7 nil { return err % % return nil

// New ranslation obtains a translation function obDect for the // specifie) locales. func New ranslation(userLocale string (t i08n. ranslateMunc+ err error) { t+ err 7 i08n. func(userLocale) if err ?7 nil { return t+ err % return t+ err % // Loa)9/3N takes a Dson )ocument of translations an) manually // loa)s them into the system. func Loa)9/3N(userLocale string+ translation&ocument string) error { tran&ocuments (7 UVmapUstringVinterface{%{% err (7 Dson.6nmarshal(UVbyte(translation&ocument)+ Gtran&ocuments) if err ?7 nil { return err % for W+ tran&ocument (7 range tran&ocuments { tran+ err (7 translation.New ranslation(tran&ocument) if err ?7 nil { return err % % % i08n..)) ranslation(locale.MustNew(userLocale)+ tran)

return nil

"0"

? 8 2 8 " 8 8 8 G 8 H 8 I 8 L 8 < G 0 G ? G 2 G " G 8 G G G H G I G L G < H 0 H ? H 2 H " H 8 H G H

"08

H H I H L view raw locali(ationFf)ll.go hosted with by 9itE)b +he Init f)nction creates the defa)lt locale for the application. C)rrently the /eego 3ample #pp only s)pports 6nglish for the 7nited 3tates. 6vent)ally* we can add cases for the other locales. 'bvio)sly this can all be done thro)gh config)ration in the f)t)re. +he Init f)nction )ses the -oadO3'& f)nction to load the goCi?Ln datastore with the internal messages for the defa)lt locale. -ater on we can )se the -oadO3'& f)nction again to load more O3'& doc)ments for the same or different locales. +he Init f)nction also )ses the &ew+ranslation f)nction to obtain a new i?Ln.+ranslateF)nc ob0ect for the defa)lt locale. +his ob0ect is )sed to retrieve messages from the goCi?Ln datastore. If we have a scenario where the defa)lt locale is not valid* we can )se the &ew+ranslation f)nction at any time to obtain an ob0ect for the locale we need.

Beego Integration
+o see how I integrated the goCi?Ln package into the sample app* we need to look at the controllerB ? 2 " // -etrieve/tation han)les the eBample E tab. 8 func (buoy5ontroller $!uoy5ontroller) -etrieve/tation() { params (7 struct { G /tation:) string Xform("station:)" H error("invali)WstationWi)" vali)("-e\uire)"X I %{% L if buoy5ontroller.#arse.n)Pali)ate(Gparams) 77 false { < return ? % 0 ... ? % ? ? 2 view raw locali(ationFcontroller.go hosted with by 9itE)b #s disc)ssed in my previo)s post abo)t the /eego 3ample #pp* we define a str)ct with tags that are )sed by the /eego validation mod)le. I added s)pport for defining the error to be ret)rned when validation fails* by providing a new tag called error. In this case the error tag contains the id of the locali(ed message we want to ret)rn. +he $arse#nd=alidate f)nction will handle the restB ? 2 "
// #arse.n)Pali)ate will run the params through the vali)ation framework an) then // response with the specifie) localiNe) or provi)e) message func (base5ontroller $!ase5ontroller) #arse.n)Pali)ate(params

"0G

8 G H I L < ? 0 ? ? ? 2 ? " ? 8 ? G ? H ? I ? L ? < 2 0 2 ? 2 2 2 " 2 8 2 G 2 H 2 I 2 L 2 < " 0 " ?

interface{%) bool { err (7 base5ontroller.#arseMorm(params) if err ?7 nil { base5ontroller./erve@rror(err) return false % vali) (7 vali)ation.Pali)ation{% ok+ err (7 vali).Pali)(params) if err ?7 nil { base5ontroller./erve@rror(err) return false % if ok 77 false { // !uil) a map of the error messages for each fiel) messagesE (7 mapUstringVstring{% val (7 reflect.Palue3f(params).@lem() for i (7 1< i = val.NumMiel)()< i>> { // Look for an error tag in the fiel) typeMiel) (7 val. ype().Miel)(i) tag (7 typeMiel). ag tagPalue (7 tag.2et("error") // Was there an error tag if tagPalue ?7 "" { messagesEUtypeMiel).NameV 7 tagPalue % % // !uil) the error response errors (7 UVstring{% for W+ err (7 range vali).@rrors { // Match an error from the vali)ation framework errors // to a fiel) name we have a mapping for message+ ok (7 messagesEUerr.Miel)V if ok 77 true { // 6se a localiNe) message if one eBists errors 7 appen)(errors+ continue % // No match+ so use the message as is errors 7 appen)(errors+ err.Message) % base5ontroller./ervePali)ation@rrors(errors) return false % % return true

localiNe. (message))

"0H

" 2 " " " 8 " G " H " I " L " < 8 0 8 ? 8 2 8 " 8 8 8 G 8 H 8 I 8 L 8 < G 0 G ? G 2 G " G 8 view raw locali(ationFparseval.go hosted with by 9itE)b When the /eego validation mod)le finds a problem* then the real work begins. +he f)nction )ses reflection to find the error tag on any of the fields in the param str)ct. If an error tag is fo)nd* the id of the locali(ed message is stored along with the field name. +hen the f)nction

"0I

ranges over all the errors that the /eego validation mod)le fo)nd and if an error tag e5isted for that field* we )se the id to retrieve the locali(ed message.

Testing
If we r)n the r)nFendpointFtest.sh shell script* which can be fo)nd in the (scripts folder* we can see the locali(ed message ret)rned in the last testB ? 2 " 8 G H I 777 -6N estMissing/tationK8 L -.5@( E10J/1I/1R 0R(0I(F1 mongo.go(08S( 6nknown ( 5opy/ession ( /tarte) < ( 6se/essionUmonotonicV ? -.5@( E10J/1I/1R 0R(0I(F1 mongo.go(E11( 6nknown ( 5opy/ession ( 5omplete) 0 -.5@( E10J/1I/1R 0R(0I(F1 base5ontroller.go(FE( 6nknown ( ? !ase5ontroller.#repare ( :nfo ( 6ser:)U6nknownV #athU/buoy/station/JE1V ? -.5@( E10J/1I/1R 0R(0I(F1 base5ontroller.go(SJ( 6nknown ( Minish ( ? 5omplete) ( /buoy/station/JE1 2 -.5@( E10J/1I/1R 0R(0I(F1 mongo.go(EJ1( 6nknown ( 5lose/ession ( /tarte) -.5@( E10J/1I/1R 0R(0I(F1 mongo.go(EJJ( 6nknown ( 5lose/ession ( ? 5omplete) " -.5@( E10J/1I/1R 0R(0I(F1 buoy@n)pointsWtest.go(01I( testing ( ? est/tation ( :nfo ( 5o)eUJ1QV 8 { "errors"( U ? ":nvali) /tation :) 3r Missing" G V ? % H /ubDect( est /tation @n)point ? I /tatus 5o)e /houl) !e J1Q ? he -esult /houl) Not !e @mpty L here /houl) !e .n @rror :n he -esult ? < Q assertions thus far 2 KKK #.//( estMissing/tationK8 (1.11 secon)s) 0 2 ? 2 2 2 " view raw locali(ationFres)lt.go hosted with by 9itE)b +he last test is designed to validate the locali(ed message is ret)rned.

$oncl'sion
+he /eego framework has been great for developing my 9o web service. It has the right

"0L

amo)nt of framework and mod)les* like the validation mod)le* when yo) need it. +he ability to bring in a package like goCi?Ln and integrate it so easily is another big win for /eego. If yo) are in need for locali(ing yo)r 9o application* take a look at goCi?Ln and see if it can work for yo).

E@ported-.ne@ported Identi6iers In Go

Introd'ction
'ne of the first things I learned abo)t in 9o was )sing an )ppercase or lowercase letter as the first letter when naming a type* variable or f)nction. It was e5plained that when the first letter was capitali(ed* the identifier was p)blic to any piece of code that wanted to )se it. When the first letter was lowercase* the identifier was private and co)ld only be accessed within the package it was defined. I have come to reali(e that the )se of the lang)age p)blic and private is really not acc)rate. It is more acc)rate to say an identifier is e5ported or )ne5ported from a package. When an identifier is e5ported from a package* it means the identifier can be directly accessed from any other package in the code base. When an identifier is )ne5ported from a package* it can.t be directly accessed from any other package. What we will soon learn is that 0)st beca)se an identifier is )ne5ported* it doesn.t mean it can.t be accessed o)tside of its package* it 0)st means it can.t be accessed directly.

Direct Identi6ier )ccess


-et.s start with a simple e5ample of an e5ported typeB ? package counters 2 " // .lert5ounter is an eBporte) type that 8 // contains an integer counter for alerts. type .lert5ounter int G view raw e5portedtypesF?a.go hosted with by 9itE)b Eere we define a )serCdefined type called .lert5ounter inside the package counters. +his type is an alias for the b)iltCin type int* b)t in 9o .lert5ounter will be considered a )ni2)e and distinct type. We are )sing the capital letter C.C as the first letter for the name of the type* which means this type is e5ported and accessible by other packages. &ow let.s access o)r .lert5ounter type in the main programB ? 2 " 8 G H I L < ?
package main import ( "fmt" "test/counters" ) func main() { // 5reate a variable of the eBporte) type an) // initialiNe the value to 01. counter (7 counters..lert5ounter(01)

"0<

0 ? ? ? fmt.#rintf("5ounter( ')*n"+ counter) 2 % ? " ? 8 view raw e5portedtypesF?b.go hosted with by 9itE)b 3ince the .lert5ounter type has been e5ported* this code b)ilds fine. When we r)n the program we get the val)e of ?0. &ow let.s change the e5ported .lert5ounter type to be an )ne5ported type by changing the name to alert5ounter and see what happensB ? 2
package counters // contains an integer counter for alerts.

"// alert5ounter is an uneBporte) type that 8type alert5ounter int G view raw e5portedtypesF2a.go hosted with by 9itE)b ? 2 " 8 package main G H import ( I "fmt" "test/counters" L ) < ? func main() { 0 // .ttempt to create a variable of the uneBporte) type // an) initialiNe the value to 01. his will N3 compile. ? counter (7 counters.alert5ounter(01) ? ? fmt.#rintf("5ounter( ')*n"+ counter) 2 % ? " ? 8 view raw e5portedtypesF2b.go hosted with by 9itE)b #fter making the changes to the counters and main packages* we attempt to b)ild the code again and get the following compiler errorB

"?0

./main.go(00( cannot refer to uneBporte) name counters.alert5ounter ./main.go(00( un)efine)( counters.alert5ounter #s e5pected we can.t directly access the alert5ounter type beca)se it is )ne5ported. 6ven tho)gh we can.t access the alert5ounter type directly anymore* there is a way for )s to create and )se variables of this )ne5ported type in the main packageB ? 2 " package counters 8 // alert5ounter is an uneBporte) type that G // contains an integer counter for alerts. H type alert5ounter int I L // New.lert5ounter creates an) returns obDects of // the uneBporte) type alert5ounter. < func New.lert5ounter(value int) alert5ounter { ? return alert5ounter(value) 0 % ? ? view raw e5portedtypesF"a.go hosted with by 9itE)b ? 2 " 8 package main G H import ( I "fmt" "test/counters" L ) < ? func main() { 0 // 5reate a variable of the uneBporte) type using the // eBporte) New.lert5ounter function from the package counters. ? counter (7 counters.New.lert5ounter(01) ? ? fmt.#rintf("5ounter( ')*n"+ counter) 2 % ? " ? 8 view raw e5portedtypesF"b.go hosted with by 9itE)b In the counters package we add an e5ported f)nction called New.lert5ounter. +his f)nction creates and ret)rns ob0ects of the alert5ounter type. In the main program we )se this f)nction and the programming logic stays the same. What this e5ample shows is that an identifier that is defined as )ne5ported can still be accessed and )sed by other packages. It 0)st can.t be accessed directly.

"??

.sing Str'cts
;efining e5ported and )ne5ported members for o)r str)cts work in the e5act same way. If a field or method name starts with a capital letter* the member is e5ported and is accessible o)tside of the package. If a field or method starts with a lowercase letter* the member is )ne5ported and does not have accessibility o)tside of the package. Eere is an e5ample of a str)ct with both e5ported and )ne5ported fields. +he main program has a compiler error beca)se it attempts to access the )ne5ported field directlyB ? 2 "
package animals

8// &og represents information about )ogs.


type &og struct { Name string !ark/trength int age int H%

I L view raw e5portedtypesF8a.go hosted with by 9itE)b ? package main 2 import ( " "fmt" 8 "test/animals" G ) H func main() { I // 5reate an obDect of type &og from the animals package. L // his will N3 compile. < )og (7 animals.&og{ Name( "5hole"+ ? !ark/trength( 01+ 0 age( F+ ? % ? fmt.#rintf("5ounter( 'Ov*n"+ )og) ? % 2 ? " ? 8 ? G ? H ? "?2

I ? L view raw e5portedtypesF8b.go hosted with by 9itE)b Eere is the error from the compilerB ./main.go(0J( unknown animal.&og fiel) CageC in struct literal #s e5pected the compiler does not let the main program access the age field directly. -et.s look at an interesting e5ample of embedding. We start with two )serCdefined types where one type embeds the otherB ? 2 " 8 package animals G .nimal represents information about all animals. H // type .nimal struct { I Name string L .ge int < % ? // &og represents information about )ogs. 0 type &og struct { ? .nimal !ark/trength int ? ? % 2 ? " view raw e5portedtypesFGa.go hosted with by 9itE)b We added a new e5ported type called .nimal with two e5ported fields called Name and .ge. +hen we embed the .nimal type into the e5ported &og type. +his means that the &og type now has three e5ported fields* Name* .ge and !ark/trength. -et.s look at the implementation of the main programB ? 2 " 8 G H I L < ? 0 ? ?
package main import ( "fmt" "test/animals" ) func main() { // 5reate an obDect of type &og from the animals package. )og (7 animals.&og{ .nimal( animals..nimal{ Name( "5hole"+ .ge( 0+ %+ !ark/trength( 01+ %

"?"

? 2 ? " ? 8 ? G fmt.#rintf("5ounter( 'Ov*n"+ )og) ? % H ? I ? L ? < view raw e5portedtypesFGb.go hosted with by 9itE)b In main we )se a composite literal to create and initiali(e an ob0ect of the e5ported &og type. +hen we display the str)ct)re and val)es of the )og ob0ect. +o make things more interesting* let.s change the .nimal type from e5ported to )ne5ported by changing the first letter of the type.s name to a lowercase letter CaCB ? 2 " 8 package animals G animal represents information about all animals. H // type animal struct { I Name string L .ge int < % ? // &og represents information about )ogs. 0 type &og struct { ? animal !ark/trength int ? ? % 2 ? " view raw e5portedtypesFHa.go hosted with by 9itE)b +he animal type remains embedded in the e5ported &og type* b)t now as an )ne5ported type. We keep the Name and .ge fields within the animal type as e5ported fields. In the main program we 0)st change the name of the type from .nimal to animalB ? package main 2 import ( " "fmt" 8 "test/animals" "?8

G H I L < ? 0 ? ) ? ? func main() { // 5reate an obDect of type &og from the animals package. 2 // his will N3 compile. ? )og (7 animals.&og{ " animal( animals.animal{ Name( "5hole"+ ? .ge( 0+ 8 %+ ? !ark/trength( 01+ G % ? fmt.#rintf("5ounter( 'Ov*n"+ )og) H % ? I ? L ? < 2 0 view raw e5portedtypesFHb.go hosted with by 9itE)b 'nce again we have a main program that can.t compile beca)se we are trying to access the )ne5ported type animal from inside the composite literalB ./main.go(00( cannot refer to uneBporte) name animals.animal ./main.go(0J( unknown animals.&og fiel) CanimalC in struct literal We can fi5 the compiler error by initiali(ing the e5ported fields from the )ne5ported embedded type o)tside of the composite literalB ? 2 " 8 G H I L < ? 0 ?
package main import ( "fmt" "test/animals" ) func main() { // 5reate an obDect of type &og from the animals package. )og (7 animals.&og{ !ark/trength( 01+ % )og.Name 7 "5hole" )og..ge 7 0

"?G

? ? 2 ? " ? fmt.#rintf("5ounter( 'Ov*n"+ )og) 8 % ? G ? H ? I view raw e5portedtypesFI.go hosted with by 9itE)b &ow the main program b)ilds again. +he e5ported fields that were embedded into the &og type from the animal type are accessible* even tho)gh they came from an )ne5ported type. +he e5ported fields keep their e5ported stat)s when the type is embedded.

Standard Library
+he e5ported ime type from the time package is a good e5ample of a type from the standard library that provides no access to its internalsB ? type 2 " 8 G H I L < ? 0 ? ? ? 2 ? " ? 8 ? % G ? H ? I ? L ?
ime struct { // sec gives the number of secon)s elapse) since // 9anuary 0+ year 0 11(11(11 6 5. sec intSJ // nsec specifies a nonKnegative nanosecon) // offset within the secon) name) by /econ)s. // :t must be in the range U1+ QQQQQQQQQV. // // :t is )eclare) as uintptr instea) of intIE or uintIE // to avoi) garbage collector aliasing in the case where // on a SJKbit system the intIE or uintIE fiel) is written // over the low half of a pointer+ creating another pointer. // 3&3(rsc)( When the garbage collector is completely // precise+ change back to intIE. nsec uintptr // loc specifies the Location that shoul) be use) to // )etermine the minute+ hour+ month+ )ay+ an) year // that correspon) to this ime. // 3nly the Nero ime has a nil Location. // :n that case it is interprete) to mean 6 5. loc $Location

"?H

< 2 0 2 ? 2 2 2 " 2 8 view raw e5portedtypesFL.go hosted with by 9itE)b +he lang)age designers are )sing the )ne5ported fields to keep the internals of the ime type private. +hey are KhidingK the information so we can.t do anything contrary to how the time data works. With that being said* we still can )se the )ne5ported fields thro)gh the interface they provide. Witho)t the ability to )se and access )ne5ported fields indirectly* we wo)ld not be able to copy ob0ects of this type or embed this type into o)r own )serCdefined types.

$oncl'sion
# solid )nderstanding of how to hide and provide access to data from o)r packages is important. +here is a lot more to e5porting and )ne5porting identifiers than meets the eye. In setting o)t to write this post* I tho)gh a co)ple of e5amples wo)ld do the trick. +hen I reali(ed how involved the topic can get once we start looking at embedding )ne5ported types into o)r own types. +he ability to )se e5ported or )ne5ported identifiers is an implementation detail* one that 9o give )s fle5ibility to )se in o)r programs. +he standard library has great e5amples of )sing )ne5ported identifiers to hide and protect data. We looked at one e5ample with the time. ime type. +ake the time to look at the standard library to learn more.

Introd'ction To ?'meric $onstants In Go

Introd'ction
'ne of the more )ni2)e feat)res of 9o is how the lang)age implements constants. +he r)les for constants in the lang)age specification are )ni2)e to 9o. +hey provide the fle5ibility 9o needs at the compiler level to make the code we write readable and int)itive while still maintaining a type safe lang)age. +his post will attempt to b)ild a fo)ndation for what n)meric constants are* how they behave in their simplest form and how best to talk abo)t them. +here are a lot of little n)ances* terms and concepts that can trip )s )p. /eca)se of this* the post is going to take things slowly. 3o if yo) are ready to peek )nder the covers 0)st a bit* roll )p yo)r sleeve and let]s get startedB

.ntyped and Typed ?'meric $onstants


Constants can be declared with or witho)t a type in 9o. When we declare literal val)es in o)r code* we are act)ally declaring constants that are both )ntyped and )nnamed.

"?I

+he following e5amples show typed and )ntyped n)meric constants that are both named and )nnamedB con,t untype):nteger 7 12%?L con,t untype)Mloating#oint 7 %.1?1LT2 con,t type):nteger int 7 12%?L con,t type)Mloating#oint f(oat>? 7 %.1?1LT2 +he constants on the left hand side of the declaration are named constants and the literal val)es on the right hand side are )nnamed constants.

Iinds o6 ?'meric $onstants


:o)r first instinct may be to think that typed constants )se the same type system as variables* b)t they don]t. Constants have their own implementation for representing the val)es that we associate with them. 6very 9o compiler has the fle5ibility to implement constants as they wish* within a set of mandatory re2)irements. When declaring a typed constant* the declared type is )sed to associate the type]s precision limitations. It does not change how the val)e is being internally represented. /eca)se the internal representation of constants can be different between the different compilers* it is best to think of constants as having a kind* not a type. &)meric constants can be one of fo)r kindsB integer* floatingCpoint* comple5 and r)neB 12%?L // kin)( integer %.1?1LT2 // kin)( floatingKpoint 1E> // kin)( floatingKpoint In the e5ample above* we have declared three n)meric constants* one of kind integer and two of kind floatingCpoint. +he form of the literal val)e will determine what kind the constant takes. When the form of the literal val)e contains a decimal or e5ponent* the constant is of kind floatingCpoint. When the form does not contain a decimal or e5ponent* the constant is of kind integer.

$onstants )re 2athematically E@act


1egardless of the implementation* constants are always considered to be mathematically e5act. +his is something that makes constants in 9o )ni2)e. +his is not the case in other lang)ages like C and C!!. Integers can always be represented precisely when there is eno)gh memory to store their entire val)e. 3ince the specification re2)ires integer constants to have at least 2GH bits of precision* we are safe in saying integer constants are mathematically e5act. +o have mathematically e5act floatingCpoint n)mbers* there are different strategies and options that the compiler can employ. +he specification does not state how the compiler m)st do this* it 0)st specifies a set of mandatory re2)irements that need to be met.

"?L

Eere are two strategies that the different 9o compilers )se today to implement mathematically e5act floatingCpoint n)mbersB 'ne strategy is to represent all floatingCpoint n)mbers as fractions* and )se rational arithmetic on those fractions. +his is what go types does today and these floatingC point n)mbers never have any no loss of precision. #nother strategy is to )se floatingCpoint n)mbers with so m)ch precision that they appear to be e5act for all practical p)rposes. When we )se floatingCpoint n)mbers with several h)ndred bits* the difference between e5act and appro5imate becomes virt)ally nonCe5istent. +his is what the gc gccgo compilers do today. #s developers however* it is best to not consider what internal representation is being )sed by the compiler* it is irrelevant. O)st remember that all constants* regardless if they are declared with or witho)t a type* )se the same representation to store their val)es* which is not the same as variables and is mathematically e5act.

2athematically E@act E@ample


3ince constants only e5ist d)ring compilation* it is hard to provide an e5ample that shows constants are mathematically e5act. 'ne way is to show how the compiler will let )s declare constants of kind integer with val)es that are m)ch larger than the largest integer types can s)pport. Eere is a program that can be compiled beca)se constants of kind integer are mathematically e5actB package main import "fmt" // Much larger value than intSJ. con,t my5onst 7 T22%%72"%>DL?77LD"DL?%L22%?L f+nc main() { fmt.#rintln("Will 5ompile") % If we change the constant to be of type intSJ* which means the constant is now bo)nd to the precision limitations of a H8 bit integer* the program will no longer compileB package main import "fmt" // Much larger value than intSJ. con,t my5onst int>? 7 T22%%72"%>DL?77LD"DL?%L22%?L f+nc main() { fmt.#rintln("Will N3 % 1ompi(er Error: "?< 5ompile")

./i)eal.go(S( constant QEEIIRE1IS8FJRRF818FJIFEEIJF overflows intSJ Eere we can see that constants of kind integer can represent very large n)mbers and why we say they are mathematically e5act.

?'meric $onstant Declarations


When we declare an )ntyped n)meric constant* there are no type constraints that m)st be met by the constant val)eB con,t untype):nteger 7 12%?L // kin)( integer con,t untype)Mloating#oint 7 %.1?1LT2 // kin)( floatingKpoint In each case* the )ntyped constant on the left hand side of the declaration is given the same kind and val)e as the constant on the right. When we declare a typed constant* the constant on the right hand side of the declaration m)st )se a form that is compatible with the declared type on the leftB con,t type):nteger int 7 12%?L // kin)( integer con,t type)Mloating#oint f(oat>? 7 %.1?1LT2 // kin)( floatingK point +he val)e on the right hand side of the declaration m)st also fit into the range for the declared type. For instance* this n)meric constant declaration is invalidB con,t my6int8 +intD 7 1""" uint8 only can represent n)mbers from 0 to 2GG. +his is what I mean when I said earlier that the declared type is )sed to associate the type]s precision limitations.

Implicit Integer Type $on:ersions


In 9o there are no implicit type conversions between variables. Eowever* implicit type conversions between variables and constants can happen reg)larly by the compiler. -et]s start with an implicit integer conversionB var my:nt int 7 12% In this e5ample we have constant 0EI of kind integer being implicitly converted to a val)e of type int. 3ince the form of the constant is not )sing a decimal point or e5ponent* the constant takes the kind integer. Constants of kind integer can be implicitly converted into signed and )nsigned integer variables of any length as long as no tr)ncation needs to take place. Constants of kind floatingCpoint can also be implicitly converted into integer variables if the constant )ses a form that is compatible with the integer typeB var my:nt int 7 12%."

"20

We can also perform implicit type conversion assignments witho)t declaring an e5plicit type for the variableB var my:nt 7 12% In this case* the defa)lt type of intSJ is )sed to initiali(e the variable being assigned with constant 0EI of kind integer.

Implicit Cloating<Point Type $on:ersions


&e5t let]s look at an implicit floatingCpoint conversionB var myMloat f(oat>? 7 ".%%% +his time the compiler is performing an implicit conversion between constant 1.III of kind floatingCpoint to a variable of type floatSJ. 3ince the form of the constant is )sing a decimal point* the constant takes the kind floatingCpoint. +he defa)lt type for a variable initiali(ed with a constant of kind floatingCpoint is floatSJ. +he compiler can also perform implicit conversions between constants of kind integer to variables of type floatSJB var myMloat f(oat>? 7 1 In this e5ample* constant 0 of kind integer is implicitly converted to a variable of type floatSJ.

Iind Promotion
$erforming constant arithmetic between other constants and variables is something we do 2)ite often in o)r programs. It follows the r)les for binary operators in the specification. +he r)le states that operand types m)st be identical )nless the operation involves shifts or )ntyped constants. -et]s look at an e5ample of two constants that are m)ltiplied togetherB var answer 7 % 8 ".%%% In this e5ample we perform m)ltiplication between constant I of kind integer and constant 1.III of kind floatingCpoint. +here is a r)le in the specification abo)t constant e5pressions that is specific to this operationB 1 2&ept for shift operation) if the operands of a binary operation are different .inds of untyped &onstants) ...) the result use the .ind that appears later in this list: integer) rune) floating!point) &omple2.1 /ased on this r)le* the res)lt of the m)ltiplication between these two constants will be a constant of kind floatingCpoint. Kind floatingCpoint is being promoted ahead of kind integer "2?

based on the r)le.

?'meric $onstant )rithmetic


-et]s contin)e with o)r m)ltiplication e5ampleB var answer 7 % 8 ".%%% +he res)lt of the m)ltiplication will be a new constant of kind floatingCpoint. +hat constant is then assigned to the variable answer thro)gh an implicit type conversion from kind floatingCpoint to floatSJ. When we divide n)meric constants* the kind of the constants determine how the division is performed. con,t thir) 7 1 : %." When one of the two constants are of kind floatingCpoint* the res)lt of the division will also be a constant of kind floatingCpoint. In o)r e5ample we have )sed a decimal point to represent the constant in the denominator. +his follows the r)les for kind promotion that we talked abo)t before. -et]s take the same e5ample b)t )se kind integer in the denominatorB con,t Nero 7 1 : % +his time we are performing division between two constants of kind integer. +he res)lt of the division will be a new constant of type integer. 3ince dividing " into the val)e of ? represents a n)mber that is less than ?* the res)lt of this division is constant 1 of kind integer. -et]s create a typed constant )sing n)meric constant arithmeticB t=pe Numbers intD con,t 3ne Numbers 7 1 con,t wo 7 2 8 3ne Eere we declare a new type called Numbers with a base type of int8. +hen we declare constant 3ne with type Numbers and assign constant 0 of kind integer. &e5t we declare constant wo which is promoted to type Numbers thro)gh the m)ltiplication of constant E of kind integer and constant 3ne of type Numbers. +he declaration of constant wo shows an e5ample of a constant getting promoted not 0)st to a )serCdefined type* b)t a )serCdefined type associated with a base type.

(ne Practical E@ample


-et]s look at one practical e5ample right from the standard library. +he time package declares this type and set of constantsB t=pe &uration int>?

"22

con,t ( Nanosecon) &uration Microsecon) Millisecon) /econ) )

7 7 7 7

1 1""" 8 Nanosecon) 1""" 8 Microsecon) 1""" 8 Millisecon)

#ll of the constants declared above are constants of type &uration which have a base type of intSJ. Eere we are declaring typed constants )sing constant arithmetic between constants that are typed and )ntyped. 3ince the compiler will perform implicit conversions for constants* we can write code in 9o like thisB package main import ( "fmt" "time" ) con,t five/econ)s 7 L 8 time./econ) f+nc main() { now (7 time.Now() lessMiveNanosecon)s (7 now..))(BL) lessMive/econ)s (7 now..))(Kfive/econ)s) fmt.#rintf("Now ( 'v*n"+ now) fmt.#rintf("Nano ( 'v*n"+ lessMiveNanosecon)s) fmt.#rintf("/econ)s ( 'v*n"+ lessMive/econ)s) % 5+tp+t: Now ( E10JK1IKER 0I(I1(JQ.0001I8I8J K1J11 @& Nano ( E10JK1IKER 0I(I1(JQ.0001I8IRQ K1J11 @& /econ)s ( E10JK1IKER 0I(I1(JJ.0001I8I8J K1J11 @& +he power of constants are e5hibited with the method calls to .)). -et]s look at the definition of the .)) method for the receiver type imeB f+nc (t ime) .))() &uration) ime

+he .)) method accepts a single parameter of type &uration. -et]s look closer at the method calls to .)) from o)r programB var lessMiveNanosecon)s 7 now..))(BL) var lessMiveMinutes 7 now..))(Kfive/econ)s) +he compiler is implicitly converting constant KF into a variable of type &uration to allow

"2"

the method call to happen. Constant five/econ)s is already of type &uration thanks to the r)les for constant arithmeticB con,t five/econ)s 7 L 8 time./econ) +he m)ltiplication between constant F and time./econ) res)lts in constant five/econ)s becoming a constant of type &uration. +his is beca)se constant time./econ) is of type &uration and this type is promoted when determining the type of the res)lt. +o s)pport the f)nction call* the constant still needs to be implicitly converted from a constant of type &uration to a variable of type &uration. If constants didn.t behave the way they do* these kind of assignments and f)nction calls wo)ld always re2)ire e5plicit conversions. -ook at what happens when we try to )se a val)e of type int to make the same method callB var )ifference int 7 BL var lessMiveNano 7 now..))()ifference) 1ompi(er Error: ./const.go(0S( cannot use )ifference (type int) as type time.&uration in function argument 'nce we )se a typed integer val)e as the parameter for the .)) method call* we received a compiler error. +he compiler will not allow implicit type conversions between typed variables. For that code to compile we wo)ld need to perform an e5plicit type conversionB .))(time.&uration()ifference)) Constants are the only mechanism we have to write code witho)t the need to )se e5plicit type conversions.

$oncl'sion
We take the behavior of constants for granted* which is a testament to the lang)age designers and those who have worked hard on this feat)re. # lot of work and care has gone into making constants work this way and the benefits are hopef)lly clear. 3o the ne5t time yo) are working with a constant* remember yo) are working with something that is )ni2)e. # hidden gem b)ried in the compiler that doesn]t get eno)gh credit or recognition as a )ni2)e feat)re of 9o. Constants help make coding in 9o f)n and the code we write readable and int)itive. While at the same time keeping the code we write type safe.

Thanks
+hanks to &ate Finch and Kim 3hrier who have provided several reviews of the post that have helped to make s)re the content and e5amples were acc)rate* flowed well and wo)ld be interesting to 9o developers. I was ready to give )p a few times and &ate]s enco)ragement kept me going.

"28

Das könnte Ihnen auch gefallen