PortingyourcodetoPython3

Presented by
Alexandre Vassalotti

 Overview
Introduction
What's new?
10 minutes break
Migrating to Python 3
Conclusion

Introduction

WhatisPython3?
Notacompleterewrite
Abackwardincompatiblerelease
Cleanupoldwarts
Thispresentationisabout:
Themajorchanges
Howtoportyourcode.

 What'snew?
print is a function
Keyword-only arguments
Unicode throughout
New I/O library
Standard library reorganization
Iterators and views
Special methods
Syntax changes

printisnowafunction!

Notabigdeal
Moreflexible
Thestringsperatoriscustomizable
>>>print("value=",number,sep="")
value=34
Youcanoverridethefunction
importbuiltins
builtins.print=my_custom_logger

printisnowafunction!

Theweird>>sys.stderrsyntaxisgone

Python2
print>>sys.stderr,"systemfailure!"

Python3
print("systemfailure!",file=sys.stderr)

Keywordonlyarguments

Thekeywordneedstobeexplicitlywrittenout.
Neededforvariadicfunctions.
defprint(*args,file=sys.stdout):
This is valid syntax
... in Python 3!

Usefulforforcinguserstostatetheirintent.
my_list.sort(key=lambdax:x[1])

sorted(my_list,reverse=True)

Keywordonlyarguments

Syntax
defsorted(iterable,*,reverse=False,key=None):
...

The bare * indicates the

following arguments are
keyword-only.

Beware:theerrormessageissurprising!
>>>sorted([1,2],True)
Traceback(mostrecentcalllast):
File"<stdin>",line1,in<module>
TypeError:sorted()takesexactly1positionalargument(2given)

 What'snew?
print is a function
Keyword-only arguments
Unicode throughout
New I/O library
Standard library reorganization
Iterators and views
Special methods
Syntax changes

Unicode

AllstringsuseUnicodebydefault.

Python2 Python3
u"helloworld" "helloworld"

ur"\.write\(.*?\)" r"\.write\(.*?\)"

unicode(anything) str(anything)

Unicode:bytesdatatype
Newbytes()datatypefordata
b"thisisdata"

>>>bytes([1,2,3,4])
b'\x01\x02\x03\x04'

>>>bytes("hh","utf8")
b'h\xc3\xa9h\xc3\xa9'

>>>b"hello"+"world"

TypeError:can'tconcatbytestostr

Unicode:bytearraydatatype

Mutableversionformorefancyoperations
b=bytearray(20)
Only 20 bytes are
file.readinto(b) read into the buffer.

b.reverse()
b+=b"helloworld"
b[1]=0

Unicode

Thedistinctionbetweendataandtextisnot
alwaysclear.
ManysystemAPIsacceptbytesaswell.
>>>os.listdir(os.getcwd())
['eggs','monkey','spam']
>>>os.listdir(os.getcwdb())
[b'eggs',b'foo\x8f',b'monkey',b'spam']

Unicode:Otherimprovements

repr()nolongerescapesnonASCII
characters.Itstillescapesnonprintableand
controlcharacters,however.
Python2
>>>"all!\n"
Recall that repr() is implicitly
'all\xc3\xb4!\n' called at the interpreter prompt.

Python3
The old behaviour is
>>>"all!\n" still available if you
need it.
'all!\n'
>>>ascii("all!\n")

"'all\\xf4!\\n'"
Unicode:Otherimprovements

NonASCIIidentifiersaresupported
defhol(,):
return+*360
Butdon'tusethem!
Bewareofcharactersthatlookslikelatinletters.
>>>ascii("l")
"'\\u0455\\u0440\\u0435\\u0441\\u0456\\u0430l'"

NewI/Olibrary

DesignedwithUnicodeinmind.
CurrentlybeingrewritteninCforperformance.
Goodnews:youdon'thavetothinkaboutit.
withopen("readme.txt","w")asf:
f.write("hello")

open("encoded.txt","r",encoding="latin1")

NewI/Olibrary

3layers:raw,bufferedandtext.
Greatwaytoreusecode.
classStringIO(io.TextIOWrapper):
def__init__(self,initial_value=""):
super().__init__(io.BytesIO(),encoding="utf16")
self.write(initial_value)
self.seek(0)
defgetvalue(self):
returnself.buffer.getvalue().decode("utf16")

 What'snew?
print is a function
Keyword-only arguments
Unicode throughout
New I/O library
Standard library reorganization
Iterators and views
Special methods
Syntax changes

Standardlibraryreorganization

Removethe"sillyoldstuff"
ModulesrenamedtobePEP8conformant.
2to3handlesmostoftheworkforyou.

Standardlibraryreorganization

Python2 Python3
import_winreg importwinreg

importConfigParser importconfigparser

importcopy_reg importcopyreg

importQueue importqueue

importSocketServer importsocketserver

import__builtin__ importbuiltins

importrepr importreprlib

importtest.test_support importtest.support
PEP 8 violations

Poorly choosen names

Standardlibraryreorganization

Python2 Python3
try:
importcStringIOasStringIO importio

exceptImportError:
importStringIO

try:
importcPickleaspickle importpickle

exceptImportError:
importpickle Use the optimized
implementations
automatically

Standardlibraryreorganization

Python2 Python3
importHTMLParser importhtml.parser

importhtmlentitydefs importhtml.entities

importxmlrpclib importxmlrpc.client

importDocXMLRPCServer importxmlrpc.server

importSimpleXMLRPCServer

importdbhash importdbm.bsd

importdbm importdbm.ndbm

importgdbm importdbm.gnu

importanydbm importdbm

importwhichdb

Standardlibraryreorganization

Somemoduleswereremoved:compiler,
popen2htmllib,sgmllib,urllib,
md5,andmanymore.
2to3doesnothandlethese.
Rewriteyourcodetoavoidthedeprecated
modules.
SeePEP3108forreplacements

Standardlibraryreorganization

Sidenote:pickledataneedtoberegenerated.
$python2.6
>>>importpickle
>>>pickle.dump(map,open("test.pickle","wb"))
>>>pickle.load(open("test.pickle","rb"))
<builtinfunctionmap>

$python3.0
>>>importpickle
>>>pickle.load(open("test.pickle","rb"))
Traceback(mostrecentcalllast):
...
ImportError:Nomodulenamed__builtin__

Iteratorsandviews

ManyAPIsnolongerreturnlists.
dict.keys(),.values()and.items()
returnviews.
>>>{1:0}.keys()
<dict_keysobjectat0x7ffdf8d53d00>
Aviewisasetlikeobject.
fornodein(graph.keys()current_node):
...

Iteratorsandviews

map(),filter(),zip()returniterators.
Python2 Python3
a=map(lambdax:x[1],items) a=[x[1]forxinitems]

fornameinmap(str.lower,names): nochange
...

a=filter(lambdan:n%2==0,nums) a=[nforninnumsifn%2==0]

forkeyinfilter(str.isdigit,keys): nochange
...

dict(zip(sins,persons))

nochange
Iteratorsandviews

xrange()isthenewrange().
Nochangesareneededformostcode.

 What'snew?
print is a function
Keyword-only arguments
Unicode throughout
New I/O library
Standard library reorganization
Iterators and views
Special methods
Syntax changes

Specialmethods:slicing

__getslice__andfriendsarenolonger
supported.
Use__getitem__instead.
classArray:

def__getitem__(self,x):

ifisinstance(x,slice):

start,stop,step=x.indices(len(self))

...

else:

try:

index=x.__index__()

exceptAttributeError:

raiseTypeError("indicesmustbeintegers")

...
Specialmethods:richcomparaisons

3waycomparaisonsaregone.
Python2
classNumber:
...
def__cmp__(self,other):
ifself.value==other.value:
return0
elifself.value<other.value:
return1
else:
return1
...

Specialmethods:richcomparaisons

Python3
classNumber:
...
def__eq__(self,other):
returnself.value==other.value
def__lt__(self,other):
returnself.value<other.value:
def__gt__(self,other):
returnself.value>other.value:
def__le__(self,other):
returnself.value<=other.value:
def__ge__(self,other):
returnself.value>=other.value:

...
 What'snew?
print is a function
Keyword-only arguments
Unicode throughout
New I/O library
Standard library reorganization
Iterators and views
Special methods
Syntax changes

Syntaxchanges:exceptions

Python2 Python3
try: try:

withopen(fn,'r')asf: withopen(fn,'r')asf:

lines=list(f) lines=list(f)

except(IOError,OSError),err: except(IOError,OSError)aserr:

log_error(err) log_error(err)

This variable does

not leak anymore.

Syntaxchanges:relativeimports

json/
|encoder.py
|decoder.py
|__init__.py

Inthe__init__.pyfile:

from.encoderimportJSONEncoder
from.decoderimportJSONDecoder

Syntaxchanges:setanddictcomprehension

Newsyntaxforsetliterals
{1,3,5}
set() No syntax for empty sets.

Setcomprehension
{xforxiniterable}
Dictionarycomprehension
{k:vfork,viniterable}

Syntaxchanges:manyotherniceties

Extendediterableunpacking
a,b,*c=(1,2,3,4,5)
Thenonlocaldeclarationforaccessing
variablesinouterscopes.
Functionannotations
defreadinto(b:bytearray)>int:
...

 MigratingtoPython3
Introduction
Migration strategy
Runtime warnings
Backported features
2to3 source code translator

Introduction

Thereismorethanonewaytodoit.
PortingCextensionsisanotherbeast.

Migrationstrategy

1.Improveyourtestsuite.
2.PortyourcodetoPython2.6
3.EnablePython3warnings
4.Fixallthewarnings
5.Modernizeyourcode
6.Run2to3

Codemodernization

Reducethesemanticgap
Decreasetheamountofwork2to3hastodo.
Examples:
Usedict.iterkeys(),xrange(),etc
Avoidimplictstrandunicodecoercion
Prefer__getitem__over__getslice__

RuntimeWarnings

python2.63scriptname.py
Warnaboutfeaturesthatwereremovedin
Python3.
Warnaboutchanges2to3cannothandle
automatically.

Runtimewarnings

Demo

Backportedfeatures

ManyfeaturesofPython3areavailablein2.6
NewI/Olibrary
Unicodeandbytesliterals
from__future__importunicode_literals

Futurebuiltinfunctions
fromfuture_builtinsimportmap,zip,hex

Newsyntaxforcatchingandraisingexceptions
ABCs,newastmodule,advancedstring
formatting,richcomparaisons,etc

Backportedfeatures

Demo

2to3sourcecodetranslator

Convertfilesordirectories
Generateaunifieddiffformattedpatch
2to3project/>python3.patch
Canalsofixdoctests
2to3dtests.py
Fixerscanberunindividually

2to3sourcecodetranslator

Limitations
Handleonlysyntactictransformationsi.e,
thereisnotypeinference.
Cannotfixthingslike:
m=d.has_key attr="has_key"
ifm(key): ifgetattr(d,attr)(key):
... ...

eval("d.has_key(key)")

2to3sourcecodetranslator

Demo

Upcomingchanges

%styleformattingmaybecomedeprecated.
Performanceimprovements.
Newimportlibmodule.

Conclusion

Python3haslotofnewfeatures.
Therearemanytoolsavailabletohelpyou
duringthetransition.
Sendbugreportstohttp://bugs.python.org/
Subscribetothepythonportingmailinglistfor
additionalhelp.