Eective packaging

Petr Uzel
L3/Maintenance puzel@suse.cz ptr uzl @ IRC

About this talk

collection of tips and tricks that make (some) package maintenance tasks more ecient no rocket science, no cool technologies based on experience there are denitely better ways

2/1

c Novell Inc. All Right Reserved

Which package is broken?

given a bug (e.g. /path/to/bin crashes), sometimes it is not obvious which package should be xed
bugzilla does not track package info

rpm -qf /path/to/file gives name of the RPM we need source package, not RPM $ rpm -qf /lib64/libblkid.so.1 libblkid1-2.18-56.1.x86_64 $ osc search -s --package blkid No matches found for blkid in packages
3/1 c Novell Inc. All Right Reserved

Source package name

lets ask RPM:
$ rpm -q --queryformat %{SOURCERPM}\n \ libblkid1-2.18-56.1.x86_64 util-linux-2.18-56.1.src.rpm Everything combined together (le source package) $ rpm -q --queryformat %{SOURCERPM}\n \ $(rpm -qf /lib64/libblkid.so.1) | \ sed -e s/\(^.*\)-.*-.*src.rpm/\1/ util-linux Where did I install the package from? $ rpm -q --queryformat %{DISTURL}\n util-linux obs://build.opensuse.org/home:puzel:staging/openSUSE_11.3/6a[CUT]
4/1 c Novell Inc. All Right Reserved

Get package source from OBS

With branching: slow: osc branch openSUSE:Factory foo; osc co home:*:branches:PRJ faster: osc branch --checkout openSUSE:Factory foo fastest: osc getpac foo or osc bco foo hyperspeed: osc getpac + osc bash completion Without branching: function show_devel { osc meta pkg openSUSE:Factory ${1} | fgrep devel } go directly to devel project
5/1 c Novell Inc. All Right Reserved

Building package

osc build in package working copy usually does the right thing more control: osc build [OPTS] REPOSITORY ARCH SPECFILE REPOSITORY, ARCH and SPECFILE detected automatically if not specied list REPO-ARCH combinations: osc repos [PRJ]

6/1

c Novell Inc. All Right Reserved

osc build options

build debuginfo and debugsource packages: osc build --debuginfo/-d skip post build checks: osc build --nochecks delete old build root before build: osc build --clean use alternative project for build: osc --alternative-project=openSUSE:Factory use ccache to speed up build: osc build --ccache

7/1

c Novell Inc. All Right Reserved

Use custom RPMs in buildroot

foo depends on bar, which is not in buildservice yet

but we have bar.rpm

osc build -p/--prefer-pkgs DIR: prefer packages from DIR when installing buildroot $ cd foo $ osc build -p /path/to_dir_with_bar.rpm foo.spec osc build -x/--extra-pkgs PKG: add this package to buildroot

8/1

c Novell Inc. All Right Reserved

Buildroot tips
by default, buildroot is /var/tmp/build-root OSC BUILD ROOT overrides the default $ export OSC_BUILD_ROOT=/var/tmp/build-root-temp $ osc build use when parallelly building multiple packages

buildroot on tmpfs 5% speedup of parted build $ grep build-root /etc/fstab tmpfs /var/tmp/build-root tmpfs
9/1 c Novell Inc. All Right Reserved

size=5G

Avoid password for building

by default, osc build asks for root password; to avoid it set su-wrapper = sudo in ~/.oscrc add yourlogin ALL = (root) NOPASSWD: /usr/bin/build to sudoers le

10/1

c Novell Inc. All Right Reserved

Build log

build log is stored in $OSC BUILD ROOT/.build.log slow: less /var/tmp/build-root/.build.log faster: osc lbl/localbuildlog
respects $OSC BUILD ROOT from working copy

osc rbl/remotebuildlog PRJ PKG REPO ARCH - get remote build log

11/1

c Novell Inc. All Right Reserved

Version update workow

1 2 3 4 5 6 7 8 9 10 11

get package source download tarball verify tarball repack tarball check dierences adjust specle refresh patches run test build test the package write changelog entry commit
c Novell Inc. All Right Reserved

12/1

Downloading tarball with ncftp

many projects publish tarballs on FTP sites ncftp is a command-line FTP client with bookmarking support
useful to remember locations where upstream publishes tarballs

supported by bash-completion $ ncftp ut<tab> -> ncftp util-linux ncftp... ls ncftp... get fil<tab> -> filename ncftpbookmarks: bookmark editor

13/1

c Novell Inc. All Right Reserved

Repacking tarball

packaging policy requires the tarball to be bzip2-ed (or xz-ed) sometimes upstream provides only .tar.gz $ bznew parted-2.3.tar.gz $ ls *.bz2 parted-2.3.tar.bz2

14/1

c Novell Inc. All Right Reserved

Whats new in tarball?

Compare outputs from configure --help $ git clone git://gitorious.org/opensuse/pack-tools.git $ diffconf gnupg-2.0.10.tar.bz2 gnupg-2.0.16.tar.bz2

15/1

c Novell Inc. All Right Reserved

Change specle

usually just open specle and increase the version number small helper: $ alias vs=vim -p *.spec in vim (normal mode), press Ctrl+A/Ctrl+X to increase/decrease numbers

16/1

c Novell Inc. All Right Reserved

Packaging with quilt

about quilt

Quilt allows you to easily manage large numbers of patches by keeping track of the changes each patch makes. Patches can be applied, un-applied, refreshed, and more.

17/1

c Novell Inc. All Right Reserved

Packaging with quilt

quilt setup

initialize a source tree from RPM spec le

1 2 3

unpack tarball apply patches un-apply patches (but remembers the series)

IOW: %prep phase $ quilt setup -v foo.spec $ cd foo-1.0.0 shortcut: $ alias qs=quilt setup -v *.spec
18/1 c Novell Inc. All Right Reserved

Packaging with quilt

working with series of patches

now we are in the directory created by quilt setup (unpacked tarball) none of the patches is applied (but quilt knows about them) quilt push applies one patch (-a: all patches) quilt pop un-applies one patch (-a: all patches) quilt top prints name of the last applied patch quilt refresh refreshes last patch (so that it applies cleanly, without fuzz and osets) quilt series prints the names of all patches

19/1

c Novell Inc. All Right Reserved

Packaging with quilt

inspecting patches

$ $ $ $ $

quilt quilt quilt quilt quilt

diff diff --color=auto or quilt diff | colordiff files or quilt diff | lsdiff diff | diffstat diff -U num

view dierences in vimdi (side by side):

$ alias qdv=quilt diff -p0 -C 9999 --color=never --diff=vimdiff

20/1

c Novell Inc. All Right Reserved

Packaging with quilt

creating new patch

1 2

quilt push -a quilt new foo-fix-bug.patch quilt add src/file.c vim src/file.c hack, hack quilt refresh

steps 3 and 4 can be combined to quilt edit src/file.c the patch appears in parent directory (after quilt refresh) you have to add it to specle

3 4 5 6

21/1

c Novell Inc. All Right Reserved

Packaging with quilt

importing patch

goal: import patch (e.g. taken from upstream) to quilt

1 2 3 4

quilt push -a quilt import /path/to/bugfix.patch quilt push quilt refresh bugfix.patch appears in the parent directory -p num controls how many directory levels to strip (as with patch(1)) -P name overrides name of the patch multiple patches can be imported at once

22/1

c Novell Inc. All Right Reserved

Packaging with quilt

rejected patches

when quilt push fails, use quilt push -f applies as much as possible, leaves rejects (file.c + file.c.rej)
$ quilt push Applying patch gnupg-files-are-digests.patch patching file g10/gpg.c Hunk #3 succeeded at 1998 (offset -4 lines). Hunk #4 succeeded at 2487 (offset -4 lines). patching file g10/options.h Hunk #1 FAILED at 194. 1 out of 1 hunk FAILED -- rejects in file g10/options.h patching file g10/sign.c Patch gnupg-files-are-digests.patch does not apply (enforce with -f)

23/1

c Novell Inc. All Right Reserved

Packaging with quilt

rejected patches - continued

failed hunks in *.c.rej have to be applied manually (vim *.c) then quilt refresh helper: vim quilt plugin (zypper install vim-plugin-quilt) (Contrib) vim foo.c - if there is also foo.c.rej - opens vertically split view easier to copy code from rejected hunks to original le the plugin can do more: :help quilt

24/1

c Novell Inc. All Right Reserved

Packaging with quilt

make quilt more friendly

$ $ $ $ $

alias alias alias alias alias

q=quilt qd=quilt diff --color=always qdv=quilt diff -p0 -C 9999 --color=never --diff=vimdiff qp=quilt push --color=auto qs=quilt setup -v *.spec

$ cat ~/.quiltrc # show C function name in hunk header QUILT_DIFF_OPTS=-p

25/1

c Novell Inc. All Right Reserved

osc status, osc addremove

we have to tell osc about new/deleted les example: update foo-1.0 to foo-2.0
$ ls foo*bz2 foo-1.0.tar.bz2 foo-2.0.tar.bz2 $ osc status ? foo-2.0.tar.bz2 $ rm foo-1.0.tar.bz2 $ osc ar # osc addremove D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 $ osc st # osc status D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 osc add + osc remove = osc ar
26/1 c Novell Inc. All Right Reserved

Committing changes
osc diff displays changes in the package more readable: $ alias osccd=osc diff | colordiff | less osc vc opens foo.changes le (prepares new entry header, unless -e option is given) vim can open compressed tarballs:
:tabnew foo-2.0.tar.bz2 search for Changelog, NEWS, . . . ; open copy, paste and edit changelog entry

osc commit osc sr

27/1 c Novell Inc. All Right Reserved

Vim and changelogs

to make changelog editing with vim more convenient, add to .vimrc:

set nocompatible filetype plugin on syntax on autocmd BufRead,BufNewFile *.changes.* set filetype=changes autocmd FileType changes set spelllang=en_us spell textwidth=67

28/1

c Novell Inc. All Right Reserved

osc bash completion

supports completing:
osc subcommand names (checkout, commit, . . . ) global long osc options (e.g. --quiet, --debug, . . . ) long subcommand options project names package names (in given project) le names (in given project/package) repository names (openSUSE Factory, . . . ) ...

http://gitorious.org/opensuse/osc-bash-completion git clone git://gitorious.org/opensuse/osc-bash-completion.git see README

29/1 c Novell Inc. All Right Reserved

osc bash completion

examples

osc osc osc osc osc osc

com<tab> => osc commit build --a<tab> => osc build --alternative-project ls Ar<tab> => osc ls Archiving ls Base:System pa<tab> => osc ls Base:System parted meta a<tab> => osc meta attribute request <tab> => osc request accept approvenew co decline checkout list log revoke show wipe

...

30/1

c Novell Inc. All Right Reserved

Browsing source code

packager often needs to understand code written by somebody else

application crashes patch does not apply to new version build fails

there are tools that signicantly help:

ctags cscope

both work with C/C++, ctags supports many other languages zypper install ctags cscope

31/1

c Novell Inc. All Right Reserved

Browsing source code

ctags

ctags creates database of symbols in the source code tree editor (or other tools) use this database to quickly search for symbols and jump to the denition to create the database (tags le): $ cd foo-1.0.0 $ ctags -R . vim automatically uses ctags database if present Ctrl+] / Ctrl+L click jump to denition of symbol under cursor Ctrl+t / Ctrl+R click jump back :tag symbol jump to denition of symbol vim -t symbol open vim and jump directly to denition of symbol :help ctags
32/1 c Novell Inc. All Right Reserved

Browsing source code

vim + ctags + bash completion

sometimes you forget exact name of the symbol bash completion becomes handy: $ ctags -R . $ vim -t sym<tab> => vim -t symbol_name_full
http://vim.wikia.com/wiki/Using bash completion with ctags and Vim put it to ~/.bash completion zypper install bash completion

33/1

c Novell Inc. All Right Reserved

Browsing source code

cscope

ctags can only jump to symbol denition; cscope can do more whith cscope, it is possible to quickly lookup e.g.
where is the symbol dened (same as ctags) where is the symbol used what functions call this function what functions are called by this function which les include this le ...

support in vim http://cscope.sourceforge.net/cscope vim tutorial.html

34/1

c Novell Inc. All Right Reserved

Browsing source code

taglist

source code browser plugin for vim provides overview of the structure of source code les zypper install vim-plugin-taglist (Contrib) set up taglist binding in .vimrc: nnoremap <silent> <F8> :Tlist<CR>

35/1

c Novell Inc. All Right Reserved

Browsing source code

taglist screenshot

36/1

c Novell Inc. All Right Reserved

Patches from Fedora

do not reinvent wheel - before writing patch, make sure it does not already exist Fedora uses git now: # Credits: Pavol Rusnak function fedora_getpkg() { git clone git://pkgs.fedoraproject.org/${1}.git }

37/1

c Novell Inc. All Right Reserved

Patches from git

. . . with tig

tig is ncurses interface for git (zypper install tig) run tig from the cloned repository, press h for help to nd and extract patch form git repository, add the following to /.tigrc: bind generic P !git format-patch %(commit)^..%(commit) run tig (e.g. tig v2.2..) select the patch you want to extract hit P - the patch will be saved as sth like 0001-fix-name.patch
38/1 c Novell Inc. All Right Reserved

RPM macros
sometimes not obvious what rpm %macro does rpmbuild --showrc lists all dened RPM macros rpm -E %macro prints expanded macro supported by bash completion:
$ rpm -E %la<tab> => rpm -E %lang_package %package lang Summary: Languages for package %{name} Group: System/Localization Requires: %{name} = %{version} Provides: %{name}-lang-all = %{version} Supplements: packageand(bundle-lang-other:%{name}) BuildArch: noarch %description lang Provides translations to the package %{name}
39/1 c Novell Inc. All Right Reserved

EOF

Questions?

40/1

c Novell Inc. All Right Reserved