Beruflich Dokumente
Kultur Dokumente
o Multitasking
o Remote tasking ("real networking")
o Multiuser
o Access to shell, programming languages, databases, open-source projects
o Better performance, less expensive (free), more up-to-date
o Many more reasons
UNIX variants
LINUX distributions
Handy shortcuts
$ . # refers to pwd
$ ~/ # refers to user's home directory
$ history # shows all commands you have used recently
$ !<number> # starts an old command by providing its ID number
$ up(down)_key # scrolls through command history
$ <incomplete path/file_name> TAB # completes path/file_name
$ <incomplete command> SHIFT&TAB # completes command
$ Ctrl a # cursor to beginning of command line
$ Ctrl e # cursor to end of command line
$ Ctrl d # delete character under cursor
$ Ctrl k # delete line from cursor, content goes into kill buffer
$ Ctrl y # paste content from Ctrl k
3. UNIX HELP
4. FINDING THINGS
$ df # disk space
$ free # memory info
$ uname -a # shows tech info about machine
$ bc # command-line calculator (to exit type 'quit')
$ wget ftp://ftp.ncbi.nih.... # file download from web
$ /sbin/ifconfig # give IP and other network info
$ ln -s myfilename # creates symbolic link to myfilename, so that it can be
opened from any directory
$ du -sh # displays disk space usage of current directory
$ du -sh * # displays disk space usage of individual files/directories
$ du -s * | sort -nr # shows disk space used by different directories/files
sorted by size
11 JOB/PROCESS MANAGEMENT
11 TEXT VIEWING
$ less <my_file> # more versatile text viewer than 'more', 'G' moves to end of
text, 'g' to beginning, '/' find forward, '?' find backwards
$ more <my_file> # views text, use space bar to browse, hit 'q' to exit
$ cat <my_file> # concatenates files and prints content to standard output
11 TEXT EDITORS
VI and VIM
Non-graphical (terminal-based) editor. Vi is guaranteed to be available on any
system. Vim is the improved version of vi.
EMACS
Window-based editor. You still need to know keystroke commands to use it.
Installed on all Linux distributions and on most other Unix systems.
XEMACS
More sophisticated version of emacs, but usually not installed by default. All
common commands are available from menus. Very powerful editor, with built-in
syntax checking, Web-browsing, news-reading, manual-page browsing, etc.
PICO
Simple terminal-based editor available on most versions of Unix. Uses keystroke
commands, but they are listed in logical fashion at bottom of screen.
VIM MANUAL (essentials marked in red)
BASICS
$ vim my_file_name # open/create file with vim
$ i # INSERT MODE
$ ESC # NORMAL (NON-EDITING) MODE
$ : # commands start with ':'
$ :w # save command; if you are in editing mode you have to hit ESC first!!
$ :q # quit file, don't save
$ :q! # exits WITHOUT saving any changes you have made
$ :wq # save and quit
$ R # replace MODE
$ r # replace only one character under cursor
$ q: # history of commands (from NORMAL MODE!), to reexecute one of them,
select and hit enter!
$ :w new_filename # saves into new file
$ :#,#w new_filename # saves specific lines (#,#) to new file
$ :# go to specified line number
HELP
$ Useful list of vim commands: Vim Commands Cheat Sheet, VimCard, Vim
Basics
$ vimtutor # open vim tutorial from shell
$ :help # opens help within vim, hit :q to get back to your file
$ :help <topic> # opens help on specified topic
$ |help_topic| CTRL-] # when you are in help this command opens help
topic specified between |...|, CTRL-t brings you back to last topic
$ :help <topic> CTRL-D # gives list of help topics that contain key word
$ : <up-down keys> # like in shell you get recent commands!!!!
MOVING AROUND IN FILE
$ $ # moves cursor to end of line
$ A # same as $, but switches to insert mode
$ 0 (zero) # moves cursor to beginning of line
$ CTRL-g # shows at status line filename and the line you are on
$ SHIFT-G # brings you to bottom of file, type line number (isn't displayed) then
SHIFT-G # brings you to specified line#
DISPLAY
WRAPPING AND LINE NUMBERS
$ :set nowrap # no word wrapping, :set wrap # back to wrapping
$ :set number # shows line numbers, :set nonumber # back to no-number
mode
WORKING WITH MANY FILES & SPLITTING WINDOWS
$ vim *.txt # opens many files at once; ':n' switches between files
$ :wall or :qall # write or quit all open files
$ vim -o *.txt # opens many files at once and displays them with horizontal
split, '-O' does vertical split
$ :args *.txt # places all the relevant files in the argument list $ :all # splits
all files in the argument list (buffer) horizontally $ CTRL-w # switch between
windows
$ :split # shows same file in two windows
$ :split <file-to-open> # opens second file in new window
$ :vsplit # splits windows vertically, very useful for tables, ":set scrollbind" let's
you scroll all open windows symultaneously
$ :close # closes current window
$ :only # closes all windows except current one
SPELL CHECKING & Dictionary
$ aspell -c <file> # shell command
$ aspell -l <my_file> # shell command
$ :! dict <word> # meaning of word
$ :! wn 'word' -over # synonyms of word
PRINTING FILE
$ :ha # prints entire file
$ :#,#ha # prints specified lines: #,#
MERGING/INSERTING FILES
$ :r <filename> # inserts content of specified file after cursor
UNDO/REDO
$ u # undo last command
$ U # undo all changes on current line
$ CTRL-R # redo one change which was undone
DELETION/CUT (switch to NORMAL mode)
$ x # deletes what is under cursor
$ dw # deletes from curser to end of word including the space
$ de # deletes from curser to end of word NOT including the space
$ cw # deletes rest of word and lets you then insert, hit ESC to continue with
NORMAL mode
$ c$ # deletes rest of line and lets you then insert, hit ESC to continue with with
NORMAL mode
$ d$ # deletes from cursor to the end of the line
$ dd # deletes entire line
$ 2dd # deletes next two lines, continues: 3dd, 4dd and so on.
PUT (PASTE)
$ p # uses what was deleted/cut and pastes it behind cursor
COPY & PASTE
$ yy # copies line, for copying several lines do 2yy, 3yy and so on
$ p # pastes clipboard behind cursor
SEARCH IN FILE (most regular expressions work)
$ /my_pattern # searches for my_pattern downwards, type n for next match
$ ?my_pattern # seraches for my_pattern upwards, type n for next match
$ :set ic # switches to ignore case search (case insensitive)
$ :set hls # switches to highlight search (highlights search hits)
REPLACE WITH REGULAR EXPRESSIONS (great intro: A Tao of Regular Expressions)
$ :s/old_pat/new_pat/ # replaces first occurence in a line
$ :s/old_pat/new_pat/g # replaces all occurence in a line
$ :s/old_pat/new_pat/gc # add 'c' to ask for confirmation
$ :#,#s/old_pat/new_pat/g # replaces all occurence between line
numbers: #,#
$ :%s/old_pat/new_pat/g # replaces all occurence in file
$ :%s/\(pattern1\)\(pattern2\)/\1test\2/g # regular expression to
insert, you need here '\' in front of parentheses (<# Perl)
$ :%s/\(pattern.*\)/\1 my_tag/g # appends something to line containing
pattern (<# .+ from Perl is .* in VIM)
$ :%s/\(pattern\)\(.*\)/\1/g # removes everything in lines after pattern
$ :%s/\(At\dg\d\d\d\d\d\.\d\)\(.*\)/\1\t\2/g # inserts tabs
between At1g12345.1 and Description
$ :%s/\n/new_pattern/g #Replaces return signs
$ :%s/pattern/\r/g #Replace pattern with return signs!!
$ :%s/\(\n\)/\1\1/g # insert additional return signs
$ :%s/\(^At\dg\d\d\d\d\d.\d\t.\{-}\t.\{-}\t.\{-}\t.\
{-}\t\).\{-}\t/\1/g # replaces content between 5th and 6th tab (5th
column), '{-}' turns off 'greedy' behavior
$ :#,#s/\( \{-} \|\.\|\n\)/\1/g # performs simple word count in
specified range of text
$ :%s/\(E\{6,\}\)/<font color="green">\1<\/font>/g # highlight
pattern in html colors, here highlighting of >= 6 occurences of Es
$ :%s/\([A-Z]\)/\l\1/g # change uppercase to lowercase, '%s/\([A-
Z]\)/\u\1/g' does the opposite
$ :g/my_pattern/ s/\([A-Z]\)/\l\1/g | copy $ # uses 'global'
command to apply replace function only on those lines that match a certain
pattern. The 'copy $' command after the pipe '|' prints all matching lines at the
end of the file.
$ :args *.txt | all | argdo %s/\old_pat/new_pat/ge | update #
Command 'args' places all relevant files in the argument list (buffer); 'all' displays
each file in separate split window; command 'argdo' applies replacement to all
files in argument list (buffer); flag 'e' is necessary to avoid stop at error messages
for files with no matches; command 'update' saves all changes to files that were
updated.
MATCHING PARENTHESES SEARCH
- place curser on (, [ or { and type % # curser moves to matching parentheses
HTML EDITING
-Convert text file to html format:
$ :runtime! syntax/2html.vim # run this command with open file in Vim
SHELL COMMAND IN VIM
$ :!<SHELL_COMMAND> <ENTER> # executes any shell command, hit <enter>
to return
$ :sh # switches window to shell, 'exit' switches back to vim
USING VIM AS TABLE EDITOR
$ v # starts visual mode for selecting characters
$ V # starts visual mode for selecting lines
$ CTRL-V # starts visual mode for selecting blocks (use CTRL-q in gVim under
Windows). This allows column-wise selections and operations like inserting and
deleting columns. To restrict substitude commands to a column, one can select it
and switch to the command-line by typing ':'. After this the substitution sytax for a
selected block looks like this: '<,'>s///.
$ :set scrollbind # starts simultaneous scrolling of 'vsplitted' files. To set to
horizontal binding of files, use command ':set scrollopt=hor' (after first one). Run
all these commands before the ':split' command.
$ :AlignCtrl I= \t then :%Align # This allows to align tables by column
separators (here '\t') when the Align utility from Charles Campbell's is installed.
To sort table rows by selected lines or block, perform the visual select and then
hit F3 key. The rest is interactive. To enable this function one has to include in
the .vimrc file from Gerald Lai the Vim sort script.
MODIFY VIM SETTINGS (in file .vimrc)
- see last chapter of vimtutor (start from shell)
- useful .vimrc sample
- when vim starts to respond very slowly then one may need to delete the .viminf*
files in home directory
When you log into UNIX/LINUX the system starts a program called SHELL. It provides
you with a working environment and interface to the operating system. Usually there are
many different shell programs installed.
$ finger <user_name> # shows which shell you are using
$ chsh -l # gives list of shell programs available on your system (does not
work on all UNIX variants)
$ <shell_name> # switches to different shell
STDIN, STDOUT, STDERR, REDIRECTORS, OPERATORS & WILDCARDS (more on
this @ LINUX HOWTOs)
By default, many UNIX commands read from standard input (STDIN) and send
their output to standard out (STDOUT). You can redirect them by using the
following commands:
$ file* # * is wildcard to specify many files
$ ls > file # prints ls output into specified file
$ command < my_file # uses file after '<' as STDIN
$ command >> my_file # appends output of one command to file
$ command | tee my_file # writes STDOUT to file and prints it to screen;
alternative way to do this:
$ command > my_file; cat my_file
$ command > /dev/null # turns off progress info of applications by redirecting
their output to /dev/null
$ grep my_pattern my_file | wc # Pipes (|) output of 'grep' into 'wc'
$ grep my_pattern my_non_existing_file 2 > my_stderr # prints
STDERR to file
Useful shell commands
$ cat <file1> <file2> > <cat.out> # concatenate files in output file
'cat.out'
$ paste <file1> <file2> > <paste.out> # merges lines of files and
separates them by tabs (useful for tables)
$ cmp <file1> <file2> # tells you whether two files are identical
$ diff <fileA> <fileB> # finds differences between two files
$ head -<number> <file> # prints first lines of a file
$ tail -<number> <file> # prints last lines of a file
$ split -l <number> <file> # splits lines of file into many smaller ones
$ csplit -f out fasta_batch "%^>%" "/^>/" "{*}" # splits fasta
batch file into many files at '>'
$ sort <file> # sorts single file, many files and can merge (-m) them, -b
ignores leading white space, ...
$ sort -k 2,2 -k 3,3n input_file > output_file # sorts in table
column 2 alphabetically and column 3 numerically, '-k' for column, '-n' for numeric
$ sort input_file | uniq > output_file # uniq command removes
duplicates and creates file/table with unique lines/fields
$ join -1 1 -2 1 <table1> <table2> # joins two tables based on
specified column numbers (-1 file1, 1: col1; -2: file2, col2). It assumes that join
fields are sorted. If that is not the case, use the next command:
$ sort table1 > table1a; sort table2 > table2a; join -a 1 -t
"`echo -e '\t'`" table1a table2a > table3 # '-a <table>' prints all
lines of specified table! Default prints only all lines the two tables have in
common. '-t "`echo -e '\t'`" ->' forces join to use tabs as field separator in its
output. Default is space(s)!!!
$ cat my_table | cut -d , -f1-3 # cut command prints only specified
sections of a table, -d specifies here comma as column separator (tab is default),
-f specifies column numbers.
$ grep and egrep # see chapter 4
Useful One-Liners
$ perl -p -i -w -e 's/pattern1/pattern2/g' input_file # replace
something (e.g. return signs) in file using regular expressions; use $1 to
backreference to pattern placed in parentheses
'-p' lets perl know to write program; '-i.bak' creates backup file *.bak, only -i
doesn't; '-w' turns on warnings; '-e' executeable code follows
$ perl -ne 'print if (/my_pattern1/ ? ($c=1) : (--$c > 0)) ;
print if (/my_pattern2/ ? ($d = 1) : (--$d > 0))' my_infile
> my_outfile # parses lines that contain pattern1 and pattern2
following lines after pattern can be specified in '$c=1' and '$d=1'; for OR function
use this syntax: '/(pattern1|pattern2)/'
Examples
Copy file from Server to Local Machine (type from local machine prompt):
$ scp user@remote_host:file.name . # '.' copies to pwd, you can specify
here any directory, use wildcards to copy many files at once.
Copy file from Local Machine to Server:
$ scp file.name user@remote_host:~/dir/newfile.name
Copy entire directory from Server to Local Machine (type from local machine
prompt):
$ scp -r user@remote_host:directory/ ~/dir
Copy entire directory from Local Machine to Server (type from local machine
prompt):
$ scp -r directory/ user@remote_host:directory/
Copy between two remote hosts (e.g. from bioinfo to cache):
similar as above, just be logged in one of the remote hosts:
$ scp -r directory/ user@remote_host:directory/
NICE FTP
$ open ncftp
$ ncftp> open ftp.ncbi.nih.gov
$ ncftp> cd /blast/executables
$ ncftp> get blast.linux.tar.Z (skip extension: @)
$ ncftp> bye
11 UNPACKING FILES
options:
f: use archive file
p: preserve permissions
v: list files processed
x: exclude files listed in FILE
z: filter the archive through gzip
Systems-wide installations
Installations for systems-wide usage are the responsibility of system
administrator
To find out if an application is installed, type:
$ which <application_name>
$ whereis <application_name> # searches for executeables in set of
directories, doesn't depend on your path
Most applications are installed in /usr/local/bin or /usr/bin. You need root
permissions to write to these directories.
Perl scripts go into /usr/local/bin, Perl modules (*.pm) into
/usr/local/share/perl/5.8.0/. To copy executables in one batch, use command: cp
`find -perm -111 -type f` /usr/local/bin
Applications in user accounts
Create a new directory, download application into this directory, unpack it (see
chapter 13) and follow package-specific installation instructions.
Usually you can then already run this application when you specify its location
e.g.: /home/user/my_app/blastall.
If you want you can add this directory to your PATH by typing from this directory:
$ PATH=.:$PATH; export PATH # this allows you to run application by
providing only its name; when you do echo $PATH you will see .: added to
PATH.
Intstallation of RPMs
$ rpm -i application_name.rpm
To check which version of RPM package is installed, type:
$ rpm --query <package_name>
Help and upgrade files for RPMs can be found at http://rpmfind.net/.
Installation of Debian packages
Check whether your application is available at: http://www.debian.org/intro/about,
then you type (no download):
$ apt-cache search phylip #searches for application "phylip" from
command line
$ apt-cache show phylip #provides description of program
$ apt-get install phylip # example for phylip install, manuals can be
found in /usr/doc/phylip/, use zless or lynx to read documentation (don't unzip).
$ apt-get update # do once a month do update Debian packages
$ apt-get upgrade -u # to upgrade after update from above
$ dpkg -i # install data package from local package file (e.g. after download)
111 DEVICES
Mount/unmount floppy/cdrom
$ mount /mnt/floppy
$ mount /mnt/cdrom
$ eject /mnt/floppy
111 ENVIRONMENT VARIABLES