Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
command-line_ninjary [2013/03/06 23:40] – ishivverscommand-line_ninjary [2013/04/01 14:59] (current) – [Navigation] c.axen
Line 1: Line 1:
 +====== Command-line Ninjary GSPS 2013 ======
  
 +This is meant to be a quick reference guide to command-line usage for those of us 
 +who have the basics down, but aren't yet true ninjas.  Please add your own tips
 +or quick examples!  We spend a surprising amount of our lives inside a terminal,
 +so learn to love it and use it well.  Note: this page was written for ''bash'',
 +but much of this stuff should translate to ''csh'' or your shell of choice.  
 +
 +------------------------------------
 +
 +==== Navigation ====
 +General notes about navigation commands: First, "Ctrl" commands are the same as in the text editor emacs, so you may find that other emacs line navigation commands work as well. Second, if you are on a Unix machine, note that the line navigation commands work almost anywhere you can edit text - you will learn that accidentally if you start using these commands compulsively. Third, if you use "Ctrl" commands often, you may consider setting the "Caps Lock" key as an additional "Control" key - this is just a computer setting for Unix (e.g. System Preferences > Keyboard > Modifier Keys) but requires slightly more work on Windows (e.g. the AutoHotkey program).
 +
 +
 +  * ''cntrl-a'': go to beginning of line
 +  * ''cntrl-e'': go to end of line
 +  * ''cntrl-k'': delete everything to right
 +  * ''cntrl-u'': delete everything to left
 +  * ''cntrl-w'': delete word to left
 +  * (''option'' or ''cntrl'')-(''left'' or ''right''): move left/right. ''cntrl'' for linux, ''option'' for mac.
 +  * Ctrl-f: move forward one character
 +  * Ctrl-b: move backward one character
 +  * use backslash ''\'' to literally interpret any command characters
 +  
 +  * ''cd''  : go home
 +  * ''cd -'' : go back to previous working directory
 +  * ''~'' : stand-in for home directory
 +      * e.g.: ''ls ~/Documents''
 +  
 +  * ''!!'': repeat previous command
 +  * ''!xxxx'': repeat most recent command that starts with //xxxx//
 +  * ''cntrl-r xxxx'': search recursively through history for match to //xxxx//
 +      * keep typing to refine search==
 +      * ''cntrl-r'' again to cycle through matches
 +      * and hit ''enter'' to execute
 +  * ''^this^that'': rerun most recent command, but replace //this// with //that//
 +  * ''history'': shows history of recent commands
 +      * ''history | grep xxxx'': search for command //xxxx// in history
 +      * ''!xxxx'': repeat item number //xxxx// in history
 +  
 +  * ''cntrl-x-e'': invoke your editor to compose a hairy command
 +      * executes command upon exit
 +      * can be invoked in the middle of composing command
 +  
 +  * curly braces perform parameter expansion, inserting a copied string with contents of braces swapped
 +      * ''cp file.{txt,txt.backup}'': same as ''cp file.txt file.txt.backup''
 +      * ''cp file_{,copied}one.txt'': same as ''cp file_one.txt file_copied_one.txt''
 +      * ''rm file_{a,b,d,g}.txt'': same as ''rm file_a.txt file_b.txt file_d.txt file_g.txt''
 +
 +
 +
 +------------------------------------
 +
 +
 +
 +==== Really Useful Built-ins ====
 +
 +  * ''clear'': clear the terminal screen
 +  * ''cntrl-z'': shunt something to the background
 +      * ''fg'': to get it back
 +      * really helpful when in some other program like a text editor or python/IRAF terminal
 +  
 +  * ''man xxxx'': display manual for //xxxx//
 +  * (''which'' or ''whereis'') ''xxxx'': display path for //xxxx//
 +  
 +  * ''wc xxxx'': character, word, and line count on file //xxxx//
 +      * ''wc -l'': return only line count
 +  
 +  * ''cat xxxx'': display contents of file //xxxx//
 +  * ''head -n 10 xxxx'': display first 10 lines of file //xxxx//
 +  * ''tail -n 10 xxxx'': display last 10 lines of file //xxxx//
 +  
 +  * ''grep xxxx yyyy'': search for //xxxx// in file //yyyy//
 +      * //xxxx// can be simple string or regular expression
 +  * ''find path xxxx'': search for files with names matching //xxxx//
 +      * //path// is optional - defaults to working directory
 +      * //xxxx// can be simple string or regular expression
 +  
 +  * ''kill xxxx'': kill (end) process with process id //xxxx//
 +  * ''top'': display running processes and system info
 +      * ''q'': quit
 +      * ''k xxxx'': in linux, kills process with id //xxxx//
 +      * sadly, no equivalent to above on most macs
 +  * ''ps -e'': show all running processes
 +      * ''ps -e | grep xxxx'': search for //xxxx// amongst running processes
 +  
 +  * ''date'': print date & time
 +      * can choose output format - i.e.: ''date +%H:%M:%S''
 +  * ''crontab'': fully-functional command scheduler
 +      * good examples [[http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples/|here]]
 +  * ''at'': a simple one-off scheduler
 +      * ''cmd | at time'' : schedule //cmd// to run at //time//
 +      * i.e.: ''ls ~/* > ~/home_dir.list | at 3:05pm''
 +      * understands pretty broad range of //time// definitions
 +  
 +  * ''screen'': like VNC but only for the terminal, lets you start a session and then disconnect and reconnect at will
 +      * ''screen'' to connect
 +      * ''screen -d'' to disconnect
 +      * ''screen -r'' to reconnect
 +      * ''screen -list'' to show current sessions
 +      * ''screen -r xxx'' to reconnect to session with PID //xxx//
 +      * great for ''ssh'' sessions
 +      * full guide [[http://www.cyberciti.biz/tips/how-to-use-screen-command-under-linux.html|here]]
 +
 +==  Get A Package Manager ==
 +
 +Best way to install and manage command-line tools. Native for all linux builds; have a couple good choices for macs.
 +
 +  * [[http://mxcl.github.com/homebrew/|Homebrew]]
 +  * [[http://www.macports.org/|MacPorts]]
 +
 +
 +
 +------------------------------------
 +
 +
 +
 +==== Pipes & Redirects ====
 +
 +=== Invoking multiple commands ===
 +  * ''cmd &'' : run //cmd// in subshell
 +  * ''cmd1 ; cmd2'' : run //cmd1// then run //cmd2//
 +  * ''cmd1 && cmd2'' : run //cmd2// after //cmd1// only if //cmd1// is successful
 +  * ''cmd1 || cmd2'' : run //cmd2// after //cmd1// only if //cmd1// is not successful
 +
 +=== Piping stuff around ===
 +  * ''cmd > file'' : sends output of //cmd// to //file//
 +      * overwrites //file//
 +      * ''%%cmd >> file%%'' : appends output of //cmd// to //file//
 +  
 +  * ''cmd1 | cmd2'' : connects stdout of //cmd1// to stdin of //cmd2//
 +  
 +  * ''cmd 2> file'': sends error of //cmd// to //file//
 +      * ''%%cmd 2>> file%%'' : appends error of //cmd// to //file//
 +      * e.g.: ''ls not.a.file* 2> output; cat output''
 +  * ''cmd > output 2> errors'' : save output and errors of //cmd// separately, in files //output// and //errors//
 +      * e.g.: ''ls * not.a.file* > success.out 2> failure.err''
 +  
 +  * ''cmd < file'' : sends //file// to //cmd//
 +
 +
 +== Useful tools when piping ==
 +
 +  * ''sort'': sorts stuff alphanumerically
 +      * ''ls | sort''
 +  
 +  * ''xargs cmd'' : a piping glue - runs //cmd// with standard input as arguments
 +      * can be used with a pipe or with a file
 +      * use it to translate between lines in a file or pipe and arguments to a function
 +      * best to understand by example
 +      * ''echo one two three > test.txt; xargs echo < test.txt''
 +      * ''echo -e 'one\ntwo\nthree' > test.txt; xargs echo < test.txt''
 +      * ''ls | sort | xargs echo''
 +  
 +  * ''awk'': programming language to edit pipes on the fly
 +      * basic usages are powerful, learn more [[http://www.gnu.org/software/gawk/manual/gawk.html|here]]
 +      * ''awk '{print $1}'': print the first item of each line, items split by spaces/tabs
 +      * ''awk -F, '{print $2}'': print the second item of each line, items split by '',''
 +  * Example: search for all running python instances, find their //PID//, and kill them
 +      * ''ps -e | grep python | awk '{print $1}' | xargs kill''
 +  * Example: count and display all the items in current working directory
 +      * ''ls | awk '{sum+=1; print sum ": " $1}'''
 +  * Example: replace every occurrence of ''NA'' with the number ''0'' in a file
 +      * ''cat file.txt | awk '{gsub("NA", "0"); print $0}' > clean.file.txt''
 +
 +------------------------------------
 +
 +
 +==== Loops & Functions ====
 +
 +=== Variables ===
 +Variables that have already been declared are prefaced by a ''$'', but you don't need
 +one when first declaring a variable.  Any variables declared normally (''x=4'') are considered strings,
 +to declare a number use the ''let'' keyword: ''let x=4''.  You can also declare a variable to be a command by 
 +wrapping it in back-ticks or parentheses.  I.E. ''LIST=(ls); $LIST''.
 +There are some system-wide variables that should always be available, like ''$SHELL'', ''$PATH'', and ''$HOME''.
 +Note that though convention says system-wide variables should be all caps, they don't need to be.
 +
 +=== Simple Loops ===
 +There are several ways to write loops, the most useful of which are below.
 +Note that in bash an ''$'' always indicates a variable, and that all variables are,
 +by default, treated as strings.  The actions to be repeated must be wrapped in 
 +''do/done'' statements, and the semicolon marks an end-of-statement (equivalent to a line return).
 +A few examples:
 +
 + for i in {1..20}
 +   do
 +     echo $i
 +   done
 +
 + for i in {a,b,d,g}; do echo $i; done
 +
 + for i in a 2 b 6;
 +   do
 +     echo $i;
 +   done
 +
 + for file in *filePattern*;
 +   do
 +     echo file;
 +   done
 +
 +
 +=== Simple Tests ===
 +
 +The shell can do arithmetic and logic tests, but I think if you're doing
 +anything very complex you should move into ''Python'' or ''Perl''.  Logic comparisons
 +should be put in square brackets (spacing is important), ''if'' statements must
 +be closed with a ''fi'', and the operators you have at your access are:
 +
 +  * ''=='': string equality
 +  * ''!='': string inequality
 +  * ''-eq'': number equality
 +  * ''-ne'': number inequality
 +  * ''-gt'': greater than
 +  * ''-lt'': less than
 +  * ''-ge'': greater than or equal
 +  * ''-le'': less than or equal
 +  * ''&&'': and
 +  * ''||'': or
 +
 +
 +== Use more aliases! ==
 +Use aliases to make hard-to-remember commands less terrible.
 +
 +Example:
 +http://xkcd.com/1168/:
 +
 +  * ''alias maketar='tar -czvf '''
 +  * ''alias untar='tar -xzvf '''
 +
 +
 +== Defining your own functions ==
 +
 +You can declare functions and use them at any time.  The most common way
 +to interact with functions is through required arguments, as shown below.
 +Adding in flags (like ''-v'') for optional arguments takes some more work
 +(look up ''getopts'').
 +
 +  * ''$#'': number of parameters passed
 +  * ''$@'': array of all parameters
 +  * ''$N'': the Nth parameter passed to the function, with $0 being the function name
 +
 +Some examples:
 +
 + function speak () {
 +   echo here are all $# arguments: $@
 +   echo here is argument 1: $1
 +   echo here is argument 3: $3
 + }
 +
 + function killme () {
 +   ps -e | grep $1 | awk '{print $1;}' | xargs kill
 +   echo $1 is dead
 + }
 +
 + function cls () {
 +   cd $1
 +   ls
 + }
 +
 +
 +== .profile or .bashrc ==
 +
 +There are a set of system files in your home directory that are executed every time
 +a terminal is opened - use those to define aliases, functions, or variables
 +you want always available.

Trace: