Linux tips and tricks

System configuration
Bash shotcuts | Bash command output and status | Bash script | Set system time | ls aliases | File permission | Process limitation | Command line mail | Fuser | pkill | System bell | Computer startup | System log | Network configuration | Wireless network | Fat32 | xinitrc, Xclients and xsession | Collect hardware informations | yum / rpm | rescue and recovery | decimal separator | PBS / TORQUE | 32 / 64 bits |

File renaming and conversion
File renaming | Replace space in filename | Convert pdf to png | Convert pdf to booklet | Print .pdf with several page per output page (pdfnup) | Decrease PDF file size | Extract pictures from pdf | Convert eps to png | Convert mp3 to wav | Convert ogg to wav | Convert wma to wav | Convert 3gp to avi | pdfnup | Save video stream to disk | Remove transparent background from png picture |

Misc
Screenshots | Awk | nvidia | CD burning | CD copy | DVD burning | Gtk2 fonts | Nedit | wget | Cron | Gtklp | SSH port forwarding | SSH login without password | Search and dig | Search and destroy | Search and list | Search and archive | Search and copy | Wipe and erase hard disk | System version |


Bash shortcuts

!!          recall the last command
!cmd         recall the last command starting by cmd
!$          last argument of the last command
CTRL + r    search in the last commands
CTRL + a    jump to the start of the line
CTRL + e    jump to the end of the line
CTRL + u    delete backward from cursor
CTRL + k    delete forward from cursor
CTRL + w    delete the word before the cursor

Bash command output and status

Store in variables output and status of a bash command

output=$(command)
status=$?

A command succeeds if status code = 0

Bash script

Automate the use of a program with a bash script:
#! /bin/bash

program > results << !BLOC!
param1
param2
param3
!BLOC!
The tags !BLOC! delineate parameters (param1, param2 and param3) send to program. Output is send to results file.

Change system time or date

Define date and time manually:

date -s "MM/JJ/AAAA HH:MM:SS"

One can also synchronize to a time server (ntp package), for instance pool.ntp.org or europe.pool.ntp.org. The configuration of ntp is done in /etc/ntp.conf. The time synchronization daemon is loaded via /etc/init.d/ntpd restart or /etc/init.d/ntp restart
To check if the synchonization is working: ntpq -np, columns 'when' and 'reach' should not be null or undefined. List of French time servers

ls aliases

Color accordong to file types and add '/' after directory names: alias ls="ls --color=auto -F"
Sort by modification date (last modified file last listed): alias lt="ls -rt"

File permissions

Change file permissions to rwx: chmod 777 file.test
Change directory access permissions recursively: find . -type d -print0 | xargs -0 chmod g+x
Binary ----- Logic --- Decimal
000 -------- (---) ------- 0
001 -------- (--x) ------- 1
010 -------- (-w-) ------- 2
011 -------- (-wx) ------- 3
100 -------- (r--) ------- 4
101 -------- (r-x) ------- 5
110 -------- (rw-) ------- 6
111 -------- (rwx) ------- 7
chown to change owner: chown group file or chown login.group file or chown login file
chgrp to change group

Processus number limitation (avoid fork bombs)

Check for limitations: limit To add limitations, append the file /etc/pam.d/login with:
session    required     /lib/security/pam_limits.so
And edit /etc/security/limits.conf and add:
@users          hard    nproc           23
@users          hard    data            2000
@users          hard    nofile          20
@users          hard    stack           4000
@users          hard    core            8000
@users          hard    fsize           100000

Command line mail

echo "done" | mail -s "process in $HOSTNAME done" user@domain.com
For a more sophisticated output, one can use a bash script:
#! /bin/bash

mail -s "error with $USER" root@domain.com << !message!
Hello,
There is an error with $USER on $HOSTNAME computer.
Can you fix it ?
By by
!message!

Wich process uses which files?

List processes using the sound card:
/sbin/fuser -v /dev/dsp0
List processes using the cdrom drive (usefull for a clean unmount):
/sbin/fuser -v /mnt/cdrom
Option -k instead of -v automatically kills processes found.

pkill, more powerfull than killall

Kill firefox: pkill firefox or also pkill firef

Stop system bell

From the text consol: setterm -blength 0
From the X windows consol: xset b off or xset -b
In the .bashrc file, add the line set bell-style none

Load a program after computer startup

On Linux systems, manual additions have to be done in /etc/rc.d/rc.local or /etc/rc.local since this file is loaded after at the end and after all paths and definitions are performed.

System log

System log is in /var/log/message and can be print on screen by:
color-xterm -e tail -f /var/log/message &
or
color-xterm -ls -geometry 80x5+45+705 -rv -sb -name "System messages" -fn 5x7 -T "System messages" -e tail -f /var/log/messages &
One can also redirect syslog outputs toward a terminal (for instance tty8). Add kern.* /dev/tty8 in /etc/syslog.conf.

Network configuration

ifconfig       : list network interfaces "up" (activated)
ifconfig -a    : list all network interfaces
dhclient eth0  : get an ip address from DHCP for eth0 interface
ifup eth0      : activate eth0 interface

Wirelesse network configuration (wifi)

List all network interfaces and label wireless interfaces: ipwconfig
Scan reachable wifi networks: iwlist eth0 scanning

Assign files from a FAT32 partition to specific user

Modify the file /etc/fstab with the uid and gid of the user. For instance:
/dev/hda1 /mnt/win_c vfat iocharset=iso8859-15,codepage=850,umask=0,uid=501,gid=501 0 0
The uid and gid may be found in /etc/passwd (first two numbers of the user line).

Difference between .xinitrc, .Xclients and .xsession

If X windows is launched by startx, .xinitrc is read first, and then .Xclients. If X windows is automatically launched and a user session is opened, .xsession is read followed by .Xclients. Scripts .xsession and .xinirc can be symbolic links to .Xclients. Whatever script is used, it has to be executable (chmod u+x file).

Collect hardware informations

cat /proc/cpuinfo gives information of the CPU(s) (number of processors, model and flags)
cat /proc/cpuinfo |grep processor |wc -l outputs the number of cpus.

/sbin/lspci gives information on the PCI bridges
/sbin/lspci |grep 'PCI Express x16' outputs the PCI bridge for video card (PCI Express 16x)

/sbin/lspci |grep VGA gives info on the video card
glxinfo |grep OpenGL gives more info on nVidia video card

cat /proc/meminfo |grep 'MemTotal' outputs the total RAM

fdisk -l (as root) lists all connected hard drives (internal and external)

yum / rpm

Install the package test: yum install test
Install the package test.rpm: rpm -ivh test.rpm

Remove the package test: yum remove test or rpm -e test

List packages available for installation: yum list
List installed packages: yum list installed or rpm -qa
List all packages (installed and available): yum list all
List updated packages: yum list updates
Search the package test: yum list *test*

Find if package test is installed : yum list installed test or rpm -qa |grep test
List and location of files contained in the package test.rpm: rpm -qlp test.rpm
List and location of files contained in the installed package test: rpm -ql test
Find which package provides the file /path/to/file: yum whatprovides /path/to/file or rpm -qf /path/to/file

Check the packages to be updated: yum check-update
Update the package test: yum update test
Update the package test.rpm: rpm -Uvh test.rpm
Update all installed packages: yum update
Clean all installed packages from disk cache: yum clean all

Rescue and recovery

Mount the read-only root partition in read-write mode: mount -n -o remount,rw /
Check disk (fsck) at next reboot: shutdown -F now
Scan for bad blocks on /dev/hda2 device: badblock -sv /dev/hda2

Redefine decimal separator

On French operating systems, the comma , is the default decimal separator. To use the dot . separator instead, declare LC_NUMERIC=C
Either redefine the decimal separator for a given program (for instance xmgrace):
alias xmgrace="LC_NUMERIC=C xmgrace"
Either redefine the decimal separator globally (in .bashrc file):
export LC_NUMERIC=C

PBS / TORQUE (cluster jobs management)

Get list of available nodes
pbsnodes -l free
Get list of active (working) nodes
pbsnodes -l active
Get number of available cores (assuming a cluster of 13 nodes x 16 cores)
qstat -n | sed 's/node/node\n/g' | grep -c node | awk '{print 13*16 - $1}'

32 / 64 bits

Find if your system is 32 or 64 bits:
uname -m
Find if a binary is compiled for 32 or 64 bits:
file -be elf /usr/bin/nedit
answer could be
ELF 64-bit LSB executable, x86-64, version 1 (SYSV)
or
ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV)

Batch file renaming

Rename all .txt files from the current directory into .dat
for i in *.txt; do mv $i ${i%txt}dat; done
or
rename 's/\.txt$/\.dat/' *.txt

Replace spaces by _ in file names

for i in *.dat; do mv "$i" `echo $i | tr ' ' '_'`; done

Convert the first page of a .pdf document into .png image

pdftoppm file.pdf page
convert page-000001.ppm page.png
rm page-000001.ppm

Convert a .pdf file to a booklet

pdf2ps -sPAPERSIZE=a4 file.pdf file.ps
psbook file.ps | psnup -2 | a2ps -1 -s tumble -o booklet.ps
Print on both side with the option "long edge (standard)". If the original document is not in the A4 format (for instance 14,88 cm x 20,98 cm), we have to do then:
pdf2ps file.pdf file.ps
psbook file.ps | psnup -2 -W18.88cm -H20.98cm -pa4 | a2ps -1 -s tumble -o booklet.ps

Print .pdf with several page per output page (pdfnup)

pdfnup input.pdf --nup 1x3 --frame true --scale 0.96
pdfnup input.pdf --nup 1x3 --frame true --scale 0.96 --outfile output.pdf

Decrease PDF file size (with gs)

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH -sOutputFile=output.pdf input.pdf
Verbosity can be decreased with the -dQUIET option

Extract pictures from .pdf file

With the packages xpdf-utils and imagemagik
pdfimages file.pdf picture
for i in picture-*.ppm; do convert $i ${i%ppm}png; done

Convert .eps or .ps file to .png

pstopnm -xborder=0 -yborder=0 -xsize=new-size-x -ysize=new-size-y -stdout file.eps > file.ppm
pnmtopng file.ppm > file.png
rm -f file.ppm
or
mogrify -density 120 -rotate 90 -format png file.ps

Convert .mp3 to .wav

for i in *.mp3; do lame --decode $i `basename $i .mp3`.wav; done

Convert .ogg to .wav

for i in *.ogg ; do ogg123 -d wav -f `basename $i .ogg`.wav $i; done

Convert .wma to .wav

mplayer file.wma -ao pcm -aofile file.wav
or
mplayer -vo null -vc dummy -af resample=44100 -ao pcm -waveheader file.wma -aofile file.wav

Convert .3gp movies to .avi

.3gp files are movies from mobile phone. To convert them into .avi files:
ffmpeg -i input.3gp -f avi -vcodec xvid -acodec mp3 output.avi
or with more advanced option :
ffmpeg -i video.3gp -b 250 -s 160×120 -r 15 -f avi -an video.avi

Convert .pdf to .pdf with 4 pages per page

pdfnup test.pdf --nup 2x2 --orient landscape --frame true

pdfnup is provided with the package pdfjam.

Save to disk a video stream

mplayer -dumpstream -dumpfile test.wmv -nocache mms://stream-address.wmv
or
mencoder mms://stream-address.wmv -oac copy -ovc copy -o toto.avi

Remove transparent background from png picture

convert -background white -flatten pic.png pic_white.png 

Sum the numbers of a given column with awk

With a file containing:
TestA       3        11
TestB      26        45
TestA       7        87
TestB      18        14
To sum numbers of the second column of all lines starting by "TestA":
awk '/TestA/ {sum+=$2; print sum}' file
To print numbers of the second column and their sum:
awk '/TestA/ {sum+=$2; printf "%d %d\n", $2, sum}' file

NVidia drivers setup

  1. type CTRL+ALT+F1 to open a text terminal
  2. connect to root
  3. type init 3
  4. type sh NVIDIA-Linuxxxxxxx.run to load the settup utility
  5. open /etc/X11/XF86config-4 and replace nv by nvidia
  6. type init 5

CD burning

With /dev/cdrom defined as CD-ROM drive device
Burn .iso file: cdrecord dev=/dev/cdrom speed=8 -v file.iso
Clean (blanck) CD-RW: cdrecord dev=/deV/cdrw blanck=fast

CD copy

mkisofs, then cdrecord
or
dd if=/dev/cdrom of=image.iso
or
cdda2wav -D /dev/cdrom -s -O wav -S 1à -B
cdrecord -v fs=6m speed=10 dev=0,0,0 -audio *.wav

DVD burning

With /dev/dvd the DVD-ROM drive.
Files:
growisofs -dvd-compat -speed=2 -Z /dev/dvd -R -J files
Iso image:
growisofs -dvd-compat -speed=2 -Z /dev/dvd=file.iso
Directory content:
growisofs -dvd-compat -speed=2 -Z /dev/dvd -R -J directory_name
One or several directories (with their directory name):
growisofs -dvd-compat -speed=2 -Z /dev/dvd -R -J -graft-points /dir1=/path/to/dir1 /dir2=/path/to/dir2
Clean DVD:
dvd+rw-format -force /dev/dvd
Note: is filenames are too long, use the -joliet-long option instead of -J

Gtk2 fonts

Create the file .gtkrc-2.0 in the home directory and append the line

gtk-font-name = "Arial 11"

Nedit scripts

Scripts have to be Unix type files (and not Dos type).

Wget

Download of a directory recursively: wget -r -np url

Contrab

List existing crontab: crontab -l
Edit contrab: contrab -e
Automatically append a new job to crontab:

crontab -l > old.cron
echo "# add new cron job at 3:00 AM" > append.cron
echo "0 3 * * * script.sh >& /tmp/script.log" >> append.cron
cat old.cron append.cron > new.cron
crontab new.cron

Screenshots

import -window root screenshot.jpg or import screenshot.jpg

Gtklp

gtklp is a Gtk-interface above cups that may be used on its own or via other software. The configuration of each printer is written in ~/.cups/lpoptions.
The set up of applications such as firefox, openoffice.org or acrobat reader to use gtklp is explained here (FR): http://doc.ubuntu-fr.org/gtklp

SSH port forwarding

Build a SSH connection directly from computerA to computerC via serverB (computerA — serverB — computerC):

ssh -L 2222:computerC:22 login@serverB
And in a different shell:
ssh -X localhost -p 2222

The local port (2222 here) must be > to 1024 since port < 1024 are reserved to root.

SSH login without password

To perform SSH login without password from computerA (user loginA) on computerB (user loginB).
First, create a public/private key pair on computer A:
ssh-keygen -t rsa
Accept all default choices and leave an empty passphrase.
Then, append the public key of computerA to the authorized keys of computerB:
ssh-copy-id -i ~/.ssh/id_rsa.pub computerB
Enter one last time your password of userB on computerB to accept the copy.
You can now connect via SSH from computerA to computerB without any password.

Search and dig

Search for text files (*.txt) and search into:

find ./path/ -name '*.txt' | xargs grep 'string'

Search and destroy

Search text files (*.txt) and remove:

find ./path/ -name '*.txt' -exec rm {} \;

Search and list

Search files with size bigger than 5 Mb and sort them by size:

find ./ -size +5M -type f -print0 | xargs -0 ls -Ssh

Search and archive

Search files modified less than last 2 days ago with size bigger than 100 kb and archive:

find ./path/ -type f -mtime -2 -size +100k | xargs tar -zcvf archive.tgz

Search and copy

Search all pdf files and copy them in a directory:

find ./path/ -name "*pdf" -print0 |xargs -0 cp -t ./directory/

Wipe and erase hard disk

shred is installed on most Linux systems.
shred -v -n 1 -z /dev/hdc

-v   : show progress
-n 1 : overwrite hard disk with one pass of random data
-z   : finish cleaning with one pass of zeros 
One can clean full hard disk (/dev/hdc) or single partition (/dev/hdc1).
More than 1 pass of random data is preferred but it also takes more time.

Obtain the version of Linux systems

lsb_release -a

Valid XHTML 1.0!