Useful Bash tips

[n473]

Do any of you have any simple and useful Bash tips? Minor scripts, one-liners, etc. Would love to hear about them.

I'll get us started.

I recently discovered that Bash can handle substitutions on the previously-used command:

$ cat /etc/resolve.cnf
    cat: resolve.cnf: No such file or directory
$ ^cnf^conf
    cat /etc/resolve.conf

The syntax is pretty straight-forward; you're defining a pattern found in the previous command, and then you're defining a replacement of that pattern. Kinda like sed.

But it can be used for way more than mere typo-correction. Like when you're copying files:

$ cp /home/derp/herp/sherp/dog.txt /opt/dog/txt/
$ cp ^dog^cat
    cp /home/derp/herp/sherp/cat.txt /opt/cat/txt/

And speaking of last-used commands, '!!' can be used as an "alias" for the last-used command:

$ visudo
    visudo: /etc/sudoers: Permission denied
$ sudo !!
    sudo visudo

 

[Bar0n]

Not so much a tip as an enhancement, but install tmux to open multiple terminals within one terminal window.

 

[DrJohnZoidberg]

Some distro's have a command for long list (ll), some don't which really annoys me. It's easy to create your own though.

printf '#!/bin/bash\nls -alh' > /usr/local/bin/ll | chmod +x /usr/local/bin/ll

It's also useful having a long list less (lll), which paginates if you have a long directory listing, you could just do this for that one:

printf '#!/bin/bash\nls -alh | less' > /usr/local/bin/lll | chmod +x /usr/local/bin/lll

Now you have two new commands which are very quick to access without having to type in a bunch of extra parameters.

 

[Sinbad]

Do any of you have any simple and useful Bash tips? Minor scripts, one-liners, etc. Would love to hear about them.


And speaking of last-used commands, '!!' can be used as an "alias" for the last-used command:

$ visudo
    visudo: /etc/sudoers: Permission denied
$ sudo !!
    sudo visudo

$_ is the last file you worked on

so
$ vi ./script.sh
$ $_
will run the script.
$ cat $_
will cat it as well.

 

[n473]

$_ is the last file you worked on

so
$ vi ./script.sh
$ $_
will run the script.
$ cat $_
will cat it as well.

That... is something I had no idea about. This is going to have a vast impact on my workflow. Thank you for this!

 

[Nod]

Some distro's have a command for long list (ll), some don't which really annoys me. It's easy to create your own though.

printf '#!/bin/bash\nls -alh' > /usr/local/bin/ll | chmod +x /usr/local/bin/ll

It's also useful having a long list less (lll), which paginates if you have a long directory listing, you could just do this for that one:

printf '#!/bin/bash\nls -alh | less' > /usr/local/bin/lll | chmod +x /usr/local/bin/lll

Now you have two new commands which are very quick to access without having to type in a bunch of extra parameters.

Would an alias not work "better"?

alias ll='ls -alh | less'

Put that in your .bashrc file.

 

[n473]

Another thing worth mentioning is your ssh config file. Many people seem to forget about this, and if you find yourself using ssh on a daily basis, it's well worth setting up some things like ControlMasters and aliases.

I wrote an article on ssh tips a while ago. I'm not generally a fan of blog spam, so feel free to not click on that link. I'll paste the important bits here:

general aliasing and host configuration

You can break down each host entry into a single configuration stanza. Within this stanza you can pass specific configuration options on a per-host basis. For example:

Host myhost

will alias "myhost" to whatever you specify for the Hostname parameter, so if you type the command ssh user@myhost it will connect you to that hostname.

Hostname 10.0.0.1

specifies the IP you want bound to the name used in Host.

IdentityFile ~/.ssh/id_rsa

is the path to whichever private key you wish to use for that connection

User nate is the user with which you wish to establish the session.

All together now:

Host myhost  
  Hostname 10.0.0.1
  IdentityFile ~/.ssh/config
  User nate

This will set an alias, the private key, and the user for this specific host. This means that when you want to connect to the specified host, with the specified key, as the specified user, all you have to do is type ssh myhost. Keep in mind that you can also specify Port if some of your hosts are on non-standard ports.

Protip:

Host github.com  
  IdentityFile ~/.ssh/github.key

re-using connections

This is a pretty neat trick if you're not too far away from your servers latency-wise. Instead of opening up a socket every time you create an additional ssh session into a server, you can configure ssh to just re-use the background connection you've already established. No password prompts, or authentication handshaking, you just drop right in to the encrypted connection established by your initial session. Here's how:

Host *  
  ControlMaster auto
  ControlPath ~/.ssh/control/%h-%l-%p
  ControlPersist 600

The first line tells ssh to open the "initial" sockets as needed, and fall back to a normal connection if something breaks.

The second line is where your connection sockets will reside. Each file will be named for the connection's hostname-username-port.

The third line tells your "initial" connections how long to remain idle for before terminating. In the example above, it's 10 minutes. The default is for them to just remain open as long as you retain network connectivity, which is great if you have five servers, but if you're working on hundreds per day, you're gonna end up with a whole lot of ssh processes on your local machine.

If for some weird reason you need to be more of a control freak, you can be more specific by wrapping the ControlPath into a per-host stanza (as seen above).

jump hosts

Picture this:

[localhost]---[webserver]---[CDN]

The CDN is in a DMZ which can only be accessed via the webserver's network range; a range your localhost doesn't share. How do you gain ssh access to the CDN from your localhost when you don't have the routing for it? You use the webserver as a jump host. This is pretty much the same as sshing into the webserver, and then sshing into the CDN. Unless you want to access it via HTTP...

ssh -L 8888:cdn.derp.com:80 webserver.derp.com

Using our friendly -L flag we learned about earlier, we can forward a local port through an intermediary and on to destination port 80 on the CDN. Thus, if you open your web browser and go to http://localhost:8888, you should be presented with the webserver on the CDN itself.

You could add this to your ssh config, but I tend to chop and change ports on a regular basis and don't want something static. So for your homework, go and do some reading up on the ssh config parameter ProxyCommand to find out the most dynamic way to configure a host stanza for a jump host that is able to forward various ports.

 

[battletoad]

jump hosts

Picture this:

[localhost]---[webserver]---[CDN]

The CDN is in a DMZ which can only be accessed via the webserver's network range; a range your localhost doesn't share. How do you gain ssh access to the CDN from your localhost when you don't have the routing for it? You use the webserver as a jump host. This is pretty much the same as sshing into the webserver, and then sshing into the CDN. Unless you want to access it via HTTP...

ssh -L 8888:cdn.derp.com:80 webserver.derp.com

Using our friendly -L flag we learned about earlier, we can forward a local port through an intermediary and on to destination port 80 on the CDN. Thus, if you open your web browser and go to http://localhost:8888, you should be presented with the webserver on the CDN itself.

I love ssh tunnels. Sometimes, setting plenty of -L's can become tiresome, especially if they're all simply for webservers. In that case use

ssh -D 8888 foo.bar.com

You now essentially have a SOCKS proxy running on localhost:8888. Simply setting this as the proxy in firefox allows all traffic from your browser to be tunneled thru foo.bar.com. Don't forget to tick "Remote DNS" if you're resolving hostnames.

 

[Light]

1. Ensure that bash_completion is installed and enabled and that if you have anything installed with bash_completion plugins (eg. git) that the plugins are enabled too. You'll be able to use the tab key to auto-complete command parameters, hosts you're previously ssh'ed to and other contextual magic such as service names with systemctl, database names with mysql and branch names with git.

2. Learn the readline keyboard shortcuts (http://www.bigsmoke.us/readline/shortcuts). These are applicable not only to bash but to almost anything that allows you to type in text. While you're at it learn the bash history shortcuts too (http://ss64.com/bash/syntax-keyboard.html). In addition to what is listed on these pages, use Ctrl-D on an empty command line to exit the current shell.

3. Install z (https://github.com/rupa/z) for quick navigation to commonly used directories. Never use cd again.

4. Create a bin directory in your homedir and add this to your PATH in your .bashrc (PATH=~/bin:${PATH}) for any scripts you write. Use aliases for shorter commands.

5. Enable colours for any tools that support it (alias ls='ls --color=auto; eval $(dircolors -b); alias grep='grep --color=auto'), again in your .bashrc. To colour less see this page http://blog.0x1fff.com/2009/11/linux-tip-color-enabled-pager-less.html

6. Change your EDITOR and PAGER settings to the ones you prefer. If you're using anything other than vi(m) as your EDITOR you should think carefully about your life choices. Install most as your pager to embrace your inner hipster.

7. Alias ssh to automatically load your identity into ssh-agent the first time you use it (ssh-add -l > /dev/null || alias ssh='ssh-add -l > /dev/null || ssh-add && unalias ssh; ssh')

8. Go here https://github.com/alebcay/awesome-shell for a whole list of further useful apps, fancy prompts and other shell related items.

 

[n473]

If you're using anything other than vi(m) as your EDITOR you should think carefully about your life choices.

This is accurate.

 

[etienne_marais]

I use command substitution a lot (note sure of correct term)

./myscript `find . -iname *.gif`

to pass the output of find to myscript without using pipes

 

[n473]

I use command substitution a lot (note sure of correct term)

./myscript `find . -iname *.gif`

to pass the output of find to myscript without using pipes

Note that you can also use $() for command substitution. Syntactically, $() can easily nest within itself - $(echo foo$(echo bar)) without the need for escape characters - `echo foo\`echo bar\``

From what I've read, backticks are not deprecated in favor of $(), it's just another option for syntactical reasons.