Use the Past to Conquer the Future - A How-To on BASH History Substitution https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977 # Use The Past To Conquer The Future - History Substitution Since I had to get **_really_** close with bash recently and found myself typing the same things over and over again, I decided to open the amazing toolbox called _"History Substitution"_ and was amazed by how much you can do with it. It's a feature built into bash that allows you to get commands and arguments from your history. In this How-To I hope to be able to share this amazement with you! So let's go! *** ## 1. Introduction A long time ago, the Grandmaster who taught me Linux brought a small detail to my attention. It went a little something like this: > Me: apt update $ apt update: permission denied Me: sudo apt upda ... Him: wait! I shall teach you ... $ sudo !! sudo apt update Suffice to say, my mind was blown! After learning this, I used it as often as I could, not realizing that this rabbit hole went even deeper than I imagined. *** ## 2. Down the hole we go In our daily life with bash we often come across situations like this: > less /this/path/is/really/long/and/annoying/randomfile Now if we're done looking at the file and want to open it in vi we have to: press the up arrow -> pos1 -> del/del/del/del -> vi which is ok but still a pain in the butt if we have to do it more than once in our entire lifetime. Instead with history subsitution we can just do: > vi !!$ So how does this work? As you may have noticed by now, to start a history substitution command you begin with two exclamation points. Given only two consecutive exclamation points returns the whole last command. If we use history substitution with other options we can always omit the second exclamation point. When used with a dollar sign it returns only the last argument. The other way around, an exclamation point followed by a circumflex (this thing -> ^ ) takes only the first argument. Let's look at some examples with **_"echo a b c"_** as the last command used: >!! -> echo a b c echo !! -> echo echo a b c echo !$ -> echo c echo !^ -> echo a echo !* -> echo a b c The first command above would also execute it immediately. Notice that the two exclamations points also get the command and the one with ^ ignores it. To get all arguments but not the command, the asterisk is used. Additionally you can get specific arguments using a colon, in this context it's called the word designator: > echo !:0 -> echo echo echo !:2 -> echo b echo !:2-3 -> echo b c echo !:2* -> echo b c echo !:1 !:3 -> echo a c But let's say that you had to run a different command in the meantime and the things you need are not in the last command anymore. This is also easily done by telling it "how long ago" the command was used. > $ echo a b c $ echo d e f !-1 -> echo d e f !-2 -> echo a b c This is very helpful when using them in quick succession but nigh impossible to keep track of over longer sessions. In those cases it's a lot easier to use the built-in string function. >$ uname -r $ uname -a $ echo a b c echo !un -> echo uname -a echo !un:* -> echo -a echo !?nam -> echo uname -a echo !?me -r? -> echo uname -r As we can see, only giving it the beginning of the command it automatically looks for the first occurence (going upwards) in the history and returns it. Using the question marks we can look for parts inside the command to further narrow it down and filter alike commands. ### Now onto paths! Another great feature built into this, is the path handling. It requires some getting used to but is also very straightforward. let's assume we have this path: > /home/user/folder/file.txt This is what we can do: > Get the path up to file: echo !:h -> /home/user/folder >Get the filename plus extension: echo !:t -> file.txt >Get the path up to file extension: echo !:r -> /home/user/folder/file >Get only file extension: echo !:e -> .txt >Get only filename: echo !:t:r -> file ### Substitution! We can also substitute certain strings with other strings. For example if we have **_/path/file1_** and we want to change **file1** to **file2** we do: >!:s/file1/file2 But this only works on the first occurence found! If we want to do this globally, so for every occurence of **file1**, we use: >!:gs/file1/file2 Example: >$ cp /path/file1 /path/folder/file1 !:gs/file1/file2 -> cp /path/file2 /path/folder/file2 If you want to repeat a successful substitution in a different context then you don't need to write it again. You can just use: >!:& -> for single occurence substitution !:g& -> for global substitution *** ## 3. Summary [details="Summary/Cheat-Sheet"] >!! -> execute last command !$ -> return last argument !^ -> return first argument !* -> return all arguments !:n -> return the string on nth position !:n-x -> return position n to x !:n* -> return all arguments starting with n !n -> execute command with history number n !-n -> exectue command that was run n commands back !?str -> execute first command (going up) that matches _str_ !?str? -> execute first command (going up) that contains _str_ !:h -> return path up to bas filename !:t -> return only base filename !:r -> return path up to extension !:e -> return only extension !:s/str1/str2 -> substitute first occurence of _str1_ with _str2_ !:gs/str1/str2 -> substitute all occurences of _str1_ with _str2_ !:& -> repeat last successful substitution !:g& -> repeat last successful substitution and make it global `!:p` -> don't execute, print only [/details] *** ## 4. Extras * If you missed it in the summary, you can only print the return value of the history substitution without executing it by using `!:p` * Of course you can connect all of these to do some crazy things! * To use this feature in bash scripts, supply it with `set -H` * To run a command and not have it in the history, just type a space in front of it! This works for any command, not just history substitution. *** And that's all! If you have any questions or found a mistake just leave a comment below. I sincerely hope you were able to learn something new today and if you already knew about it then I hope I was able to refresh your memory! :smiley: ### Happy Hacking! Fri, 03 May 2019 10:13:16 +0000 Linux Use the Past to Conquer the Future - A How-To on BASH History Substitution This topic was automatically closed after 30 days. New replies are no longer allowed.

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/9 Mon, 13 May 2019 11:45:12 +0000 d.clarkee.co.uk-post-12977-9 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution Interestingly, zsh does the same thing, but replaces expressions not when enter is hit, but when the expression is done, wich means for zsh that there is a space after it.
For this reason, this example doesn’t work :

As echo !?me -r? is replaced by echo uname -a before we can type -r.
A simple fix is to use echo !?-r? instead.

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/8 Fri, 03 May 2019 10:13:16 +0000 d.clarkee.co.uk-post-12977-8 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution Oh man very nice post with nice explanation
Carry on man !!!
Happy Hacking :smile:

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/7 Fri, 03 May 2019 07:04:59 +0000 d.clarkee.co.uk-post-12977-7 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution I’m glad to hear you enjoyed it!
Your malicious use idea intrigued me @hunter so I tried to strace the history substitution, as a starting point, and to my surprise it didn’t work!
Maybe it’s also something to do with how I traced it!?
I used:
stty; cat | strace bash > /dev/null
But I always get "!!" - command not found
Anyone know why this could be? It doesn’t really make sense to me since it’s a shell keyword and built into bash…

EDIT: it works when attaching to an already active bash process… Not quite sure why though, maybe I’ll make a follow up post exploring this in more detail

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/5 Tue, 16 Apr 2019 15:58:01 +0000 d.clarkee.co.uk-post-12977-5 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution Oh man, I’ve been using the linux command line for like 10 years and I never knew about these shortcuts.

Thank you so much!

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/4 Mon, 15 Apr 2019 11:27:31 +0000 d.clarkee.co.uk-post-12977-4 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution Another feature it could be good to point out is that you can use ctrl-r to search the command history.
https://www.gnu.org/software/bash/manual/html_node/Searching.html

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/3 Mon, 15 Apr 2019 07:58:44 +0000 d.clarkee.co.uk-post-12977-3 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution That’s a very useful feature that i’ve not heard about. Thank you! I’m now wondering if it could be used maliciously though :thinking: Maybe poisoning the bash_history with evil commands? idk

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/2 Sat, 13 Apr 2019 23:39:07 +0000 d.clarkee.co.uk-post-12977-2 Use the Past to Conquer the Future - A How-To on BASH History Substitution
Use the Past to Conquer the Future - A How-To on BASH History Substitution Use The Past To Conquer The Future - History Substitution

Since I had to get really close with bash recently and found myself typing the same things over and over again, I decided to open the amazing toolbox called “History Substitution” and was amazed by how much you can do with it. It’s a feature built into bash that allows you to get commands and arguments from your history.
In this How-To I hope to be able to share this amazement with you! So let’s go!


1. Introduction

A long time ago, the Grandmaster who taught me Linux brought a small detail to my attention.
It went a little something like this:

Me: apt update
$ apt update: permission denied
Me: sudo apt upda …
Him: wait! I shall teach you …
$ sudo !!
sudo apt update

Suffice to say, my mind was blown!

After learning this, I used it as often as I could, not realizing that this rabbit hole went even deeper than I imagined.


2. Down the hole we go

In our daily life with bash we often come across situations like this:

less /this/path/is/really/long/and/annoying/randomfile

Now if we’re done looking at the file and want to open it in vi we have to:

press the up arrow → pos1 → del/del/del/del → vi

which is ok but still a pain in the butt if we have to do it more than once in our entire lifetime.
Instead with history subsitution we can just do:

vi !!$

So how does this work?
As you may have noticed by now, to start a history substitution command you begin with two exclamation points.
Given only two consecutive exclamation points returns the whole last command.
If we use history substitution with other options we can always omit the second exclamation point.

When used with a dollar sign it returns only the last argument.
The other way around, an exclamation point followed by a circumflex (this thing → ^ ) takes only the first argument.
Let’s look at some examples with “echo a b c” as the last command used:

!! → echo a b c
echo !! → echo echo a b c
echo !$ → echo c
echo !^ → echo a
echo !* → echo a b c

The first command above would also execute it immediately.
Notice that the two exclamations points also get the command and the one with ^ ignores it.
To get all arguments but not the command, the asterisk is used.
Additionally you can get specific arguments using a colon, in this context it’s called the word designator:

echo !:0 → echo echo
echo !:2 → echo b
echo !:2-3 → echo b c
echo !:2* → echo b c
echo !:1 !:3 → echo a c

But let’s say that you had to run a different command in the meantime and the things you need are not in the last command anymore. This is also easily done by telling it “how long ago” the command was used.

$ echo a b c
$ echo d e f
!-1 → echo d e f
!-2 → echo a b c

This is very helpful when using them in quick succession but nigh impossible to keep track of over longer sessions. In those cases it’s a lot easier to use the built-in string function.

$ uname -r
$ uname -a
$ echo a b c
echo !un → echo uname -a
echo !un:* → echo -a
echo !?nam → echo uname -a
echo !?me -r? → echo uname -r

As we can see, only giving it the beginning of the command it automatically looks for the first occurence (going upwards) in the history and returns it. Using the question marks we can look for parts inside the command to further narrow it down and filter alike commands.

Now onto paths!

Another great feature built into this, is the path handling.
It requires some getting used to but is also very straightforward.
let’s assume we have this path:

/home/user/folder/file.txt

This is what we can do:

Get the path up to file:
echo !:h → /home/user/folder

Get the filename plus extension:
echo !:t → file.txt

Get the path up to file extension:
echo !:r → /home/user/folder/file

Get only file extension:
echo !:e → .txt

Get only filename:
echo !:t:r → file

Substitution!

We can also substitute certain strings with other strings.
For example if we have /path/file1 and we want to change file1 to file2 we do:

!:s/file1/file2

But this only works on the first occurence found!
If we want to do this globally, so for every occurence of file1, we use:

!:gs/file1/file2

Example:

$ cp /path/file1 /path/folder/file1
!:gs/file1/file2 → cp /path/file2 /path/folder/file2

If you want to repeat a successful substitution in a different context then you don’t need to write it again.
You can just use:

!:& → for single occurence substitution
!:g& → for global substitution


3. Summary

Summary/Cheat-Sheet (click for more details)

4. Extras

  • If you missed it in the summary, you can only print the return value of the history substitution without executing it by using !:p
  • Of course you can connect all of these to do some crazy things!
  • To use this feature in bash scripts, supply it with set -H
  • To run a command and not have it in the history, just type a space in front of it! This works for any command, not just history substitution.

And that’s all! If you have any questions or found a mistake just leave a comment below.

I sincerely hope you were able to learn something new today and if you already knew about it then I hope I was able to refresh your memory! :smiley:

Happy Hacking!

Read full topic

]]>
https://d.clarkee.co.uk/t/use-the-past-to-conquer-the-future-a-how-to-on-bash-history-substitution/12977/1 Sat, 13 Apr 2019 11:45:10 +0000 d.clarkee.co.uk-post-12977-1 Use the Past to Conquer the Future - A How-To on BASH History Substitution