Hiding secret keys from shell history: Part 1

During the course of working, I occasionally have to write commands like this one, usually to test some old commands

export AWS_ACCESS_KEY_ID=${something}
export AWS_SECRET_ACCESS_KEY=${something}

It may not seem like a big deal to have them in the history, but remember every process that runs as you has permission to read all your command history. If you end up downloading something malicious, leave machine unattended for even a split second, someone could steal important secrets and leave any trail. Every time you type commands, these are getting written in history, and sometimes this may mean getting compromised in big way.

First I’m going to show some ways that you can manipulate history, or avoid writing history altogether. Next part of this series will go over how you can integrate password managers to keep these secrets encrypted and only unlock when needed.

Managing History File

Location

The path is stored in the $HISTFILE environment variable. So in zsh it would look something like /home/myuser/.zsh_history or in bash it would look like /home/myuser/.bash_history

Accessing history

You can use a cat or history command to read all commands you have entered previously.

cat $HISTFILE
#or
history

Interactively, it is mapped to Ctrl+r key to be able to do a lookup. I have it mapped to fzf so that I can do fuzzy search on it. I have described it in a previous post on effective shell usage.

Limiting History (or not)

This used to be crucial for performance before SSDs became common place but it’s still important to keep a limit of the number of commands to keep in history. Since these are kept for quick lookup in memory, there may still be performance implication for not cleaning these up.

both bash and zsh use SAVEHIST and HISTSIZE to determine how many commands to keep in the file and consequently in memory

HISTFILE="$HOME/.zsh_history"
HISTSIZE=10000
SAVEHIST=1000
setopt BANG_HIST                 # Treat the '!' character specially during expansion.
setopt EXTENDED_HISTORY          # Write the history file in the ":start:elapsed;command" format.
setopt INC_APPEND_HISTORY        # Write to the history file immediately, not when the shell exits.
setopt SHARE_HISTORY             # Share history between all sessions.
setopt HIST_EXPIRE_DUPS_FIRST    # Expire duplicate entries first when trimming history.
setopt HIST_IGNORE_DUPS          # Don't record an entry that was just recorded again.
setopt HIST_IGNORE_ALL_DUPS      # Delete old recorded entry if new entry is a duplicate.
setopt HIST_FIND_NO_DUPS         # Do not display a line previously found.
setopt HIST_IGNORE_SPACE         # Don't record an entry starting with a space.
setopt HIST_SAVE_NO_DUPS         # Don't write duplicate entries in the history file.
setopt HIST_REDUCE_BLANKS        # Remove superfluous blanks before recording entry.
setopt HIST_VERIFY               # Don't execute immediately upon history expansion.
setopt HIST_BEEP                 # Beep when accessing nonexistent history.

There are lot more possibilities for managing the history as mentioned in this stackoverflow post.

You want to at least set the hist_ignore_all_dups variable to ignore duplicate commands with setopt hist_ignore_all_dups

Another trick that works by default in bash but zsh needs setting another option. You can configure zsh to ignore storing commands prefixed by a space by using setopt hist_ignore_space command or `setopt histignorespace`

Disable recording in history

In zsh setting the `setopt histignorespace` and typing commands with space in front is the only way I know that avoids putting in history. If you press up, the commands will show up unlike bash but that’s a feature for convenience, it’s not saved in the history file.

For bash you can use `set +o history` to disable saving in the current shell’s history. You can add this to ~/.bashrc to disable it for all sessions.

In Bash, you can use HISTIGNORE env variable to only exclude certain commands from history. A nice trick would be to do something like

export HISTIGNORE="ls*:cat*:*AWS*:*SECRET*

For zsh, the equivalent is HISTORY_IGNORE and they’re evaluated as a pattern, so the same thing would look like

export HISTORY_IGNORE="(ls|cat|AWS|SECRET)"

See more from the zshparam manual.

Cleaning up history

You can either delete $HISTFILE or run `history -c` to do this.

In the next part of the series, I’ll include LastPass CLI tool to keep the actual secrets in a password manager and only include when needed. I’ve found this is great to write shared scripts in enterprise/team settings and to establish sane rotation policies.