A short tale about environment variables, dotenv, nodeJS and shell

Photo by Martijn Baudoin on Unsplash

There are many occasions when I end up working in projects that either use dotenv, shell scripts, or, in many cases a little bit of first and little bit of the second. So what options do you have especially when there’s a large amount of logic put inside shell scripts or simply there’s a large amount of export this, export that. You can of course grab a large cup of coffee and start to do manual work, but is there any other simpler way maybe? How could we trick someone or something into giving us our variables in a format that we can easily use and read later.

Let’s dig a bit into shells and -x flag. As ShellExplain explains it:

The shell shall write to standard error a trace for each command after it expands the command and before it executes it. It is unspecified whether the command that turns tracing off is traced.

Looking a bit on how this looks in sh, we can see something like this over the terminal:

sh-4.4$ export TEST=123
+ export TEST=123
+ TEST=123

Looks like this is something that we can use and maybe even filter to get some useful data out of it. Let’s start with a simple test shell script:

# file sourceable.sh
export A=1
export V=3
export B=78
export D="${V}93"

This yields the following output into the shell:

source ./test.sh 
+ source ./test.sh
++ export A=1
++ A=1
++ export V=3
++ V=3
++ export B=78
++ B=78
++ export D=393
++ D=393

Ok, now let’s try to use bash in order to gather all this data into a separate log file:

exec 13>logfile
export BASH_XTRACEFD=13
set -x
source ./sourceable.sh
set +x

Now, we should end up with the same content as above, but this time this content will be inside a file called logfile inside working directory. This contains everything that was run inside the shell between the -x and +x. We can filter it using common tools like sed and grep:

cat logfile | grep -P '^[^a-z]*export (?!DO_NOT_NEED_THESE)' | sed -r 's/^[^a-z]+//g' | sed -r "s/^export '(.+)'\$/export \1/" | sed -r 's/^export ([^=]+)=(.*)$/process.env.\1="\2\"/'

And get something similar to this (or any other format):

process.env.A=”1"
process.env.V=”3"
process.env.B=”78"
process.env.D=”393"

You can put it in an alias and just use it in your shell to easily get the format you need:

dump_envs() {
if [ $# -eq 0 ]; then
   echo "You should specify a file to be sourced" >&2
return 1
fi
exec 13>logfile
export BASH_XTRACEFD=13
set -x
source "$1"
set +x
grep -P '^[^a-z]*export (?!PATH|.*MARATHON_URL|VIRTUAL_SERVICE)' < logfile \
| sed -r 's/^[^a-z]+//g' \
| sed -r "s/^export '(.+)'\$/export \1/" \
| sed -r 's/^export ([^=]+)=(.*)$/process.env.\1="\2\"/'
}

Happy envs hunting.