Repo Browser: Bash Commons You need to enable JavaScript to run this app.
Gruntwork Website
Bash Commons A collection of reusable Bash functions for handling common tasks such as logging, assertions, string manipulation, and more.
Bash Commons
This repo contains a collection of reusable Bash functions for handling common tasks such as logging, assertions,
string manipulation, and more. It is our attempt to bring a little more sanity, predictability, and coding reuse to our
Bash scripts. All the code has thorough automated tests and is packaged into functions, so you can safely import it
into your bash scripts using source.
Examples
Once you have bash-commons installed (see the install instructions ), you use source to import the
modules and start calling the functions within them. Before you import any modules, make sure you source the
bootstrap.sh file which sets some important defaults to encourage good code:
source /opt/g runtwork/bash-commons/ bootstrap.sh
source /opt/g runtwork/bash-commons/ log.sh
source /opt/g runtwork/bash-commons/ assert.sh
source /opt/g runtwork/bash-commons/ os.sh
log_info "Hello, World!"
assert_not_empty "--foo" "$foo" "You must provide a value for the --foo parameter."
if os_is_ubuntu "16.04" ; then
log_info "This script is running on Ubuntu 16.04!"
elif os_is_centos; then
log_info "This script is running on CentOS!"
fi
Install
The first step is to download the code onto your computer.
The easiest way to do this is with the Gruntwork Installer
(note, you'll need to replace <VERSION> below with a version number from the releases
page ):
gruntwork-install \
--repo https: //github.com/gruntwork -io/bash-commons \
--module -name bash -commons \ --tag <VERSION>
The default install location is /opt/gruntwork/bash-commons, but you can override that using the dir param, and
override the owner of the install dir using the owner and group params:
gruntwork-install \
--repo https: //github.com/gruntwork -io/bash-commons \
--module -name bash -commons \ --tag <VERSION> \
--module -param dir =/foo /bar \ --module -param owner =my -os -username \ --module -param group =my -os -group
If you don't want to use the Gruntwork Installer, you can use git clone to get the code onto your computer and then
copy it to it's final destination manually:
git clone --branch <VERSION> https://github.com/gruntwork-io/bash-commons.git
sudo mkdir -p /opt /gruntwork
cp -r bash-commons/modules/bash-commons/src /opt /gruntwork/bash-commons
sudo chown -R "my-os-username:my-os-group" /opt /gruntwork/bash-commons
Instance Metadata Service versions
bash-commons supports both Instance Metadata Service (IMDS) version 1 and 2. Gruntwork and AWS both recommend using version 2 of the Instance Metadata Service whenever possible. Although version 1 is still supported and considered fully secure by AWS, version 2 has been specially hardened against specific threat vectors and is therefore preferable.
To understand more about Instance Metadata Service version 2 and its features, read the official AWS documentation on IMDSv2 .
There are two ways to specify the version of the Instance Metadata Service that bash-commons should use:
Set the environment variable GRUNTWORK_BASH_COMMONS_IMDS_VERSION to the version of IMDS that you wish to use. Valid values are either 1 or 2.
Change the value of default_instance_metadata_version to either 1 or 2.
Example of dynamic-ubuntu-wait.sh usage:
You can use the dynamic-ubuntu-wait.sh command after you install bash-commons :
bash /opt /gruntwork/bash-commons/dynamic-ubuntu-wait.sh
Alternatively, you can call the script without installing by curling it during your existing provisioning/automated installation process:
curl -LsS https://raw.githubusercontent.com/gruntwork-io/bash-commons/ [VERSION ]/modules/bash-commons/src/dynamic-ubuntu-wait.sh | bash`
Where [VERSION] could be: v0.0.3. The latest release can be found here
Importing modules
You can use the source command to "import" the modules you need and use them in your code:
source /opt /gruntwork/bash-commons/log .sh
This will make all the functions within that module available in your code:
log_info "Hello, World!"
Available modules
Here's an overview of the modules available in bash-commons:
array.sh: Helpers for working with Bash arrays, such as checking if an array contains an element, or joining an
array into a string with a delimiter between elements.
assert.sh: Assertions that check a condition and exit if the condition is not met, such as asserting a variable is
not empty or that an expected app is installed. Useful for defensive programming.
aws.sh: A collection of thin wrappers for direct calls to the AWS CLI and EC2
Instance Metadata . These thin
wrappers give you a shorthand way to fetch certain information (e.g., information about an EC2 Instance, such as its
private IP, public IP, Instance ID, and region). Moreover, you can swap out aws.sh with a version that returns mock
data to make it easy to run your code locally (e.g., in Docker) and to run unit tests.
aws-wrapper.sh: A collection of "high level" wrappers for the AWS CLI and EC2
Instance Metadata to simplify common
tasks such as looking up tags or IPs for EC2 Instances. Note that these wrappers handle all the data processing and
logic, whereas all the direct calls to the AWS CLI and EC2 metadata endpoints are delegated to aws.sh to make unit
testing easier.
dynamic-ubuntu-wait.sh: A script that dynamically waits for Ubuntu automatic update mechanism to
release all locks so that apt-get may run without errors.
file.sh: A collection of helpers for working with files, such as checking if a file exists or contains certain text.
log.sh: A collection of logging helpers that write logs to stderr with log levels (INFO, WARN, ERROR) and
timestamps.
os.sh: A collection of Operating System helpers, such as checking which flavor of Linux (e.g., Ubuntu, CentOS) is
running and validating checksums.
string.sh: A collection of string manipulation functions, such as checking if a string contains specific text,
stripping prefixes, and stripping suffixes.
Coding principles
The code in bash-commons follows the following principles:
Compatibility Code style Everything is a function Namespacing Testing
Compatibility
The code in this repo aims to be compatible with:
Bash 3
Most major Linux distributions (e.g., Ubuntu, CentOS)
Code style
All the code should mainly follow the Google Shell Style Guide .
In particular:
The first line of every script should be #!/usr/bin/env bash.
All code should be defined in functions.
Functions should exit or return 0 on success and non-zero on error.
Functions should return output by writing it to stdout.
Functions should log to stderr.
All variables should be local. No global variables are allowed at all.
Make as many variables readonly as possible.
If a variable is both local and readonly, use local -r.
If calling to a subshell and storing the output in a variable (foo=$( ... )), do NOT use local -r in the same
statement or the exit code will be lost .
Instead, declare the variable as local on one line and then call the subshell on the next line.
Quote all strings.
Use [[ ... ]] instead of [ ... ].
Use snake_case for function and variable names. Use UPPER_SNAKE_CASE for constants.
Everything in a function
It's essential that ALL code is defined in a function. That allows you to use source to "import" that code without
anything actually being executed.
Namespacing
Bash does not support namespacing, so we fake it using a convention on the function names: if you create a file
<foo.sh>, all functions in it should start with foo_. For example, all the functions in log.sh start with log_
(log_info, log_error) and all the functions in string.sh start with string_ (string_contains,
string_strip_prefix). That makes it easier to tell which functions came from which modules.
For readability, that means you should typically give files a name that is a singular noun. For example, log.sh
instead of logging.sh and string.sh instead of strings.sh.
Testing
Every function should be tested:
Automated tests are in the test folder.
We use Bats as our unit test framework for Bash code. Note: Bats has not been
maintained the last couple years, so we may need to change to the bats-core
fork at some point (see #150 ).
We run all tests in the gruntwork/bash-commons-circleci-tests Docker
image so that (a) it's consistent with how the CI
server runs them, (b) the tests always run on Linux, (c) any changes the tests make, such as writing files or
creating OS users, won't affect the host OS, (d) we can replace some of the modules, such as aws.sh, with mocks at
test time. There is a docker-compose.yml file in the test folder to make it easy to run the tests.
To run all the tests: docker-compose up.
To run one test file: docker-compose run tests bats test/array.bats.
To leave the Docker container running so you can debug, explore, and interactively run bats: docker-compose run tests bash.
If you ever need to build a new Docker image, the Dockerfile is in the .circleci folder :
cd .circleci
docker build -t gruntwork/bash-commons-circleci-tests .
docker push gruntwork/bash-commons-circleci-tests
TODO
Add automated tests for aws.sh and aws-wrapper.sh. We have not tested these as they require either running an
EC2 Instance or run something like LocalStack .
License
This code is released under the Apache 2.0 License. Please see
LICENSE and
NOTICE for more details.
Copyright © 2018 Gruntwork, Inc.
Questions? Ask away. We're here to talk about our services, answer any questions, give advice, or just to chat.
Ready to hand off the Gruntwork? "https://cdn.gruntwork.io/gruntwork-website/"
{"index":{"js":"https://cdn.gruntwork.io/gruntwork-website/index.bundle.c7884255553b53fbca3a.js","map":"https://cdn.gruntwork.io/gruntwork-website/index.bundle.1b14c1b7d19f1f5eb35d6e118e838255.map"},"styles":{"css":"https://cdn.gruntwork.io/gruntwork-website/styles.bundle.f22938926651ddec7c49.css","js":"https://cdn.gruntwork.io/gruntwork-website/styles.bundle.e782420e74a20dcb8691.js","map":"https://cdn.gruntwork.io/gruntwork-website/styles.bundle.d5e2af49807c6ca33f8367d621ece507.map"},"vendors":{"css":"https://cdn.gruntwork.io/gruntwork-website/vendors.bundle.29f7d0366a0978763f96.css","js":"https://cdn.gruntwork.io/gruntwork-website/vendors.bundle.fa8174a130797d75d12c.js","map":"https://cdn.gruntwork.io/gruntwork-website/vendors.bundle.57243d94deeeb29d5061288a338b4eb6.map"}}
{"treedata":{"name":"root","toggled":true,"children":[{"name":".circleci","children":[{"name":"aws-local.sh","path":".circleci/aws-local.sh","sha":"3e72e4614374c6b1be34dac9f2fa297118aa0a7b"},{"name":"config.yml","path":".circleci/config.yml","sha":"5b5951a91d2a49d17c8a3b48093dbf1b029c68d6"},{"name":"shellcheck.sh","path":".circleci/shellcheck.sh","sha":"68a7007c1798d1d275c85219d366d89369ad93a8"}]},{"name":".gitignore","path":".gitignore","sha":"e2e7cb0ae833d7bb394b1d6ef647573b91339ff6"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"6bddb3ff6e1b3dfaba7cf180e56bca12c245be56"},{"name":"Dockerfile.shellcheck","path":"Dockerfile.shellcheck","sha":"f67999ce1662aa9645114037c7c50fde071d8101"},{"name":"Dockerfile.ubuntu16.04.bats","path":"Dockerfile.ubuntu16.04.bats","sha":"43df0df44d1e62c8190fd8ed2994ebe4192af4a2"},{"name":"Dockerfile.ubuntu18.04.bats","path":"Dockerfile.ubuntu18.04.bats","sha":"3ff1d3affab4fee65be3a71cf46f0f4fe0b19c3f"},{"name":"Dockerfile.ubuntu20.04.bats","path":"Dockerfile.ubuntu20.04.bats","sha":"3b251170a17935232bf70f8c21443914fb13d83d"},{"name":"LICENSE","path":"LICENSE","sha":"7a4a3ea2424c09fbe48d455aed1eaa94d9124835"},{"name":"NOTICE","path":"NOTICE","sha":"6590c2e84b922cb6d56d43b31c09735604b0149d"},{"name":"README.md","path":"README.md","sha":"db8482c056fa9bd2c9cdd1a1fd9ba44902bc1106","toggled":true},{"name":"docker-compose.yml","path":"docker-compose.yml","sha":"ce5a462f46d2391b967077f01fe3fec3991d7aa3"},{"name":"examples","children":[{"name":"dynamic-ubuntu-wait","children":[{"name":"packer-build.json","path":"examples/dynamic-ubuntu-wait/packer-build.json","sha":"53c4c3b1e28abb8b0033d2cf77c46436b0bc8fbc"}]}]},{"name":"integration-test","children":[{"name":"dynamic_ubuntu_wait_test.go","path":"integration-test/dynamic_ubuntu_wait_test.go","sha":"2836ffbbe189d07be960e669d92c2f8f2f04fed8"},{"name":"go.mod","path":"integration-test/go.mod","sha":"e25d63461bcab4592c20165509c84d3dfa04003e"},{"name":"go.sum","path":"integration-test/go.sum","sha":"e9ed2463f62d3104af374de7366d4d5c91a30301"}]},{"name":"modules","children":[{"name":"bash-commons","children":[{"name":"install.sh","path":"modules/bash-commons/install.sh","sha":"c67605f37c01a0583765bc6d34998fef1fac325e"},{"name":"src","children":[{"name":"array.sh","path":"modules/bash-commons/src/array.sh","sha":"960b8d46bd56acd9e6dc6038c5fcde8b2074f19f"},{"name":"assert.sh","path":"modules/bash-commons/src/assert.sh","sha":"1e767e308c222378aacfd743c2648e09177b94ff"},{"name":"aws-wrapper.sh","path":"modules/bash-commons/src/aws-wrapper.sh","sha":"4234e7ae5a211ce58b098d705eb52b3e60495154"},{"name":"aws.sh","path":"modules/bash-commons/src/aws.sh","sha":"be683e06d9f5fc132a866fd75f6182b686df163a"},{"name":"bootstrap.sh","path":"modules/bash-commons/src/bootstrap.sh","sha":"c9aac3f64e3489b1cf70b0a60492e881a7882362"},{"name":"dynamic-ubuntu-wait.sh","path":"modules/bash-commons/src/dynamic-ubuntu-wait.sh","sha":"ecd15e82642770e0621e5a1f74e130e6ad8186e5"},{"name":"file.sh","path":"modules/bash-commons/src/file.sh","sha":"4028b2f6960921823e9e8640ea91566391dda751"},{"name":"log.sh","path":"modules/bash-commons/src/log.sh","sha":"2f0817c944c3ede164fb1b21d45237b6471ace3e"},{"name":"os.sh","path":"modules/bash-commons/src/os.sh","sha":"a2b190f05431c86a013bf64476dc5d4a55b96b9d"},{"name":"string.sh","path":"modules/bash-commons/src/string.sh","sha":"40467ebacef0a825c8e304064c532f9950ee9b7d"}]}]}]},{"name":"test","children":[{"name":"array.bats","path":"test/array.bats","sha":"bd6c2f198c23f3af5f2a11e424a27c6e47434a4a"},{"name":"assert.bats","path":"test/assert.bats","sha":"d1500c8d743dbb331fed699796c2ab3167f9b5c7"},{"name":"aws-cli.bats","path":"test/aws-cli.bats","sha":"414b646e4951e27da808b41c117cea4ce384ff74"},{"name":"aws-ec2-metadata.bats","path":"test/aws-ec2-metadata.bats","sha":"1bb65b58c7e1e01eb18dea3b47cbf228b94a1a9e"},{"name":"aws-helper.bash","path":"test/aws-helper.bash","sha":"24b224638be5d89a807d5f37b39f82243cc71343"},{"name":"aws-mock","children":[{"name":"aws.sh","path":"test/aws-mock/aws.sh","sha":"b311f32fd84f36fc57e8d7d25c64eb3f92696c91"}]},{"name":"aws-wrapper.bats","path":"test/aws-wrapper.bats","sha":"269c1a332b78a1fbd1ac4ea293d76f6f073677ec"},{"name":"ec2-metadata-mock","children":[{"name":"ec2-metadata-mock.py","path":"test/ec2-metadata-mock/ec2-metadata-mock.py","sha":"ce1e638cdd70c2978cade5df5aea542ad315d471"}]},{"name":"file.bats","path":"test/file.bats","sha":"757f7c229fc1a6fb774684babf4f83687b15eadf"},{"name":"fixtures","children":[{"name":"checksum","children":[{"name":"foo.txt","path":"test/fixtures/checksum/foo.txt","sha":"19102815663d23f8b75a47e7a01965dcdc96468c"}]}]},{"name":"log.bats","path":"test/log.bats","sha":"6a6eeb7a39b235e9ffe1562551f5ab0f74bbdb9e"},{"name":"os.bats","path":"test/os.bats","sha":"5d560d984d8f7a3f4152cba927f15cd0e498ded3"},{"name":"string.bats","path":"test/string.bats","sha":"aa8401120a4bf91e53335db6341ffaa5784610a5"},{"name":"test-helper.bash","path":"test/test-helper.bash","sha":"829da0c5f7ab359fe28465a71045c71db2d95a03"}]}]},"detailsContent":"<p><a href=\"https://gruntwork.io/?ref=repo_bash_commons\" class=\"preview__body--description--blue\" target=\"_blank\"><img src=\"https://img.shields.io/badge/maintained%20by-gruntwork.io-%235849a6.svg\" alt=\"Maintained by Gruntwork.io\" class=\"preview__body--diagram\"></a></p>\n<h1 class=\"preview__body--title\" id=\"bash-commons\">Bash Commons</h1><div class=\"preview__body--border\"></div><p>This repo contains a collection of reusable Bash functions for handling common tasks such as logging, assertions,\nstring manipulation, and more. It is our attempt to bring a little more sanity, predictability, and coding reuse to our\nBash scripts. All the code has thorough automated tests and is packaged into functions, so you can safely import it\ninto your bash scripts using <code>source</code>.</p>\n<h2 class=\"preview__body--subtitle\" id=\"examples\">Examples</h2>\n<p>Once you have <code>bash-commons</code> installed (see the <a href=\"#install\" class=\"preview__body--description--blue\">install instructions</a>), you use <code>source</code> to import the\nmodules and start calling the functions within them. Before you import any modules, make sure you <code>source</code> the\n<code>bootstrap.sh</code> file which sets some important defaults to encourage good code:</p>\n<pre><span class=\"hljs-keyword\">source</span> <span class=\"hljs-regexp\">/opt/g</span>runtwork<span class=\"hljs-regexp\">/bash-commons/</span>bootstrap.sh\n<span class=\"hljs-keyword\">source</span> <span class=\"hljs-regexp\">/opt/g</span>runtwork<span class=\"hljs-regexp\">/bash-commons/</span>log.sh\n<span class=\"hljs-keyword\">source</span> <span class=\"hljs-regexp\">/opt/g</span>runtwork<span class=\"hljs-regexp\">/bash-commons/</span>assert.sh\n<span class=\"hljs-keyword\">source</span> <span class=\"hljs-regexp\">/opt/g</span>runtwork<span class=\"hljs-regexp\">/bash-commons/</span>os.sh\n\nlog_info <span class=\"hljs-string\">\"Hello, World!\"</span>\n\nassert_not_empty <span class=\"hljs-string\">\"--foo\"</span> <span class=\"hljs-string\">\"$foo\"</span> <span class=\"hljs-string\">\"You must provide a value for the --foo parameter.\"</span>\n\n<span class=\"hljs-keyword\">if</span> os_is_ubuntu <span class=\"hljs-string\">\"16.04\"</span>; then\n log_info <span class=\"hljs-string\">\"This script is running on Ubuntu 16.04!\"</span>\nelif os_is_centos; then\n log_info <span class=\"hljs-string\">\"This script is running on CentOS!\"</span>\nfi\n</pre>\n<h2 class=\"preview__body--subtitle\" id=\"install\">Install</h2>\n<p>The first step is to download the code onto your computer.</p>\n<p>The easiest way to do this is with the <a href=\"/repos/gruntwork-installer\" class=\"preview__body--description--blue\">Gruntwork Installer</a>\n(note, you'll need to replace <code><VERSION></code> below with a version number from the <a href=\"#open_modal\" class=\"preview__body--description--blue\">releases\npage</a>):</p>\n<pre>gruntwork-install \\\n -<span class=\"ruby\">-repo <span class=\"hljs-symbol\">https:</span>/<span class=\"hljs-regexp\">/github.com/gruntwork</span>-io/bash-commons \\\n</span> -<span class=\"ruby\">-<span class=\"hljs-class\"><span class=\"hljs-keyword\">module</span>-<span class=\"hljs-title\">name</span> <span class=\"hljs-title\">bash</span>-<span class=\"hljs-title\">commons</span> \\</span>\n</span> -<span class=\"ruby\">-tag <VERSION>\n</span></pre>\n<p>The default install location is <code>/opt/gruntwork/bash-commons</code>, but you can override that using the <code>dir</code> param, and\noverride the owner of the install dir using the <code>owner</code> and <code>group</code> params:</p>\n<pre>gruntwork-install \\\n -<span class=\"ruby\">-repo <span class=\"hljs-symbol\">https:</span>/<span class=\"hljs-regexp\">/github.com/gruntwork</span>-io/bash-commons \\\n</span> -<span class=\"ruby\">-<span class=\"hljs-class\"><span class=\"hljs-keyword\">module</span>-<span class=\"hljs-title\">name</span> <span class=\"hljs-title\">bash</span>-<span class=\"hljs-title\">commons</span> \\</span>\n</span> -<span class=\"ruby\">-tag <VERSION> \\\n</span> -<span class=\"ruby\">-<span class=\"hljs-class\"><span class=\"hljs-keyword\">module</span>-<span class=\"hljs-title\">param</span> <span class=\"hljs-title\">dir</span>=/<span class=\"hljs-title\">foo</span>/<span class=\"hljs-title\">bar</span> \\</span>\n</span> -<span class=\"ruby\">-<span class=\"hljs-class\"><span class=\"hljs-keyword\">module</span>-<span class=\"hljs-title\">param</span> <span class=\"hljs-title\">owner</span>=<span class=\"hljs-title\">my</span>-<span class=\"hljs-title\">os</span>-<span class=\"hljs-title\">username</span> \\</span>\n</span> -<span class=\"ruby\">-<span class=\"hljs-class\"><span class=\"hljs-keyword\">module</span>-<span class=\"hljs-title\">param</span> <span class=\"hljs-title\">group</span>=<span class=\"hljs-title\">my</span>-<span class=\"hljs-title\">os</span>-<span class=\"hljs-title\">group</span></span>\n</span></pre>\n<p>If you don't want to use the Gruntwork Installer, you can use <code>git clone</code> to get the code onto your computer and then\ncopy it to it's final destination manually:</p>\n<pre><span class=\"hljs-symbol\">git</span> clone --<span class=\"hljs-keyword\">branch </span><VERSION> https://github.com/gruntwork-io/<span class=\"hljs-keyword\">bash-commons.git\n</span>\n<span class=\"hljs-symbol\">sudo</span> mkdir -p /<span class=\"hljs-meta\">opt</span>/gruntwork\n<span class=\"hljs-symbol\">cp</span> -r <span class=\"hljs-keyword\">bash-commons/modules/bash-commons/src </span>/<span class=\"hljs-meta\">opt</span>/gruntwork/<span class=\"hljs-keyword\">bash-commons\n</span><span class=\"hljs-symbol\">sudo</span> chown -R <span class=\"hljs-string\">\"my-os-username:my-os-group\"</span> /<span class=\"hljs-meta\">opt</span>/gruntwork/<span class=\"hljs-keyword\">bash-commons\n</span></pre>\n<h2 class=\"preview__body--subtitle\" id=\"instance-metadata-service-versions\">Instance Metadata Service versions</h2>\n<p><code>bash-commons</code> supports both Instance Metadata Service (IMDS) version 1 and 2. Gruntwork and AWS both recommend using version 2 of the Instance Metadata Service whenever possible. Although version 1 is still supported and considered fully secure by AWS, version 2 has been specially hardened against specific threat vectors and is therefore preferable.</p>\n<p>To understand more about Instance Metadata Service version 2 and its features, read <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html\" class=\"preview__body--description--blue\" target=\"_blank\">the official AWS documentation on IMDSv2</a>.</p>\n<p>There are two ways to specify the version of the Instance Metadata Service that <code>bash-commons</code> should use:</p>\n<ol>\n<li>Set the environment variable <code>GRUNTWORK_BASH_COMMONS_IMDS_VERSION</code> to the version of IMDS that you wish to use. Valid values are either <code>1</code> or <code>2</code>.</li>\n<li>Change the value of <code>default_instance_metadata_version</code> to either <code>1</code> or <code>2</code>.</li>\n</ol>\n<h4 id=\"example-of-dynamic-ubuntu-wait-sh-usage\">Example of <code>dynamic-ubuntu-wait.sh</code> usage:</h4>\n<p>You can use the <code>dynamic-ubuntu-wait.sh</code> command after you <a href=\"#install\" class=\"preview__body--description--blue\">install bash-commons</a>:</p>\n<pre><span class=\"hljs-keyword\">bash </span>/<span class=\"hljs-meta\">opt</span>/gruntwork/<span class=\"hljs-keyword\">bash-commons/dynamic-ubuntu-wait.sh\n</span></pre>\n<p>Alternatively, you can call the script without installing by curling it during your existing provisioning/automated installation process:</p>\n<pre>curl -LsS <span class=\"hljs-link\">https://raw.githubusercontent.com/gruntwork-io/bash-commons/</span>[<span class=\"hljs-string\">VERSION</span>]/modules/bash-commons/src/dynamic-ubuntu-wait.sh | bash`\n</pre>\n<p>Where <code>[VERSION]</code> could be: <code>v0.0.3</code>. The latest release can be found <a href=\"#open_modal\" class=\"preview__body--description--blue\">here</a></p>\n<h2 class=\"preview__body--subtitle\" id=\"importing-modules\">Importing modules</h2>\n<p>You can use the <code>source</code> command to "import" the modules you need and use them in your code:</p>\n<pre><span class=\"hljs-keyword\">source</span> /<span class=\"hljs-keyword\">opt</span>/gruntwork/bash-commons/<span class=\"hljs-built_in\">log</span>.<span class=\"hljs-keyword\">sh</span>\n</pre>\n<p>This will make all the functions within that module available in your code:</p>\n<pre><span class=\"hljs-attribute\">log_info</span> <span class=\"hljs-string\">\"Hello, World!\"</span>\n</pre>\n<h2 class=\"preview__body--subtitle\" id=\"available-modules\">Available modules</h2>\n<p>Here's an overview of the modules available in <code>bash-commons</code>:</p>\n<ul>\n<li>\n<p><code>array.sh</code>: Helpers for working with Bash arrays, such as checking if an array contains an element, or joining an\narray into a string with a delimiter between elements.</p>\n</li>\n<li>\n<p><code>assert.sh</code>: Assertions that check a condition and exit if the condition is not met, such as asserting a variable is\nnot empty or that an expected app is installed. Useful for defensive programming.</p>\n</li>\n<li>\n<p><code>aws.sh</code>: A collection of thin wrappers for direct calls to the <a href=\"https://aws.amazon.com/cli/\" class=\"preview__body--description--blue\" target=\"_blank\">AWS CLI</a> and <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html\" class=\"preview__body--description--blue\" target=\"_blank\">EC2\nInstance Metadata</a>. These thin\nwrappers give you a shorthand way to fetch certain information (e.g., information about an EC2 Instance, such as its\nprivate IP, public IP, Instance ID, and region). Moreover, you can swap out <code>aws.sh</code> with a version that returns mock\ndata to make it easy to run your code locally (e.g., in Docker) and to run unit tests.</p>\n</li>\n<li>\n<p><code>aws-wrapper.sh</code>: A collection of "high level" wrappers for the <a href=\"https://aws.amazon.com/cli/\" class=\"preview__body--description--blue\" target=\"_blank\">AWS CLI</a> and <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html\" class=\"preview__body--description--blue\" target=\"_blank\">EC2\nInstance Metadata</a> to simplify common\ntasks such as looking up tags or IPs for EC2 Instances. Note that these wrappers handle all the data processing and\nlogic, whereas all the direct calls to the AWS CLI and EC2 metadata endpoints are delegated to <code>aws.sh</code> to make unit\ntesting easier.</p>\n</li>\n<li>\n<p><code>dynamic-ubuntu-wait.sh</code>: A script that dynamically waits for Ubuntu automatic update mechanism to\nrelease all locks so that <code>apt-get</code> may run without errors.</p>\n</li>\n<li>\n<p><code>file.sh</code>: A collection of helpers for working with files, such as checking if a file exists or contains certain text.</p>\n</li>\n<li>\n<p><code>log.sh</code>: A collection of logging helpers that write logs to <code>stderr</code> with log levels (INFO, WARN, ERROR) and\ntimestamps.</p>\n</li>\n<li>\n<p><code>os.sh</code>: A collection of Operating System helpers, such as checking which flavor of Linux (e.g., Ubuntu, CentOS) is\nrunning and validating checksums.</p>\n</li>\n<li>\n<p><code>string.sh</code>: A collection of string manipulation functions, such as checking if a string contains specific text,\nstripping prefixes, and stripping suffixes.</p>\n</li>\n</ul>\n<h2 class=\"preview__body--subtitle\" id=\"coding-principles\">Coding principles</h2>\n<p>The code in <code>bash-commons</code> follows the following principles:</p>\n<ol>\n<li><a href=\"#compatibility\" class=\"preview__body--description--blue\">Compatibility</a></li>\n<li><a href=\"#code-style\" class=\"preview__body--description--blue\">Code style</a></li>\n<li><a href=\"#everything-is-a-function\" class=\"preview__body--description--blue\">Everything is a function</a></li>\n<li><a href=\"#namespacing\" class=\"preview__body--description--blue\">Namespacing</a></li>\n<li><a href=\"#testing\" class=\"preview__body--description--blue\">Testing</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"compatibility\">Compatibility</h3>\n<p>The code in this repo aims to be compatible with:</p>\n<ul>\n<li>Bash 3</li>\n<li>Most major Linux distributions (e.g., Ubuntu, CentOS)</li>\n</ul>\n<h3 class=\"preview__body--subtitle\" id=\"code-style\">Code style</h3>\n<p>All the code should mainly follow the <a href=\"https://google.github.io/styleguide/shell.xml\" class=\"preview__body--description--blue\" target=\"_blank\">Google Shell Style Guide</a>.\nIn particular:</p>\n<ul>\n<li>The first line of every script should be <code>#!/usr/bin/env bash</code>.</li>\n<li>All code should be defined in functions.</li>\n<li>Functions should exit or return 0 on success and non-zero on error.</li>\n<li>Functions should return output by writing it to <code>stdout</code>.</li>\n<li>Functions should log to <code>stderr</code>.</li>\n<li>All variables should be <code>local</code>. No global variables are allowed at all.</li>\n<li>Make as many variables <code>readonly</code> as possible.</li>\n<li>If a variable is both local and readonly, use <code>local -r</code>.</li>\n<li>If calling to a subshell and storing the output in a variable (foo=<code>$( ... )</code>), do NOT use <code>local -r</code> in the same\nstatement or the <a href=\"https://blog.gruntwork.io/yak-shaving-series-1-all-i-need-is-a-little-bit-of-disk-space-6e5ef1644f67\" class=\"preview__body--description--blue\" target=\"_blank\">exit code will be lost</a>.\nInstead, declare the variable as <code>local</code> on one line and then call the subshell on the next line.</li>\n<li>Quote all strings.</li>\n<li>Use <code>[[ ... ]]</code> instead of <code>[ ... ]</code>.</li>\n<li>Use snake_case for function and variable names. Use UPPER_SNAKE_CASE for constants.</li>\n</ul>\n<h3 class=\"preview__body--subtitle\" id=\"everything-in-a-function\">Everything in a function</h3>\n<p>It's essential that ALL code is defined in a function. That allows you to use <code>source</code> to "import" that code without\nanything actually being executed.</p>\n<h3 class=\"preview__body--subtitle\" id=\"namespacing\">Namespacing</h3>\n<p>Bash does not support namespacing, so we fake it using a convention on the function names: if you create a file\n<code><foo.sh></code>, all functions in it should start with <code>foo_</code>. For example, all the functions in <code>log.sh</code> start with <code>log_</code>\n(<code>log_info</code>, <code>log_error</code>) and all the functions in <code>string.sh</code> start with <code>string_</code> (<code>string_contains</code>,\n<code>string_strip_prefix</code>). That makes it easier to tell which functions came from which modules.</p>\n<p>For readability, that means you should typically give files a name that is a singular noun. For example, <code>log.sh</code>\ninstead of <code>logging.sh</code> and <code>string.sh</code> instead of <code>strings.sh</code>.</p>\n<h3 class=\"preview__body--subtitle\" id=\"testing\">Testing</h3>\n<p>Every function should be tested:</p>\n<ul>\n<li>\n<p>Automated tests are in the <a href=\"/repos/v0.1.9/bash-commons/test\" class=\"preview__body--description--blue\">test</a> folder.</p>\n</li>\n<li>\n<p>We use <a href=\"https://github.com/sstephenson/bats\" class=\"preview__body--description--blue\" target=\"_blank\">Bats</a> as our unit test framework for Bash code. Note: Bats has not been\nmaintained the last couple years, so we may need to change to the <a href=\"https://github.com/bats-core/bats-core\" class=\"preview__body--description--blue\" target=\"_blank\">bats-core</a>\nfork at some point (see <a href=\"https://github.com/sstephenson/bats/issues/150\" class=\"preview__body--description--blue\" target=\"_blank\">#150</a>).</p>\n</li>\n<li>\n<p>We run all tests in the <a href=\"https://hub.docker.com/r/gruntwork/bash-commons-circleci-tests/\" class=\"preview__body--description--blue\" target=\"_blank\">gruntwork/bash-commons-circleci-tests Docker\nimage</a> so that (a) it's consistent with how the CI\nserver runs them, (b) the tests always run on Linux, (c) any changes the tests make, such as writing files or\ncreating OS users, won't affect the host OS, (d) we can replace some of the modules, such as <code>aws.sh</code>, with mocks at\ntest time. There is a <code>docker-compose.yml</code> file in the <code>test</code> folder to make it easy to run the tests.</p>\n</li>\n<li>\n<p>To run all the tests: <code>docker-compose up</code>.</p>\n</li>\n<li>\n<p>To run one test file: <code>docker-compose run tests bats test/array.bats</code>.</p>\n</li>\n<li>\n<p>To leave the Docker container running so you can debug, explore, and interactively run bats: <code>docker-compose run tests bash</code>.</p>\n</li>\n<li>\n<p>If you ever need to build a new Docker image, the <code>Dockerfile</code> is in the <a href=\"/repos/v0.1.9/bash-commons/.circleci\" class=\"preview__body--description--blue\">.circleci folder</a>:</p>\n<pre><span class=\"hljs-symbol\">cd</span> .circleci\n<span class=\"hljs-symbol\">docker</span> <span class=\"hljs-keyword\">build </span>-t gruntwork/<span class=\"hljs-keyword\">bash-commons-circleci-tests </span>.\n<span class=\"hljs-symbol\">docker</span> <span class=\"hljs-keyword\">push </span>gruntwork/<span class=\"hljs-keyword\">bash-commons-circleci-tests\n</span></pre>\n</li>\n</ul>\n<h2 class=\"preview__body--subtitle\" id=\"todo\">TODO</h2>\n<ol>\n<li>Add automated tests for <code>aws.sh</code> and <code>aws-wrapper.sh</code>. We have not tested these as they require either running an\nEC2 Instance or run something like <a href=\"https://github.com/localstack/localstack\" class=\"preview__body--description--blue\" target=\"_blank\">LocalStack</a>.</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"license\">License</h2>\n<p>This code is released under the Apache 2.0 License. Please see\n<a href=\"/repos/v0.1.9/bash-commons/LICENSE\" class=\"preview__body--description--blue\">LICENSE</a> and\n<a href=\"/repos/v0.1.9/bash-commons/NOTICE\" class=\"preview__body--description--blue\">NOTICE</a> for more details.</p>\n<p>Copyright © 2018 Gruntwork, Inc.</p>\n","repoName":"bash-commons","repoRef":"v0.1.9","serviceDescriptor":{"serviceName":"Bash Commons","serviceRepoName":"bash-commons","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws","gcp","azure"],"description":"A collection of reusable Bash functions for handling common tasks such as logging, assertions, string manipulation, and more.","imageUrl":"grunt.png","licenseType":"open-source","technologies":["Bash"],"compliance":[],"tags":[""],"noDisplayInUI":true},"serviceCategoryName":"Reference Architecture","fileName":"README.md","filePath":"","title":"Repo Browser: Bash Commons","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
.circleci
.gitignore
