Browse the Repo

file-type-icon.circleci
file-type-icon_ci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconaws-helpers
file-type-iconbuild-helpers
file-type-iconcheck-url
file-type-iconcircleci-helpers
file-type-icondocs-generator
file-type-icontest-fixtures
file-type-iconGopkg.lock
file-type-iconGopkg.toml
file-type-iconREADME.md
file-type-iconcli.go
file-type-iconcli_test.go
file-type-iconerrors.go
file-type-iconfile.go
file-type-icongenerator.go
file-type-icongenerator_test.go
file-type-iconglob.go
file-type-iconglob_test.go
file-type-iconlogger.go
file-type-iconmain.go
file-type-iconplaceholder.go
file-type-iconplaceholder_test.go
file-type-iconurls.go
file-type-iconurls_test.go
file-type-iconec2-backup
file-type-iconecs-deploy-runner-invoke-iam-policy
file-type-iconecs-deploy-runner-standard-configuration
file-type-iconecs-deploy-runner
file-type-icongit-helpers
file-type-icongruntwork-module-circleci-helpers
file-type-iconiam-policies
file-type-iconinfrastructure-deploy-script
file-type-iconinfrastructure-deployer
file-type-iconinstall-jenkins
file-type-iconjenkins-server
file-type-iconkubernetes-circleci-helpers
file-type-iconterraform-helpers
file-type-icontest
file-type-icontestdep
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME-CircleCI.adoc
file-type-iconREADME-Jenkins.adoc
file-type-iconREADME-Terraform-Terragrunt-Pipeline.adoc
file-type-iconREADME-TravisCI.adoc
file-type-iconREADME.adoc
file-type-iconsetup.cfg

Browse the Repo

file-type-icon.circleci
file-type-icon_ci
file-type-icon_docs
file-type-iconexamples
file-type-iconmodules
file-type-iconaws-helpers
file-type-iconbuild-helpers
file-type-iconcheck-url
file-type-iconcircleci-helpers
file-type-icondocs-generator
file-type-icontest-fixtures
file-type-iconGopkg.lock
file-type-iconGopkg.toml
file-type-iconREADME.md
file-type-iconcli.go
file-type-iconcli_test.go
file-type-iconerrors.go
file-type-iconfile.go
file-type-icongenerator.go
file-type-icongenerator_test.go
file-type-iconglob.go
file-type-iconglob_test.go
file-type-iconlogger.go
file-type-iconmain.go
file-type-iconplaceholder.go
file-type-iconplaceholder_test.go
file-type-iconurls.go
file-type-iconurls_test.go
file-type-iconec2-backup
file-type-iconecs-deploy-runner-invoke-iam-policy
file-type-iconecs-deploy-runner-standard-configuration
file-type-iconecs-deploy-runner
file-type-icongit-helpers
file-type-icongruntwork-module-circleci-helpers
file-type-iconiam-policies
file-type-iconinfrastructure-deploy-script
file-type-iconinfrastructure-deployer
file-type-iconinstall-jenkins
file-type-iconjenkins-server
file-type-iconkubernetes-circleci-helpers
file-type-iconterraform-helpers
file-type-icontest
file-type-icontestdep
file-type-icon.gitignore
file-type-icon.pre-commit-config.yaml
file-type-iconCODEOWNERS
file-type-iconLICENSE.txt
file-type-iconREADME-CircleCI.adoc
file-type-iconREADME-Jenkins.adoc
file-type-iconREADME-Terraform-Terragrunt-Pipeline.adoc
file-type-iconREADME-TravisCI.adoc
file-type-iconREADME.adoc
file-type-iconsetup.cfg
EC2 backup

EC2 backup

Snapshot your EC2 instances on a scheduled basis.

Code Preview

Preview the Code

mobile file icon

README.md

down

Docs Generator

This folder contains a Go app that can be used to generate public documentation from a private GitHub repo. It loops over all the files in a given input folder (typically a private GitHub repo), and copies each documentation file (e.g. markdown files such as this Readme) mostly unchanged to an output folder (typically a public GitHub repo). It also copies each non-documentation file (e.g. source files) to the output folder, but it replaces their contents with placeholder text.

For an example of the output of this process, check out the public GitHub repo module-ci-public.

Motivation

At Gruntwork, we have developed a number of infrastructure "packages" or "modules" that are thoroughly tested and documented. Each of these modules is in a private GitHub repo with a name of the format module-XXX, such as module-vpc, module-ecs, and module-aws-monitoring. We want to make the documentation from these repos available online publicly so that:

  1. The documentation gets indexed by Google.
  2. Existing customers find the documentation while searching online.
  3. Potential new customers find the documentation while searching online.
  4. We can link to the documentation from gruntwork.io to make it clearer to potential customers exactly what we're offering.

The question is, how can you publish the documentation publicly, but not the actual source code?

This Docs Generator project is the answer. It allows us to add a CI job to each module-XXX repo that extracts the documentation from the repo and puts it into a module-XXX-public repo.

Install

The Docs Generator can be installed using the Gruntwork Installer:

gruntwork-install --binary-name 'docs-generator' --repo 'https://github.com/gruntwork-io/module-ci' --tag '0.0.1'

Alternatively, you can download the Docs Generator binary for your OS from the Releases Page.

Usage

Command-line options

The docs-generator binary supports the following options:

  • input-path PATH (required): Generate documentation from the files and folders in PATH.
  • output-path PATH (required): Write the output to PATH.
  • github-org NAME (required): The name of the GitHub organization. E.g. The 'foo' in github.com/foo/bar. Used to update URLs.
  • github-repo-name NAME (required): The name of the GitHub repo. E.g. The 'bar' in github.com/foo/bar. Used to update URLs.
  • doc-pattern PATTERN (optional): Copy files that match PATTERN to the output path, unchanged. Supports standard file patterns (e.g. *.txt, foo/**/bar). Make sure to quote PATTERN so bash doesn't expand it. May be specified more than once.
  • exclude-pattern PATTERN (optional): A PATTERN to exclude while copying to the output path. Supports standard file patterns (e.g. *.tf, foo/**/bar). Make sure to quote PATTERN so bash doesn't expand it. May be specified more than once.
  • help, (optional): Show the help text and exit.
  • version, (optional): Print the version and exit.

Run docs-generator --help to see the full docs, including all teh default values.

Examples

Generate documentation from the folder /home/ubuntu/module-vpc into the folder /home/ubuntu/module-vpc-public for the GitHub repo module-vpc:

docs-generator --input-path /home/ubuntu/module-vpc --output-path /home/ubuntu-module-vpc-public --github-org gruntwork-io --github-repo-name module-vpc

Generate documentation from the folder /home/ubuntu/module-vpc into the folder /home/ubuntu/module-vpc-public for the GitHub repo module-vpc, including only Markdown files (*.md as documentation) and excluding the vendor folder:

docs-generator --input-path /home/ubuntu/module-vpc --output-path /home/ubuntu-module-vpc-public --github-org gruntwork-io --github-repo-name module-vpc --doc-pattern "*.md" --exclude-pattern "vendor" --exclude-pattern "vendor/*"

Note that you need to quote the --doc-pattern and --exclude-pattern parameters so that bash does not expand them immediately.

Developing

To run the Docs Generator locally:

go run `ls *.go | grep -v _test.go`

To run the Docs Generator tests:

go test -v -parallel 128

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?