As a Test Automation specialist, I see lot of people who uses test credentials such as emails, passwords, webtokens, client credentials placed all over their scripts, It is not a crime to store and access the credentials in our framework, however sometimes it becomes a major problem when the tools like GitGuardian, SonarQube, CodeScan starts to trouble you by showing the policy breaking incidents to the management if suppose it was enforced by your organization.
So in this tutorial we are going to see not only to remove secrets and rewrite the history from the git, we will also learn what are all the different ways we can use to store our credentials in the workspace.
- What is a Secret?
- What are the threats of exposing a secret to the outer world?
- What is GitGuardian?
- What is GGShield and How to Use It?
What is a Secret?
Any sensitive information that we desire to keep hidden might be referred to as a secret in comon usage. Secrets are frequently used to refer to digital authentication credentials that provide access to systems or data when talking about them in the context of software development. These are most frequently security certificates, usernames, passwords and API keys.
What are the threats of exposing a secret to the outer world?
When secrets are spread across multiple systems, the so-called “security attack” increases. This is the point through which an unauthorized user can gain access to your systems or data. In the case of compromised secret , every time when a secret gets into another system, unauthorized access to the systems gets multiplied.
Most internal systems are not a suitable place to store sensitive information, even if those systems are private. No company wants plain text credit card numbers in databases, personal information in application logs, bank account information in a Google Docs or Microsoft Excel.
As a general security principle, data should remain secure to the most extent possible.
What is GitGuardian?
GitGuardian monitors Git repository round the clock for your secrets and sensitive data. It catch the leaks, intrusions. Enforce security rules across your VCS, DevOps tools, and infrastructure-as-code configurations – with GitGuardian’s code security policy engine.
If you are personal user or running a freelance project , you can also utilize the free version of the GitGuardian and integrate that with your GitHub, GitLab, BitBucket etc.
Here in this case, I had setup my GitGuardian workspace as a personal user and linked my GitHub repostory into my workspace
https://www.gitguardian.com/pricing
https://www.gitguardian.com/faq
What is GGShield and How to Use It?
GGShield is a CLI application that runs in your local environment or in a CI environment to help you detect more than 350+ types of secrets, as well as other potential security vulnerabilities or policy breaks.
https://www.gitguardian.com/ggshield
A pre-commit hook is a client-side git hook that runs right before the commit is created. So whenever we commit our code, it first runs through the GGShield and checks for any potential secrets included and if detects something, it won’t allow you to commit the code.
Step -1 – Install Python and add it to the PATH
https://www.python.org/downloads/
Step -2 – Install Git SCM (ignore if already available in your system)
Download the package from https://git-scm.com/downloads and Install in your machine.
Step -3 – Install pre-commit using pip
Launch Command Prompt and run the following command
pip install pre-commitStep -4 – Create a .pre-commit-config.yaml file in your repository’s root path with the following content:
Clone my below project for test purpose or use your test project if you have one set up already.
https://github.com/vigneshram20/SecretsRemovalTest.git
Navigate to the project root folder and create the below file
repos:
- repo: https://github.com/gitguardian/ggshield
rev: v1.14.1
hooks:
- id: ggshield
language_version: python3
stages: [commit]
Step -5 – Open GitBash from your project root folder and execute the below command
pre-commit install
Step – 6 – Open Command Prompt & Install ggshield using pip installer
pip install ggshield 
Step -7 -Authenticate your GitGuardian credentials for ggshield secret scan
ggshield auth login
Once the GitGuardian URL opened in your browser. Enter the credentials and login to your workspace, that’s it, Your PAT token will be automatically created and added to your system.
Step -8 -Navigate to the repository root path and open GitBash, Install GGhield local for that particular repo
ggshield install --mode local
Step -9 – Time for some real action .
Import the project in eclipse add any Base64 type encoded secret in your file.
For Test Tokens, Use base64encode.org or https://canarytokens.org/generate
Open GitBash from the project root folder and commit the file which contains secret.
As soon as you execute the commit command, the pre-commit hook will be iniated and since we have accidentally placed our secret in the class file, it now tells you to remove the secrets from the file before commiting.
So now whenever you are commiting your files through the GitBash, it will first go through the pre-commit step, validates the code for any potential secret and allow you to do the commit.
Make sure we have to use only GitBash tool to commit our files otherwise it won’t get detected if you commit the files using eclipse native GIT tool.
Note!
