Betterleaks

     ○
     ○
ghp_ ● qOomCIZBWchHR4v5FPp9UiQRS9CyigrCkXXuIJQPfe63f12a
     ○

Betterleaks is a configurable, fast, and thorough secrets scanner. It is maintained by the folks who made Gitleaks, including the original author. Check out this series of blog posts to learn how the detection engine works: 1. Regex is all you need, 2. Rare Not Random, 3. Express YourCELf.

Development is supported by

Notable Features

Feature	Description
Expr-based filtering	Write contextual rule filters that evaluate fragment (data chunks) attributes (like git author, commit message, and file path) and finding data to reduce false positives. If you're coming from Gitleaks, think of this feature as a more expressive [[allowlist]] system.
Secrets Validation	Validate if a detected secret is active by making asynchronous HTTP requests directly from within the rule definition using Expr.
Token Efficiency filtering	Filter out natural language false positives by using BPE tokenization to measure how "rare" or non-human a string is.
Fast scans	Achieve fast performance through sane default parallelization settings, ahocorasick keyword filters, and re2.
New Sources	Support for sources like GitHub, GitLab, Hugging Face, S3, and more. It's easy to add new sources too!
Portability	Runs on any modern OS/Arch. The small binary can be integrated in any system.

Installation

# Package managers
brew install betterleaks
brew install betterleaks/tap/betterleaks

# Fedora Linux
sudo dnf install betterleaks

# Containers
docker pull ghcr.io/betterleaks/betterleaks:latest

# Go
go install github.com/betterleaks/betterleaks@latest

# Source
git clone https://github.com/betterleaks/betterleaks
cd betterleaks
make build

Usage

# Scan Git
betterleaks git /path/to/repo -v --git-workers=16

# Scan local filesystem
betterleaks dir /path/to/file/or/dir -v

# Scan GitHub org
betterleaks github https://github.com/betterleaks
# Scan GitHub user
betterleaks github https://github.com/cooluser123456789 --include issues,prs,actions,releases,gists
# Scan specific resource, like a PR... but exclude the description (only scan comments)
betterleaks github https://github.com/betterleaks/betterleaks/pull/113

# Scan GitLab group or project
betterleaks gitlab https://gitlab.com/mygroup
betterleaks gitlab https://gitlab.com/mygroup/myproject --include issues,mrs,releases,ci-jobs
# Scan a specific GitLab merge request
betterleaks gitlab https://gitlab.com/mygroup/myproject/-/merge_requests/42

# Scan Hugging Face models, datasets, Spaces, and buckets
betterleaks huggingface https://huggingface.co/myorg
betterleaks hf https://huggingface.co/datasets/myorg/mydataset
betterleaks hf --include=discussions,prs https://huggingface.co/myorg/model
betterleaks hf hf://buckets/myorg/mybucket/path

# Scan a public s3 dataset (Common Crawl).
betterleaks s3 https://commoncrawl.s3.us-east-1.amazonaws.com/crawl-data/CC-MAIN-2018-17/segments/1524125937193.1/warc/
# Enumerate and scan every bucket in a Cloudflare R2 account
betterleaks s3 'https://<account-id>.r2.cloudflarestorage.com/*'

# Scan stdin
cat some_file.txt | betterleaks stdin -v

For more advanced scanning examples check out the scanning doc.

Configuration

Betterleaks' strength comes from its expressive configuration. Filtering and validation logic are defined as Expr. Previously this logic was implemented in CEL; existing CEL-shaped configs are still accepted for compatibility, but new configs should use Expr. prefilters run before any regex matching occurs and only have access to the attributes map. attributes describe a resource like a git patch. Use prefilters to quickly bail out before more expensive scanning happens. filters, on the other hand, get evaluated post-regex match and have access to the attributes map and candidate finding data like finding["secret"] or finding["match"].

# Global prefilter, it runs before expensive regex calls
prefilter = '''
filter.matchesAny(get(attributes, "path", ""), [
  `(?i)\.(?:bmp|gif|jpe?g|png|svg|tiff|pdf|exe)$`,
  `(?:^|/)node_modules(?:/.*)?$`,
  `(?:^|/)vendor(?:/.*)?$`
])
|| get(attributes, "git.author_name", "") == "renovate[bot]"
'''

# Global filter, it runs for _every_ candidate secret.
filter = '''
filter.containsAny(finding["secret"], [
  "EXAMPLE",
  "CHANGEME",
  "YOUR_API_KEY_HERE",
  "0000000000000000"
])
'''

# An array of tables that contain data on how to detect secrets
[[rules]]
id = "github-fine-grained-pat"
description = "GitHub Fine-Grained Personal Access Token, risking unauthorized repo access."
regex = '''github_pat_\w{82}'''
keywords = ["github_pat_"]

# Rule-level filter
filter = '''
(
    get(attributes, "git.author_name", "") == "ci-runner" &&
    filter.matchesAny(get(attributes, "path", ""), [`^mocks/`]) &&
    finding["secret"] contains "TESTING"
)
|| (filter.entropy(finding["secret"]) <= 3.0)
'''

# Post-match-and-filter async validation check
validate = '''
let r = http.get("https://api.github.com/user", {
    "Accept": "application/vnd.github+json",
    "Authorization": "token " + secret
  });
r.status == 200 && (r.json?.login ?? "") != "" ? {
    "result": "valid",
    "username": r.json?.login ?? "",
    "name": r.json?.name ?? "",
    "scopes": get(r.headers, "x-oauth-scopes", "")
  } : r.status in [401, 403] ? {
    "result": "invalid",
    "reason": "Unauthorized"
  } : validate.unknown(r)
'''

Refer to the default betterleaks config for examples and the config docs for more information about the betterleaks.toml config. If you're using Betterleaks in production, it is recommended you maintain your own config instead of extending the upstream default config directly. This keeps your rule set stable across Betterleaks upgrades and lets you review new upstream rules before adopting them.

Test out your rules in the Betterleaks Playground