Zurück zum Ranking

makcedward/nlpaug

Jupyter Notebookmakcedward.github.io

Data augmentation for NLP

nlpaugmentationmachine-learningartificial-intelligencedata-sciencenatural-language-processingadversarial-attacksadversarial-exampleaiml
Sterne-Wachstum
Sterne
4.7k
Forks
473
Wochenwachstum
Issues
71
2k4k
Jan. 2023März 2024Mai 2025Juli 2026
README



Build Code Quality Downloads

nlpaug

This python library helps you with augmenting NLP, audio, and spectrogram data for machine learning projects. Visit this introduction to understand about Data Augmentation in NLP. Augmenter is the basic element of augmentation while Flow is a pipeline to orchestrate multiple augmenters together.

Features

  • Generate synthetic data for improving model performance without manual effort
  • Simple, easy-to-use and lightweight library. Augment data in a few lines of code
  • Plug and play with common machine learning and neural network frameworks
  • Support textual, audio, and spectrogram inputs
  • Python 3.12-ready V2 baseline with offline-first tests and GitHub Actions coverage

Textual Data Augmentation Example


Acoustic Data Augmentation Example


Section Description
Quick Demo How to use this library
Augmenter Introduce all available augmentation methods
Installation How to install this library
Recent Changes Latest enhancement
Extension Reading More real life examples or researchs
Reference Reference of external resources such as data or model

Quick Demo

Augmenter

Augmenter Target Augmenter Action Description
Textual Character KeyboardAug substitute Simulate keyboard distance error
Textual OcrAug substitute Simulate OCR engine error
Textual RandomAug insert, substitute, swap, delete Apply augmentation randomly
Textual Word AntonymAug substitute Substitute opposite meaning word according to WordNet antonym
Textual ContextualWordEmbsAug insert, substitute Feeding surroundings word to BERT, DistilBERT, RoBERTa or XLNet language model to find out the most suitlabe word for augmentation
Textual RandomWordAug swap, crop, delete Apply augmentation randomly
Textual SpellingAug substitute Substitute word according to spelling mistake dictionary
Textual SplitAug split Split one word to two words randomly
Textual SynonymAug substitute Substitute similar word according to WordNet/ PPDB synonym
Textual TfIdfAug insert, substitute Use TF-IDF to find out how word should be augmented
Textual WordEmbsAug insert, substitute Leverage word2vec, GloVe or fasttext embeddings to apply augmentation
Textual BackTranslationAug substitute Leverage two translation models for augmentation
Textual ReservedAug substitute Replace reserved words
Textual Sentence ContextualWordEmbsForSentenceAug insert Insert sentence according to XLNet, GPT2 or DistilGPT2 prediction
Textual AbstSummAug substitute Summarize article by abstractive summarization method
Textual LambadaAug substitute Using language model to generate text and then using classification model to retain high quality results
Signal Audio CropAug delete Delete audio's segment
Signal LoudnessAug substitute Adjust audio's volume
Signal MaskAug substitute Mask audio's segment
Signal NoiseAug substitute Inject noise
Signal PitchAug substitute Adjust audio's pitch
Signal ShiftAug substitute Shift time dimension forward/ backward
Signal SpeedAug substitute Adjust audio's speed
Signal VtlpAug substitute Change vocal tract
Signal NormalizeAug substitute Normalize audio
Signal PolarityInverseAug substitute Swap positive and negative for audio
Signal Spectrogram FrequencyMaskingAug substitute Set block of values to zero according to frequency dimension
Signal TimeMaskingAug substitute Set block of values to zero according to time dimension
Signal LoudnessAug substitute Adjust volume

Flow

Augmenter Augmenter Description
Pipeline Sequential Apply list of augmentation functions sequentially
Pipeline Sometimes Apply some augmentation functions randomly

Installation

The library targets Python 3.12+.

Install the core package:

pip install nlpaug

Install feature extras as needed:

pip install "nlpaug[transformers]"
pip install "nlpaug[nltk]"
pip install "nlpaug[word-embs]"
pip install "nlpaug[audio]"
pip install "nlpaug[lambada]"

Install the latest GitHub version:

pip install "git+https://github.com/makcedward/nlpaug.git"

If you use WordEmbsAug (word2vec, glove or fasttext), download the pretrained assets first:

from nlpaug.util.file.download import DownloadUtil
DownloadUtil.download_word2vec(dest_dir='.')
DownloadUtil.download_glove(model_name='glove.6B', dest_dir='.')
DownloadUtil.download_fasttext(model_name='wiki-news-300d-1M', dest_dir='.')

If you use SynonymAug with PPDB, download the language pack from:

http://paraphrase.org/#/download

Testing

Run the default offline suite:

python -m pytest

Run the optional integration suite:

python -m pytest -m integration

uv workflow

Create a Python 3.12 environment and run the default suite:

make test

Install the common optional extras and run the fuller local suite:

make test-full

Install the heaviest optional extras and run only integration tests:

make test-integration

If you prefer direct uv commands:

uv venv --python 3.12
uv pip install -p .venv/bin/python -e ".[dev]"
uv run --python .venv/bin/python pytest

Or use the repo scripts directly:

./scripts/setup_uv.sh core
./scripts/test_uv.sh core
./scripts/setup_uv.sh full
./scripts/test_uv.sh full
./scripts/setup_uv.sh integration
./scripts/test_uv.sh integration

Recent Changes

2.0.0 Jun 2026

  • Upgrade runtime baseline to Python 3.12+
  • Refresh major optional dependencies, including transformers 5.9, gensim 4.4, librosa 0.11, and NumPy 2.x
  • Add uv-based setup and test scripts for core, full, and integration workflows
  • Modernize offline-first tests so the default suite runs without downloading real models
  • Mock transformer-backed augmenters in tests and add broader regression coverage
  • Add GitHub Actions coverage reporting and local coverage scripts
  • Refactor shared augmenter hot paths and sentence generation internals for better readability and performance

1.1.11 Jul 6, 2022

See changelog for more details.

Extension Reading

Reference

This library uses data (e.g. capturing from internet), research (e.g. following augmenter idea), model (e.g. using pre-trained model) See data source for more details.

Citation

@misc{ma2019nlpaug,
  title={NLP Augmentation},
  author={Edward Ma},
  howpublished={https://github.com/makcedward/nlpaug},
  year={2019}
}

This package is cited by many books, workshop and academic research papers (70+). Here are some of examples and you may visit here to get the full list.

Workshops cited nlpaug

Book cited nlpaug

Research paper cited nlpaug

Contributions


sakares saengkaew


Binoy Dalal


Emrecan Çelik

Ähnliche Repositories
huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

PythonPyPIApache License 2.0nlpnatural-language-processing
huggingface.co/transformers
162.8k34k
hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

PythonPyPIApache License 2.0fine-tuningllama
llamafactory.readthedocs.io
73.4k9k
microsoft/AI-For-Beginners

12 Weeks, 24 Lessons, AI for All!

Jupyter NotebookMIT Licensedeep-learningartificial-intelligence
52.5k10.6k
apachecn/ailearning

AiLearning:数据分析+机器学习实战+线性代数+PyTorch+NLTK+TF2

PythonPyPIOtherfp-growthapriori
ailearning.apachecn.org
42.4k11.5k
666ghj/BettaFish

微舆:人人可用的多Agent舆情分析助手,打破信息茧房,还原舆情原貌,预测未来走向,辅助决策!从0实现,不依赖任何框架。

PythonPyPIGNU General Public License v2.0agent-frameworkdata-analysis
deepwiki.com/666ghj/BettaFish
41.8k7.6k
rohitg00/ai-engineering-from-scratch

Learn it. Build it. Ship it for others.

PythonPyPIMIT Licenseagentsai
aiengineeringfromscratch.com
41.6k6.9k
google-research/bert

TensorFlow code and pre-trained models for BERT

PythonPyPIApache License 2.0nlpgoogle
arxiv.org/abs/1810.04805
40.1k9.7k
google/langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

PythonPyPIApache License 2.0llmnlp
pypi.org/project/langextract/
37.6k2.6k
hankcs/HanLP

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

PythonPyPIApache License 2.0nlpnatural-language-processing
hanlp.com
36.5k10.9k
ashishpatel26/500-AI-Machine-learning-Deep-learning-Computer-vision-NLP-Projects-with-code

500 AI Machine learning Deep learning Computer vision NLP Projects with code

awesomemachine-learning
35.6k7.4k
explosion/spaCy

💫 Industrial-strength Natural Language Processing (NLP) in Python

PythonPyPIMIT Licensenatural-language-processingdata-science
spacy.io
33.8k4.7k
stanford-oval/storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

PythonPyPIMIT Licenselarge-language-modelsnlp
storm.genie.stanford.edu
30.2k2.8k