Volver al ranking

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

audiogptmusicsoundspeechtalking-head
Crecimiento de estrellas
Estrellas
10.2k
Forks
852
Crecimiento semanal
Issues
44
4k6k8k10k
abr 2023may 2024jun 2025jul 2026
ArtefactosPyPIpip install audiogpt
README

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

arXiv GitHub Stars visitors Hugging Face

We provide our implementation and pretrained models as open source in this repository.

Get Started

Please refer to run.md

Capabilities

Here we list the capability of AudioGPT at this time. More supported models and tasks are coming soon. For prompt examples, refer to asset.

Currently not every model has repository.

Speech

Task Supported Foundation Models Status
Text-to-Speech FastSpeech, SyntaSpeech, VITS Yes (WIP)
Style Transfer GenerSpeech Yes
Speech Recognition whisper, Conformer Yes
Speech Enhancement ConvTasNet Yes (WIP)
Speech Separation TF-GridNet Yes (WIP)
Speech Translation Multi-decoder WIP
Mono-to-Binaural NeuralWarp Yes

Sing

Task Supported Foundation Models Status
Text-to-Sing DiffSinger, VISinger Yes (WIP)

Audio

Task Supported Foundation Models Status
Text-to-Audio Make-An-Audio Yes
Audio Inpainting Make-An-Audio Yes
Image-to-Audio Make-An-Audio Yes
Sound Detection Audio-transformer Yes
Target Sound Detection TSDNet Yes
Sound Extraction LASSNet Yes

Talking Head

Task Supported Foundation Models Status
Talking Head Synthesis GeneFace Yes (WIP)

Acknowledgement

We appreciate the open source of the following projects:

ESPNetNATSpeechVisual ChatGPTHugging FaceLangChainStable Diffusion

Repositorios relacionados
huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

PythonPyPIApache License 2.0nlpnatural-language-processing
huggingface.co/transformers
162.8k34k
FFmpeg/FFmpeg

Mirror of https://git.ffmpeg.org/ffmpeg.git

COtherffmpegvideo
ffmpeg.org
62.3k14k
mpv-player/mpv

🎥 Command line media player

COthermpvmplayer
mpv.io
36.1k3.4k
OpenBMB/VoxCPM

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

PythonPyPIApache License 2.0audiodeeplearning
voxcpm.com
34k3.9k
ossrs/srs

SRS is a simple, high-efficiency, real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181, with codec support for H.264, H.265, AV1, VP9, AAC, Opus, and G.711.

C++MIT Licensertmplive-streaming
ossrs.io
29.1k5.7k
Anjok07/ultimatevocalremovergui

GUI for a Vocal Remover that uses Deep Neural Networks.

PythonPyPIMIT Licensevocal-removerpytorch
25.5k1.9k
goldfire/howler.js

Javascript audio library for the modern web.

JavaScriptnpmMIT Licensejavascriptaudio
howlerjs.com
25.3k2.3k
ExistentialAudio/BlackHole

BlackHole is a modern macOS audio loopback driver that allows applications to pass audio to other applications with zero additional latency.

COtheraudiodriver
19.4k828
modelscope/FunASR

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

PythonPyPIMIT Licensepytorchspeech-recognition
modelscope.github.io/FunASR/
19.4k1.9k
kyleneideck/BackgroundMusic

Background Music, a macOS audio utility: automatically pause your music, set individual apps' volumes and record system audio.

C++GNU General Public License v2.0audioaudio-utility
19.2k775
alyssaxuu/screenity

The free and privacy-friendly screen recorder with no limits 🎥

JavaScriptnpmGNU General Public License v3.0screen-capturescreencast
screenity.io
18.4k1.5k
mahmoud/awesome-python-applications

💿 Free software that works great, and also happens to be open-source Python.

Jupyter Notebookpythonapplication
ftp//you:relookin@it
17.9k2.7k