PaddlePaddle/FastDeploy

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A flexible, high-performance serving system for machine learning models

A Cloud Native Batch System (Project under CNCF)

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny vLLM + Qwen.

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

Serve, optimize and scale PyTorch models in production

A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.

Database system for AI-powered apps

TensorFlow template application for deep learning

A comprehensive guide to building RAG-based LLM applications for production.

A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.