AI Model Training & Runtime

SGLang is a high-performance serving framework for large language models and multimodal models

A framework for efficient model inference with omni-modality models

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025)

A modular Swift SDK for audio processing with MLX on Apple Silicon

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way

A retargetable MLIR-based machine learning compiler and runtime toolkit

Pytorch domain library for recommendation systems

On-device AI across mobile, embedded and edge for PyTorch

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI

An API standard for multi-agent reinforcement learning environments, with popular reference environments and related utilities

Low-code framework for building custom LLMs, neural networks, and other AI models

RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI

PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle

Open Machine Learning Compiler Framework

FlagGems is an operator library for large language models implemented in the Triton Language

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators

Overview of Japanese LLMs

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data

AI on the way. An RDBMS approach to deep learning. Declarative, explainable, scalable, optimizable, easy to deploy, all that good stuff

Ongoing research training transformer models at scale

Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application

Flower: A Friendly Federated AI Framework

Community maintained hardware plugin for vLLM on Ascend

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem)

LiteRT-LM is an open-source GitHub project focused on developer-centric engineering workflows, multi-agent orchestration, workflow automation

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed

A high-throughput and memory-efficient inference and serving engine for LLMs

A Datacenter Scale Distributed Inference Serving Framework

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

Google efficiency-focused multimodal model for coding, tool use, and complex agent workflows

Mistral's trillion-parameter open-weight multimodal frontier model

Meta multimodal reasoning model for tool use, computer use, coding, and agentic tasks

Anthropic’s Claude Sonnet 5.5 model for capable, efficient work

Describe model needs in natural language and get an optimized production API

TrueFoundry provides enterprise AI deployment, model gateway, observability, and scaling workflows to run LLM and agent apps in production

Open Source Continuous Inference Benchmarking Qwen3.5, DeepSeek, GPTOSS - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 vs H100 & soon™ TPUv6e/v7/Trainium2/3

Analyze logs, metrics and traces for AI applications

Access multiple AI model families through one API

Access chat, image, video, and audio models through one gateway

Use a typed decision model through a playground and API

Six open-weight 2B-class models for phones, laptops, and robots

Local model routing layer for coding agents, subscriptions, keys, and local models

Desktop AI workspace with local models, connected tools, and creative workflows