Explore live open-source projects and AI models.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

LLM inference in C/C++

A high-throughput and memory-efficient inference and serving engine for LLMs

Inference code for Llama models

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

Port of OpenAI's Whisper model in C/C++

TypeScript-first schema validation with static type inference

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Making large AI models cheaper, faster and more accessible

Official inference framework for 1-bit LLMs