Transformers documentation
Run training on Amazon SageMaker
Get started
Tutorials
Run inference with pipelinesWrite portable code with AutoClassPreprocess dataFine-tune a pretrained modelTrain with a scriptSet up distributed training with 🤗 AccelerateLoad and train adapters with 🤗 PEFTShare your modelAgentsGeneration with LLMs
Task Guides
Natural Language Processing
Audio
Computer Vision
Multimodal
Generation
Prompting
Developer guides
Use fast tokenizers from 🤗 TokenizersRun inference with multilingual modelsUse model-specific APIsShare a custom modelTemplates for chat modelsTrainerRun training on Amazon SageMakerExport to ONNXExport to TFLiteExport to TorchScriptBenchmarksNotebooks with examplesCommunity resourcesCustom Tools and PromptsTroubleshootContribute new quantization method
Performance and scalability
OverviewQuantization Instantiate a big modelDebuggingXLA Integration for TensorFlow ModelsOptimize inference using `torch.compile()`
Efficient training techniques
Methods and tools for efficient training on a single GPUMultiple GPUs and parallelismFully Sharded Data ParallelDeepSpeedEfficient training on CPUDistributed CPU trainingTraining on TPU with TensorFlowPyTorch training on Apple siliconCustom hardware for trainingHyperparameter Search using Trainer API
Optimizing inference
Contribute
How to contribute to 🤗 Transformers?How to add a model to 🤗 Transformers?How to convert a 🤗 Transformers model to TensorFlow?How to add a pipeline to 🤗 Transformers?TestingChecks on a Pull Request
Conceptual guides
PhilosophyGlossaryWhat 🤗 Transformers can doHow 🤗 Transformers solve tasksThe Transformer model familySummary of the tokenizersAttention mechanismsPadding and truncationBERTologyPerplexity of fixed-length modelsPipelines for webserver inferenceModel training anatomyGetting the most out of LLMs
API
Main Classes
Agents and ToolsAuto ClassesBackbonesCallbacksConfigurationData CollatorKeras callbacksLoggingModelsText GenerationONNXOptimizationModel outputsPipelinesProcessorsQuantizationTokenizerTrainerDeepSpeedFeature ExtractorImage Processor
Models
Text models
Vision models
Audio models
Video models
Multimodal models
Reinforcement learning models
Time series models
Graph models
Internal Helpers
You are viewing v4.40.0 version. A newer version v5.8.1 is available.
Run training on Amazon SageMaker
The documentation has been moved to hf.co/docs/sagemaker. This page will be removed in transformers 5.0.
Table of Content
- Train Hugging Face models on Amazon SageMaker with the SageMaker Python SDK
- Deploy Hugging Face models to Amazon SageMaker with the SageMaker Python SDK