huggingface多卡加载模型

Hug[gin](https://geek.csdn.net/educolumn/1aef582107c22aa4d9f243890549bf39?spm=1055.2569.3001.10083)g Face是一个自然[语言](https://geek.csdn.net/educolumn/05c8de7581a582e6d46821757663ed30?spm=1055.2569.3001.10083)处理（NLP）模型和工具库的开源社区，提供了许多用于加载和使用预训练模型的工具。在Hug[gin](https://geek.csdn.net/educolumn/1aef582107c22aa4d9f243890549bf39?spm=1055.2569.3001.10083)g Face中，多卡加载模型可以通过使用`torch.nn.DataParallel`或`torch.nn.parallel.DistributedDataParallel`来实现。 1. 使用`torch.nn.DataParallel`加载模型： ```python from transformers import BertModel, BertTokenizer import torch # 加载模型和分词器 model_name = "bert-base-uncased" model = BertModel.from_pretrained(model_name) tokenizer = BertTokenizer.from_pretrained(model_name) # 将模型放在多个GPU上 model = torch.nn.DataParallel(model) # 使用模型进行推理 inputs = tokenizer("Hello, how are you?", return_[tensor](https://geek.csdn.net/educolumn/0ebc891269ff76b86c4b41f64bffd5db?spm=1055.2569.3001.10083)s="pt") [output](https://geek.csdn.net/educolumn/3fa40697d361ab41848a3d891b656e90?spm=1055.2569.3001.10083)s = model(**inputs) ```