biomap-research

/

proteinglm-1b-mlm

like 7

Follow

Biomap 29

Dear Biomap-research group, @Bo1015

Thank you for the new SOTA model!

Following the instructions on the Model Card, loading a model for sequence classification task results in an error:

# model for the sequence-level tasks
model = AutoModelForSequenceClassification.from_pretrained("Bo1015/proteinglm-10b-mlm",  trust_remote_code=True, torch_dtype=torch.bfloat16)

AttributeError                            Traceback (most recent call last)
Cell In[22], line 4
      1 # model for the sequence-level tasks
      2 # config = AutoConfig.from_pretrained('Bo1015/proteinglm-1b-mlm', trust_remote_code=True, num_labels=7)
      3 # model = AutoModelForSequenceClassification.from_pretrained("Bo1015/proteinglm-1b-mlm",  trust_remote_code=True, torch_dtype=torch.bfloat16)
----> 4 model = AutoModelForSequenceClassification.from_pretrained("Bo1015/proteinglm-10b-mlm",  trust_remote_code=True, torch_dtype=torch.bfloat16)
      7 # tokenizer  = AutoTokenizer.from_pretrained("Bo1015/proteinglm-1b-mlm", trust_remote_code=True, use_fast=True)
      8 # model = AutoModelForMaskedLM.from_pretrained("Bo1015/proteinglm-1b-mlm",  trust_remote_code=True, torch_dtype=torch.bfloat16)
      9 
   (...)     33 #     model_classification = model_classification.to(device)
     34 #     print(f'Using device: {device}')

File ~/bioenv/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:564, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    562     cls.register(config.__class__, model_class, exist_ok=True)
    563     model_class = add_generation_mixin_to_remote_model(model_class)
--> 564     return model_class.from_pretrained(
    565         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    566     )
    567 elif type(config) in cls._model_mapping.keys():
    568     model_class = _get_model_class(config, cls._model_mapping)

File ~/bioenv/lib/python3.11/site-packages/transformers/modeling_utils.py:309, in restore_default_torch_dtype.<locals>._wrapper(*args, **kwargs)
    307 old_dtype = torch.get_default_dtype()
    308 try:
...
    209 if key != "attribute_map" and key in super().__getattribute__("attribute_map"):
    210     key = super().__getattribute__("attribute_map")[key]
--> 211 return super().__getattribute__(key)

AttributeError: 'ProteinGLMConfig' object has no attribute 'activation_func'

Thank you for the help,
Vadim