issue #AttributeError: 'LatentAttentionConfig' object has no attribute '_attn_implementation_internal'
(genie) khushwant@Khushwant-OMEN:/CodeGenie$ python app/services/main.py/CodeGenie$
Traceback (most recent call last):
File "/home/khushwant/CodeGenie/app/services/main.py", line 22, in
model = AutoModel.from_pretrained('nvidia/NV-Embed-v2', trust_remote_code=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/khushwant/miniconda3/envs/genie/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/khushwant/miniconda3/envs/genie/lib/python3.12/site-packages/transformers/modeling_utils.py", line 3832, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/khushwant/.cache/huggingface/modules/transformers_modules/nvidia/NV-Embed-v2/26db444e203771ea231d32a795344655e1119d40/modeling_nvembed.py", line 323, in init
self.latent_attention_model = AutoModel.from_config(config.latent_attention_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/khushwant/miniconda3/envs/genie/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 438, in from_config
return model_class._from_config(config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/khushwant/miniconda3/envs/genie/lib/python3.12/site-packages/transformers/modeling_utils.py", line 1478, in _from_config
if config._attn_implementation_internal is not None:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/khushwant/miniconda3/envs/genie/lib/python3.12/site-packages/transformers/configuration_utils.py", line 264, in getattribute
return super().getattribute(key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'LatentAttentionConfig' object has no attribute '_attn_implementation_internal'
(genie) khushwant@Khushwant-OMEN:
Hi, @khushwant04 . This is related to transformers package version related to this issue: https://huggingface.co/nvidia/NV-Embed-v1/discussions/50