Hugging Face - Part6 - Repo model xyz is gated and you must be authenticated to access it

Today, while working locally on my machine with mistralai/Mistral-7B-Instruct-v0.2 from Hugging Face, I encountered the following issue:

 

Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/resolve/main/config.json.
Repo model mistralai/Mistral-7B-Instruct-v0.2 is gated. You must be authenticated to access it.

OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2.
403 Client Error. (Request ID: Root=1-66266e88-14951c696b21d7515a1dd516;df373d0d-261c-41ec-9142-bca579e082fc)

Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/resolve/main/config.json.
Access to model mistralai/Mistral-7B-Instruct-v0.2 is restricted and you are not in the authorized list. Visit https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2 to ask for access.

Upon conducting a Google search, I observed that certain Hugging Face repositories are restricted, requiring an access token for downloading models locally from these gated repositories.

Following are the discussion threads:

https://huggingface.co/google/gemma-7b/discussions/31

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/discussions/93

Therefore, if you intend to utilize this code for downloading and engaging with a Mistral model on your local system, you'll require a Hugging Face access token and must implement minor adjustments as outlined below:

import os

MODEL_ID = "mistralai/Mistral-7B-Instruct-v0.2"

access_token = os.environ["HFREADACCESS"]

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, token=access_token)

model = AutoModelForCausalLM.from_pretrained(MODEL_ID, token=access_token)

Note: You can pass the access token to the script as an environment variable. If this is running on Kubernetes as a pod, then you can consider creating a secret with the access token, inject the secret to the container as env using secretKeyRef.  

Next, you'll need to log in to Hugging Face, navigate to the model card you wish to download, and select "Agree and access repository". Once completed, executing the Python script should enable you to download the model locally and interact with it seamlessly. 

Hope it was useful. Cheers!