Deployment issues on sagemaker
I am trying to deploy on AWS sagemaker using the above example.
But after running it I get a failure message
Cloudwatch said that the model type gemma3 is Unsupported
Is there a solution?
Hi @tychen6677 , Apologies for late reply,
Gemma 3 models on SageMaker has been a known problem due to compatibility issues with earlier versions of the Hugging Face transformers library, which SageMaker relies on.
To use Gemma-3 models, you need the latest development version of the Transformers library (4.50.0 or higher).. A new stable version of Transformers is now available which is compatible to Gemma3. Please update it using pip install -U transformers and try again. Let us know if this helps!
Thank you.
Hello @tychen6677 , @juanjucm tested with the latest TGI DLC and it should work fine now! make sure you upgrade sagemakersdk to get the latest dlc uri.
Have a good day,
Simon