Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amazon Bedrock Workshop - [Module 3 Model Customization - ] fails to create model-customization-job for Titan or llama #233

Closed
zdarova opened this issue Apr 16, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@zdarova
Copy link
Contributor

zdarova commented Apr 16, 2024

Describe the bug
In module 3 of the workshop in Workshop Studia as test event, when I try to run in SageMaker Studio Classic the titan or llama notebook for creating a model-customization-job, it returns the following error:

AccessDeniedException: An error occurred (AccessDeniedException) when calling the CreateModelCustomizationJob operation: User: arn:aws:sts::****:assumed-role/bedrock-workshop-studio-SageMakerExecutionRole-rVM4toc9ynQ4/SageMaker is not authorized to perform: bedrock:CreateModelCustomizationJob on resource: arn:aws:bedrock:us-west-2:****:model-customization-job/* with an explicit deny in a service control policy

To Reproduce
Run the amazon-bedrock-workshop/03_Model_customization/02_fine-tuning_llama2.ipynb

Expected behavior
No errors

Screenshots

Desktop (please complete the following information):

  • OS: Win10
  • Browser: Chrome

Smartphone (please complete the following information):
ND

! Additional context !
Region: us-west-2

@zdarova zdarova added the bug Something isn't working label Apr 16, 2024
@johngiles12345
Copy link

This is still an issue a year later.
The 01_fine-tuning-titan-lite.ipyn lab does not work. An issue was submitted for this last year. Maybe remove this module?
On the SageMakerExecutionRole, there is an explicit deny for CreateModelCustomizationJob. But even if you remove the deny permission from the role, there is still an SCP that will deny this action. So there is no way to create a fine-tuning job if you are an event participant that I can find.
AccessDeniedException: An error occurred (AccessDeniedException) when calling the CreateModelCustomizationJob operation: User: arn:aws:sts::362720299941:assumed-role/bedrock-workshop-studio-v2-SageMakerExecutionRole-8h7CgWyI4zUh/SageMaker is not authorized to perform: bedrock:CreateModelCustomizationJob on resource: arn:aws:bedrock:us-west-2:362720299941:model-customization-job/* with an explicit deny in a service control policy

@w601sxs
Copy link
Contributor

w601sxs commented Mar 25, 2025

Sorry for the late response - this is known internally by workshop facilitators - workshop studio team blocks customization and long running jobs. We also think this is a bad idea for AWS hosted workshops, as customers may be waiting for many hours to finish a job. Customers can try this on their own accounts . Closing for now @zdarova and @johngiles12345

@w601sxs w601sxs closed this as completed Mar 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants