添加链接 注册    登录
link管理
链接快照平台
  • 输入网页链接，自动生成快照
  • 标签化管理网页链接
相关文章推荐
打盹的斑马  ·  JavaScript Date Formats·  1 月前    · 
行走的键盘  ·  Delphi-使用TNetHTTPReque ...·  3 周前    · 
深情的黄瓜  ·  So I m stuck on S3 ...·  1 周前    · 
乐观的火锅  ·  VS 2012 TCP server ...·  1 周前    · 
买醉的煎饼果子  ·  木材科学与技术考研科目是什么-高顿教育·  7 月前    · 
刚失恋的炒面  ·  喂饭级人工智能版Photoshop（Beta ...·  7 月前    · 
深情的韭菜  ·  迪士尼收购福克斯了影响我看漫威吗？_Hulu·  9 月前    · 
含蓄的红薯  ·  Pierre-Luc Arsenault ...·  9 月前    · 
乐观的黄花菜  ·  童书·专访｜邓正祺：鳄鱼是我对群体生活不太行 ...·  1 年前    · 
link管理  ›  So I m stuck on S3 access I created a service account called Flyte #flyte-support
pod try
https://discuss.flyte.org/t/2586554/so-i-m-stuck-on-s3-access-i-created-a-service-account-called
深情的黄瓜
1 周前
https://flyte.org logo
Join Slack
Powered by
So I’m stuck on S3 access - I created a service ac...
# flyte-support
s
So I’m stuck on S3 access - I created a service account called 
flyte-executor
with the 
flyte-user-role
(which has full s3 access) attached as an annotation and running Flyte executions with this service account, but it’s giving me PutObject access denied error. This service account is in the project+domain namespace. What am I doing wrong?
t
stupid question but the iam role has the putobject permission right?
this is annoying to do I apologize, but maybe we can also try to run the pod manually, but instead of the pyflyte command run instead 
aws sts get-caller-identity
s
flyte-user-role has 
AmazonS3FullAccess
policy attached to it
How can I run the pod manually?
I see the pod for the task that failed but the status is failed so I can’t exec into it
When I log the pod, I see some other errors as well: for example, 
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
@ thankful-minister-83577 Should I maybe add 
sleep infinity
to the args in the yaml and 
exec
into it to run 
aws sts get-caller-identity
? Let me know when you got a minute - thanks!
t
cc: @ thankful-minister-83577
t
hey sorry is this still an issue @ sticky-angle-28419
s
Yes - can you provide me a bit more detail on what to do to start debugging why I have s3 permission issues?
Thanks! @ thankful-minister-83577
t
if you want to run the pod manually you have to create a yaml file, but yeah this will work
create the pod, make sure you have the correct service account, remove the status field, ownership and anything else that looks like it might be related to an existing flyteworkflow execution
update the args to sleep infinity and then yeah you can attach to it and investigate.
s
OK so create a completely new pod
I’ll try it tonight and report back - thank you!
t
i’m not convinced that’s necessary though. are you sure the service account has the iam role attached
maybe easier to first try to assume the role manually just on your laptop through the aws cli
run the same s3 command
just isolate out the aws side first, and then deal with the k8s side
s
Hmm ok - here’s my sa yaml that’s deployed
Copy code 
apiVersion: v1
imagePullSecrets:
- name: gcr-json-key
kind: ServiceAccount
metadata:
  annotations:
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: arn:aws:iam::xxx:role/flyte-user-role
  labels:
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: pulumi
  name: flyte-executor
  namespace: shelly-robotics-bipedal-robot-development
  resourceVersion: "57747250"
  uid: 9db3e9da-cf32-4a78-8b06-81d83b66c611
secrets:
- name: flyte-executor-token-l6rkj
As you can see the annotation is there - unless that’s not sufficient to link an SA to an iam role
t
can you 
get pod <pod name> -o yaml
and grep for “iam”
you should see aws-iam-token a couple times
and if that’s the case then yeah, can you try assuming the role locally
s
So 
get pod <pod name> -o yaml | grep 'iam'
returns this
Copy code 
value: arn:aws:iam::xxx:role/flyte-user-role
      name: aws-iam-token
  - name: aws-iam-token
Does this look right?
t
Yeah
Can you try assuming locally?
s
When I try 
aws sts assume-role …
locally, I get this error: 
An error occurred (AccessDenied) when calling the AssumeRole operation
t
mm
okay sure, try copying the pod yaml and editing so that the command is just a sleep
maybe your role is not set up for you to assume it
s
Hmm - I have admin access to all resources. I do have multiple clusters running, one of which is flyte and here’s the trusted relationships:
Copy code 
{
  "Version": "2012-10-17",
  "Statement": [
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::xxx:oidc-provider/oidc.eks.us-west-1.amazonaws.com/id/2A6739B7813451087E3258C60BC37CF4"
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "<http://oidc.eks.us-west-1.amazonaws.com/id/2A6739B7813451087E3258C60BC37CF4:aud|oidc.eks.us-west-1.amazonaws.com/id/2A6739B7813451087E3258C60BC37CF4:aud>": "<http://sts.amazonaws.com|sts.amazonaws.com>"
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "<http://ec2.amazonaws.com|ec2.amazonaws.com>"
      "Action": "sts:AssumeRole"
}
Where oidc is from flyte eks cluster
t
what do you mean?
s
Oh I was responding to your comment regarding the role not being setup to be assumed
I think a pod in the flyte cluster should be able to assume this role via service account
t
yeah try that i guess, just make the pod yaml
let us know if you need an example
s
OK would it be fine to just get rid of the status from the yaml I get from the failed task pod? And then update the args to 
sleep infinity
?
OK I just changed the pod name and created it and ran 
aws sts get-caller-identity
and here’s the response:
Copy code 
{
  "UserId": "xxx:botocore-session-1663707573",
  "Account": "xxx",
  "Arn": "arn:aws:sts::xxx:assumed-role/flyte-user-role/botocore-session-1663707573"
}
t
can you try to run the failing s3 command also?
s
Hmm where can I grab this command?
You mean the pyflyte command?
t
no it was in the error
putobject access denied error
just
Copy code 
cat > abc
hello
^C
and then try to aws s3 cp abc s3:/….
s
ok one sec
No error - confirmed that the file was uploaded to the right bucket path in s3 
aws s3 cp abc <s3://sidetrek-flyte-cluster-flyte-bucket/metadata/propeller/shelly-robotics-bipedal-robot-development-an6gvhl5dn8vr44nn9ds/n0/data/0/abc.txt>
t
but when the task runs with the same role, it can’t?
s
Yes that seems to be the case
One thing though - maybe I’m doing it wrong
So when I launch the execution in the flyte dashboard
I only fill out Kubernetes Service Account in the 
Security Context
section - leaving IAM Role field empty
Could that be what’s creating the problem??
Do I need to fill out both? I assumed service account is enough since it references the iam role in it
t
no one should be enough
s
Oh ok then it’s strange
t
can you change your task code to call sts get caller identity
s
I seem to be able to run s3 commands manually in the pod fine but it errors out when I run it from dashboard
t
s
You mean replace the actual code of the failing task with just this?
Copy code 
>>> import boto3
 
推荐文章
打盹的斑马  ·  JavaScript Date Formats
1 月前
行走的键盘  ·  Delphi-使用TNetHTTPRequest.Post发送JSON - Delphi学习大师  Delphi教程 视频教程 文档大全
3 周前
深情的黄瓜  ·  So I m stuck on S3 access I created a service account called Flyte #flyte-support
1 周前
乐观的火锅  ·  VS 2012 TCP server side diconnected by accident, how reconnect that after server run again?-VBForums
1 周前
买醉的煎饼果子  ·  木材科学与技术考研科目是什么-高顿教育
7 月前
刚失恋的炒面  ·  喂饭级人工智能版Photoshop（Beta）安装与使用教程 - 哔哩哔哩
7 月前
深情的韭菜  ·  迪士尼收购福克斯了影响我看漫威吗？_Hulu
9 月前
含蓄的红薯  ·  Pierre-Luc Arsenault | Professionals | Kirkland & Ellis LLP
9 月前
乐观的黄花菜  ·  童书·专访｜邓正祺：鳄鱼是我对群体生活不太行的人的祝愿_翻书党_澎湃新闻-The Paper
1 年前
Link管理   ·   51好读   ·   Sov5搜索   ·   小百科
link管理 - 链接快照平台