Questions tagged [boto3]

1

votes
1

answer
2.1k

Views

DynamoDB Primary Key strategy

I'm dabbling with DynamoDB (using boto3) for the first time, and I'm not sure how to define my Partition Key. I'm used to SQL, where you can use AUTO_INCREMENT to ensure that the Key will always increase. I haven't seen such an option in DynamoDB - instead, when using put_item, the "primary key attr...
scubbo
1

votes
1

answer
3.1k

Views

Get the Tag “Name” of an EBS volume with Boto 3 AWS Lambda

I am using the script https://github.com/xombiemp/ec2-take-snapshots-lambda/blob/master/ec2-take-snapshots-lambda.py for AWS lambda. I would like create snapshot and set tag "Name" with the same tag "Name" of the EBS volume, but in the documentation of boto 3 not find how get tag value of EBS volume...
1

votes
1

answer
1.9k

Views

aws boto3 client Stubber help stubbing unit tests

I'm trying to write some unit tests for aws RDS. Currently, the start stop rds api calls have not yet been implemented in moto. I tried just mocking out boto3 but ran into all sorts of weird issues. I did some googling and found http://botocore.readthedocs.io/en/latest/reference/stubber.html So I ha...
Justin S
1

votes
1

answer
233

Views

Incorrect padding on a Celery SQS consumer

I'm trying to set up a Celery consumer for a SQS queue. I'm using Celery 4.1.0 with Python3 and I sent a message with the library boto3 (1.5.28) this way: response = queue.send_message(MessageBody='Hello World') and everything it's OK so far. Then I tried to setup a Celery consumer this way: @app.ta...
alauri
1

votes
1

answer
464

Views

Boto3 get only S3 buckets of specific region

The following code sadly lists all buckets of all regions and not only from "eu-west-1" as specified. How can I change that? import boto3 s3 = boto3.client("s3", region_name="eu-west-1") for bucket in s3.list_buckets()["Buckets"]: bucket_name = bucket["Name"] print(bucket["Name"])
lony
1

votes
1

answer
23

Views

Python boto3: receive versionId after uploading a file to S3

I want to upload a file to a version-enabled S3 bucket and need its version number. Ideally, without a separate API call to avoid any possibility of a race condition. I'm using the following code snippet for upload (which is working fine): s3 = boto3.client("s3") s3.upload_fileobj(file_handle, bucke...
SaturnFromTitan
0

votes
1

answer
16

Views

AWS Python Lambda with multiple functions

I have an AWS Lambda function in python3.7. The way its set up im running the lambda_handler(event, context) function and passing data to a separate function that calls itself multiple times depending on what is passed into it. How do I then return data from the second function? import json import b...
bjwhip
2

votes
2

answer
57

Views

When to use dynamodb.client, dynamodb.resource and dynamodb.Table?

I am working on Dyanamodb using Boto3 and Python. One of the issues I am finding is when should we use the dynamodb.client, dynamodb.resource and dynamodb.Table?
Shash
1

votes
1

answer
1.1k

Views

AWS boto3 source code

How can we get a look at the AWS boto library code. Not all the code is present in the python package. For example if I wanted to understand how the wait_until_running() method of Waiter class is implemented i.e. if it is implemented serialized fashion and other features of the code. Although the do...
Mikki
1

votes
1

answer
3.9k

Views

Upload file to AWS S3 using Boto3 with full access permissions

Hi i need help in setting permissions to an object when uploading to S3. I want to make it public read/write. I have a function as follows. def UploadFile(self, destFilePath, data): self._bucket.put_object(Key=destFilePath, Body=data, ACL='public-read-write') When it runs, i have a permissions error...
RStyle
1

votes
1

answer
546

Views

upload file to amazon cloud subfolder using python boto3

I'm writing some python script and I am trying to upload a file to the amazon cloud using boto3. The problem is I want to upload the file to a particular subfolder...in some cases I will need to upload the file to a subfolder of a subfolder. I'm trying to do this: s3.meta.client.upload_file( "C:\\...
J4ce
0

votes
0

answer
3

Views

Search file within folders of s3 bucket

I am able to find a file which is directly placed inside s3 bucket but if file is placed inside folders like A/B/test.csv I am NOT able to find it. # This gives 0 result key='A/B/test.csv' bucket = s3.Bucket(bucket_name) objs = list(bucket.objects.filter(Prefix=key)) print len(objs)
user3186866
1

votes
2

answer
5.1k

Views

boto3 file_upload does it check if file exists

I was looking through the boto3 documentation and could not find if it natively supports a check to see if the file already exists in s3 and if not do not try and re-upload. Here's what I have working: import boto3 s3_client = boto3.client('s3') s3_bucket = 'bucketName' s3_folder = 'folder1234/' tem...
chowpay
1

votes
1

answer
740

Views

Python 3 Boto 3, AWS S3: Get object URL

I need to retrieve an public object URL directly after uploading a file, this to be able to store it in a database. This is my upload code: s3 = boto3.resource('s3') s3bucket.upload_file(filepath, objectname, ExtraArgs={'StorageClass': 'STANDARD_IA'}) I am not looking for a presigned URL, just the U...
HyperDevil
1

votes
1

answer
206

Views

AWS Cognito - Resetting User Password Documentation Seems to Contradict Itself

The Problem I have worked with both AdminResetPassword and ForgotPassword and I found a contradiction in the API documentation that is confusing and seems to indicate there is no current solution for full admin initialized reset password flow. The AdminResetPassword documentation says: When a devel...
guardhunt
1

votes
1

answer
33

Views

'S3' object has no attribute 'put_object_retention'

I'm trying to implement the object lock function introduced recently by AWS s3. The code works perfectly when i run it locally. But as soon as i deploy it on amazon it breaks and gives me the error. I have tried deploying it using zappa, ECS as well as plain lambda. It appears that amazon python pac...
Ninad Gaikwad
0

votes
0

answer
10

Views

Python AWS Boto3 DynamoDB Paginator hangs when used

I am trying to get a paginator working on a DynamoDB query running in Python using Boto3. When I remove the pagination, the query runs fine. When I introduce the pagination the query finishes, but the pagination loop hangs. paginator = client.get_paginator('query') response_iterator = paginator.pa...
Garet Jax
0

votes
0

answer
6

Views

Use S3 metadata or a database to search and filter objects in amazon buckets?

I have a bucket with json files, I want to expose those files along with a basic description of what they contains. So far I have a list of files that could be like that: {'key' : 'value', 'description' : 'text'} and the key object that match each file, it is the filename. So, for fetching a list o...
user305883
0

votes
0

answer
3

Views

Is there an alternative way to know when DynamoDb table descreased?

I have a weird problem with one of my tables in DynamoDb. When I make a petition to describe it I find that it was decreased three times today, while in the AWS console I can see only one scale down, that coincides with the one returned by LastDecreaseDateTime when performing a describe_table(TableN...
Drubio
1

votes
2

answer
924

Views

Fetching ElastiCache Tags

I am trying to fetch ElastiCache Tags using Boto3 and Python. In boto3, there is a function called- list_tags_for_resource(). But, the problem I am facing is, how to find the resource name. I am using the following code: from boto3.session import Session sess = Session(aws_access_key_id=id,aws_secr...
upaang saxena
1

votes
2

answer
458

Views

get number of used resources in AWS

AWS sets the limit on number of resources. I need to figure out the number of used resources for my account using boto3 and python in a script Is there any way to do this ? I am beginner in both boto3 and python For example: EBS limits which has multiple resources and its limits defined under it.
user3351750
1

votes
1

answer
2.7k

Views

Using AWS ECS with Boto3

I would like to lunch a Task on ECS cluster and wait for the task termination. import boto3 client = boto3.client('ecs') response = client.run_task( cluster='default', taskDefinition='RGB', overrides={ 'containerOverrides': [ { 'name': 'RGB', 'command': [ 'python', '-u', 'rgb.py' ] } ] }...
p.magalhaes
1

votes
1

answer
1.2k

Views

copy file from gcs to s3 in boto3

I am looking to copy files from gcs to my s3 bucket. In boto2, easy as a button. conn = connect_gs(user_id, password) gs_bucket = conn.get_bucket(gs_bucket_name) for obj in bucket: s3_key = key.Key(s3_bucket) s3_key.key = obj s3_key.set_contents_from_filename(obj) However in boto3, I am lost trying...
fahad daniyal
1

votes
3

answer
1.5k

Views

Amazon SES send_email Text body with spacing/newlines

I am facing an issue where all of my text e-mails are scrunched together and do not have new lines persisting through the sending process. Here is the code: def send_ses_message(email_to, subject, body): ses = init_ses_internal() if ses: ses.send_email( Source='[email protected]', Destination={ 'T...
morissette
1

votes
2

answer
725

Views

find OS and licensed software for EC2 instances

I am trying to figure out the OS and licensed software of an AWS ec2 instance by using API (specifically python's boto3 but if the solution comes from another SDK, that's fine by me). I naturally tried using describe instances: client = session.client('ec2') ec2.describe_instances() But the data is...
Avi
1

votes
1

answer
4.5k

Views

boto : ResourceNotFoundException Dynamodb

I have a table - user_details that has 500 read/write capacity + 500 on indexes as well with over 5k items, currently. I am querying the table based on username and it seems to be giving me "ResourceNotFoundException" all the time - ClientError: An error occurred (ResourceNotFoundException) when ca...
user3089927
1

votes
2

answer
1.9k

Views

Get the names and count of all buckets in AWS S3

I'm new to Boto3 and AWS API, and I want to get the list of buckets' names and the count of the available buckets in S3. Any help is appreciated.
l3_08
1

votes
1

answer
6.4k

Views

How to download files from s3 given the file path using boto3 in python

Pretty basic but I am not able to download files given s3 path. for eg, I have this s3://name1/name2/file_name.txt import boto3 locations = ['s3://name1/name2/file_name.txt'] s3_client = boto3.client('s3') bucket = 'name1' prefix = 'name2' for file in locations: s3_client.download_file(bucket, 'file...
Atihska
2

votes
0

answer
13

Views

AWS : Running Rekognition on DeepLens device

I am creating a DeepLens project to recognise people, when one of select group of people are scanned by the camera. The project uses a lambda, which processes the images and triggers the 'rekognition' aws api. On AWS lambda console ( which has 1.8.9 boto version ), I get following issue when I try t...
Deep
2

votes
1

answer
148

Views

Boto3 Client Session Failing in Kivy App

I'm working on a Kivy app which needs to upload a file to S3 using boto3 but fails in the JSON decoder.py when trying to load the endpoint model and establish a client connection. This works fine locally running on 16.04 but fails after compiling with buildozer and running on Lollipop 5.1.1. Buildo...
darthpaul
2

votes
0

answer
76

Views

Howto put object to s3 with Content-MD5

I have tried to upload an XML File to S3 using boto3. As recommended by Amazon, I would like to send a Base64 Encoded MD5-128 Bit Digest(Content-MD5) of the data. https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html https://boto3.amazonaws.com/v1/documentation/api/latest/reference/serv...
Meschkov
2

votes
1

answer
38

Views

writer.writerow() doesn't write to the correct column

I have three DynamoDB tables. Two tables have instance IDs that are part of an application and the other is a master table of all instances across all of my accounts and the tag metadata. I have two scans for the two tables to get the instance IDs and then query the master table for the tag metadata...
dmn0972
3

votes
1

answer
803

Views

Using Boto3 and Python How Make a Folder Public

I want to make a folder called img that already exists within my private bucket public. I'm using Boto3. I just want to make this folder public not anything else using a script.. This is how I'm currently connecting to the bucket and how far I have got.... ACCESS_KEY_ID = 'xxxxx' ACCESS_KEY_SECRET =...
Prometheus
1

votes
3

answer
963

Views

Remote signing of boto request for python client

I want to directly upload/download files to Amazon S3 from python clients, running in some users machines. I have a server, that hosts the access Id and Secret keys, as they cannot be in the users side, that can be used to generate a pre-signed url, and that the clients can connect via API to reques...
drodri
-2

votes
0

answer
28

Views

What are the three dots for in an python class object?

For example, I am working with a SDK and I am not understanding why this object has 3 dots. Example: s3.buckets.list Shouldn't it just be a attribute and a method in this notation?
Mdot
1

votes
3

answer
2.8k

Views

To check whether AWS instance is up after reboot using python

Is there a way to check if an AWS instance has finally come up in python either using boto3 or otherwise. The running state doesnt differentiate between rebooting and finally up phase.
Oliver Blue
1

votes
2

answer
3.4k

Views

How to invoke another lambda async and pass context to it?

I'm trying to get working two basic lambdas using Python2.7 runtime for SQS message processing. One lambda reads from SQS invokes and passes data to another lambda via context. I'm able to invoke the other lambda but the user context is empty in it. This is my code of SQS reader lambda: import boto3...
Centurion
1

votes
1

answer
7.1k

Views

How to upgrade botocore1.2.6 to 1.4 or above present with boto3?

I have boto3 installed on my linux machine. When I pass below command, I get botocore version as 1.2.6 >>> import boto3 >>> print boto3.__version__ 1.2.6 How to upgrade to botocore present within boto3 to 1.4 or latest version? In case I need to downgrade botocore 1.4 to botocore 1.2.6 , what are th...
Dave
1

votes
2

answer
99

Views

How can I extract all instances of a key from a list within a python dictionary?

I have a python dictionary named 'available_volumes'. This dictionary contains a key 'Volumes', and within this key a list of 'VolumeIds.' How can I print each of the 'VolumeId's within this list? Here is some simplified sample data for my 'available_volumes' data: {'Volumes': [ {'VolumeId': 'vol-...
Xenidious

View additional questions