Questions tagged [amazon-s3]

Amazon S3 (simple storage service) is an online object storage service from Amazon Web Services.

0
votes
0answers
7 views

How to save multipart form data to a file in AWS Lambda

I am writing a serverless api using Node.js and Typescript. I want to send a .xlsx file from the client side and save the file in S3 bucket. I tried using the npm package busboy. And the file that ...
0
votes
0answers
8 views

Enable TLS in bareos client

I have a bareos client trying to communicate with aws s3 server. As per http://doc.bareos.org/master/html/bareos-manual-main-reference.html#x1-400006 I can use TLS Enable & TLS Require options to ...
0
votes
1answer
11 views

AWS S3 basic Usage Report - recent operations not included?

I've noticed that when you generate the AWS Usage Report (the basic old type as referenced here https://docs.aws.amazon.com/AmazonS3/latest/dev/aws-usage-report.html), there is a delay between the ...
0
votes
0answers
14 views

spark s3 incremental load parquet overwrite

My data is parquet and partitioned in S3. This is essentially an incremental load but with S3. I have two data sets, one is current and other is changed data. I can filter to partitions I need with ...
0
votes
0answers
6 views

How to use FineUploader to upload to S3 with asp.net?

I am working on getting FineUploader to work with uploading to S3 using asp.net. Does anyone have an example for creating the endpoint code in C#? I really appreciate the help. Thank you. Warren
0
votes
1answer
13 views

How to Troubleshoot 'Cannot Connect to Proxy' Error - AWS S3

New to AWS and AWS CLI, I have installed and configured the AWS CLI, and I am simply trying to list the buckets in S3, but I am behind a proxy. How do I troubleshoot and resolve and the following ...
0
votes
1answer
9 views

AWS S3 bucket encryption - bucket property setting vs. bucket policy setting

I have created an AWS S3 bucket and in property, I have enabled "Default encryption". Then Do I have still need to create a bucket policy similar to below? Note I just want to make s3 encrypted in a ...
0
votes
3answers
22 views

Writing csv files to local host from a docker container

I am trying to set up a very basic data processing project where I use docker to create an ubuntu environment on an EC2, install python, take an input csv, perform some simple data manipulation, then ...
0
votes
0answers
9 views

Copy nested files to root directory in AWS s3 bucket

I have a nested file structure in an AWS bucket, and I want to move all the file contents to another folder without copying the folder hierarchy. For example, I'd like to move these files: s3://my-...
0
votes
0answers
12 views

AWS Cognito and AWS S3 integration?

Is it possible to restrict AWS S3 objects such that only users who authenticate via AWS Cognito gain access to the object? I haven't figured out a way to do this, but it seems obvious to me that this ...
0
votes
0answers
8 views

AWS S3 SVG File is read as binary/octet-stream

I have an icon assets located on s3, and have a page in angular that looks like this <img alt="Some Logo" src="assets/icons/logo.svg" width="200"> It used to always work properly. But ...
1
vote
1answer
16 views

Accessing S3 bucket from local pyspark using assume role

Background: In order to allow developers to build and unit test code on an easy-to-use environment, we built a local Spark environment with other tools integrated to it. However, we also want to ...
-4
votes
2answers
26 views

Using php with angular application hosted on AWS

I'm new to AWS. I created a DB and I am currently hosting the website right now. This is great! The problem is that I don't know how to integrate my PHP files that I was using before the start of AWS....
0
votes
1answer
12 views

Make batch copy requests to AWS S3 with Python

I wrote a lambda function to copy files in an s3 bucket into another s3 bucket and I need to move a very large number of these files. To try and meet the volume requirements I was looking for a way to ...
0
votes
0answers
8 views

How to send a file to a S3 with vuejs with a presigned URL?

I wanto upload a file to minio using vuejs. So for I have added the following code to my vue project <template> <div class="app-container"> <h1>Minio Code</h1> <input ...
0
votes
1answer
22 views

How to copy Redshift S3 Unloaded Data on daily basis to Azure?

I want to build a mechanism that will unload the data from redshift to S3 on daily basis in a folder and that data will get copied to Auzre on daily basis, I want to build it in such a way so that ...
2
votes
0answers
14 views

How to authenticate digitalocean spaces obejcts

I am using digital ocean spaces to store user files(images, videos, and others). I can access it by using amazon-sdk boto3. Tried using oAuth2 authentication to authenticate the user for accessing the ...
0
votes
0answers
10 views

Can't make CloudFront supply cache-control headers

I've done lots of reading on this but none of the solutions I have seen seem to work for IIS websites - most seem to suggest some server-side solution but none of that works for me. I'm optimising ...
0
votes
0answers
15 views

Incremental update of the data in AWS S3

Incremental update of S3 buckets without natural keys I need to design an etl flow. OLTP systems are sharing customer, product, campaign and sales record via files. I want to transfer these files ...
0
votes
0answers
7 views

CSV upload to aws s3 is not in utf-8

Using Python, I am uploading multiple CSVs into Amazon S3 and found that they are not in UTF-8. Is there any setting or a method wherein I could set the encoding to UTF-8 at the folder level so that ...
0
votes
0answers
15 views

Cross region Export/Import DynamoDB data - Oregon to Frankfurt

I'm trying to export/import DynamoDB data from one account to another account. For that im using AWS Data Pipeline. Here is my scenario. I've two accounts, Account A (in Oregon region). Account B (...
0
votes
0answers
21 views

Thread Pools ThreadStart and Finished Problem xamarin iOS

I am trying to download an XML file of size 85MB in my Xamarin iOS iPad App. Sometimes download completes successfully, but other times download hangs inbetween somewhere without giving any error. ...
0
votes
0answers
10 views

Your Amazon Music marketplace is not configured correctly for this device

I am trying to test my audio sample, but getting error like 'Your Amazon Music marketplace is not configured correctly for this device.UnsupportedAmazonMarketplaceMusic....
2
votes
1answer
20 views

Amazon S3 response does not return date header

I've written a js SDK that fetches a static file from S3: fetch(new Request("https://url.to.my.S3.file.js")).then((response) => { //logging the response headers response.headers.forEach((...
0
votes
0answers
7 views

How to merge videos in S3 Bucket to form one Video

I have uploaded a video as multiple chunks into S3 bucket, I have these videos eTags List. How can I merge these videos to form a single video using java? 1) I don't want to download the chunks to my ...
0
votes
0answers
19 views

How to make a updater for electron application

I created a desktop application for my ionic app using electron. Now what I need is I want to implement auto update function for this. I followed several documentations, but neither helps. https://...
0
votes
1answer
13 views

S3 serving mpeg-dash, will load the manifest but CORS fails on the packages

I've been using S3 to serve MPEG-Dash videos and it's been working fine. I've made some changes to the video processing jobs, but they're still publishing to the same place but now I'm getting the ...
0
votes
0answers
20 views

AWS S3 Bucket upload all only zip files

I'm trying to upload all my zip files in folder to my s3 bucket using this command aws s3 cp recursive s3://<bucket-name>/%date:~4,2%-%date:~7,2%-%date:~10,4% -- recursive --include="*.zip" --...
1
vote
1answer
31 views

Incremental updates of data in an S3 data lake

I'm new to AWS and coming from a Data Warehousing ETL background. We are currently moving to cloud using AWS services Data Lake and trying to load data into Amazon s3 landing layer (Bucket) from our ...
-1
votes
0answers
9 views

Using a queue for bulk inserts

I'm using S3 to save log messages. But since appending data isn't a thing in S3 I have to fetch the original log again and append my message and then push it. I think a good way to make this more ...
1
vote
2answers
27 views

Read S3 bucket files from Intellij using scala/spark

How to read Amazon s3 bucket files from Intellij ide installed on local machine using scala/spark?
0
votes
1answer
18 views

Gitlab: How to deal with caches larger than 5GB with an autoscaled runner with S3

I configured my yocto project to use an auto-scaled gitlab-runner to run on AWS and now I noticed as the project has grown, that the cache fails to upload every time. Uploading cache.zip to https://...
-1
votes
1answer
15 views

Issue with Uploading Files from Ruby On Rails to S3 Bucket

I'm currently working on a proof-of-concept application that accepts a user-uploaded file and uploads it to an S3 bucket. However, whenever I try to generate a Upload model to represent the upload, I ...
0
votes
0answers
14 views

django cms media files dont show up in production environment

I'm using Django cms. Media files are not displayed after I deploy the application. For instance when I try to upload an image using carousel slides, the request times out. Image uploads are not ...
1
vote
1answer
24 views

S3 enable versioning affect on version-unaware code

I'd like to turn on versioning so I as a human can browse the console and see the history of files (my code won't be interacting with versioning). but I'd like make sure that my current system doesn't ...
2
votes
0answers
20 views

S3 upload from browser with presigned URL and SSE-C - 307 and 403s

Here's my scenario: using a server to generate presigned URLs that are sent down to the browser on-demand. The browser uploads files to these URLs and then sends the results of the uploads to the ...
0
votes
0answers
15 views

Deleted S3 File Still Appears

I got a problem to parse a file coming from s3 on Python. I found the file then I tried to delete this one. When I tried to list all my files into this folder I got the parse problem again. I cameback ...
-2
votes
0answers
15 views

How the VOD platform architecture should look?

It's like above. I am not sure how my service should delivering content to users. I was reading a lot but i had found there is something like AWS products (AWS Cloudfront, AWS S3) that can helps me ...
0
votes
0answers
8 views

TravisCI deploy to multiple S3 buckets failing with 'Aws::S3::Errors::NotFound'

This error recently appeared, I have gone back to previous version of travis.yml and I get the same error: Deploying application uploading "favicon.ico" with {:content_type=>"image/x-icon", :...
0
votes
1answer
16 views

S3 uploading and serving image with pre signed URL

I am trying to upload an image to my S3 bucket through a pre-signed url. Everything works well except that when I hit the public URL for that image, the browser downloads it instead of showing it. ...
0
votes
0answers
7 views

How to invoke AWS CLI command from CodePipeline?

I want to copy artifacts from S3 bucket in Account 1 to S3 bucket in Account 2. Though I was able to setup replication but I want to know whether there is a way to invoke AWS CLI command from within a ...
4
votes
1answer
40 views

Password protect s3 bucket with lambda function in aws

I added website authentication for s3 bucket using lambda function and then connect the lambda function with the CloudFront by using behavior settings in distribution settings and it worked fine and ...
0
votes
0answers
20 views

S3 Bucket policy - restrict access to objects

My setup: images stored on a S3 bucket bucket linked to a CloudFront CDN using custom CNAME (i.e. cdn.mydomain.com) My intentions: consider an object whose original location is mybucket.s3.amazonaws....
0
votes
0answers
5 views

How to add custom headers to AWS S3 .NET SDK GetObjectRequest?

How can I add custom headers to the GetObjectRequest using the .NET SDK? If the SDK does not allow this functionality, why does the .NET SDK not implemented this functionality? Is it the AWS S3 ...
0
votes
0answers
21 views

What is the rate limit on S3 list objects calls?

The documentation doesn't seem to be totally clear re. api limits on s3 operations. I see this doc But it doesn't specifically mention list-objects. I think its saying that GET operations on ...
0
votes
0answers
18 views

AWS MediaLive MediaPackage - How to store realtime streaming videos to S3?

I'm using AWS MediaLive to push video content in real-time from mobile devices. Then I'm using AWS MediaPackage to digest the input, i.e. the output from AWS MediaLive. With AWS MediaLive I create ...
0
votes
0answers
12 views

Buffer a gz file from s3 using lambda nodejs

i have a gz file in s3 and need to fetch the content of the file. The below code doesn't print the actual content of the file. const params = { Bucket: mybucket, Key: mykey.gz, ...
0
votes
0answers
15 views

Ionic 3 Resolve URI file for Upload (iOS)

From Ionic 3, I'm using the Cordova FileChooser plugin (for Android) which returns a URI resembling "content://com.android.providers.downloads.documents/document/6784". I then upload the URI as such ...
0
votes
1answer
21 views

Can I set the “Endpoint Override URL” to a presigned POST URL in NiFi's PutS3Object processor?

I recently learned that it's possible to generate a presigned upload URL for a client to upload to S3. It is possible to set this URL as the value of "Endpoint Override URL" in the PutS3Object ...
0
votes
0answers
14 views

Allow only a few servers access to resources on Amazon S3 Bucket via CORS?

I need help implementing and understanding how to write a CORS rule for an Amazon S3 bucket. I have a few dedicated servers with a hosting company and I want any site on these servers to be able to ...