Questions tagged [amazon-sagemaker]

Amazon SageMaker is a fully-managed AWS service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale.

0
votes
0answers
21 views

Load S3 bucket Data in AWS SageMaker

Related to the question below but I'm still struggling: Load S3 Data into AWS SageMaker Notebook I'm trying to load a parquet file from a local S3 bucket (it contains "sagemaker" in the name. I'm ...
0
votes
1answer
19 views

Can I make Amazon SageMaker deliver a recommendation based on historic data instead of a probability score?

We have a huge set of data in CSV format, containing a few numeric elements, like this: Year,BinaryDigit,NumberToPredict,JustANumber, ...other stuff 1954,1,762,16, ...other stuff 1965,0,142,16, ......
0
votes
0answers
9 views

Blankspace and colon not found in firstline

I have a jupyter notebook in SageMaker in which I want to run the XGBoost algorithm. The data has to match 3 criteria: -No header row -Outcome variable in the first column, features in the rest of ...
0
votes
0answers
54 views

Difference in usecases for AWS Sagemaker vs Databricks?

I was looking at Databricks because it integrates with AWS services like Kinesis, but it looks to me like SageMaker is a direct competitor to Databricks? We are heavily using AWS, is there any reason ...
1
vote
0answers
38 views

Inconsistent “'float' object is not iterable” error in sklearn

I know that there are a lot of questions with the same problem but none of them solves my problem. I am using a Jupyter Notebook in Amazon Sagemaker and I want to use the hashing trick for some ...
0
votes
1answer
27 views

SageMaker Train job is not creating /opt/ml/input/data/training directory

I am trying to create a custom algorithm by following the instruction given in this tutorial. When I am running the train job it is failing with error No such file or directory: '/opt/ml/input/data/...
0
votes
1answer
15 views

Install graphiz on AWS Sagemaker

I'm on a Jupyter notebook using Python3 and trying to plot a tree with code like this: import xgboost as xgb from xgboost import plot_tree plot_tree(model, num_trees=4) On the last line I get: ~/...
0
votes
1answer
14 views

Install graphiz on AWS Sagemaker

I'm on a Jupyter notebook using Python3 and trying to plot a tree with code like this: import xgboost as xgb from xgboost import plot_tree plot_tree(model, num_trees=4) On the last line I get: ...
0
votes
0answers
27 views

How can I install the R kernel on an AWS SageMaker notebook instance without internet access?

I am trying to install the R kernel on an AWS SageMaker notebook instance that has no internet access. AWS provides documentation on how to install the R kernel on a notebook instance with internet ...
-2
votes
1answer
19 views

Is multi-node Sagemaker training batched per-node or shared?

I am using Tensorflow, and am noticing that individual steps are slower with multiple nodes than with one, so I am a bit confused as to what constitutes a step on multiple training nodes on Sagemaker. ...
0
votes
1answer
22 views

Discrepancy between AWS Glue and its Dev Endpoint

My understanding is Dev Endpoints in AWS Glue can be used to develop code iteratively and then deploy it to a Glue job. I find this specially useful when developing Spark jobs because every time you ...
0
votes
1answer
38 views

How do I start an AWS Sagemaker training job with GPU access in my docker container?

I have some python code that trains a Neural Network using tensorflow. I've created a docker image based on a tensorflow/tensorflow:latest-gpu-py3 image that runs my python script. When I start an ...
0
votes
1answer
35 views

MissingRequiredParameter: Missing required key 'FunctionName' in params

I'm working on a supervised machine learning problem, and I am setting up a custom labeling task to send out to Amazon Mechanical Turk for human annotation. I have uploaded the data to AWS S3 in the ...
1
vote
0answers
28 views

Issues with prediction time not proportional w.r.t. number of trees in RF

My estimation strategy consists on using a set of Random Forest models, each one concerns some subset of data (ex : RF_A if feature == A). This has been said seek of completeness as I dont think this ...
0
votes
2answers
45 views

How to deploy pre-trained model using sagemaker?

I have a pre-trained model that will translate text from English to Marathi. You can find it here... git clone https://github.com/shantanuo/Word-Level-Eng-Mar-NMT.git Clone and Run the notebook. I ...
0
votes
1answer
23 views

Training Job in Sagemaker gives error in locating file in S3 to docker image path

I am trying to use scikit_bring_your_own/container/decision_trees/train mode, running in AWS CLI, I had no issues. Trying to replicate in Creating Sagemaker Training Job , facing issue in loading data ...
-2
votes
0answers
23 views

Implement fast style transfer using AWS

I am trying to train a model as explained on this page... https://github.com/lengstrom/fast-style-transfer What is the best way to implement this in AWS environment? Is sagemaker a better choice ...
1
vote
1answer
16 views

Download an entire folder from AWS sagemaker to laptop

I have a folder with predicted masks on AWS Sagemaker. ( It has 4 folders inside it and lot of files inside those folders. ) I want to download the entire folder to my laptop. This might sound so ...
0
votes
1answer
21 views

Call Sagemaker Tensorflow Resnet50 Endpoint from node.js

The Question I want to know the equivalent in node.js to this Python code: from keras.preprocessing import image from PIL import Image from keras.applications.resnet50 import preprocess_input ...
0
votes
1answer
53 views

Sagemaker LDA topic model - how to access the params of the trained model? Also is there a simple way to capture coherence

I'm new to Sagemaker and am running some tests to measure the performance of NTM and LDA on AWS compared with LDA mallet and native Gensim LDA model. I'm wanting to inspect the trained models on ...
0
votes
2answers
29 views

Using cloudwatch api get_metric_data to get sagemaker endpoints invocation metrics

I am trying to use aws cloudwatch api get_metric_data to get sagemaker endpoint invocation metrics in python, and it returns me empty timestamps and Values, but there are some invocations between the ...
0
votes
1answer
73 views

With AWS SageMaker, is it possible to deploy a pre-trained model using the sagemaker SDK?

I'm trying to avoid migrating an existing model training process to SageMaker and avoid creating a custom Docker container to host our trained model. My hope was to inject our existing, trained model ...
0
votes
0answers
30 views

Problem connecting Livy to EMR via sagemaker

I have followed this tutorial: https://aws.amazon.com/fr/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/ in order to be able to run pyspark code on EMR via ...
0
votes
0answers
40 views

Preprocess input data before making predictions inside Amazon SageMaker

I have a Keras/tensorflow model that we have trained by ourselves which does image related prediction. I have followed this trained keras model tutorial to deploy the model in Sagemaker and can invoke ...
0
votes
0answers
21 views

SageMaker Neo PyTorch 1.0.0

I've updated the torch version in my SageMaker pytorch_36 kernel to torch version 1.0.0. I then tried running the example notebook pytorch_torchvision_neo.ipynb, also changing the framework_version to ...
0
votes
1answer
21 views

create aws sagemker endpoint with lambda function

I created an endpoint in aws sagemaker and it works well, I created a lambda function(python3.6) that takes files from S3, invoke the endpoint and then put the output in a file in S3. I wonder if I ...
1
vote
2answers
32 views

SageMaker delete Models and Endpoint configurations with python API

I've tried deleting/recreating endpoints with the same name, and wasted a lot of time before I realized that changes do not get applied unless you also delete the corresponding Model and Endpoint ...
0
votes
0answers
24 views

fastai torch JIT save and SageMaker Neo compilation

I'm attempting to get a fastai model optimized with SageMaker Neo compilation. It looks like I can save out my model in one of three ways. regular: learn.save which outputs e.g. resnet34.pth onnx: ...
2
votes
1answer
31 views

“Kernel appears to have died” error when I try to train my model. Is it too big? What could be the issue?

None of the other solutions to this question on here have worked for me. I am trying to train a model on a Jupiter notebook on amazon Sagemaker instance ml.c4.8xlarge with 15,000GB memory. However, ...
0
votes
2answers
47 views

How do I create a Sagemaker training job with my own Tensorflow code without having to build a container?

I'm trying to define a Sagemaker Training Job with an existing Python class. To my understanding, I could create my own container but would rather not deal with container management. When choosing "...
0
votes
1answer
53 views

How can i call sagemaker inference endpoint using API gateway

I am trying to call sagemaker inference endpoint from api gateway with AWS Integration.I don't want to use lamdba in between of API gateway and sagemaker runtime. I followed this doc to setup api ...
0
votes
1answer
30 views

Sagemaker endpoint(with VPC) url accessible from internet

I have created a model with VPC, Private subnet, and appropriate security group. The endpoint URL can, however, be reached through the internet though failing due to the lack of security token Things ...
0
votes
1answer
22 views

Converting Cifar10 dataset to RecordIO format for Amazon SageMaker

I have downloaded cifar-10 dataset and need to convert it to RecordIO format. If you unzip the downloaded cifar-10-python.tar.gz dataset, you will see below cifar-10-batches-py/ cifar-10-batches-py/...
0
votes
2answers
71 views

AWS SageMaker GroundTruth permissions issue (can't read manifest)

I'm trying to run a simple GroundTruth labeling job with a public workforce. I upload my images to S3, start creating the labeling job, generate the manifest using their tool automatically, and ...
0
votes
2answers
46 views

Is there a way to use sagemaker lifecycle configuration to run an EMR cluster on notebook start

I would like to start an EMR cluster every time a sagemaker notebook is started. However, I have find out that Lifecycle configuration scripts cannot run for longer than 5 minutes. Sadly my EMR ...
1
vote
1answer
34 views

Amazon SageMaker Unsupported content-type application/x-image

I have a tensorflow/keras based CNN model deployed in Sagemaker. Now to invoke the inference, I followed this tutorial Below code snippet def inferImage(endpoint_name): # Load the image bytes ...
0
votes
2answers
40 views

Invoking SageMaker endpoint with data payload in S3 from a python boto client

I have an image that I upload to s3 in mybucket. Suppose my s3 endpoint for this data is s3://mybucket/imgname Now I also have a model deployed in SageMaker at sagemaker-model-endpoint I looked into ...
0
votes
1answer
33 views

SageMaker Script Mode + Pipe Mode

I'm training in SageMaker using TensorFlow + Script Mode and currently using 'File' input mode for my data. Has anyone figured out how to stream data using 'Pipe' data format in conjunction with ...
0
votes
1answer
49 views

Tensorflow Serving in Amazon SageMaker

I am facing an issue with serving tensorflow models on AWS SageMaker. I have trained the model outside of the SageMaker environment, now I have a savedmodel.pb file and I need to deploy it on a ...
0
votes
1answer
61 views

Invoke aws sagemaker endpoint

I have some data in S3 and I want to create a lambda function to predict the output with my deployed aws sagemaker endpoint then I put the outputs in S3 again. Is it necessary in this case to create ...
0
votes
1answer
32 views

Access Denied read open data into Sagemaker

Cannot read AWS open data datasets into Sagemaker. Error is download failed: s3://fast-ai-imageclas/cifar100.tgz to ../../../tmp/fastai-images/cifar100.tgz An error occurred (AccessDenied) when ...
0
votes
2answers
24 views

Correct parameters for training AWS Sagemaker with multiple classes per image

I've found consistently that "multi_label" to "1" for image classification jobs, they crash with the following error: Algorithm Error: Internal Server Error [15:56:08] /opt/brazil-pkg-cache/packages/...
0
votes
1answer
61 views

call sagemaker endpoint using lambda function

I have some data in S3 and I want to create a lambda function to predict the output with my deployed aws sagemaker endpoint then I put the outputs in S3 again. Is it necessary in this case to create ...
0
votes
1answer
39 views

Amazon Sagemaker object detection expected number of batch error

I am getting the following error while training my model with more than 1 epoch [02/06/2019 13:37:08 WARNING 140231582721856] Expected number of batches: 15, did not match the number of batches ...
0
votes
0answers
39 views

Amazon S3 SelectObjectContent error - OverMaxParquetBlockSize

I'm trying to use the select functionality of S3 but for some files, I get OverMaxParquetBlockSize error. My select is pretty simple: r = s3.select_object_content( Bucket='prod-bucket', Key='...
0
votes
0answers
23 views

How to UTF-8 encode .pkl in Sagemaker?

I am using the fast.ai v1 libraries in Sagemaker and find that learn.export() errors with the note: First question -- Where in the Console would I look for more details? I checked the CloudWatch ...
2
votes
1answer
80 views

aws sagemaker for detecting text in an image

I am aware that it is better to use aws Rekognition for this. However, it does not seem to work well when I tried it out with the images I have (which are sort of like small containers with labels on ...
0
votes
1answer
57 views

SageMaker deploying to EIA from TF Script Mode Python3

I've fitted a Tensorflow Estimator in SageMaker using Script Mode with framework_version='1.12.0' and python_version='py3', using a GPU instance. Calling deploy directly on this estimator works if I ...
0
votes
1answer
38 views

AWS Sagemaker - ClientError: An error occurred (ValidationException) when calling the CreateTransformJob operation

I am trying to run a batch transform job on AWS sagemaker. However, I keep getting the following error when I create a transformer and run the transform method: ClientError: An error occurred (...
1
vote
0answers
46 views

SageMaker GetBucketLocation operation: Access Denied

I have a SM notebook in Oregon region and I've ran estimator.fit with data in a S3 bucket of the same region several times previously. Today on attempting to rerun this script I'm getting the ...

http://mssss.yulina-kosm.ru