Aws Cli Cheat Sheet

Amazon S3

• Aws Cli Cheat Sheet
• Aws Cli Cheat Sheet Pdf
• Aws Cheat Sheet Pdf

DynamoDB CLI Operation Examples. This cheat sheet will help you perform basic query operations, table manipulations and item updates with DynamoDB and AWS CLI. If you're looking for similar cheat sheet but for Python, you can find it here, and for Node.js - here. Table of Contents. Setting up; Important Environment Variables; Create Table. CLI AWS CLI is a unified tool to manage AWS services & control multiple services from the command line & automate them through. AWS Services Cheat Sheet.

What it is S3

Amazon S3 (Simple Storage Service) is a Amazon’s service for storing files. It is simple in a sense that one store data using the follwing:

• bucket: place to store. Its name is unique for all S3 users, which means that there cannot exist two buckets with the same name even if they are private for to different users.
• key: a unique (for a bucket) name that link to the sotred object. It is common to use path like syntax to group objects.
• object: any file (text or binary). It can be partitioned.

Sign up

First go tohttps://s3.console.aws.amazon.com/s3

and sign up for S3. You can also try to create a bucket, upload files etc. Here we will explain how to use it porogramatically.

Data

But first let’s get data we are going to use here. We take the dataset train.csv from https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge. We locally store in data directory.

Sampling data

We also sample this dataset in order to have one more example (and faster execution).

Installing AWS Command Line Interface and boto

In order to install boto (Python interface to Amazon Web Service) and AWS Command Line Interface (CLI) type:

Then in your home directory create file ~/.aws/credentials with the following:

If you add these configuration as [default], you won’t need to add --profile myaws in CLI commands in Section CLI Basic Commands.

Where to get credentials from

1. Go to https://console.aws.amazon.com/console/home and log in
2. Click on USER NAME (right top) and select My Security Credentials.
3. Click on + Access keys (access key ID and secret access key) and then on Create New Acess Key.4 Choose Show access key.

CLI Basic Commands

List buckets

List all buckets

Create buckers

Warning The bucket namespace is shared by all users of the system so you need to change the name.

Upload and download files

Upload

The last 4 commands can be done in shell calling:

Download

List files in path

Remove file(s)

Delete bucket

For deleting a bucket use

in order to delete non empty backet use --force option.

In order to empty a backet use

What Boto is

Boto is a Python package that provides interfaces to Amazon Web Services. Here we are focused on its application to S3.

Creating S3 Resource

We start using boto3 by creating S3 resorce object.

From evironment variables

Mozilla firefox download. If your credentials are stored as evirionment variables AWS_SECRET_KEY_ID and AWS_SECRET_ACCESS_KEY then you can do the following:

List buckets

Create a bucket

Warning As before, bucket’s namespace is shared, so the following command may not poroduce a bucket if a bucket with the name exists.

And you have the followng Access Control List (ACL) options while creating it:

• `‘private’,
• ‘public-read’,
• ‘public-read-write’,
• ‘authenticated-read’`.

Deleting

List keys in the bucket

The object of class ObjectSummary has to properties Bucket (that returns Bucket object), bucket_name and key that return strings.

Filter keys and sort them

Download file

Aws Cli Cheat Sheet

Transform to pandas.DataFrame

One way to do this is to download the file and open it with pandas.read_csv method. If we do not want to do this we have to read it a buffer and open it from there. In order to do this we need to use low level interaction.

Another way, using higher level download_fileobj requires transform bytes streaiming into text streaming.

Upload file

With buffer

Delete

S3 client: low level access

Access through http(s)

Change Access Control

Uri

There are two formats of uri:

Example

Streaming with smart_open

Install

Passing session

It is smart enough to recognize from where it has to read

Aws Cli Cheat Sheet Pdf

Writing

Links:

• https://github.com/boto/boto3
• https://boto3.amazonaws.com/v1/documentation/api/latest/index.html
• https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge

Last update:2018-11-03

The AWS CLI is a handy and extremely powerful tool for managing resources on AWS from a local shell. In this post we will walk through the process of setting up the AWS CLI on a Linux Machine ( I use the Linux Subsystem on Windows which gives me an Ubuntu bash). Then we begin using the AWS CLI to perform administrative tasks on the environment.

What you need

• AWS IAM User with the correct permissions to administer EC2
• The Access keys of said AWS IAM User with the correct permissions
• Linux
• AWS CLI

Lets get started…

Setting Up The AWS CLI

Setup an IAM User with Permission to Administer EC2

Aws Cheat Sheet Pdf

I am always in the habit of thinking about what group a user should be in before I create the user account. In this case my new user is going to be a System Administrator with some elevated permissions. First we create a group called EC2SysAdmins and give it full access permission to EC2. Then we pull the user into that group.

1. Log into the AWS Console
2. Services > IAM > Groups
3. Create New Group
   1. Group Name : EC2SysAdmins > Next Step
   2. Filter : AmazonEC2FullAccess > Check – AmazonEC2FullAccess > Next Step
   3. Review > Create Group
   4. You are returned to the Groups Screen
4. Create a User
   1. Users > Add User
      1. User Name : EC2Admin
      2. Access Type : Check – Programmatic Access > Next: Permissions
   2. Add user to group
      1. Check – EC2SysAdmin > Next: Review
   3. Review > Create user
   4. You then see a Success screen listing the Access key ID and the Secret Access Key. You will need these to configure the AWS CLI later on. To make sure you have these credentials later (just in case you can not memorize long alphanumeric strings) download them.
      1. Click download CSV > Optionally, go to where the the file was downloaded and rename the file from credentials.csv to EC2Admin_AccessKey.csv or something a bit more fitting as you will refer to it shortly.

Install the AWS CLI Tools on Linux

In your bash shell (get bash set up on Windows 10)

sudo apt-get install awscli

Configure the AWS CLI Tools on Linux

In your bash shell

aws configure

Here you need to specify:

1. Your access Key ID (refer the the csv file you downloaded)
2. Your Secret Access Key (refer the the csv file you downloaded)
3. Your preferred default region code (for example, us-east-1). Choose a region that is closest to you for now.
4. Default output format. Choose json

There may be cases where you need to reconfigure the CLI tools, such as using different account credentials. You can always reconfigure any of these items later by

just re-running the aws configure command again.

AWS CLI Command Cheat Sheet – 101

Here is a list of commands that I’ve found helpful. I’ve made an effort to describe their uses. I hope they are helpful to you too.

List Regions and Availability Zones

Let’s first see what regions are available

You’ll get something like this.

With a list of regions in front of us, let’s take a look at which availability zones are present in a particular region. In this case lets check us-east-1.

We get a text list of availability zones. TAKE NOTE: Since we already configured the region in the AWS CLI, we could just omit the --region us-east-1 parameter in the command above which would return the same list of availability zones. Include the region parameter if you have not configured a preferred default region in the AWS CLI.

Get the Name, Instance ID, IP Addresses of EC2 Instances

Sometimes you just want a human readable list of instances that you can then take action on, like quickly stopping or starting an instance based on the InstanceID.

Shimoneta characters. This command is handy for a tabular list of information that you can then take action on, like quickly stopping or starting an instance based on the InstanceID.

Stop and Start Instances

Starting an instance

Stopping an instance

Launch an Instance in an Availability Zone

Spot Price History