Comparison between AWS and GCP resources

GCP (Google Cloud Platform) & AWS (Amazon Web Services) Cloud services comparison

AWS	GCP
Availability Zone	Zone
Virtual Private Cloud (VPC)	Virtual Private Cloud (VPC)
Identity Access Management (IAM)	Identity Access Management (IAM)
AWS CloudShell	Cloud Shell
Market Place	Market Place
EC2	Compute Engine (VM)
S3	Cloud Storage
EFS	Filestore
EBS	Persistent Disk
ElastiCache / Cloud Front	Cloud CDN
Device Farm	Cloud Test Lab
Elastic Beanstalk	App Engine
RDS	Cloud SQL
Redshift	BigQuery
DynamoDB	Bigtable
Elastic Container Service	Kubernetes Engine
Lambda	Cloud Functions
API Gateway	Cloud Endpoints
Elastic Transcoder	Cloud Video
Fargate	Cloud Run/Anthos
Elastic Load Balancer	Cloud Load Balancing
Direct Connect	Cloud Interconnect/Cloud VPN
Route 53	Domains, Cloud DNS
EMR	Cloud DataProc
Kinesis/Data Pipeline	Cloud Dataflow
SNS/Kinesis Data Steams	Pub/Sub
Data Pipeline	Cloud Composer
Code Pipeline	Cloud Build
Auto Scaling	Autoscaler
CloudWatch	Operations Suite (Stackdriver)
Cloud Trail	Cloud Logging
AWS Snowball	Transfer Appliance
Tags	Labels
Reserved instances	VM with committed use discounts
Spot instances	Preemptible VM
Sagemaker	Cloud AutoML
AWS Cloud9	Cloud Code
AWS Glue Data Catalog	Cloud Data Catalog

Other Softwares/Platforms	GCP
Docker Hub	Container Registry
Kafka	Cloud Pub/sub)

Related Articles: 

awscli commands

Amazon Web Services Command Line Interface (AWSCLI)

Installing aws cli in Python:
    pip install awscli
    pip install awscli --upgrade --user

aws [options] <command> <subcommand> [parameters]

aws --version
aws-cli/1.15.4 Python/3.6.3 Darwin/16.7.0 botocore/1.10.4

aws help
aws ec2 help
aws autoscaling create-auto-scaling-group help

aws configure
$ aws configure --profile user2

output format
json
text
table
export AWS_DEFAULT_OUTPUT="text"

$ aws s3 ls
aws s3 ls s3://satya-sparks/NYC_Parking_Tickets/
aws s3 ls s3://satya-sparks --recursive --human-readable --summarize
aws s3 ls s3://mybucket --recursive
aws s3 cp mnist.csv s3://thirumani-bucket
aws s3 cp /tmp/dir/ s3://thirumani-bucket/ --recursive
aws s3 cp /tmp/my_dir/ s3://thirumani-bucket/ --recursive --exclude "*" --include "*.jpg"
sudo aws s3 cp $OUTDIR.tar.gz s3://sr-cache/perfResults/$DATE_DIR/$RUN_ID/
aws s3 cp -R <dir_name>
time aws s3 cp --recursive --quiet . s3://test_bucket/test_smallfiles/
aws s3 cp s3://mybucket/test.txt test_local.txt
aws s3 cp s3://srcbucket/ s3://destbucket/ --recursive --exclude "a*" --exclude "b*"
aws s3 cp s3://mybucket/test.txt s3://mybucket/test2.txt --acl public-read-write
aws s3 cp file.txt s3://mybucket/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers full=emailaddress=user@example.com
aws s3 mv test.txt s3://mybucket/test2.txt
aws s3 mv s3://mybucket/test.txt s3://mybucket/test2.txt
aws s3 mv s3://mybucket/ s3://mybucket2/ --recursive --exclude "mybucket/another/*"
aws s3 rm s3://mybucket/test2.txt
aws s3 rm s3://mybucket --recursive
aws s3 mb s3://mybucket # make bucket
aws s3 mb s3://mybucket --region us-west-1
aws s3 rb s3://mybucket # remove bucket
aws s3 rb s3://bucket-name --force
aws s3 sync . s3://mybucket
time aws s3 sync --quiet . s3://test-bucket/test_randfiles/
aws s3 sync s3://mybucket s3://mybucket2
aws s3 website s3://my-bucket/ --index-document index.html --error-document error.html
aws s3api head-object --bucket test-bucket --key test_bigfiles/bigfile

$ aws ec2 start-instances --instance-ids i-X13836c
aws ec2 stop-instances --instance-ids i-X13836c
aws ec2 terminate-instances --instance-ids i-X13836c
$ aws ec2 describe-instances
aws ec2 describe-instances --filter Name=instance-type,Values=t2.nano
aws ec2 describe-instances --filters Name=instance-state-name,Values=stopped  --region  eu-west-1  --output json  |  jq  -r  .Reservations[].Instances[] .StateReason.Message
aws ec2 describe-instances --filters "Name=tag:Type,Values=Build" \
  --query "Reservations[0].Instances[0].PublicDnsName" | sed 's/"\(.*\)"/http:\/\/\1\/manage/'

aws ec2 run-instances --image-id ami-785bae10 --count 1 --instance-type t2.micro --key-name newpair.pem --security-group new-sg
aws ec2 delete-vpc --vpc-id vpc-a01106c2
aws ec2 describe-key-pairs
aws ec2 describe-key-pairs --key-name MyKeyPair
aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output text > MyKeyPair.pem
aws ec2 create-key-pair --key-name newpair.pem
aws ec2 delete-key-pair --key-name MyKeyPair
aws ec2 describe-tags --output table
aws ec2 describe-spot-price-history help
aws ec2 create-security-group --group-name my-sg --description "My security group"
aws ec2 describe-vpcs --region us-west-2
aws ec2 describe-vpcs --region us-west-2 --filter "Name=tag:Name,Values=Web VPC"
aws ec2 describe-vpc-attribute --region us-west-2 --attribute enableDnsHostnames --vpc-id vpc-24be405c

aws emr add-steps --cluster-id j-XXXXXXXX --steps file://./step.json
aws emr add-steps --cluster-id j-XXXXXXXX --steps Type=IMPALA,Name='Impala program',ActionOnFailure=CONTINUE,Args=--impala-script,s3://myimpala/input,--console-output-path,s3://myimpala/output
aws emr list-steps --cluster-id j-3SD91U2E1L2QX
aws emr create-cluster --ami-version=3.3.0 --applications Name=Hue Name=Hive Name=Pig --use-default-roles --ec2-attributes KeyName=myKey --instance-groups \
  InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m3.xlarge InstanceGroupType=CORE,InstanceCount=2,InstanceType=m1.large

aws iam create-user --user-name Satya
aws iam create-group --group-name SysAdmin
aws iam add-user-to-group --group-name SysAdmin --user-name Satya
aws iam list-users
aws iam list-users --output table
aws iam list-users --query 'Users[*].[UserName,CreateDate]'
aws iam list-users --query 'Users[*].{Name:UserName,CreateDate:CreateDate}'
aws iam list-groups-for-user --user-name Satya
aws iam list-groups
aws iam get-user-policy --user-name myuser --policy-name mypolicy
aws iam put-group-policy --group-name SysAdmin --policy-name admin-policy --policy-document file://sytem_admin_policy_doc.json
aws iam list-group-policies --group-name SysAdmin
aws iam delete-group-policy --group-name SysAdmin --policy-name admin-policy
aws iam remove-user-from-group --user-name myuser --group-name mygroup
aws iam delete-group --group-name SysAdmin

aws rekognition detect-labels --image "S3Object={Bucket=photo-collection,Name=photo.jpg}" --region us-west-2

aws dynamodb list-tables
aws dynamodb scan --table-name clients_table
aws dynamodb delete-table --table-name clients_table

aws rds create-db-instance \
    --engine MySQL \
    --db-instance-identifier mydbinstance \
    --allocated-storage 20 \
    --db-instance-class db.m1.small \
    --db-security-groups mydbsecuritygroup \
    --db-subnet-group mydbsubnetgroup \
    --master-username masterawsuser \
    --master-user-password masteruserpassword \
    --backup-retention-period 3
aws rds create-db-instance ^
    --engine oracle-se2 ^
    --db-instance-identifier oradbinstance ^
    --allocated-storage 40 ^
    --db-instance-class db.t2.small ^
    --db-security-groups mydbsecuritygroup ^
    --db-subnet-group mydbsubnetgroup ^
    --master-username masterawsuser ^
    --master-user-password masteruserpassword ^
    --backup-retention-period 4
aws rds describe-db-instances
aws rds modify-db-instance \
    --db-instance-identifier mydbinstance \
    --backup-retention-period 7 \
    --no-auto-minor-version-upgrade \
    --no-apply-immediately
aws rds modify-db-instance ^
    --db-instance-identifier mydbinstance ^
    --backup-retention-period 7 ^
    --auto-minor-version-upgrade ^
    --apply-immediately
aws rds download-db-log-file-portion --db-instance-identifier mydbinstance --log-file-name trace/sqlnet-parameters --output text
aws rds modify-db-instance \
    --db-instance-identifier <oradbinstance> \
    --engine-version <12.1.0.2.v10> \
    --option-group-name <default:oracle-ee-12-1> \
    --db-parameter-group-name <default.oracle-ee-12.1> \
    --allow-major-version-upgrade \
    --no-apply-immediately
aws rds modify-db-snapshot --db-snapshot-identifier <mydbsnapshot> --engine-version <11.2.0.4.v12>  --option-group-name <default:oracle-se1-11-2>

Related Articles: AWS Certified Cloud Practitioner Practice Exam

Difference between NACL and Security Groups in AWS

What are the differences between Security Groups and NACL in Amazon Web Services ?

Security Groups (SG)
Security groups, act as a firewall for associated Amazon EC2 instances, controlling both inbound and outbound traffic at the instance level.

Security groups can also be used by AWS services such as Amazon RDS, Amazon Redshift, Amazon EMR, and Amazon ElastiCache.

By default, Security Groups allow only outbound traffic and block all incoming traffic. To enable inbound traffic, we have to specify the protocol, port and source.

VPC is like a house and subnets are like rooms, security group is like (fire)wall.

Network Access Controls Lists (NACL)
Network access controls lists (ACLs) act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level.

NACL is at subnet level.

In a VPC, the default NACL is set to allow all inbound and outbound traffic. If you deploy a custom NACL, then all inbound and outbound traffic is blocked.

Security Groups control traffic flow at the instance level whereas NACLs control traffic at the subnet level.

Security Groups can thus be used to enable traffic flow between different types of instances within a subnet itself which NACLs cannot control.

NACL and Security Groups work in collaboration with each other to offer a more complete security setup.

Differences between Security Groups and Network ACL

Security Groups	NACL
Security groups are stateful (changes applied to an incoming rule will be automatically applied to the outgoing rule)	NACLs are stateless (changes applied to an incoming rule will not be applied to the outgoing rule)
By default, Security Groups allow only outbound traffic and block all incoming traffic	By default, NACL is set to allow all inbound and outbound traffic
Security groups are tied to an instance	Network ACLs are tied to the subnet
Firewall of EC2 Instances	Firewall of Subnet
Security group support ALLOW rules only (by default all rules are denied)	Network ACL support ALLOW and DENY rules
All rules in a security group are applied	NACLs apply rules in number/priority order, from top to bottom
Applies to an instance only if specified the security group	Automatically applies to all instances in the subnets it's associated with
Can not block specific IP address	Can block specific IP address

Related AWS Articles:  AWS Certified Cloud Practitioner Practice Exam 2

AWS Cloud Practitioner Exam 1

AWS Certified Cloud Practitioner Practice Exam 1

Related Articles: AWS Certified Cloud Practitioner Practice Exam 2

AWS Certified Cloud Practitioner Practice Exam 2

AWS Certified Cloud Practitioner Practice Exam 2

Related Articles: AWS Certified Cloud Practitioner Practice Exam 1

Terraform commands

Terraform Command Line Interface (Terraform CLI)

$ terraform
Usage: terraform [--version] [--help] <command> [args]

Common commands:
    apply              Builds or changes infrastructure
    console            Interactive console for Terraform interpolations
    destroy            Destroy Terraform-managed infrastructure
    fmt                Rewrites config files to canonical format
    get                Download and install modules for the configuration
    graph              Create a visual graph of Terraform resources
    import             Import existing infrastructure into Terraform
    init               Initialize a new or existing Terraform configuration
    output             Read an output from a state file
    plan               Generate and show an execution plan
    providers          Prints a tree of the providers used in the configuration
    push               Upload this Terraform module to Terraform Enterprise to run
    refresh            Update local state file against real resources
    show               Inspect Terraform state or plan
    taint              Manually mark a resource for recreation
    untaint            Manually unmark a resource as tainted
    validate           Validates the Terraform files
    version            Prints the Terraform version
    workspace          Workspace management

All other commands:
    debug              Debug output management (experimental)
    force-unlock       Manually unlock the terraform state
    state              Advanced state management

terraform -version
terraform taint -help

terraform init # will install provider plugins
terraform init -upgrade commands_exec/
terraform get

terraform plan
terraform plan -out=demo.tfplan
terraform plan --destroy -out destroy-plan
terraform apply
terraform apply demo.tfplan
terraform apply -var 'zones=["us-east-1b", "us-east-1d"]'
terraform apply -var-file=sample.tfvars
terraform destroy
terraform destroy -target=aws_security_group.elb_sg

terraform workspace list
terraform workspace new dev
terraform workspace select
$ terraform import aws_instance.example i-abcd1234 # import existing infrastructure
$ terraform import aws_key_pair.deployer deployer-key
terraform import aws_vpc.web_vpc vpc-25b7495d

terraform refresh
terraform show # to show the attributes in the state file
terraform state list
terraform state show aws_vpc.web_vpc
terraform graph

terraform output
terraform output ips_var
terraform output -module=network
site_address=$(terraform output site_address)
web_server_ip=$(terraform output -module=vm | cut -d " " -f 3)

terraform console
echo "2 + 8" | terraform console
terraform providers
terraform fmt
terraform fmt -check
terraform validate # performs syntax checks

Related Articles:  awscli commands