AWS Other Application Services Exam Preparation
SQS – Simple Queue Service (must know)
- Amazon SQS ensures delivery of each message at least once and supports multiple readers and writers interacting with the same queue.
- SQS –
- Does not guarantee the order of delivery.
- Duplicates possible.
- SQS FIFO –
- Delivery guaranteed based on FIFO.
- No Duplicates.
- SQS –
- It is a poll based system.
- SQS by default stores message for 4 days and for max of 14 days.
- Default timeout is 30 sec, longest configurable is 12 hrs.
- SQS facilitates horizontal scaling.
- Each message can have up to 10 attributes metadata(date,timestamp…).
- Configure dead letter queues to handle messages that can’t be processed,primary benefit of using a dead letter queue is the ability to sideline and isolate the unsuccessfully processed messages.
- Delay queues allow you to postpone the delivery of new messages.
- With long polling, you send a WaitTimeSeconds argument to ReceiveMessage of up to 20 seconds.
- SQS can directly subscribe to SNS topic.
- Whenever you see decoupling and/or scaling microservices, SQS can be recommended.
- Encryption – SSE can be used to protect the content.
- An SQS request can contain up to TEN (10) individual messages, as long as he total size of the request does not exceed 256KB.
- whenever the application lagging in performance and cannot process tasks within stipulated time, consider using SQS to pass those message and EC2/other service can pull from the queue.
SWF – Simple Workflow Service
- An SWF workflow ensure that actions are executed only once as against SQS.
- Components :
- Actors can be workflow initiator / starter, decider, or activity workers. Like booking a ticket with Web portal.
- Due to above action, what are the tasks need to be performed.
- Activity tasks, Lambda tasks, Decision tasks.
- An instance/service outside AWS can perform worker task.
SNS – Simple Notification Service
- Push based.
- Amazon Simple Notification Service follows the Publish-Subscribe paradigm.
- Create Topic (like ELB launched) -> Subscribe to Topic – and protocol (email, lambda) -> go to ELB -> Create metrics -> Set alram = SNS topic name.
- All SNS messages are stored redundantly on multiple servers and in multiple data centers, which means that no single computer or network failure renders Amazon SNS inaccessible.
- A fanout scenario is when an Amazon SNS message is sent to a topic and then replicated and pushed to multiple Amazon SQS.
- Amazon API Gateway is a fully managed service that makes it easy for developers to publish, maintain, monitor, and secure APIs at any scale.
- API Gateway logs API calls, latency, and error rates to Amazon CloudWatch.
- To control access use IAM Permissions , like to develop API developer, to access API Caller permission.
- Private API Gateway – when the API will not be required to be accessed over internet (uses PrivateLink).
- Regional API Gateway – Over the internet,but closer your Region (uses direct access over the Internet .
- Edge Optimized API Gateway – Over the internet and users distributed worldwide(uses CloudFront).
AWS Lambda (must know)
- A serverless compute service. Allows you to run your code with you provisioning an instance, the resource provision is done automatically according to your running code.
- To DecoupleApplication services.
- Lets you run code without provisioning or managing servers.
- Lambda automatically scales your application by running code in response to each trigger.
- AWS Lambda to execute code in response to triggers such as changes in data, shifts in system state, or actions by users.
- Lambda can be directly triggered by AWS services such as S3, DynamoDB, Kinesis, SNS, and CloudWatch,Step functions.
- Click here for Lambda Example.
Kinesis (must know)
- Collect, process, and analyze real-time streaming data.
- Amazon Kinesis Firehose(preferably for S3)
- Configured to save a live stream to Amazon S3, Amazon Kinesis Firehose sends the data directly to Amazon S3.
- Amazon Kinesis Data Streams
- A service enabling you to build custom applications for more complex analysis of streaming data in real time.
- As analysis is realtime, it is for processing lightweight application.
- A Kinesis data stream is a set of shards. Each shard has a sequence of data records. Each data record has a sequence number that is assigned by Kinesis Data Streams.
- A data record is the unit of data stored in a Kinesis data stream.
- Data records are composed of a sequence number, a partition key, and a data blob, which is an immutable sequence of bytes.
- Limitless data streams by distributing incoming data across a number of shards.
- Each shard can support up to 5 transactions per second for reads, up to a maximum total data read rate of 2 MB per second and up to 1,000 records per second for writes.
- Use case – log data, application logs, social media, market data feeds, and web clickstream data.
- Amazon Kinesis Data Analytics
- A service enabling you to easily analyze streaming data real time with standard SQL.
- Kinesis Video Stream
- For securely streaming Video for ML/AI processing.
- Fire hose only writes directly to S3 (or other aws storage service).
- Data streams allow you to write your own consumer applications to process the data.
- Amazon Kinesis Firehose(preferably for S3)
AWS Directory Service
- is designed to reduce identity management task, same as MS AD with AWS services connected.
KMS – Key Management Service
- Generate, store, enable/disable, and delete symmetric keys(same key 4 encrypt & decrypt) – encryption keys.
- Customer Master Key (CMK) to encrypt and decrypt data upto 4kb and this never goes out unencrypted.
- Data keys to encrypt large data objects.
- Envelope Encryption to protect data.
- Helps you meet corporate, contractual, and regulatory compliance requirements for data security.
- HSM is a hardware appliance that provides secure key storage and cryptographic operations.
- AWS CloudHSM allows to generate our own encryption keys and we can manage it fully(ie) we have complete control.
EMR – Elastic MapReduce:
- EMR provides you with a fully managed, on-demand Hadoop framework.
- Define -> Instance type, number of nodes, hadoop version and application tools like hive, pig,spark, presto.
- HDFS is the standard file system can use EC2 instance storage or Amazon EBS for HDFS.
- EMRFS is an implementation of HDFS that allows clusters to store data on Amazon S3.
- EMR starts your instances in two Amazon Elastic Compute Cloud (Amazon EC2) security groups, one for the master and another for the slaves.
AWS CI/CD (Codecommit, codebuild , codedeploy & Datapipeline.)
- AWS Code repository
- To build and validate the above committed code.
- is a service that automates code deployments to any instance, including Amazon EC2 instances and instances running on-premises.
- Use Case:
- CodeDeploy helps in canary deployment, blue & Green deployment –> new version, failover to older version.
AWS Data Pipeline:
- AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services.
- Use case – batch processing / jobs.
- Tasks can be scheduled and run.
AWS OpsWorks (to Manage Stacks) :
- OpsWorks is a configuration management service that helps you configure and operate applications using Chef / Puppet.
- AWS OpsWorks provides a simple and flexible way to create and manage stacks and applications.
- Stack is the core AWS OpsWorks component. It is basically a container for AWS resources—Amazon EC2 instances, Amazon RDS database instances, and so on.
- A layer represents a set of resources that serve a particular purpose, such as load balancing, web applications, or hosting a database server.
- AWS OpsWorks sends all of your resource metrics to Amazon CloudWatch.
Cloud Formation (must know):
- Scripting the infrastructure creation.
- You can author AWS CloudFormation templates in JSON or YAML formats.
- The Resources section is the only required section. It specifies the stack resources and their properties, such as an Amazon Elastic Compute Cloud instance or an Amazon Simple Storage Service bucket.
- When a stack is deleted all the underlying services also gets deleted.Deletion policy to retain some resources out of the stack can be used incase.
AWS Trusted Advisor (must know):
- AWS Trusted Advisor inspects your AWS environment and makes recommendations when opportunities exist to save money, improve system availability and performance, or help close security gaps.
- Use case – cost optimization, security, fault tolerance, and performance improvement.
- Red: Action recommended
- Yellow: Investigation recommended
- Green: No problem detected
- Amazon Inspector is an automated security assessment service that helps improve the security and compliance of applications deployed on AWS or deviations from best practices.
AWS Config (any change to AWS resource):
- AWS Config, you can discover existing and deleted AWS resources,determine your overall compliance against rules.
- Capabilities enable compliance auditing, security analysis, resource change tracking, and troubleshooting.
- AWS Config integrates with AWS CloudTrail, a service that records AWS API calls for an account and delivers API usage log files to an Amazon S3 bucket.
- AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. automate the evaluation of recorded configurations against desired configurations.
- Config continuously monitors and records your AWS resource configurations.
- is a managed threat detection service that continuously monitors for malicious or unauthorized behavior to help you protect your AWS accounts and workloads.
Amazon Athena (S3-Sql):
- Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL.
- can be used to analyze logs, such as CloudTrail logs, to help you identify trends and further isolate activity by attribute, such as source IP address or user.(serverless interactive query service).
AWS Shield (for DDoS):
- is a managed distributed denial of service (DDoS) protection service that safeguards web applications running on AWS.
- is a web application firewall that helps protect your web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources.
- Monitors the HTTP and HTTPS requests that are forwarded to Amazon CloudFront or an Application Load Balancer.
- Use case:
- Allow all requests except the ones that you specify.
- Block all requests except the ones that you specify
- Count the requests that match the properties that you specify.
AWS Step Functions:
- to coordinate a sequence of steps to automate an incident response process.
AWS X-Ray (must know):
- traces user requests as they travel through your entire application, enabling analysis and debugging of distributed applications.
- Use Case: X-Ray can be used for debugging/analyze MicroService or components
- is a business analytics service that makes it easy to build visualizations, perform ad-hoc analysis, and quickly get insights from your data.
- Amazon Elasticsearch Service is a managed service that makes it easy to deploy, operate, and scale Elasticsearch clusters in the AWS Cloud.
- Elasticsearch is an open-source, RESTful, distributed search and analytics engine.
- When coupled with Kibana, a visualization tool, Elasticsearch can be used to provide near-real time analytics using large volumes of log data.
- Amazon ES makes it easy to deploy, secure, operate, and scale Elasticsearch for log analytics, and application monitoring.
- is a fully-managed, pay-as-you-go, extract, transform, and load (ETL) service that automates the time-consuming steps of data preparation for analytics.
- Glue can automatically discover both structured and semi-structured data stored in your data lake on Amazon S3, data warehouse in Amazon Redshift, and various databases running on AWS.
Amazon Elastic Network Adapters (ENA):
- provide further optimization by delivering 20 Gbps of network capacity for your instances within a single placement group.
AWS FPS & AWS DevPay:
- both leverage the Amazon Payments infrastructure to process payments from customers.
Amazon Cognito (must know):
- lets you easily add user sign-up and authentication to your mobile and web apps. (through an external identity provider like google, Facebook, Twitter, Amazon,FB).
- provides temporary security credentials.
AWS Direct Connect:
- to have dedicated network between on-prem and AWS, without internet.
- makes it simple and cost-effective to set up, manage, and scale a search solution for your website or application.
- offers policy-based management for multiple AWS accounts.
- you can create groups of accounts, automate account creation, apply and manage policies for those groups.
- you can create Service Control Policies (SCPs) that centrally control.
- Lightsail provides developers compute, storage, and networking capacity and capabilities to deploy and manage websites, web applications, and databases in the cloud.
- It creates instance based on our choice including the instance image(Linux / Windows), blue print(for wordpress development, node.js,Mean) and instance price plan (monthly plan ranging from $3.5 – depending upon compute, storage capacities and tranfer).
- Use Case:Running web site/applications.