AWS CLOUD

Kinesis:

1. Amazon Kinesis Data Streams enables you to build custom applications that process or analyze streaming data for specialized needs. You can continuously add various types of data such as clickstreams, application fogs, and social media to an Amazon Kinesis data stream from hundreds of thousands of sources.
2. Data will be available for your Amazon Kinesis Applications to read and process from the stream.
3. . An Amazon Kinesis Application is a data consumer that reads and processes data from an Amazon Kinesis data stream. You can build your applications using either Amazon Kinesis API or Amazon Kinesis Client Library (KCL).
4. Amazon Kinesis Data Streams manages the infrastructure, storage, networking, and configuration needed to stream your data at the level of your data throughput. You do not have to worry about provisioning, deployment, ongoing maintenance of hardware, software, or other services for your data streams. In addition, Amazon Kinesis Data Streams synchronously replicates data across three availability zones, providing high availability and data durability.
5. The throughput of an Amazon Kinesis data stream is designed to scale without limits via increasing the number of shards within a data stream.
6. By default, Records of a stream are accessible for up to 24 hours from the time they are added to the stream. You can raise this limit to up to 7 days by enabling extended data retention.
7. The maximum size of a data blob (the data payload before Base64 encoding! within one record a 1 megabyte (MB).
8. Each shard can support up to 1000 PUT records per second.
9. Kinesis vs SOS: Amazon Kinesis Data Streams enables real-time processing of streaming big data. It provides ordering of records, as well as the ability to read and/or replay records in the same order to multiple Amazon Kinesis Applications Amazon SQS lets you easily move data between distributed application components and helps you build applications in which messages are processed independently (with message-level ack/fail semantics), such as automated workflows.
10. A record is the unit of data stored in an Amazon Kinesis data stream. A record is composed of a sequence number, partition key, and data blob. Data blob is the data of interest your data producer adds to a data stream. The maximum size of a data blob (the data payload before Base64-encoding) is 1 megabyte (MB).
11. Partition key is used to segregate and route records to different shards of a data stream. A partition key is specified by your data producer while adding data to an Amazon Kinesis data stream. For example, assuming you have a data stream with two shards (shard 1 and shard 2). You can configure your data producer to use two partition keys (key A and key 8) so that all records with key A are added to shard 1 and all records with key B are added to shard 2.
12. A sequence number is a unique identifier for each record. Sequence number is assigned by Amazon Kinesis when a data producer calls PutRecord or PutRecords operation to add data to an Amazon Kinesis data stream. Sequence numbers for the same partition key generally increase over time; the longer the time period between PutRecord or PutRecords requests, the larger the sequence numbers became.
13. Amazon Kinesis Data Firehouse is the easiest way to load streaming data into data stores and analytic took it can capture, transform, and load streaming data into Amazon S3, Amazon Redshift Amazon Elastic search Service, and Splunk, enabling