➜ GettingStarted git:(master) ✗ aws configure –profile “sandbox”
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]: us-west-2
Default output format [None]:
➜ GettingStarted git:(master) ✗ aws s3 ls s3://xxxx-bucket –profile sandbox
getShardIterator, by its name, it gets the iterator for the shard.
get-shard-iterator --stream-name <value> --shard-id <value> --shard-iterator-type <value> [--starting-sequence-number <value>]
Let’s say we use getShardIterator, we got iterator on shard2, seq2_0. Then we use this iterator to iterate message in Kinesis shard2 from seq0_0. After one iteration, it returns next iterator which points to seq2_1.
If we want to iterator message in all shards, we need to generate the iterators on each shard.
Another caveat is that the neighbor sequence numbers for neighbor message in one shard is not sequential order. For example, it’s not the case that first message sequence number is 2355, then next message sequence number is 2366.
In this case, we can understand why in Kinesis when we talk about “lag” concept in Kafka, it only talks about millisbehindlatest instead of sequence number.