Aws cli, put/get kinesis record

By | May 21, 2021
Share the joy
  •  
  •  
  •  
  •  
  •  
  •  
AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY=xxxx AWS_SESSION_TOKEN=xxxx \
        aws kinesis put-record \
        --region us-east-1 \
        --stream-name ${streamName} \
        --data "{\"pengliid\": \"pengli\",\"value\": [\"1\", \"2\", \"3\", \"4\"]}" \
        --partition-key pengli-id

In the output, remember the sequence number and shardId

AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY=xxxx AWS_SESSION_TOKEN=xxxx \
        aws kinesis get-shard-iterator \
        --region us-east-1 \
        --stream-name ${streamName} \
        --shard-id ${shard_id} \
        --shard-iterator-type AT_SEQUENCE_NUMBER \
        --starting-sequence-number ${sequence_number}

remember the shardIterator

AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY=xxxx AWS_SESSION_TOKEN=xxxx \
        aws kinesis get-records \
        --region us-east-1 \
        --shard-iterator ${shardIterator}

Output is base64 format. Transform it.

kinesis_output

base64

Kinesis get-records shard-iterator

command1: aws kinesis get-records --shard-iterator xxxxxx
return:
  {
    records: [record1]
      nextiterator: yyyyyyy
  }

command2: aws kinesis get-records --shard-iterator yyyyyyy
return:
  {
    records: [record2]
      nextiterator: zzzzzzz
  }
Category: aws