DynamoDB Query
- Kalpana Malladi
- Aparna Tayi (Unlicensed)
- Lakshmi Manda
On this Page
| Snap type: | Read | ||||||
|---|---|---|---|---|---|---|---|
| Description: | This Snap retrieves data from DynamoDB by executing a DynamoDB Query via the REST API. It processes Snap properties and formats the HTTP request entity body according to the syntax defined in the AWS DynamoDB Query API Reference. It can be used to retrieve data from DynamoDB table as well as Global Secondary Index or Local Secondary Index. Refer to AWS DynamoDB Query API Reference for further details. Input & Output
Modes
| ||||||
| Prerequisites: | None | ||||||
| Limitations and Known Issues: | None at this time. | ||||||
| Configurations: | Account & AccessA valid DynamoDB account is required. Views
| ||||||
| Troubleshooting: | The Snap produces an error document if DynamoDB fails to execute the submitted query. The "request_entity" field in the error document contains the HTTP entity body of the submitted query and may be useful when troubleshooting the pipeline. The Snap processes the Snap properties and prepares the HTTP entity body for the query. The "error_entity" fields in the error document is the error message from the DynamoDB server describing which part of the query has caused the error. | ||||||
Settings | |||||||
Label | Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | ||||||
| Table name | Required. The name of the DynamoDB table. Table names can be suggested if the expression property is disabled. The input document schema and pipeline parameters can be suggested when the expression property is enabled. Example: EmployeeEvents, $TableName, _TableName Default value: None | ||||||
| Secondary index name | The name of Global Secondary Index or Local Secondary Index if used in the query. The Amazon DynamoDB document explains this as following: "Amazon DynamoDB provides fast access to items in a table by specifying primary key values. However, many applications might benefit from having one or more secondary (or alternate) keys available, to allow efficient access to data with attributes other than the primary key." Please refer to Improving Data Access with Secondary Indexes for more details. Example: EffectiveDate-index, $secondary_index, _secondary_index Default value: None | ||||||
| Projection expression | A string that identifies one or more attributes to retrieve from the table. The attributes in the expression must be separated by commas. If no attribute names are specified, then all attributes will be returned. If any of the requested attributes are not found, they will not appear in the result. Example: "EffectiveDate, EmpID, EventID, #name" (without double-quote), or $attributes or _attributes Default value: None | ||||||
| Key condition expression | The condition that specifies the key value(s) for items to be retrieved by the Query action. The condition must perform an equality test on a single partition key value. The condition can also perform one of several comparison tests on a single sort key value. Query can use KeyConditionExpression to retrieve one item with a given partition key value and sort key value, or several items that have the same partition key value but different sort key values. Refer to the KeyConditionExression section in the AWS DynamoDB Query API Reference for more details. Example: EffectiveDate = :val1 Default value: None | ||||||
| Filter expression | A string that contains conditions that DynamoDB applies after the Query operation, but before the data is returned to the Snap. Items that do not satisfy the filter expression criteria are not returned. A filter expression does not allow key attributes. You cannot define a filter expression based on a partition key or a sort key. Note: A filter expression is applied after the items have already been read; the process of filtering does not consume any additional read capacity units. Example: #v >= :num Default value: None | ||||||
| Expression attribute values | Specify the placeholders for the Expression Attribute Names and/or Expression Attribute Values.ed words as attribute names in expressions. DynamoDB reserves certain words and they are called reserved names. You must not use reserved words as attribute names in expressions. If you must write an expression that contains a word that conflicts with a DynamoDB reserved word, then you can define an expression attribute name to use in the place of the reserved word. For more information, see Expression Attribute Names. Similarly, you cannot use certain values as attribute values in expressions and instead must define an expression attribute name. For more information, see Expression Attribute Values.
Example If your table contains a column named uuid (a reserved word) and you want to read a value from the uuid column, then you must assign a placeholder for the column name because you cannot use a reserved word directly in your expression. We can define the Attribute #U, leave the Type blank, and enter the value as uuid. In the Snap Filter expression condition, we enter #u=:val1. :val1 is the placeholder for the value in the table which is defined as Attribute :val1 of Type B with Value dGhpcyB0ZXh0.
| ||||||
| Action | Enter COUNT to get only the number of items after query execution or DESCRIBE TABLE to get the table metadata at the output view, or QUERY for normal query operations. Example: QUERY, COUNT, DESCRIBE TABLE Default value: QUERY | ||||||
| Consistent read | Determines the read consistency model. If set to true, then the operation uses strongly consistent reads. Otherwise, the operation uses eventually consistent reads. Strongly consistent reads are not supported on global secondary indexes. If you query a global secondary index with 'Consistent read' set to true, you will receive an Exception. Default value: False | ||||||
| Pass through | If true, the input document is passed through to the output view under the 'original' key. Default value: False | ||||||
Snap Execution | Select one of the following three modes in which the Snap executes:
Default Value: Execute only | ||||||
Examples
Sample query for Global Secondary Index:
Refer to the attached file: DynamoDB Query for GlobalSecondaryIndex.slp
The 'EmployeeEvents' table has the following items in the above example:
| EmpID | Name | EventID | EffectiveDate |
|---|---|---|---|
| {"N":"3"} | {"S":"Paul"} | {"S":"103"} | {"N":"20170703"} |
| {"N":"2"} | {"S":"John"} | {"S":"102"} | {"N":"20170703"} |
| {"N":"3"} | {"S":"Paul"} | {"S":"102"} | {"N":"20170703"} |
| {"N":"20170210"} | {"S":"foobar"} | ||
| {"N":"1828"} | {"S":"Preferred Name Change Event"}, | {"N":"1475218800000"}, | |
| {"N":"1"} | {"S":"101"} | {"N":"20170802"} |
The Snap processes Snap properties and sends the following HTTP request entity to DynamoDB:
{
"TableName": "EmployeeEvents",
"IndexName": "EffectiveDate-index",
"ProjectionExpression": "EffectiveDate, EmpID, EventID, #nam",
"KeyConditionExpression": "EffectiveDate = :val1",
"ExpressionAttributeNames": {
"#name": "Name"
},
"ExpressionAttributeValues": {
":val1": {
"N": "20170703"
}
},
"ReturnConsumedCapacity": "INDEXES"
}
Then, the Snap produces the output documents as following:
| EmpID | Name | EventID | EffectiveDate |
|---|---|---|---|
| {"N":"3"} | {"S":"Paul"} | {"S":"103"} | {"N":"20170703"} |
| {"N":"2"} | {"S":"John"} | {"S":"102"} | {"N":"20170703"} |
| {"N":"3"} | {"S":"Paul"} | {"S":"102"} | {"N":"20170703"} |
2. Sample query for Table:
Refer to the attached file: DynamoDB Query for Table.slp
The Snap processes Snap properties and sends the following HTTP request entity to DynamoDB:
{
"TableName": "EmployeeEvents",
"ProjectionExpression": "EffectiveDate, EmpID, EventID, #nam",
"KeyConditionExpression": "EventID = :val2 AND EmpID > :val3",
"ExpressionAttributeNames": {
"#name": "Name"
},
"ExpressionAttributeValues": {
":val2": {
"S": "102"
},
":val3": {
"N": "1"
}
},
"ReturnConsumedCapacity": "INDEXES"
}
Then, the Snap produces the output documents as following:
| EmpID | Name | EventID | EffectiveDate |
|---|---|---|---|
| {"N":"2"} | {"S":"John"} | {"S":"102"} | {"N":"20170703"} |
| {"N":"3"} | {"S":"Paul"} | {"S":"102"} | {"N":"20170703"} |
Snap Pack History
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.