Expression Tester for DynamoDB Conditions, Filters and Updates

Check the expressions a DynamoDB request carries the way DynamoDB checks them: key conditions, filters, conditions, updates and projections. The tester answers with the same error message DynamoDB gives, word for word, and finds the same problem first when there are several. When the request is fine, it runs it on items you give it and shows what comes back, or what an update leaves behind. It all runs in your browser.

Nothing you paste leaves this page. The tool doesn’t send anything anywhere, and the page’s security policy stops it from fetching or loading anything from another site, so your items stay on your machine.

On a wide screen you can open it full screen.

How to Use It

  1. Pick the operation, then write the expressions and the two maps, or paste the whole request your code sends. Values can be typed ({":one": {"N": "1"}}), the way the API and the AWS command line write them, or plain ({":one": 1}), the way the document clients take them.
  2. Give the key schema, the table’s or index’s partition key and sort key, so the tester can apply the rules DynamoDB has for keys.
  3. Add items: for a Query or Scan, the items in the table; for the other operations, the item as it is now, or nothing for an item that doesn’t exist yet.
  4. Read the answer. A refused request shows DynamoDB’s message, what it means and the spot in the expression. An accepted one shows how DynamoDB groups the conditions, what each placeholder stands for, and the result: the items returned, each with the parts of the filter that held or didn’t, or the item before and after an update.

Put #placeholders on names that need them rewrites every reserved word in the expressions as a placeholder and adds it to ExpressionAttributeNames. It does the same for a name DynamoDB can’t read as one name, such as first-name or _id. In the values of an update’s SET, a-b means a minus b, so it stays as it is there.

What DynamoDB Expressions Are

DynamoDB has no query language like SQL. A request names an item by its key, or a range of items by their partition key, and carries small expressions that say what to do. A key condition picks the items a Query reads. A filter drops some of them before they come back, though they still count as read. A condition makes a write happen only if the item is in a given state, and an update says how to change it. A projection picks the attributes that come back.

The values never go in the expression itself. Each is a placeholder such as :one, defined in ExpressionAttributeValues. Names can be placeholders too, such as #st, and they have to be when a name is one of DynamoDB’s 563 reserved words, which include everyday names like status, name, data, count, value and ttl.

DynamoDB is strict about all of it. A reserved word, a placeholder that isn’t defined, one that is defined but not used, two paths in an update that overlap, or a key condition with OR gets the request refused with a ValidationException that names one problem at a time. The tester also makes a few surprises visible:

  • AND binds tighter than OR, so a OR b AND c means a OR (b AND c). The tester draws the condition as DynamoDB groups it.
  • <> is true when the attribute is missing, and = is false.
  • Strings compare by their UTF-8 bytes, so "¿" sorts after "z".
  • In an update, every value comes from the item as it was, so SET a = b, b = a swaps two attributes.
  • SET list[10] = :v on a list of three adds the value at the end, and REMOVE list[0], list[2] uses the positions from before the update.

How It Was Tested

The tester was checked against DynamoDB Local 3.3.1, the version of DynamoDB that AWS publishes for development and testing.

  • Random requests. A seeded generator wrote 4,800 requests, more than half of them broken on purpose: 1,500 scans with filters and projections, 600 projections, 1,200 queries and 1,500 updates. The tester gave DynamoDB Local’s answer to every scan, projection and query, the same message word for word or the same items in the same order, and to 1,472 of the 1,500 updates. The other 28 each had two or more problems that show only while the update runs, and DynamoDB Local named a different one first.
  • Written by hand. 214 requests each pin down one rule: how names and placeholders are read, which error comes first, number precision, list and set updates, key conditions, projections. The tester matched all 214.
  • Reserved words. All 573 words on AWS’s published list went in as attribute names. DynamoDB Local refused 563 of them as reserved. CONVERT and SIZE are on the list but accepted, and eight words such as AND and SET are keywords that give a syntax error instead. Where the list and DynamoDB disagree, the tester does what DynamoDB does.

The Code

The tester’s logic is one JavaScript file with no dependencies, open source under the Apache License 2.0. It also runs as a command line in Node.js, where a CI job can check every expression a codebase sends:

node expressions/cli.js --request request.json --items items.json --key pk:S,sk:N
node expressions/cli.js --escape 'status = :s AND size(data) > :n'

The full manual, the tests and the requests they replay are in the expressions folder on GitHub. The other tools work the same way.