How to interact with IoT Core shadows from AppSync

Read and modify the shadows for things with the HTTP data source

Tamás Sallai

6 mins

Want to learn AWS serverless development? Click here

IoT Core shadows

Devices connect to AWS IoT Core via MQTT and use the protocol to publish and subscribe to topics. For example, a thermometer can report the current temperature by publishing to a specific topic, or a door might write when it's opened, or a smart light might subscribe to a topic that controls whether it is on or off.

In AWS IoT Core, devices can publish and subscribe to any topics they want which makes it a bit messy when you have a lot of devices writing to a lot of different topics. Because of this, AWS provides a shadows service as well. Devices, called things in AWS, can have different shadows, usually per physical function. Then these shadows provide a structured way to communicate data to and from the cloud. For example, a light might report movement and temperature in separate shadows, and listen for turn on/turn off events in a third one.

Apart from namespacing, shadows also support getting the previous state, provide accept/reject topics, document deltas, persistence, and a lot more features. Usually, when you have a device that you want to connect to AWS, it's best practice to use shadows.

Shadows have reserved topics for Device <-> Cloud communication, as well as an HTTP interface for reading and writing. This means if a service can send a signed request to the data endpoint then it can read and write the shadow state.

Which is something AppSync is perfectly capable of.

In this article we'll build a simple GraphQL API that reads a value from a shadow of a specific thing, and has a mutation that increments that value. This provides the basis of reading and writing the state from AppSync.

Here's the schema we'll implement:

type Mutation {
	increase: Int
}

type Query {
	current: Int
}

schema {
	query: Query
	mutation: Mutation
}

Use the HTTP data source to interact with HTTP APIs directly from AppSync

How to send HTTP requests from AppSync resolvers

Book

Building GraphQL APIs with AWS AppSync

How to design, implement, and deploy GraphQL-based APIs on the AWS cloud

IoT setup

In IoT Core, we have a thing:

resource "aws_iot_thing" "thing" {
  name = "thing_${random_id.id.hex}"
}

Note that we don't need a device certificate to interact with the HTTP interface of the shadow. This is because it supports the AWS Signature version 4 to authenticate:

The HTTPS interface uses the AWS Signature algorithm for authentication

IAM permission

The AWS signature algorithm relies on IAM credentials, which are usually short-lived ones issued for IAM roles.

This is also what AppSync supports, so we need to add a permission to AppSync's role:

data "aws_iam_policy_document" "appsync" {
  statement {
    actions = [
      "iot:GetThingShadow",
      "iot:UpdateThingShadow",
    ]
    resources = [
      "${aws_iot_thing.thing.arn}/*"
    ]
  }
}

The permission after deployment:

The IAM permission needed to interact with shadows

Even though the documentation says the resource is "arn:aws:iot:region:account:thing/thing" I found that it's not enough. That is why there is a /* in the end.

HTTP data source

The HTTP data source defines the endpoint and the signing options along with the role.

data "aws_region" "current" {}

data "aws_iot_endpoint" "iot_endpoint" {
	endpoint_type = "iot:Data-ATS"
}

resource "aws_appsync_datasource" "shadow" {
  api_id           = aws_appsync_graphql_api.appsync.id
  name             = "shadow"
  service_role_arn = aws_iam_role.appsync.arn
  type             = "HTTP"
	http_config {
		endpoint = "https://${data.aws_iot_endpoint.iot_endpoint.endpoint_address}"
		authorization_config {
			authorization_type = "AWS_IAM"
			aws_iam_config {
				signing_region = data.aws_region.current.name
				signing_service_name = "iotdevicegateway"
			}
		}
	}
}

The endpoint is the iot:Data-ATS. As I learned, ATS stands for "Amazon Trust Services" and this endpoint is special as it is signed by Amazon's root certificate instead of VeriSign's.

The signing region and the service comes from where the shadow is. In this examle, the thing is in the current region, and the service is always iotdevicegateway.

Reading values

The resolver needs to send a request to the HTTP endpoint with the specific path and query parameters:

{
	"version": "2018-05-29",
	"method": "GET",
	"params": {
		"query": {
			"name": "test"
		},
	},
	"resourcePath": "/things/${aws_iot_thing.thing.name}/shadow"
}

The resourcePath contains the thing name, and the name query parameter defines the shadow name. Here, it will read the shadow named test.

The structure of the JSON in a shadow follows a strict structure, at least on the top level. There is a state object with a reported and a desired properties. These then contain the payload.

The idea behind this is that the device reports its state (state.reported), while the cloud can request a state change (state.desired). Combined with persistence, it does not matter if the device is offline; when it comes online next time it can read the desired state and act accordingly.

After the request is sent, the resolver needs to extract the value from the response:

#if ($ctx.error)
	$util.error($ctx.error.message, $ctx.error.type)
#end
#if ($ctx.result.statusCode == 404)
	#return(0)
#end
#if ($ctx.result.statusCode < 200 || $ctx.result.statusCode >= 300)
	$util.error($ctx.result.body, "StatusCode$ctx.result.statusCode")
#end
$util.toJson($util.parseJson($ctx.result.body).state.reported.value)

Error handling

The data source populates the $ctx.error only if there was a problem with the request itself. This can happen, for example, when the resulting template does not conform to the data source's schema. This kind of error is rare and usually means there is an issue with the mapping template.

A more common error is when the HTTP response contains a non-2XX status code. The third part checks that.

Then there is a special case: the shadow does not exist. This is a 404 status and depending on the use-case it may or may not be an error. Here, the resolver returns 0 in that case.

Writing values

Writing works similarly, only the parameters are a bit different:

#set($newVal = $ctx.prev.result + 1)
{
	"version": "2018-05-29",
	"method": "POST",
	"params": {
		"query": {
			"name": "test"
		},
		"body": $util.toJson({
			"state": {"reported": {"value": $newVal}}
		})
	},
	"resourcePath": "/things/${aws_iot_thing.thing.name}/shadow"
}

The resourcePath is the same and the method: POST defines that it's a change operation. The name query parameter defines the shadow, same as before. Then the body defines the value to be set in the shadow. In this example, it writes the state.reported.value.

Testing

Let's see how all these work in practice!

First, send a request to get the current value:

query MyQuery {
	current
}

The response is 0, as the shadow does not exist:

{
	"data": {
		"current": 0
	}
}

Run the mutation:

mutation MyMutation {
	increase
}

The response is 1, and it creates the shadow:

{
	"data": {
		"increase": 1
	}
}

The IoT Core console shows that the shadow is created and there is activity for this thing:

The state of the shadow shows the value:

{
	"state": {
		"reported": {
			"value": 1
		}
	}
}

Conclusion

IoT Core device shadows provide a powerful way to implement communication with devices. They offer MQTT topics specific to a shadow and an operation that is easy to use from embedded devices. Then these values are available via a HTTP interface, making it simple to integrate with other services.