How to Update AWS Cognito Users in Bulk

Introduction

AWS Cognito is a powerful serverless user management product. But no product or service is ever 100% complete, and, unfortunately, AWS Cognito is no exception.

Amazon's philosophy of 80% dictates that upon reaching this threshold of "readiness", the managed service would be published for early adopters. From then on, it'd be improved based on enthusiasts' feedback, mainly through GitHub and AWS's own re:Post community one complaint at a time.

AWS Cognito undoubtedly undergoes rolling changes. It's a very mature product, but some things are still not up to speed, so frustrated developers subscribe to long-trailing GitHub threads, hoping they'll be addressed soon. Consequently, some operations still require you to roll up your sleeves and un-dust your command line kung fu. Updating AWS Cognito users in bulk is one of them.

The Problem

If you're managing your user pools with AWS Cognito, you'll quickly notice that this service is much less "malleable" than other managed AWS services. Some of these limitations are by design, such as being unable to migrate users from one pool to another easily due to passwords not being stored. However, some are a byproduct of the development team not keeping up with feature requests.

For example, there is no way to edit user attributes in bulk. This is the case across the board: AWS CLI, AWS Web Console, AWS CloudFormation, AWS SAM, AWS SDK, AWS CDK, etc. If you want to groom large quantities of user attributes, you need to get a tad geeky at the time of writing.

The Solution

Note: AWS Cognito does not allow you to modify global attribute settings on the pool level. What we can modify are these attributes on a user level. If you need to change your global user pool settings, you'll have no choice but to create a new one and implement a gradual user migration mechanism to pull users into the new pool from the old one. Reach out to us if you need help doing it.

Step 1: Install and Configure AWS CLI

Since AWS's web console doesn't let you do that, we'll fall back to the AWS CLI. Therefore, the first prerequisite is installing and adequately configuring this fundamental tool.

The installation and configuration of AWS's command line are beyond the scope of this article, so if you still don't have it set up, follow the official instructions to do so. Please make sure you're installing the latest version, v2, at the time of writing.

Step 2: Install the jq Command Line Utility

You'll also need a jq command-line utility, which is the most comfortable way of dealing with JSON data in your favourite shell environment. You can learn more about it from its official website.

Using Homebrew, run the following:

brew install jq && jq --version

The step was successful if the console displayed the package version similar to the output below.

jq-1.7.1

Step 3: Construct and Run the Run the Command

Since we're working with AWS Cognito, the associated CLI command will be cognito-idp. To learn more about using this service's CLI interface, run aws cognito-idp help. In doing so, you'll, once again, realise that none of the commands can affect multiple users at once.

We'll be running the admin-delete-user-attributes command to illustrate the point. Still, you can extrapolate this approach to other operations that would otherwise be permitted only on a single user. Again, you can run the following command to learn more about the specifics.

aws cognito-idp admin-delete-user-attributes help

The first thing we want to do is to retrieve all of the usernames from our pool.

aws cognito-idp list-users --user-pool-id yourUserPoolID --no-paginate --profile yourCustomAWSProfile

In this example, we're not filtering any users out, but if you need to, use the --filter flag. You can learn more about it by running

aws cognito-idp list-users help

You know the drill.

We also want all results simultaneously, without pagination—Hence, the --no-paginate flag.

The result of the above code block would return us a JSON file formatted something like this:

{
    "Users": [
        {
            "Username": "some-username-or-uuid",
            "Attributes": [
                {
                    "Name": "sub",
                    "Value": "uuid"
                },
                {
                    "Name": "email_verified",
                    "Value": "true"
                },
                {
                    "Name": "custom:yourCustomAttribute",
                    "Value": "some-value"
                },
                {
                    "Name": "email",
                    "Value": "test@test.com"
                }
            ],
            "UserCreateDate": "2024-02-14T00:30:59.508000+08:00",
            "UserLastModifiedDate": "2024-02-14T00:31:19.059000+08:00",
            "Enabled": true,
            "UserStatus": "CONFIRMED"
        }
    ]
}

The contents of Attributes might vary, but the structure will be similar.

Now, we need to process this output with jq.

jq -r '.Users.[].Username'

The above snippet traverses the predefined JSON tree data structure returned by aws cognito-idp list-users. We start at the root ., unpack the Users attribute, whose value is an array [], and finally grab the Username attribute value for every item in this list. The flag -r simply indicates to jq that it should return the value without leading and trailing quotes. If you don't use this flag, the quotes will be considered part of the value to search for in the next step, yielding no results. So, this flag is essential here.

Let's combine the above operations to get a list of pure usernames, one per line:

aws cognito-idp list-users --user-pool-id yourUserPoolID --no-paginate --profile yourCustomAWSProfile | jq -r '.Users.[].Username'

The | character passes (or "pipes") the output of the first command as an input of the second command. So, the AWS CLI Cognito request output will be parsed by jq.

Let's now iterate on this clean list of usernames to perform single-user operations:

for username in $(aws cognito-idp list-users --user-pool-id yourUserPoolID --no-paginate --profile yourCustomAWSProfile | jq -r '.Users.[].Username'); do SOME VALUABLE WORK HERE; done

As you can see, the sky is the limit at this stage. For the sake of completing the example, let's delete an attribute:

for username in $(aws cognito-idp list-users --user-pool-id yourUserPoolID --no-paginate --profile yourCustomAWSProfile | jq -r '.Users.[].Username'); do aws cognito-idp admin-delete-user-attributes --user-pool-id yourUserPoolID --username $username --user-attribute-names custom:yourCustomUserAttributeToDelete --profile yourCustomAWSProfile; done

Voilà, your user pool has been groomed. Almost all user-level CRUD operations can be automated this way. And now you're equipped to do so.

Conclusion

AWS Cognito is a potent, highly customisable user management solution with room for extensibility. It should be able to cover most of your authentication needs. Yet some basic operations you'd expect to be there aren't present either by design or because the feature's still in the team's backlog, such as an automatic redirect upon sign-up verification through a link. Oh, well.

Although power users are likely to run into limitations, in most circumstances, there are workarounds, even if they require an "escape hatch", such as the one described above, where you have to "break out" of the AWS toolset and run it through an external grinder before "pouring it" back in, to achieve the desired effect. And, if that's any consolation, it makes all of us better engineers.

Update AWS Cognito Users in Bulk