How to build an automated test suite in Postman, step by step.

I am currently working on a web service. A tiny message board where users can post messages and retrieve messages that others have written. Like Twitter but less fancy and probably with less traffic. The basic functionality this service needs is posting, retrieving, editing and deleting messages. These functions are provided via an HTTP API.

To test this API I have been using Postman. I have used Postman before in a rather ad-hoc fashion, but this time I wanted to build a proper, automated test suite. And I wanted to use the testing tools Postman provides instead of just the “send a request and see if the response looks alright” method I had been using before.

One of my testing scripts

So I did, and it worked out pretty well. I did have some problems during this process that eventually led to some insights. Some if these problems/insights were of the kind where I wished that someone had written a blog post about them so that I wouldn’t have had to to figure them out by myself. And that is how this blog came about. I definitely am not a Postman expert, so it should probably be considered a guide by a Postman newbie, for other Postman newbies.

In this post I address a specific topic that gave me some trouble: the Postman environment variables. If what you need is a general introduction to Postman, their Getting Started and short blogpost about testing worked well for me.

That’s it for this introduction, the rest of the post will be divided into suitable titled parts so that you can easily find what you actually came here looking for, promise!

Variables in Postman

When testing with Postman, you generally use the collection runner to run an entire collection of requests at a time, tests included. It is natural, then, to want to save values between requests and between collection runs.

You may for example receive a user id in response to one request, and then want to use that id in the body of the next request. It is also useful to have more-or-less permanent variables with things like URLs and test user names. This is what the Postman variables, stored in the pm javascript object, let you do.

Postman variables

There is a doc here, but I didn’t really think it told me everything I wanted to know, and certain parts were not that clear to me. Also, I didn’t see it mentioned anywhere that ordinary Javascript variables also persist throughout a collection run if defined without a prefix! So here are some things that I spent a few hours figuring out, so that you hopefully won’t have to.

Scopes

There are four variable “scopes” in Postman: globals, collection, environment and iterationData. This is in addition to the ordinary Javascript variables in your scripts of course. They are retrieved with pm.variables.get(“varname”) in code, and with {{varname}} in the various non-Javascript text fields of the user interface (pm is a javascript object available in Postman, see the API docs here.)

Getting Postman scoped variables in input fields and code

The same variable name can be defined in different scopes, and which one is retrieved by variable.get is determined by their hierarchy: iterationData overshadows environment overshadows collection overshadows globals. This is of course why they are called “scopes.” Oh, and the entire hierarchy is per workspace.

“Local” variable scope

In the Postman docs there is talk about a fifth scope called “local” that is supposed to be the innermost scope. This apparently just refers to the regular javascript variables, and personally I wouldn’t call them a scope in relation to the others.

They are completely outside the hierarchy, since they cannot be retrieved with pm.variables.get(“var”) or {{varname}}. They only way to make javascript variables interact with Postman variables is by explicitly setting and getting.

Local variables

Note that Javascript variables declared without a prefix (i.e. just foo, not var foo, let foo or const foo) persist throughout the collection run. And this works for functions declared as variables too! But they only persist during a run, so if you set a variable x in one request and then manually run another, x will not be available in the latter.

Setting and getting variables

As mentioned, all Postman variables can be retrieved with pm.variables.get(“varname”) in Javascript code and with {{varname}} in the various interface text fields. With the exception of collection, it is also possible to explicitly get variables from a specific scope. Other than that, they are set and retrieved in various ways:

globals are the outermost scope in a workspace. They can be set in the UI, or in code with pm.globals.set(“varname”, varval). They can also be retrieved specifically with pm.globals.get(“varname”), ignoring scope. I have been using them for things such as JSON schemata, that are constant throughout a project.
collection variables are different for each collection. They can only be set in the UI, and only retrieved with pm.variables.get and {{varname}}. I use them for mock text and usernames that need to be consistent throughout a test suite.

environment variables are bound to the environment they are defined in. They can be set in the UI, and with pm.environment.set in code, and specifically retrieved with pm.environment.get. You choose the environment every time you run a a collection, so they are useful for storing for example URLs, so that you can have a “local” environment that points to localhost, and a “remote” environment that points to your cloud service.
iterationData variables are read from a file that is selected in the collection runner. They can be specifically retrieved with pm.iterationData.get. They are used when iterating collection runs with different variable values each time. For details read the docs or this blog. Tiny bug warning: If you have an iterationData file loaded when running a collection, and want to repeat the run by pressing “Retry,” the data will no longer be available. Instead, you have to go back to the first tab of the runner and run from there.
Javascript variables aren’t really part of the pm object, but I thought I’d mention them here anyway since they can be semi-persistent. If declared without a prefix, i.e. no var, let or const, a variable will persist during the single run in which they were defined. Pro tip: Remember that functions can be declared as variables!

Types and how to store them

All variables set via text fields in the UI become strings. And if you take a look in the API docs of the pm object you will see that the set functions take string parameters. But the get functions return “*”? That is, they can return things other than strings?

Javascript is of course dynamically typed so nothing is actually stopping us from providing other types as input. And some experimentation indeed shows that it is possible to save types other strings with globals.set, environment.set, and with the JSON file that is provided to iterationData.

Some more experimentation shows that the types we can store like this are string, number, boolean and null, and array and object consisting of these. Functions however will not persist between requests, and undefined turns into null. If you absolutely need to store a function permanently you will have store its code as a string and then use eval when you retrieve it.

Do you really need permanently persistent variables other than strings? Well for instance, I use Tiny Validator to validate the JSON responses I get. And for that I need the JSON templates that Tiny Validator compares to. The API JSON schema is fixed (well, it should be,) so it is natural to store the templates at a global workspace level. I could of course enter them as strings in the UI and use JSON.parse every time I need them. But it was easier for me to hard-code them into globals in a script. It can also be easier to write complex objects in code than in text fields.

Since only the setters can store non-string variables, the architecture to properly use them globally requires a little bit of extra thought. I have been setting them in the pre-request script of the first request of a collection. This works for me as I am the only person (pretty much) working on the tests, and I have only one long test collection.

But if you have several collections then this will not be DRY. And if you are on a team you have to consider that this would mean that the variables are reset to the hard coded values every time a collection is run, so centrally managing global variables could be harder. In these cases, maybe you could have a collection with a single request in it, whose sole purpose is to set non-string globals. This request could then be managed by whoever manages global variables.

Built-In Dynamic Variables

I should probably also mention that there are three built-in variables in Postman. These are used for generating GUIDs (random codes used as identifiers,) timestamps and random integers from 0 to 1000. These are accessed with {{$guid}}, {{$timestamp}} and {{$randomInt}} respectively. They can only be accessed where the double brackets syntax is used, so not in javascript code.

Next Time

Maybe JSON validation with Tiny Validator, maybe something else, who knows?