API guidelines — Part C: API development 2022

Learn about the technical details in developing an API (Application Programming Interface) and the standards that need to be met.

1. API design and development

This section covers key considerations such as good API design, correct use of the Hyper Text Transfer Protocol (HTTP), version control and various technical standards.

1.1. API artefacts

Some fundamental artefacts that comprise an API deliverable are:

• interface specification, accessible via some form of catalogue
• API code and policies
• tests
• Service Level Agreement (SLA) for the API, identifying performance, availability.

These artefacts are the responsibility of the API developer and should be mandated as a minimum set of deliverables when developing an API. All of these artefacts must be managed in a source code management system so that changes can be tracked and collaborative development can be performed.

1.2. API design

Learn about good API designs and how it will attract other developers.

Application developers will invest heavily in using your API. They will invest in learning the design and behaviour of your API, in developing and testing around your API and may even invest in developing an entire business model on your API.

Good API designs will attract more developers. Conversely, bad API designs will drive away developers and drive up costs — support costs, maintenance costs and operating costs.

1.2.1. When an API is appropriate

In the past, the default reaction to a requirement for capability has been to develop a web application. This is now gravitating towards APIs as the default. So, when is it better to build an API than a web application?

Situations where an API may be more appropriate are when:

• commercial organisations or non-governmental organisations (NGOs) are screen scraping data from the agency’s website
• the agency holds a single authoritative source of the truth
• there is a need for (near real time) information exchange with a partner agency
• parts of a business process are (or may be) outsourced
• the agency’s service can be included as part of a larger business process
• the agency needs internal systems to interact with cloud-based commercial off-the-shelf (COTS) solutions (software as a service (SaaS))
• businesses require easy access to (public) information
• commercial organisations want to build capability into their own applications that will benefit the public (mobile / web applications)
• the agency expects other parties to act as agents or intermediaries for their services
• the agency expects to support service delivery across multiple channels.

1.2.2. Types of API

There are several different types of API and the type you choose may depend on the technical use cases that you apply to both consumption and provision of your API.

Figure 1 demonstrates a combination of a synchronous API and an asynchronous API.

Detailed description of figure

Figure 1 illustrates an asynchronous and synchronous claims API in a Unified Modelling Language (UML) sequence diagram. It shows interactions between different actors in the use case. The sequence is divided into the following 4 sections and associated steps:

Section 1 — Register

1. Service provider system sends POST / register to claims API.
   This call is to enable the service provider to register their notification endpoint. Note that this is illustrative only. In reality most consumers will register with an agency using a developer portal or equivalent and create client application credentials with registered notification endpoint(s).

2. Claims API sends return registration details to service provider system.

Section 2 — Create claim (synchronous API process)

3. Customer sends access funded product or service to service provider.
4. Service provider sends create claim details to service provider system.
5. Service provider system sends register claim details to claims API.
6. Claims API sends create claim details to agency core system.
7. Agency core system creates claim and assigns to workflow.
8. Agency core system returns claim identifier (ID) and status to claims API.
   This interaction is illustrated as a RESTful API call for the purposes of this example.

9. Claims API returns claim ID and status to service provider system.
10. Service provider system returns claim ID and status to service provider.
11. Service provider delivers service update to customer.

Section 3 — Subscribe to events

12.  Service provider system sends SUB claims / (claimed) to claims API.

Section 4 — Receive updates (asynchronous API process)

13. Agency core system provides workflow update.
14. Agency core system sends PUB claims / (claimed) to claims API.
15. Claims API receives subscription message to service provider system.
    Alternative to 15:
    15a. Claims API sends hook notification to notification endpoint.
    15b. Notification endpoint forwards notification to service provider system.
16. Service provider system notifies service provider.
17. Service provider notifies customer.

View larger image (PNG 250KB)

Figure 1 has 2 associated API specifications. The first is an OpenAPI specification that describes the ‘create claim’ and ‘get claim’ RESTful interactions and the second is an AsyncAPI specification that describes the ‘receive updates’ interaction. These example specifications have been provided in Appendix D.

1.2.3. API design principles

This section assumes that API principles defined in Part A: API concepts and managment 2022 of these guidelines have already been read.

1.2.3.1 Future-focused design

APIs should not be tightly coupled to legacy applications, exposing whatever capabilities the legacy system offered. Nor should they be designed to work in the way the legacy system currently works.

Instead APIs should be consumer driven — built to expose the resources that consumers need, whether those resources are in legacy systems or new. There should not be a drive to expose an entire product via the API or to wait for the perfect backend system to be available. The recommendation is to just offer as much as is practically useful, then evolve it to meet needs. The API interface should be abstracted from the backend, so that backend systems can be changed or replaced over time without needing to change the interface.

The aim is to be future focused (while still pragmatic) and develop APIs to meet future needs. A good example of being future focused is building APIs to support Hypermedia as the Engine of Application State (HATEOAS). This is where an API, in response to a consuming application’s request, is programmed to return hyperlinks (URLs) that indicate the options for subsequent actions or information.

1.2.3.2. Layering

When designing and developing an API it is important to consider that an API is made up of the following distinct functional layers:

Security

Every API will have a security component. It is important to recognise that this is not only authentication and authorisation for access to an API, it also includes threat protection (Distributed Denial of Service (DDoS), Structured Query Language (SQL) Injection, Cross Site Scripting, and so on) as well as availability and Quality of Service (QoS). When designing and developing APIs it is often cost effective to create a common framework that handles security for all APIs. See Part B: API security 2022 for more details.

Caching

Caching can dramatically improve the performance of an API. When designing APIs consider what, when and where to cache. Understanding how data is changed and how often it is changed is an important consideration, as well as which layer it is most appropriate to cache at. A common caching strategy should be developed for APIs that would benefit from it.

Representation

When designing and developing an API it is important to consider the representation of that API. This commonly includes an interface specification that fully describes the API. To ensure the success of an API it should be easy to consume driving a well-considered representation layer.

Detailed description of figure

Figure 2 illustrates the functional layers that make up an API. A stylised image of a person sits at the top above an image of a computer, the consuming application. An arrow points down from the consuming application through the layers of security, caching and representation to the API at the bottom. An arrow from the API points back up through the same layers to the consuming application.

View larger image (PNG 21KB)

1.2.3.3. Standards based

Web standards have rapidly become powerful agreements, which span not just local regions but are internationally accepted and enable commonality and consistency. Using standard HTTP and URLs, the core technologies of the web, along with standards such as JSON and OAuth (Open Authorization) ensures that agencies are not creating bespoke or proprietary technologies.

Hence the principle is to build to the latest versions of existing open and accepted standards such as:

• HTTP
• OpenAPI
• AsyncAPI
• REST
• JSON
• OAuth
• OpenID Connect (OIDC).

Refer to Appendix E for more details.

1.2.4. Designing an API

When designing an API, it is important to perform business process analysis to ensure that API development is business driven rather than technology driven. Technology-driven projects rarely meet customers’ needs in the long run, so it is important to gain background in who could be using the API, and for what. As mentioned previously, co-design is fundamental to driving the right API development. To help identify potential partners to involve in the co-design, consider processes that:

• currently depend on information the API could expose
• require a capability an API could expose.

Some actors will be human (for example, application developers) while some actors will be the systems that will interact with, or depend on, the API. There may be different types of actors, some public sector, some commercial entities, some public.

When representatives for the potential actors are identified, start co-designing with these representatives. First and foremost consider the requirements for the API. Application developers often couch their requirements in terms of how the API should work, rather than what the API needs to do. Do not get bogged down in the variety of proposed solutions from each developer.  Focus on extracting the true requirements by performing functional analysis (for example, use cases) and data flow analysis, and then identify resources and work out the granularity (see section 1.2.7. Granularity).

It is especially important that security and information privacy impacts are identified up front and addressed early on in the design process. Assess the information being passed and the types of access different customers and / or consuming applications should have to the API. This will help drive development of security policies alongside the design of the API.

One common pitfall in API design is to map all existing data tables onto resources and develop the associated CRUD (Create, Read, Update and Delete) capabilities into an API. Usually this results in a poor API design and tightly couples the API design to the underlying data structure.

Another common pitfall is to design APIs as an extension to, or way into, monolithic legacy systems. This should be avoided as it tightly couples the API to the legacy system. Both of these pitfalls will create issues for both provider and consumers in the long term.

At this point in the design process, agility is probably more important than completeness. Share early design thoughts and interface specifications with the developer community and quickly make changes in response to their feedback. Work through some sequence diagrams with them to help pin down API interactions and inform API design thinking.

The correct API design will likely not please every API developer, so do not try to be all things to all developers. A rule of thumb is that the API design is probably on the right track if most developers are a little unhappy, but all are able to achieve their aims with the proposed design.

It is important not to try to bypass evolution steps and try to build for all potential use cases right from the offset. Starting simple and focusing on a single channel or interaction initially is a more measured progression towards API delivery. By building onto these simple building blocks, API evolution over time naturally progresses towards omnichannel applications.

1.2.5. Design considerations

The following considerations emerge when looking to develop APIs in the public sector.

• Understand the data, understand the consumers. Before starting work on APIs, gain a good understanding of what data is held, the complexity of that data, its relationships to other data, the downstream dependents on that data, and potential consumers of the data. It is also a good idea to understand the consumers’ drivers and downstream usage or customers.
• Design for REST. If the interaction appears RESTful then develop a REST API, for example, one to many. In some cases, however, a gRPC or GraphQL API may be appropriate and are supported by these guidelines. REST APIs are widely understood, easier to govern and suitable for most interoperability use cases. It is also important to note here that if there is an appropriate supported standard such as Fast Healthcare Interoperability Resource (FHIR) then the resources defined in that standard should be used rather than a set of agency-defined resources.
• The information supplied via an API may still be dependent on overnight batch processes for updates. It is worth making the information’s timeliness and update period clear to potential consumers of the API.
• APIs should not be used to handle large batch transfers because if one single data write fails then the whole batch fails. APIs work better for multiple individual update transactions.

1.2.6. Design-driven development — Required

When building APIs, a design-driven approach must be taken. This includes:

Interface specification first

The best way to design an API is in collaboration with potential consumers of that API. Creating the interface specification first makes it easier for application developers to see what the API is going to offer and how it could be used.

Consumers’ early feedback will ensure that the API design is heading in the right direction and will be usable and appropriate to their needs. Using a simple modelling language to define the interface specification makes it easy to view and edit API definitions.

Iterative approach

It has been acknowledged that big bang releases rarely deliver business or customer benefit. An iterative approach, with ongoing releases offering gradual improvement or evolving capability, gives a finer tuned delivery, better sense of momentum, illustrates progress and enables third parties to coordinate efforts with API developments.

An iterative approach incorporates continuous improvement, which recognises the need to support continuing evolvement of the API beyond the first delivery. The API is a product whose capabilities should undergo continuing improvements based on consumer feedback, performance analytics and emerging needs.

Continuous integration / testing

Automation provides a quick turnaround for informing API developers about breaking changes being submitted by developers who are all working on the same code base or product, for example, an API. The idea is that all developers submit code changes as often as possible (for example, into a version control system) allowing code to go through an automated, integrated build process that ensures that nothing has broken the build. This build process could happen many times a day. Any errors are quickly identified and alerted to the team who can rectify them in a timely fashion.

Tests can be written against the interface specification quite early on in the development process by developing just enough API code to enable the test to be run (stubs). The tests can then be incorporated into the automated build process, giving early warning of regression test failures. API code should not be able to progress through Software Development Life Cycle (SDLC) environments until successful test execution.

1.2.7. Granularity

There is a question as to how fine-grained an API resource should be. APIs should be designed at the lowest practical level of granularity because this makes each service simpler and allows them to be combined in ways that suit the application developer. The key principle is to design services that can be re-used and combined in different ways, and not tie the application developer to a specific way of working or sequence of API consumption just because that is how it is configured or built in the backend.

If an API offers very fine-grained resources, then consuming applications end up having to make many more calls to the API in order to collect all the information they need, resulting in chattier communications. On the other hand, if the API is based on very coarse-grained resources (returning everything about the resource) the response payloads could end up being enormous, may not support all API application developers’ needs, and the API could become cumbersome to use and maintain.

There may also be a need for varying granularity within one API, depending on the purpose and use of that API. For example, if a blogging API was being created, it may make sense to offer a coarse-grained resource for posting a new blog entry as a bundle of content including pictures, sound, keywords as well as the textual blog entry. But when offering the ability to like or comment on a blog entry it would make sense, for clarity and ease of use, to offer these as separate sub-resources, one for indicating a ‘like’ and one for submitting a comment.

It is important to aim for a granularity that will prevent business logic from leaking into the API, for example, requiring API calls to be made in a certain sequence to meet with internal business processes. With the blogging example, consider the impact of a fine-grained API being offered for posting blog entries, with separate calls for adding images, sounds and metadata. If metadata is mandatory, then the API application developer needs to ensure that every blog post call is followed by a metadata call. This puts the onus on the application developer to get the sequence of calls correct and could lead to inconsistent data if they do not adhere to the rules. It also leads to close coupling between the consuming application and the API, and if the business logic changes in the future there is a downstream impact on all application developers, requiring them to modify their applications.

General guidelines are:

• Do not aim for the finest granularity — an API should be built around each discrete and updateable resource.
• There does not need to be a 1-to-1 mapping between a manual service and a corresponding API. APIs should support the process but not try to encapsulate the process.
• A rough guide is to have one API operation for each entity life cycle state transition.
• Resources can be identified by reviewing a business process and identifying the key entities that underpin the process. Any entities that the organisation manages or maintains throughout its full life cycle will be potential API resources. There is typically one API operation for each entity life cycle state transition.
• There should be only 1 API function for 1 business outcome, for example, change an address.
• Consider using query parameters in URLs for different granularity of resources. For example, ‘/passengers.json’ could return a list of names, while ‘/passengers.json?detail=full’ could return detailed information about each passenger in a list.

1.2.8. Interface specification

An API represents a contract between the provider and the consumer for access to provider resources. Because API consumption is a programmatic exercise, it is important to have a clear definition of what the API offers and how those API resources are to be accessed. This definition is the interface specification.

The interface specification should be designed in advance of developing the API, as the act of working out the specification often helps think through all the issues that could impact the design of the underlying resource-handling code. It also helps consuming application developers to review the capabilities being offered to see if the capabilities meet their needs before they start developing to the specification. When outsourcing API development, the interface specification can be written in abstract as a means of defining the API the vendor should build. The specification can be handled as a separate entity, is version controllable, and it should be possible to use it as the main API documentation or reference.

There are several API modelling languages available for defining the interface specification. Some are proprietary (for example, REST API Modelling Language (RAML), API Blueprint) while others are machine readable but not human readable (for example, Web API Description Language (WADL)).

The most commonly used API modelling language today is an open standard known as the OpenAPI Specification (formerly Swagger) maintained by the Open API Initiative. This initiative is supported by many of the main commercial entities in IT (Google, IBM, Microsoft, Atlassian, Paypal). It offers a standard, programming language-agnostic (JSON / YAML Ain’t Markup Language (YAML)) interface to REST APIs that allows both humans and computers to discover and understand the capabilities of the service.

It is important to use a modelling language to define the API interface specification as essentially it is text or code and can be maintained using source code management systems.

It is recommended to use OpenAPI / Swagger as the interface specification language for all APIs being developed.

OpenAPI/Swagger — OpenAPIs

1.2.9. Orchestration

As a general rule using APIs as an orchestration tool is not recommended. This is due to the complexity that this can introduce to an API architecture. That said, however, simple orchestration may in some cases be appropriate. An example of this may be a mashup API that creates a new logical API resource by combining related data held in 2 or more backend systems.

A mashup is not appropriate when you create an API that merges 2 distinct API products together. If a consumer requires this functionality the mashup should be performed by the consuming application.

Other examples of simple orchestration could include mediation such as message transformation (JSON to eXtensible Markup Language (XML) or canonical message model transformation).

1.2.10. Software development kits (SDK)

It is recommended that API providers offer an SDK to developers of consuming applications.

An SDK can be considered the implementation toolset for use of an agency’s APIs. By providing developers with an SDK they can build applications faster without having to understand all of the API’s complexities. An SDK should provide sample code that explains the functionality of your API to potential application developers.

An example of a public service API SDK can be found at the US Department of Labor GitHub repository Publication / Catalogue — Github.

Once an API is in sufficient state to be offered to API consumers, the API definition should be published to an API catalogue. The primary API discoverer is the developer, so an external API must be well documented, and provide accurate and up-to-date guidance via the catalogue.

Government are considering options for a central point of discovery for externally accessible public sector APIs.

Publishing to a catalogue is recommended for internal APIs.

Publishing to a catalogue is required for external APIs.

1.3. Architectural and deployment patterns

Table 2 identifies some architectural and deployment patterns that bear further investigation. There are a number of vendor specific references that are also available however these guidelines do not recommend any particular one.

1.4. HTTP verbs

Access to REST APIs must be via the standard HTTP verbs: GET, PUT, POST, DELETE in line with the W3C Standard.

1.5. URI construction

Uniform Resource Identifier (URI) construction is important in that it is the door through which consumers enter to obtain agency resources. It should be intuitive and easy to guess what an endpoint does just by looking at the URI and HTTP verb, without needing to see a query string. As basic guidance, endpoint URLs should advertise resources, and avoid verbs.

In some situations an implemented standard may define the URI structure for you. A good example of this is Resource — FHIR v4.0.1 — Fast Healthcare Interoperability Resources (FHIR).

1.5.1. API offering — Recommended

Is it recommended that agencies make it clear in the URL that their offering is an API:

1.5.2. Version

APIs should have a clear indication of the version, so that application developers can ensure they are using the appropriate version for their consuming application.

Header-based versioning is recommended (see section 1.10. API version control) however, it is recognised that some API infrastructure does not readily support header-based versioning.

URL-based versioning is a viable alternative, as the version number in the URL should only change when major revisions have been made and the interface has changed substantially without backwards compatibility. For URL-based versioning the URI should include /vN with the major version (N) and v as a prefix. Agencies should not include minor version numbers when using version numbers in the API paths.

1.5.3. Namespaces

For APIs where an agency holds multiple responsibilities that may result in overlapping resource naming (for example, MBIE could have the resource ‘accommodation’ in 2 contexts: tenancy/contracts, tourism/contracts) it is recommended that namespaces be used to avoid any ambiguity. The namespace would be the first noun in the URI and should reflect the function of government being offered by this API. Namespaces may be singular or plural, depending on the situation.

1.5.4. Resources and sub-resources — Recommended

Resource names should be noun-based and collection resource names should be plural nouns, for example, ‘/passengers’ in lower case. Resource naming should be short, simple and clearly understandable. It should also be human-guessable, avoiding technical or specialist terms where possible.

Sub-resources must appear under the resource they relate to, but should go no more than 3 deep, for example, ‘/resource/id/sub-resource/id/sub-sub-resource’. If you reach a third level of granularity (sub-sub-resource), it may be worth reviewing your resource construction to see if it is actually a combination of multiple first or second level resources.

The URI references for resources should consistently use the same path structure to refer to resources. Sub-namespace or sub-folders should be avoided, to maintain path consistency. This allows application developers to have a predictable experience in case they are building URIs in code.

1.5.5. Word separation

Hyphens have traditionally been used as word separators in URLs, as search engines (particularly Google) prefer a hyphen to split words because a hyphen is not a word character (as defined in regular expression language). This has led to hyphens, or kebab case, being the de facto standard in the interests of readability and Search Engine Optimisation (SEO).

Therefore, in order to keep URLs consistently formatted, path and query string parameters should be lower case with hyphen separators for multiword names.

1.5.6. Query arguments

Query arguments should be used to filter a response, for example, modify a returning result set. The general rule is if it changes the behaviour of the:

• result set, then it should be a query argument
• API, then it should be in the path.

Query arguments are generally used for:

• sorting or ordering the result set — for example, sort-order=ascending
• pagination — pagination is a query argument because it effectively acts as a filter and limits the result set returned. This is particularly useful with large response data sets. When using pagination, it is a good idea to inform the consumer where they can find previous and subsequent result sets using hypermedia (see section 1.7.7. HATEOAS).

• limiting the result set, for example, by specifying which fields to return. This approach can be complicated and is often a decision based on functionality versus complexity. For example, it may be desirable to be able to filter a result set to a specific set of objects. While this is possible, it is not a recommended approach. If this kind of flexibility is required in an API it could be a good time to consider the use of Open Data Protocol (OData).
• in cases where response filtering is used, providers should ensure that they use a JSON schema by default so that consumers have the ability to understand the entire resource and do not need to query the resource for the message structure.

1.6. HTTP headers

1.6.1. Request headers

1.6.2. Response headers

1.6.3. Customer X-HTTP headers

X-notation headers have been deprecated as per Request for Comments (RFC) 6648 and should not be used. This standard appreciates that X-notation headers are widely used however this guideline recommends that where an agency is defining a custom header the X-notation should not be used and the agency defines a custom header notation that is relevant. For example, X-Request-Id should be redefined as Request-Id.

RFC 6648 — Internet Engineering Task Force (IETF)

1.7. Returned content

The returned content is the body of the message that the API returns in response to a request. This usually relates to the resource that was requested.

1.7.1. Resource scope

The amount of data to return about a resource depends on several factors:

• Business context — does all the resource information being returned have relevance to the business process or activity consumers will use it for?
• Payload size and network efficiency — how much information does the resource contain, and is it so bulky it could impact network performance?
• Usefulness — is the information in the returned content always useful to the consumer, or is it something they may need once but never again?

Analysis of the full resource data against these factors should help determine how much resource data is returned in the response.

1.7.2. Formats

REST responses should be of a format that is both human and machine readable. Even if your initial consumer is via business to business (B2B), the API should be designed, where possible, for all potential consumers. REST APIs should, by default, return content in JSON format. But it is good practice to support multiple message formats (for example, XML, Multi-part Multipurpose Internet Mail Extensions [MIME]) and allow consuming applications to request the format they wish to consume (in line with the design for the consumer principle in section 2.1. Design for the consumer of Part A: API concepts and management 2022).

The response format required from a GET request must be indicated by the consumer in their request using the Accept header.

The request format for a request containing a request body (POST, PUT, PATCH) must be supplied in the Content-Type request header.

For a good example of this in action, see the tabbed format options for presenting patient information on the FHIR site.

Where JSON format is used in responses, it must conform to the JSON Standard (RFC 7159 — IETF). In line with this standard, textual content should be Unicode Transformation Format-8 (UTF-8) encoded by default, and any deviations from this must be limited.

It is inadvisable to use APIs for returning binary data such as images in the response content — hyperlinks to images are the preferred returned response. But where images are returned, ensure that the image encoding is MIME, HTTP Multipart.

Hypertext Markup Language (HTML) is allowed but is less flexible for consumer usage (for example, how can it be consumed by a native mobile app?), ties content to presentation and loses its benefits when building a single page application. It also reduces the ability to filter the content in transit for consumers who have fine-grained control on what they can access. However, it may be necessary for some forms of API like APIs with geospatial content.

Avoid using bespoke formats in returned content.

Recommended

Returned content format should be JSON by default.

Required

Requests for a specific return format must be defined in the Accept header.

1.7.3. Layout

Responses should be a JSON object (not an array) by default. Using an array to return results limits the ability to include metadata about results and limits the API’s ability to add additional top-level properties in the future. Do not use unpredictable properties. Parsing a JSON response where properties are unpredictable (for example, derived from data) is difficult, and adds friction for clients.

1.7.4. JSON property names

Within content, property names should conform to the following guidelines:

• Property names should be meaningful names with defined semantics.
• Property names must be camel-case American Standard Code for Information Interchange (ASCII) strings.
• The first character must be a letter or underscore (_).
• Subsequent characters can be a letter, a digit or underscore.
• Reserved JavaScript keywords should be avoided.

1.7.5. Consistency

Preserve backwards compatibility with existing consumers of the API by returning expected fields and employing sensible default values for missing fields. Keep consistency of terminology throughout, so that the consumer is not misled. Avoid modifying the semantics of content to new meanings, for example, do not change a ‘title’ field from being used for the title of a page to meaning a person’s job title.

1.7.6. Singletons versus collections

Some API GET calls will return a single resource item (for example, GET https://api.example.govt.nz/passengers/12345) whereas some calls will return a collection of resource items (for example, GET https://api.example.govt.nz/passengers), and the handling and error treatment can be different.

A single item will generally consist of all the pertinent fundamental details about the resource, and the GET will return these fundamental details.

If passenger 12345 did not exist, the HTTP return code would be 404.

If a requested sub resource did not exist — for example, addresses for passenger 54321: GET https://api.example.govt.nz/passengers/54321/addresses an HTTP 404 response would not be valid. The correct response would contain an empty array:

A collection will take the form of an array of the given resources, including any related metadata. The collection returned should be the complete set, leaving it to the consumer to reduce the quantity of information returned by sending filters in the resource request (see section 1.5.6. Query arguments).

The only reason the collection should be filtered by the API provider is based on the security context — the API consumer should only gain access to what that consumer is allowed to see. It is advisable to aim for consistent naming of collection resource fields, as this enables application developers to create generic handling of the data from across various resource collections offered by the API. Collection resources should not contain binary attachments or other content that will lead to large response payloads.

If there were no results from a query against a filter— for example, passengers booked on a specific flight (GET https://api.example.govt.nz/passengers/flights/1234), the return value would be a 200 response code, because the query is not invalid and there may be such data in the future.

1.7.7. HATEOAS

Hypermedia as the Engine of Application State (HATEOAS) is the principle of not just returning a response to a request, but also returning links to other useful capabilities. So, if a POST has just been used to create an item, the response could return links to options to update the item, delete the item and view other items of the same type. This makes it possible for user interfaces (UIs) that utilise an API to be automatically generated (in terms of hyperlinks) and adaptable to change (response content links).

If a web page effectively captures a single state in time, then hyperlinks are transitions to other states. The HATEOAS principle suggests that REST should return not only response content (current state) but also links to transitions to other states. This could result in thinner clients and fatter APIs, and means UIs need to be built to adapt to potential new transitions, but makes it possible to update all UIs no matter what platform they sit on (mobile app, web application).

For example, consider the following representation of a person and sequence of requests.

1. User submits a request for a person record:

   # Request person details:

   Example

   GET https://api.example.govt.nz/people/c70d37cf-3314-4a24-a2c3-0fca7ec0943f
   Host: www.example.com
   Accept: application/json;q=1.0

   # Response with person details:

   Example

   HTTP/1.1 200 OK
   Content-Location: https://api.example.govt.nz/people/c70d37cf-3314-4a24-a2c3-0fca7ec0943f
   Content-Type: application/json;charset=UTF-8
   Cache-control: max-age=3600,must-revalidate
   {
     "personId": "d9e1a2f6-6bc9-46af-9dbb-50de7f6eacd5",
     "name": "Joe Bloggs",
     "email": "joe.bloggs@example.com",
     "addresses": [
       {
         "id": "5046b8f2-e8aa-4ec9-a597-34d01af0fd32",
         "streetNumber": "12",
         "streetName": "Somewhere Street",
         "City": "Springfield",
         "type": "home"
       },
       {
         "id": "18a1a85c-3b1e-41b9-909c-668f2a803b69",
         "streetNumber": "1",
         "streetName": "Some Other Street",
         "City": "Springfield",
         "type": "business"
       }
     ],
     "_links": [
       {
         "rel": "self",
         "href": "https://api.example.govt.nz/people/d9e1a2f6-6bc9-46af-9dbb-50de7f6eacd5"
       },
       {
         "rel": "address1",
         "href": "https://api.example.govt.nz/addresses/046b8f2-e8aa-4ec9-a597-34d01af0fd32"
       },
       {
         "rel": "address2",
         "href": "https://api.example.govt.nz/addresses/18a1a85c-3b1e-41b9-909c-668f2a803b69"
       }
     ]
   }

2. The user then updates the address details changing the street number:

   # Request to update the address details:

   Example

   PUT https://api.example.govt.nz/addresses/d9e1a2f6-6bc9-46af-9dbb-50de7f6eacd5 HTTP/1.1
   Host: api.example.govt.nz
   Content-Type: application/json
   {
       "streetNumber": "12a",
       "streetName": "Somewhere Street",
       "City": "Springfield"
   }

   # Response that the update was successful:

   Example

   HTTP/1.1 204 No Content
   Location: https://api.example.govt.nz/addresses/046b8f2-e8aa-4ec9-a597-34d01af0fd32

   If the user were now to re-fetch the person details again as per step 1, the representation would contain a stale version of the address (street number 12 instead of 12a) because the address representation had been cached. This caching could occur anywhere along the network path (including public internet) between the server and client.

   Using hypermedia links for related resources improves the visibility of the API and avoids these types of problems. Consider the revised person representation below:

   # Response that the update was successful:

   Example

   {
     "personId": "d9e1a2f6-6bc9-46af-9dbb-50de7f6eacd5",
     "name": "JoeBloggs",
     "email": "joe.bloggs@example.com",
     "addresses": [
       {
         "rel": "self",
         "href": "https://api.example.govt.nz/addresses/5046b8f2-e8aa-4ec9-a597-34d01af0fd32"
       },
       {
         "rel": "self",
         "href": "https://api.example.govt.nz/addresses/18a1a85c-3b1e-41b9-909c-668f2a803b69"
       }
     ],
     "_links": [
       {
         "rel": "self",
         "href": "https://api.example.govt.nz/people/d9e1a2f6-6bc9-46af-9dbb-50de7f6eacd5"
       }
     ]
   }

Using this approach, the embedded object is replaced with a link to the ‘real’ object. Although this approach requires a larger number of network requests, it avoids the stale data problems and reduces payload size, which is an important consideration for APIs being used by mobile devices or over high latency networks.

1.8. Consideration of state

In almost all cases a REST API should be entirely stateless. As part of processing, it is possible for an API to gather context and pass this to a downstream system, however an API should not maintain that context for future requests. This should not be confused with caching, as discussed in section 1.12. Caching.

1.9. Batch handling and transactions

APIs are not designed for large payloads like batch handling for retrieving or uploading batches of data. APIs are geared towards stateless, usually synchronous, web-like requests for individual discrete data transactions. However, batch handling can be achieved through bundling multiple calls to the same API. This helps achieve atomicity of transactions and aids recoverability in event of errors.

When handling transactions, it is important to consider the troubleshooting and recovery aspects of error handling. This includes visibility of transaction progress and the ability to perform root cause analysis. To achieve this, it is advisable that logging is performed on all transactions coming through an API, with accurate timestamping, so that monitoring tools can visualise transaction progress. It also requires transaction identifiers (see section 1.6.1. Request headers) to be built into transactional API calls to ensure the transaction is traceable end to end.

In some cases, it may be deemed appropriate to provide an asynchronous batch type capability using an API. This would usually be implemented in a scenario where legacy system impact is a concern. An example of this could be a bulk creation of person records in a database based on a batch event in a consuming legacy application.

In such a scenario it would be preferable for the consuming application to treat each person as a unique event and to POST to a person API for each new unique resource. While this may seem like an unnecessary overhead, by individualising the transactions each success or error scenario can be handled in its own right, and the consumer can be informed of their transaction status.

If this is not possible, due to some system restriction, it is possible to asynchronously POST multiple person details to an API. This type of interaction should not be attempted synchronously as large batches will tie up HTTP threads and in some cases require client / server timeout control.

1.10. API version control

Learn more about the commonly-used types of versioning when developing an API.

1.10.1. API version control methods

There are 2 main API versioning methodologies commonly used. There are positives and negatives to both approaches and a large amount of debate over which is the most ‘RESTful’. Rather than stipulate a methodology to use in these guidelines, each methodology is described below in order of preference. Both, however, are acceptable. What is more important is that APIs are versioned and that there is an understanding of when and why to version an API.

1. Accept header versioning

This is usually considered the most RESTful way to version APIs because the resource path remains ‘pure’ and it is possible to provide more version flexibility for clients. It is, however, technically more difficult to implement and in many cases commercial API management / gateway products do not support or work well with this approach.

Header-based versioning should be performed using the Accept header where a consuming application requests an API version as defined in an Accept header. Wildcards (*) are used by the consuming application to indicate acceptance of the latest major or minor version of an API.

Example

#Get details for provider id 123435 – latest minor version of the API
GET https://api.example.govt.nz/providers/12345
Accept: application/json, version=1.*
Host: api.example.govt.nz

#Get details for provider id 123435 – version 1.1 of the API
GET https://api.example.govt.nz/providers/12345
Accept: application/json, version=1.1
Host: api.example.govt.nz

#Get details for provider id 123435 – latest version of the API
GET https://api.example.govt.nz/providers/12345
Accept: application/json, version=*
Host: api.example.govt.nz

The response should include the version of the API that responded to the client request in the ‘Content-Type’ header.

Example

Content-Type: application/json,version=1.2

2. URI (path) versioning

URL-based versioning is a viable alternative, as the version number in the URL should only change when major revisions have been made and the interface has changed substantially without backwards compatibility. For URL-based versioning the URI should include /vN with the major version (N) and v as a prefix. Agencies should not include minor version numbers when using version numbers in the API paths.

This is probably the easiest method to implement, but a number of REST purists feel that it compromises the URI and does not offer enough flexibility to consumers. If there is a requirement to support a large number of historical versions of APIs it can mean that there is complexity introduced with the number of URIs to maintain.

Example

#Query version of version 1 of the search API
GET https://api.example.govt.nz/customers/v1/search?first-name=John&last-name=Smith&page-size=10

#Query version 2 of the search API
GET https://api.example.govt.nz/customers/v2/search?first-name=John&last-name=Smith&page-size=10

The response should still indicate the version of the API that was called. This can be done as in the example above, using the Content-Type header or in the Location header, as the version in the path indicates the API that was called.

1.10.2. When to version

Simply put, an API should be versioned when a change is considered a breaking change. One of the benefits of an API is that, if it is well designed, there should be fewer breaking changes. In government however, there are likely to be situations where legislative changes enforce a new version of an API and deprecation of all previous versions.

1.10.2.1. Breaking changes

A change is a breaking change if any consuming application requires changes to work with the new version. In other words, the new version will not successfully process messages provided by existing consumers. A breaking change should be considered as a major version change, for example, 1.3 to 2.0.

Examples of breaking changes include:

• the removal of any property from the response representation
• the change of datatype for an existing property or a change from optional to required
• the removal of any resource or HTTP Verb support
• a change to the way errors are handled
• any change to existing resource URIs.

1.10.2.2. Non-breaking changes

A change is a non-breaking change if any message that would have been processed by the previous version will be successfully processed by the new version (that is, backwards compatible). This will enable an existing consumer of the previous version to work with the new version without requiring modification. A non-breaking change should be considered as a minor version change, for example, 1.1 to 1.2.

Such changes include:

• the addition of new properties to the JSON representation
• the addition of new resources
• the addition of support for new HTTP Verbs (new operations) on existing resources
• support for new custom headers, for example, for request tracing.

1.10.3. Software configuration management

It is important to remember that version control is more than just versioning the resource. An API will inherently have associated code and artefacts. Consider what comprises an API and include these as a logical artefact stored and managed in a software configuration management (SCM) system.

It is a good idea to try and capture all components in a format that can be controlled by SCM. A good example of this is Unified Modeling Language (UML) diagrams — these cannot be version-controlled unless captured in a text-like format. For examples of textual UML modelling, see: 

• Web Sequence Diagrams
• PlantText UML editor — PlantText

An API artefact should, where possible, comprise the following elements:

• any API code
• API specification (OpenAPI)
• unit test scripts such as JUnit or Mocha
• continuous integration support files such as grunt.js or Ant scripts.

1.11. Search

Search should be implemented using GET requests with filters provided as query string parameters.

Search can be used by API consumers to filter or sort information they want from an API.

For instance, ‘GET /groups/search?status=active’ would enable the consumer to filter the groups resource on status.

Alternatively, ‘GET /groups/search?status=active&sort=status,name’ enables the consumer to return active groups ordered by status and name. It can also be useful to allow the consumer to choose which fields they want back from the API, so they only receive the content they need in the API response.

In rare circumstances it is possible that an extremely complex search API may run into query string or URL length limitations, although this is probably an indication that the design of the search operation is too complicated. The exact length limit is server- and client-dependent. For example, the Apache server has a default limit of 8KB, Microsoft Information Internet Services (IIS) has a default of 16KB, and many browsers are limited to around 2KB maximum URL length.

Where the total length can potentially exceed 2KB, the only option is to implement the search operation using the POST verb with an appropriately designed request object to contain the search parameters.

Other considerations include special characters or filters that add additional complexity due to URL encoding requirements. For example, the search function might need to support ‘<’ or ‘>’ operators, or special characters for non-English language support.

Additionally, SQL words should be avoided, as common threat detection filters may block requests containing words such as ‘DROP’, ‘ALTER’ or ‘DELETE’ in case they are SQL injection attacks. For more details, see the following Open Web Application Security Project (OWASP) cheat sheets:

• Cross-Site Scripting (XSS) Prevention Cheat Sheet — OWASP
• SQL Injection Prevention Cheat Sheet — OWASP

Paging behaviour for search results should be consistent with the interaction as described in section 1.5.6. Query arguments. Pagination can be implemented as a page number and page size, offset and limit, or continuation token, depending on the scale and changeability of the search results.

1.12. Caching

Caching enables faster responses from APIs and reduces server load. It is good for information that is frequently requested but does not change very often. The aim is to retrieve information once but re-use many times.

There are a number of caching methodologies available. This section focuses on 2 types of caching that should be owned and controlled by the provider, primarily for APIs.

1.12.1. Response cache

A response cache is a cache that contains the response to a GET request on a unique resource. The response should be cached as close to the consumer as possible while remaining inside the control boundaries of the agency. Response caches are usually fast, in-memory caches. Many off-the-shelf API gateway / management solutions provide a built-in response cache.

When using a response cache, agencies should ensure that they understand and monitor their cache to ensure that stale objects are kept to a minimum and that they have sufficient system memory to service caching loads. This means ensuring that the cache is refreshed once a user has updated the information or a cache timeout has occurred.

Consider the example in figure 3 of a response cache.

Detailed description of figure

Figure 3 illustrates a caching sequence diagram in a Unified Modeling Language (UML) sequence diagram showing interactions between different actors in the use case. The sequence is divided into the following 6 sections and associated steps:

Section 1 — User1 first GET request

1. User1 sends GET customers / 1234 to API gateway.
2. API gateway sends check cache for key ‘cachedid_customers_1234’ to response cache.
3. Response cache sends cache miss to API gateway.
4. API gateway sends process request to providing application.
5. Providing application retrieves data from agency data.
6. Providing application sends response to API gateway.
7. API gateway sends cache response with key ‘cacheid_customers_1234’ to response cache.
   The response cache object time to live is 1,800 seconds or 30 minutes.
8. API gateway sends response to User1.

Section 2 (happens 5 minutes later) — User1 second GET request

In this case, response is cached and valid.

9. User1 sends GET / customers / 1234 to API gateway.
10. API gateway sends check cache for key ‘cachedid_customers_1234’ to response cache.
11. Response cache sends cache hit to API gateway.
12. API gateway sends response to User1.

Section 3 — User2 first GET request

In this case, response is cached and valid.

13. User2 sends GET / customers / 1234 to API gateway
14. API gateway sends check cache for key ‘cachedid_customers_1234’ to response cache
15. Response cache sends cache hit to API gateway
16. API gateway sends response to User2

Section 4 — User1 first PUT request

In this case the response is cached and valid, request is an update, API must invalidate cache.

17. User1 sends PUT / customers / 1234 to API gateway.
18. API gateway sends process request to providing application.
19. Providing application updates data to agency data.
20. Providing application sends response to API gateway.
21. API gateway sends check cache for key ‘cachedid_customers_1234’ to response cache.
22. Response cache sends cache hit to API gateway.
23. API gateway sends invalidate cache to response cache.
24. API gateway sends response to User1.

Section 5 — User1 third GET request

25. User1 sends GET / customers / 1234 to API gateway.
26. API gateway sends check cache for key ‘cachedid_customers_1234’ to response cache.
27. Response cache sends cache miss to API gateway.
28. API gateway sends process request to providing application.
29. Providing application retrieves data from agency data.
30. Providing application sends response with updated data to API gateway.
31. API gateway sends cache response with key ‘cacheid_customers_1234’ to response cache.
    The response cache object time to live is 1,800 seconds or 30 minutes.
32. API gateway sends response with updated data to User1.

Section 6 (happens 45 minutes later) — User2 second GET request

33. User2 sends GET customers / 1234 to API gateway.
34. API gateway sends check cache for key ‘cachedid_customers_1234’ to response cache.
35. Response cache sends cache miss to API gateway.
    Cache object expired.
36. API gateway sends process request to providing application.
37. Providing application retrieves data from agency data.
38. Providing application sends response with updated data to API gateway.
39. API gateway sends cache response with key ‘cacheid_customers_1234’ to response cache.
    The resource is re-cached with a time to live of 1,800 seconds or 30 minutes.
40. API gateway sends response with updated data to User2.

1.12.2. Object cache

An object cache is used to cache objects that are fundamental to the function of an API but may not change on a regular basis. For example, an API may include a tax code validation step. Tax code references are held in a backend database. To prevent redundant requests to the database, the API layer could cache tax codes in its object cache and refer to these until the cache becomes invalid. Figure 4 details an example sequence.

Detailed description of figure

Figure 4 illustrates an object sequence diagram in a Unified Modeling Language (UML) sequence diagram showing interactions between different actors in the use case. The sequence is divided into the following 4 sections and associated steps:

Section 1 — User PUT request

1. User sends PUT / returns / paye / 1234 to API.
2. API validates tax code.
3. API checks object cache for key ‘cacheid_tax_codes’.
4. Object cache sends cache miss to API.
5. API retrieves tax codes for object cache from agency data.
6. API populates object cache with key ‘cached_tax_codes’.
   Tax codes cache object time to live is 36,000 seconds or 10 hours.
7. API checks tax code in object cache.
8. API sends process request to providing application.
9. Providing application updates agency data.
10. Providing application sends response to API.
11. API sends response to User.

Section 2 (happens 5 minutes later) — User PUT request

12. User sends PUT /returns / GST / 9876 to API.
13. API validates tax code.
14. API checks object cache for key ‘cacheid_tax_codes’.
15. Object cache returns cache hit to API.
    Cache object is cached and valid.
16. API checks tax code in object cache.
17. API sends process request to providing application.
18. Providing application updates agency data.
19. Providing application sends response to API.
20. API sends response to User.

Section 3 — Tax codes updated in backend database

21. Agency administrator updates tax codes in agency data.
22. Agency data sends cache refresh request to API.
23. API updates object cache with key ‘cached_tax_codes’.
    Tax codes object time to live is 36,000 seconds or 10 hours.

Section 4 (happens 11 hours later) — User PUT request

24. User sends PUT / returns / corporation / 7654 to API.
25. API validates tax code.
26. API checks object cache for key ‘cacheid_tax_codes’.
27. Object cache sends cache miss to API.
    Tax codes cache object expired.
28. API retrieves tax codes for object cache from agency data.
29. API populates object cache with key ‘cached_tax_codes’.
    Tax codes object time to live is 36,000 seconds or 10 hours.
30. API checks tax code in object cache.
31. API sends process request to providing application.
32. Providing application updates agency data.
33. Providing application sends response to API.
34. API sends response to User.

1.13. Error handling

Learn more about how important errors in the development of API are being handled.

Error handling is important because API consumers see the API as a black box, and when an error occurs they need to know how to handle it. Therefore error responses need to be informative yet avoid information leakage about the internals of the backend system. Errors should be handled in both a human- and machine-consumable way. When an error occurs, the response body should contain:

• the HTTP status code
• an API-specific error code, which API support staff will be able to look up to identify what has gone wrong
• a human readable error message (possibly including selective technical details if the API is a development / test release for developer consumption only).

1.13.1. HTTP status codes

RESTful APIs should always use the standard HTTP error responses when an error occurs, but different codes may be pertinent depending on the HTTP verb being used and the quantity of data being retrieved (for example, single item versus list / collection of data).

For a tutorial, see HTTP Status Codes — REST API Tutorial

The following table gives an example of good practice use of HTTP error codes.

1.13.2. API-specific error code

Error responses need to supply enough information to allow the API consumer to react appropriately to the error, yet not give away too much. But when a critical error occurs it is important for the API provider to be able to trace the root cause and fix it as soon as possible.

One way of achieving this is to return an API-specific error code in the response to the consumer. When the consumer reports the error to the API support team, the consumer can relate the associated API-specific code. The support team can then look this code up and ascertain exactly what has gone wrong and who needs to address it.

1.13.3. Human-readable error message

The human readable error message should be as informative as is useful to an end customer, without offering too many technical details (for example, “An account with this ID already exists.”).

It is important to avoid revealing system information in the human readable response, such as composition of the backend system like component names, as this informs malicious consumers as to what vulnerabilities and back doors to look for. It is also important not to confirm or deny sensitive information, such as username in a username / password combination, as this informs potential attackers which criteria they have got correct.

A07: 2022 Identification and Authentication Failures — OWASP

In non-production instances of the API it may be appropriate to offer more verbose error messages with some technical detail (for example, “Body should be a JSON object.”). However, it is still recommended not to reveal too much about the internal systems that underpin the API, as non-production systems still reflect a lot of the composition of production.

2. API governance

API governance helps save time and money because it ensures that APIs are built proactively to achieve specific goals and bring value to the agency.

API governance also helps agencies make intelligent decisions regarding API programmes and establish best practices for building, deploying and consuming APIs.

API governance should include:

• specification and resource definitions — governing contracts and resource definitions help improve the consistency and reusability of APIs
• style and pattern guidelines — standardising API design ensures APIs remain consistent and improve usability
• automation — establishing guardrails to make it easier to comply with policies than not (for example, API scaffolding that automatically generates security and audit componentry for APIs or build pipelines that enforce security policies, such as coding standards and security scans)
• life cycle — governing the life cycle of an API, from publication to versioning and deprecation to retirement
• tracking and analytics — keeping track of where APIs are deployed, who is using them and how they are being used (this information helps inform other aspects of API governance)
• a robust code review process — ensuring that your development teams understand the concepts laid out in these guidelines, and establishing a practice where collaborative code reviews are performed (it is recommended that software is developed using a branch strategy with at least one peer review required prior to code merge and deployment).

When your API designers and developers are on the same page regarding APIs, the more efficient, valuable and successful your API programmes will be. Governance is especially beneficial for agencies that have a large API programme or microservices architectures.

For more information, see API Governance: How Important Is It for API Strategy? — NORDIC APIs.

3. Appendix A — HTTP verbs — Required

3.1. GET

GET is used:

• for retrieval of information
• if the interaction is more like a question, that is, it is a safe operation such as a query.

The HTTP GET method is used to read (or retrieve) a representation of a resource. It can be used to retrieve information about an individual resource (for example, ‘customers/123456’) or to retrieve a collection (list) of resources (for example, customers).

In the ‘happy’ (non-error) path, GET returns a representation in JSON and an HTTP response code of 200 (OK). In an error case, it most often returns a 404 (NOT FOUND) or 400 (BAD REQUEST) if the requested resource does not exist.

According to the HTTP specification, GET (along with HEAD) requests are used only to read data and not change it. Therefore, they are considered safe when used this way. That is, they can be called without risk of data modification or corruption — calling it once has the same effect as calling it 10 times. GET is idempotent, which means that making multiple identical requests ends up having the same result as a single request. Do not expose unsafe operations via GET — it should never modify any resources on the server.

3.1.1.Examples

Some API GET calls will return a single resource item while some calls will return a collection of resource items. See section 1.7.6. Singletons versus collections for more details.

3.2. POST

POST is used:

• to create a piece of information or resource item
• to spawn an action
• if the interaction is like placing an order
• if the interaction changes the state of the resource in a way that the user would perceive (for example, a subscription to a service)
• if the user is held accountable for the results of the interaction.

The POST verb is most often utilised to create new resources. In particular, it is used to create sub-resources, that is, subordinate to some other (for example, parent) resource. When creating a new resource by a POST to the parent, the API takes care of associating the new resource with the parent, assigning an ID (new resource URI) and so on. On successful creation, POST returns an HTTP status 201 (CREATED), with a Location header containing a link to the newly created resource.

POST is neither safe nor idempotent. It is therefore recommended for non-idempotent resource requests. Making 2 identical POST requests will most likely result in 2 resources containing the same information.

The body content of a POST request will differ to that of a PUT, as the provider will create certain values such as a unique identifier for the resource and its creation date.

In many cases a POST may have a small subset of the data that can exist on a resource — a subsequent PUT to the resource can be performed to populate the data.

3.2.1. Examples

3.3. PUT

PUT is used to :

• update an existing resource item
• replace an existing resource item.

PUT is most often utilised to update information, PUT-ing to a known resource URI with the request body containing the newly updated representation of the original resource.

However, PUT can also be used to create a resource in the case where the resource ID is chosen by the consumer instead of by the API itself: where PUT is to a URI that contains the value of a non-existent resource ID. Again, the request body contains a resource representation. This method of creation should be avoided, if possible, or used sparingly. If consumer-defined resource IDs are required, use POST to create the new resource and provide the consumer-defined ID in the body representation (see section 3.2. POST).

On a successful update, PUT returns HTTP status 200 (or 204 if not returning any content in the body). If using PUT to create, PUT returns HTTP status 201 on successful creation. Content in the body in the response is optional. It is not necessary to return a link via a Location header in the creation case since the consumer already knows the resource ID, having set it themselves.

PUT is not a safe operation — it modifies (or creates) state on the server, but it is idempotent. That is, if you create or update a resource using PUT and then make that same call again, the resource is still there and still has the same state as it did with the first call. However, were the PUT call to increment a counter within the resource, the call is no longer idempotent. Sometimes this is necessary behaviour, and it may be enough to document that the call is not idempotent. However, it’s recommended to keep PUT requests idempotent. It is strongly recommended to use POST for non-idempotent requests.

It is good practice to perform a GET on a resource before you perform a PUT. This means that the consuming application has the latest representation of that resource. PUT should contain the entire resource content in the message body. By performing a GET prior to the PUT the consuming application can re-send the results of the GET with any modifications.

When using a resource response cache, any PUT to a resource should invalidate that cache or re-populate the cache with the updated resource.

3.3.1. Examples

3.4. DELETE

DELETE is used to delete a resource item.

DELETE is only used to delete a resource identified by a URI.

On successful deletion, DELETE returns an HTTP status 200 (OK) along with a response body, and optionally the representation of the deleted item, or a wrapped response. Alternatively, DELETE can return HTTP status 204 (NO CONTENT) with no response body.

DELETE operations are idempotent. If you DELETE a resource, it is removed. Repeatedly calling DELETE on that resource ends up as the same result — the resource is gone. However, calling DELETE on a resource a second time will often return a 404 (NOT FOUND) since it was already removed and therefore is no longer findable. This, in some opinion, makes DELETE operations no longer idempotent, but the end state of the resource is the same. Returning a 404 is acceptable and communicates accurately the status of the call. If calling DELETE were to decrement a counter (within the resource), the DELETE call is no longer idempotent.

3.4.1. Examples

3.5. OPTIONS

OPTIONS is used to retrieve information about what the consumer is allowed to do with the resource.

3.6. Other

PATCH is a valid HTTP verb but is still in draft and its use is discouraged due to complexity. An exception to this might be where an agency-adopted standard, such as Restful API — FHIR , supports the use of PATCH.

HEAD is used to retrieve metadata about a resource, such as when it was last updated or the associated response headers. The response to a HEAD request must not contain a body. If a response body is returned it must be ignored.

4. Appendix B — HTTP headers

4.1. Request headers — Required

Request headers are supplied by the consuming application and must include the following elements:

• Accept
• Content-Type
• Authorization.

Header definitions can be found in Header Field Definitions: RFC 2616 — W3C.

4.1.1. Accept — Required

An Accept header is required to indicate what format the consuming application wants the information to be returned as (for example, JSON or XML). It is preferable for the consuming application to specify the response content type that they expect, however it is acceptable to use the ‘q’ parameter to specify the quality of support.

If the client has specified via the Accept header that it does not support any formats provided by the API, the server should return an HTTP 406 (Not Acceptable) response. The response should also include a Link header with a link to documentation or help on supported formats. The body of the representation should include an error message in human readable plain text or HTML.

The Accept header is required, however the ‘version’ and ‘q’ are recommended as this will depend on a provider’s implementation and in some cases technology choice.

4.1.2. Content-Type — Required

The Content-type header is required for all requests that include a request body like POST, PUT, DELETE.

4.1.3. Authorization

Most API requests will be authorised. The Authorization header should be used for this purpose and no other. Note that the Authorization header is not the best place for an API key.

The Authorization header is comprised of a type and a token depending on the authorization type.

4.1.4. Accept-Encoding — Recommended

In some cases, especially where API responses are very large, it may be worth compressing HTTP responses.

An Accept-Encoding value of gzip instructs a provider that supports HTTP compression that the consuming application supports compressed responses.

4.1.5. API key — Required

An API key header issued to a consuming application should be sent with every request made to the API. The name of the header is up to the agency, but it should not be an X- prefixed header as this use is deprecated. Sometimes API keys are passed in URIs, however, this is not considered best practice and should be avoided.

4.1.6. If-Modified-Since / If-None-Match — Recommended

The If-Modified-Since header is a pre-condition header used to instruct a client on the validity of the resource requested. Note that pre-condition handling is not required, but should be considered where appropriate. Pre-condition control can be a complex topic so in some cases the complexity may eclipse the value.

Conditional header standards are available in Hypertext Transfer Protocol (HTTP/1.1): Conditional Requests RFC 7232 — IETF.

A consuming application can specify an If-Modified-Since header with a date and time. The server checks the resource to see the last modified time. If the last modification was before the date / time specified in the header, the server will respond with an HTTP 304 (NOT MODIFIED) status code telling the consuming application that the resource they requested has not changed and therefore there is no need to return the full payload in the response.

When using If-Modified-Since headers you should also implement ETag as discussed in section 1.6. HTTP headers.

The If-None-Match header indicates to the provider the ETag for the resource that it currently holds for that resource.

4.1.7. Request tracking headers — Recommended

To support the tracking of API requests it is a good idea to apply a transaction ID header that will enable analysts to trace a request throughout its life cycle.

There are 2 common approaches to this:

1. To request that the consuming application generate a unique transaction ID and the provider will capture it and respond with a corresponding correlation ID.
2. To allow the provider to generate the transaction ID as close to the edge of its domain as possible, and respond to the consuming application with that transaction ID. In this case a consuming application administrator or developer should capture and log that transaction ID for audit purposes so that it can be relayed to agency analysts if required.

4.2. Response headers

Response headers are supplied by the provider and must contain Content-Type.

4.2.1. Content-Type — Required

The Content-Type must be returned in the response, and indicate the format type the response content is in, for example, ‘Content-Type: application/json’. The Content-Type header should also include the version of the API that processed the request, for example, ‘Content-Type: application/json,version=1.1’.

4.2.2. Location — Recommended

Where a new resource item has been created (POST), the server should respond with a 201 (Created) HTTP header and a Location header indicating the URI of the newly created resource item, for example, ‘Location: https://api.example.govt.nz/v1/crown-property/198373’.

A Location header should also be returned by a server when it sends a 3** HTTP response, indicating a temporary or permanent redirection.

A Location header can be, as recommended by this standard, absolute, for example, ‘Location: https://api.example.govt.nz/v1/crown-property/198373’ or relative, for example, ‘Location: /v1/crown-property/198373’.

4.2.3. Content-Location — Recommended

The Content-Location header indicates the location of the requested resource. It should be an absolute and not a relative link.

4.2.4. Cache-Control — Recommended

It is important to consider caching when designing APIs, to ensure that the APIs you build can perform at scale. The Cache-Control header is only concerned with caching that occurs externally to the API layer. Common examples may be client-side caching or caching within a Content Delivery Network (CDN). This header should not be concerned with caching mechanisms within the API layer such as response caches or object caches, which require a different caching strategy.

The Cache-Control header is comprised of the following:

• private|public — indicates if the cache is specific to the consumer or not. A setting of private will prevent the cache from being persisted in a public cache.
• no-cache — instructs a cache to revalidate the resource every time.
• no-store — instructs a cache not to store the response. It also ensures that no part of the request is stored too.
• max-age — instructs the cache on expiry time. This can be used but it makes more sense to use the Expires header discussed in section 4.2.5. Expires.
• s-max-age — similar to max-age, however it refers to a shared cache therefore only relevant to CDNs and other intermediary caches.
• must-revalidate — instructs the client to resend request headers and receive confirmation that the resource has not changed.
• proxy-revalidate — similar to must-revalidate, however specific to shared caches.

Care should be taken when using caches, especially shared or public caches. In some cases it may be more prudent to use the no-cache directive to ensure that content is not cached outside your network. This does not rule out caching within your API layer. For more information, see section 1.12. Caching.

HTTP Cache Control standards can be found in Hypertext Transfer Protocol (HTTP/1.1): Caching RFC 7234 — IETF.

4.2.5. Expires — Recommended

When using Cache-Control you should include an Expires header in the response for backward compatibility.

4.2.6. ETag (Entity Tag) — Recommended

The ETag header is a concurrency control header, and when in a response it is a tag that the consuming application can use when making a subsequent request to a resource.

Note that pre-condition handling is not required, but should be considered where appropriate. Pre-condition control can be a complex topic so in some cases the complexity may eclipse the value.

4.3. Custom X-HTTP headers

X- notation headers have been deprecated and should not be used.

Deprecating the “X-” Prefix and Similar Constructs in Application Protocols RFC 6648 — IETF

This is because the X- name often became a standard, making the X-notation inaccurate (for example, x-gzip became gzip) and requiring support for both the X- and the non-X- notation. Common uses of the X- headers are examples such as the X-HTTP-Method-Override header or the X-Forwarded-For header.

Where an agency feels that a custom HTTP header is required that is not available within the current HTTP standard they can use a custom header without the X- prefix.

5. Appendix C — Example interface specification

Figures 5 to 8 illustrate examples of an agency API interface. The examples are:

• Create customer
• Get customer
• Modify customer
• Delete customer.

Following the figures is the interface specification written in Swagger and visualised via a Confluence plugin, however most API developer portal products will support importing OpenAPI specifications.

The raw JSON, YAML and Swagger code is below. This is best viewed via the Swagger (OpenAPI) editor. Simply open the editor and paste the contents of the JSON, YAML and Swagger examples into the editor window.

6. Appendix D — Synchronous / asynchronous example specifications

6.1. OpenAPI 3.0

To visualise, paste the content below into the OpenAPI editor SwaggerEditor.

6.2. AsyncAPI 2.0

To visualise, paste the content below into the AsyncAPI playground AsyncAPI 2.0.0 playground.

7. Appendix E — Standards for developing APIs

Learn more about the relevant standards for developing APIs.

In addition to the security standards captured in Part B: API security 2022, table 7 captures (current) API and web standards that should be considered as part of any API strategy.

For recommended standards and their current status, see the Government Digital Standards Catalogue.

8. Glossary of terms

Analytics

Analytics in the context of these guidelines is the capturing and reporting of API usage.

API (Application Programming Interface)

An API is a piece of software that provides a way for other disparate pieces of software (applications, systems) to talk to one another.

API catalogue

The API delivery component that lists the APIs offered, along with their interface specification and guidance on how to gain access and use the APIs.

API developer

The organisation (or person) who creates the API and maintains the interface specification for the API. An API developer creates and documents an agency’s APIs. API developers generally have a good understanding of an agency’s function.

API developer portal

The API delivery component that allows API providers to engage with, onboard, educate and manage application developers whether inside or outside the organisation. These management activities will generally include registration, documentation, analytics and so on.

API gateway

The API delivery component that allows API providers to offer APIs to the outside world. This component (physical or virtual) hosts the APIs ready for consumers to access. It provides an API provider with the ability to control who has access to their APIs by enforcing policies for access control. Some API gateways also offer additional capabilities.

API manager

The API delivery component that allows API providers to control an API’s visibility and behaviour. It is usually exposed as a UI / console to internal staff only, as it is seen as an administration component. It offers a variety of capabilities, including API registration and catalogue administration.

API provider

The organisation that provides the API to expose a resource (information or functionality) for consumers.

Application

The software behind the API that provides the business logic for the resource.

Application developer

Software developer or organisation that builds consuming applications that use the API. An application developer needs to be able to discover, understand and access your APIs. They can be internal to your agency, developers that work with trusted partners, developers from other agencies or developers from the private sector.

Atomicity

One of the ACID (Atomicity, Consistency, Isolation, Durability) transaction properties. An atomic transaction is an indivisible and irreducible series of database operations such that either all occurs, or nothing occurs. A guarantee of atomicity prevents updates to the database occurring only partially, which can cause greater problems than rejecting the whole series outright. As a consequence, the transaction cannot be observed to be in progress by another database client. At one moment in time, it has not yet happened, and at the next it has already occurred in whole (or nothing happened if the transaction was cancelled in progress).

Consumers

Customers, consuming applications and application developers who use the API.

Consuming application

Any application (on any device) that consumes or uses an API.

Context

Context in these guidelines generally refers to request context. For example, a JWT (JSON Web Token) may contain information about the customer initiating an API request.

Customers

People (or organisations) that use the consuming applications to access the API resources the API provider is offering.

DELETE

One of the most common HTTP methods, DELETE is used to delete a resource specified by its URI.

Discovery

The ability for application developers to find resources and associated APIs to use in their consuming applications.

GET

An HTTP request method, GET is applied while requesting information from a particular source. It is also used to get a specific variable derived from a group.

HEAD

An HTTP request method, HEAD asks for a response identical to a GET request, but without the response body. It is used to send the request and retrieve just the HTTP header as response. For example, a client application can issue a HEAD request to check the size of a file (from HTTP headers) without downloading it.

Interface specification

Provides technical information to the application developer community about the API. It includes information about how the API works, any access control features and any error conditions.

Idempotent

An HTTP method where making multiple identical requests ends up having the same result as a single request.

PATCH

An HTTP request method, PATCH is a request for making partial changes to an existing resource. The PATCH method provides an entity containing a list of changes to be applied to the resource requested using the URI.

POST

One of the most common HTTP methods for retrieving from and sending data to a server, the POST method sends data to the server and creates a new resource. The resource it creates is subordinate to some other parent resource. When a new resource is POSTed to the parent, the API service will automatically associate the new resource by assigning it a new resource URI. In short, this method is used to create a new data entry.

Publish

The act of releasing the interface specification and associated API to a location accessible by application developers.

PUT

One of the most common HTTP methods for retrieving from and sending data to a server, the PUT method is most often used to update an existing resource. If you want to update a specific resource (which comes with a specific URI), you can call the PUT method to that resource URI with the request body containing the complete new version of the resource you are trying to update.

Resource

The information or functionality exposed by the API.

State

State defines the point in time record of an in-flight transaction. Some systems maintain ‘user state’ for a period of time to enable a transaction to be continued from the point of last recorded state. APIs are usually, but not always, considered stateless.

9. Glossary of acronyms

API

Application Programming Interface

ASCII

American Standard Code for Information Interchange

B2B

Business to business

CA

Certificate Authority

CDN

Content Delivery Network

CRUD

Create, Read, Update and Delete

DDoS

Distributed Denial of Service

EDA

Event-Driven Architectures

ETag

Entity Tag

FAPI

Financial Grade API

FHIR

Fast Healthcare Interoperability Resources

GEAG

Government Enterprise Architecture Group

HATEOAS

Hypermedia As The Engine Of Application State

HISO

Health Information Standards Organisation

HTML

Hypertext Markup Language

HTTP

Hyper Text Transfer Protocol

IETF

Internet Engineering Taskforce

IIS

Information Internet Services

JSON

Javascript Object Notation

JWA

JSON Web Algorithms

JWE

JSON Web Encryption

JWK

JSON Web Key

JWS

JSON Web Signature

JWT

JSON Web Token

MIME

Multipurpose Internet Mail Extensions

OAuth

Open Authorization

OData

Open Data Protocol

OIDC

OpenID Connect

OWASP

Open Web Application Security Project

Peppol

Pan European Public Procurement Online

PNZ

Payments New Zealand

RAML

REST API Modelng Language

REST

Representative State Transfer

RFC

Request for Comments

SCM

Software Configuration Management

SDLC

Software Development Life Cycle

SEO

Search Engine Optimisation

SLA

Service Level Agreement

SOAP

Simple Object Access Protocol

SQL

Structured Query Language

UI

User interface

UML

Unified Modeling Language

URI

Uniform Resource Identifier

UTF-8

Unicode Transformation Format-8

WADL

Web API Description Language

XML

eXtensible Markup Language

XSS

Cross-Site Scripting

YAML

YAML Ain’t Markup Language

10. Further reading

• HTTP 1.1 Standards RFCs — IETF:
  • RFC7230 HTTP/1.1: Message Syntax and Routing
  • RFC7231 HTTP/1.1: Semantics and Content
  • RFC7232 HTTP/1.1: Conditional Requests
  • RFC7233 HTTP/1.1: Range Requests
  • RFC7234 HTTP/1.1: Caching
  • RFC7235 HTTP/1.1: Authentication
  • RFC7236 HTTP/1.1: Authentication Scheme Registrations
  • RFC7237 HTTP/1.1: Method Registrations
• NZ Protective Security — Protective Security Requirement (PSR)
• OpenAPI Specification — GitHub
• OWASP API Security Project — OWASP
• OWASP REST Security — REST Security Cheat Sheet
• OWASP Secure Coding Principles — OWASP
• OWASP Top Ten Project — OWASP
• Reserved JavaScript Keywords — W3Schools
• REST API Resource Modelling — Thoughtworks
• Strategy for a Digital Public Service
• Using HTTP Methods for RESTful Services — REST API Tutorial