TL;DR
This blog post is a collaboration between the Service Innovation Lab, the Family Services Directory team from the Ministry of Social Development (MSD), the SmartStart team in the Department of Internal Affairs (DIA) and Catalyst IT (the SmartStart developer). Our many thanks also to the data.govt.nz team for their support in making this success story. Please see the new SmartStart “Services near me” functionality for the first use case of our first reusable component from the Service Innovation Lab, a publicly available family and social services register.
The Approach: Family Services Directory Team
In mid-2017 the Family Services Directory (FSD) team was asked if they were open to providing data to deliver information on nearby services for the SmartStart life event service. The SmartStart team from DIA were collaborating with the Service Innovation Lab to meet a user need around identifying relevant family and social services when having a child, and our FSD Directory was a great fit given it has over 20,000 family and social services, which are updated reasonably regularly by service providers across New Zealand.
The Family Services Directory has been making the published provider and service data available in several formats since 2014 when a “Build your Community Directory“ feature was added to the Directory. Initially this was in Printable Document Format (PDF) and was later complemented by the addition of a Rich Text Format (RTF) option. In 2016 FSD made the public data available as “Regional data sets” in an Excel format. The request to collaborate with the Lab to support SmartStart was seen as a natural progression of the pragmatic approach we had been taking to make the FSD data available for the benefit of New Zealanders, and we could see real opportunities for others to build upon the data through the use of Application Programme Interfaces (APIs).
The Method: FSD, Service Innovation Lab and SmartStart
When we collectively examined the FSD dataset on the FSD website, there were a few challenges. The Lab found the data was published in human but not machine readable formats extracted from an application that was not API enabled or machine discoverable. Given the source data was actually a database within MSD (where it is well maintained and kept reasonably up to date through online update functionality for service providers), we looked at how we could publish directly from the database to the open data service, data.govt.nz.
We also discovered there were fields in the source database that were not exposed through the current publishing process that would lend themselves nicely to service delivery, such as categories (to ensure SmartStart users got relevant rather than all categories), unique service and provider identifiers, and organisation “purpose”, to name a few. The contact information was largely free text, but the physical addresses were not geocoded and that was as a key requirement to making this dataset consumable for service delivery. Any changes needed to be built into the MSD maintenance and publishing process to be sustainable. As with most data projects, there was a fair level of “grunt” required to get the data into the shape required for service delivery.
First we did a manual database extraction and provided it to the Lab where they geocoded around 86% of the provider addresses. A proportion of the remainder were necessarily without location (for national helplines or sensitive services like refuges), and we manually updated the remaining several hundred physical addresses. The SmartStart team and Catalyst also assessed the data extract for SmartStart service requirements. We then updated the database to be able to store the geocoding by adding latitude and longitude fields to the FSD database. The manual process to complete the missing latitudes and longitudes was completed within 2 weeks by the FSD Admin team (and it is a small team), and the addition of the coordinates is now part of the BAU work process.
The FSD team started providing the brave new machine readable extract to data.govt.nz in September 2017. Because data.govt.nz uses CKAN, and now supports hosting of agency data, it means the FSD data is available publicly with automatically generated APIs. The SmartStart team (including Catalyst) mapped the data to the SmartStart topics and user needs. Just like Rome, further work was required, and we subsequently released 2 more iterations of the extract structure as the service requirements emerged. The first adding the level 2 service categories from FSD, and the most recent the unique service identifiers to enable clustering of services under each provider in a service.
All up, to deliver the data set in a form that enables location based service delivery required us to:
- modify the FSD database
- add latitude and longitude information to physical addresses
- modify our BAU work process to maintain this information in new and amended listings
- develop the data extract direct from the database
- implement a secure file transfer process to provide the extract for publishing (with thanks to the MSD IT team)
- write code to collect the extract, check the structure and publish it to data.govt.nz for use by SmartStart and anyone else (with thanks to the Service Innovation Lab team).
The Outcome: helping people & enabling innovation
Even if we only put in the effort to support SmartStart users, it would have been worth it. FSD has good provider data and processes in place for keeping that data up to date, whereas SmartStart is not in the business of maintaining family and social services or provider data. So the approach taken provides a good and sustainable outcome for all parties and SmartStart users.
But the Lab really wanted to ensure the functionality would be available to any service, whether provided by government or non-government organisations. The idea of extending FSD to be a reusable component was extremely attractive to the FSD team as we wanted to provide greater value and support to New Zealanders. Work is already underway to leverage off the open data with the Lab developing a demonstrator mobile responsive Family Services Directory function that allows people to more easily search for nearby services from their mobiles. This work not only takes advantage of the open data but also reuses code from the SmartStart “Services near me” functionality.
Of course we look forward to seeing how other people and organisations make use of FSD, because machine readable data available with a simple API provides the potential for anyone to use the data to service myriad use cases and user needs. It also provides greater incentive for family and social service providers to update their data in the Family Services directory. Within days of the SmartStart “Services near me” going live, we have been contacted about new service providers that should be on SmartStart. We will contact them to get them to list on the Family Services Directory and therefore on SmartStart too. We like to think this creates a virtuous cycle of natural motivation, benefits realisation and value creation.
By sharing the FSD data we also hope to enable purpose built (bespoke) service directories to meet targeted audiences’ needs in terms of focus and functionality by using a (broader) data source that also supports a FSD, which is a broader, more general directory of services. Sharing the data on FSD should reduce the overhead that “directories” have in maintaining the data and the burden on providers to maintain the same information across multiple databases. Providers could direct other directory consumers to draw their data from this API and therefore have to update it in fewer places. As an example we see services for youth as a key opportunity for other parties to develop a “youth friendly” service directory that is supported by the open data from the Family Services Directory (FSD has over 1,800 providers classified under Youth Services).
Another key benefit of better sharing the FSD data with SmartStart is promotion through SmartStart’s profile: providers will want to be on SmartStart and by pointing them to FSD to list we increase the depth of providers on our Directory and concurrently enable their services to be accessible through multiple pathways.
Last Word
Overall working together we have delivered greater functionality to users of SmartStart and subsequent life event services, greater benefits to family and social service providers, greater value to MSD, and enabled service innovation across New Zealand. At the same time, it has been a really good experience and true collaboration focused on the needs of New Zealanders, working across departmental boundaries. We hope to be able to participate and see more of this kind of shared work for shared benefits in future.
