Considering the Big Picture Before We Mandate Government APIs

A new bill in New York City needs the attention of a broader audience.

Many thanks to Kin Lane, API Evangelist, for his thoughtful contributions.

Imagine that you and your family have fallen on hard times. Through your community — perhaps a church or mosque, you hear that there are government services available to help. An internet search on your smartphone reveals an app which claims it can help you apply easily and quickly for government assistance. You download the app and glance briefly at the first few sentences of the EULA before tapping “I agree”. You follow the instructions, give the final application for benefits a once-over, and then click submit. The next day, you start getting emails advertising “cash now!”, but they’re really high-interest small loans. The following day, emails pushing debt restructuring services. The day after that, offers of credit score repair. A few days later, similar letters start arriving in your real mailbox. A week later, nearly buried in the pile of correspondence, is a letter from the government which asks you to meet with a case manager who has reviewed your application and would like to help you take the next steps. You almost missed it.

Concerned? You should be. That scenario might be a more extreme example, but some recently proposed technology legislation in New York City could make it a reality. Hidden among a batch of other bills [PDF], was a broadly scoped proposal which would require the city’s government to use Application Programming Interfaces (APIs) to send and receive information. If you aren’t sure what an API is, you may want to quickly look at Kin Lane’s API primer, or perhaps the description on Wikipedia.

This bill would require that information disseminated and accepted by City agencies also be transmittable through an Application Program Interface (API). It would also require the creation of a website with information on how to find and use such API’s.

With good planning, it is valuable to set the expectation that a more “open” city includes APIs for transactions. Tim O’Reilly has been talking about Government as a Platform since at least 2009. The Sunlight Foundation wrote in support of government APIs in 2012, but also recommended a bulk-data-first approach. We are already seeing efforts such as Open311, Fast Healthcare Interoperability Resources, Payment Services, or the Waze Connected Citizens Program that provide benefit to governments, their partners, and the public. This bill is a laudable first-of-its-kind attempt to mandate the use of APIs, but it represents serious risks to both the public and the government. In its current form, it shouldn’t be passed. As members of the community, we should not miss the opportunity to improve it so New York City can continue a great trend of smart legislation. If it does pass, it will likely become a model for similar policies elsewhere.

How can we make this proposed law better? I’m glad you asked! First, let’s lay out some bigger-picture issues that need to be considered. A bit later, we’ll lay out some specific recommendations to the existing bill.

1. Perhaps most important, define the goal. How will the successful use and sustainability of APIs be recognized? How will progress towards that goal be measured? NYC’s open data law at least requires an annual report to be published. As proposed, the law will require the City to spend millions of dollars to achieve a confusing objective which won’t be clear to the taxpayers who are paying for it.
2. It may be more helpful to use terms like interoperability or observability, which can help focus work where there is a clear need for city systems to exchange data/messages with non-city systems. APIs are somewhat technical, and generally refer to a subset of stateless machine-to-machine communications over the web, such as SOAP or REST-style services. There are plenty of other mechanisms like Remote Procedure Calls, Message Queues, WebRTC, SCADA, and more.
3. Explicit funding is needed. Asking the city to retrofit legacy systems (defined as “anything in production”) carries an enormous price tag. Most major systems have been capitally funded, so there’s usually very little expense fund flexibility to draw from.
4. Allowing third-party intermediaries to connect constituents to services creates great opportunities for innovation, but it also creates risks to privacy and security. An organization which promises (with the best of intentions) faster approval for SNAP benefits will have to meet the same expectations of privacy that the city is expected to meet. There are a variety of other risks here — for example, how would the identities of constituents be validated when necessary? What might intermediaries do with the data they collect? The city will need to audit those intermediaries, and be able to act in defense of its constituents when things go wrong. And they will go wrong. Since we don’t yet have suitable state or federal laws related to cybersecurity, the burden of this would fall on the city’s Chief Information Security Officer.
5. Under the leadership of Mayor De Blasio, New York City government strives for equity. Efforts like Open Data for All work to reduce the digital divide, which these days is less about getting people connected and more about helping them become tech-savvy. APIs present great opportunities to strengthen this work, but they also present significant opportunities to widen the gap between those who know how to build tools that use APIs, and those who do not. New legislation should try to take this into account, perhaps by requiring the creation of training programs — or at the very least connecting existing training efforts to these new public APIs.
6. The NYC Mayor’s Office of Technology and Innovation has developed guidelines for adoption of Internet of Things (IoT) technologies. How does IoT fit into this picture? When projects are created through public-private partnerships (like LinkNYC), where third parties may also provide APIs in the public interest, should this law apply to them as well?
7. The Comptroller’s Office, Public Advocate, City Council, and other government entities including Boards and Commissions also have highly valuable information and services that could benefit by being accessible via a publicly available and documented APIs. The proposed legislation doesn’t define which parts of the government must carry out this new law. Agencies all have their own IT operations, which also means compliance will be complicated. NYC’s open data law contains the following definition, which is slightly narrower, but may be a helpful starting point:

“Agency” means an office, administration, department, division, bureau, board, commission, advisory committee or other governmental entity performing a governmental function of the city of New York.

Finally, here are some specific proposed language changes.

The legislation doesn’t make a great effort to define an API, so perhaps we should start by improving that.

§ 23–202 Open application program interfaces.
a. Any information, including both text in a narrative form and data as defined in section 23–501 [NYC’s Open Data Law], posted online or otherwise made available electronically to the public by an agency shall also be made available through a web application program interface that permits programs to request and receive such information directly from a city web portal.

Technically, normal communication between a web browser and and the city’s web servers meets this very loose definition. We need a slightly better definition which gets both more specific about the goal, while at the same time not constraining how this would work 5–10 years from now. Here’s the rewrite:

a. Any information, including both text in a narrative form and data as defined in section 23–501 [NYC’s Open Data Law], posted online or otherwise made available electronically to the public by an agency shall also be made available through a mechanism that permits machine-to-machine communication over the internet.

b. Any non-emergency city services for which intake information is accepted from the public by phone, paper form, web application or mobile application, including but not limited to 311, shall also permit such intake to be made through a web application program interface that permits programs to transmit such information directly to a city web portal.

We learned from NYC’s equally broad open data law that expending time and taxpayer money to publicly release tons of data doesn’t necessarily mean it will get used. Publishing open data really does take a lot of resources; deploying APIs where data can also flow in will take way more. In addition, NYC has been operating a few APIs for years now, and there doesn’t appear to be a huge demand for more. A better approach might be to require APIs where there is a demonstrated public demand (and a mechanism for measuring that demand), and provide a timeframe within which the API must be made available. We also need to leave the city room to identify situations in which it will not fulfill the expectation of creating an API, such as when the services may present too great a risk to privacy, are too complex (e.g. software wizards), or the investment of implementing them isn’t worth the return.

b. Any city service available online shall support the use of APIs within one year of recording a measurable demand from public. The Department of Information Technology & Telecommunications must promulgate rules for identifying and measuring demand, which should include community meetings, analysis of web server metrics, implementation of similar APIs in other cities around the globe, and others as reasonable.

c. Any public data sets posted to the single web portal pursuant to chapter five of this title shall be exempted from the requirements of this section.

The NYC Open Data Portal already has built-in APIs for many of its datasets, though it isn’t required by any law or policy. Therefore this seems like a strange exemption, and we recommend not having it.

c. There is no c.

d. The department of information technology and telecommunications shall post in a single portal on the city website information on how to utilize each of the web application program interfaces required pursuant to subdivisions a and b of this section, including both plain language descriptions and technical details. Such portal shall also include a listing of the application program interfaces available pursuant to subdivisions a and b, the endpoint for each such application program interface, and, for those required pursuant to subdivision a, a description of the information contained therein.

The great news here is that this already exists! The NYC Developer Portal was launched almost five years ago. So this part is fine as-is, or could be updated to say that existing work to have government APIs should be leveraged.

§ 2. This local law takes effect one year after enactment.

While there’s nothing wrong with waiting a year, if the recommended changes to paragraph (b.) are made, the law probably should go into effect immediately.

§ 2. This local law takes effect immediately.

It’s great to see legislators thinking about how to improve government engagement through technology. NYC’s Open Data law was built through active engagement and consensus. Hopefully this proposed legislation will continue to build upon that model for success. One path for that could be to share it on Madison and actively invite community participation at events and through social media.