CFCamp Notes -- API Management from the Trenches, Matt Gifford

October 25, 2015

APIs are where it's at .
We should all be building APIs for most things these days.

DISCOVERY --
is the API still active?
how strong is the community?
what kind of feedback is there on it?

try not to rely on ONE api to manage everything under your code

look at them as micro-services
pick multiple APIs to deal with manageable things.
micro-services are where it's at
ex: hotel booking
1 api for hotel rooms
1 for the weather on those days
1 for travel, 1 for car booking, etc.
1 hotel site, but broken down into sections - micro services

abstraction --
consider this from the start
weather API. if it's used in your site…what if the Weather API gets canceled.
if you deal specifically with the (for ex) Yahoo site API and it gets canceled, it's difficult to swap that out of your code. write an abstraction layer so you can swap APIs in/out easily.

Version Monitoring --
Twitter API changed their authentication and "broke" the API (it was public knowledge but still a challenge) when they moved it to OAuth

API Changelog (via Digital Ocean)
if you're using a public API, look on here first
if it's there, you log in, find the API, click "follow"
if that api changes, it will notify you so you're never caught by surprise

will notify you AS the change happens. they "poll" the public APIs. as SOON as the change happens, your'e notified, so you can react very quickly.

Network --
connecting to the API
if it's a private API, we might be connecting to the client, dealing with firewalls, etc.

traceroute
Fiddler

Authentication --
really hard
Twitter's OAuth connection, for example
DONT rewrite authentication if you don't have to. it's REALLY hard. use open source libs if you can so you have fewer headaches.

Test in Isolation --
first things first -- interact with the API directly
isolate all the little bits as you go
there are OAuth token generators that will generate the tokens/secrets for you. use these as part of your testing process.

Direct API calls
take the wrapper out and just hit the API directly for testing. so you know your wrapper isn't causing a problem.
can increase speed, interact much quicker with the API. instant gratification.

Scope --
check the moving parts
REST Client for Firefox plugin
any time you need the various HTTP methods, this client plugin can make testing much easier

cURL -- command line easy way to test it as well.
curl -i -X PUT [url goes here]

HTTPie
same as cURL but much simpler command line syntax

POSTMAN
amazing.
free and premium versions
the new version 2 is a big improvement

Payload --

Mock API --
to make sure what we're building is right

Schema
discuss as early as possible
what things will we pass in/out as parameters? how will it look?
the earlier we agree, the sooner people can start building against the API and save time.

Rate limits
another benefit of a mock API, rate limits are easier to deal with.

have your mock API run locally so you can work anywhere, don't need a wifi connection, etc.

mockable.io
apiary.io
mocky.io (similar UI to POSTMAN)

a fake API might be better than the online resources like mocky.io -- if your wifi goes down, the local fake API will continue working.
also means less security, noone trying to hit Mocky.io, etc.
everything is local, everything is fake. we can drastically minimize dataloss.

fake APIs can also be dynamic
can create fake data, etc

WebPro Dyson
various other Node-based options

json-server
another good one for local fake APIs

proxying

tunneling
various proxy tools allow us to do this

if someone else needs access to your local fake API
can expose a port to make that fake api visible to someone else

charlesproxy.com
httpkit.com/wiretap
runscope.com/docs/inspector
ngrok.com

Cache Management --
always make sure your data is clean and fresh
if your' working locally, don't worry about about caching. you won't gain enough performance locally to worry about it.

author debugging--
FIRST things first -- make sure your API actually WORKS

Logging --
as an author, logging is a top priority
consider logging as early as possible. NOT an afterthought
will need to see what's going on in your API, regardless of it it's public or private.

Find Issues --
add identifiers to URLs so you can track a particular request all the way down the stack
allows you to do a lot of analysis, see what's happening at a granular level

Cloud Logging
don't need to dump to massive data files any more. don't need mysql dumps any more
sumologic.com
-- real time searching logging facility, dashboards, monitoring, etc.

logstash.net
part of the Elastic family
can parse logs and pipe them to other services, etc.

loggly.com
nice dashboard and search capabilities, etc.

Logging is cruicial
not just error logs, but can also log exception logs
but if it's all in the same place, it's harder to analyze -- massive log with all these exceptions + everything else in there
better - segment the 2, don't send exception logs and trace logs to the same place

bugsnag.com
java wrapper for if you're doing CFML stuff

airbrake.io
similar to bugsnag

getsentry.com
similar to AirBrake and Bugsnag

pricing levels on the tools can vary

Monitoring --
we need a central area to inspect everything.
pipe all that info to a central dashboard
can build your own, or use cloud services
zapier.com
apimetrics.io
runscope.com/radar
-- can run unit tests similar to Selenium but more specific to APIs
ducksboard.com
-- great dashboard, can get everything thrown into 1 place

Load Testing
if you are launching something, get ti right. you don't want products to fail
you'll have al to of people hitting it
don't rush your API out the door without testing it
make sure a lot of people can hit it at once
github.com/JoeDog/siege
- can see how many requests/sec you can serve

loader.io
depends on an network connection to work

github.com/wg/wrk
great benchmarking tool
>> cmd -t12 -c400 -d30s http://site:8080/testthis.html

running 30s test @ site
12 threads
400 connections
super easy to use

Other considerations for authors -
as an author you have  a lot of people depending on you
we don't want those people to be frustrated
first -- DOCUMENTATION!
let ppl know how to acutely use it
if you want it to be successfully adopted, build a rich suite of docs for your API
what can users send thru, what responses come back
basically spoon-feed them
they'll thank you for it if you help them out
various tools to help write docs
apibluepring.org
-based on markdown files, generates docs for you

raml.org
swagger.io
- nice tool, available in lots of languages.
- uses YAML
write .yaml file
will generate a local faked api for you
CF12 can import Swagger into the API manager in CF

bit.ly/designApi
-- great blog post on the benefits of designing your API using these tools

SDKs --
provide an SDK if possible, a CF wrapper, whatever
it might not be perfect, but provide the helping hand whenever possible.

authors -- as part of the help, allow users to talk to you
users -- ask others don't just work by yourself in a box. look for assistance.

follow these people on Twitter for api stuff
@kinlane
@johnsheehan
@synedra

above all else, have fun with it.

if you'er building something, there are lots of things to consider.
building an API isn't necessarily easy
"i can just hit an end point, i can just hit a GET URL"
it's not quite as easy as that. have to consider the RIGHT way of returning that data.
do you auto-redirect the API?
do you give them a status code? which one? is it 200 for everything?
there are specific rules, but you have to dig deeper to find them.

Book: REST Assured by Adam Tuttle
restassuredbook.com