Many modern web applications have embraced the tenets of REST to gain clean separation between resources, take advantage of HTTP features such as caching and verbs, and describe discoverable relationships between resources. Ideally, it’d be as simple as defining a REST interface that could be used by multiple consumers targeting any number of resource producers. The underlying data sources could be shared or siloed depending on the service architecture.
In practice, however, adhering fully to the REST tenets and creating a perfect implementation is hard to achieve. Most complex applications face some degree of compromises and pitfalls that result in the interface being classified in more of a “RESTful” category.
One example would be missing hypermedia usage to reference resources, which opens up the possibility of resources duplicating and tweaking the representation of a resource. This makes it hard for consumers to intuit the interface and for producers to create consistent documentation for the interface.
An over-adherence to REST may also result in an interface that requires suboptimal data-fetching patterns. This includes a resource that may be too fat or too thin for a consumer’s needs, which can lead to specialized endpoints to contain exactly what a specific consumer would need. Having smaller resource representations also results in making an abundance of network calls to fetch data for a content-rich page or to determine complex UI interactions.
Implementing better processes and review standards will help address some of these difficulties, but there’s a different way to think about the interface to both optimize and increase consistency in the way consumers and producers interact.
Web applications have exploded in terms of number of users and output of content. This is a real concern for many applications that need to be performant and highly interactive. Facebook and Netflix are two such applications that have been leading the forefront in trying to solve this at scale, using GraphQL and Falcor respectively. They’ve both introduced new concepts that address some of the difficulties of REST, which enable them to be more flexible and efficient with the consumer and producer interaction.
GraphQL is a new query language specification to fetch and mutate data. It takes into account that your data may be nested and that your data will have complex relationships. GraphQL does not make any assumptions about what your persistence layer looks like, so you can use whatever systems you already have in place or feel makes sense with the data storage needs of your application. Most commonly, we see usage with NodeJS and MongoDB, so the examples in this article will highlight those use cases specifically.
GraphQL has an expressive query language available to the client to request data. Rather than having multiple endpoints based on the entities the client is querying, GraphQL will expose a single endpoint like /data, and the client will supply its queries to it.
The query language has multiple parts, but let’s start with a simple example: an employee database that we want to pull employee names from.
With this example, we used a root query field called employees and asked the GraphQL middleware to give us just the name field from those employee data objects. But GraphQL is much more expressive than that. Remember that GraphQL is hierarchical, so we can do queries like getting us the employee names but also the names and members’ names of projects that each employee is staffed on.
Below is an example of a REST API endpoint. Using a GET request at this endpoint most likely means the consumer would be receiving the entire employee object in the queries. Many REST APIs require a special header, query param, or use an ad hoc endpoint as an attempt to limit the data coming back to be a certain subset. This means the consumer will have little control and the API designer will have to specify very tersely what the subsets vs. full sets of the employee data look like.
Queries can also contain arguments defined by the GraphQL schema—for example, limits, ids, or sort orders. These arguments can do almost anything the author of the schema desires.
Let’s look back at our employee data and pretend we have more than one employee. We can do queries such as:
This isn’t much different than how a REST API might behave. You can think of these arguments like query parameters. For example, in a typical REST API, you might have the endpoints listed below, but the API maintainer needs to ensure that the compact mode listed in the last example is also wired into this endpoint so the consumer can request a compact form of the employee data (if needed). Also, GraphQL enables introspection on the queries you can perform, so the REST API will need to have documentation that will tell consumers about the headers, query params, and endpoints available to get certain employees and sort them.
Facebook has its open source implementation of GraphQL in JavaScript located here, but many others have ported it to various languages.
GraphQL provides a hierarchical query system shaped like the response data, enabling consumers to request exactly what they want—and giving them some flexibility without requiring changes on producers.
As you can see in the examples above, GraphQL seems convenient due to the ability to supply query arguments and desired fields—but how does the consumer know what’s available? Introspection is the key to making life easier for anyone that wants to interact with GraphQL.
First, let’s look at the example above where the consumer could request an employee by ID, and dive into how the consumer would find the argument name and availability of that feature in GraphQL.
These simple introspection queries come for free with GraphQL and can be expanded on with deprecation notices, descriptions, and more. Some of these fields, like description, do require the developer to maintain them. But these are just some examples of free, out-of-the-box introspections that make documentation extremely simple to keep up-to-date and available. Also, because you can query for the documentation from the GraphQL endpoint, there’s no need for a separate page or application to maintain in order to display documentation information. There are a lot of great documentation tools you can use with REST APIs, like Swagger, but it’s hard to compete with an almost entirely self-documenting system when it comes to ease-of-maintenance.
Mutations provide a way for the client to create, update, and destroy data. They occur one after the other, and because of this, errors can be detected and propagated very easily. Mutations can be achieved in REST via PUT and POST methods of course, but you have to know which verbs to use based on the API implementations (some use these verbs differently). With GraphQL, you simply fire the POST request to the same/data endpoint in the above examples, which you can of course rename. That way, you don’t need to use the exact verb, as GraphQL schema manages creations and updates.
Mutations—unlike queries—have names and parameter lists. Here’s an example mutation that will add a new employee:
In GraphQL, fragments are a helpful way to group commonly used fields for queries, which enable you to use fewer key strokes. Fragments are a great way to get consistent data sets back when you’re querying multiple similar types in GraphQL. Because the consumer specifies the queries, the backend maintainer has an easier time designing the API. The maintainer doesn’t have to keep all the domain models and endpoints in sync with what they return—compact vs. full set—like they do in a more traditional REST API.
To illustrate fragments, here’s an example of how we can save queries to variables:
You can imagine times when it might be useful to save the result of queries into variables, but in this example, the repetition of fields was disruptive and unnecessary. This is where fragments come in. Fragments must specify the type of field they’re working on, and then they can be used for each query.
Fragments can contain nested fields, compose other fragments, or be mixed with fields at the query level. For example:
I recommend trying out Facebook’s GraphiQL tool, which has a sandbox mode with a very convenient autocomplete feature. Also, here is a great repo showing a small example and a link to a tutorial on how to build it.
GraphQL seems to be a very promising alternative to RESTful APIs. Unlike REST, GraphQL gives consumers a significant amount of power to format data. GraphQL’s hierarchical nature also plays very nicely into the composition-style architecture most modern web engineers are embracing. However, the setup of producers for GraphQL does take a decent amount of effort and more advanced use cases—such as granular authorization patterns—and may be more difficult to set up before GraphQL fully matures.
Most of my usage of GraphQL comes from the JavaScript implementation, but below are the pros and cons I’ve found using GraphQL:
Falcor is a web interface platform that unifies backend data into a single virtual JSON-model object. Interactions with your traditional services are encapsulated through the Falcor model, which uses a new convention, JSON Graph, to describe resource relationships and reduce duplication. This approach strives to reduce the network traffic required to populate most web consumers and better describe interconnected data.
Netflix recently open-sourced Falcor as a developer preview.
Paths are sequences of keys used to traverse a JSON object from its root. This is similar to object traversal in JavaScript, with a few notable differences. Falcor enables paths to be described as an array of keys or a string, but string paths are parsed into arrays at run time. Paths can also begin with an indexer for the first key in the path. For example: `[“employees”][0][“project”]`.
References are paths contained within an object’s properties which describe symbolic links to other locations within the JSON object. This is the magic that turns the JSON object into a JSON Graph. Let’s take our previous JSON object and add references to turn it into a JSON Graph:
With these references, we can identify an employee working on the first project by querying `[“projects”, 0, “employees”]`.
One caveat here is that the number of records to retrieve must be explicitly declared.
Similar to GraphQL, we can define special queries and use them to retrieve our data.
JSON Graph is a convention for modeling graph information as a JSON object.
The JSON Graph is one of its biggest divergences from REST—and also where Falcor gets a lot of its power. Instead of tying one endpoint to a single resource type, the JSON Graph enables a single JSON object to describe multiple resource types and any relations between them. The name itself emphasizes the design choice to model a graph within a hierarchal format in JSON. This is done by adding new primitive types, like references.
JSON Graph is a convention for modeling graph information as a JSON object
- http://netflix.github.io/falcor/documentation/jsongraph.html
By using a path, references within a JSON Graph point to another key in the graph. A path is dereferenced as part of resolving the value for a reference. This allows for only one instance of a particular resource to be fully populated within the graph. The main benefit of this is avoiding multiple declarations of the same resource to not only reduce the JSON size but to avoid having stale versions of the resource.
Unified Falcor models provide a single endpoint for consumers to use. Similar to GraphQL, this enables consumers to have access to the entire data set but request only the parts they want. Typically, this would compose different backend systems into a single interface. Producers can optimize fetching data by only requesting from systems that are part of the consumers’ route. The unified model becomes the documentation for the API, since it’s the path to interact with the application data.
A crucial part of balancing interactivity and performance is designing a strong model and route. A guiding principle of model and route design is that consumer interaction needs to be explicit through bounded interactions. This means it’s not possible to do a fetch-all type of interaction when fetching data. This is intentional—to make the consumer pick the right amount of data for their particular view. It also encourages producers to bound their downstream interactions to prevent source data growth, which adversely affects upstream consumer performance.
As a consumer, the main optimization is the ability to request all relevant data at one time, regardless of the number of downstream systems. This is due to a combination of the previous design choices—references and a unified model.
Falcor also has the ability to cache the model on the consumer side and batch any requests to the producer. The cache is an in-memory cache of all previously fetched data. The size of this cache is configurable and automatically overwrites the oldest data first, once it has reached capacity.
Batching enables the consumer to specify multiple paths that should be used by the producer to return in the unified model. Subsequent requests for data get added to the consumer-side cache.
Falcor is an interesting take on optimizing the web interface and is based on Netflix’s evolution of their own REST API. The ideal use case would be a paged catalog and detail view with referential data, where the consumer could explicitly control how many results to use but take advantage of the optimizations provided by JSON Graph.