Total Surrender to the API Culture is the New Thing especially in the space of payments for easier merchant integration, easier SME solutions of businesses (Invoicing, Salary Payout, Disbursements), easier Future Tech adaptations (SaaS Tools Integrations), more convenient Payment Use Cases Integration.
The quality & quantity of efforts being dedicated towards Payments APIs shall decide the success of that Payments Platform.
The Stages across which an API Development goes through comprises of below :
- Brainstorm – First, it is necessary to identify key services your business offers and capabilities of the business. Figure out the kinds of APIs that should be built and which services should be offered via APIs. Also, figure out and write down the use cases for each API. Write down potential endpoints based on those use cases.
- Establish API stakeholders – Who are the stakeholders within your organisation? As many people as possible should be involved in your API initiative – you need company-wide buy-in and a vision that is shared by teams within your organisation. Also, allow stakeholders to weigh in on the design of the API. Stakeholders can then agree on interactions across the organisation so that APIs remain consistent.
- Design an API contract – The contract establishes a set of standards and best practices for designing APIs. Be sure to describe and document all APIs. Ensure that all APIs work the same, from endpoint names and URLs to error codes and versioning. Consistency is key.
- Create a style guide – A comprehensive, cohesive style guide ensures consistency across the teams of an organisation who are building services. API status codes, versioning, error handling, and more will be standardised ensuring that APIs are designed the same way. Use a tool like SwaggerHub to create a style guide for all APIs across your organisation.
- Implement API governance – An API governance process can help enforce established standards and reinforce desired outcomes. We discuss API governance in an upcoming blog article. Conducting peer code reviews can also help ensure that API design standards are followed and that developers are producing quality code.
- Automate processes – Use tools like SwaggerHub to automate processes like generating API documentation, style validation, API mocking, and versioning. Also, make APIs self-service so that developers can get started building apps with your APIs right away. Provide interactive documentation or a sandbox so that developers can try out API endpoints.
- Track and manage your API portfolio – Avoid duplicating code and building redundant APIs by tracking and managing your API portfolio. Implement a system that helps you track and manage your APIs. The larger your organization and platform becomes, the harder it gets to track APIs and their dependencies.
- Create a portal for internal developers – Create a central place for internal developers, a place where everything for all your APIs is stored- API specification, documentation, contracts, etc. For example, PayPal has built a portal for its developers and it’s “one of the most visited internal apps in PayPal,” according to an InfoQ article. PayPal’s portal includes an inventory of all APIs, documentation, dashboards, and more.
- Implementing an API-first transformation in a large organization is as much a people problem as a technical one
- Organizational chutzpah will strongly influence your API-first success strategy
- Treating the transformation itself as a product promotes long term success
- Process and governance are necessary, but keep it light and make it work for your customers
- Don’t underestimate the investment in tools, infrastructure, and people required to make it work
Getting Your API Strategy on
It wasn’t that long ago the idea of exposing your core plumbing externally was verboten. The real products were the apps and experiences you built on top of that internal stuff. Most product investments resulted in vertically integrated solutions that served a relatively limited set of functionality. Leveraging that investment for a totally different set of use cases or a different product line was often somewhere between difficult and impossible. In that context, it was natural to think of the underlying infrastructure as sunk cost or internal-only plumbing. It was hard to imagine it differently.
That’s changed. More or less every company now has an API strategy that strives to leverage a much larger portion of its technology investment by exposing the outcome in the form of reusable services. The high-level goal is to accelerate business agility by making core business capabilities easily accessible and reusable. Reducing the cost of integration not only has a positive impact on internal velocity, it also provides for increased flexibility and speed incorporating customers, partners, and acquisitions. The days of monolithic, vertically aligned mudballs are rapidly coming to the end. What’s now generically referred to as “microservices,” exposed via nicely designed and documented APIs, with a clear bounded context and semantically distinct business value are where we’re going.
The API Economy
Why the change? The emergence of the API economy has changed the rules.
Some of the highest value companies in the world – Facebook, Google, Amazon – all figured out a while ago that opening the kimono and providing external developer access to more of the plumbing of their core products was a great way to leverage their tech investment, reinforce their product portfolio, and expand partnership opportunities. They built cultures that reinforced share and re-use principles, which were key to making this possible. This catalyzed the API economy and enabled many, many new products to be built and market-tested faster and more cheaply than before. API providers reaped the benefits of more customers, more traffic, deeper integration, and, at least for some, the seeds of new, multi-billion dollar businesses. Even Twitter owes much of its early success to an API strategy even if they subsequently pulled back from it.
The second wave – companies like Twilio and Stripe – skipped the traditional “product” part altogether. They made the APIs themselves their core product and focused on providing great developer experiences as their primary mission. Their customers quickly integrated and got the benefit of essential, needed capabilities without having to build and maintain a lot of non-differentiating functionality themselves. Agility improved. Hackathons happened. Apps and start-ups exploded. It became the default way to build new experiences, iterate, and verify product-market fit.
To remain competitive as an established player your organisation also needs to adopt an API-first approach. It is a nontrivial change to how you look and think about your tech portfolio.
Actualising Your Future
For newer companies, this approach is already second nature. They’ve likely grown up inside the new reality. But what if you are a large, established company with a lot of existing infrastructure and customers? How do you go about making this transition while still running your business?
First, your tech landscape probably looks quite different. Your legacy code base – probably influenced by SOA, but developed before “microservices” was even a word – is likely sub-optimal. You may have built a lot of tech debt over the years and you may not have a culture that natively reinforces the principles of clear componentization and reuse. Testing may be spotty and there are likely skeletons left over from various re-orgs and abandoned projects. Documentation may leave a lot to be desired. This is not a nice, level foundation on which to start a major microservices transformation. There are, as we say, a lot of “challenges”.
The other major consideration is organizational chutzpah. Executives may be sold on the need for an API-first strategy, but their understanding of the commitment required to get there may vary. Understanding where you are on this spectrum is very important to your success.
On one end, you get the Manhattan project. The bus stops, everybody gets 100% with the program, it’s the top priority, and almost everything else is put on hold. This is rare. If you find yourself in this situation, run with it. Treat it as a massive project and move quickly. You’re not likely to get a second chance.
More likely, practical reality puts you somewhere in the middle. There may be a commitment to slow the bus a little, but it’s definitely going to keep moving and the wheels will need to be changed along the way. As a leader of the transformation, this feels a lot messier. It raises a lot of difficult and important questions that don’t have obvious answers. How is it going to be prioritized? Which teams are going to do it? How is it going to be coordinated? When will it be done? Your job just got a lot more complex. It’s no longer a project. It’s now a complex transformation that may continue for years and it’s as much about organizational change as it is about a technology transformation.
Put on Your Product Hat
At PayPal, we’ve been on this journey for over three years. We’ve used a customer-oriented approach to go from a very monolithic, siloed architecture to a much more loosely coupled set of over 150 services with well designed, modern APIs.
There is a tendency to treat programs like this as engineering projects. We didn’t. Instead, we framed this as a larger, organizational change problem with the “product” being a fundamental shift in how we design and build APIs. Viewed that way, we were forced to identify and serve all key “customers” – developers who build and consume APIs as well as the executives that support them. It shaped our strategy, the tooling we built, how we communicated, and how we measured success. Identifying your customers and focusing on their satisfaction is a fundamental product principle. This mindset has been key to our success and it continues to shape how we manage the program today.
While no two companies are the same, many of the lessons we’ve learned and approaches we’ve used are applicable to other organizations on the same path.
To break this down, we’ve organized the effort using a framework we call the “3P’s”.
- Product – the infrastructure, standards, and tools used to manage the portfolio of APIs and underlying service implementations
- Portfolio – the catalog of business capabilities, represented as APIs and underlying services
- Program – the metrics, training, and levers used to incent changes in organisational behaviour and technology investment
This article is the first of three that explores these dimensions in more detail.
In a small company of a few dozen to maybe 50 or 60 developers, it’s not that hard for the key stakeholders to get in a room, hash out a plan on a whiteboard, divide responsibility, and get to work. There will be a pretty good, shared understanding of the plan and goals, and, after some iteration and course correction, you’ll likely end up with a nicely integrated and cohesive set of APIs and underlying services representing your platform. Your timeline is probably represented in months, or maybe quarters.
In a large organization, things are different. You may have hundreds or thousands of developers spread across multiple business units and geographies, all with differing objectives and, often, different tech stacks. For many reasons, not the least of which is Dunbar’s number – you will not have a Kumbaya moment where everybody holds hands, agrees on the goal, and gets to work. What you need is something more distributed, more scalable, something that reinforces your objectives without depending upon the frailty of social contracts or large scale project coordination. In this context, the “Product” is actually the infrastructure that supports the transformation (not the APIs themselves). This is a significant investment in and of itself. At a minimum, it should probably include the following:
- A playbook defining common principles and standards for APIs
- An engagement process to ensure compliance when developing APIs
- A means of governance including clear and objective measurement
- Some kind of centralized portal to consolidate APIs and manage the process and metrics
Principles and Standards
At PayPal, we started off thinking about what common ideas would guide us towards our goal of nicely encapsulated services that exposed well-designed, reusable APIs. These became our core principles and they framed much our subsequent standards documentation.
- Loose Coupling – Services and consumers must be loosely coupled from each other.
- Encapsulation – A domain service can access data and functionality it does not own through other service contracts only.
- Stability – Service contracts must be long-lived with a predictable lifecycle.
- Reusable – Services must be developed to be reusable across multiple contexts and by multiple consumers.
- Contract-Based – Functionality and data must only be exposed through standardized service contracts.
- Consistency – Services must follow a common set of rules, interaction styles, vocabulary and shared types.
- Ease-of-Use – Services must be easy to use and composable by consumers.
- Externalizable – Services must be designed so that the functionality they provide is easily externalizable.
Setting The API Structure within Organisation :
Step 1: Understand Your API’s purpose
What problem does your API offering promise to solve? Having an outcome-driven approach to API development can help you define better metrics of success, build a sustainable and scalable development strategy, and facilitate faster stakeholder buy-in. Some famous use-cases for APIs include:
- Facilitating faster development and creating an adaptable architecture by exposing all internal services via APIs – Amazon
- Creating a successful platform to drive higher engagement of users – Facebook
- Grow traffic to main website by incentivizing sending users through a partner API model – eBay
It’s important to fully understand the purpose of your API to see how it maps into your business taxonomy. This taxonomy is composed of smaller building blocks of your business from a functional stance. Identifying this, and defining the problem space of your API, helps provide a more focused approach to conceptualizing the exact purpose of your API, and how it maps into your existing business.
Step 2: Understand Your API’s Audience
It’s easy to throw the word “developer” around. What’s important to understand is what outcome the end developer is trying to achieve, and what domain they are in. There can be two consumers for your APIs:
- Consumers in your direct vertical: This type of API consumer builds services that directly benefit the main consumer of your core product.
For example, Slack’s end users are people within a company trying to communicate better with each other. To better facilitate collaboration and communication, Slack’s API can easily integrate with many popular third party tools that companies use and centralize notifications received from these tools. This way, Slack’s API can directly benefit the same people who use their tools and drive higher engagement and value.
- Consumers in tangential verticals: This type of API consumer would build services that may not necessarily help the direct consumer of the product, but solves the need of another vertical.
For example, ride sharing companies like Uber could integrate with the Yelp API to give suggestions for good restaurants depending on the location their passengers are driving through. This is a classic example where Yelp’s API is benefitting the consumers of Uber, a company that can’t necessarily be considered as Yelp’s direct consumer. In general, this allows you to have the vision of your full API value chain, made up of your API’s direct and indirect consumers.
Step 3: Align your Team
A successful, scalable API program first needs a solid team behind it that’s fully aware of the API’s objectives and purpose. A great team starts with alignment. The belief that APIs are products, that they solve problems, and they have customers who care about quality, is a mindset that needs to be instilled among all of your teams. A culture of digital transformation involves full buy-in on the power of APIs to be the foundation of a successful platform.
Alignment can come when you clearly articulate the current and future state of your APIs, and how they tie into your entire business strategy. After this, figure out how you tie all of your development, operations, and marketing teams together though well-defined processes.
The way you choose the right people, and define processes that keep them and their workflow intact, can be a determining factor in the success of your API programs. You should also implement tooling that supports a collaborative approach to API development.
Step 4: Develop your API Incrementally
Having a product driven approach involves building APIs that provide immense value to your consumers. A proven tactic to delivering APIs that people love is the build-measure-learn philosophy. This is taken directly from the Lean Startup methodology of having a set of assumptions around the validity of your offering, and iteratively proving and disproving these assumptions. APIs are built incrementally, with regular stakeholder and customer feedback in every iteration, to eventually deliver a great API.
We can divide development into all of its individual life-cycle phases. It’s to be noted that all of these phases should be supported by good tooling infrastructure that align with your company’s culture, methodologies and practices. Tools like SwaggerHub, Apiary and Reprezen can help development teams design and build APIs in a collaborative environment.
Designing the API
At the end of the day, your API is just a contract between a server and client, with the request-response cycle acting as the terms of the contact. If your end consumers are the ones finding value from your API, this contract needs to be designed with their needs in mind. Designing an API means providing an effective interface that helps your internal stakeholders aligned on the API’s request-response cycles, and external consumers better understand, use, and integrate with the API.
Design is the step before you develop your technological artifacts and business logic. The design of the API acts as the forcing function to really put your API objectives and persona targeting into a human and machine-readable document for your executives and technical teams to consume. As a product management technique, this is an easy way to also focus on the consumer, designing an experience that caters to the type of audience your API is marketed towards.
In the REST ecosystem, the OpenAPI Specification (OAS) has emerged as the world’s standard for designing the interface. The OAS is a human and machine readable specification that’s language agnostic. It’s currently in its third version, with rich expressive capabilities for new security definitions, call backs, linking and more. The API’s definition in the OAS also opens up a world of possibilities in the form of mocking, generating server stubs, SDKs, test cases, and more.
Virtualising the API’s Dependencies
Your API is built for interoperability. As applications become increasingly complex with many dependencies and components, both internal and external, it becomes challenging to navigate all of these dependencies. This is where API virtualisation comes into the picture. API virtualisation is the process of creating a virtual called copy of your dependencies and mirrors specifications of your intended API. By creating a virtualised back-end off your API design, as well as mocking your dependencies, you can –
- Allow your frontend and backend teams to work in parallel
- Create a mock API with dummy data, and distribute that to a few stakeholders both internal and external and get that early feedback and deliver API greatness
API virtualization and mocking is is fast catching on due to its many advantages. This article can help you better understand the concept and benefits of API virtualisation.
Building the API’s Business Logic
This is the phase where you take your machine and human-readable API design layer and convert that into actual code and business logic. Choosing the right technology framework is really important in this phase and luckily they open API design layer being language-agnostic can allow different developers in teams to do API business logic in different languages. Of course there are many different ways to choose the right programming languages, so here’s an article that can help. Tools like the Swagger Codegen project allow you to prototype APIs faster in the programming language of your choice by auto-generating boilerplate code from your API design. The code generator takes care of all the scaffolding in the API’s code, so your development seems to focus on what truly matters – the APIs business logic.
Testing the API
The testing phase is intended to reveal bugs, inconsistencies or any sort of deviation from what the expected behavior of the API was. depending on the mission criticality off your APIs you can perform different types of tests to ensure they function as intended. There are many different types of tests you can do like functional testing, reliability testing, load testing, and security testing, all of which can be prioritized based on your API’s objectives and target audience.
Creating API Documentation
Your API documentation is the face of your API. Think of it as a technical content deliverable that contains instructions about how to effectively use and integrate with your API. It’s a concise reference manual with all the information required to work with the API, supported by tutorials and examples.
Developers can auto-generate documentation from the OpenAPI definition layer through various API documentation tools. Documentation has a direct impact on adoption and growth of your APIs. The level of sophistication of your documentation can vary depending on the APIs use case. If it’s internal APIs, then maybe just reference documentation can suffice. External APIs tend to require higher levels of investment, with SDKs, tutorials, and a getting started guide.
Step 5: Define KPIs
A Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively your API is achieving its business objectives. While your API is being developed, it’s valuable to understand exactly what are some quantifiable measures of success for your API program. It’s easy to get lost in a sea of potential KPIs you can measure yourself against, so tying your measurable outcomes back to your API’s purpose and audience, defined in steps 1 and 2, can help. There’s two types of metrics you should measure yourself against.
Audience Journey Metrics
This is characterised by measuring the various steps it takes for your end user to reach and consume your API. Here’s a classic example of KPIs for a traditional API model.
|Interest for using API
||Registering for API key
||Time to register and obtain key
|Desire for using API
||Understanding how to use the API
||Time to first 200 response from API
|Desire for using API
||Integrating with API
||Number of API calls made over time
|Advocating for the API
||Sharing APIs among social and professional circles
||Number of shares made over time
The importance of measuring yourself against quantifiable metrics could decrease when building a private API, which may not necessarily require the same amount of rigor. That said, In case of private APIs, qualitative feedback and anecdotes can serve as good yardsticks to measure yours API against too.
Business Goal Metrics
These metrics are tied more closely to your API’s purpose. If it’s a public or partner based API model, then pricing and growth are good indicators of your API success. If your API is for private, internal use, then the number of applications and processes integrating with the API is a good measure of success. Here’s a few examples of business-centric KPIs.
|Business Goal of API
||Increase in MRR
||New customers referred through API
||Customer lifetime value of customers using apps built through API
Step 6: Collecting and Acting on Feedback
The metrics you define above can help you set benchmarks when collecting feedback on the quality and business value of your API. Once you’re done with the above phases, deliver the APIs on a basic portal to enable consumption by the right stakeholders, and get feedback. This step is all about getting some valuable feedback from an initial sample size of potential users. There are two parts to this step –
- Identifying early adopters who can provide you with feedback. You can leverage relationships with some of your existing customers, or scour for early adopters in channels where they usually hang out. This article from Lean Startup can be beneficial in understanding how to obtain early adopters
- Collecting and consolidating the feedback, and iterating on it.
This feedback needs to be processed and acted upon quickly, as part of the build-measure-learn philosophy. New feedback is again incorporated into the API design and development process, releasing new updates constantly.
Step 7: Marketing the API
Marketing your API is about educating your consumers on its benefits, and advocating for why it can be an integral part of the developer’s toolchain. At the end of the day, you want to make sure your APIs are being discovered and consumed, both internally and externally. In general, marketing is a combination of inbound content, including (includes inbound messaging and documentation) and outbound evangelism. This Nordic API ebook explains in detail the various stages to build a successful marketing program around your API, including various stages like establishing and maintaining developer relations, creating tutorials, and creating an advocacy program around your APIs.