Blog posts about Distributed Systems

Introduction Capacity is a key component of reliability. Uber's services require enough resources in order to handle daily peak traffic and to support our different kinds of business units. These services are deployed across different cl...

Introduction Cadence is a multi-tenant orchestration framework that helps developers at Uber to write fault-tolerant, long-running applications, also known as workflows. It scales horizontally to handle millions of concurrent executions ...

Introduction Each day, Uber moves millions of people around the world and delivers tens of millions of food and grocery orders. This generates a large number of financial transactions that need to be stored with provable completeness, co...

Background Apache Kafka® is widely used across Uber’s multiple business lines. Take the example of an Uber ride: When a user opens up the Uber app, demand and supply data are aggregated in Kafka queues to serve fare calculations. When a ...

Scaling our data infrastructure with lower hardware costs while maintaining high performance and service reliability has been no easy feat. To accommodate the exponential growth in both Data Storage and Analytics Compute at Uber, the Dat...

  Introduction The Fulfillment Platform is a foundational Uber domain that enables the rapid scaling of new verticals. The platform handles billions of database transactions each day, ranging from user actions (e.g., a driver starting a ...

Co-authors: Andy Li and Hongbin Wu Indexing plays the key role in modern search engines for fast and accurate information retrieval, and the ability to swi...

Uber recently launched a new capability: Ads on UberEats. With this new ability came new challenges that needed to be solved at Uber, such as systems for ad auctions, bidding, attribution, reporting, and more. This article focuses on how...

Part one of a series on how we provide powerful, automated, and scalable data privacy and security engineering capabilities at Airbnb.

Co-authors: Konstantin V. Shvachko, Chen Liang, and Simbarashe Dzinamarira LinkedIn runs its big data analytics on Hadoop. During the last five years, the ...

OMNI is an intuitive, homegrown platform that supports message creation, processing, and distribution to engage our guests and hosts at the…

Access Control at scale for a complex product

Co-authors: Hunter Lee and Dru Pollini LinkedIn was built to help professionals achieve more in their careers, and every day millions...

Learn about the background, challenges, and future of Airbnb’s distributed scheduling and queueing system.

We regularly play host to a series of meetups here at the LinkedIn office in the Empire State Building. Open to the community, these events cover a range o...

In this blog post, we’ll share how we migrated Espresso, LinkedIn’s distributed data store, to a new Netty4-based framework and achieved a large performanc...

Pinot is an open source, scalable distributed OLAP data store that entered the Apache Incubation recently. Developed at LinkedIn, it...

How we built a generic idempotency framework to achieve eventual consistency and correctness across our payments micro-service…