Coinbase Reviews July 14 Service Outage: Misconfiguration Caused 50-Minute Disruption, User Funds Unaffected

Odaily reported that Coinbase has published a post-mortem report on the service outage that occurred on July 14, stating that the incident was caused by a misconfiguration of a critical network component triggered by a routine configuration update. The impact lasted approximately 50 minutes, during which user funds remained safe at all times.

Coinbase stated that at 12:34 Beijing time on July 14 (12:34 AM Eastern Time), the company deployed a routine configuration change to a shared production Kubernetes cluster hosting core infrastructure services. The update was intended to migrate to a new service deployment model to improve system performance and reliability.

However, due to a resource name conflict that was not detected by pre-production environment testing mechanisms, the configuration modification unexpectedly affected Istio Ingress Gateway-related Kubernetes resources within the cluster, rendering this network component unavailable. Approximately 3 minutes later, all inbound traffic to the cluster was disrupted, preventing internal services from accessing multiple infrastructure components.

Coinbase noted that because a large number of asynchronous workflows depend on these infrastructure services—including transaction settlement, fund transfers, and debit card transaction authorization—multiple business lines were affected. Affected services included:

Retail users were unable to complete off-chain transactions, deposits, and withdrawals; some in-progress transactions appeared stuck and were completed after service was restored;

Coinbase Card debit card transactions temporarily failed, credit card payments were unaffected, but some card management functions were unavailable;

Coinbase DEX's on-chain exchange services on Base and Solana were suspended;

Coinbase Exchange and Prime institutional clients experienced failures or delays in transfers and settlements;

Coinbase Developer Platform customers were unable to complete account opening, fund transfers, and deposit services.

Coinbase said that the Ingress Gateway was restored at 13:20 Eastern Time, and the incident was mitigated at 13:23. Subsequently, the system began processing backlogged tasks, with most services returning to normal quickly, and the remaining queues were completed within a few hours. During the recovery process, Coinbase faced two additional challenges. First, the deployment tool itself depended on the affected Ingress Gateway to operate, preventing the standard rollback process from being executed, creating a circular dependency where "the failure impacted the repair tool." Second, although the emergency recovery process (break-glass access) was effective, the permission verification and manual review procedures increased recovery time.

Coinbase stated that this incident did not involve asset security risks, but it highlighted the need for continuous optimization of large-scale financial infrastructure in areas such as automated deployment, system dependencies, and disaster recovery mechanisms. The company will further strengthen system resilience to reduce the likelihood of similar incidents occurring again.