Technology

S3 scales to petabytes a second on top of slow HDDs

Source

Hacker News

Published

Sep 24, 2025

TL;DR

AI Generated

Amazon Web Services' S3 storage service operates at a massive scale, handling 1 petabyte per second and 150 million requests per second using commodity hard drives. Despite hard drives being slower than SSDs due to mechanical limitations, S3 leverages their cost-effectiveness and sequential access optimization. Through techniques like erasure coding and massive parallelism, S3 achieves high throughput and tolerable latency, even under random access patterns. The system employs load-balancing strategies, such as shuffle sharding and the Power of Two Random Choices, to distribute data effectively and avoid hot spots. As S3 continues to grow, it focuses on rebalancing data, optimizing parallelism, and benefiting from workload decorrelation to enhance performance and scalability.

Read Full Article

Cloud storage company releases its 2025 hard drive reliability report — overall Annualized Failure Rate drops to 1.36%, 21 percentage points lower than last year

Backblaze's 2025 hard drive reliability report shows an overall Annualized Failure Rate of 1.36%, a significant improvement from previous years. The report covers 344,196 drives and 4,317 failures, providing a robust dataset for analysis. Some standout drives with low failure rates include the Seagate ST16000NM002J 16TB and the Western Digital WUH722626ALE6L4 26TB. However, models like the HGST HUH728080ALE600 8TB and the Seagate ST10000NM0086 10TB had notably higher failure rates in the fourth quarter. Backblaze attributes some failures to factors like vibration and firmware issues, highlighting ongoing improvements in drive technology despite recent price surges.

Tom's Hardware•

3 months ago

Distributed Authentication Framework Leveraging Multi-Party Computation In A Scalable Tree-Based Architecture (Univ. of Central Florida, Louisiana State)

Researchers from the University of Central Florida and Louisiana State University have introduced a new distributed authentication framework called AuthenTree, leveraging multi-party computation (MPC) in a scalable tree-based architecture. This framework aims to enhance security in chiplet-based heterogeneous systems by enabling secure chiplet validation without exposing raw signatures, distributing trust across multiple integrator chiplets. AuthenTree has shown minimal overhead, with an area as low as 0.48%, an overhead power under 0.5%, and an authentication latency below 1 microsecond, surpassing previous solutions. The framework is designed to address security threats in post-fabrication environments and establish a more efficient and scalable security solution for next-generation chiplet-based systems.

SemiEngineering•

7 months ago

Dr. L.C. Lu on TSMC Advanced Technology Design Solutions

Dr. L.C. Lu, a key figure at TSMC, focuses on design-technology co-optimization, packaging innovations, and AI-driven methodologies for next-gen semiconductor systems. TSMC emphasizes DTCO and DDCL innovations for scaling from N5 to A14 nodes, with NanoFlex and NanoFlex Pro architectures offering efficiency gains. N2P and N2U nodes incorporate advanced DTCO and power delivery optimizations, with hybrid dual-rail architectures achieving significant energy savings. TSMC collaborates with EDA partners for AI integration, enhancing productivity and design quality. Advanced packaging technologies like CoWoS and SoIC play a crucial role in enabling AI scaling, with memory bandwidth and interconnect performance scaling aggressively. TSMC addresses power delivery and thermal management challenges in AI systems through advanced solutions. TSMC's advancements in design methodologies and AI-driven automation promise improved productivity and scalability in chip-package co-design.

SemiWiki•

5 hours ago

MindsEye's sabotage mission is being slammed as dull and pointless

Build A Rocket Boy accuses individuals of sabotaging MindsEye's launch, spending over €1 million on damaging efforts. The studio integrates the controversy into a new in-game mission, Blacklist, to showcase evidence of sabotage to players. However, reports describe the mission as lackluster and failing to deliver a compelling narrative. Critics attribute the launch issues to internal problems, such as management and design decisions, casting doubt on the sabotage claims. The saga continues as MindsEye's post-launch turmoil unfolds.

TweakTown•