line·Jun 23, 2026·4 min readCurated summary

Total Capacity Exceeds 1 EB! How Do You Connect Two HDFS Systems with Different Histories? Challenges and Design Decisions in Data Platform Integration

LY Corporation’s Tech-Verse 2026 article examines how its former LINE and Yahoo Japan organizations operated HDFS platforms exceeding one exabyte in total capacity. Although both platforms used Hadoop at scale, their access models, namespace architectures, permission systems, and operational practices differed substantially. The article argues that large-scale data platforms must be designed around actual usage patterns, not just storage capacity, and previews how the two environments were later connected after organizational integration. ## Different Operating Models - Former LINE built a unified analytics environment for broad, cross-departmental data use. - Users accessed data through a web portal that managed catalogs, permissions, and role-based approval workflows rather than interacting directly with HDFS or Apache Ranger. - BI tools, reporting systems, and ETL pipelines supported diverse use cases, but integrating multiple existing clusters made operations complex. - Former Yahoo Japan evolved from a limited-purpose Hadoop deployment into a company-wide platform. - Its user interfaces and access methods were intentionally restricted, making the system easier to stabilize and support. - Yahoo Japan retained HDFS-style POSIX permissions, which limited flexibility compared with newer data-governance models. ## Different HDFS Architectures - Both platforms split their storage across multiple namespaces to overcome NameNode scaling limitations. - Each namespace used two to four NameNodes for redundancy, but the way namespaces were exposed differed: - LINE used **ViewFS**, requiring clients to maintain mount-table configurations. - Yahoo Japan used **Router-Based Federation (RBF)**, allowing routers to direct requests to the correct NameNode. - LINE shared DataNodes across namespaces, improving resource efficiency but increasing operational complexity. - LINE also had mixed NameNode and DataNode versions because several legacy platforms had been consolidated. - Yahoo Japan’s RBF design included Observer NameNodes to distribute read load. - These architectural differences affected later federation work, including connection endpoints, client configuration, network reachability, and permission management. ## Capacity and Network Challenges at LINE - Data growth exceeded forecasts, causing HDFS capacity shortages before new servers could be delivered. - Older servers were temporarily reused, leading to frequent node additions and removals. - Large changes in node count triggered HDFS Balancer activity and block redistribution, generating substantial network traffic. - Network engineers therefore had to coordinate closely with the Hadoop operations team during infrastructure changes. ## NameNode Metadata and Small-File Problems - As file and block counts increased, NameNode heap usage and processing load grew. - Larger heaps also increased garbage-collection times, making NameNodes slower and less stable. - The team analyzed regularly dumped FSImage data stored in Hive tables to identify users, paths, file counts, block counts, and data volumes. - They prioritized tables containing many small files where file compaction could significantly reduce block counts without requiring data deletion or schema changes. - File merging reduced both NameNode metadata pressure and the number of HDFS operations, improving response times for jobs. ## Namespace-Specific Load Patterns - Different namespaces experienced different types of pressure. - Temporary-file namespaces saw frequent Spark staging-file creation and deletion, producing repeated metadata updates requiring NameNode write locks. - When HDFS Balancer moved blocks, read-lock activity increased and could delay file creation and deletion. - Increasing Balancer parallelism initially worsened contention. - The team reduced parallelism to a level compatible with available DataNode disk capacity, balancing migration speed against cluster impact. ## Connecting the Two Platforms - Organizational integration introduced additional challenges beyond storage: - Determining which platform and entry point users should access - Reconciling different permission-management models - Establishing data-transfer paths between platforms - LINE’s ViewFS model depends on correctly distributed client mount tables. - Yahoo Japan’s RBF model depends on reliable, scalable, and reachable router infrastructure. - These differences directly influence cross-platform data movement, including transfers using DistCP. Large HDFS environments should be managed according to real workload behavior, namespace characteristics, and operational dependencies. Capacity planning alone is insufficient; teams should monitor metadata growth, small-file patterns, lock contention, balancing traffic, network effects, and the distinct access models of each platform.

Read original(opens in new tab)

Data PlatformApache SparkHdfsApache Hadoop+3

toss·Mar 25, 2026·3 min readCurated summary

Metric Review, Driving Execution

Metric Review is Toss Place’s weekly operating system for turning data insights into product and business action. By connecting OKRs to a hierarchy of driver metrics, analysts continuously detect risks, test hypotheses, and encourage execution rather than merely reporting results. The approach has improved data literacy and helped teams contribute directly to company-level Key Results. ### Building a Data-Literate Organization - Toss Place aims for everyone—not only analysts—to perform effective analysis. - The Data Platform Team strengthens data quality and infrastructure, while the Data Analysis Team provides domain knowledge and delivery capabilities. - Analysts are expected to develop three complementary skills: - Technical expertise with data and analysis tools - Logical communication - Deep product and business knowledge ### Why Metric Review Matters - Metrics serve as a shared language for aligning teams around organizational goals. - Metric Review helps teams identify: - Whether goals are on track - Emerging risks - New opportunities - Analysts act as **Metric Owners**, providing insights that support better decisions and following through until actions and outcomes are verified. ### Operating Model #### OKR-Linked Metric Hierarchy - Company-level Key Results flow down to team and silo-level Key Results. - The levers that influence each team’s KR become its driver metrics. - This hierarchy provides the structure for identifying opportunities and threats. #### A Continuous Analysis Cycle - The operating cycle is: - Goal setting → hypothesis formation → validation and execution → insight discovery - Metric Review translates this into: - Metric analysis → hypothesis testing → insight sharing → driving action - Exploratory data analysis (EDA) is also conducted when metric movements suggest deeper questions. #### Weekly Consistency - Reviewing metrics weekly helps teams detect small changes before they become significant. - Regular analysis also builds domain knowledge by requiring analysts to understand why metrics rise or fall. - Monthly or occasional reporting may explain past performance but often misses the window for timely action. ### Examples of Business Impact #### Growth Tribe: Establishing Shared Metrics - Weekly metric reviews initially focused on reporting performance and interpretation. - Over time, the practice changed how teams worked: - Designers defined product hypotheses around target metrics and incorporated logging requirements into designs. - Backend developers collaborated with analysts on analysis-friendly data structures. - Client developers prioritized measurable events when implementing logs. - Product Owners combined qualitative feedback with quantitative results to determine whether goals were on track. - This created a feedback loop that contributed to successful product launches and improved company metrics. #### POS Tribe: Segment-Specific Solutions - POS adoption varied significantly across partner dealerships. - Analysts used clustering to identify groups with different adoption patterns. - Product teams combined cluster analysis with interviews to design tailored interventions: - Low-adoption groups received stronger education and onboarding. - High-adoption groups received simplified store creation and installation flows. - Segment-specific actions accelerated POS expansion more effectively than a single broad solution. #### Supply Chain: Forecast-Based Optimization - Because Toss Place manufactures and distributes hardware, supply-chain metrics are strategically important. - Analysts and the SCM team monitored: - Device shipments - Market installation rates - Inventory and ordering forecasts - Potential improvement areas - Hypothesis-driven actions helped optimize distribution and reduce costs. ### How the Organization Changed - Analysts became Metric Owners rather than report writers. - Product teams began asking, “Which metric should we move?” before asking what to build. - Business teams increasingly aligned strategies using quantitative evidence. - Repeated Metric Reviews strengthened organization-wide data literacy and contributed to meaningful company Key Result achievement. The practical recommendation is to evaluate analysis by whether it leads to measurable action. Teams should structure problems, create testable hypotheses, define follow-up metrics, and maintain a consistent review rhythm until the execution loop is closed.

Read original(opens in new tab)

Data PlatformDatabase DesignData AnalysisOkrs+4

toss·Oct 30, 2025Original article

Toss People: Designing a structure (opens in new tab)

Data architecture is evolving from a reactive "cleanup" task into a proactive, end-to-end design process that ensures high data quality from the moment of creation. In fast-paced platform environments, the role of a Data Architect is to bridge the gap between rapid product development and reliable data structures, ultimately creating a foundation that both humans and AI can interpret accurately. By shifting from mere post-processing to foundational governance, organizations can maintain technical agility without sacrificing the integrity of their data assets. **From Post-Processing to End-to-End Governance** * Traditional data management often involves "fixing" or "matching puzzles" at the end of the pipeline after a service has already changed, leading to perpetual technical debt. * Effective data architecture requires a culture where data is treated as a primary design object from its inception, rather than a byproduct of application development. * The transition to an end-to-end governance model ensures that data quality is maintained throughout its entire lifecycle—from initial generation in production systems to final analysis and consumption. **Machine-Understandable Data and Ontologies** * Modern data design must move beyond human-readable metadata to structures that AI can autonomously process and understand. * The implementation of semantic-based standard dictionaries and ontologies reduces the need for "inference" or guessing by either humans or machines. * By explicitly defining the relationships and conceptual meanings of columns and tables, organizations create a high-fidelity environment where AI can provide accurate, context-aware responses without interpretive errors. **Balancing Development Speed with Data Quality** * In high-growth environments, insisting on "perfect" design can hinder competitive speed; therefore, architects must find a middle ground that allows for future extensibility. * Practical strategies include designing for current needs while leaving "logical room" for anticipated changes, ensuring that future cleanup is minimally disruptive. * Instead of enforcing rigid rules, architects should design systems where following the standard is the "path of least resistance," making high-quality data entry easier for developers than the alternative. **The Role of the Modern Data Architect** * The role has shifted from a fixed, corporate function to a dynamic problem-solver who uses structural design to solve business bottlenecks. * A successful architect must act as a mediator, convincing stakeholders that investing in a 5% quality improvement (e.g., moving from 90 to 95 points) provides significant long-term ROI in decision-making and AI reliability. * Aspiring architects should focus on incremental structural improvements, as any data professional who cares about how data functions is already operating on the path to data architecture.

Data PlatformData GovernanceData ArchitectureData Modeling+4