Data Analysis

13 posts

cloudflare3 min readCurated summary

How the 2026 World Cup affected Internet traffic

The 2026 World Cup demonstrated how a rare shared global event can reshape Internet behavior across countries. Cloudflare used traffic data from its global network to show that match timing, teams, and major storylines significantly affected online activity. Overnight matches often doubled traffic, while games during normal active hours could reduce browsing as people focused on watching. ## Measuring “Normal” Internet Activity - Cloudflare established a baseline using the median traffic from the previous four weeks. - Traffic changes were expressed as a log₂ ratio: - `0` means normal activity. - `+1` means traffic doubled. - `−1` means traffic fell to half its normal level. - This approach made traffic changes comparable between high-volume and low-volume countries. ## Kickoff Time Shaped Online Behavior - The largest traffic changes occurred during matches played between midnight and 8 a.m. local time. - Fans staying awake or waking early caused traffic to rise well above normal, sometimes by more than 2x. - Daytime matches had little effect because viewers were likely already online. - Evening matches produced a smaller increase on weekdays, but could also cause traffic to fall when people stopped browsing to watch. - In Bosnia and Herzegovina: - A 2 a.m. match caused traffic to more than double. - An evening match reduced traffic to roughly 70% of normal. - Brazil and Japan showed opposite patterns during the same match: - Japan’s overnight viewing produced traffic around twice its usual level. - Brazil’s daytime match coincided with traffic about 40% below normal. ## Matches With the Greatest Global Impact - Cloudflare measured traffic during the two hours after kickoff. - For each match, it calculated the absolute deviation from normal across countries and then used the median result. - Simultaneous matches were excluded because their effects could not be reliably separated. - The most impactful match was Argentina vs. Switzerland, with traffic changing by a factor of about 1.26. - France vs. Spain followed at 1.21. - The highest-impact games included quarterfinals, knockout matches, and some round-of-32 games—not only the final or semifinals. ## Argentina Led the Teams Drawing Global Attention - Argentina produced the largest average worldwide traffic impact, at 1.17x normal activity. - France, Brazil, Portugal, Morocco, Spain, and Norway also ranked highly. - Argentina’s appeal was linked to its status as defending champion and the possibility that the tournament could be Lionel Messi’s final World Cup. - Haiti and Iraq appeared as outliers because matches involving major teams caused especially large changes relative to their usual traffic levels. ## More Traffic to Sports Betting Websites - Requests to gambling-related websites increased after the tournament began compared with the preceding month. - Before the World Cup, betting traffic followed a regular weekly cycle. - Once matches began nearly every day, that weekly pattern flattened into a more consistent level of activity. Overall, the data shows that the World Cup affected Internet use in two distinct ways: inconvenient kickoff times brought previously offline fans online, while matches during normal usage hours often diverted people away from their usual browsing. For analyzing global events, normalized, country-by-country traffic measurements provide a clearer picture than raw request volumes alone.

Read original(opens in new tab)
line4 min readCurated summary

Unifying Analysis Through the Power of Analytics Agents: Work Innovation and Role Transformation in the Generative AI Era at a Professional Organization

PJ One Piece is LY Corporation’s initiative to connect business questions, data analysis, insight generation, and next-action planning through generative AI. Its analysis agent reduced typical turnaround times from about two weeks to roughly 10 minutes, enabling hundreds of analyses each month and adoption by more than half of an early-adopter business unit. The project treats AI not as a chat interface, but as an analysis platform that connects data, knowledge, people, and organizational processes. ## Three Disconnects Behind the Project - **Business and data:** Even with a data warehouse and BI tools, business users still needed to understand SQL, tables, column definitions, KPI rules, and result interpretation. - **Within the analysis process:** Task definition, analysis design, execution, review, and action planning were often handled by different people or tools, causing context loss, rework, delays, and inconsistent quality. - **Across domains:** Useful analysis patterns and domain knowledge remained isolated because services used different KPIs, table structures, business assumptions, and review criteria. ## The Analysis Agent as a Connector - Users ask questions in natural language without needing to know SQL or database structures. - The agent: - Clarifies the business objective and missing assumptions. - Finds relevant data and creates an analysis plan. - Executes queries and specialized analyses. - Interprets results and produces visualizations or reports. - Suggests further analysis and possible next actions. - The platform consists of: - A user-facing application. - An LLM-based agent for reasoning and tool use. - Tools for SQL, Python, document search, and visualization. - A knowledge base containing domain information, skills, and table metadata. - Logging, feedback, monitoring, and evaluation systems. - Domain knowledge is added through a plugin-like structure, while logs and feedback continuously improve the system. ## Turning Business Questions into Analysis Requirements - Natural-language questions often leave important assumptions unspecified, such as: - Target population or campaign definition. - Analysis period and comparison group. - KPI definitions. - Aggregation level. - Exclusion conditions. - Rather than requiring users to write detailed prompts, the agent uses domain knowledge to determine what can be inferred and asks only about unresolved points. - Knowledge bases document service context, KPI definitions, aggregation cautions, policy information, and review requirements. - Table metadata explains available tables, columns, appropriate use cases, samples, partition requirements, and usage restrictions. ## Reaching Data Safely and Reliably - Table metadata is revealed progressively: - The agent first narrows down relevant tables. - It then retrieves detailed definitions and usage rules only for those tables. - Analysis-oriented wide tables or logical views combine transaction data with commonly needed attributes, reducing complicated joins and SQL-generation errors. - SQL is checked before and after execution to enforce: - `SELECT`-only access. - Approved tables and usage rules. - Required partition conditions. - Restrictions on sensitive or personal data. - Result-size limits. - These guardrails allow the agent to perform analysis flexibly without exposing data or infrastructure to unnecessary risks. ## Preserving Context Across the Analysis Process - PJ One Piece uses a supervisor-style multi-agent architecture. - A main agent maintains: - The user’s request and business objective. - The current analysis plan. - Findings and constraints discovered so far. - Remaining questions and decision points. - Specialized sub-agents handle tasks such as statistical testing, time-series analysis, clustering, and independent review. - This separates complex or specialized work from the main context while preserving overall continuity. - Progress updates expose discoveries, design decisions, data limitations, and constraints so users can adjust direction during longer analyses. ## Building Reusable Organizational Capability - Logs record agent actions, assumption checks, analysis designs, generated SQL, errors, and outputs. - User and analyst feedback helps identify whether improvements are needed in prompts, tools, data, or reusable skills. - Repeated workflows are formalized as skills, including: - General-purpose methods such as time-series and clustering analysis. - Domain-specific workflows such as monthly reporting or policy monitoring. - Skills document required assumptions, comparison axes, cautions, and interpretation methods. - Over time, isolated domain knowledge becomes reusable organizational analysis capability. ## Business Impact - In early deployment, the platform expanded data use beyond data scientists to product owners and frontline employees. - More than half of the participating business unit’s members use it. - Analysis turnaround fell from an average of approximately two weeks to about 10 minutes. - The platform now supports hundreds of analyses per month and serves as a daily starting point for business questions. PJ One Piece’s main recommendation is to design AI analysis as an end-to-end operating platform—not merely an automated SQL or chatbot tool. Combining structured domain knowledge, safe data access, contextual multi-agent workflows, reusable skills, and continuous evaluation can make analysis faster while steadily improving its quality and organizational reach.

Read original(opens in new tab)
gitlab2 min readCurated summary

Claude Opus 4.8 on GitLab: Complex agentic work, less disruption

Claude Opus 4.8 is now available in GitLab Duo Agent Platform for complex, multi-step coding and agentic workflows. The model is designed to follow instructions more precisely over extended autonomous runs, reducing the need for human correction and intervention. It also supports mid-conversation system-prompt updates without invalidating the prompt cache, making it better suited to changing contexts. ## Improved Long-Horizon Agentic Execution - Handles complex workflows involving multiple tools and extended sequences. - Provides more comprehensive planning and reasoning. - Produces more accurate end states with fewer interruptions or redirects. - Interprets instructions more precisely than previous models. - Supports professional tasks beyond coding, including: - Document drafting - Data analysis - Structured knowledge work ## Mid-Conversation System Prompts - System instructions can change during an active session. - Prompt-cache validity is preserved when context changes. - This is useful when: - Files change on disk - Token budgets shift - User context is updated - Asynchronous information arrives ## Availability and Pricing - Available through model selection in Agentic Chat and across GitLab Duo agent workflows. - Runs on GitLab Credits. - GitLab Premium and Ultimate subscribers can use included credits. - New users can access it through a GitLab Duo Agent Platform trial or the GitLab Free tier. Teams seeking more reliable autonomous coding and documentation workflows can use Opus 4.8 in GitLab Duo to reduce manual oversight while agents complete longer, more complex tasks.

Read original(opens in new tab)
toss5 min readCurated summary

Introducing Toss Place's Data Bot 'PANDA': How every team member works like a data expert

PANDA, short for Place Analytics & Data, is Toss Place’s AI data-analysis assistant, designed to let employees retrieve and interpret approved data without waiting for analysts. It was created after the team found that 70% of data requests involved simple metric lookups rather than complex analysis. The project’s main conclusion is that reliable AI analytics depends less on prompting alone and more on standardized data, business definitions, controlled table selection, and iterative validation. ## Why Toss Place Built PANDA - Employees previously relied on analysts to search dashboards, write SQL, or manually investigate data requests. - PANDA provides self-service access within each employee’s security permissions. - It reduces routine extraction work for analysts, allowing them to focus on deeper analysis. - The goal is to establish a stronger culture of “data democracy,” where employees can access and use data immediately. ## Challenges with a Simple AI Chatbot Early experiments showed that asking an AI model to search all company data produced unreliable and expensive results: - Referencing thousands of tables and internal documents consumed excessive tokens. - The model sometimes selected different tables for identical questions, producing inconsistent answers. - It often misunderstood business definitions. For example, “active stores” could mean stores with completed installations or stores that had processed payments. - Inefficient SQL caused unnecessary Snowflake data scans and higher warehouse costs. ## Standardized Data Marts as a Single Source of Truth Toss Place collaborated across its Data Analysis and Data Platform teams to establish reliable standard data marts. - Core concepts, such as store information, were consolidated into standardized tables. - Naming conventions made table and column purposes easier for both people and AI to understand: - Tables follow `{mart_type}_{domain}_{subject}`, such as `fact_device_error_log`. - Columns follow `{prefix}_{entity}_{attribute}_{suffix}`, such as `is_merchant_active`. - Table and column descriptions were documented comprehensively. - The standardization effort reduced ambiguity by ensuring the same business concepts were represented consistently. ## Connecting Business Language to Data Data structures alone could not answer questions about terms such as “installed store” or “store category.” - Domain-specific terms and metric definitions were documented. - These business definitions were linked to the relevant standard data marts. - Data analysts helped reconcile differing interpretations and establish shared organizational definitions. - This gave PANDA the context needed to apply the correct business logic. ## Scoring and Ranking for Reliable Table Selection PANDA limits its search to well-managed tables and uses dbt tags to import selected metadata into a Manifest file. - Tables are ranked using: - **Similarity score:** Based on relationships between the question and table, including table-name matches and description relevance. - **Hierarchy weight:** Reflecting the reliability of the data layer. - The final score is calculated as: `similarity score × hierarchy weight` - Weights are assigned as follows: - Company-wide SSOT metrics: ×4 - Validated standard marts: ×3 - Domain analysis marts: ×2 - Raw bronze data and logs: ×1 - This improves accuracy, consistency, and trustworthiness while reducing unnecessary warehouse exploration. ## Agentic Loop for Querying and Validation Rather than expecting a correct answer in one attempt, PANDA uses an agentic loop. - It selects appropriate tools based on the question. - It explores tables, generates and executes queries, and reviews the results. - If the result appears inaccurate, it can inspect the schema again, modify the query, and retry. - If necessary, it asks the user for clarification. - This approach allows PANDA to handle exceptions dynamically instead of relying only on predefined rules. ## Answers Designed for Practical Use PANDA structures responses so users can understand and apply the results: - **Result:** The requested data or metric. - **Query criteria:** The period, filters, and aggregation method used. - **Insight:** An interpretation that can support practical decisions. This makes PANDA more than a number-retrieval chatbot; it also exposes part of the reasoning process normally provided by a data analyst. ## Adoption and User Response PANDA quickly became part of everyday work at Toss Place. - One-third of employees used it on its first day. - Half of the organization had tried it within a week. - More than 4,000 messages were exchanged during that period. - Current adoption is approximately 70%. - Employees reported feeling more comfortable asking small questions and using data while away from their desks. - Users particularly valued receiving insights alongside raw figures. - Unexpectedly, developers and even data professionals used PANDA actively, suggesting that its answers achieved a meaningful level of trust. ## Future Development PANDA was developed and launched in just one month, but the team plans further improvements. - Increase data coverage to more than 90%. - Raise answer accuracy above 97%. - Use real user questions, follow-up behavior, and abandonment patterns to identify unmet needs. - Expand beyond basic data retrieval to reduce more of the data team’s workload. PANDA’s central lesson is that effective enterprise AI does not require the most complicated technology. It requires solving a real business pain point with trustworthy data foundations, clear definitions, and a workflow that users can rely on.

Read original(opens in new tab)
toss3 min readCurated summary

Metric Review, Driving Execution

Metric Review is Toss Place’s weekly operating system for turning data insights into product and business action. By connecting OKRs to a hierarchy of driver metrics, analysts continuously detect risks, test hypotheses, and encourage execution rather than merely reporting results. The approach has improved data literacy and helped teams contribute directly to company-level Key Results. ### Building a Data-Literate Organization - Toss Place aims for everyone—not only analysts—to perform effective analysis. - The Data Platform Team strengthens data quality and infrastructure, while the Data Analysis Team provides domain knowledge and delivery capabilities. - Analysts are expected to develop three complementary skills: - Technical expertise with data and analysis tools - Logical communication - Deep product and business knowledge ### Why Metric Review Matters - Metrics serve as a shared language for aligning teams around organizational goals. - Metric Review helps teams identify: - Whether goals are on track - Emerging risks - New opportunities - Analysts act as **Metric Owners**, providing insights that support better decisions and following through until actions and outcomes are verified. ### Operating Model #### OKR-Linked Metric Hierarchy - Company-level Key Results flow down to team and silo-level Key Results. - The levers that influence each team’s KR become its driver metrics. - This hierarchy provides the structure for identifying opportunities and threats. #### A Continuous Analysis Cycle - The operating cycle is: - Goal setting → hypothesis formation → validation and execution → insight discovery - Metric Review translates this into: - Metric analysis → hypothesis testing → insight sharing → driving action - Exploratory data analysis (EDA) is also conducted when metric movements suggest deeper questions. #### Weekly Consistency - Reviewing metrics weekly helps teams detect small changes before they become significant. - Regular analysis also builds domain knowledge by requiring analysts to understand why metrics rise or fall. - Monthly or occasional reporting may explain past performance but often misses the window for timely action. ### Examples of Business Impact #### Growth Tribe: Establishing Shared Metrics - Weekly metric reviews initially focused on reporting performance and interpretation. - Over time, the practice changed how teams worked: - Designers defined product hypotheses around target metrics and incorporated logging requirements into designs. - Backend developers collaborated with analysts on analysis-friendly data structures. - Client developers prioritized measurable events when implementing logs. - Product Owners combined qualitative feedback with quantitative results to determine whether goals were on track. - This created a feedback loop that contributed to successful product launches and improved company metrics. #### POS Tribe: Segment-Specific Solutions - POS adoption varied significantly across partner dealerships. - Analysts used clustering to identify groups with different adoption patterns. - Product teams combined cluster analysis with interviews to design tailored interventions: - Low-adoption groups received stronger education and onboarding. - High-adoption groups received simplified store creation and installation flows. - Segment-specific actions accelerated POS expansion more effectively than a single broad solution. #### Supply Chain: Forecast-Based Optimization - Because Toss Place manufactures and distributes hardware, supply-chain metrics are strategically important. - Analysts and the SCM team monitored: - Device shipments - Market installation rates - Inventory and ordering forecasts - Potential improvement areas - Hypothesis-driven actions helped optimize distribution and reduce costs. ### How the Organization Changed - Analysts became Metric Owners rather than report writers. - Product teams began asking, “Which metric should we move?” before asking what to build. - Business teams increasingly aligned strategies using quantitative evidence. - Repeated Metric Reviews strengthened organization-wide data literacy and contributed to meaningful company Key Result achievement. The practical recommendation is to evaluate analysis by whether it leads to measurable action. Teams should structure problems, create testable hypotheses, define follow-up metrics, and maintain a consistent review rhythm until the execution loop is closed.

Read original(opens in new tab)
google3 min readCurated summary

Improving breast cancer screening workflows with machine learning

Google Research’s AIMS studies evaluated whether machine learning could support the UK’s mammography double-reading workflow. Across five NHS screening services, the AI system improved cancer detection sensitivity without reducing specificity, detected some cancers missed by human readers, and processed cases far faster. The studies also showed that safe deployment requires local calibration, monitoring for distribution shifts, and evaluation of how clinicians interact with AI results. ## NHS Screening Challenges - The UK NHS uses two human readers for each mammogram, with arbitration when their assessments require review. - A projected shortage of clinical radiologists—currently around 30% and expected to reach 40% by 2028—threatens the sustainability of this model. - AI could help increase detection while reducing pressure on radiology services. ## Study 1: Standalone Performance - The retrospective evaluation included mammograms from approximately 116,000 women screened across five NHS services. - The services represented three different double-reading and arbitration workflows. - AI thresholds were calibrated separately for each service to account for local populations and procedures. - Performance was measured against the original first reader using a 39-month follow-up period, including interval and subsequent-round cancers. - Researchers also assessed: - Comparisons with second and consensus readers - Lesion-level localization - Performance across demographic groups ## Study 1: Results - Cancer detection increased from 7.54 to 9.33 cases per 1,000 women. - The AI system achieved significantly higher sensitivity than the original first reader without compromising specificity. - It detected 25% of interval cancers missed by the original double-reading process. - Performance was especially strong for invasive cancers and women attending their first screening. - The study found no notable systematic disparities by age, ethnicity, breast density, or socioeconomic status. ## Prospective Technical Deployment - The system was deployed non-interventionally at 12 sites across two London screening services. - It processed 9,266 cases over roughly two months per service. - Mammograms were pseudonymized and sent to a secure Google Cloud-based system. - Median AI processing time was 17.7 minutes, compared with more than two days for the first human read. - The deployment detected a distribution shift between historical training data and current clinical data. - Researchers adjusted operating points during deployment to maintain safe and appropriate recall rates for local workflows. ## Study 2: AI in the Double-Reading Workflow - The second study examined how human readers performed when using AI as part of arbitration, rather than evaluating AI in isolation. - Twenty-two readers reviewed thousands of cases using real screening-service rules. - Two workflows were compared: - **Standard care:** decisions from the historical first and second human readers - **AI-enabled care:** the historical first-reader decision paired with the AI decision - This design aimed to assess the practical effects of replacing the second human read with an AI reader. The findings support AI as a potential second reader in breast cancer screening, but broader prospective clinical validation is still needed. Successful adoption should include phased deployment, local calibration, continuous monitoring, and careful evaluation of human-AI decision-making.

Read original(opens in new tab)
google3 min readCurated summary

Where wild things roam: Identifying wildlife with SpeciesNet

SpeciesNet is an open-source AI tool that identifies wildlife in camera-trap images, making large-scale monitoring faster and more practical. Trained on more than 65 million labeled images, it can classify nearly 2,500 animal categories and process tens of thousands of images per day. Its adoption by researchers, governments, and conservation groups is expanding wildlife research and enabling more responsive conservation efforts. ## A New Era for Wildlife Monitoring - Motion-triggered camera traps generate enormous volumes of images, often far beyond what human teams can classify manually. - Automated identification helps researchers: - Track population health and changes. - Study migration and climate-related movement. - Estimate population sizes. - Detect rare or endangered species. - SpeciesNet uses deep learning to identify animals in camera-trap photos, accelerating analysis and improving wildlife-management decisions. - The tool is part of Google Earth AI, a collection of geospatial AI tools intended to support environmental and conservation work. ## SpeciesNet’s Training and Performance - SpeciesNet classifies 2,498 categories of mammals, birds, reptiles, and other animals. - It works with MegaDetector, another open-source model that identifies which images and pixels contain animals. - The system provides: - Species names. - Confidence scores. - Multiple identifications when several animals appear in one image. - Processing capacity is approximately: - 30,000 images per day on a standard laptop. - 250,000 or more images per day on a low-end gaming GPU. - SpeciesNet was trained on more than 65 million images from Wildlife Insights and public repositories. - On held-out camera-trap projects, it: - Detected animals in 99.4% of relevant images. - Reached species-level classification 83% of the time. - Produced correct species-level predictions in 94.5% of those cases. - Human-verified labels from Wildlife Insights can be reused as additional training data, creating a feedback loop for improving the model. ## Conservation Projects Using SpeciesNet - **Snapshot Serengeti:** Researchers can analyze roughly 11 million images collected since 2010 in just days, rather than relying exclusively on citizen scientists. Field processing also allows cameras to be redeployed based on recent sightings. - **Wildlife Observatory of Australia:** The organization trained a regional version of SpeciesNet to recognize Australian species missing from the original label set, including musky rat-kangaroos and orange-footed scrubfowl. - **Idaho Department of Fish and Game:** SpeciesNet serves as a first-pass classifier for images of deer, elk, black bears, coyotes, and other wildlife, speeding up human verification. - **Public and private platforms:** Tools including Animl and AddaxAI have integrated SpeciesNet, while companies such as Okala use it alongside Google’s Perch audio model to monitor biodiversity in Africa. - The model has also supported studies of pumas and ocelots in Colombia, cassowaries in Australia, and lions and elephants in Tanzania. SpeciesNet demonstrates how open-source AI can turn massive camera-trap datasets into usable scientific evidence. Its strongest role is as a scalable first-pass system combined with human review, while regional adaptations can extend its usefulness to local and threatened species.

Read original(opens in new tab)
grammarlyOriginal article

Scaling Always-On Writing Support at Florida Atlantic University (opens in new tab)

Florida Atlantic University successfully implemented Grammarly as a campus-wide writing support tool to improve student outcomes while reducing the grading burden on faculty. By integrating the software directly into students' existing workflows, the university observed significant gains in course completion, retention, and average GPAs across diverse student populations. This strategic approach demonstrates that providing low-friction, automated feedback on mechanics allows students to submit stronger drafts and enables instructors to focus their critiques on higher-order ideas and arguments. ### Strategic Integration and Low-Friction Access * The university opted for a campus-wide rollout that prioritized instructor autonomy, allowing faculty to decide how to best onboard students within their specific writing-intensive courses. * The tool was integrated into students' existing digital ecosystems, including Microsoft Word, Google Docs, Outlook, and Gmail, as well as via browser extensions to ensure adoption didn't require new platforms. * Grammarly was positioned as a “first line of instruction” for recurring mechanical issues, acting as a private, on-demand support system that reduced the friction typically associated with seeking help. ### Measurable Impact on Student Success * Data analysis revealed a +5.3-point persistence lift, with Grammarly users reaching a 79.5% completion rate compared to 74.2% for their peers. * Significant gains were noted in "gateway" courses that unlock further degree progress, with completion rates rising by +3.3 points in writing-intensive courses and +4.3 points in STEM sections. * Frequent users achieved an average GPA of 3.69, which was 0.4 points higher than non- or low-frequency users, even when controlling for baseline demographics and prior academic performance. ### Continuous Writing Performance Gains * Writing performance scores increased by +2.14 points in Fall 2023 and +1.28 in Fall 2024, suggesting that the tool supports ongoing skill development rather than just short-term corrections. * Continuous users showed a year-over-year improvement in writing scores from 76.7 to 81.3. * The visibility of recurring patterns in the students' own drafts allowed them to make sustainable changes to their writing habits over multiple terms. ### Shift in Faculty Instruction * The implementation acted as a "classroom pressure release," making student drafts easier to read by filtering out repeated mechanical errors. * Instructors were able to shift their focus away from basic proofreading and toward guiding students on complex structural and argumentative elements. * The university utilized the rich usage datasets provided by the software to inform broader student-success initiatives and institutional analysis. To replicate these results, institutions should focus on broad access and low-barrier implementation, ensuring the tool meets students where they already write. Anchoring the rollout to specific momentum metrics—such as first-year retention and STEM course completion—allows administrators to track the tangible impact of the technology on institutional goals.

tossOriginal article

Improving Business Data Literacy: (opens in new tab)

Toss’s Business Data Team addressed the lack of centralized insights into their business customer (BC) base by building a standardized Single Source of Truth (SSOT) data mart and an iterative Monthly BC Report. This initiative successfully unified fragmented data across business units like Shopping, Ads, and Pay, enabling consistent data-driven decision-making and significantly raising the organization's overall data literacy. ## Establishing a Single Source of Truth (SSOT) - Addressed the inefficiency of fragmented data across various departments by integrating disparate datasets into a unified, enterprise-wide data mart. - Standardized the definition of an "active" Business Customer through cross-functional communication and a deep understanding of how revenue and costs are generated in each service domain. - Eliminated communication overhead by ensuring all stakeholders used a single, verified dataset rather than conflicting numbers from different business silos. ## Designing the Monthly BC Report for Actionable Insights - Visualized monthly revenue trends by segmenting customers into specific tiers and categories, such as New, Churn, and Retained, to identify where growth or attrition was occurring. - Implemented Cohort Retention metrics by business unit to measure platform stickiness and help teams understand which services were most effective at retaining business users. - Provided granular Raw Data lists for high-revenue customers showing significant growth or churn, allowing operational teams to identify immediate action points. - Refined reporting metrics through in-depth interviews with Product Owners (POs), Sales Leaders, and Domain Heads to ensure the data addressed real-world business questions. ## Technical Architecture and Validation - Built the core SSOT data mart using Airflow for scalable data orchestration and workflow management. - Leveraged Jenkins to handle the batch processing and deployment of the specific data layers required for the reporting environment. - Integrated Tableau with SQL-based fact aggregations to automate the monthly refresh of charts and dashboards, ensuring the report remains a "living" document. - Conducted "collective intelligence" verification meetings to check metric definitions, units, and visual clarity, ensuring the final report was intuitive for all users. ## Driving Organizational Change and Data Literacy - Sparked a surge in data demand, leading to follow-up projects such as daily real-time tracking, Cross-Domain Activation analysis, and deeper funnel analysis for BC registrations. - Transitioned the organizational culture from passive data consumption to active utilization, with diverse roles—including Strategy Managers and Business Marketers—now using BC data to prove their business impact. - Maintained an iterative approach where the report format evolves every month based on stakeholder feedback, ensuring the data remains relevant to the shifting needs of the business. Establishing a centralized data culture requires more than just technical infrastructure; it requires a commitment to iterative feedback and clear communication. By moving from fragmented silos to a unified reporting standard, data analysts can transform from simple "number providers" into strategic partners who drive company-wide literacy and growth.

tossOriginal article

The era when everyone does research (opens in new tab)

In an era where AI moderators and non-researchers handle the bulk of data collection, the role of the UX researcher has shifted from a technical specialist to a strategic guide. The core value of the researcher now lies in "UX Leadership"—the ability to frame problems, align team perspectives, and define the fundamental identity of a product. By bridging the gap between business goals and user needs, researchers ensure that products solve real problems rather than just chasing metrics or technical feasibility. ### Setting the Framework in the Idea Phase When starting a new project, a researcher’s primary task is to establish the "boundaries of the puzzle" by shifting the team’s focus from business impact to user value. * **Case - AI Signal:** For a service that interprets stock market events using AI, the team initially focused on business metrics like retention and news consumption. * **Avoiding "Metric Traps":** A researcher intervenes to prevent fatigue-inducing UX (e.g., excessive notifications to boost CTR) by defining the "North Star" as the specific problem the user is trying to solve. * **The Checklist:** Once the user problem and value are defined, they serve as a persistent checklist for every design iteration and action item. ### Aligning Team Direction for Product Improvements When a product already exists but needs improvement, different team members often have scattered, subjective opinions on what to fix. The researcher structures these thoughts into a cohesive direction. * **Case - Stock Market Calendar:** While the team suggested UI changes like "it doesn't look like a calendar," the researcher refocused the effort on the user's ultimate goal: making better investment decisions. * **Defining Success Criteria:** The team agreed on a "Good Usage" standard based on three stages: Awareness (recognizing issues) → Understanding (why it matters) → Preparation (adjusting investment plans). * **Identifying Obstacles:** By identifying specific friction points—such as the lack of information hierarchy or the difficulty of interpreting complex indicators—the researcher moves the project from "simple UI cleanup" to "essential tool development." ### Redefining Product Identity During Stagnation When a product's growth stalls, the issue often isn't a specific UI bug but a fundamental mismatch between the product's identity and its environment. * **Case - Toss Securities PC:** Despite being functional, the PC version struggled because it initially tried to copy the "mobile simplicity" of the app. * **Contextual Analysis:** Research revealed that while mobile users value speed and portability, PC users require an environment for deep analysis, multi-window comparisons, and deliberate decision-making. * **Consensus through Synthesis:** The researcher integrates data, user interviews, and market trends into workshops to help the team decide where the product should "live" in the market. This process creates team-wide alignment on a new strategic direction rather than just fixing features. The modern UX researcher must move beyond "crafting the tool" (interviewing and data gathering) and toward "UX Leadership." True expertise involves maintaining a broad view of the industry and product ecosystem, structuring team discussions to reach a consensus, and ensuring that every product decision is rooted in a clear understanding of the user's context and goals.

kakaoOriginal article

Were We Solving the Real Problem (opens in new tab)

The POPM (Product Owner/Product Manager) training course at Kakao focuses on restructuring existing professional knowledge into a cohesive framework for solving real-world business problems. Rather than simply delivering new information, the program emphasizes aligning strategy with execution, transforming "strategy" from a vague concept into a practical set of decision-making criteria. The ultimate goal is to move teams away from a "release-only" mindset toward a cycle of continuous hypothesis verification and learning. ### Strategic Thinking and Metric Modeling * **Strategic Decision Criteria**: Strategy is redefined as the standard for team judgment, utilizing frameworks like MECE, MVP, and priority-setting models to align daily tasks with long-term goals. * **Metrics as Problem-Solving Language**: Key indicators such as Funnel, Retention, Cohort, and LTV are treated not just as data points, but as a language used to define and reveal underlying product issues. * **Context-Based Design**: UX design is approached through "context-based logic" rather than intuition, encouraging teams to ask which specific design fits the current user journey. ### Systematic Experimentation and A/B Testing * **The MASS Framework**: Experiments are designed and evaluated based on being Measurable, Attributable, Sensitive, and having a Short-term cycle. * **Failure Analysis Routines**: The curriculum emphasizes the importance of establishing a routine for interpreting failed experiments, ensuring that every test contributes to the team's institutional knowledge. * **Incremental Testing**: Encourages a culture of "starting small," giving teams the confidence to run experiments without requiring massive resource allocation. ### Building Repeatable Execution Loops * **Metric-Based Retrospectives**: Teams transition from simply finishing a release to a structured loop of "Problem Definition → Hypothesis → Metric → Verification → Retrospective." * **Formalizing Problem Definitions**: Using templates to 명문화 (formally document) the problem, expected behavior, and success metrics ensures that the entire team—not just the PO—understands the "why" behind every task. * **Operational Rhythms**: Teams are adopting fixed weekly or bi-weekly cycles for sharing insights and adjusting priorities, turning data-driven execution into a natural habit. The most critical takeaway for product teams is to constantly ask: "Is the work we are doing right now actually a solution to a defined problem, or are we just busy releasing features?" Success lies in moving beyond the sense of accomplishment from a launch and establishing a repeatable rhythm that validates whether those efforts truly move the needle.

kakaoOriginal article

How the POPM Course Became a (opens in new tab)

Kakao developed its internal POPM (Product Owner/Product Manager) training program by treating the curriculum itself as an evolving product rather than a static lecture series. By applying agile methodologies such as data-driven prioritization and iterative versioning, the program successfully moved from a generic pilot to a structured framework that aligns teams through a shared language of problem-solving. This approach demonstrates that internal capability building is most effective when managed with the same rigor and experimentation used in software development. ## Strategic Motivation for POPM Training * Addressed the inherent ambiguity of the PO/PM role, where non-visible tasks often make it difficult for practitioners to define their own growth or impact. * Sought to resolve the disconnect between strategic problem definition (PO) and tactical execution (PM) within Kakao’s teams. * Prioritized the creation of a "common language" to allow cross-functional team members to define problems, analyze metrics, and design experiments under a unified structure. ## Iterative Design and Versioning * The program transitioned through multiple "versions," starting with an 8-session pilot that covered the entire lifecycle from bottleneck exploration to execution review. * Based on participant feedback regarding high fatigue and low efficiency in long presentations, the curriculum was condensed into 5 core modules: Strategy, Metrics, Experiment, Design, and Execution. * The instructional design shifted from "delivering information" to "designing a rhythm," utilizing a "one slide, one question, one example" rule to maintain engagement. ## Data-Driven Program Refinement * Applied a "Product Metaphor" to education by calculating "Opportunity Scores" using a matrix of Importance vs. Satisfaction for each session. * Identified "Data/Metrics" as the highest priority for redesign because it scored high in importance but low in satisfaction, indicating a structural gap in the teaching method. * Refined the "features" of the training by redesigning worksheets to focus on execution routines and converting mandatory practice tasks into selective, flexible modules. ## Structural Insights for Organizational Growth * Focused on accumulating "structure" rather than just training individuals, ensuring that even as participants change, the framework for defining problems remains consistent within the organization. * Designed practice sessions to function as "thinking structures" rather than "answer-seeking" exercises, encouraging teams to bring their training insights directly into actual team meetings. * Prioritized scalability and simplicity in the curriculum to ensure the structure can be adopted across different departments with varying product needs. To build effective internal capabilities, organizations should treat training as a product that requires constant maintenance and versioning. Instead of focusing on one-off lectures, leaders should design structural "rhythms" and feedback loops that allow the curriculum to evolve based on the actual pain points of the practitioners.

datadog3 min readCurated summary

The Old Datadog and the Sea

Wouter de Bie describes upgrading his 1992 Hunter Legend sailboat, *Second Wind*, into a connected, data-driven vessel. His goals were to unify onboard instruments, improve safety with vessel tracking, add accurate wind measurements, and collect data for later analysis. The central solution was an NMEA 2000 network, which allowed compatible devices to share data and power over a single backbone. ## Building a Unified Instrument Network - The boat originally had a depth sounder, speedometer, autopilot, chartplotter, and GPS, but the systems were not centrally connected. - The existing Raymarine equipment used SeatalkNG, which is compatible with NMEA 2000 despite having different cables and connectors. - The older speed and depth displays used NMEA 0183 rather than NMEA 2000. - Instead of adding protocol converters, Wouter replaced the older displays with new NMEA 2000-compatible instruments. - Once connected to the backbone, speed and depth data appeared on the chartplotter. ## Why NMEA 2000 Was Chosen - NMEA 2000 is similar to the CAN bus used in automobiles. - Devices connect to a shared backbone and broadcast small data messages for other equipment to consume. - The network operates at 250 kbit/s, which is sufficient for marine instruments. - Its cables carry both power and data, reducing wiring complexity. - Most marine electronics vendors support the standard. ## Improving Safety with AIS - Wouter added an Automatic Identification System (AIS) transponder to detect nearby vessels. - AIS broadcasts a vessel’s name, identifier, speed, heading, and coordinates over VHF radio. - The system only detects AIS-equipped vessels within line of sight, but offers a lower-cost alternative to radar. - After installing the transponder and routing its antenna cable to the stern, nearby ships appeared on the chartplotter. - The transponder’s built-in GPS also supplied positioning data to the NMEA 2000 network. ## Adding Wind Measurements - The original wind indicator was only an analog vane, requiring Wouter to look up at the mast and providing no wind-speed measurement. - He purchased a NMEA 2000 wind transducer and cockpit display. - The transducer measures wind angle with a vane and wind speed with a rotor, typically from the top of the mast. - Installation required routing a cable through the mast and boat. - A friend was hoisted up the mast on a calm evening to drill the mounting hole and install the sensor. The project demonstrates how replacing incompatible legacy instruments and connecting modern devices through NMEA 2000 can turn a boat into an integrated safety and analytics platform. Once all sensor data is centralized, it can support both better sailing decisions and longer-term analysis of performance and conditions.

Read original(opens in new tab)