toss·Aug 11, 2026·5 min readCurated summary

Toss’s Speed and Quality: Are Commercial Tools Enough? — Tossion

Toss’s QA Platform team built Tossion to replace a rigid commercial test case management system with a platform they could continuously adapt. It unifies test cases, manual and automated results, evidence, history, and release snapshots while preserving a clear record of what was tested at each point in time. The platform later expanded into AI-assisted PR analysis, test case generation, and real-device regression testing, enabling QA processes to evolve at Toss’s speed. ## Why Tossion Was Needed - Previous QA information was scattered across automation results, manual test results, test cases, and decision evidence. - Answering questions such as “How did this test perform last time?” required checking multiple systems. - The commercial TCM could not be modified quickly, and requests for new features often stalled. - Toss needed a platform that could be changed directly as new QA requirements emerged. ## Preserving Test History - Tossion organizes work as: - Project - Suite - Section - Test Case - Test cases continue to change as products evolve, but test runs must preserve historical reality. - When a test run is created, it copies the test case’s assignee, steps, and description instead of merely referencing the current test case. - Status changes create history entries showing who made the decision, when, and against which version. - Closed test runs store snapshots of test cases, comments, and automation results, so reports remain available even if the original test cases are later deleted. - Manually selected test cases override Type or Platform filters. ## Collaborative Test Execution - Test runs progress through `Active → Completed → Closed`. - Assignee-based charts show each person’s remaining work at a glance. - Fields such as Status, Type, Assignee, Version, Platform, RNR, and History are added or removed based on actual usage. - Multiple users can work in the same test run simultaneously: - Active viewers appear as avatars. - Editing locks prevent conflicts. - Locks are automatically released when users leave or disconnect. - Status changes are synchronized without requiring a refresh. - The main benefit is short feedback loops: requested improvements can be built and deployed immediately. ## AI-Assisted Release and PR Analysis - Toss planned to use AI for test case creation, PR analysis, regression automation, and execution. - Tossion analyzes every PR included in a release, separating those with QA labels from those marked as not requiring QA. - The goal is not merely to summarize changes, but to audit whether “no QA needed” classifications are correct. - An agent running on a QA server: - Registers with Tossion. - Polls for work. - Executes the AI already authenticated on the server. - Hundreds of PRs are divided into smaller batches and analyzed in parallel for deeper review. - Results are checked for vague or unusable content, such as: - Missing screens or conditions. - Repeated PR titles. - Raw function names. - Missing reproduction steps, expected results, failure symptoms, or reasoning. - Invalid analyses are retried, and the number of merged PRs is compared with the number of analyzed PRs to detect omissions. - Historical incident data increases risk when a new PR touches files associated with past outages. - Results are uploaded after each batch, allowing interrupted jobs to resume without repeating completed analysis. - The resulting “must-test” list defines the scope of the Sprint test run. ## AI-Generated Test Cases - AI generates test cases for new features, but Tossion controls their placement in the test case tree. - AI returns paths such as `Assets > Account Connection > Select Bank`; Tossion converts them into actual sections, reusing existing sections or creating missing ones. - Reliability is checked in three layers: - AI reviews its own output for missing branches, error cases, and boundary values. - Scripts validate naming, numbering, screen coverage, and requirements coverage. - A separate AI creates an independent test plan based on ISTQB and ISO/IEC 29119 practices. - The plan and generated cases are compared: - Planned but missing cases indicate omissions. - Cases outside the plan indicate unplanned scope. - This catches state-based scenarios that screen-oriented test generation might miss. - Generated cases are uploaded through a portable CLI rather than direct UI integration, reducing dependency on local packages, runtimes, and paths. - Stable cases can later become regression tests. ## Running Regression Tests on Real Devices - A Tossion execution modal specifies: - Device - Build - Test scope - Target test run - Runners connected to Android and iOS physical devices register themselves with Tossion but require administrator approval before receiving work. - Runners report device status every 30 seconds. - Tossion supplies the correct build for installation, ensuring results are tied to a known version. - Users can run the full regression suite or a selected section. - Progress is streamed as scenarios finish, including duration and failure messages. - Results are stored at the step level: - Status - Duration - Error message - Screenshot - Scenario-level video - Automation results can be attached directly to the relevant Sprint test run, making them part of the same record used for manual testing. ## Linking Automation Results to Test Cases - Aggregate reports such as “200 regression tests, 3 failures” do not identify which manual test case rows were covered. - Tossion aims to connect test cases and automation in both directions: - Generate automation code from test cases. - Write automation results back into individual test cases. - This removes the need for QA engineers to manually reconcile separate reports with test case lists. Tossion’s central value is not just test management, but ownership and adaptability. By combining immutable test history, collaborative execution, AI validation, and real-device automation in one extensible platform, Toss’s QA team can adjust its tools and processes as quickly as product requirements change.

Read original(opens in new tab)

Test AutomationAIAndroidiOS+3

toss·Jun 26, 2026·4 min readCurated summary

While Others Test Toss, We Build the Way to Test.

Every week, Toss releases a new version containing roughly 300–400 code changes, making quality assurance essential to protecting users from broken critical features. The QA Platform team combines smoke tests, regression testing, release monitoring, automation, and AI-assisted tools to make releases reliable and efficient. Its central lesson is that teams often want QA work handled responsibly—not merely more tools to operate themselves—so the team is shifting toward directly owning more of the testing process. ## Weekly Release Quality - QA begins when a Release Candidate is available. - **Toss Doctor** runs smoke tests covering core flows such as login through account deletion. - A pull-request analyzer identifies changed areas, potential impact, bug risk, and testing priorities. - **Toss Checker** performs regression testing to ensure new changes have not damaged stable functionality. - After release, the team monitors crash metrics and decides whether an immediate hotfix is necessary or whether a safer fix can wait for the next release. - Custom dashboards track crashes and hotfixes, including causes and prevention measures. - The team also supports product groups starting QA, improves internal tools, and helps establish organization-wide QA processes. ## Defining Toss’s Quality Standard The team’s goal is to go beyond basic testing and establish consistent quality standards across Toss. - **Reliable releases every time:** Quality must remain dependable week after week, not just during individual successful launches. - **High-quality testing:** The focus is on finding defects that could become real incidents, rather than simply increasing test volume. - **Efficient quality assurance:** Manual repetition alone cannot keep pace with the company’s release speed, so automation and sustainable workflows are necessary. - AI is intended to handle suitable decisions and repetitive work, allowing people to focus on areas requiring human judgment. ## Building the Tossion Platform Commercial tools did not provide enough flexibility for Toss’s release pace and evolving AI experiments, so the team built its own platform, **Tossion**. - Tossion replaced TestRail and brought test-case creation, execution, and result tracking into one system. - Multiple bots were consolidated into **Toss Butler**, optimized for the team’s workflow. - **PRCheck** analyzes pull requests and highlights where testers should focus. - **tcgen** uses PRDs, design documents, and surrounding context to generate initial test cases for review. - An automation testing platform displays manual and automated test results together. - **Crash Trend** tracks crash patterns using metrics tailored to Toss. - A hotfix dashboard categorizes causes and records measures intended to prevent recurrence. - These tools are connected by one objective: handling the growing volume of weekly changes more effectively. ## Learning What Teams Really Need The team initially assumed that making test-case creation easier would encourage more people to test. However, tcgen received less adoption than expected. - Users did not necessarily want better tools for doing QA themselves. - What they really wanted was for someone to perform testing quickly and accurately while taking responsibility for its quality. - Providing a tool could feel less like removing work and more like assigning a new task. - As a result, the team shifted toward directly handling more testing and pursuing tenfold efficiency without transferring responsibility to product teams. ## Staying Flexible as AI Evolves AI has solved many problems but has also made long-term planning difficult. - Initial hypotheses may be only partly correct, as the tcgen experience demonstrated. - Tools and methods can become outdated rapidly as AI capabilities change. - The team discarded an API testing tool, **API Labs**, after only eight hours when it proved misaligned with its goals. - Tossion, Toss Doctor, Toss Checker, and internal skills are designed with replacement in mind rather than as permanent, finished systems. - AI can accelerate tool creation, but people must still define what quality means, establish priorities, and decide what standards must be preserved. The QA Platform team’s ongoing approach is to build adaptable systems, learn from actual usage, and remain willing to replace anything that no longer serves its purpose. Its upcoming work will explore Tossion, the release-gate tools, regression automation, and intelligent AI bots in greater detail.

Read original(opens in new tab)

Test AutomationAIRegression TestingRelease Management+4

slack·Jun 11, 2026·3 min readCurated summary

Agentic Testing: Where Agents Fit in the E2E Testing Stack

Agentic E2E testing validates whether users can achieve goals rather than enforcing one fixed sequence of UI actions. Slack’s experiment with more than 200 runs found that agents can reliably explore workflows, especially through Playwright MCP, but they are slower and more expensive than deterministic tests. The conclusion is that agents should complement—not replace—traditional E2E tests. ## Goal-Based Testing vs. Fixed Journeys - Traditional tests follow predefined steps: click, type, navigate, and assert. - Agent-driven tests receive a goal and adapt their actions to reach it. - Agents may use different paths to achieve the same result, such as: - Selecting a search suggestion or pressing Enter - Reusing existing navigation state or reopening a view - Adding or skipping intermediate actions - This flexibility improves exploration but introduces tradeoffs in reliability, runtime, and cost. ## Experiment Design Slack evaluated three execution models across more than 200 runs: - **Agent + Playwright MCP** - Uses predefined browser actions and persistent DOM snapshots and logs. - **Agent + Playwright CLI** - Runs Playwright commands through the shell and reassesses the UI after each step. - **Generated Playwright tests** - Produces deterministic test code from natural language, then iteratively refines it. The experiments used Claude Sonnet 4.5 for MCP and CLI workflows and Claude Opus 4.6 for generated tests. All tests ran in non-production Slack workspaces using test data. Two workflows were tested 20 times per configuration: - **Thread Reply:** A simple 15–20-step flow involving channel creation, messaging, thread replies, and verification. - **Search Discovery:** A 25–30-step flow involving search, result navigation, channels, threads, and state verification. Inputs were provided either as detailed natural-language instructions or structured YAML describing actions and expected outcomes. ## Results: Reliability, Cost, and Runtime | Approach | Thread Reply failures | Search Discovery failures | Average runtime | |---|---:|---:|---:| | Agent with Playwright MCP | 0% | Approximately 12% | 5–8 minutes | | Agent with Playwright CLI | Approximately 12% | Approximately 20% | 9–11 minutes | | Generated Playwright tests | Approximately 8% | Approximately 48% | About 3 minutes | - Playwright MCP was the most reliable agentic approach, particularly for simple workflows. - Playwright CLI failed more often due to authentication, navigation timing, and session instability. - Generated tests were fast and reasonably successful on simple flows but degraded sharply as workflows became more complex. - Generated tests often completed 70–80% of a complex workflow before failing on a final interaction or assertion. ## Why Complexity Exposes Differences - MCP maintains a live, stable view of the application through persistent context. - CLI-based agents reconstruct state from updated snapshots, allowing small timing or interpretation inconsistencies to accumulate. - Generated tests can suffer from: - Variable UI state - Imprecise element targeting - Mismatches between generated code and existing page-object abstractions - The results suggest agent-native execution models handle increasingly complex exploratory flows better than generated deterministic tests, despite taking longer. Agentic testing is best used as an exploratory layer for validating user goals and discovering unexpected paths. Deterministic Playwright tests remain preferable for fast, repeatable regression checks, while Playwright MCP appears to be the strongest option when flexible, goal-oriented E2E coverage is needed.

Read original(opens in new tab)

Test AutomationLarge Language ModelsAI AgentsModel Context Protocol+3

line·May 14, 2026·4 min readCurated summary

AI Didn’t Replace QA; It Expanded It

Generative AI has not replaced QA at LINE Album; it has expanded QA’s scope and influence. The team found that QA productivity depends less on executing tests quickly than on organizing and interpreting large amounts of scattered information. By embedding AI into event-driven quality workflows, QA engineers now focus more on risk assessment, test strategy, and final decisions. ## QA as a Quality Architect - QA operates across the entire product lifecycle: planning, development, testing, release, and post-release feedback. - Its responsibilities include: - Identifying design risks during planning - Assessing the impact of code changes - Designing test strategies - Validating releases - Connecting user feedback and operational data to product improvements - QA information comes from many sources: - Planning and technical documents - Slack discussions and decisions - Jira tickets and pull requests - Automated test scripts and logs - App Store and Google Play reviews - The central challenge is therefore managing information volume and complexity, not merely increasing testing speed. ## From AI Assistant to AI-Driven Workflow - Initially, AI was used interactively to: - Summarize documents - Draft test cases - Organize bug reports - Document reproduction steps - This improved individual productivity but required QA engineers to manually collect and prepare information. - LINE Album QA instead built an automated quality platform with more than 30 workflows. - AI now reacts automatically to events such as: - Jira issue creation - Code changes and pull requests - Test execution - User feedback collection - AI gathers, analyzes, and structures quality information, while QA engineers interpret risks and make decisions. ## Scheduling and Webhook Automation ### Scheduled Analysis - Scheduled workflows periodically collect and summarize quality data. - Examples include: - Daily App Store review classification - API test result summaries sent to Slack - UI automation reports - Weekly QA activity and issue reports - QA engineers spend less time gathering data and more time evaluating risks and verifying important findings. ### Webhook-Based Analysis - Webhook workflows run immediately when quality-related events occur. - Examples include: - Summarizing the potential impact of merged code changes - Creating meeting notes when Slack discussions end - Analyzing and visualizing automated test results - This allows the team to recognize important quality signals much earlier. ## The AI-Supported QA Workday - UI tests run through MagicPod for Android and iOS, with results updated in Jira and shared in Slack. - Failed tests trigger analysis to determine whether they are flaky tests and identify possible causes. - Pytest-based API tests are similarly reported to Jira and Slack. - Daily Scrum workflows automatically provide: - Current test progress - Scrum board and issue dashboard links - Unresolved issues - Jira mentions requiring QA attention - App reviews are analyzed daily, classified as positive or negative, translated into Japanese and Korean, and summarized for the team. - During focused work periods, QA engineers use AI-generated information to plan quality activities, execute tests, monitor workflows, and summarize relevant discussions or documents. - End-of-day workflows summarize completed work and remaining issues. ## AI as a Test Design Partner - By 2026, AI generated approximately 90% of LINE Album QA’s test-case drafts. - Simple prompting produced many generic scenarios but failed to capture: - The reason a feature was introduced - Historical defect patterns - Effects on existing user flows - The team improved results by supplying broader context, including: - Specifications and development tickets - Change rationale - Previous Jira issues - Test history - Recurring bug patterns ### Multi-Agent Test Generation - An orchestrator coordinates five specialized sub-agents: - **Plan-Analyzer:** Examines requirements, feature descriptions, and images. - **Dev-Analyzer:** Adds implementation and development-ticket context. - **TestCase-Generator:** Produces normal, exceptional, boundary-value, platform-specific, and prioritized scenarios. - **TestCase-Validator:** Checks coverage, traceability, completeness, Given/When/Then structure, priorities, and platform coverage. - **Quality-Inspector:** Uses prior feedback and quality evaluations to improve future generations. - The workflow expands testing beyond stated requirements by incorporating defects that have historically occurred. - Validation feedback creates an iterative loop, making the output more executable and useful over time. AI is most valuable when it is connected to the organization’s accumulated context and quality signals—not when it is used merely as a chat-based drafting tool. The recommended approach is to automate information collection and analysis while keeping QA responsible for interpretation, prioritization, and final quality decisions.

Read original(opens in new tab)

Test AutomationGenerative AIAutomationSlack+4

toss·Jan 20, 2026Original article

Toss Income QA Platform: The Beginning (opens in new tab)

Toss's QA team developed an internal "QA Platform" to solve the high barrier to entry associated with using Swagger for manual testing and data setup. By transforming complex, multi-step API calls into a simple, button-based GUI, the team successfully empowered non-QA members to perform self-verification. This shift effectively moved quality assurance from a final-stage bottleneck to a continuous, integrated part of the development process, significantly increasing product delivery speed. ### Lowering the Barrier to Test APIs * Existing Swagger documentation was functionally complete but difficult for developers or planners to use due to the need for manual JSON editing and sequential API execution. * The QA Platform does not create new APIs; instead, it provides a GUI layer over existing Swagger Test APIs to make them accessible without technical documentation. * The system offers two distinct interfaces: "Normal Mode" for simplified, one-click testing and "Swagger Mode" for granular control over request bodies and parameters. ### From Manual Clicks to Automation and Management * Phase 1 focused on visual accessibility, allowing users to trigger complex data states via buttons rather than manual API orchestration. * Phase 2 integrates existing automation scripts into the platform, removing the need for local environment setups and allowing anyone to execute automated test suites. * The final phase aims to transition into a comprehensive Test Management System (TMS) tailored to the team's specific workflow, reducing reliance on third-party external tools. ### Redefining Quality as a Design Choice * By reducing the time and mental effort required to run a test, verification became a frequent, daily habit for the entire product team rather than a chore for the QA department. * Lowering the "cost" of testing replaced guesswork with data-driven confidence, allowing the team to move faster during development. * This initiative reflects a philosophical shift where quality is no longer viewed as a final checklist item but as a core structural element designed into the development lifecycle. The primary takeaway for engineering teams is that the speed of a product is often limited by the friction of its testing process. By building internal tools that democratize testing capabilities—making them available to anyone regardless of their technical role—organizations can eliminate verification delays and foster a culture where quality is a shared responsibility.

Test AutomationQuality AssuranceApi TestingTest Management+3

toss·Dec 23, 2025Original article

Tax Refund Automation: AI (opens in new tab)

At Toss Income, QA Manager Suho Jung successfully automated complex E2E testing for diverse tax refund services by leveraging AI as specialized virtual team members. By shifting from manual coding to a "human-as-orchestrator" model, a single person achieved the productivity of a four-to-five-person automation team within just five months. This approach overcame the inherent brittleness of testing long, React-based flows that are subject to frequent policy changes and external system dependencies. ### Challenges in Tax Service Automation The complexity of tax refund services presented unique hurdles that made traditional manual automation unsustainable: * **Multi-Step Dependencies:** Each refund flow averages 15–20 steps involving internal systems, authentication providers, and HomeTax scraping servers, where a single timing glitch can fail the entire test. * **Frequent UI and Policy Shifts:** Minor UI updates or new tax laws required total scenario reconfigurations, making hard-coded tests obsolete almost immediately. * **Environmental Instability:** Issues such as "Target closed" errors during scraping, differing domain environments, and React-specific hydration delays caused constant test flakiness. ### Building an AI-Driven QA Team Rather than using AI as a simple autocomplete tool, the project assigned specific "personas" to different AI models to handle distinct parts of the lifecycle: * **SDET Agent (Claude Sonnet 4.5):** Acted as the lead developer, responsible for designing the Page Object Model (POM) architecture, writing test logic, and creating utility functions. * **Documentation Specialist:** Automatically generated daily retrospectives and updated technical guides by analyzing daily git commits. * **Git Master:** Managed commit history and PR descriptions to ensure high-quality documentation of the project’s evolution. * **Pair Programmers (Cursor & Codex):** Handled real-time troubleshooting, type errors, and comparative analysis of different test scripts. ### Technical Solutions for React and Policy Logic The team implemented several sophisticated technical strategies to ensure test stability: * **React Interaction Readiness:** To solve "Element is not clickable" errors, they developed a strategy that waits not just for visibility, but for event handlers to bind to the DOM (Hydration). * **Safe Interaction Fallbacks:** A standard `click` utility was created that attempts a Playwright click, then a native keyboard 'Enter' press, and finally a JS dispatch to ensure interactions succeed even during UI transitions. * **Dynamic Consent Flow Utility:** A specialized system was built to automatically detect and handle varying "Terms of Service" agreements across different sub-services (Tax Secretary, Hidden Refund, etc.) through a single unified function. * **Test Isolation:** Automated scripts were used to prevent `userNo` (test ID) collisions, ensuring 35+ complex scenarios could run in parallel without data interference. ### Integrated Feedback and Reporting The automation was integrated directly into internal communication channels to create a tight feedback loop: * **Messenger Notifications:** Every test run sends a report including execution time, test IDs, and environment data to the team's messenger. * **Automated Failure Analysis:** When a test fails, the AI automatically posts the error log, the specific failed step, a tracking EventID, and a screenshot as a thread reply for immediate debugging. * **Human-AI Collaboration:** This structure shifted the QA's role from writing code to discussing failures and policy changes within the messenger threads. The success of this 5-month experiment suggests that for high-complexity environments, the future of QA lies in "AI Orchestration." Instead of focusing on writing selectors, QA engineers should focus on defining problems and managing the AI agents that build the architecture.

Test AutomationAIAI AgentsReact+5

woowahan·Dec 4, 2025Original article

Test Automation with AI: Plugin Development Story (opens in new tab)

This blog post explores how a development team at Woowahan Tech successfully automated the creation of 100 unit tests in just 30 minutes by combining a custom IntelliJ plugin with Amazon Q. The author argues that while full AI automation often fails in complex multi-module environments, a hybrid approach using "compile-guaranteed templates" ensures high success rates and maintains operational stability. This strategy allows developers to bypass repetitive setup tasks while leveraging AI for logic implementation within a strictly defined, valid structure. ### Evaluating AI Assistants for Testing * The team compared various AI tools including GitHub Copilot, Cursor, and Amazon Q to determine which best fit their existing IntelliJ-based workflow. * Amazon Q was selected for its superior understanding of the entire project context and its ability to integrate seamlessly as a plugin without requiring a switch to a new IDE. * Initial manual use of AI assistants highlighted repetitive patterns: developers had to constantly specify team conventions (Kotest FunSpec, MockK) and manually fix build errors in 15% of the generated code. * On average, it took 10 minutes per class to generate and refine tests manually, prompting the team to seek a more automated solution via a custom plugin. ### The Pitfalls of Full Automation * The first version of the custom plugin attempted to generate complete test files by gathering class metadata through PSI (Program Structure Interface) and sending it to the Gemini API. * Pilot tests revealed a 90% compilation failure rate, as the AI frequently generated incorrect imports, hallucinated non-existent fields, or used mismatched data types. * A critical issue was the "loss of existing tests," where the AI-generated output would completely overwrite previous work rather than appending to it. * In complex multi-module projects, the AI struggled to identify the correct classes when multiple modules contained identical class names, leading to significant manual correction time. ### Shifting to Compile-Guaranteed Templates * To overcome the limitations of full automation, the team pivoted to a "template first" approach where the plugin generates a valid, compilable shell for the test. * The plugin handles the complex infrastructure of the test file, including correct imports, MockK setups, and empty test stubs for every method in the target class. * This approach reduces the AI's "hallucination surface" by providing it with a predefined structure, allowing tools like Amazon Q to focus solely on filling in the implementation details. * By automating the 1-minute setup and letting the AI handle the 2-minute implementation phase, the team achieved a 97% success rate across 100 test cases. ### Practical Conclusion For teams looking to improve test coverage in large-scale repositories, the most effective strategy is to use IDE plugins to automate context gathering and boilerplate generation. By providing the AI with a structurally sound template, developers can eliminate compilation errors and significantly reduce the time spent on manual refinement, ensuring that even complex edge cases are covered with minimal effort.

Test AutomationAIAI AgentsKotlin+5

toss·Dec 1, 2025Original article

Toss Income Tax Refund Service: (opens in new tab)

Toss Income’s QA team transitioned from traditional manual testing and rigid class-based Page Object Models (POM) to a stateless Functional POM to keep pace with rapid deployment cycles. This shift allowed them to manage complex tax refund logic and frequent UI changes with high reliability and minimal maintenance overhead. By treating automation as a modular assembly of functions, they successfully reduced verification times from four hours to twenty minutes while significantly increasing test coverage. ### Transitioning to Functional POM * Replaced stateful classes and complex inheritance with stateless functions that receive a `page` object as input and return the updated `page` as output. * Adopted a clear naming convention (e.g., `gotoLoginPage`, `enterPhonePin`, `verifyRefundAmount`) to ensure that test cases read like human-readable scenarios. * Centralized UI selectors and interaction logic within these functions, allowing developers to update a single point of truth when UI text or button labels change. ### Modularizing the User Journey * Segmented the complex tax refund process into four distinct modules: Login/Terms, Deduction Checks, Refund/Payment Info, and Reporting. * Developed independent, reusable functions for specific data inputs—such as medical or credit card deductions—which can be assembled like "Lego blocks" to create new test scenarios rapidly. * Decoupled business logic from UI interactions, enabling the team to create diverse test cases by simply varying parameters like amounts or dates. ### Robust Interaction and Page Management * Implemented a 4-step "Robust Click Strategy" to eliminate flakiness caused by React rendering timings, sequentially trying an Enter key press, a standard click, a forced click, and finally a direct JavaScript execution. * Created a `waitForNetworkIdleSafely` utility that prevents test failures during polling or background network activity by prioritizing UI anchors over strict network idleness. * Standardized page transition handling with a `getLatestNonScrapePage` utility, ensuring the `currentPage` object always points to the most recent active tab or redirect window. ### Integration and Performance Outcomes * Achieved a 600% increase in test coverage, expanding from 5 core scenarios to 35 comprehensive automated flows. * Reduced the time required to respond to UI changes by 98%, as modifications are now localized to a single POM function rather than dozens of test files. * Established a 24/7 automated validation system that provides immediate feedback on functional correctness, data integrity (tax amount accuracy), and performance metrics via dedicated communication channels. For engineering teams operating in high-velocity environments, adopting a stateless, functional approach to test automation is a highly effective way to reduce technical debt. By focusing on modularity and implementing fallback strategies for UI interactions, teams can transform QA from a final bottleneck into a continuous, data-driven validation layer that supports rapid experimentation.

Test AutomationTypeScriptPlaywrightFunctional Programming+3

toss·Nov 13, 2025Original article

Frontend Code That Lasts 1 (opens in new tab)

Toss Payments evolved its Payment SDK to solve the inherent complexities of integrating payment systems, where developers must navigate UI implementation, security flows, and exception handling. By transitioning from V1 to V2, the team moved beyond simply providing a library to building a robust, architecture-driven system that ensures stability and scalability across diverse merchant environments. The core conclusion is that a successful SDK must be treated as a critical infrastructure layer, relying on modular design and deep observability to handle the unpredictable nature of third-party runtimes. ## The Unique Challenges of SDK Development * SDK code lives within the merchant's runtime environment, meaning it shares the same lifecycle and performance constraints as the merchant’s own code. * Internal logging can inadvertently create bottlenecks; for instance, adding network logs to a frequently called method can lead to "self-DDoS" scenarios that crash the merchant's payment page. * Type safety is a major hurdle, as merchants may pass unexpected data types (e.g., a number instead of a string), causing fatal runtime errors like `startsWith is not a function`. * The SDK acts as a bridge for technical communication, requiring it to function as both an API consumer for internal systems and an API provider for external developers. ## Ensuring Stability through Observability * To manage the unpredictable ways merchants use the SDK, Toss implemented over 300 unit tests and 500 E2E integration tests based on real-world use cases. * The team utilizes a "Global Trace ID" to track a single payment journey across both the frontend and backend, allowing for seamless debugging across the entire system. * A custom Monitoring CLI was developed to compare payment success rates before and after deployments, categorized by merchant and runtime environment (e.g., PC Chrome vs. Android WebView). * This observability infrastructure enables the team to quickly identify edge-case failures—such as a specific merchant's checkout failing only on mobile WebViews—which are often missed by standard QA processes. ## Scaling with Modular Architecture * To avoid "if-statement hell" caused by merchant-specific requirements (e.g., fixing installment months or custom validation for a specific store), Toss moved to a "Lego-block" architecture. * The SDK is organized into three distinct layers based on the "reason for change" principle: * **Public Interface Layer:** Manages the contract with the merchant, validating inputs and translating them into internal domain models. * **Domain Layer:** Encapsulates core business logic and payment policies, keeping them isolated from external changes. * **External Service Layer:** Handles dependencies like Server APIs and Web APIs, ensuring technical shifts don't leak into the business logic. * This separation allows the team to implement custom merchant logic by swapping specific blocks without modifying the core codebase, reducing the risk of regressions and lowering maintenance costs. For developers building SDKs or integration tools, the shift from monolithic logic to a layered, observable architecture is essential. Prioritizing the separation of domain logic from public interfaces and investing in environment-specific monitoring allows for a highly flexible product that remains stable even as the client-side environment grows increasingly complex.

Test AutomationObservabilityTypeScriptSoftware Architecture+3

line·Sep 26, 2025Original article

PD1 AI Hackathon: Into the (opens in new tab)

The PD1 AI Hackathon 2025 served as a strategic initiative by LY Corporation to embed innovative artificial intelligence directly into the LINE messaging ecosystem. Over 60 developers collaborated during an intensive 48-hour session to transition AI from a theoretical concept into practical features for messaging, content, and internal development workflows. The event successfully produced several high-utility prototypes that demonstrate how AI can enhance user safety, creative expression, and technical productivity. ## Transforming Voice Communication through NextVoIP * The "NextVoIP" project utilized Speech-to-Text (STT) technology to convert 1:1 and group call audio into real-time data for AI analysis. * The system was designed to provide life security features by detecting potential emergency situations or accidents through conversation monitoring. * AI acted as a communication assistant by suggesting relevant content and conversation topics to help maintain a seamless flow during calls. * Features were implemented to allow callers to enjoy shared digital content together, enriched by AI-driven recommendations. ## Creative Expression with MELODY LINE * This project focused on the intersection of technology and art by converting chat conversations into unique musical compositions. * The system analyzed the context and emotional sentiment of messages to automatically generate melodies that matched the atmosphere of the chat. * The implementation showcased the potential for generative AI to provide a multi-sensory experience within a standard messaging interface. ## AI-Driven QA and Test Automation * The grand prize-winning project, "IPD," addressed the bottleneck of repetitive manual testing by automating the entire Quality Assurance lifecycle. * AI was utilized to automatically generate and manage complex test cases, significantly reducing the manual effort required for mobile app validation. * The system included automated test execution and a diagnostic feature that identifies the root cause of failures when a test results in an error. * The project was specifically lauded for its immediate "production-ready" status, offering a direct path to improving development speed and software reliability. The results of this hackathon suggest that the most immediate value for AI in large-scale messaging platforms lies in two areas: enhancing user experience through contextual awareness and streamlining internal engineering via automated QA. Organizations should look toward integrating AI-driven testing tools to reduce technical debt while exploring real-time audio and text analysis to provide proactive security and engagement features for users.

Test AutomationAIGenerative AIMobile Development+2