1. Technical issues exposed in this SNS-1 Decentralization Sale
1.1 User repeated deductions
The rules for the SNS-1 decentralization sale were simple: total 3141 SNS-1 tokens are for decentralization sale, and up to 1 ICP can be used to purchase 1 SNS-1 token.
But many users report that they have been deducted more than once for 1 ICP. I can’t collect specific numbers of total number of users affected (DFINITY should have that numbers), but here are just three examples of community feedback that I have collected:
Although after 48 hours the extra ICP deducted has been returned to users, this problem occurred at the launch of the sale, reflecting the serious functional bug in the SNS sale code.
Any computer science major student knows how to develop an e-commerce system that meets basic functional requirements. Products are either traded for the price amount or not traded without deducting more money from the user (no DOUBLE SPENDING). Even if you encounter the traffic of Black Friday online shopping, users may face long delays, service degradation, or even service interruption, but basic functions must also be guaranteed, and users should not pay more than once!
1.2. Sybil Attack / Front-run Prevention
NNS/SNS team blamed bots for causing issues and they have acknowledged (https://twitter.com/dfinity/status/1598399777284427776 ) bots heavily impacted this launch, with over 1/3 of SNS-1 being purchased by robots. This however raises questions about whether the NNS/SNS team responsible for the launch had any experience in product design. Bots are very common in both the web2 and web3/crypto worlds, yet the team seemingly ignored them beforehand and later blamed the robots for the problems with the launch.
The crypto industry already has well-established sybil-attack solutions for airdrops or token launches (of course each with its own advantages and disadvantages). Even within the IC ecosystem, there have been proposals like DOM’s favorite project People Parties(People Parties - Community Proposal), as well as community projects like ModClub(http://modclub.app) , the Supernova-winner project Proof of Personhood(https://devpost.com/software/web3-mkdgh6) , and even there is the cycle faucet anti-robot solution (DFINITY DEV OFFICIAL ) . It’s clear that the NNS/SNS team was living in their own bubble without sufficient communication with the other DFINITY team members.
Additionally, the presence of bots also raises the issue of front-running. It’s likely that bots or scripts were used to directly invoke the NNS backend to complete front-running. Moreover, NNS initiated the public call for the function through the front end, added orders and notifications to SNS through its own backend, inevitably concentrating all traffic in one place. This easily led to congestion. Regardless of how strong the ledger’s performance was, the traffic was still congested at the NNS frontend and its single Canister. The distributed processing of concurrent issues was not well considered.
1.3. The Subnet node outage and congestion issues
Why did 5 of SNS subnet’s 34 nodes go offline during the initial SNS-1 sale? Why did the SNS subnet also crash for several hours after the SNS-1 sale was over?
Some members of the community celebrated (https://twitter.com/danostrovsky/status/1597637014341357568 ) the huge tps achieved during the SNS-1 sale, but we need to be aware that during this 1-2 hour period, only 3141 SNS-1 tokens were finally bought by 3141 NNS wallets. Therefore, most of the transactions are invalid, blocking the network and wasting a lot of computing resources. The number of effective transactions is far lower than the data shown on the panel, which is only close to 1 TPS.
1.4. Lack of Rigorous Testing
If SNS-1 decentralization sale had undergone thorough and rigorous testing, particularly stress testing, many of the issues mentioned above could have been discovered and resolved.
While SNS-1 was intended as a test, it was essentially a PUBLIC BETA test in a production environment, with thousands of users investing 1-2 hours and their own real money to participate. Low-level bugs should not have been discovered at this stage. Public beta testing should be used as a last line of defense to discover complex software defects or human economic incentive issues.
I have heard that members of the NNS/SNS team previously worked at well-known big tech companies such as Google/IBM. It is hard to believe that Google/IBM would have such low engineering quality standards.
If the NNS/SNS team’s testing engineering issues are not addressed fundamentally, the community will have no confidence in the immediate release of the OpenChat token.
1.5. Lock-up time
Why was the lock-up time for SNS-1 so random, when it was previously announced as 30 days, and why was it adjusted before the launch? This is a big product issue.
Summary
DFINITY‘s Manu has made a detailed post-incident analysis, which can be found here: https://forum.dfinity.org/t/degraded-performance-during-SNS-1-decentralization-sale-incident-retrospective-tuesday-november-29-2022/17005. His analysis identifies the cause of the performance issues mentioned in 1.3 and suggests solutions, but does not provide satisfactory answers for issues such as the problem with repeat charges and other issues. And I hope to see NNS/SNS team come forward to provide their post-mortem.
In summary, NNS, the first usable wallet launched by IC and developed by DFINITY, not only takes on the functions of transfer transactions, neuron pledge and community governance, Canister creation, and top-up. Coin holders and community users had high expectations.
However, since its launch more than 1 year ago, poor loading performance, poor user experience and inability to interact with DApp have been the criticisms of NNS. DFINITY wanted to improve user experience by revamping the front-end, but lacked knowledge and experience in building user-side products and did not achieve good results.
The experimental release of SNS1 Decentralized Sale saw transaction congestion and robotic rush, which DFINITY explained was an experiment to expose the problem. However, as a product for end users and ICP holders, this attitude is quite irresponsible. Participants are not only investing real money but also with a real desire for the performance and performance of the IC. The experiment was not meant to be 100% successful, but the problems and reasons for failure were not officially pinpointed and held accountable.
The biggest reason for SNS1’s big failure in experience is the design of NNS, and the front-end and back-end teams are responsible. Apparently, the team did not have enough knowledge and testing on the completeness of business processes, sybil-attack, and performance guarantees.
The successful completion of the decentralization mechanism to offer is to decentralize while achieving the effect of horizontal scale out, which should allow more builders and users to help tokens sale and circulate. In the design process of SNS, the team did not engage enough with the community. It was only in the public beta testing before the final launch that so many problems were discovered, giving the community a huge surprise.
1. 本次SNS-1发行暴露的技术问题
1.1 用户重复扣款
SNS decentralization sale 发售的规则很简单：发售3141个SNS-1代币，最多1 ICP购买1个SNS-1，但是有多位用户汇报他们被扣了超过一次SNS-1的ICP。具体数字我无法统计，但以下仅为我收集到社区反馈的三个例子：
虽然在48小时后多扣的ICP已退回给了用户，但这个问题在发售上线时发生，反映了SNS发售代码存在的严重功能性Bug。
任何一个计算机专业的大学生都清楚如何实现一套电商系统，满足基本功能要求，商品要么按照金额成交，要么没有成交，而不会多扣用户的资金。即使碰到黑色星期五电商的流量，用户可能面临长延迟，服务降级甚至服务中断，但基本的功能也必须会得到保障，而不会重复扣款。
1.2 机器人女巫攻击/抢跑
DFINITY事后承认 了机器人严重影响了这次发售，超过1/3的SNS-1被机器人抢购。这难免不让人猜测，负责这次发售的NNS/SNS团队是不是完全没有产品设计的经验？机器人无论是在web2还是Web3/crypto世界里，都是广泛存在的，他们竟然完全无视，并事后把发售的问题归咎于机器人。
crypto业界已经有比较成熟的针对机器人的空投或者抢售抗女巫攻击方案（各有不同利弊）。即使在IC生态，Dom提出过People Parties并且DFINITY基金也通过提案, 社区里也有像ModClub项目，Supernova获奖项目Proof of Personhood, 以及cycle领取水龙头防机器人 的方案。显然 NNS/SNS团队活在自己的泡泡里，跟DFINITY内部团队也没有充分的交流。
另外，有机器人就必然有Front-run的问题。大概率有机器人或者有人用脚本直接调用NNS后端完成Front-Run。而且NNS通过前端发起公募的函数的调用，通过它自己的后端来去对SNS进行订单添加和通知，无疑让所有的流量都集中到了一个地方。非常容易造成堵塞。无论ledger本身性能多强，流量依然塞到了NNS前端和它自己的单个Canister上。分布式处理并发问题并没有很好的考虑到。
1.3 节点掉线和堵塞问题
为什么在 SNS 初始代币交换其间，SNS 的 34个节点掉线了 5个。为什么同时 SNS-1 售完后 SNS 子网也宕机了几个小时？
有社区成员为SNS-1发售时达到巨量的tps而欢呼 ，但我需要清醒的认识到，在这1-2小时内，只有3141个SNS-1 token，被最终3141个NNS钱包购买成功。所以绝大多数的交易都是无效交易，阻塞网络并浪费了大量计算资源。真正有效交易远低于面板展示的数据，接近1 TPS而已。
1.4 缺乏严谨测试
如果SNS-1上线前经过完整严格的测试，特别是压力测试，以上这些问题大部分是能被发现并解决的。虽然SNS-1本身是一次测试，但这是一次在生产环境里的公开BETA测试，有上千的用户投入1-2个小时，而且消耗用户自己真金白银来参与的。低级bug不应该有在这个阶段才发现。公开BETA测试应该作为最后一道防线，用来发现复杂的软件缺陷，或者经济激励问题。
我听说有NNS/SNS的团队成员之前在谷歌/IBM这些知名Web2企业工作，难以相信谷歌/IBM会有如此低的工程质量门槛。
如果NNS/SNS团队的测试工程问题不被根本性解决的话，接下来直接发布OpenChat代币，社区都不会再有信心。
1.5 锁仓规则
这是产品设计问题 ，为什么SNS-1的锁仓时间是随机的，之前说过的30天，最后上线前为什么做了调整？
总结
DFINITY Manu做了详细的事后分析。他的分析指出了上面1.3的性能问题的原因和解决方案，但重复扣款的功能问题和其他问题并没有很好的回答。
综上，NNS作为IC推出的第一个可用型钱包，由 DFINITY 开发，不仅承担了转账交易，神经元质押和社区治理，Canister创建和充值等职能。 持币人和社区用户寄予厚望。
然而上线以来，加载性能差，用户体验差，不能和 DApp 互动一直是 NNS 的诟病，DFINITY 想通过改造前端来提升用户体验，但缺乏对用户侧产品打造的认知与经验，没有取得良好的效果。
此次 SNS1 的实验性发行，发生了交易拥堵和机器人抢购，DFINITY 解释这是为了暴露问题做的实验。但作为一个面向持币人的产品，这种态度是极不负责的，参与者不仅是真金白银的在投入，而且是带有对IC的性能和表现的真实愿望。实验并不是要百分百成功，但是失败的问题和原因官方并没有准确定位和进行问责。
SNS1 在体验上的大失败，最大的原因就是 NNS 的设计问题，前端和后端团队要承担责任。显然，团队对业务流程的完备，防刷，以及性能保障都没有足够的认知和测试。
成功的完成去中心化机制来发售，就是要去中心化而达到水平扩容的效果，应该让更多人帮助代币发售和流通。在SNS的设计过程中，官方并没有这个思路，而是所有的环节都默认放在NNS或者SNS自己这里做，导致了能够参与的社区力量无法参与，进一步的无法获得准确有效的反馈。 在最后上线前的公测环节，才发现如此之多的问题，给社区一个巨大的意外。
iclighthouse
