TL;DR: HTTP outcalls in smart contracts have potential benefits, like event alerts and trusted data provision, but can lead to fragile, centralized systems. Weighing mitigation strategies against simpler, economic incentivization methods is crucial. The community should develop guidelines and best practices for HTTP outcalls, while exploring alternatives to ensure a robust, decentralized ecosystem on the Internet Computer platform.
As we continue to build and improve the Internet Computer ecosystem, itās essential to have an open dialogue about the strengths and weaknesses of various features we use in our smart contracts. One such feature is HTTP outcalls, which has the potential to both enhance and undermine the decentralized nature of our projects. In this post, I will discuss the potential downsides of relying on HTTP outcalls and suggest that we, as a community, should consider developing guidelines and best practices to ensure we maintain the robustness and reliability of our smart contract systems.
HTTP outcalls can provide valuable functionality in certain cases, such as alerting the outside world to interesting events happening on-chain, or when a single trusted party is authorized to provide a specific piece of data. However, as data becomes more subjective and its provision relies on non-dedicated providers, the system becomes more fragile and centralized.
The recent debate over the KYT methodology to use for BTC was rooted in the fact that HTTP outcalls exist at all. Most chains donāt have the functionality in the first place, so relying on decentralized data collection is the default mode of operation. Iāve previously expressed my skepticism of HTTP outcalls (Oracles. Iām suspicious. - re: coming http requests) and I think that this situation shows the slippery slope to ālazy architectureā. Iām not suggesting that we donāt use them, just that we set up a proper framework and set of best practices when doing so. For the Exchange Rate canister, I have serious āblack swanā fears that the method could stall(for example, Binance blocks access to their price feeds in the USā¦and if suddenly 75% of crypt companies shuttered over night due to regulatory, military, or social chaos, things could get chippy). I DFINITY has thought long and hard about fallbacks and has a plan in place, but not everyone has the resources or time that DFINITY does, and HTTP Outcalls could become a juicy shortcut that could come back to bite projects in the long run.
While some mitigation strategies exist, like using multiple data sources, fallback mechanisms, and data verification techniques, their complexity should be weighed against simpler, economic incentivization-based methods. By relying on these incentives, we can drive truth-seeking behavior and encourage the development of more decentralized solutions.
Itās important to recognize that we cannot predict every scenario, and there are cases where HTTP outcalls might be the most practical or efficient choice. Iāve spent some time thinking through this and added several patterns to the ICDevs Language Project. They are open for public comment and you are encouraged to contribute. By building these patterns, we can foster a more informed and productive discussion within our community and further strengthen the decentralized nature of the Internet Computer platform.
In summary, HTTP outcalls are a double-edged sword with both advantages and drawbacks. As developers, it is our responsibility to carefully consider the implications of using this feature and to explore alternative approaches when appropriate.
I encourage you to share your thoughts and experiences with HTTP outcalls and letās work together to build a more resilient and decentralized Internet Computer ecosystem.
New Patterns: See ICDevs.org for a more dynamic presentation, and contribute at Icdevs_fleeksite/language_project at main Ā· icdevs/Icdevs_fleeksite Ā· GitHub
Decentralized Data Provision
Supports: BLOCKCHAIN AND DECENTRALIZATION PRINCIPLES
Leverage decentralized data sources to promote system robustness. Minimize reliance on single points of failure.
Context:
Smart contract systems often require external data to function, and the choice of data source can have a significant impact on the overall system architecture.
Problem:
Relying on centralized data sources can introduce fragility and undermine the decentralized nature of a smart contract system, making it susceptible to single points of failure.
Forces:
- Decentralization: The desire to maintain a decentralized architecture in smart contract systems.
- Data reliability: The need for accurate and reliable data for smart contract execution.
- System robustness: Ensuring the system can withstand failures or attacks on data sources.
- Integration complexity: The challenge of integrating with multiple decentralized data sources.
Solution:
Incorporate decentralized data sources and oracles into the system architecture whenever possible. By diversifying data sources and reducing reliance on single points of failure, smart contract systems can become more robust and resilient. In cases where decentralized data sources are not available, strive to implement mitigation strategies to ensure data reliability and maintain system integrity.
Supported By : OUT. OUTSIDE-IN, DVR. DATA VERIFICATION AND REDUNDANCY, EIA. ECONOMIC INCENTIVES ALIGNMENT
Data Verification and Redundancy
Supports: DDP. DECENTRALIZED DATA PROVISION
Cross-check data from multiple sources. Maintain data integrity and system reliability.
Context:
Smart contract systems often rely on external data for critical operations. Ensuring the accuracy and reliability of this data is essential for maintaining the integrity of the system.
Problem:
Inaccurate or unreliable data from external sources can compromise the functionality and reliability of a smart contract system, potentially leading to undesirable consequences.
Forces:
-
Data accuracy: Ensuring that the data used by the smart contract system is accurate and reliable.
-
System integrity: Maintaining the overall reliability and robustness of the smart contract system.
Redundancy: Utilizing multiple data sources to cross-check information and minimize the impact of individual data source failures.
- Verification complexity: Implementing data verification techniques without overly complicating the system architecture.
Solution:
Use multiple data sources and implement verification techniques to cross-check information and ensure data accuracy. By incorporating redundancy into the system design and verifying data against multiple sources, developers can minimize the impact of individual data source failures and inaccuracies, thereby maintaining system integrity and reliability.
To implement this solution, consider the following strategies:
-
Data source diversity: Select data sources that offer a range of perspectives or methodologies, reducing the risk of systemic bias or errors. Diversifying data sources can also help mitigate the impact of a single point of failure or manipulation.
-
Cross-validation: Implement mechanisms to cross-validate data from different sources, comparing and reconciling discrepancies to establish a higher degree of confidence in the dataās accuracy. This may include employing statistical methods, consensus algorithms, or other techniques to ensure data consistency.
-
Monitoring and alerts: Establish monitoring and alert systems to quickly identify and respond to potential data inaccuracies or failures in the data provisioning process. This enables the developers or system maintainers to take corrective action promptly and minimize the impact on the smart contract system.
-
Graceful degradation: Design the system to handle data inaccuracies or unavailability gracefully, ensuring that the smart contract system can continue to operate, albeit with reduced functionality or precision, until the issue is resolved.
-
Continuous improvement: Regularly evaluate the performance of the data verification and redundancy mechanisms, refining and updating them as needed to maintain optimal system performance and reliability.
By incorporating these strategies into the smart contract system design, developers can create a more resilient and reliable system that is less vulnerable to data inaccuracies or failures. This, in turn, helps maintain the overall integrity of the smart contract system and ensures that critical operations can continue even in the face of external data challenges.
It is essential to recognize that the appropriate level of redundancy and verification complexity will depend on the specific requirements and risks associated with the smart contract system. For example, systems handling high-value transactions or sensitive information may warrant more robust verification and redundancy mechanisms than those with lower stakes. Therefore, developers should carefully consider the specific needs and potential risks associated with their smart contract system when designing and implementing data verification and redundancy measures.
By prioritizing data accuracy and system integrity, and by implementing a robust and adaptable verification and redundancy framework, developers can create smart contract systems that are better equipped to navigate the uncertainties and challenges associated with external data provision. This ultimately helps ensure the long-term success and reliability of the smart contract system.
Supported By: FAL. FALLBACK MECHANISMS
Adaptable System Design
Supports: DDP. DECENTRALIZED DATA PROVISION, DVR. DATA VERIFICATION AND REDUNDANCY
Build systems capable of transitioning between centralized and decentralized data sources. Enable seamless adaptation as viable alternatives emerge.
Context:
Smart contract systems often require external data and may initially rely on centralized data sources due to limitations in the availability or maturity of decentralized alternatives.
Problem:
A rigid system design that does not account for the potential emergence of decentralized data sources can hinder the transition to a more decentralized architecture, compromising the long-term robustness and resilience of the smart contract system.
Forces:
-
Decentralization: Encouraging a shift towards decentralized data sources as they become more viable and mature.
-
System adaptability: Designing a smart contract system that can easily transition between centralized and decentralized data sources.
-
Future-proofing: Ensuring the system remains relevant and robust as the external data landscape evolves.
-
Implementation complexity: Developing an adaptable system without introducing excessive complexity or overhead.
Solution:
Design smart contract systems capable of seamlessly transitioning between centralized and decentralized data sources. By building systems with adaptability in mind, developers can future-proof their projects and better align with the evolving data landscape. As decentralized alternatives become more mature and viable, these adaptable systems can easily integrate with them, reducing reliance on centralized data sources and promoting a more resilient and decentralized architecture.
In order to build adaptable smart contract systems, developers should consider the following strategies:
-
Modular design: Create modular components within the system that can be easily updated or replaced as new data sources or technologies become available. This approach allows for the smooth integration of decentralized data sources without the need for extensive system rewrites.
-
Open standards: Utilize open standards and protocols when interacting with external data sources, as this will facilitate seamless interoperability between the smart contract system and a wide range of centralized or decentralized data providers.
-
Community engagement: Engage with the broader developer and user community to stay informed about emerging technologies and trends in decentralized data provision. This will enable the systemās developers to proactively identify and adopt new solutions as they become available.
-
Continuous improvement: Regularly evaluate and iterate on the system design to ensure that it remains adaptable and capable of supporting the integration of new data sources and technologies. This may include conducting regular audits, addressing technical debt, and refining the system architecture.
By incorporating these strategies into the system design process, developers can create smart contract systems that are more resilient and better prepared to adapt to the ever-changing data landscape. This will ultimately result in a more robust and decentralized architecture, better aligned with the principles of blockchain technology and capable of supporting the long-term success of the project.
Economic Incentives Alignment
Supports: DDP. DECENTRALIZED DATA PROVISION
Design incentive structures that drive truth-seeking behavior. Foster competition among external data providers.
Context:
Smart contract systems often rely on external data providers for essential information. Ensuring the accuracy and reliability of this information is crucial for the proper functioning of the system.
Problem:
When data providers lack proper incentives, they may not be motivated to deliver accurate and reliable data, potentially compromising the systemās integrity and functionality.
Forces:
-
Truth-seeking behavior: Encouraging data providers to supply accurate and reliable data.
-
Competition: Fostering a competitive environment among data providers to drive higher quality data provision.
-
Incentive design: Creating effective and balanced incentive structures that motivate data providers.
-
System integrity: Ensuring the smart contract system remains robust and reliable despite potential inaccuracies in external data.
Solution:
Design economic incentive structures that motivate external data providers to deliver accurate and reliable data. By aligning the interests of data providers with the requirements of the smart contract system, truth-seeking behavior can be encouraged. Additionally, fostering competition among data providers can drive higher quality data provision and further support system integrity. Incentive structures should be carefully designed to balance potential rewards and penalties, ensuring they are effective and sustainable over time.
Supported By: CIT. COMPLEXITY VS. INCENTIVIZATION TRADE-OFF
Complexity vs. Incentivization Trade-off
Supports: EIA. ECONOMIC INCENTIVES ALIGNMENT
Evaluate the balance between implementing complex mitigation strategies and using economic incentivization. Optimize system design for simplicity and robustness.
Context:
Smart contract systems often require external data, and developers must choose between implementing complex mitigation strategies for HTTP outcalls or relying on simpler, economic incentivization-based methods.
Problem:
Complex mitigation strategies can add unnecessary layers of complication and overhead to the system architecture, potentially undermining its efficiency and robustness. Conversely, economic incentivization might not always be sufficient or applicable in every scenario.
Forces:
-
System simplicity: Designing a smart contract system that is easy to understand, implement, and maintain.
-
Robustness: Ensuring that the system can withstand failures, attacks, or inaccuracies in external data.
-
Incentive design: Evaluating the effectiveness and applicability of economic incentives in various scenarios.
-
Trade-off analysis: Weighing the benefits and drawbacks of complexity and incentivization in system design.
Solution:
Evaluate the trade-off between implementing complex mitigation strategies for HTTP outcalls and using simpler, economic incentivization-based methods. Carefully consider the system requirements, available resources, and potential risks to make informed decisions about the most appropriate approach. Strive to optimize system design for simplicity and robustness, leveraging economic incentives where possible and implementing more complex strategies only when necessary or when economic incentives are not sufficient or applicable. Be aware that the effectiveness of economic incentives may vary depending on the nature of the data and the specific context of the smart contract system.
When making these decisions, consider factors such as the reliability and accuracy requirements of the data, the potential consequences of data inaccuracies, and the costs associated with implementing and maintaining various mitigation strategies. Additionally, examine the potential for future developments in decentralized data provision technologies, as these may impact the optimal balance between complexity and incentivization.
By carefully weighing these factors and prioritizing system simplicity and robustness, developers can create smart contract systems that effectively address the challenges associated with external data provision while remaining adaptable to changes in the data landscape. This approach ensures that smart contract systems can continue to operate efficiently and securely, even as the external environment evolves and new data sources or technologies become available.
Outside In
Supports: DDP. DECENTRALIZED DATA PROVISION
Prioritize notifying external data providers of data needs. Encourage data provision through decentralized push mechanisms.
Context:
Smart contract systems often require external data for critical operations. Ensuring the accuracy and reliability of this data while maintaining decentralization is essential for system integrity.
Problem:
Relying on HTTP outcalls to pull data from external sources may compromise decentralization and introduce fragility into the system. A more resilient approach involves pushing data into the system from external sources through decentralized means.
Forces:
-
Decentralization: Encouraging a decentralized approach to data provision in smart contract systems.
-
Data reliability: Ensuring the data provided by external sources is accurate and reliable.
-
System robustness: Maintaining the overall reliability and resilience of the smart contract system.
Communication efficiency: Facilitating seamless data provision without relying on HTTP outcalls.
Solution:
Design smart contract systems to prioritize notifying external data providers of their data needs, thereby encouraging data provision through decentralized push mechanisms. By shifting the responsibility of data provision to external providers and promoting a push-based approach, developers can reduce reliance on HTTP outcalls and maintain the decentralized nature of their smart contract systems. This approach can also foster competition among data providers and incentivize them to deliver accurate and timely data. To implement this solution, use on-chain events or other communication channels to notify data providers about data requests, and design the system to accept data input through transactions or decentralized protocols. This outside-in approach helps ensure system robustness and resilience while preserving the core principles of decentralization.
Single Trusted Party Authorization
Use HTTP outcalls when only one trusted party is authorized to provide data. Recognize the necessity of centralization in specific scenarios.
Context:
Smart contract systems sometimes require data from a single, authorized provider due to regulatory requirements, specialized expertise, or unique access to information.
Problem:
In cases where a single trusted party is the only authorized source of data, relying on decentralized data sources may not be feasible.
Forces:
-
Centralization trade-off: Balancing the need for decentralization with the reality of single trusted party authorization.
-
Data accuracy: Ensuring the data provided by the trusted party is accurate and reliable.
-
Compliance: Adhering to regulatory or contractual requirements that mandate single trusted party authorization.
-
System integrity: Maintaining the overall reliability and robustness of the smart contract system.
Solution:
In scenarios where only one trusted party is authorized to provide specific data, use HTTP outcalls to integrate with the external data source. Acknowledge that centralization may be necessary or more efficient in certain situations, and carefully evaluate the trade-offs involved. To maintain system integrity, implement additional verification or fallback mechanisms where possible to minimize the risks associated with relying on a single data source.
).