Ic-py Development and Ongoing Maintenance

Roadmap Update

Automatic retrieval of the Candid file directly from the canister.

Reference implementation: fixed ordering of function and code · bodily11/ic-py@bf17bfd · GitHub

Proposed by @bob11

2 Likes

Last Week’s Progress

  1. I investigated how the Rust ic-agent crate verifies certificates and mapped out which core functions were still missing in ic-py. I then added those missing building blocks—chiefly certificate parsing and hash-tree handling—into ic-py.
  2. Finished parsing the public key from certificates and prepared all prerequisite variables needed for signature verification.

This Week’s Tasks

  • Implement certificate-signature verification in ic-py using the same approach as ic-agent and fully integrate the verification workflow.

Looking Ahead

If everything goes smoothly, we expect a major breakthrough next week: the security-fix release of ic-py (fulfilling Milestones 1 and 2) will be merged into the main repository and published on PyPI.

Stay tuned!

4 Likes

Recently, more and more people have shown interest in the maintenance and updates of ic-py, so I’d like to provide a brief progress update:

I am currently improving ic-py’s checks for certificate delegation and its BLS signature verification. This work is expected to be completed this week, followed by thorough testing next week. If all goes smoothly, I will merge both milestone 1 and milestone 2 into the ic-py main branch next week, and notify the previous maintainers of ic-py to publish the latest updates to PyPI.

I will push the work forward as quickly as possible!

4 Likes

Great that it will be published to PyPI, which is how most users will discover that there is a new version. You can expect a LOT of response when that happens. :flexed_biceps:

2 Likes

Because Milestones 1 and 2 involve a large number of file changes—especially in certificate verification—I’ve almost completely refactored ic-py’s original certificate-related files (and the endpoint upgrade happens to be related to this as well).

As a result, I’ve made extensive changes to ic-py’s request-related functions, which makes me very cautious about the next upgrade. In addition to completing the code, thorough testing is extremely important.

I’ll finish this work as soon as possible, and once Milestones 1 and 2 are complete, I believe ic-py will be just as easy to use as the Rust version of ic-agent.

1 Like

Quick update:

I’m happy to let everyone know that all the work for milestone 2 of ic-py has been completed. The only things left are final testing and some code cleanup before release. Very soon, you’ll be able to update to the latest version of ic-py on PyPI!

4 Likes

Quick update:

All the preparatory work for ic-py Milestones 1 and 2 has been completed.

This week, I will submit a PR to the main repository with the updates for Milestones 1 and 2 and publish them to PyPI.

Any further updates will be posted under this thread.

2 Likes

PR: v2.0.0: Endpoint Upgrade, Timeout/Error Handling, and Certificate Verification via blst by eliezhao · Pull Request #121 · rocklabs-io/ic-py · GitHub

2 Likes

:loudspeaker: ic-py v2.0.0 Release Announcement

We are excited to announce the release of ic-py v2.0.0, a major update to the Python Agent Library for the Internet Computer.

This version delivers significant improvements in endpoint migration, reliability, and certificate verification.

The PR for Milestone 1 and Milestone 2 has already been merged into the main repository. The maintainer will soon publish ic-py 2.0 to PyPI. Once released, I’ll update this thread with the PyPI package link.

:link: PR: #121


:rocket: What’s New in v2.0.0

1. Endpoint Upgrade

  • Migrated update calls from legacy /api/v2/.../call to the new BN v3 call endpoint (/api/v3/canister/.../call).
  • Adapted response parsing to handle v3 responses.
  • Improved retry logic and fixed the incomplete poll implementation.
  • Provides more stable request handling across boundary nodes.

References:


2. Timeouts & Error Classification

  • Added configurable timeout handling for agent calls.
  • Introduced structured error classification for common canister rejections.

References:


3. Certificate Verification (Security)

  • Added optional certificate verification using the official blst binding.
  • New parameter:
agent.update_raw(..., verify_certificate=True)
  • When enabled, the agent verifies the IC’s certified responses with BLS12-381 (G1 signature, G2 public key).
  • Full unit test coverage included for both successful and failure cases.

References:

:warning: Security Note: Certificate verification is disabled by default. For production workloads, we strongly recommend enabling verify_certificate=True and installing blst.


:hammer_and_wrench: Installation

pip3 install ic-py

If you enable certificate verification, you’ll need to install blst manually:

# Clone the official repo
git clone https://github.com/supranational/blst
cd blst/bindings/python

# For Apple Silicon (M1/M2) if ABI issues occur:
# export BLST_PORTABLE=1

python3 run.me

# Add to PYTHONPATH
export PYTHONPATH="$PWD:$PYTHONPATH"

Alternatively, copy blst.py and _blst*.so into your site-packages.

:backhand_index_pointing_right: Windows users: WSL2 is recommended for a smoother setup.


:bookmark: Version

  • Release version: v2.0.0

:raising_hands: Acknowledgments

Big thanks to the community for raising issues and highlighting security concerns. This release directly addresses several long-standing requests and vulnerabilities.


:backhand_index_pointing_right: GitHub Repository

2 Likes

awesome, thanks for the contribution! :slight_smile:

I assume then the new release will also appear under github releases, right?

Yes, I will urge them to release and update the new version to ic-py. I have already communicated with the original author of ic-py, and as you can see, he quickly merged my PR into the main repository.

However, since the original author has stopped maintaining ic-py and nearly three years have passed, I have doubts about whether he can still access his account or related information. If unexpected situations arise, I may have no choice but to open a new PyPI repository, and I might even have to use a new name. This is currently what I’m worried about and considering.

1 Like

GitHub release:

I’ve published the v2.0.0 release to my repository first, so that future users of ic-py can easily locate the release versions.

I’m currently considering updating the package name on PyPI to ic-agent, with myself as the primary maintainer. This would better support more frequent updates and maintenance. The current release process is too slow (since communication with the original author takes a long time), which makes it unsuitable for stable, long-term releases and updates.

1 Like

Considering the need for timely maintenance and easier releases of ic-py in the future, I’ve decided to publish a new Python agent for ICP.

This new package is a fork of the current ic-py codebase and will be actively maintained by me on an ongoing basis .

This package will primarily include modules for candid parsing , identity management , and interaction with IC / canisters on IC .

I’m currently evaluating names, and I think ic-sdk-py and ic-agent-py are both good candidates. At the moment, I’m leaning towards ic-agent-py.

If you have better suggestions, please let me know!

I’ll conclude the naming discussion by Friday at 12:00 noon UTC+8.

If there are no objections to ic-agent-py or no better alternatives, I’ll adopt ic-agent-py as the new name for the updated ic-py repository.

cc: @Luka @raymondk @bob11 @icprobot @Jupes_M @lastmjs @icpp
(hope I tagged the right people)

3 Likes

In my opinion, the naming should depend on what APIs the package offers. I can provide the JS libraries example to help you decide.

We grouped the @dfinity/agent, @dfinity/candid, @dfinity/identity, @dfinity/identity-secp256k1, @dfinity/principal packages under the @icp-sdk/core package. The idea is to have a single, module-based package from which devs can pick and import what they need, without worrying about searching and installing several packages (and eventually being blocked by their versioning). This is the lower level package on top of which other packages can be built, that’s why “core”.

Additionally, we renamed the repo that hosts their source code from dfinity/agent-js to dfinity/icp-js-core. The reason behind the repo renaming is that it aligns with the name of the package published from the repo, plus the agent is just a module among the others offered by the repo.

What APIs/modules will the new Python package offer? Will this be a low level package that devs will use to build other packages on top of? Have you considered the “core” naming?

2 Likes

Very happy to hear that you will publish and maintain a new package :+1:

I like the name ic-agent-py .

1 Like

Thanks for the detailed explanation :raising_hands:

For the new Python SDK, my plan is to follow the same modular grouping approach as the JS SDK. Specifically, the Python tooling will provide three core packages:

  • principal
  • candid – for candid parsing and types
  • agent – for communication with the IC and canisters

I’d also like to keep the naming style aligned with the Foundation’s SDKs. This way, developers working across languages will see a consistent structure, and it emphasizes a unified developer tooling experience across ecosystems.

4 Likes

I guess this means that there will be a single Python package exporting those modules. If this is correct, I would rather go with something like icp-core-py. Is the -py suffix a standard in the Python ecosystem?

1 Like

I think this is a great idea. Striving to keep the Python agent as a true Python-version of the JavaScript agent might make the most sense. At least at Demergent Labs, having the exact equivalent (as much as the language allows) of the JavaScript agent feels like it will help us to switch between contexts much more easily than an esoteric Python agent.

Similar naming conventions, PyPI packages, modules, and APIs to the JavaScript agent feels very good to me.

1 Like

The -py suffix is not really a standard in the Python ecosystem . For example, well-known libraries like NumPy , Pandas, and httpx don’t use it.

I only considered adding the -py suffix to help distinguish the Python SDK from SDKs written in other languages. However, it seems unnecessary, since anything available for download on PyPI is naturally understood to be a Python SDK.

2 Likes

I think that might be more important for the github repository then. icp-py-core would then be the name of the repository if we follow GitHub - dfinity/icp-js-core: The base library to build applications on the Internet Computer.

2 Likes