Open Source the IC-API that powers the Internet Computer Dashboard

Thank you! Let me know please.

Hello. We have pushed out the change to return the total as a single data point: https://ic-api.internetcomputer.org/api/v3/timeseries/reward-node-providers.

3 Likes

Thank you. I would like to have also an endpoint for Disbursed Voting Rewards (ICP Minted from Maturity) always as a total value it it’s possible.

Why are there some endpoints where the API result seems to be limited at the beginning, even though the data shows up on the dashboard?
For example, the timestamp to start in 12/01/2021

/governance-metrics/governance_dissolving_neurons_e8s?start=1638316800

Returns values from 1692057600 (08/15/2023).

Any way I can get this older data?

Hi @Fernando, the ICP Dashboard has never used that particular endpoint. Try this instead:
https://ic-api.internetcomputer.org/api/v3/metrics/governance_dissolving_neurons_e8s_1year?start=1638316800

1 Like

/v3/canisters/ started to return {"code":500,"status":"Internal Server Error"}, more often for large offset

offset 10k OK https://ic-api.internetcomputer.org/api/v3/canisters?offset=10000&sort_by=canister_id&limit=10
offset 100k ERROR https://ic-api.internetcomputer.org/api/v3/canisters?offset=100000&sort_by=canister_id&limit=10

Are this APIs showing only the staked maturity into neurones? (The maturity re-staked)

/api/v3/metrics/governance_not_dissolving_neurons_staked_maturity_e8s_equivalent_1year
/api/v3/metrics/governance_dissolving_neurons_staked_maturity_e8s_equivalent_1year

While this shows only the stake (as we can see form the official dashboard)

api/v3/metrics/governance_dissolving_neurons_e8s_1year
api/v3/metrics/governance_not_dissolving_neurons_e8s_1year

I could not find any description … but as I understood what we see here: Neurons - ICP Dashboard it is only the stake not the staked maturity.

That’s correct.

Also correct.

That’s right, the Neurons page does not have charts that show staked maturity, though it’s something we’re considering adding in the future.

1 Like

Just to be sure, this shows only the “Staked” maturity, not the “Available” maturity. Right?
If it is so, would be nice to know also the Available maturity…

Correct.

The NNS Governance canister doesn’t export available maturity (i.e., maturity that is not staked) broken down by dissolve delay, but you can get the total available maturity using https://ic-api.internetcomputer.org/api/v3/governance-metrics/governance_total_maturity_e8s_equivalent. That value is just the available maturity and does not include staked maturity.

2 Likes

@diegop It’s been a year and a half since any update on Open Sourcing this API. I haven’t heard or seen this prioritized anywhere, but I see people touting cycle and growth metrics.

It’s feels like all this was brushed aside.

How can we trust any of this aggregate data if the collection methods and APIs aren’t’ open sourced? Is there a backdoor into the ICP to retrieve and scrape customer and application canister data that we don’t know about?

There really isn’t any way to know without more transparency here.

2 Likes

Totally reasonable. I am not aware of the dashboard teams work and priorities so I have pinged folks internally.

1 Like

This seems like quite an important issue, and I agree with the OP. Any more details please? @Dylan

1 Like

Not gonna lie, this is starting to become suspicious.

1 Like

Keeping this one warm for the monday morning

Hey folks,

I asked R&D leadership at DFINITY and the current thinking we are currently working on providing more data about the IC operation that then anyone can use. Resources are limited so that is higher priority in the stack.

This include data about the replica operation (so that anyone can verify whether or not nodes behave correctly) as well as data for canister developer (cycles consumptions etc) that will allow them to better develop their code.

So essentially, we are prioritizing work on public metrics that are to the benefit of the IC.

I realize its not exactly what folks are asking, but it is the current thinking when the “reality of limited resources” meets “balancing protocol-level work.”

4 Likes

@diegop

I know you’re doing your best to communicate word from R&D leadership, but their stonewall message communicated through your extraordinarily kind persona really feels like a slap in the face.

DFINITY has indeed been open sourcing the majority of its repositories over the past two years, but conveniently the IC API has been left out, regardless of the fact that this is the only repository that has a thread on the forum asking DFINITY to open source it.

Getting priorities from R&D leadership has nothing to do with open sourcing the IC API.

I know that there is a small team within DFINITY that is specifically in charge of off-chain tooling, and data aggregation for the analytics that power the IC Dashboard. This team is NOT working on AI, ckERC-20, or whatever the current roadmap items and priorities are.

I’m sorry again, but this is a deflection. You’re talking about developer metrics, and I’m talking about network metrics, for which all of the bases of growth and usage of the Internet Computer Network are based off of, both in terms of marketing materials and read/used by investor analyses.

If it isn’t a deflection, can you be more specific on the metrics and data that will be made visible?Hopefully through a pathway that is open source?

Again, this is not work that requires protocol engineers to work on it. Many of the engineers that work on the IC Dashboard and the IC API don’t work at the protocol level. Unless there was a huge shake-up recently and that team was fired, that team has primarily been siloed to off-chain tooling & infrastructure for DFINITY. All of that is off of the IC (AWS), and there’s nothing wrong with that…it’s A LOT of data and would be unmanageable to all have on the IC.

There are a few thoughts that go through my mind seeing this lack of response over the past 18+ months, ranging from most likely all the way to conspiratorial.

  1. DFINITY does not care about open sourcing off-chain components of their system.
  2. DFINITY believes the IC-API and off-chain components of their infrastructure are insecure, and are therefore uncomfortable open sourcing them.
  3. The off-chain metrics and aggregation is error prone, so it is possible that a bug will reveal false statistic in the IC Dashboard which could be embarrassing.
  4. The off-chain components reveal a data export from the IC that DFINITY engineers can use to read private canister metrics and/or state for “debugging purposes” that would normally only be available to the controller of the canister.
  5. DFINITY is purposely manipulating/specific data that is released through the IC-API to the dashboard. (i.e. hiding the neuron information of several high level team members so that their neuron activity and voting power is not searchable/viewable through the Dashboard)
5 Likes