Hey fellow IC developers, I’m interested in learning your opinion on whether would it be valuable if DFINITY implemented a framework for managing large amounts of structured data. In the past we had plans to implement something called BigMap, however, this project was never fully released in the o…

Yes! Byron is doing some great work on this and we recently had a number of entries that came at it from different angles for our QuickStart bounty and Sudograph is coming along as well. In my opinion, we need a Manhattan Project style effort like what is happening with the ledger right now to sett…

@icme @lastmjs - thanks for the input. Here’s where I’m coming from on this topic: It’s great to see that the community is coming up with various implementations of a DB framework As a product manager, I’m interested in learning more about the developer use cases and the demand. I want to know wh…

@skilesare Thanks for the ideas! [image] skilesare: mydata.getCertificateRoot((); mydata.getWitness(collection, key); mydata.validate(witness); What would this code do?

Create a Merkle tree of your data so that you can certify it and return it with queries to applications wanting to know they can trust the data. You can use the out put for CertifiedData | Internet Computer Developer Portal edit: To be blunt, all the apps we are writing are just web2 wanna be apps…

I think that scalable storage architecture is better to form on a case by case basis based on the specifics of a system/dapp. Bigmap-like libraries can cover many use cases, but for a sample, the ledger block storage and the openchat user storage use different scaling techniques. Openchat gives each…

In short, I definitely still want and need a BigMap. Speaking from a Motoko developer’s POV, the status quo is insufficient for a variety of reasons: Stable variables are the easiest way to persist data across canister upgrades. Currently, stable variables are limited by the size of the canister…

[image] mikhail-turilin: I want to know what types of apps will be easier to build if we had a database framework. I would say that most cloud developers coming from web2 expect not just scalability, but having the complexities of that scalability abstracted away so that they can focus less o…

[image] mikhail-turilin: Since today building a true DBMS on IC is challenging because of platform constraints, I’d like to build a better mental map of potential developer benefits to make a good case for underlying platform improvements. I think this would be a great opportunity and time to…

I don’t know if people really want BigMap or a NoSQL database. I suspect that what people actually want is the freedom to write code using whatever data structures they deem appropriate for the task at hand, and that they would love to never have to think about the limits imposed by the implementat…

@paulyong I want to give some thoughts on what you said. I think you’re basically correct, or at least in line with my thinking. As in, I think DFINITY/Internet Computer should provide basic stable/scalable data structures. Then developers can take those data structures and do what they need. So f…

Do we still need BigMap / NoSQL database?

Developers

icme May 21, 2022, 4:25am 12

I think this would be a great opportunity and time to start an “Infrastructure and Scalability” developer working group

Topic		Replies	Views
Database-like searching/querying with BigMap/BigSearch Developers	32	5698	July 15, 2021
Abstract away the 4GB canister memory limit Developers	26	3898	June 29, 2021
Sqlite backed by stable memory: the perfect match? Developers	36	4956	July 12, 2022
ICP.Lab Storage & Scalability Summaries Developers	18	4862	April 9, 2025
CanDB, the first scalable NoSQL Database on the Internet Computer Developers	13	5065	July 21, 2023

Do we still need BigMap / NoSQL database?

Related topics