Create a nit module to combine all latest information in AssetTree

[sync-by-unito]

Problem:

Current NSE shows the information from the latest assetTree. However, it is also possible that the important information such as creator is not included in the latest commit.

It will be good if we can pull information from all AssetTree and create a latest snapshot of the asset. This will has different meaning of the latest AssetTree but should be useful.

┆Issue is synchronized with this Asana task by Unito
┆Created By: Tammy Yang

[sync-by-unito]

➤ Tammy Yang commented:

Working on status page this sprint, push the task to 23.03.27

[sync-by-unito]

➤ Tammy Yang commented:

Suggest to create some miro flow and discuss with Bofu Chen first.

[sync-by-unito]

➤ Olga commented:

Tammy YangBofu Chen There are two different implementations:

1. In the "claap ( https://app.claap.io/numbers-protocol/nse-update-and-api-to-combine-asset-tree-files-c-O35CsUM4Uy-CBov1Sh1Dv-v )" implementation, the API combines the list of asset trees, treating each asset tree as a separate piece of the overall asset tree structure.
2. In the "nit ( https://github.com/numbersprotocol/nit/blob/main/src/run.ts#L513 )" module implementation, the newly added asset tree is combined with the latest one, treating each asset tree as a complete asset tree structure.

I believe that the performance of #2 is better because it always retrieves the latest asset tree and combines it with the added one. On the other hand, #1 needs to retrieve all asset trees and combine them with the added one.

I suggest updating the #1 implementation to match the approach used in #2. However, if you prefer to keep the "nit" module aligned with the #1 implementation, we can introduce a new command called "nit merge" to display the merged asset tree.

Please let me know your thoughts on this approach.

[sync-by-unito]

➤ Bofu Chen commented:

Olga I'm checking the material above

[sync-by-unito]

➤ Bofu Chen commented:

Olga 先走 nit merge (align with the #1 implementation)

data sources 為四個鏈：Numbers, Ethereum, Avalanche, NEAR

我們的目標是先有第一版實作，並瞭解執行效能

可以預期效能一開始不會太好，因此麻煩測試實作時，紀錄幾個效能指標

• 從每個鏈取回單筆資料的所需時間
• 從每個鏈取回全部資料的所需時間與資料筆數
• 完成整個動作的所需時間
  • 若平行處理不好做，可以先單線處理

[sync-by-unito]

➤ Olga commented:

Bofu Chen please help review PR 28 for the nit #28 ( #28 ).

The performance varies for each blockchain. The time to retrieve assetTrees depends on whether the file is already on the public gateway.

$ ./bin/nit.js merge aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

For Numbers with 24 blocks:

• Get commits: 10 seconds
• Get assetTrees: 7 seconds
• Total: 17 seconds

For Avalanche with 21 blocks:

• Get commits: 8 seconds
• Get assetTrees: 1 second
• Total: 9 seconds

For Ethereum with 1 block:

• Get commits: 2 seconds
• Get assetTrees: 2 seconds
• Total: 4 seconds

$ ./bin/nit.js merge bafkreid2izzo5nk4g5edzcd7toyxpfg57m53nx2ggvaropy755a5j2k3pu

I use the CID because aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is not available on Aurora.

For Aurora with 2 blocks:

• Get commits: 1 second
• Get assetTrees: 6 seconds
• Total: 7 seconds

[sync-by-unito]

➤ Tammy Yang commented:

bafkreiafr2brcwink7zfoa4tbvjrmhpnw2hyr22gygk2ccjwyt4bqwehmm

[sync-by-unito]

➤ Olga commented:

$ ./bin/nit.js merge bafkreiafr2brcwink7zfoa4tbvjrmhpnw2hyr22gygk2ccjwyt4bqwehmm

The assetTrees are timing out from the w3s, so I have changed the IPFS source to Numbers.

Jade 4 blocks:

• Get commits: 3 seconds
• Get assetTrees: 2 second
• Total: 5 seconds

Aurora 1 block:

• Get commits: 2 seconds
• Get assetTrees: 2 second
• Total: 4 seconds

[sync-by-unito]

➤ Olga commented:

claap https://app.claap.io/numbers-protocol/nit-merge-asset-trees-c-O35CsUM4Uy-v8umJHntvBS5 ( https://app.claap.io/numbers-protocol/nit-merge-asset-trees-c-O35CsUM4Uy-v8umJHntvBS5 )