Author Topic: Introducing MOFOWALLET.com  (Read 20287 times)

Eliphaz Fimk #30 on: November 05, 2014, 05:22:31 PM

  • FIMKrypto coordinator
  • Administrator
  • *****
  • Posts: 806
    • View Profile
    • FIMKrypto
I was just suggesting technical adjustments for the head developer publicly.

vamos2014 #31 on: November 05, 2014, 05:31:32 PM

  • Fresh Fimker
  • *
  • Posts: 7
    • View Profile
I was just suggesting technical adjustments for the head developer publicly.

It seems as if the customer would be continuously connecting and disconnecting from the nodes, spending many resources.

I have seen that the nodes can be configured, if there is any change that would help the client work better, say that to me.

Thanks

Dirk Diggler #32 on: November 06, 2014, 12:34:54 AM

  • FIMK Staff
  • *****
  • Posts: 486
    • View Profile
    • Krypto Fin ry
It seems as if the customer would be continuously connecting and disconnecting from the nodes, spending many resources.
I have seen that the nodes can be configured, if there is any change that would help the client work better, say that to me.
Thanks

Hi. I don't think there is anything you could do about the problem you are experiencing the adjustment of the nodes is limited to:

1. blacklisting public nodes that are down
2. adding more nodes
3. turning on/off per node if they support CORS (special kind of header for the browser) if they don't mofowallet will communicate with those nodes through special proxy servers that add those headers (this is not important in your case)

I too have noticed the absurd amount of requests and it's something I will be working on tomorrow - for now it's just there - are you on a fast connection all works well - on a slow(er) connection and you'll have the problem you are experiencing.
« Last Edit: November 06, 2014, 12:48:50 AM by Dirk Diggler »
FIMK Developer | GPG fingerprint: CEF2 7C39 43BE 6800 504E  71BC 7E87 A7B0 AC34 E2D5 | mofowallet.com | blog

Dirk Diggler #33 on: November 06, 2014, 12:35:53 AM

  • FIMK Staff
  • *****
  • Posts: 486
    • View Profile
    • Krypto Fin ry
This is a somewhat technical explanation of the reason mofowallet seems to perform slow right now. In case anyone is interested read on, otherwise wait a few days and mofowallet will perform much better.

About why mofowallet seems slow at the moment.

Lets start with the fact that I personally have a very fast internet connection. This is nice when downloading movies but not for developing web apps. My os on the other hand is very slow, I use ubuntu and I have a slow graphics card which gives over-all much worse performance than on a comparable windows or osx machine. I see this as a good thing since I know the apps that I develop will run fast everywhere as long as they perform decent on my own system. If I had a slower internet connection I would have noticed much sooner what many users are experiencing now with mofowallet.

It comes down to this: There are many actors in mofowallet, dozens of views a handful of services, multiple blockexplorers, then you have multiple opened accounts that need to know their balance, check for unconfirmed transactions etc etc. All of that stuff wants to know something from the network. Needing to know something from the network means you have to make an AJAX request to one of the many pre-set public servers in the database. Not just any server is selected for your current request, mofowallet looks at which one was accessed the longest ago (to even the load on the public servers) and mofowallet looks at how that node performed in the past (you want to skip the very slow or down nodes) (we call servers nodes - a node is a public FIM/NXT server).

So much for optimization one, then there is the database. All data you see on the screen comes straight from the database - only if data (some transaction, some accounts public key, some asset name etc etc) is not in the database mofowallet will ask for it from the network (in other words make an ajax request to one of the nodes). All is fine when the network can keep up with mofo and data is returned approximately before other data is requested (there can be some overlap - but not too much) but when the ajax response is taking longer than expected and some other part of mofo starts another request, they'll soon start piling or queuing up. I figured that would happen so I already cancel all outstanding requests that were started while looking at account A, navigate to account B and mofo will cancel any outstanding request for account A (and there are a lot, most transactions are downloaded one by one).

I was probably thinking something like: "Wow Dirk, thats smart!" . And partially I was right but unfortunately not completely. A somewhat similar trick was done for the blockexplorer also, look at one block and transactions are downloaded, navigate to another block and all outstanding AJAX requests for transactions for the previous block are cancelled.

But this of course is not the way to fix this. Instead of starting the requests and cancelling them (cancelling started AJAX requests in a cross browser environment is not so straightforward) we need to nip it in the butt by not even starting the requests. Et voila there you have the entire problem in a single sentense:

Don't start the requests! Just don't!!

Then what to do? Thats not so hard luckily. Mofowallet uses Promises for all it's asynchronous operations and Promises can perfectly be stored in for instance an Array which could act like a queue. When I do this instead of just keep starting new requests all will work out just fine. There will be no more active/waiting AJAX requests polluting the browsers memory and I no longer need to cancel active AJAX requests, just removing a pending request from the Array and it's promise and possibly even ignore them completely will be sufficient.

The memory usage will decrease and the responsiveness of mofowallet will increase by that but there remains a problem of the piling requests. Doing it this way mofowallet will perform very snappy but not much data will be showing up in the UI. Thats optimization number two that needs to be made. Certain requests should have higher priority. Right now every 10 seconds a check is done if there are any unconfirmed transactions, this has to be done for both NXT and FIMK (so thats 2 requests every 10 seconds) then there is the check for new blocks which is done every 15 seconds for FIMK and every 30 seconds for NXT and there are a couple more of these repeating requests. When you look at the blockexplorer you want to see A. the blocks and B. the transactions and you want that NOW. So it would be reasonable that when you load the blockexplorer the request for the blocks and the transactions will have higher priority than any other. Same goes for showing your balance when opening an account page or for sending FIM/NXT or a message etc.

So to conclude this much too long forum post - mofowallet will be fixed by implementing:

1. A request queue where AJAX requests are not started until it is actually their turn
2. Give priority to any request for data that the user actually needs to see now

After these fixes mofowallet will probably be so fast that it will travel back in time  8). Really excited to see what comes from that.

Thanks
Dirk
FIMK Developer | GPG fingerprint: CEF2 7C39 43BE 6800 504E  71BC 7E87 A7B0 AC34 E2D5 | mofowallet.com | blog

Eliphaz Fimk #34 on: November 06, 2014, 01:12:35 AM

  • FIMKrypto coordinator
  • Administrator
  • *****
  • Posts: 806
    • View Profile
    • FIMKrypto
Excellent explanation for the less coding inclined. Looking forward for the meaner and leaner mofo!

Hyo #35 on: November 06, 2014, 10:46:49 AM

  • Fresh Fimker
  • *
  • Posts: 24
    • View Profile
Will the possibility of leasing forging-power be available anytime soon?
Could somebody write a step by step introduction how that would work?
Iam in a country with v. weak internet but would like to participate with forging Fimk
best regards

Polarpanda #36 on: November 06, 2014, 11:45:12 AM

  • Fimker
  • **
  • Posts: 70
    • View Profile
I see in your file path that you use the wallet from https://wallet.fimk.fi I looked at that and it seems to not have the latest version, we are fixing that.
To use Mofo Wallet from your browser it's best to go to http://mofowallet.com and launch it from there.
It always points you to the latest version which we host on a fast and secure CDN.
Hosting for the web version is expected to change but http://mofowallet.com will stay the same.

The current version seems to grow fairly large. After being online 18 hours the database had reached 50 MB and was stopped at the Firefox limit.

$ du -m
51   ./Profiles/6u850szr.default/storage/persistent/https+++fimkrypto.github.io/idb

Now I allowed it to exceed the 50MB limit and it's seemed catch up with the missing blocks in 30 minutes and grow pretty fast:

$du -m
101   ./Profiles/6u850szr.default/storage/persistent/https+++fimkrypto.github.io/idb

The database might be big but I doesn't mean all the data was downloaded though.


Dirk Diggler #37 on: November 06, 2014, 11:45:47 AM

  • FIMK Staff
  • *****
  • Posts: 486
    • View Profile
    • Krypto Fin ry
Will the possibility of leasing forging-power be available anytime soon?

This should be possible yes it's a relatively small task to add that. As soon as some other urgent tasks are completed we'll add that.

Could somebody write a step by step introduction how that would work?
Iam in a country with v. weak internet but would like to participate with forging Fimk

We are in the process of setting up blog. Questions like this will be answered in the form of a series of articles on that blog.
FIMK Developer | GPG fingerprint: CEF2 7C39 43BE 6800 504E  71BC 7E87 A7B0 AC34 E2D5 | mofowallet.com | blog

Lunero #38 on: November 06, 2014, 08:33:14 PM

  • Senior Fimker
  • ***
  • Posts: 166
    • View Profile
This should be possible yes it's a relatively small task to add that. As soon as some other urgent tasks are completed we'll add that.

We are in the process of setting up blog. Questions like this will be answered in the form of a series of articles on that blog.
Would it be possible to integrate escrow to payments? So that, once payment has been made, it is locked by the system, so that no one can access it unless both parties agree that the transaction is fully completed?

Dirk Diggler #39 on: November 06, 2014, 10:38:03 PM

  • FIMK Staff
  • *****
  • Posts: 486
    • View Profile
    • Krypto Fin ry
Would it be possible to integrate escrow to payments? So that, once payment has been made, it is locked by the system, so that no one can access it unless both parties agree that the transaction is fully completed?

Yes. That would be a new transaction type in FIMK.
FIMK Developer | GPG fingerprint: CEF2 7C39 43BE 6800 504E  71BC 7E87 A7B0 AC34 E2D5 | mofowallet.com | blog

abctc #40 on: November 07, 2014, 09:00:29 AM

  • Fresh Fimker
  • *
  • Posts: 44
    • View Profile
Would it be possible to integrate escrow to payments?
Yes. That would be a new transaction type in FIMK.

- it has been done in another Nxt clone, so maybe it is possible to acquire to FIMk:
Quote
Escrow transactions

Usage:

In the sidebar, after clicking Transactions, there is an entry for escrow. Click it to go to the escrow screen.
In the escrow screen, there is a button in the upper right labelled "Create Escrow" which will open the escrow create dialogue.
Fill it out like this:
Recipient: The recipient of the escrow in either BURST- or numeric id(you can hit backspace to remove the existing BURST-___ entry to type numbers)
Amount: The amount you want to transfer
Escrow deadline: A time limit in seconds that if runs out, the escrow will be automatically completed with the deadline action selected in the next step. This deadline must be 1 - 7776000(up to 3 months) seconds.
Deadline Action: The action that will occur automatically if the deadline runs out without the escrow being completed. This can be release, refund, or split.
Required signers: A number from 1 - 10, saying how many people need to sign off on a decision to cause that action to be taken. Regardless of what this number is set to, a sender can always release, and a recipient can always refund.
Signers: A list of accounts allowed to sign off on a decision. BURST- addresses and numeric ids can be used. They are separated by semicolons ( ; ). 1 - 10 must be specified. (ex: BURST-AAAA-AAAA-AAAA-AAAAA;1234556;987654321;BURST-BBBB-BBBB-BBBB-BBBBB;1357913579 )
Message: A message can optionally be attached. If you want signers to be able to see the message, it must not be encrypted.

Fees: There is a network fee that is 1 burst by default, and an additional fee equal to the number of signers specified. The additional fee is given to the signers, so they have a coin to make their decision transaction with if necessary. This also makes it easy for people to participate in escrows as signers without having an existing funded burst account.

Once the escrow transaction has been included in a block, it will be listed on the escrow screen if you are involved in it(sender, recipient, or listed as a signer is considered involved)
Clicking on the transaction id in the screen brings up the decision making screen. Various information is shown on it(signers' ids and decisions, deadline time and action, etc)
A decision can be selected from the dropdown and submitted to sign off on that decision.
Senders can select release which will release the funds when it gets included in a block.
Recipients can select refund which will refund the funds when it gets included in a block.
Signers can select release, refund, split, or undecided. When their decision gets included in a block, the escrow details will be updated to show their decision. If their decision makes the number of signers with that decision at least the number of required signers, that decision is executed.

Dirk Diggler #41 on: November 12, 2014, 03:48:28 AM

  • FIMK Staff
  • *****
  • Posts: 486
    • View Profile
    • Krypto Fin ry
Hi,

After learning the kind of problems people were having with mofowallet we decided to take a good hard look at what is actually going on in this enormous pile of code that we wrote and designed in the short period that it took (6/7 weeks in total). Such a short period for the amount of code that is mofowallet is probably as short as you can get it and it is no surprise that there were/are bugs.

As you may have noticed (or not) mofo has been a bit slow to work with, it's not snappy so to say.

A major design flaw that we didn't foresee was to rely on the browser to manage the starting, stopping and queueing of network requests (AJAX requests) this works fine for smaller projects but not for a network hogging beast like mofowallet. Until both FIM and NXT JSON API's are optimized for mofowallet (in some rare cases) mofowallet simply cannot display all the info that should go on a page without doing 20 or more AJAX requests. That said the second time you view that same page the data will be there instantly because of the use of the client side IndexedDB database. But lets continue.. The slowness of mofo was due to a pile up of (AJAX) requests that quickly consumed large parts of the browsers memory. So far, not good.

A second shortcoming was the inability to either prioritize or compartmentalize (AJAX) requests. Let me explain.
If you're on a slow connection and you just opened a page in the mofowallet blockexplorer you could easily kick off 20 to 30 (really small) network requests, each weighing some bytes each. No problem there, just let it load. But what if I click on an asset in the blockexplorer or on an account? If the clicked item's data is not in the database it means we have to fetch that from the network (from one of the public nodes) in the old situation the request for that data that you want to see now had to wait until all previous 20 to 30 requests were done loading. Not ideal, not snappy.

A third shortcoming was the quality of the public nodes, some were down some were on forks, in short a mess. But thank god for that mess! Since now we have solution for that that will keep working even when people start adding their own public nodes. (Did you know mofowallet workded like that? There is not one single NXT or FIM API server, in fact there are many. Mofowallet comes loaded with 14 public FIM nodes and it's easy to add your own. Mofo will optimize communications between those servers to give the best experience possible on a thin client).

Sow how did we solve that?

1.) Mofowallet no longer relies on the browser to schedule it's Ajax requests. There is a service for that now (for the scripters out there; there is a copy of that service here http://paste.ubuntu.com/8952187/ - the prose about podiums and actors is something I use to conceptualize the problem in my mind) the service only runs 6 simultaneous Ajax requests at a time and allows for full control over all pending and active requests.

2.) The new request handling service allows to prioritize requests based on a priority value you give each request. Requests with a higher priority get executed before requests with a lower priority. To compartmentalize requests we introduced the concept of a podium (the requests are the actors that perform on a podium) this was needed to quickly give priority to or completely delete a podium. Such an action can be expected when you navigate from one block in the blockexplorer to another, the requests for loading the transactions in a block go in their own podium. When you navigate to the next block all remaining transaction requests are instantly destroyed. This way prioritizing what a user needs now.

3.) To solve the public node badness and the forking problems we introduced another new service in mofowallet. The startup service, other modules (or plugins that you write) can register with the startup service to do things before mofowallet is fully started (the startup service will show a nice progress window). One of the first startup modules we added was the Fork Scanner this code will ping all registered public nodes and builds an in-memory graph from the data it gets back. It then calculates a median height shared by most nodes and then does a second scan but now for a specific block height. The returned data tells us exactly on what forks each public node is. We then blacklist those nodes and that solves a lot of weird behavior.

You can check out the new mofowallet online only for now. I'll prepare a desktop version tomorrow or the day after. For now please try it out (especially the blockexplorer) and let us know if it works better for you. There are more speed ups of mofo on the way since the new priority feature for requests has not been turned on everywhere.

I have indicated a week ago that around this time we would release an Asset Exchange enabled mofowallet. Unfortunately this has to come later because of the mandatory fixes I described above. We figured that since everything builds on top of the networking layer it had to be done first before continuing with the rest of mofo.

Thanks
Dirk

TL;DR

New version here http://mofowallet.com, online only. Download will follow in the coming days.
It could be your browser cache keeps showing the old version. in most browsers/OS's you can hit CTRL+F5 to refresh your browser cache.
If you have any problems loading pages or you have to wait a long time.. Simply refresh your browser, refreshing your browser will scan the public nodes again that could have been blacklisted the last time you loaded that page. If in doubt look in the Settings / Node section to see if the public nodes are considered to be on a fork
FIMK Developer | GPG fingerprint: CEF2 7C39 43BE 6800 504E  71BC 7E87 A7B0 AC34 E2D5 | mofowallet.com | blog

Eliphaz Fimk #42 on: November 12, 2014, 09:41:38 AM

  • FIMKrypto coordinator
  • Administrator
  • *****
  • Posts: 806
    • View Profile
    • FIMKrypto
Good work from Dirk! Throttling and request prioritization seem to work nicely.

I'm seeing some errors with the new build that need fixing before the downloadable is made available. I'd like to clarify a few basic points related to the structure of Mofo:

Question 1: As "thin client" is the online version downloading partial or full blockchain to the local machine?

Question 2: If not downloading the blockchain, is it downloading and building an own version of the blockchain to a local database, every transaction for every account? If so, what's the benefit over downloading full blockchain, other than being able to query stuff inbetween loading?

Question 3: "Thin client" implies that it wouldn't be downloading much, not quite anything before the user clicks on a link or requests data. Would it be difficult to make a version of mofo that works so? It  should use the resources on the mofowallet server and display them for the user like a standard web page, not using local database on the user's machine at all for other than encrypted wallet purposes.

Dirk Diggler #43 on: November 12, 2014, 11:39:30 AM

  • FIMK Staff
  • *****
  • Posts: 486
    • View Profile
    • Krypto Fin ry
Question 1: As "thin client" is the online version downloading partial or full blockchain to the local machine?

Only partial. It downloads only the blocks and transactions that you are looking at at your screen. Apart from that when you click an account in the blockexplorer Mofo will start downloading the transactions for that account (I call that backfilling) when you go to another account the backfilling stops. Come back to that same account and it will pick up where it left. I figured that if you have interest in an account you would likely want to store it's transactions locally. There also is a repeated check called a getState request, this requests the current *state* from a public node and Mofo will know from that if for instance the last block changed (we have a new block). Such a repeated call is also done for unconfirmed transactions so Mofo can display transactions even when they are unconfirmed.

Question 2: If not downloading the blockchain, is it downloading and building an own version of the blockchain to a local database, every transaction for every account? If so, what's the benefit over downloading full blockchain, other than being able to query stuff inbetween loading?

It is not doing that. There would be no benefit in that.

Question 3: "Thin client" implies that it wouldn't be downloading much, not quite anything before the user clicks on a link or requests data. Would it be difficult to make a version of mofo that works so? It  should use the resources on the mofowallet server and display them for the user like a standard web page, not using local database on the user's machine at all for other than encrypted wallet purposes.

The database is at the center of everything and it's a very special one. With the help of the Dexie library the IndexedDB available in your browser is turned into an observable data store. This is extremely powerful, I can stick something somewhere in the database and other parts of Mofo simply *listen* to changes to the database and then update the UI. For plugin authors this will be priceless and combined with the template system from Angular allows them to build unique and powerful plugins. As for your question. Thin client means several things I agree that Mofo is not so thin when it comes to the amount of code, you might even consider it a fat client in that regard. But in crypto wallet terms it is thin in that it relies on an external API and an external blockchain.
Data/network usage is something that will get much better, the changes made in this version are only the tip of the iceberg of what is still available in the optimization space.
The encrypted wallet is not stored in the database btw (it will probably soon) the encrypted wallet.dat is a file on your computer that the webclient can read through some html/js hocus pocus and the user clicking a button.
FIMK Developer | GPG fingerprint: CEF2 7C39 43BE 6800 504E  71BC 7E87 A7B0 AC34 E2D5 | mofowallet.com | blog

Eliphaz Fimk #44 on: November 12, 2014, 12:13:06 PM

  • FIMKrypto coordinator
  • Administrator
  • *****
  • Posts: 806
    • View Profile
    • FIMKrypto
It is not doing that. There would be no benefit in that.
Ok, not the full blockchain, but some of that it does. Creates a local database of its own format, IndexedDB, while not downloading the full blockchain. Which is fine.

I'm just wondering why it makes 5-10 requests per second while I'm NOT looking at any particular account. I'd like mofo to be silent, or send http requests more rarely (1 per sec, 1 per 3 secs or so) when I'm not requesting info.