Since the beginning of this year untill now I have had exceptional progress on the OctoMY™ project, in large part thanks to the
few bold goals set in the beginning of the year.
I have managed to
deliver on this plan, and I consider this strategy change was a success. By maintaining motivation for development and also digging the project out of some really bad places, it is now in a much better shape. So where is the MVP?
During the process many bad things have come to light both from architecture but also from a code perspective, which is good. And most of them have been mitigated somehow, either by refactoring, removing or rewriting, which is awesome! But
one big issue remains which has proven hard to fix; the asynchronous stores.
At some point in the distant past I decided that I needed a generic component that would allow me to access data asynchronously. The problem that prompted this was keystore. Generating a key takes several seconds, and large parts of the application are useless unless there is a key present. However, having the UI pause for several seconds at start up was rightfully deemed unacceptable, so I created a way to generate the key in a background thread and wrapped all access to the key in a layer of futures with stored list of callbacks. So if you wanted the key, instead of doing
myKey=keystore.getKey();
you would do the following:
keystore.getKey().onFinished([](myKey){ /*do whatever with the key*/})
While this system seemed simple enough, it soon became clear that the implementation would be non-trivial. How could I ensure that the callback was executed from the same thread that booked it?
In Qt this is very important. Also I decided that I should use this fancy new asyncronous API for other parts of the program, not just keystore. Before I knew it it had bloated up and become difficult to manage. The API had diverged into several simplification layers on top and it was getting really messy.
Also, one of the main ideas of OctoMY™ was the concept of a "plan", which basically is a small language that would describe a configuration of many nodes completely, and allow them to collaborate on pretty elaborate tasks from the rules of their common "plan". While some effort was made to allow for easily editing and parsing plan files, it never really took the center stage of storing data about nodes. Currently data is spread out among the following stores:
- keystore - keypair for local and pubkeys for friends
- addressbook - friends data such as name gender and type (except key, which is in keystore)
- localidentity - local data such as name gender and type (except key, which is in keystore)
- agentconfig - agent spesific configuration such as controller card config and similar
- settings - all non-essential settings such as window placements and last selected items.
Many of these "stores" are customers of the aforementioned asynchronous API and all of them show bugs, preventing us from getting to an MVP.
So now it's time for a new update to the strategy, to attack this last problem standing in the way of our glorious MVP! This time around we will focus on the following:
- plan spec - Spec up a decent first version of the plan. Take whatever we learned form the asynchronous store into consideration and ensure the spec is clean and nice.
- plan implementation - Implement the plan spec to have a first working version.
- plan test - Implement a full test suite for the plan implementation to gain full confidence in it.
- plan integration - Integrate the plan by replacing each store one by one.
Once this plan business is out of the way then:
- MVP - Continue work towards MVP.
While this approach in many ways constitutes "one step back, two steps forward", I think it is the better way. By doing this we address the real problems in front of us in a good way and can move forward with our pride intact. The alternative is soldiering on with debugging the asynchronous API, but I think that is not a fruitful use of time at this point.
Work on the plan spec has already begun, and I will try to keep this blog with some updates along the way. Stay tuned!
