2019-12-21

Productivity matrix

I have been thinking about productivity lately. I have dubbed my idea "swat matrix team", deriving from the fact that it talks about how to organize teams in a way that both resembles "swat" teams and leverages the skills of individual team members as a "matrix. (Sorry Neo, you are not the inspiration this time).
Swat robots. Image credit Morgan Allen.
The gist of it is that we create a small team of resources each expert in their own domains working together like a unit ("swat" team). For example you have one dev-ops, one UX, one back-end and one front-end specialist working on the same project.

All should take part in git review for each other and try to learn the trades of each other. In fact the learning should be formalized in rotating bi-weekly "apprenticeships designations" where pairs of team members are responsible for mentoring each other in their respective fields of expertise. For example dev-ops will teach deployment in kubernetes to UX, UX will teach user testing to dev-ops.

The actual work for the project will be the learning material used. In this example UX Tasks will be assigned 25% to the UX apprentice with the tasks with most learning potential being assigned first.

The idea is that by doing things you are not comfortable with you will be on high alert and your attention to detail and best practices will be heightened. By having an expert by your side you can maintain confidence that the result will pass the bar.

For larger projects multiple swat teams will work together. The apprenticeship is maintained within each team, and each team is responsible for one easily separable part of the project. Rotation of the members will happen between teams, so that the UX of one team will swap with the UX of another, facilitating cross-team knowledge sharing while still maintaining coherent teams.

2019-12-20

Screens vs productivity

It's been awfully quiet lately in the OctoMY™camp. Things are moving under the hood as always, albeit in a different manner than you might think.

All development has been put on hold the last few months an a new secret project has been put into motion in an effort to gather some funds for OctoMY™.

Without spoiling anything, here is a tip that might help you improve productivity in your own projects!

Buy more screens. Lots of them. My current setup is like this:
Big-H configuration
I have two vertically aligned 1080P 24" screens on the sides and two 4k 28" horizontal screens in the middle. The three "tiny" 15" screens above are actually touch screen computers. I use them for testing OctoMY builds during development and for showing operations statistics.

Anyways, getting used to this amount of screen real estate takes some time, but after you get used to it there is no going back. Your brain will accept and learn to take advantage of windows staying put in one location without having to switch around too much, and your way of working and speed will gradually increase.


Also a bonus tip that has also contributed to my current level of working speed; leave your computer and screens on even when you are not using it. I know this is a big no-no in todays environment-focused mindset. I think of it this way; the innovation I create with this setup will in the future far offset the power I spent having my screens on. And if you don't feel the same, convert your energy supply to a environmentally friendly one. You should have done that regardless if you really care....

Have fun and happy coding!

2019-07-04

Plan is dead, long live Dogma!

As explained in the strategy update of 2019-06-16, We will continue the effort of completing the "plan" part of OctoMY™. But "plan" is not the best of word to describe this concept. In this post will explore the alternatives. I want the word we settle on to be self explanatory, short and memorable.



So what is it that we will be naming?

It is a declaration of all the rules by which a group of agents will abide. The rules are written in a declarative language where the possible states and the possible transitions between them are described in high level terms.

The transitions are triggered by external events available to the individual agent via their respective hardware, and deterministic logic, and it is up to the individual agent to determine in which state they ought to be at any point in time.

The collaboration of agents is supported by the specification of shared secrets that allow agents to communicate securely whenever within range of one another. Each agent may interpret communication received from both trusted and distrusted agents.

Agents are expected to have a set of sensors that provide good input to form events from. Events may be such things as a change in location, a discovery of a landmark or identification of a person.

While the "rules" are usually handwritten by the people that own the agents, it should be possible to generate the rules programatically, and the syntax of the language may be extended in the future to support powerful open-ended ways of controlling the agents.

Using https://www.thesaurus.com I have arrived at the following alternatives that more succinctly describe what may be the most important part of the OctoMY™ project:
From this, I think the one I like the most is "Dogma". While edict is maybe more accurate, dogma sounds cooler and I never heard the word edict before I did this research. So from now on, "plan" will be renamed to "Dogma".

2019-06-16

Migration from github to gitlab complete

After the last post on migration away from github, I have finally been able to act on it!



From today the official git repository provider of OctoMY™ will be gitlab under the following url:

https://gitlab.com/octomy/octomy

A mirror has been set up so that all pushes to gitlab will automatically be available on github. This will maintain any SEO and existing users or users preferring github. Also Travis CI only supports github. For the future we are looking into using some of gitlab's built in CI functionality as well.

Strategy Update

Since the beginning of this year untill now I have had exceptional progress on the OctoMY™ project, in large part thanks to the few bold goals set in the beginning of the year.

I have managed to deliver on this plan, and I consider this strategy change was a success. By maintaining motivation for development and also digging the project out of some really bad places, it is now in a much better shape. So where is the MVP?



During the process many bad things have come to light both from architecture but also from a code perspective, which is good. And most of them have been mitigated somehow, either by refactoring, removing or rewriting, which is awesome! But one big issue remains which has proven hard to fix; the asynchronous stores.

At some point in the distant past I decided that I needed a generic component that would allow me to access data asynchronously. The problem that prompted this was keystore. Generating a key takes several seconds, and large parts of the application are useless unless there is a key present. However, having the UI pause for several seconds at start up was rightfully deemed unacceptable, so I created a way to generate the key in a background thread and wrapped all access to the key in a layer of futures with stored list of callbacks. So if you wanted the key, instead of doing


myKey=keystore.getKey();

you would do the following:

keystore.getKey().onFinished([](myKey){ /*do whatever with the key*/})


While this system seemed simple enough, it soon became clear that the implementation would be non-trivial. How could I ensure that the callback was executed from the same thread that booked it?

In Qt this is very important. Also I decided that I should use this fancy new asyncronous API for other parts of the program, not just keystore. Before I knew it it had bloated up and become difficult to manage. The API had diverged into several simplification layers on top and it was getting really messy.

Also, one of the main ideas of OctoMY™ was the concept of a "plan", which basically is a small language that would describe a configuration of many nodes completely, and allow them to collaborate on pretty elaborate tasks from the rules of their common "plan". While some effort was made to allow for easily editing and parsing plan files, it never really took the center stage of storing data about nodes. Currently data is spread out among the following stores:

  • keystore - keypair for local and pubkeys for friends
  • addressbook - friends data such as name gender and type (except key, which is in keystore)
  • localidentity - local data such as name gender and type (except key, which is in keystore)
  • agentconfig - agent spesific configuration such as controller card config and similar
  • settings - all non-essential settings such as window placements and last selected items.
Many of these "stores" are customers of the aforementioned asynchronous API and all of them show bugs, preventing us from getting to an MVP.

So now it's time for a new update to the strategy, to attack this last problem standing in the way of our glorious MVP! This time around we will focus on the following:
  • plan spec - Spec up a decent first version of the plan. Take whatever we learned form the asynchronous store into consideration and ensure the spec is clean and nice.
  • plan implementation - Implement the plan spec to have a first working version.
  • plan test - Implement a full test suite for the plan implementation to gain full confidence in it.
  • plan integration - Integrate the plan by replacing each store one by one.
Once this plan business is out of the way then:

  • MVP - Continue work towards MVP.
While this approach in many ways constitutes "one step back, two steps forward", I think it is the better way. By doing this we address the real problems in front of us in a good way and can move forward with our pride intact. The alternative is soldiering on with debugging the asynchronous API, but I think that is not a fruitful use of time at this point.

Work on the plan spec has already begun, and I will try to keep this blog with some updates along the way. Stay tuned!

2019-05-19

Short progress report

Despite hectic personal life with a dash of illness on top I have manged to make progress on OctoMY!

Here is a screenshot of my latest creation; the debugger widget. Basically I decided to make all internals of the node explicitly available through a debugging user interface. The feedback and interaction this provides greatly expands on what has been available through the log output that has served as the primary debugging data so far:

2019-05-04

Introducing the GibiNeuron!

The following are terms to describe the number of neurons in a neural network.

Metric prefix

UnitShorthandPowerNumber of Neurons
Neuron N 1 Neuron
Kiloneuron KN1 0001 1,000 Neurons
Meganeuron MN1 0002 1,000,000 Neurons
Giganeuron GN1 0003 1,000,000,000 Neurons
Teraneuron TN1 0004 1,000,000,000,000 Neurons
Petaneuron PN1 0005 1,000,000,000,000,000 Neurons
Exaneuron EN1 0006 1,000,000,000,000,000,000 Neurons
ZettaneuronZN1 0007 1,000,000,000,000,000,000,000 Neurons
YottaneuronYN1 0008 1,000,000,000,000,000,000,000,000 Neurons

Binary prefix

UnitShorthandPowerNumber of Neurons
Neuron N1 Neuron
KibineuronKiN1,0241 1,024 Neurons
MebineuronMiN1,0242 1,048,576 Neurons
GibineuronGiN1,0243 1,073,741,824 Neurons
TebineuronTiN1,0244 1,099,511,627,776 Neurons
PebineuronPiN1,0245 1,125,899,906,842,624 Neurons
ExbineuronEiN1,0246 1,152,921,504,606,846,976 Neurons
ZebineuronZiN1,0247 1,180,591,620,717,411,303,424 Neurons
YobineuronYiN1,0248 1,208,925,819,614,629,174,706,176 Neurons

2019-04-22

QtCreator - Not the hero we deserved, but the hero we needed.

When people ask me what my favorite programming language is (for some reason I get asked this a lot), I answer C++ without hesitating, then after a short pause I add "...and Python as a second".

At this point I usually see the person in front of me smirking ever so faintly and I can almost hear their thoughts "such an old fashioned language luckily he got to his senses with Python". I don't mind, I just smirk right back at them because the people asking this are usually coming from a background in Java/C# or JavaScript :D

The follow-up question that I get when I am facing someone more cultivated than to be dissing on a developer's favorite language is "what is your favorite editor or IDE". And to this I answer QtCreator without as much as blinking, followed by a lengthy praise of it's stability, speed, portability and carefully curated set of features that I have come to love above all else.

So in this post I decided to share why exactly I love QtCreator so much in more detail.

Speed


When I go on about how quick QtCreator feels I just don't seem to get the point across. If I am talking to someone using JetBrains' latest incarnation or VisualStudio people look puzzled as why I mention speed as the main reason. I guess they don't value their time.

Working as a consultant I a will encounter all kinds of environments and yes I have used versions of Visual Studio from version 1.0 to the latest 2019. And in each version of VS I wait for what feels like hours every day. Every little nudge of the mouse seems to activate some kind of intellisense that blocks the entire UI for seconds while loading, also blocking the view under the empty box that is displayed. Or the project needs to be refreshed and it takes forever even for moderately sized solutions.

Or I have to manually delete obj and bin folders, just because. When re-scaling windows or re-arranging the UI it just staggers along. Scrolling stutters. Loading files is slow. Searching in project is slow. Everything feels... sluggish. Like swimming in syrup.

In QtCreator you never even think about it. It just is fast and you never loose track of your really important thoughts and ideas. The IDE simply never gets in your way.

VS, JetBrains and even Eclipse has really awesome refactoring tools that I love, but it is just not worth the wait for me.

Super important features™


There are many small things that combined gives you an edge as QtCreator user. Here are some examples:

  • The locator thing to open files, classes, line numbers etc is just completely awesome.
  • Splitting panes is awesome.
  • Search and replace tool is awesome.
  • I didn't initially see the point of kits, but they really grew on me, and now I think they are a really useful feature which I find is lacking in other IDEs. 
  • The excellent integrated example projects with full writeup and screenshots were a big hit when I was learning to use Qt and QtCreator for the first time.
  • The embedded help system actually works and is useful. And best of all... it is not full of "news".

Portability


Qt is portable. It is one of it's main selling points. And QtCreator is too. I love the fact that I can decide what platform to develop on, and not have to install Java or Mono first. QtCreator is in fact a snappy native app developed in Qt to spend no amount of memory, load fast, work fast and feel responsive on all platforms that it is built for. I especially love that it is native on Linux.

qmake

I see a lot of Qt devs hating on qmake and using cmake instead. Or automake. Or something else. I think about it differently. If you look at the goals of those tools, I think qmake makes a far better job of meeting it's goal than cmake. qmake was designed to build Qt applications while cmake is to build "any C++ application with any dependencies". I have not tried Qbs yet, maybe it is even better, but qmake serves me really really well and the tight integration in QtCreator makes me very happy. I have a huge project with lots of code. All of it builds with just one qmake build on all the platforms we support. There are no strange error messages to google or changes to the path needed on windows only. It just works.

Active development

QtCreator has gone through a bunch of really nice updates since I started using it a few years ago. To name a few:
  • Dark theme
  • Real-time compiler hints via clang
  • New icons/profile
  • Many small fixes and features such as separating out the -j parameter for parallel builds in make.

Can it be better?

While I love QtCreator, I stil have some wishes for the QtCreator devs going forward!

  • A place to download plugins/extensions. I come across posts online that state QtCreator has a rich plugin architecture but I never found a single plugin or extension, nor any information on where to find them or install/use them. If it was made available somehow in an app-store that would be awesome!
  • Faster loading speed, and integrate it into O/S like text editor so I can use my IDE as a text editor. This is in the "nice to have" category for sure, but just imagine it. It would definitely be a killer feature.
  • Better support for other languages such as Python out of the box. I would love to add a Pyhon project and at least be able to run it using the Python environments that are already on my computer without having to do any manual configuration.
  • I know this is probably a big change, but. I think that having the UI designer as a separate "mode" is a big mistake. It should be opened inside an editor tab instead of the completely separate mode screen. The UX for switching between source and UI editing is clumsy at best.
  • The maintenance tool is awesome except... the UI. I mean com'on guys, Qt stands for excellent UIs, why this abomination from the 1990'? The update process could be streamlined so that download happens automatically in background then a subtle button would simply ask if you want to restart for update to take effect or similar.
  • More C++ refactoring options. Basic things like "move class to separate file" and "make const version of method" etc.
  • Include native support for cling. Not only would CERN love you, but the rest of the world would too. Think about the shift in development cycle? Just edit code in real time while app is running. It would definitely be a killer feature.
  • Apply dark theme to integrated help. I want my choice in dark to be for ALL of QtCreator!
  • Better help integration. I mentioned that that the system help works and is useful, but there are some times when I hit F1 to see relevant help (for example with cursor on a member inside Qt header file) and nothing happens. It should recognize context for help more places. Even mark it somewhere that hitting F1 now will give you relevant help.
  • Search and replace is nice and fast. But it is not as powerful as in notepad++. I didn't know I missed those features until I used them once. Also the UI for search/replace is sometimes cumbersome. I don't know how the case-insensitive, regex and whole words features works. Adding some context help and a list of examples would improve it a whole lot.
  • The project tree is either too sparse or too cluttered. Giving more control of how to show the project and the files in the tree would be nice.
  • The highlighted lines feature where errors, warnings, search hits and more are super useful, but it is not possible to click them to jump directly to the line in question, which in my opinion is a sorely needed feature. Also I think they could be symbols that are grouped when there are many in the same area. When you hover the group it will expand to how all the marks so you can click them individually. It is difficult to remember the different colors. I still don't know what white and green means.
  • I wish the integrated examples were updated as some are showing their age. I understand it is a conscious choice to not spend time on this, but marketing apt. of The Qt Company / Digia should know how huge an impact this feature has on beginners (read: potential customers). Keeping it updated and fresh should be among top 10 points on marketing plan!
There, all done!

Now I can go back to programming.

2019-04-16

March/April update 2019

A silent blog for a one-man project usually means one of two things:

1. The blog or project (or man) died
2. The project is in high gear and there is no time for silly things like blog updates.

Luckily for all involved, in this case it is the second, as we are making huge strides of progress in the project! Here is a short summary of recent improvements:


  • A debugger has been implemented that shows internal variables and state for a variety of features in real time in the UI of the nodes. This will allow for much quicker debugging.
  • A service manager has been implemented to automatically resolve dependencies of "services" for the node while activating or de-activating features such as discovery.
  • The UI of agent and remote has both been rebuilt from scratch and they now have discovery working.
Future effort will be to gradually add support for one new feature at the time from the list below while maintaining a working system. Slowly but steadily we will bring the platform up to full potential while ensuring no parts stop working along the way:

  1. Pairing
  2. UI for enabling/disabling discovery
  3. UI for enabling/disabling pairing
  4. AgentStateCourier
  5. Multi client support in remote
  6. Multi client support in agent
  7. UI for selecting client in remote
  8. UI for selecting client in agent
  9. UI for selecting control level in remote
  10. Generalized support for adding arbitrary UI widgets for different control levels in remote
  11. UI for actuator control level
  12. UI for animation control level

Beyond this we need to revise the plan (and probably it will be revised many times before we reach this stage).

Again I have to extend my gratitude to the breathtaking Ms.Y for her gracious mental support!


Army of Spots spotted

So scary. But so cool!

2019-02-05

February update

This is just a quick update on the progress of the OctoMY™ project.

The work to implement the plan set forth at the anniversary is well underway!

Today I pushed maybe one of the most substantial commits made so far in the entire history of the project. Nothing has changed in the main branch of the code, but if you look under integration/v2 you will find a complete copy of the root.

Why did I do this instead of using a git branch? Because it allows me to directly compare the two and copy between them without having to check out a separate repository.

In the new "branch" the following has so far been done:

  • Number of compiler warnings has been greatly reduced. The entire build now generates just 76 warnings! The work to remove them will continue.
  • Libraries have been renamed, merged, split, and removed so that the whole namespace is much cleaner.
  • librest has been renamed to more appropriate libcombined.
  • Widgets and their respective .ui files have now been distributed out to the libraries where they belong.
  • Resource files have also been distributed out to the libraries where they belong. Some more tweaking of the build is needed for resources to be properly linked though.
  • Many folders including those of libraries have received their own README.md file that describe them better. There is still room for more.
  • Many files have had more information put into comments, especially for the build system!
  • Build system now looks for .ui and resource files like it does for sources.
  • Build system was modularized a little and cleaned up a bit.
  • A lot of sources have been brought up to spec with regard to coding style!
  • The build time has been cut in half (!!) mainly because of splitting Utility header into parts with the most used parts (OC_METHODGATE, OC_NEW and OC_CONTYPE) being put alone into dedicated headers (but many other small changes may have contributed).
  • The initial startup code of Node & al. has been massaged to work much better and tested with a Mock Node. The startup of Node is now well documented.
  • Lots of  objects are now passed around as QSharedPointers instead of raw pointers or raw references. Countless classese now have empty contstructor parameter lists and instead have a separate .configure() method to do the actual initializations which avoided the bad practice of running code from constructor initialization list (althought RIIA suffered a little, I think this was a worthwhile update).
  • A completely empty Agent and matching AgentWindow class has been created ready for constructing a new Agent from the ground up.
I was honestly worried that the goal I set for this project was too big. A huge thanks must go to Mrs. Y for her motivational support!

Now that we are here, the "real" work starts! I am really excited!

Update will follow as soon as there is anything to report...

2019-01-07

Happy Anniversary 2019

The 7th of January 2019 marks the three year anniversary of the OctoMY™ project. As promised, a roadmap for the project will follow. Great things are ahead!

A new version has been created in-tree simply called "v2". It started as an empty folder, and will be gradually filled with all the parts of the project code that works.

Each part will be scrutinized thoroughly and polished up to meet a high standard. The following rules apply:

  • No compiler warnings or errors allowed.
  • Some libraries renamed to better more descriptive names.
  • Some libraries merged.
  • Some libraries split up, especially libcore and libutil.
  • Some libraries or parts will be removed outright.
  • Widgets and ui files are distributed among the libraries they belong to.
  • Resource files are distributed among the libraries they belong to.
  • All libraries receive README.md to describe them better.
  • Formatting of sources modified to follow the standard.
  • Translation unit optimizations to reduce buildtime.
Once all the dependencies have been cleaned up the next phase begins;

  1. Empty dummy classes called NodeV2 and AgentV2 are created that will build and run but not actually work.
  2. The messy interfaces of node and agent are cleaned up and homogenized.
  3. Each feature is re-integrated one by one into the V2 classes ensuring that they work, creating unit and integration tests as the project progresses.
This will eventually leading to the completion of the MVP, our first minimum viable product.

Once this MVP works it will be tweaked, optimized and polished in stages until we have a realease candidate for testing.

This roadmap has the advantage that the progress can be measured and followed closely. Instead of crossing fingers and hoping, we can actually see what remains on a burn down sheet.

We are looking forward to work further on this, the work already started. Happy anniversary!