QtCreator - Not the hero we deserved, but the hero we needed.

When people ask me what my favorite programming language is (for some reason I get asked this a lot), I answer C++ without hesitating, then after a short pause I add "...and Python as a second".

At this point I usually see the person in front of me smirking ever so faintly and I can almost hear their thoughts "such an old fashioned language luckily he got to his senses with Python". I don't mind, I just smirk right back at them because the people asking this are usually coming from a background in Java/C# or JavaScript :D

The follow-up question that I get when I am facing someone more cultivated than to be dissing on a developer's favorite language is "what is your favorite editor or IDE". And to this I answer QtCreator without as much as blinking, followed by a lengthy praise of it's stability, speed, portability and carefully curated set of features that I have come to love above all else.

So in this post I decided to share why exactly I love QtCreator so much in more detail.

Speed

When I go on about how quick QtCreator feels I just don't seem to get the point across. If I am talking to someone using JetBrains' latest incarnation or VisualStudio people look puzzled as why I mention speed as the main reason. I guess they don't value their time.

Working as a consultant I a will encounter all kinds of environments and yes I have used versions of Visual Studio from version 1.0 to the latest 2019. And in each version of VS I wait for what feels like hours every day. Every little nudge of the mouse seems to activate some kind of intellisense that blocks the entire UI for seconds while loading, also blocking the view under the empty box that is displayed. Or the project needs to be refreshed and it takes forever even for moderately sized solutions.

Or I have to manually delete obj and bin folders, just because. When re-scaling windows or re-arranging the UI it just staggers along. Scrolling stutters. Loading files is slow. Searching in project is slow. Everything feels... sluggish. Like swimming in syrup.

In QtCreator you never even think about it. It just is fast and you never loose track of your really important thoughts and ideas. The IDE simply never gets in your way.

VS, JetBrains and even Eclipse has really awesome refactoring tools that I love, but it is just not worth the wait for me.

Super important features™

There are many small things that combined gives you an edge as QtCreator user. Here are some examples:

• The locator thing to open files, classes, line numbers etc is just completely awesome.
• Splitting panes is awesome.
• Search and replace tool is awesome.
• I didn't initially see the point of kits, but they really grew on me, and now I think they are a really useful feature which I find is lacking in other IDEs. 
• The excellent integrated example projects with full writeup and screenshots were a big hit when I was learning to use Qt and QtCreator for the first time.
• The embedded help system actually works and is useful. And best of all... it is not full of "news".

Portability

Qt is portable. It is one of it's main selling points. And QtCreator is too. I love the fact that I can decide what platform to develop on, and not have to install Java or Mono first. QtCreator is in fact a snappy native app developed in Qt to spend no amount of memory, load fast, work fast and feel responsive on all platforms that it is built for. I especially love that it is native on Linux.

qmake

I see a lot of Qt devs hating on qmake and using cmake instead. Or automake. Or something else. I think about it differently. If you look at the goals of those tools, I think qmake makes a far better job of meeting it's goal than cmake. qmake was designed to build Qt applications while cmake is to build "any C++ application with any dependencies". I have not tried Qbs yet, maybe it is even better, but qmake serves me really really well and the tight integration in QtCreator makes me very happy. I have a huge project with lots of code. All of it builds with just one qmake build on all the platforms we support. There are no strange error messages to google or changes to the path needed on windows only. It just works.

Active development

QtCreator has gone through a bunch of really nice updates since I started using it a few years ago. To name a few:

• Dark theme
• Real-time compiler hints via clang
• New icons/profile
• Many small fixes and features such as separating out the -j parameter for parallel builds in make.

Can it be better?

While I love QtCreator, I stil have some wishes for the QtCreator devs going forward!

• A place to download plugins/extensions. I come across posts online that state QtCreator has a rich plugin architecture but I never found a single plugin or extension, nor any information on where to find them or install/use them. If it was made available somehow in an app-store that would be awesome!
• Faster loading speed, and integrate it into O/S like text editor so I can use my IDE as a text editor. This is in the "nice to have" category for sure, but just imagine it. It would definitely be a killer feature.
• Better support for other languages such as Python out of the box. I would love to add a Pyhon project and at least be able to run it using the Python environments that are already on my computer without having to do any manual configuration.
• I know this is probably a big change, but. I think that having the UI designer as a separate "mode" is a big mistake. It should be opened inside an editor tab instead of the completely separate mode screen. The UX for switching between source and UI editing is clumsy at best.
• The maintenance tool is awesome except... the UI. I mean com'on guys, Qt stands for excellent UIs, why this abomination from the 1990'? The update process could be streamlined so that download happens automatically in background then a subtle button would simply ask if you want to restart for update to take effect or similar.
• More C++ refactoring options. Basic things like "move class to separate file" and "make const version of method" etc.
• Include native support for cling. Not only would CERN love you, but the rest of the world would too. Think about the shift in development cycle? Just edit code in real time while app is running. It would definitely be a killer feature.
• Apply dark theme to integrated help. I want my choice in dark to be for ALL of QtCreator!
• Better help integration. I mentioned that that the system help works and is useful, but there are some times when I hit F1 to see relevant help (for example with cursor on a member inside Qt header file) and nothing happens. It should recognize context for help more places. Even mark it somewhere that hitting F1 now will give you relevant help.
• Search and replace is nice and fast. But it is not as powerful as in notepad++. I didn't know I missed those features until I used them once. Also the UI for search/replace is sometimes cumbersome. I don't know how the case-insensitive, regex and whole words features works. Adding some context help and a list of examples would improve it a whole lot.
• The project tree is either too sparse or too cluttered. Giving more control of how to show the project and the files in the tree would be nice.
• The highlighted lines feature where errors, warnings, search hits and more are super useful, but it is not possible to click them to jump directly to the line in question, which in my opinion is a sorely needed feature. Also I think they could be symbols that are grouped when there are many in the same area. When you hover the group it will expand to how all the marks so you can click them individually. It is difficult to remember the different colors. I still don't know what white and green means.
• I wish the integrated examples were updated as some are showing their age. I understand it is a conscious choice to not spend time on this, but marketing apt. of The Qt Company / Digia should know how huge an impact this feature has on beginners (read: potential customers). Keeping it updated and fresh should be among top 10 points on marketing plan!

There, all done!

Now I can go back to programming.

The curse of complexity

Most software development projects start out small and cute. They might have well defined goals and plenty of time and resources.

Why is it then, that they always end up as horrible nightmares of missed deadlines, bugs and fury? Many a wise man has tried to uncover the reasons to this conundrum. While gallantly skipping the all too pervasive problem of incompetent (read: non-technical) management, the answer is simply the staggering amount of unexpected complexities that emerge en masse in the middle to late stages of development. We are unable as mere human beings to foresee how difficult it is to craft even the simplest of software.

As you mature as a developer you learn how to cope with this on many fronts. You learn to moderate your own and then later your peer's expectations to the progress of the project and the quality of code. You learn to spend more time architecting solutions before you start coding. You learn to incorporate  such fancy concepts as OOP, SoC, KISS, DRY and TDD. You learn to adapt to defensive programming conventions, and that healthily paranoid mindset and lack of trust in your own code that is the mark of the experienced hacker. And this all helps a lot.

But even after 25+ years of experience developing software I am still finding myself flabbergasted by some hidden complexities. And when it happens I usually feel my heart sink. It is a disgusting feeling of being let down by your own intellect, proof that you really should not feel so confident in your ability as you hoped (queue Japan - Ghosts). I have found that coping with this is the last big frontier for the experienced developer. Those who does not cope find greener pastures in management.

Even though I know this to be true of myself, I have never really cared to collect any evidence. I have just accepted whatever "temporary" hairy solution the team comes up with and moved on, trying to forget the technical debt we just incurred that will never be mitigated.

But this time, it happened to me in the middle of the night. More importantly, it happened while working on my spare time project that I really care about, and that I have set the highest of standards for. So I decided, that this time, I am not going to fold. I will present to you the raw unadulterated case in all its glory! Finally we will have some evidence. I give you first Exhibit A.

Exhibit A: A working program

We have a working program. The program consists of a P2P client where each client will either have the role of "remote" or "agent". To establish communications it is expected for both clients to contact a server with their details. The server will then report back with a list of all recent connections effectively aiding the clients "discover" each other (hole punching).

Once discovery is complete, both clients will have the network address and port of the other.

The client stores its own local network address and listen port as member variables mLocaladdress and mLocalPort respectively. The local address is determined from the local network configuration at client startup and cannot be changed. The port is set to a default value (8124 for agent and 8125 for remote) at first startup, and can be changed later in UI.

Further, the client will store the mLocalAddress + mLocalPort and mPublicAddress + mPublicPort for each of its communication partners. Whenever a client wants to reach it's partner, it will simply send network packets to the public address + public port of that partner.

So far so good. But now the plot thickens...

Exhibit B: The realizations

We realize after implementing this and testing to confirm that it works in a simple home network that the distinction between "local" and "public" does not matter too much in the "real world".

In a home network both clients will be on the same local network and it would be preferable to communicate via local addresses, but in other cases clients are on different networks, and in those cases, if one client starts off communicating while attached to a WiFi network then goes out of range and starts relying on LTE/4G instead, suddenly the "public" address of that client will change mid-transmission, throwing the whole connection off.

Further, we found that clients seldom have just one local address. In fact, the list of local addresses contains one entry for each of the physical and virtual network interfaces on the host, and even more depending on whatever crazy network configurations exists. Telling them apart and knowing which one of them is "the right one" is impossible.

So clearly our innocent simple model is not good enough to handle these scenarios. Who would have thought?

Exhibit C: Re-imagining things

So the correct thing to to here is re-imagine how addresses are stored for clients. Let's go through that exercise here. Now instead of having just local address + local port for our own address and local address + local port and public address + public port for each partner we could go for some kind of weighted list of addresses per partner, and some kind of list for our own addresses as well.

In the interest of KISS and DRY we could try to implement such a "list of address" class once and re-use it in those two cases. Think about that for a moment. Sounds like a good idea right?

Let's say we did that. We make a placeholder class "AddressList" and re-factor all the 37 places in the code that reference the old variables to instead call imaginary methods on some instance of this class, making sure to introduce 7 new bugs. Now the code won't work anymore, we broke it. Not to worry, we will fix it soon.

So we start implementing the list and realize that while local port needs to be stored once, no matter how many local addresses we have, each of the partner's addresses will need to be stored with different ports.

Snafu.

OK not to worry, we just throw DRY and KISS out the window and make a copy of the "AddressList" class, and  rename them to "LocalAddressList" and "PartnerAddressList". Then we refactor partner to store one port per address. Then we go over the 18 places in the code that should be using the other class and refactor that to use imaginary methods on some instance of that new class instead, again making sure to introduce 11 bugs.

OK time to implement those imaginary methods. But wait.

• How on earth are we going to know which address in each list is the "current"?
• When does the "current" item change?
• How will the "current" item be persisted between application runs?
• Should these addresses be persisted together with other data or in it's own separate configuration file?
• Or as a general application setting?
• Should persisting be asynchronous or blocking?
• How often should we persist?
• When should we persist?
• What if the user enters an invalid port?
• What if the default port is invalid?
• Which address and port should the server exchange for us?
• How long should old addresses be retained?
• .... 3 hours later
• GAAAA!

Summary

We went from a solution with 4 member variables to a solution with two more classes with non-trivial implementations and with their separate set of unit-tests, documentation, the works. Just ironing out the logic needed for all the new corner cases will take a considerable amount of effort.

And I can assure you that this new solution will contain at least one other "realization" similar to the one described in Exhibit B that will introduce even further complexity to the solution.

You could argue that this could have been avoided from the start by simply modelling the solution before writing the code. Or writing a prototype. Or just having a team so experience that they already "knew" this. All of which may be feasible for some projects funded by some organizations.

But the evil end-truth is that for my project and the resources it commands this would have been inevitable. This is what people like me must do to succeed. We go head first through conundrum after conundrum, ironing out the logic and watching keenly for the the next hidden realization lurking around the bend. We will soldier on one line of code at the time, and I guess it separates us from those who will buckle under and transition into management...