Why is it then, that they always end up as horrible nightmares of missed deadlines, bugs and fury? Many a wise man has tried to uncover the reasons to this conundrum. While gallantly skipping the all too pervasive problem of incompetent (read: non-technical) management, the answer is simply the staggering amount of unexpected complexities that emerge en masse in the middle to late stages of development. We are unable as mere human beings to foresee how difficult it is to craft even the simplest of software.
As you mature as a developer you learn how to cope with this on many fronts. You learn to moderate your own and then later your peer's expectations to the progress of the project and the quality of code. You learn to spend more time architecting solutions before you start coding. You learn to incorporate such fancy concepts as OOP, SoC, KISS, DRY and TDD. You learn to adapt to defensive programming conventions, and that healthily paranoid mindset and lack of trust in your own code that is the mark of the experienced hacker. And this all helps a lot.
But even after 25+ years of experience developing software I am still finding myself flabbergasted by some hidden complexities. And when it happens I usually feel my heart sink. It is a disgusting feeling of being let down by your own intellect, proof that you really should not feel so confident in your ability as you hoped. I have found that coping with this is the last big frontier for the experienced developer. Those who does not cope find greener pastures in management.
Even though I know this to be true of myself, I have never really cared to collect any evidence. I have just accepted whatever "temporary" hairy solution the team comes up with and moved on, trying to forget the technical debt we just incurred that will never be mitigated.
But this time, it happened to me in the middle of the night. More importantly, it happened while working on my spare time project that I really care about, and that I have set the highest of standards for. So I decided, that this time, I am not going to fold. I will present to you the raw unadulterated case in all its glory! Finally we will have some evidence. I give you first Exhibit A.
Exhibit A: A working program
We have a working program. The program consists of a P2P client where each client will either have the role of "remote" or "agent". To establish communications it is expected for both clients to contact a server with their details. The server will then report back with a list of all recent connections effectively aiding the clients "discover" each other (hole punching).
Once discovery is complete, both clients will have the network address and port of the other.
The client stores its own local network address and listen port as member variables mLocaladdress and mLocalPort respectively. The local address is determined from the local network configuration at client startup and cannot be changed. The port is set to a default value (8124 for agent and 8125 for remote) at first startup, and can be changed later in UI.
Further, the client will store the mLocalAddress + mLocalPort and mPublicAddress + mPublicPort for each of its communication partners. Whenever a client wants to reach it's partner, it will simply send network packets to the public address + public port of that partner.
So far so good. But now the plot thickens...
Exhibit B: The realizations
We realize after implementing this and testing to confirm that it works in a simple home network that the distinction between "local" and "public" does not matter too much in the "real world".
In a home network both clients will be on the same local network and it would be preferable to communicate via local addresses, but in other cases clients are on different networks, and in those cases, if one client starts off communicating while attached to a WiFi network then goes out of range and starts relying on LTE/4G instead, suddenly the "public" address of that client will change mid-transmission, throwing the whole connection off.
Further, we found that clients seldom have just one local address. In fact, the list of local addresses contains one entry for each of the physical and virtual network interfaces on the host, and even more depending on whatever crazy network configurations exists. Telling them apart and knowing which one of them is "the right one" is impossible.
So clearly our innocent simple model is not good enough to handle these scenarios. Who would have thought?
Exhibit C: Re-imagining things
So the correct thing to to here is re-imagine how addresses are stored for clients. Let's go through that exercise here. Now instead of having just local address + local port for our own address and local address + local port and public address + public port for each partner we could go for some kind of weighted list of addresses per partner, and some kind of list for our own addresses as well.
In the interest of KISS and DRY we could try to implement such a "list of address" class once and re-use it in those two cases. Think about that for a moment. Sounds like a good idea right?
Let's say we did that. We make a placeholder class "AddressList" and re-factor all the 37 places in the code that reference the old variables to instead call imaginary methods on some instance of this class, making sure to introduce 7 new bugs. Now the code won't work anymore, we broke it. Not to worry, we will fix it soon.
So we start implementing the list and realize that while local port needs to be stored once, no matter how many local addresses we have, each of the partner's addresses will need to be stored with different ports.
OK not to worry, we just throw DRY and KISS out the window and make a copy of the "AddressList" class, and rename them to "LocalAddressList" and "PartnerAddressList". Then we refactor partner to store one port per address. Then we go over the 18 places in the code that should be using the other class and refactor that to use imaginary methods on some instance of that new class instead, again making sure to introduce 11 bugs.
OK time to implement those imaginary methods. But wait.
- How on earth are we going to know which address in each list is the "current"?
- When does the "current" item change?
- How will the "current" item be persisted between application runs?
- Should these addresses be persisted together with other data or in it's own separate configuration file?
- Or as a general application setting?
- Should persisting be asynchronous or blocking?
- How often should we persist?
- When should we persist?
- What if the user enters an invalid port?
- What if the default port is invalid?
- Which address and port should the server exchange for us?
- How long should old addresses be retained?
- .... 3 hours later
We went from a solution with 4 member variables to a solution with two more classes with non-trivial implementations and with their separate set of unit-tests, documentation, the works. Just ironing out the logic needed for all the new corner cases will take a considerable amount of effort.
And I can assure you that this new solution will contain at least one other "realization" similar to the one described in Exhibit B that will introduce even further complexity to the solution.
You could argue that this could have been avoided from the start by simply modelling the solution before writing the code. Or writing a prototype. Or just having a team so experience that they already "knew" this. All of which may be feasible for some projects funded by some organizations.
But the evil end-truth is that for my project and the resources it commands this would have been inevitable. This is what people like me must do to succeed. We go head first through conundrum after conundrum, ironing out the logic and watching keenly for the the next hidden realization lurking around the bend. We will soldier on one line of code at the time, and I guess it separates us from those who will buckle under and transition into management...
It has come to the point where I decided to upgrade my printer with an octoprint.
Octoprint is a software that allows you to remote-control your 3D printer using a simple raspberry pi and a webcam. It was super easy to install and set up. I literally spent 20 min from opening the rapsberry pi packaging until the first octopi print was started:
It shows you all kinds of neat statistics like bed/nozzle temperature:
New in OctoMY™ HQ is a test-rig of sorts. On the wall hangs 3 separate touch-enabled PCs, each with an embedded variant of Debian and a script to auto-load latest build of remote, agent and hub respectively.
Of course each computer has the smiling mascots as wall papers to provide comfort and inspiration whenever the respective programs are not running (usually due to bugs).
This system will in addition to the build in TravisCI hopefully allow for much shorter debug cycles.
Solder is usually 60% Tin, 40% Lead, and this is referred to as 60-40.
When the solder joint cools, the different metals will solidify at different times, and there is a short time when one metal is solid while the other is liquid, called the "plastic phase". Unless the solder joint is kept absolutely still you risk creating artifacts in the joint such as voids and cracks that will decrease the quality of the joint considerably.
The solution is to switch up the numbers. You can find solder with the perfect balance of 63% Tin to 37% Lead which completely removes this problem.
So now you know what solder to get next time!
I decided to share the playlist containing 5 parts here.
A summary of the parts:
- Part 1: Battery layout, serial vs. parallel etc.
- Part 2: Welding batteries together using special purpose spot welder and nickel strips.
- Part 3: Choosing and installing a BMS (Battery Management System).
- Part 4: Sealing the battery with shrink wrap tube, kapton tape and foam.
- Part 5: Choosing an appropriate charger.
I will integrate this into the expression code soon, but until then, here are a few screenshots. I think it turned out pretty alright for 2 days work.
UPDATE: Here is a link to the full source code in C++, and here is the test code with the UI you see in the screenshots. Some details on the implementation follows:
- It uses 2 separate sources of 3D simplex noise, one for the streaks and one for radial perturbation of the streaks.
- It uses 20 random float parameters plus 2 colors to tweak the parameters of the simplexes (3 parameters each) plus tweak the scaling and offset and other "funky hard-coded equations" to to produce irises that kind of looks real, interesting and diverse without carefully selected boundaries.
- Rendering is done in a scale invariant way so that the eye can be scaled up "infinitely" wihtout changing its character.
- Rendering is done in floating point and produces results that go slightly into high dynamic range.
- It uses filmic2 tone mapping to convert HDR back to normal 8-bit ranges.
- The code should be fairly portable, both the Simplex implementation and the rendrer itself should only depend on Qt, and porting it away from Qt should be easy.
|Bed size||While many crave a huge print-bed, I have found that most things I print are small. Why? Because most 3D printers are much slower than you think. In fact most points on this list are about speed vs. quality. But if you have a fast printer that prints high quality, or if you really need to print big and have the required patience, then bed-size is still important.|
|Motor speed||How fast can the motors transport the extruder around the work?|
|Motor torque||How fast and steady can the motors accelerate?|
|Extruder max temperature||Will the extrude cope with the high-temperature filaments?|
|Extruder temperature stability||Will the extruder manage to hold temperature steadily during printing, even at higher temperatures?|
|Extruder temperature speed||How fast will the extruder warm up from room temperature at the start of the print?|
|Bed max temperature||Will the bed cope with the high-temperature filaments?|
|Bed temperature stability||Will the bed manage to hold temperature steadily during printing?|
|Bed temperature speed||How fast will the bed warm up from room temperature at the start of the print?|
|Enclsure temperature control||Some filaments like ABS may require the temperature around the work being managed using an enclosure and heater.|
|Direct vs. Bowden filament feed||The size of the print head directly affects the speed and accuracy of your printer. A big head will slow down and cause the printer to wobble if it is not rigid enough. To get around this many printer manufacturers look for ways to decrease the size of the head. One feature, the "bowden tube" allows the extruder motor to sit on the frame instead of the head. However this reduces the filament capacity somewhat which in turn affects print speed.|
|Interchangeable extruder head||You might only care about the standard 1.75 mm filament through 0.4 mm nozzle size. However there are lots of options out there when it comes to nozzle sizes. Having an easily interchangeable head/nozzle is desirable, and a printer that can handle the higher heat requirements of a larger nozzle/filament of course.|
|Multiple extruder head||Having more than one extruder provides some benefits, for example if you are printing with more than one material per part. Example usages are: flexible filament for rubber seals, washable support material, dual color prints etc.|
|Noise||Remember your printer will be running for a long time. Print times are usually measured in hours, and sometimes in days..|
|Build quality & basic features||This qill require a lot of "getting to know" the printer. Basically all sorts of small practical details such as positioning of the user interface, display size, where is filament dispensed from, how are the wires connected? etc. etc. You can often get an impression of these things by looking at printer reviews on YouTube.|
|iMacwear W1 in protective 3D printed case with servo mount.|
The idea was to make a case for it that would protect it while also providing better mounting options. The current version of the case sports a native RC servo horn mount, but I am also looking at different mounting options including a gimbal mount and a simple "screw me on" mount.
I submitted the whole thing on thingiverse here if you want to check out the details. It includes source CAD file, lots of pictures and details etc.
OK, so it is official. We are now on the 3D printing bandwagon.
The proverbial We got a Creality CR10 3D printer shipped to the shop straight from PRC, and it has had lots of use the last weeks. As a complete beginner I had a lot to learn, however yesterday I put out my first ever DIY design on Thingiverse.
The printer is awesome. It feels like I suddenly got a new super power; the power to create anything. But there is one big hurdle, and that is time. It takes forever to print even the tiniest of things!
So I have experimented a bit with settings and found some optimizations that I would like to share.
So what will affect the print speed?
- Infill percentage
- Layer height
- Print speed
- Travel speed
- Retraction speed
- Retraction distance
- Raft margin
- Raft layer count
I found the standard/default/recommended settings to be way conservative. Here are my new settings that cuts the total print speed in almost half:
|Infill percentage||10%||100%||How is this even possible? You do it by making sure your model contains the necessary structural elements by hollowing it out where you need less structural strength and filling in when you need more. This has a bonus benefit that you can shape your custom "infills" to improve the weaknesses in the structure.|
|Layer height||0.8 mm||2 mm||Will affect appearance, the model will look more coarse.|
|Print speed||60 mm/s||100 mm/s||If you go too high, the motors might start skipping steps or the extruder might not be able to put down enough PLA.|
|Travel speed||100 mm/s||300 mm/s||If you go too high, the motors might start skipping steps.|
|Nozzle temp||200 ℃||205 ℃||This provides better layer adhesion, and might be required if you increase print speed.|
|Bed temp||50 ℃||70 ℃||This provides better bed sticktion on clean glass with raft, and might be required if you increase print speed.|
|Retraction speed||25 mm/s||60 mm/s||If you go too high, the extruder motor might start skipping steps or retraction might not work as expected.|
|Retraction distance||6.5 mm||1 mm||If you go too low, you might get artifacts such as stringing.|
|Raft margin||15 mm||5 mm||There are many ways to do adhesion. I like best a clean glass plate with a small raft, and raft margin gets really important for small parts.|
|Raft layer count||2||1||I found 1 layer is more than enough for the raft.|
|OctoMY™ SLAM (Simultaneous Localization And Mapping)|
The data could be as simple as the readings of an ultrasonic range finder or as complex as the video streams from a stereoscopic pair of cameras. And the algorithms an heuristics that are used to derive meaningful localization and mapping data is loosely fit under the term SLAM (Simultaneous Localization And Mapping).
|Stereo Camera Sensor|
- Available processing power ("must run in real-time on smartphone").
- Indoor only or outdoor as well.
- 2D or 3D mapping
- Sparsity/size of map data
- Accuracy of localization
- Loop detection (detection and resolution of circular pathways in environment).
- Scalability with size of environment
- Must be GPU/FPGA/DSP accelerable.
Since we are basically building a platform on top of which we hope you will add all the cool plugins and features, there really arn't any limitations to what kind of SLAM methods can become part of OctoMY™. But since SLAM is such an essential concept, we will have to provide a pretty darn good default implementation from the get-go that you can get started with right away.
Nothing is set in stone yet, but here are some of the thoughts about SLAM in OctoMY™:
- Should be unsupervised as far as possible
- Automatic camera lens/other sensor calibration
- Automatic environment scale detection
- Automatic tweaking of parameters to get the best result in general
- Should generate detailed and accurate 3D map of high accuracy.
- Should support huge maps
- Should support distributed data model that allows sharing of data between agents all over the world.
- Should support input from whatever sensors are available, and make the most of it.
- Should adapt to the processing hardware available:
- Make use of hardware acceleration through OpenCL when available.
- Use less resources on lower-end platforms such as old smart phones.
- Should use less hand-crafted and more "learned" methods.
I first heard about him when I watched his ted talk, and from there my curiosity took over, and I started looking more into the research he is conducting with his company numenta. I will try to summarize the basics of his theories in this post.
If you want to learn more, a good place to start is to read his book "On intelligence".
So what is HTM? HTM is an acronym for Hierarchical Temporal Memory and basically it is a list of 3 important aspects of the algorithm in the theory. The most prominent contributor to intelligence in our brain is the "neocortex".
In contrast to the rest of the brain which has evolved longer and therefore is much more specialized, the relatively new neocortex has a homogenous structure that is re-used throughout.
Small patches of the cortex represent stages in a hierarchy, and the connections between the neurons is what dictate the boundaries of each patch. Signals pass from lower stages up to higher stages and back down.
The lowest stages are connected to our senses such as eyes, ears and skin via mechanisms of the old brain. The signals passed from here consists of patterns of impulses that make their way up the stages, and it is temporal sequences of these patterns that are learned in each stage.
And perhaps the most important part of the theory is the following; Each stage will try to remember signals coming from lower stages and later predict those signals once they have been remembered. Only the signals that have not been predicted or "learned" will pass to the next stage unchanged, so by every stage the impulses have been refined and condensed into more abstract and high-level information.
When a stage is seeing a new signal that it cannot predict, it will be sent up to the next stage until one stage recognizes it. If no stage recognize the input, it is forwarded to the hippo-campus which sits at the logical top of the hierarchy. The hippo-campus will keep the unknown information around for a some time until it is no longer useful. Hopefully, the stages below will now have managed to learn it, and if they have not, it will simply be discarded.
Beyond this introductory description of HTM, there are many important details that really describe well how this relatively simple algorithm actually explains our intelligence and sense of being completely.
I can warmly recommend to read the book "On intelligence".
So why did we bother?
The new build architecture gives us the following benefits:
- We can now select to build each part as a separate library or simply include the sources. Both has its pros and cons, and now we get to choose using a very simple syntax.
- We no longer have to specify build parameters for each library, they are inferred automatically and you only supply them if you need to override the defaults.
- All the include paths have been cleaned up so now all #include "../libs/libX/subfolder/SomeClass.hpp" are converted to #include "subfolder/SomeClass.hpp"
The reason we decided to start this was that the existing project structure was becoming increasingly difficult to work with. Now with the refactor we plan to improve on the following:
- Shorter build times
- More modular design
- Fewer inter-dependencies
- More organized and predictable structure
- Smaller modules
It took a few days of googling around to get the perfect combination of yaml settings and the result was completely worth it.
We now have a shiny green "passing" button:
The concept is simple enough; create a .travis.yml file with some markup to describe the building of your project, add said file to the root of your git repo, push.
In the other end, travis hooks into your repo (github) and detects the push and starts building your project according to the definition in your .yaml
And it works great for a lot of people, but after spending more than 2 hours with Travis you will know that it has it's limitations, especially if your language or platform is not of the hipster variety.
Since Travis is free for open-source projects it has become popular. This is a really sound business strategy, by capturing the hearts of passionate programmers, you will get sales when the weekend is over and they go back to work.
However, many users means lots of focus on operations, plenty of long wishlists and less time/opportunity for innovation/implementation.
And it shows. My experience with having Travis build OctoMY has been a mixed bag. It turns out that Linux support in Travis is "Beta" and is centered around Ubuntu 14.04 which as the name suggest is 3 years old (at the time of writing). If that was not bad enough, Ubuntu has very conservative collection of C++ compilers in the official repository which means that the only C++ compiler available in Travis does not support C++14.
Further, Ubuntu, even after signing a deal with digia, has even more ancient versions of Qt in the repos. And you can forget support for multiple versions of Qt side by side.
OK, enough whining and on to what you came here for!
I managed to muster my inner zen-patience-plus-determination combo and I ended up confirming that you too can have your favorite version of Qt and your favourite version of clang++ or g++ building your project happily on Travis CI in Ubuntu Travis 14.04LTS.
At first this sounds wonderful, but please note that the resulting binaries are pretty tied to that platform, and unless you have a server side application (rare for Qt) running on Ubuntu 14.04 this will not really be useful other than to verify that your code builds and to run the tests.
So with no further ado, here is the link to the fully working travis yaml for building Qt5/C++14 applications.
|OpenSCAD Official Logo 2017|
While the traditional CAD approach is all point-and-click in a laborious user interface, OpenSCAD takes the approach of the programmer, namely writing a definition text and viewing the reselt in an interactive 3D view. Since the language used in OpenSCAD allows the user to make relationships between parameters in the model, the result are highly flexible and customizable objects, perfect for rapid prototyping and research work. I can warmly recommend this software, I already have a substantial model build from the ground up after just 3 hours of use. It definitely made my grudge against CAD software go away!
|Schmoo||Any liquid, be it hydrophillic or hydrophobic, homogeneous or pasteurized, that has a viscosity between that of Canuckistanian Beaver Semen at one hour after ejaculation ( 9.35 +/- 0.99 centipoise ) and Canuckistanian Maple Syrup (175 +/- 25 centipoises).|
See also "Pixies".
|Pixies||Small, usually obedient particles that are expected to form lines inside wires to make contrivances chooch. Se also: "Chooch", "Angry Pixies".|
|Angry Pixies||Pixies that for some reason are no longer obedient, and may for no reason at all step out of line. Se also: "Pixies".|
|Bumblefuckery||An activity that may result in a contrivance stopping to chooch. See also "Pop fart and give up the ghost".|
|(The room formerly known as) wife's sowing||A traditional in-home style Canuckistanian man-cave. Dedicated to learning from books and staying in front of the screen with the occational work with brain boxes. Tempered at around 23 Dungrees.|
|(The) shop||A traditional out-of-home style Canuckistanian man-cave. Dedicated to performing bumblefuckery with the bridgeport. Tempered at around 15 Dungrees.|
|(The) bridgeport||An age old contrivance for manually and forcefully opening snail-mail, cracking plexiglass, bending and annealing end-mills and making vodka glasses from sheets of copper.|
|(The) screen||Special purpose contrivance that allows Canuckistanians to learn stuff from the interwebs and youtubes.|
|Keep your dick in a vise||Have a nice day. See also "Keep your stick on the ice".|
|Brainbox||A contrivance dedicated to the controlled conversion of pixies into angry pixies, thereby making one or more other contrivances chooch.|
|Frankenstein||Unit of measure for temperature.
|Dungrees||Unit of measure for temperature.
|Vijeo||Sequence of images displayed in rapid succession with accompanying audio.|
|Wank overload||An event where too much wank happened.|
|Jelly 4 ur Jam||Bang for your buck|
|Canuckistan||The land of the polar beers.|
|(Canuckistanian) Copeks||Primary unit of wealth in Canuckistan. See also "(Canuckistanian) Pesos".|
|(Canuckistanian) Pesos||Secondary unit of wealth in Canuckistan. See also "(Canuckistanian) Copeks".|
|Scienticious||Actions carried out exclusively by learned Canuckistanian. See also "Scientician".|
|Yada yada yada||I don't feel like elaborating now.|
|It's the cicle of life||At this very moment, I am having an epiphany.|
|Dog bless the nanny state||Idiomatic exclamation of pure satisfaction equivalent to "God bless the government of Canuckistan for looking after its citizens"!|
|Pop fart and give up the ghost||To stop chooching.|
|Flogg it like a Rented mule||To start chooching.|
|Son of a diddely||Mild expression of disappointment.|
|Safety squint engaged||Eyes squinting momentyarily to avoid damage. See also "Bifocals".|
|Fantabulous||Mild expression of approval.|
|Scokum||Of right Canuckistanian quality.|
|Keep your stick on the ice||Good day. See also "Keep your dick in a vise".|
|Rippums||Rotations Per Minute.|
|Meat hook abortion||Contrivance of unknown or non-Canuckistanian origin. See also "Chinesium".|
|Chees grade||Of considerable softness.|
|Bob's your auntie||Let's try this.|
|The sharpest bowling ball in the shed||Round Contrivance typically kept by land owners of Canuckistan in out-door sheds. Used for measuring relative mental capacity.|
One approach I have always liked is the concept of Fuzzing. Basically fuzzing means to feed random or cleverly provocative data into your innocent and carefree API to wreck complete havoc and expose all its bad traits/bugs/unexpected features etc. But I am reluctant to bring new dependencies into my project if I can avoid it. So how could we implement a built-in fuzzing in the least amount of code?
I came up with the approach below which I find myself being really happy with! Here the qrand() function could be exchanged with a more devious/evolutionary and data driven fuzz data generator function.
|Power bar created.|
- Main power bar was created and connected to battery.
- Relay board high end connected to power bar.
- Ribbon cable to carry signal between controller and relay boards was prepared with apropriate connectors.
- RC Servo pins soldered to stand alone PCB that allows for combining power with signal from different sources.
- RC Servo, ESC and power connected to RC servo board.
- ESC connected to power bar.
- Audio amplifier connected to power bar.
- Buzzer and warning light was connected to the power bar and relay board.
- Most circuits tested.
|Test circuit for RC servos.|
I looked long and hard in google image results before realizing there really arn't any icons that represent this in a good way. So I decided to make my own.
Please accept my humble contribution!
|OctoMY™ SVG Vector format Icons representing C/C++/Java fundamental datatypes|