An experienced software engineer with 20+ years experience writing products for Microsoft Windows based operating systems as well as 12+ years experience hosting websites on Linux and Windows including e-commerce and CMS systems.
It has taken quite a while, but version 0.3 of Constraint Capers Workbench is coming together pretty well.
I decided that for this release I needed to start solving actual constraint problems. Well, one specific problem to be more precise, the classic problem, n-queens.
Constraint repeaters provide a mechanism to expand a single constraint into many statements.
The n-queens model has the following constraint: cols[i] <> cols[j] | i,j in 8,i
The part on the right hand side is the repeater. The repeater expands the constraint into (potentially) many simpler constraints. In the above example, the following simpler constraints would be generated:
One area that needs improvement is the difficulty of debugging the repeater.
The model will often not be in a form that is very user friendly. The visualizer is designed to display the solution to the model in a manner that is likely to be understandable by the user.
The chessboard isn’t very flexible, it can only display an 8×8 board at the moment because that is all I needed for the current version.
Visualizer Binding Expression
One of the big problems with designing this software is finding a mechanism to convert the model into something that can be easily understood.
For instance, the mechanism I used to solve the n-queens puzzle is to have an aggregate variable with eight variables. The domain for the variable is the values from one through to eight. Each of the variables represent an individual column on the chess board. The domain represent the queens row in that column.
I have implemented a small language that is executed after the model is solved and is able to take the solution and convert it into something that makes sense to the visualizer.
Here’s an example from the n-queens solution:
for x,y in 1..8,1..8: if <cols,x> = y: board(x:x,y:y,side:white,piece:Queen)
The above expression loops through each position on an 8×8 chess board and checks to see if it should place a queen in that square. The <cols,x> expression gets the xth variable from the cols value ie the values bound to the cols variable. The board part is the name of the chessboard visualizer. It assigns a white queen to the x, y location.
The next release will also concentrate on a single constraint problem again, the map colouring puzzle.
Whilst the map colouring problem could be modeled using the current version of the software, the map itself could not be displayed satisfactorily. The visualizers are absolutely crucial if the workbench is to meet its design goals.
I’ve not managed to read as many books this year. Mostly due to the competition from playing Dragon Age: Inquisition for the first four months of the year. On the plus side, it has been a good year for books. I’ve had a run of superb books. I’ve tried to broaden the range of books I read. I’ve always been a sci-fi fan but not a great fan of ork and goblin style fantasy but I’ve made a conscious effort to try more contemporary fantasy and have been very pleased with the quality of work I’ve found.
I’ve read a total of 12 books in 2015, 10 fiction (mostly science fiction and fantasy) and just 2 non-fiction. Note to self, must read more non fiction in 2016. Neither of the two non-fiction books were particularly inspiring, but at least The Silent Day: A Landmark Oral History of D-Day on the Home Front gave me some insight into life in England during the preparations for D-Day.
My highlight of the year has been Ready Player One by Ernest Cline. The book is currently being made into a film directed by Stephen Spielberg. Can’t wait 🙂
Gary Johnson: Brightly Shone The Dawn: Some Experiences Of The Invasion Of Normandy (Non-Fiction)
Robert Llewellyn: News from the Clouds (Fiction)
Claire North: The First Fifteen Lives of Harry August (Fiction)
Stephen Baxter: Proxima (Fiction)
Max Arthur: The Silent Day: A Landmark Oral History of D-Day on the Home Front (Non-Fiction)
Ernest Cline: Ready Player One (Fiction)
James Blish: A Case of Conscience (Fiction)
Silvia Moreno-Garcia: Signal to Noise (Fiction)
David Nobbs: The Fall and Rise of Reginald Perrin (Fiction)
The project has been renamed. It was called the Dyna Project, it is now called the Constraint Capers project due to a name clash with another project on GitHub. The other project didn’t object or anything, it just seemed like the right thing to do.
I’ve now been working on Constraint Capers for nearly 8 months. Progress is as slow as you’d expect for a project I only work on for five hours per week or thereabouts.
Version 0.2 consisted of 10 work items the main items of note were removing the necessity to link things together when building the model and adding aggregate variables i.e variable consisting of zero or more variables.
The next version is where things start to get a bit more interesting. The work items for version 0.3 begin the process of taking the solution and making it more understandable by the user.
One of the more interesting challenges for the Workbench is the user interface. The interface isn’t like a spreadsheet or a word processor where you don’t need to worry what mode you are in. You have a document and can edit it at any time. In the Workbench, you can edit the model, or edit the solution or display the solution. To me that kind of interface is much more akin to Visual Studio or to PowerPoint. In PowerPoint you can edit the presentation or go into presentation mode and present your work. You can’t edit the presentation whilst in presentation mode.
The problem I see is that in the Workbench there are effectively two things that can be edited, the model that defines the problem and the solution that defines how the solution will be displayed to the user. So, from that perspective, it is more complex than PowerPoint. When you edit and then present there is just one thing to be edited.
Perhaps one workaround is to meld the model and solution into a single entity. Maybe eventually there won’t be a big difference between model and solution. At the moment the solution is created when the user invokes the solve function. Perhaps the computer may automatically and continually solve the model without intervention from the user.
Fortunately for the 0.3 release I don’t have to resolve the user interface problem. I am quite happy to kick that can down the road for the 0.4 release.
I’m back writing GUI code at the moment. I don’t think it would be an exaggeration to say that I hate writing GUIs. But, there’s nobody else to do it, so it has to be me.
It’s not just that I am doing GUI code though that makes my current project tough, it is that I am learning WPF at the same time. The last time I had to write a reasonable sized GUI I was using MFC and C++, circa 2002.
Why do I hate GUI programming?
It was probably incubated many years ago in the early 1990s when GUI programming was really quite painful. And fiddly, god it was fiddly. Make a change in the resource editor or C++ file, jump over to a dos box, recompile (could take anything up to 20-30 minutes), start the program, find that your change wasn’t quite right, rinse and repeat for days or weeks on end. Things haven’t changed that much, GUI programming is and always will be fiddly, but the tools have improved.
Tools will set you free
Fortunately, there are a number of tools that can help reduce feedback time.
Kaxaml – a WYSIWYG tool for XAML. Has a number of limitations, which are pretty easy to get around. A very useful tool for building XAML based user interfaces outside of Visual Studio and the change, compile, debug cycle. Enter your XAML and see it rendered in real time. Changes can then be made very quickly. When you’re happy, copy the code over to your Visual Studio project; Free
XAML Spy – a XAML version of the old Windows Spy program. Available as two version: standalone application and Visual Studio add-on; Paid app + Free
Microsoft Blend – would be the perfect tool if it wasn’t restricted to creating Windows 8.1 / Windows 10 app store applications. Not much use in my case, but if you are creating app store applications then you are in luck; Free, bundled with Visual Studio Community
LINQPad – won’t help you with XAML, but will help if you prefer writing your WPF code in C#. Type your C# code into LINQPad and execute it straight away. Even shows you the Roslyn parse tree; Paid app
Snoop – similar tool to XAML Spy, except that this is completely free and open source; Free
Visual Studio 2015 – one of the nicer new features of the new Visual Studio covers similar ground to XAML Spy, but all nicely integrated into the IDE itself. Nice. Free
A lot of programming is by its very nature an iterative activity, GUI programming is just an extreme form. The shorter you can make the change, compile, debug cycle the better. The above tools will help you reduce the cycle, but they don’t eliminate it. GUI programming has always been, is and will always be profoundly fiddly.
It is very easy to become jaded about technology, I had a reminder this week of just how wonderful it can be.
Last Thursday I started a beginners astronomy class at the local secondary school. I was more than a little surprised to learn that you can subscribe for a mere £3 per month and have access to three telescopes situated on top of a mountain in Tenerife through the Bradford Robotic Telescope.
You submit your astronomical target(s) via the telescope website and the telescope, without human intervention, schedules your job and, when it can, takes an observation and sends you the result(s) via email.
I know very little about astronomy and even I was able to schedule a job. It really is not difficult. There are no complex instructions to give, simply use one of the well known astronomical catalogs like the Messier object catalog, tell it how long an exposure you want and you’re pretty well done.
I chose M80 for my first target for no particular reason. My second target is Saturn. When I get the results I will be sure to share the results here.
Whilst version 0.1 of the Dyna Project isn’t quite finished, I thought it would make sense to take stock before work starts on version 0.2.
But first some introductions would probably be helpful.
What is the Dyna Project?
For a lot of years I’ve been interested in constraint satisfaction problems and how to solve them.
The Dyna Project is my latest attempt to create a tool for solving constraint type problems in an accessible way. I want constraint type problems to be as solvable as financial modelling is with a spreadsheet.
A retrospective on version 0.1
I’ve been working part time on the Dyna Project since 7th April this year. The project has received 122 commits so far, with approximately 5K lines of C# code. My plan was to build the simplest graphical modeller of constraint type problems possible. I think I have achieved that, it could not be much simpler.
Whilst the solution is very simple, there is the kernel of the design already. There is a place to model your problem, a mechanism for solving it and another mechanism to display the solution. The only element missing is the ability to design how the solution will be displayed.
I can’t say I like anything about the model building interface or design. It does work, but it is very painful to use. The idea of using different shapes for the variables, domains and constraints doesn’t work at all. The connectors add a lot of ceremony but don’t add much in the way of clarity. Most work in the 0.2 release will need to be used resolving the model building interface. Without a workable model interface, there’s not much point to the rest of the program.
The project name will be changing before the 0.2 release. The name clashes with another project on GitHub so I think it only polite to find another one. No idea what the new name will be yet. I am open to suggestions. 🙂
Please don’t use the project yet, it is some distance away from being usable. I know it’s buggy, and have no intention of fixing the bugs because much of the code is going to change for version 0.2.
The design of version 0.1 was only ever intended as a place marker. In that and that alone I think it has succeeded. The outline of the program is in there, all of those elements need considerable improvement in the next few releases.
I created a product a few years ago and whilst it is doing fine on new sales it is really bad at monetising the existing customer base. The reason it is doing so badly at monetising our existing customers is because I assumed that the business model could be plugged in later, like any other software feature.
I was 100% wrong.
Why didn’t I build the business model in from the start? Patience. Or rather my lack of patience.
The software took quite a while to write and I was very keen to get it out of the door as quickly as possible. I got to the stage that I was sick of the sight of the software and just wanted it finished. There is nothing wrong with wanting your project finished. But your project cannot be done if the business model isn’t baked in.
When I originally created the product, I tried to create the simplest product possible that nevertheless delivered value to the customer.
I put two things off from the first version of the software. One was the ability to notify customers when a new version of the software is available. The second was the ability to renew the software subscription after the free period had elapsed.
I thought that I could plug-in the business model at a later stage. Just like a new feature. Turns out I was wrong. Whilst I could retrofit now, an awful lot of the value has been lost. Perhaps most of the value. The vast majority of existing customers will never know about the new software and so will never upgrade.
One of the nice things about developing for the various app stores is that you don’t need to build the business model. Somebody else has done that for you.
Last week we had to fend off a scam attempt. The scam worked like this:
Order a product from a company and arrange to pay by bank transfer;
Manually pay a counterfeit cheque into the company account. The cheque has no chance of clearing. Rather cleverly, the amount of the payment was 10x higher than the amount due. So it looks like a simple transcription error;
Ask for a refund for the difference to be sent by bank transfer to another account;
Disappear without trace.
The only thing that saved us from falling for the scam was the fact that we know that cheques show up in our account before the money is actually cleared. So, the money is sort of in your account but you can’t spend it.
Be careful out there folks, there are some odd sorts out there.
Update 2015/07/13: The scammer sent us an email over the weekend asking where his money is 🙂
I have an awful lot of failed software projects. Most programmers do. It comes with the territory.
Most of the failures have been the result of running out of steam one way or another. Your early enthusiasm slowly wanes until the mere thought of carrying on makes you feel a little sick.
It is easy to forget that programming is an intensely psychological activity. Your attitude is central to the success or otherwise of your project.
In my experience, I fail most often when I don’t maintain feature discipline. At the beginning of a project, you define the requirements for the project. The minimal set of features that are useful. You then start implementing those requirements. If those requirements stay basically the same whilst the project is being developed, you’ve greatly improved your chances of delivering the project successfully. If however during the development stage new requirements are being added to the project, then your chances of success greatly diminish.
Requirements are likely to change, in the beginning of a project you’ll be coming up with loads of new ideas. How can you control this process so it doesn’t knock the project off course?
You may think that requirements are only for large projects. You’d be wrong. Requirements are important whatever the size of project.
One of the things that can kill a project is the feeling that it is never going to end. You’re working away hour after hour, day after day and don’t feel like you’re getting any closer to the finish line.
That is why requirements are so important. If you nail the requirement down early on, ideally before you’ve written any code, you will have a much better understanding of what you need to deliver.
You will also avoid the feeling of being lost.
If you have any ideas for enhancements during the project, document the requirement and schedule it into a future version of the project. Then return to implementing the first set of requirements.
Your requirements need to be quite specific. Vague requirements like “Implement X” is really of no use. Be very specific. Outline precisely what is to be implemented, what data is required with examples. The examples can then form the basis for your tests. The requirements should specify the very minimum you need.
Your first version does not have to be released, you can iterate over a number of releases before you release your project.
Tools Can Help
There are loads of web apps out there to help you organise a software project. Many implement a variant of the agile board.
If you have a team of people, then using some kind of agile board is a great idea. For one person teams I think they add a little too much ceremony. If you find yourself spending more time fiddling with your agile board than actually working, then you’re over complicating matters.
I’ve found the very minimal GitHub Issue tracker to be ideal. You can’t spend a long time configuring it because there’s very little to configure. You can create issues, label them and add your requirements in the issue text. You can organise your requirements into milestones. When you’ve completed all of the issues attached to a milestone, you’re done. Move on to the next milestone.
Make issues that take a small amount of time to complete. A single issue that takes many months to implement will prove hard to gauge progress. But, taking a large task and breaking it down into lots of smaller issues begins to give you some momentum. You’ll feel like you’re achieving something when you are continually marking issues as closed. If you feel like you’re making progress, then you’ll have momentum and are more likely to finish.
Related to requirements. You need to be sure that what you are building can be built in a time scale you are comfortable with. You are one programmer, the project you take on needs to be doable in a reasonable amount of time. I’d say that one year is the very outside of what I could cope with as a lone project. If I can’t program the project at the very outside in one year, then I don’t take it on.
In addition, if the project requires resources you don’t have, then either you need to have a plan to get hold of those resources or don’t start the project. A vague “I’ll sort that out sometime near the end of the project” will likely just mean the project is never finished. Resources like graphic designers or UX people don’t just pop out of thin air. Wishful thinking won’t help.
The temptation to create something perfect runs pretty deep in a lot of programmers. Perfectionism is very destructive because you can spend an inordinate amount of time ceaselessly tweaking something that, in the end, really isn’t very important. Worse, on lone projects, there is nobody to stop you disappearing down the perfectionism rabbit hole.
Every time you change something that already works you need to be questioning whether anybody will care about the change. Does the user really care about your obsessively edited product icon? The answer is almost always no. That is not to say that you need to put up with mediocrity. But, there is a big difference between excellence and perfection. Not understanding the difference between those two things causes an awful lot of projects to fail.
There aren’t many projects that you know how to implement every feature at the very beginning. At some point during the project you are going to be coding something that you don’t know how to implement.
Everybody gets stuck at some point. There are extra difficulties from getting stuck as a lone developer, not least that you don’t have any colleagues to bounce ideas off or to provide technical help.
How can you ensure that getting stuck does not kill your project?
You can ask questions on Q&A sites like Stack Overflow, but some questions don’t lend themselves to Q&A sites.
Let’s say you’ve exhausted the usual Q&A sites and you’re still stuck. What do you do now?
The simplest strategy is to isolate the problem. You need to focus solely on your problem, not worry about how you can integrate it into your project. You need to be able to try new ideas as quickly as possible without going to the trouble of fitting it into your existing code.
Most programmers end up with a disk full of small projects used to solve technical problems. A useful side effect is that they make asking good questions on Q&A sites a lot easier too.
If you have doubts about a particular technical area at the beginning of a project, it can help to tackle that right at the start. Then if you do get stuck you will at least be brimming full of enthusiam and therefore be less likely to give up.
No Version Control
Version control systems give you a lot of benefits whether working on your own or not. The main benefit they give is the ability to experiment, and more importantly, go back to a state before the experiment started.
I remember a lot of projects when I first started programming, when source control was not commonly used by PC programmers, a lot of projects were destroyed by experiments gone bad. I’d have an idea for a change and then hours or days into the change I’d want to abandon the changes but couldn’t. Either I somehow got the project back to a position I wanted it to be, or I’d abandon it. There is nothing worse than getting into a change and knowing you can’t go back.
There are no excuses for not using source control now. There are loads of providers, most with free plans, and the tooling is great too. You don’t even have to learn the command line anymore, Visual Studio and lots of other IDEs have source control tools baked right in.
It is easy to forget how much psychology comes into play whilst programming on your own. Momentum or the feeling that you are making progress is absolutely key. If you lose momentum, you are much more likely to give up.
Developing software is an intensely psychological activity. Developing software on your own doubly so.
Update 1015/07/13: Adds the “No Version Control” section.
…customers didn’t start the conversation with their own fault diagnosis.
Quite often the hardest part of the conversation is trying to coax the customer back a step to the original problem. And then working forward from the problem to a diagnosis.
Providing good remote tech support is one of the hardest jobs in the tech industry. You are at the end of a phone or email, cannot see the users set up and only have the highly edited highlights of what the customer is willing to tell you.
A lot of people are not very good at diagnosing problems. One of the challenges with technical support is that the customer has already, in their own head, diagnosed the problem. Sometimes the customer misdiagnoses the problem. So the interaction with tech support is then coloured by that misdiagnosis even to the point selectively missing out crucial information that may not fit the customer’s narrative.
A recent tech support call went something like this:
Customer: Hello support, I need you to send me a new X mine has stopped working.
Tech support: Oh, I’m sorry to hear that. Can I ask have you made any changes to X?
Customer: No, it just stopped working.
Turns out that the customer had made a change. The customer may not have been overtly dishonest. There may have been some time between making the change and discovering the fault.
We don’t have a problem with replacing faulty equipment. Quite the contrary. But, there is no point in going to the considerable expense of replacing equipment if that wasn’t the root cause of the fault. All that happens then is that customer now has two pieces of equipment that don’t work and are still no closer to knowing what the original problem was.
Whilst the customer won’t appreciate at the time being dragged back through a thorough diagnosis, they probably will appreciate in the end getting to a solution a little bit faster.