Tuesday, 16 August 2011

Programming Hardware vs Programming Software

I think after all these years of being a hardware engineer looking at the world of software engineering with a mixture of scorn, pity, bafflement and incomprehension I've put my finger on a fundamental truth.
First some background:
Hardware engineering has no problem with parallelism and concurrency. In fact all the languages used rely on this so that it can be synthesised into hardware with relative ease. However The development pace is regarded as relatively slow.
Software engineering is (mostly) built around the model of a von Neumann model of a processor randomly accessing memory. For better or for worse most languages are one way or another linked to C's assumptions and models. Now maybe it would be more correct to describe it as Turing architecture, but let's move on. These systems and ways of thought have huge problems with parallelism, but exhibit relatively rapid code development. The model of one thing must happen before another is one that is easy to deal with and therefore debug.
What strikes me is this falls down in complex systems where you actually are quite happy with things happening at the same time, but I haven't been able to put this into words. It became clear to me today trying to reverse engineer some C code that I didn't understand that I am going about this the wrong way.
I like a system to have clearly defined boundaries. A chip has input and output busses. You cannot from outside a chip package get inside and modify the internal registers without going through the interface, it just physically isn't possible. Well defined interfaces are not only desirable, they are all that is physically possible.
Now if you talk to a software engineer they will say they like well defined interfaces too. They like structured code and hate things that cause spaghetti. I realised today that if that were true then the world of software would be very different.
At the most simple level it is considered a good thing that I can in some code initialise a bunch of data structures and then call a do_frame() function that accesses unknown elements of a variety of objects and produces a result. Worse than that there are many situations where hidden away in hundreds of lines of code is a little reference to an object that pulls in the thing that does all the work. Throw in globals, pointers and complex objects and the aim seems to be not only to abstract out the complexity (which is good) but abstract out the functionality (which is bad). This is I think the important point that in most hardware I have seen because there is a data flow and interfaces are isolated even if you have abstracted the complexity you have to show the functionality. A hardware block cannot sneakily change the contents of your SDRAM unless you explicitly connect it up.

Now don't think that I'm trying to say hardware methodology is superior, also don't try and say that this is the difference between good code and bad code. Perhaps it is good language vs bad language, but whenever I complain about C's memory model and the dangers of how it handles structures and pointers I am told that that sort of functionality is possible in any useful language. Granted verilog will allow you to do cross hierarchy references of the type that will allow you to break hierarchy, but the syntax to do that is so obvious and rarely used that it stands out like a sore thumb when it is used and therefore isn't used except in very rare circumstances. Languages like C seem to hide this sort of trickery and therefore programmers embrace it, or at least excuse it's existance.


So I guess that has been my epiphany, that software is written to hide complexity in all its forms. Hardware has its complexity constrained by the physical need to have defined interfaces and so has complexity hidden by use of hierarchy.
As a side note I've long thought that each individual engineer will make things just as complicated as he is able to understand it. Even when they try and make things simpler they'll often do that to reflect what they think is complex potentially introducing what another person would see as extra complexity to achieve this. Further given those two forces a system will converge to be slightly more complex than the team of people working on it can comprehend. I take great pride in trying to make things simple, but realise that often I must spend so much time trying to break things down that I can't see the wood for the trees. Sometimes you read others code and it is complex, but succinct.
Well I guess if this was easy then anyone could do it.

Tuesday, 1 March 2011

Novel thoughts

So I've been working on my novel again.
I think it's reached a point whereby I can keep fiddling with it, but it either needs a complete re-write, or I just need to publish it.
I think I'll do something I've been thinking of for a while and setup a new blog to publish it but do it one section at a time. Then what should I call the blog?

You see the problem is that I have written it for myself as something I think I'd like to read. Which means it's quite geeky, full of exposition and suffers poor character development. But then whenever I read it to try and improve it, I enjoy re-reading it, to the point that it's beginning to feel like masturbation.

So I just need to finish this part about how to commit crime in in crime free society and then I think I should do something about it. The working title is "Sonnets from a proton" if that gives you some idea of how embarrassing this could be...

A new form of government

After my last post I thought I'd post this.
A while ago I came up with an idea for the full devolution of democracy, using online voting to the n'th degree. Based as much as possible on various voting/communal online systems. I'm not saying it's without flaws, but I present it here for comment and improvement.

The basic idea goes that everyone has the vote on every single subject.
But i hear you cry, this would be crazy, for a start there'd be too many decisions every person to make.
Correct. That's why you can appoint proxies.
For example if the subject to be voted upon concerned agricultural policy, and your mate Hugh nicely summed up your views on the subject, or at least summed them up enough that you trusted his views then you'd let him vote on your behalf for all fishing related things.
Let's say you met this guy Jeremy down the pub and you liked his ideas on the transport system, you could delegate your transport related votes to him.

So a bill gets proposed in true online democratic fashion (how this happens I'll get back to) and people tag it with tags in order to categorise it according to the various categories. Agriculture, Transport, Education, etc. Obviously most things would have several tags and therefore there would be conflict. This would therefore need you to have an arbiter in case of conflict. Depending on how important the subject was (determined by the amount of acticity on that topic) you would then either delegate it again (perhaps to the equivalent of your MP) or make the resolution yourself.
Ah, but how would the bill get proposed. Simple people could vote bills up or down. If your delegate chose to vote a bill up or down that was within his delegation remit then he would vote it up with the force of all the votes he had control over.

Now there are a number of issues with this system, not least the common problem of people voting for more public spending and lower taxes. There'd be so many on the dole voting for more taxes on the rich that the economy would probably collapse. Given every bill would have an economic impact and assessing the impact of this would be none trivial then how do you even assess the cost of each proposal never mind balance the budget.
My idea, (although it too is flawed) the personal tax allocation. You buy into government services. For example you can vote up or vote down the spending on military, but the cost is shared equally between all taxpayers. You can vote for more or less money spent on roads or whatever but you must pay your appropriate share. Two problems with this, how do you decide what is fair and how do you prevent the race for the bottom.
Simple, when it comes to tax we formalise the agreement that already exists through lobbyists and bribes, he who pays the piper calls the tune - sort of: on a logarithmic scale the more tax you pay the more say you have in financial matters. That is when it comes to tax policy it costs you (at least) four times as much to buy twice the votes. If you as a rich man vote for yourself to pay less tax, you then have fewer votes with which to maintain your position.

As I say it's far from perfect, but it's certainly different from our current scheme.

Who writes the rules

This may be an old theory, but it certainly isn't mine, can't remember where I came across it and it was fairly recently; but it was new to me, so I'll run with it.
The idea goes that in the old days it was the religious leaders who made the rules. It was all about absolute authority and follow these arbitrary rules or else. From there you got kings and the divine right and all that stuff.
Then the lawyers came along with ideas like "No, man has to make the laws for himself" and so you got things like the republic. The idea coming down to anyone who had the right education and connections could make the rules. Yes you had democracies and yes the population had a vote on who made the rules, (to prevent absolute abuse of power) but the general population was not viewed as overtly competent; more-so that like a legal system there were checks and balances placed in the system itself.
Now you've had Wikileaks and the proposal is that this is an attempt to apply scientific/engineering principles to the field of law. This was the new idea to me, so bear with me here. What science and engineering fundamentally say is that it doesn't matter who you are or what your education is, it doesn't matter as long as you are right.* Now this then leads to the concept that as long as information is freely available and debate open and free then the correct way to do things will be found. In fact the best thing people can do is collect data and publish it. It will occasionally cause upsets and disturb the status quo, but it all advances the state of the art.

To me this is an interesting thought, will you have a society that is ruled not by dogma, or a collection of rules debated by a subset of society but an honest effort to find the optimal behaviour for the species. I'm a big fan of Ian M Banks' writing particularly his Culture works and the anarchic society that is the culture. Not that I am an anarchist, I really don't believe that would ever work unless you have the post scarcity society that is the Culture and the benevolent god like puppet masters that are the Minds, but then at that point it's not really a human society anymore is it?
This is a concept that I've tried to explore in my own writings but it's really hard because I also have to first explore a society that is transitioning to a post scarcity society.
now all this is very tempting to try and work into my writings too, but they already becoming a tangled mess of complexity. I really should start publishing them and getting feedback...

Back to the point this is all about the evolution of the memes that that form society, potentially there are a set of memes and laws that form a society that is better than our own, but what are they. One or two very powerful men set out to give their opinion on that and you got religion. Then a greater mass of learned men tried to reason it out. Is the next step to open it up to public debate and iterative improvement?

No idea.


* "While the church knows no argument but force we know no force but argument" springs to mind. Fine there is definitely a hierarchy of respect within the scientific community, but at the end of the day if the lowliest school child came up with a demonstrable proof of anti-gravity, or some way to unify quantum mechanics and relativity then their status (hopefully) wouldn't matter, only the results.

Moving? Buying a House

I've been thinking about the process of moving house (which I've just completed recently).
To sum it up a few things I've noticed:

  • You have to love the property when you are considering buying, you have to want it and feel it is perfect for you. That way by the time you get it, you'll merely tolerate it.
  • It doesn't matter what budgets you make, it's more expensive than you think
  • It doesn't matter ho careful you are, there's more wrong with it than you thought.
  • Fixing things takes a lot longer than you thought.
  • It takes a lot longer to get stuff done once you've moved in
  • Ever wonder how old married couples get that way? I have stopped doing so.
  • It soon becomes clear the work you need to do on it will never end. This is good, and this is bad.
  • You'll always grow to fill the available space in the house, try and make sure you have room for expansion one way or another
  • You have more stuff to pack than you thought.
  • Rental agencies have become worse in the last couple of years at moving out inspections, but that's another story.

All in all a positive experience. I'm not looking forward to doing it again though.

Friday, 3 December 2010

Electronics for Dogs 6 - Our final idiot

So let's have a recap of our idiot. What can he do?
Well he has a whiteboard with boxes A to H on them.
He has a set of pigeon holes each with 8 switches and lights on them. Normally a switch will turn on the corresponding light, (but not always). Trust me when I say that it turns out to be very cheap to make these pigeon holes. Having billions of "holes" with 8 switches and lights in each hole is not a problem. Because of this we'll give each hole a number, or better yet an address.
He also has a set of instructions that we have given him. However these instructions have to be very simple. Let's have another look at these instructions. How simple can we make them? Let's take it to the limit:

Add the number in box A to the number in box B place the result in box C. If anything odd happens record the oddities with numbers in box D
Copy the value from box x to box y replacing the value in there with the new one (substitute any box designation in here)
Take a value from the pigeon hole number listed in Box D and put that value in Box A
Take the value from Box A and put it in the pigeon hole number listed in Box D
Set box A to value specified.
Test box A if the bit number as specified in Box B is a 1 then do not do the next instruction.
Stop following this instruction stream and start using a different one
And, XOR and NOT instructions. (more on these later)

And there you have it, less than a dozen instructions and capable of solving any problem you can throw at a computer.
Don't believe me? Everything I've discussed so far can be solved with those instructions. now a nice processor will make life easier for you by having more complex instructions, but let's get the basics done first.

One thing to say is with just a dozen instructions then why not have the instructions as a set of numbers? Rather than say "add this to this" then "move this to this" just say 12, 4.
When the instructions are just numbers then then the instructions can be placed into the pidgeon holes too. When our idiot turns up for work in the morning he doesn't have to think. He just looks at address 0 in the pigeon holes and sees the number 5 which tells him that he needs to start by writing 5 into box A then he looks at address 1 and sees he needs to...

Hang on though, how do you work out what numbers to put in there? That's the programming we've discussed but let me give you a final recap before we start getting into the electronics:

User Interface designer says:
We need a box that says Publish Post in it

Programmer Thinks:
First I need to draw a box, then set it's colour, then put the text publish post in it

Programmer writes - in a language I just made up, but not a million miles off:
draw_box(top_left_coordinate, top_right_coordinate, bottom_left_coordinate, bottom_right_coordinate)
Set_colour(orange)
Insert_text("Publish Post")

The Compiler (A piece of software, another program) looks at draw_box and finds the code associated with that. That code includes such things as:
create_triangle(top_left_coordinate, top_right_coordinate, bottom_left_coordinate)
create_triangle(top_right_coordinate, bottom_left_coordinate, bottom_right_coordinate)
so it then looks up create_triangle which includes code such as
vertex_x = coordinate_x + (offset*depth)

Which it translates into:
set A = offset
set B = depth
jump to multiply function
set A = coordinate_x
set B = C
add A to B and place in C
set address vertex_x to C

Which then gets passed to the assembler which can then rattle off a stream of numbers corresponding to the above instructions. e.g. set A to offset is saying set A to the value at memory address (wherever we chose to put offset in memory). so it's just a lookup that it has to do.

Now at this point I'm going to leave computers because I'm not sure more detail is necessary. If you want more detail on any of the above shout and I can always write more. However I'm not currently interested in detailing how for example the 3D rendering engine works no matter how interesting you may find that or otherwise, I just want the concepts to be clear as to what sort of processes you go through to get from a publish post button on a web form to a sequence of numbers in memory.

So next we move onto some electronics. To the point where you might be able to build one of these processors. Might...

Electronics for Dogs 5 - some loose ends

So in the first post I told you there were some things to not worry about. Some things to trust me on. Hopefully now I'll complete the picture. This should be a very short post.

Let's assume that our idiot is running a washing machine. We're going to make things a bit easier for him but in this case he, as always, has no idea what he's doing. He's just following instructions and doing as he's told.
I'll make it easy and have a number of lights on the wall labelled with numbers. In actual fact these will be connected up to sensors on the washing machine. One light will be the door open latch, another will be the half full indicator, another will be the full indicator, another will be the overflow. One will be the on switch.
There'll be another indicator that as far as he knows gives a number from 0 to 7. In fact that will be the program number. There'll also be a number of switches for him to throw. One will be connected to a pump to pump water in, one to pump water out, one to turn on the motor that runs the drum etc. But he won't know, it's just a bunch of switches to him.

So he sits there and his first instruction says wait for light 1 to switch on. When it does look at the number on indicator 1 and use that to determine which set of instructions to use next.
Having picked instruction sheet number 3 he sees that his first job is to see if light number 4 is turned on (is the door closed) if it is then turn on switch number 6.
And on and on...

So far so good, but what does this have to do with a processor? Did you notice how all of that up there was to do with numbers? If light number 3 is on could very easily be is value x 1 or 0. Throw switch number 6 could be set value y to 1.

Time for a new concept. Let's make this idiot's job even more basic and simple. Let's get rid of the lights and switches Let's just give him a wall of numbers. Think of it as a wall of pigeon holes if you like, so that instead of a light coming on the number in box x changes from being 0 to being 1. Instead of throwing a switch he puts a value into that pigeon hole.
This makes things simpler for him because now everything is a number, if he's solving the budget or operating a washing machine or running the phone exchange all he has is a wall of numbers and a set of instructions. In the real world getting things like the status of a door latch into a number in a box on the wall would be very difficult. For the computer however it's easy!

I said previously that everything in a computer is a switch. Numbers are stored by the state of a number of switches. The difference between a switch that stores the last value you told it, and one that tells you the state of the washing machine door really isn't that much. Going back to the pigeon hole analogy instead of numbers on a slip of paper let's say we had 8 switches. Each switch had a light associated with it. You'll remember from the last post that 8 switches means that you can story a value between 0 and 255. So that's what our idiot is doing, you tell him to store the number 4 and actually he's not even aware he's doing that (I did tell you he was very stupid didn't I) he's setting the 8 switches to 00000100. That is just flick on the switch of bit 2 (we start numbering at 0) and make sure all the others are off.
Sometimes these switches will be just connected to the light associated with it. Sometimes they'll go and do other things. Sometimes the light associated with the switch won't be connected to the switch at all and will be connected to (for example) a door latch. It doesn't matter he doesn't care that's a separate problem.

Believe it or not that is what your keyboard and mouse are doing. When you push a button it closes a switch. This switch closure is detected and presented as a number to the computer. But if it's a USB keyboard or mouse then the only way your computer knows about this is that somewhere in its memory a value has changed and it has to occasionally go and look at it to see if it has.
Having said that I'll immediately retract it because USB doesn't work quite like that. In most implementations it's more a case of the processor will regularly check memory and be presented with a list of devices that have had updates, then it will have to go and look at those updates to see what they are; however the principle is still valid that the only way the processor knows that something has happened is because it goes and looks at some memory and sees what value it is then goes and does something else based upon that value. Once again it's all just numbers at the end of the day.
Oh and if anyone is old enough here to chirp in and start talking about I/O address space and the like then yes, well done, have a cookie, but often it's not done that way anymore for good reasons, stop trying to confuse the issue, that's my job!

So next time we'll take our last detailed look at our idiot making him the most stupid ignorant individual you'll probably meet. But that's okay because the dumbed down version is easy to copy.