In some ways the problem with programming is that it doesn't follow or establish locks in the way that a person might. If I were working on a device and decided to do something and a dependent condition changed, then I would re-evaluate my response to compensate. In a computer the information is serial and not parallel. In a case where a tiger is hunting an antelope the situation would be updated and all solutions would be recalculated on a snapshot of time. If the tiger made a decision and in the midst of that decision the conditions changed, he would compensate. What made me consider this is an odd software glitch that happened with some code that I wrote. In a computer there are many simultaneous processes taking place and in some ways it is a schizophrenic act to program. I will just give the example and let you draw your own conclusions.
The language is Python, though that has nothing to do with the problem in any significant way. I have a program that goes through my email and checks to see if the data base files have changed upon the receipt of a new mail. Of course the email program is sliced and thus somewhat parallel. I walk through the entries and test each file for a change and set a flag to indicate that there is new mail in one or more of the accounts. I use a dictionary to start a new process for each new category of mail. After I have checked all the items in the list (dictionary) I use a flag that indicates changed data to rebuild the dictionary with the new file size. What apparently happened is possibly two concurrent events that require locking or intelligent interaction. There are two places where it can bite. If a file previous in the data base changes after a later file is identified as changed, rebuilding the entire dictionary hides the fact that an entry has changed. Secondly, when the data base is rebuilt, if changes occur between the time a change is identified and the time it is recalculated an item change can be ignored.
To solve the problem I used the dictionary of changed items to update the data base and did not update all items simultaneously ( not really in a computer that has threads ). It does highlight some of the problems with concurrent programming and failure to give much thought to the concurrent goings on. It is like considering the interaction of several cats in a frenzy. It isn't like the logic of a single sequential machine code of the "olden" days. Concurrency is a tricky game and seems to develop factorial complexity. I am actually very surprised that IBM's Watson functions as well as it does. They must have a couple of really talented people at the core of that adventure. So in this case the disk directory updates in parallel as well as the mail program. In this case I am only dealing with 3 concurrent processes, and still it can be confusing. The real problem is that I did not spend the time to analyze concurrency when the program was created. I will have to add that to my checklist when doing initial program tests. It does incorporate tests and this is reproducible, of course one would have to know that it was happening in the first place and that is is an odd coincidence where two mails are received at the same time and the data base is mid check when that happens. In the months that I have used this it has never happened before and today was the first time. It is a small window, but the cats got out.
Just because I have solved one problem and it works to my satisfaction and the logic is correct, the initial problem remains and concurrency was not considered in design. So I must go back and wash each part of the object oriented interface with thoughts of simultaneity. In considering this, I did identify another point where an even more unusual and rare effect would occur. If a second email arrived in the same group before I was to clear the reference, I would have a dead process thread hiding in my computer happily singing the song of chaos and mystery until the machine was rebooted. Because I create a database item in the dictionary by using the sub-process ID and the name of the category, an entry is created automagicly by Python, or in the case of a duplicate, simply over writes the old process ID and voila, strangeness ensues. Magic is good, but not knowing the risk of white Python magic is dangerous. So it must identify if a process is running before blindly creating or over writing an entry. This is a bit more convolved due to the fact that accessing an entry which does not exist is a big bada boom as far as the snake is concerned. So, ... it needs error handling and so it needs a try-catch block, which is try: and except: within the vernacular of the crawling reptile.
As always it is necessary to test the code to see if it works as expected and I have incorporated a dummy element which can be triggered by a key press. When I did the test by changing the element and then changing it again while it was in process, it did something completely and utterly unexpected. It is true, nobody ever expects the Spanish Inquisition.
It isn't just the local concurrency that can bite a person. The many cats that scramble about networks, screens and devices also will scratch a person when they least expect it. So here is the cat scratch. I have a key which is "t" to test the interface to mail. I focused to the console by clicking in its title and pressed "t", and all was right with the world as it spoke the proper words and created a sub process which displayed a ".png" with the proper image. I then pressed "t" again to overload the process and test what happens when two are active with the same name and BOING!, this showed up.
WTF! So the sub process "display" steals key focus when started and so the second "t" is in fact directed to the program "display" which wants to help you do a fuzz factor :)
So I realized I needed to refocus the proper window and when I did so, it did what I expected, hooray! and then ...... it did something even more odd in a few minutes. It had something to do with internal flags and something similar to link counts in the Linux file system like ext3 or ext4 and all others , I suppose. I increment a count of dictionary elements without creating a new element and as a result it could not reach zero by deleting elements as they were retired. So it ends and it works fine and is safe for concurrency, but like a bad movie , there will be 5 sequels before it is done. I expect "Scary Noises II" will arrive soon unless I go ahead and rewrite the whole thing coherently with some attention to detail. I wrote this several months ago and some how bit decay has rotted it, because I didn't write it that sloppy, it must have modified itself. If I were working at a company I could just blame the new guy, but perhaps I will blame quantum entanglement gone awry.
0 comments:
Post a Comment