Markov visits Monte Carlo again

I discovered a new ( to me ) course at MIT opencourseware that recaps many things I know and some that haven't fully sunk in yet. It is MIT_6-041 probability systems analysis and the link is lecture 17 which is the middle of  the Markov set of three which starts at lecture 16. It has new meaning when taken in context of Linear Algebra by Strang and others. When he discusses a Markov property, the matrix immediately comes to mind and many things progress from there, including the Schwarzschild radius, atomic orbital probabilities , and virtually all things that are in some way indefinite, like , well, everything.

The teacher John Tsitsiklis has a very good grasp of the concepts, which makes all the difference in the world when teaching complex concepts. Just one missing piece and many of these things become hopelessly complex. If you have any interest in statistics, this is a really good place to start.

As an anecdote, I went into the doctor and asked if he had a test for a 'Brain Cloud' and he said he had a test that was 100% accurate , if I had a 'Brain Cloud'. I said , "I need that test." He said that the test showed that I indeed had a 'Brain Cloud'.

Besides the fact that I am not Tom Hanks, I do not live in a fictional movie called "Joe versus the Volcano", I don't know Meg Ryan or her Dad from "Sea Hunt" Lloyd Bridges, and am a bit skeptical about anything that is 100% accurate .......

The example is just to highlight some non-intuitive things about probability in my own way. I like to test limits and so this is it. It turns out that in the case of "conditional" probability, I can always devise a test that is 100% accurate because it is predicated on an assumption. The test in this case is just to say yes. If I am asked if a person has a brain cloud, I will be correct 100% of the time when they have the condition. The problem arises when you don't have that condition. It makes me think of a collision avoidance system for an airliner that is 100% correct for detecting an imminent collision. You turn it on, and it says that you are about to collide with another aircraft. That would be really handy and would certainly keep the noobs on their toes, though it might be just as likely to cause a collision trying to avoid a statistical ghost plane.

So ... I like to have the lessons locally so I can run back and forth when I miss stuff or something interrupts like me falling asleep. The first script uses ffmpeg to get the names from the mp4 and convert it to a name of type that I like. I prefer not to deal with file names with spaces unless I have to. Also odd characters can make life difficult in file names. The scripts are hacked together in a few minutes and as such are certainly not state of the art. They might be instructive if somebody is learning script as it is a useful extension and saves a lot of point and clicky stuff if you can just think of what you want and the runes pour forth from your fingers. The funny thing is that things change and the older bones don't dance the way they used to because somebody is fu scr fiddling with the table. The example doesn't actually rename the files, but this a test script to see if it does what I expect before I put the last nail in the script. Usually a good approach for me as sometimes I tipe funy or just get my fnigesr crossed when enetering things and shtuff happens that is very bad. Shell scripting can be dangerous and it is best to trial the process before commit.

#!/usr/bin/env bash
#Sensible Renamer
for l in `seq 25`;
do
  fn=$(printf "MIT6_041F11_lec%02d_300k.mp4" $l)
  if [ -e "$fn" ];
  then
    ffmpeg -i $fn 2>&1 | grep 'title' 
    newn=`ffmpeg -i $fn 2>&1 | grep 'title' | cut -d ":" -f2,3 | tr " " "_" | tr ":" "_" | tr "'" "_"`
    newf=${newn:1}".mp4"
    echo $newf
    else
    echo "$fn not present"
  fi
done 

This  wget option [ --limit-rate=100k ] is to keep strict data caps from interfering with using Firefox or email connect or any data from the net while the script is busy.

#!/usr/bin/env bash
#About 18 minutes each for 100M files
for l in `seq 25`;
do
  fn=$(printf "MIT6_041F11_lec%02d_300k.mp4" $l)
  if [ -e "$fn" ];
  then
    echo $fn" Has already been downloaded, bypassing."
  else
    wget --limit-rate=100k http://www.archive.org/download/MIT6.041F10/"$fn"
    echo "Completed $fn $l of 25"
  fi
done

Just to make the bash tour complete, here is the last script I made after I did a rename because I like to see things in order. It is the actual rename part commented with a hash mark (#), so it is a neutered script and to make it effective the hash mark must be removed before the mv. Kind of a safety switch. As far as the sed stuff, you should consult the grymoire for details on why it is done that way. It looks worse than it is. I search for three things in a row ( (1)underscore,(2) a number between 0 and 9, and another (3)underscore .. then  add "0" before the second element, which is the number. Instead of 1-9, I get 01-09 for file names, which puts them in order on a normal ls sort. As always, I use kate editor and export as html to make the pretty highlighted version of the script.

#!/usr/bin/env bash
#Renamer so they show up the way I like on sort
for old_name in `ls *.mp4`;
do
    new_name=`echo $old_name | sed  's/\(\_\)\([0-9]\)\(\_\)/\10\2\3/'`
    echo "Renaming "$old_name" to "$new_name
#    mv $old_name $new_name
done