SOPA and PIPA

Worth a read

http://www.ted.com/talks/defend_our_freedom_to_share_or_why_sopa_is_a_bad_idea.html

The Higgs Discovery Near?

I love Europe. This would not be possible without their investment in Physics and LHC. (The RIM guys did good here in Canada as well)

And whether it is finally found or not, we can be sure it will be either "proved" or "disproved" in the next short while (short - relatively.. months or a few years max).

Exciting either way.

http://www.bbc.co.uk/news/science-environment-16116230

(Almost) fixed back pain, and how it relates to 2 years off from open/free source tinkering hopefully over

Since I was a teenager I had issues with my back. Perhaps not so surprisingly, they always got worse not by doing "something" (physical like sports) but rather doing "nothing" (such as sitting at a desk, thinking, and/or coding uninterrupted for weeks of 16 hours a day if not more). Playing hockey once a week did not really help much. Over the years I did more and more of it, not only doing paid work or things that would initiate it, but also did large amount of tinkering and playing with open source stuff.

Finally, over two years ago, I had an extensive busy period at work, while starting software for a shutdown planning prospective business, and also playing and involved with Etoys, Squeak and other software that interests me.

Unfortunately the load also resulted in an extended period of back trouble to the point it was hard to even do my work. So about two years ago or so, I decided to quit everything but software I need to do for living to save the sanity of my back. I still did follow things like Etoys, but spent way less time. Also I started doing things to fix things - gym, five minute breaks, better posture, and general "being aware" when things go bad.

All that waiting seems to be working so I plan get back to do more things for fun. I started to investigate Google's Native Client as a way from the prison of "web architecture", ran into some early issues on Opensuse but that got resolved. I plan to look at Yoshiki Ohshima's Squeak port to Nacl and see if Etoys can run that way instead of in a plugin. Not quite a "commitment" but a plan.

Exponential Growth of Interest in Dart

Exponential Growth of Interest in Dart

So,  the GOTO in Aarhus is long in the past. Dart was introduced. It's features are good as one would expect from people who know what they are doing. I would prefer a Smalltalkish-Newspeakish syntax but did not really hope for it too much - the world does move slowly after all.

I kept an interesting statistics the first few days after Dart was announced. I keep my links on delicious and every day I checked number of delicious bookmarks marked as dart for http://www.dartlang.org/. It was surely non exact, the count was taken at different times of the day, rounded them, some days I forgot, but it went like this:

2011-oct-10 : 712 saves
2011-oct-11 1924 saves
2011-oct-12 3800 saves
2011-oct-13 6700 saves

2011-oct-14 ??? saves

2011-oct-15 12585 saves

2011-oct-16 ??? saves

2011-oct-17 ???? saves

2011-oct-18 24000 saves

2011-oct-19 28500 saves

Now, the count is at 42121 saves. Count multiplied by 2 every day the first several days looks like an exponential growth of interest in Dart. 

The Dart mailing list goes steady around 25 messages a day and it looks like a vibrant community being  created. Dart is evolving, good and needed platform. Perhaps we are getting a safer, faster, concurrency friendly language and platform that will make programming for the web a better place. 

Gilad Bracha / Lars Bak / Google - Dart: Progress the web

Gilad Bracha / Lars Bak / Google - Dart: Progress the Web

Three weeks ago or so, I came back from a Toronto Smalltalk Users Group meeting, to a nice surprise: a bit of a  late night (here in TO) flurry of articles about Dart - or Dash(?) - "a new programming language from Google". I think Dart will be more than a "language" - rather a vm/environment that will run inside the browser (and also outside surely).

This is extremely exciting news, for a few reasons:

First, the people and the architecture: Gilad Bracha (Strongtalk, Java, Newspeak) and Lars Bak (Also Strongtalk, Javascript V8). I am sure they will do a great work; if this goes the right way and Dart becomes accepted, it may be a huge step towards secure, performing web architecture and applications, and finally, after 20 years, may progress "the web" beyond it's primitive stage (web, unlike "the internet" is a crappy construct developed from a simple sharing system of scientific articles, still bears all the architectural chains and shortsightedness of it's beginning).

Second reason for excitement, Smalltalk. Just to be brief here: 30-40 years later, depending where you start counting, Smalltalk is still more elegant, clean, and beautiful than all mainstream (or non mainstream) languages. Just one thing, would you rather createRectangle(10,20, 30, 40) (what is 10, 20, 30, 40 .. even with better naming the order is unclear), or createRectangleAtX: 10 y:20 width: 30 height: 40? I am guessing Dart syntax will borrow from Smalltalk, but that is just technicality, although a welcome one.

Third reason for excitement, security and performance above what Javascript can offer.

Some relevant links:

The leaked motivation (which is a great read by Mark Miller): http://pastebin.com/NUMTTrKj

Probably the best balanced read on the Dart subject I could find: http://css.dzone.com/news/google-dart-“ultimately

A few more Dart speculation links:
http://www.extremetech.com/computing/95417-google-announces-dart-programming-language
http://www.pcmag.com/article2/0,2817,2392791,00.asp#fbid=rl8OouL9eVJ

October 10, 2011, I will be looking forward to read news from Arhus!

PS: I am no fan of one company taking over my world (had my shares of passive/active resistence to Microsoft ...and happy to say gradually proven right), but the combination of open sourced, and developed in the open, of Nacl and Dart may be the sanity rescue of the web. The alternative is the "as we went", shortest path, ugliest solution web we watched "develop" the last 20 years. We will see in 5-15 years!

Freedesktop / Linux MimeTypes, File Extensions, MimeType-Application Associations

This is a capture of Linux KDE/Gnome/Freedesktop files that define how MIME types such as "html"are managed by the system, and used by Applications such as Firefox. This includes File Associations and Default actions.


1. Related Links:


http://standards.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html#mime-types
http://www.freedesktop.org/wiki/Specifications/mime-actions-spec
http://standards.freedesktop.org/shared-mime-info-spec/shared-mime-info-spec-0.18.html
http://www.freedesktop.org/wiki/Specifications/shared-mime-info-spec

2. Overview of What is involved:














3. Directories and Files Managing MIME (database) and Application (Database) 

3.1 Shared MIME-Info Database

Is a file-based database of all known MIME types. The database's files are located in the following directories (further down overrides) /usr/share/mime/*/*.xml, /usr/local/share/mime/*/*.xml, and~/.local/share/mime/*/*.xml (if they exist, and in this order).
Each has maintains an application which manipulates this database - for example, in KDE, Configure Desktop -> File Association

The database starts in directory named "mime" - for example /usr/share/mime - which has following files


all
application
audio
...
packages
...
text
(etc)


As an example, /usr/share/mime/text directory has following files:



css.xml
html.xml
plain.xml
richtext.xml
sgml.xml
vnd.graphviz.xml
x-cobol.xml
x-erlang.xml
x-fortran.xml

(etc)

As an example, html.xml defines MIME type "html", this way:

So html.xml defines:

The MIME type name (html)

The MIME Parent Type (text/plain)

Description (in all languages)

List of File Extensions which will assume this MIME Type

Application section of the the Shared MIME-Info database

this is in the mime/packages directory.

 For example, doing ./configure && make install with Firefox should create /usr/share/mime/packages/firefox.xml (but it does NOT).

3.2 Desktop Entries for Applications http://standards.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html#mime-types

 KDE and GNOME desktop environments have adopted a similar format for "desktop entries", or configuration files describing how a particular program is to be launched, how it appears in menus, etc. These files end with extension .desktop. The suffix "desktop" seems a misnomer - it seems .desktop files simply describe Applications

Desktop Entries Database (=Application Definitions Database) (NOT Part of MIME-Info Database, but uses it)

This database is located in /usr/share/applications/*, /usr/local/share/applications/*, and ~/.local/share/applications/* (if they exist, and in this order). Further, if desktop=KDE, ~.kde4/share/apps/* is also part of the database.

The /usr/share/applications directory has following files

kde4

YaST2

google-chrome.desktop

firefox.desktop

gimp.desktop

emacs.desktop

....etc

for example, /usr/share/applications/firefox.desktop contains

[Desktop Entry]

X-SuSE-translate=true

Categories=Network;WebBrowser;GTK;

Encoding=UTF-8

Name=Firefox

GenericName=Web Browser

Comment=Web Browser

TryExec=firefox

Exec=firefox %u

Icon=firefox

Terminal=false

StartupNotify=true

MimeType=text/html;text/xml;application/xhtml+xml;application/vnd.mozilla.xul+xml;text/mml;application/x-xpinstall;x-scheme-handler/http;x-scheme-handler/https;

Type=Application

Note: The MIME Type list above can be expanded to list of file extensions. This is displayed in "File Association" settings.

Clearly some component scrounges through the applications/*.desktop, and if the *.desktop lists the MIME under MimeType, it will add the application.



Note: To add a new application, such as emacs-client, that appears in popups of right click open-with, simply

• create /usr/share/applications/emacs-client.desktop
• copy contents from emacs.desktop
• modify Exec=run-emacs-client.sh
• modify Name=Emacs Client

3.3 ~/.local/share/applications/mimeapp.list - user defined order of application preference when open

• maintains ORDER of Applications Which Understand a certain MimeType
• Format

[Added Associations]
mimetype1=foo1.desktop;foo2.desktop;foo3.desktop;
mimetype2=foo4.desktop;
[Removed Associations]
mimetype1=foo5.desktop;

• Sample:

[Added Associations]
application/x-shellscript=run-emacs-client.sh.desktop;
text/x-emacs-lisp=run-emacs-client.sh.desktop;

[Added KDE Service Associations]
application/xhtml+xml=kwebkitpart.desktop;khtml.desktop;
application/xml=kwebkitpart.desktop;khtml.desktop;
text/html=kwebkitpart.desktop;khtml.desktop;

Webcam Stream over the Internet on Linux, with VLC and VLC-QT

I am playing with streaming Linux webcam output over the internet. Apparently one package for streaming is VLC (it is it seems the most featured video handling package overall).

Eventually I got webcam VLC streaming working on Linux (Opensuse 11.3)  in multiple ways. This was not straighforward (what with video is on Linux :) ) - so I took take some notes below, sort of a step by step process. Now, I am succesfully streaming webcam over the internet, using bandwith-saving multicast and dynamic IP for my domain milanzimmermann.com. (Next: to connect the webcam inside a bluebird' nesthouse.)

Step A: Stream an AVI file from one instance of VLC-QT, capturing and playing on other instance of VLC, without transcoding as H.264.

This is a first step to create a localhost client server setup. The bricks that need to be connected for a working simple configuration are as follows:


1. Source of the stream: an AVI file, downloaded from a multimedia test site:  Electric_Guitar_Scrape_And_Pound_DivX.avi

2. VLC setup as a streaming server. VLC reads the AVI file, and outputs it "straight" (without transcoding) as UDP, without transcoding as H.264. To achieve this, do the following:

on command line, run VLC:
Server prompt $: vlc # the VLC gui starts
click media->streaming, dialog opens. following describes clicking through the VLC wizard (each line represents a click on a UI item):

file
add - browser to the Electric_Guitar_Scrape_And_Pound_DivX.avi
stream
next
new destination: udp
check "display locally" - this will allow viewing on this streaming VLC server
add
address = localhost
port=1234
check "activate transcoding" OFF !!
next

having done the above, the screen now shows the parameters VLC generated from our steps:

"

:sout=#duplicate{dst=std{access=udp,mux=ts,dst=localhost:1234},dst=display}

"

From now on, we can simply start the server as follows: 

server prompt $: vlc -vvv /home/mzimmermann/Documents/Electric_Guitar_Scrape_And_Pound_DivX.avi --sout '#duplicate{dst=std{access=udp,mux=ts,dst=localhost:1234},dst=display}'

This streams to 2 destinations: the display, so we can see what the file contains, and to a UDP output on localhost, port 1234.

3. Server Output Stream: The above set of steps will start streaming the AVI file into UDP on localhost, port 1234, and also the VLC "streaming server" display., and at the same time, as the client input stream, processed in the "client VLC" - see step 4.

4. VLC client, setup as UDP client

on the client system (which in out case is same system as the server), run the client VLC as follows:

Client prompt $: vlc udp://@:1234   # Note no localhost is specified on the client uvc (This is because any-source multicast does figure out the source of the stream on the network!!)

If you run the above, you will see the webcam displayed on the client VLC.

5. Summary - run it all:

Now, find the streaming server VLC GUI, and click the play sign. This will start playing the AVI file in the "streaming server GUI" started in Step A, item 2, and also streaming it to UDP localhost port 1234. The UDP stream is  automatically showing the video in  the "streaming client GUI" started in Step A, item 4

Step B: Stream from AVI file (same as in Step A) from one instance of VLC-QT, capturing and playing on other instance of VLC as RTP, without transcoding as H.264. (difference from step A, we use RTP instead of UDP)

Server prompt $: vlc -vvv /home/mzimmermann/Documents/Electric_Guitar_Scrape_And_Pound_DivX.avi --sout '#duplicate{dst=rtp{dst=localhost,port=5004,mux=ts},dst=display}'

Client prompt $: vlc rtp://@:5004

Note that the WEIRD difference between the UDP and RTP specification for --sout !!!!!! - but it makes a difference for playing . I have no idea why this is so messed up... (the port number does not matter as long as client/server is same)

Step C: Stream from Webcam as source (instead of an AVI file in Step B) over Real Time Transport (RTP)  from one
instance of VLC-QT, capturing and
playing on other instance of VLC, without transcoding as H.264.

Server prompt $: vlc -vvv v4l2:// --sout

'#transcode{vcodec=mp4v,acodec=mpga,vb=800,ab=128,deinterlace}:rtp{mux=ts,dst=localhost,sdp=sap,name="TestStream"}'

vlc rtp://@:5004

Client Prompt $: vlc rtp://@:5004

This shows webcam in client (really?? - not sure this worked for me)

Note: vlc comes with it's own HTTP server, start it like:

vlc -I http --http-host localhost:8083

Step D: Multicast UDP Streaming (probably works same for RTP). This works, BUT client can only access localhost as "vlc udp://@:1234" I have NO IDEA why I can specify Multicast on server (something in range 239.255), but on client , no IP can be specified, otherwise the stream does not show in the client...  

See also  http://www.videolan.org/doc/videolan-howto/en/ch08.html

Server prompt $:  vlc -vvv v4l2:// --sout '#transcode{vcodec=mp4v,acodec=mpga,vb=3000,ab=256,vt=800000,keyint=80,deinterlace}:std{access=udp,mux=ts,url=239.255.12.13}' --ttl 12

Notes:

1. The URL can be almost anything, e.g. 239.255.12.13@milanzimmermann.com and client still works
2. The ttl 12 specifies time to live - how many routes UDP can pass. It is 1 by default. For clients outside of the local host, override the default (we use 12 here)

Client prompt $: vlc udp://@:1234 # But the client CANNOT USE ANY IP, like vlc udp://@239.255.12.13:1234 - thiis does not work why?

Step E: VLC server controlled via telnet: Serves stream as UDP-Multicast on mulitcast IP=239.255.12.13 and also HTTP-Singlecast. Cients can view the stream via HTTP or UDP.  This can be setup in 2 command prompts (client/server telnet). We also need a command prompt for testing http access- see also http://www.ringoy.org/?p=40.

Prerequisits:

1.In router, enabled multicast (Linksys Advanced->Advanced network settings), enable multicast streams
2.In router, enabled port 1234 (for both UDP and TCP, although UDP likely not needed) (Linksys Advanced, add open port)
3. V4L2 is installed, and Webcam works .

Start Shell Prompt 1 (Server) and type in:

vlc -I telnet # Start VLC server and open port 4212 for telnet.


Start Shell Prompt 2 (Client) and type in:

telnet localhost 4212  # Starts VLC client - here we use localhost, as we run from the same system as telnet. But it can be run from anywhere with telnet access to the system that started server in shell prompt 1. Would have to reaspecify some-ip instead of localhost. some-in is IP of the system wistarting server in step 1.

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Password: admin
Welcome, Master

> new cam broadcast enabled

> setup cam input "v4l2://"

> setup cam output "#transcode{vcodec=mp4v,acodec=mpga,vb=3000,ab=256,vt=800000,keyint=80}:duplicate{dst=standard{mux=ts,dst=239.255.12.13,access=udp,sap,name="CAM_UDP"},dst=standard{mux=ts,dst=:1234,access=http}}"

> setup cam enabled

> control cam play


Start Shell Prompt 3 (Accesses the system where Client telnet from prompt 2 is running):

vlc http://milanzimmermann.com:1234

(This will display the webcam stream)


Enable support for viewing webcam stream via HTTP

a) in the HTTP server, create a file similar to the contents this:( credit http://videolan)

http://milanzimmermann.com/test-vlc.html

b) From anywhere on the internet, browse to:

http://milanzimmermann.com/test-vlc.html

and see the video

Step F: FINAL "SOLUTION": Serves stream as UDP-Multicast on any-source mulitcast IP=239.255.12.13 and also HTTP-Singlecast. Clients can view the stream via HTTP or UDP.

Prerequisits for the steps below to work:

1. Your kernel has multicast enabled (if you want to use multicast, which is a must for multiple simultaneous clients).
2. V4L2 is installed, and Webcam works .
3. In your router, enable multicast (Linksys Advanced->Advanced network settings), enable multicast streams
4. In your router firewall, allow port 1234  (TCP if you use http, UDP for udp - we use both here) (Linksys Advanced, add open port)
5. If you are using Masquerading (at home most likely you are), you also need to forward port 1234 to the server - in our example, to 192.168.0.199

Server Prompt (running on system with Intranet IP=192.168.0.199, Internet IP milanzimmermann.com) $: vlc $HOME/Documents/Electric_Guitar_Scrape_And_Pound_DivX.avi --loop --sout "#transcode{vcodec=mp4v,acodec=mpga,vb=3000,ab=256,vt=800000,keyint=80}:duplicate{dst=standard{mux=ts,dst=239.255.12.13:1234,access=udp,sap,name="CAM_UDP"},dst=standard{mux=ts,dst=:1234,access=http},dst=display}" --ttl 12

Notes:

1. 239.255.12.13 is the IP of the UDP multicast group. Clients must specify this group to access the multicast. More precisely, clients must use source_system_ip@239.255.12.13 - see below for sample client use.
2. To stream from a webcam rather then the AVI file above, replace the path to the AVI file "$HOME/Documents/Electric_Guitar_Scrape_And_Pound_DivX.avi" with "v4l2://", and remove the "--loop" option.
3. To prevent the stream from showing in the VLC window on the server (does it save anything?), remove the ",dst=display" above.
4. Note that we use port 1234 both for http:// (uses TCP) and udp:// (uses UDP)
5. To remove streaming across http, remove the ",dst=standard{mux=ts,dst=:1234,access=http}" above.
6. The ttl 12 specifies time to live - how many routers UDP can pass. It is 1 by default. Non-default value (default 1) is needed for UDP clients to work outside the local host (we use 12 here).
7. The "--loop" option will play the file over and over - good for testing the client.
8. Trying to use "dst=milanzimmermann.com:1234,access=http" instead of "dst=:1234,access=http" does not work. No idea why.
   

vlc as udp client from prompt any host on localnet $:
vlc udp://@239.255.12.13:1234 # This works. Alos 1234 is default for udp

Notes:

1. The @239.255.12.13 is ANY-SOURCE-MULTICAST

vlc as udp client from prompt any host on localnet $:
vlc udp://192.168.0.199@239.255.12.13:1234 # This works.

Notes:

1. I think this only works because of coincidence. I think host only provides ANY-SOURCE MULTICAST, NOT SINGLE_SOURCE_MULTICAST_ IP@239.255.12.13
2. IP for SINGLE_SOURCE_MULTICAST_ IP must start with 232 (not 239) but I do NOT think VLC works for SINGLE-SOURCE
3. 192.168.0.199 is the source IP = IP of the system where multicast sender creates the multicast source. @239.255.12.13 is multicast IP served by the VLC server.(see the server vlc command)
4. 192.168.0.199 can NOT be replaced with 127.0.0.1 even when running from same system as server. 
5. If the client n the same system as server, you can omit 192.168.0.199 use vlc udp://@239.255.12.13

vlc as udp client prompt from any host on internet $: 
vlc udp://milanzimmermann.com@239.255.12.13:1234

1. This does NOT work - because UDP port not open in firewall??  
2. No, I think this is because milanzimmermann.com@239.255.12.13 indicates SOURCE-SPECIFIC-MULTICAST, but the server only provides ANY-SOURCE-MULTICAST

vlc as http client prompt from any host on localnet $:  
vlc http://192.168.0.199:1234 # This works if port 1234 is open on server

vlc as http client prompt from any host on internet or localnet:  
vlc http://milanzimmermann.com:1234 # This works, if port 1234 is open on the server.


http browser client from any host on internet or localnet:  
type address http://milanzimmermann.com:1234 into the browser # This does NOT work as browser is not VLC client (but it can be if stream served via embed tag on html, see below)

vlc plugin in http browser with VLC PluginFrom anywhere on the internet:

to try this, browse to:

http://milanzimmermann.com/test-vlc.html  # This works, as long as the page is written correctly with the embed tag, and VLC plugin installed in the browser. See the source of the above page


This concludes it alll.... (but test step F on source=V4l2 instead of AVI file.... and attach to the multicast via internet ... what ports does it go through?)