Using the web2py framework on Eclipse

Here is a nice post on getting web2py to work through Eclipse for faster/better debugging

web2py is one of the best Python frameworks out there are the moment and if you are into Python or looking for a framework to develop the next "killer" web app have a look at it.

Fix asterisk addons problems for Ubuntu

UPDATE Got a new,simpler, way to do this here

There is a bug in the recent asterisk-addons package shipped with Ubuntu 10.04
The result is that all the modules that are part of the package fail to load with a message like this

WARNING[13478]: loader.c:800 load_resource: Module 'app_addon_sql_mysql' could not be loaded.

Till this get fixed by the Ubuntu people here a quick fix

sudo aptitude install asterisk-dev
wget http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-addons-1.6.2.1.tar.gz
tar xvzf asterisk-addons-1.6.2.1.tar.gz
cd asterisk-addons-1.6.2.1
./configure
make

sudo su

cp -a apps/*.so /usr/lib/asterisk/modules/
cp -a cdr/*.so /usr/lib/asterisk/modules/
cp -a channels/*.so /usr/lib/asterisk/modules/
cp -a formats/*.so /usr/lib/asterisk/modules/

Generate passwords with crypt and random salt

Here is a small perl script i found sometime ago to generate crypted passwords with random salt values

#!/usr/bin/perl

$plain=$ARGV[0]; ## Read the command line argument

if (!$plain){ ## No parameter pased
print "Usage: $0 plain_text_password\n";
exit;
}

## Use the Process id & time to generate the salt.
srand($$|time); # random seed
@saltchars=(a..z,A..Z,0..9,'.','/'); # valid salt chars
$salt=$saltchars[int(rand($#saltchars+1))]; # first random salt char
$salt.=$saltchars[int(rand($#saltchars+1))]; # second random salt char
$newuser = crypt ($newuser, $salt);

## Generate the encrypted password
$crypted=crypt ($plain, $salt);

print $crypted,"\n";

Τα μαύρα χάλια μας και καλό κουράγιο

Παρακολουθώ τους τελευταίους μήνες όλα αυτά που συμβαίνουν γύρω από τα οικονομικά της χώρας και πραγματικά αναρωτιέμαι ο 2 χρόνος γιός μου σε τι κατάσταση θα ζήσει.

Ξεκινήσαμε από το "η κρίση εμάς δεν πιάνει", ακούσα για λεφτά που υπήρχαν και μετά "εξαφανίστηκαν" και έγιναν κοψίματα μισθών, νέους έμμεσους φόρους (τη χειρότερη μορφή φορολόγησης) μέτρα εδώ, μέτρα εκεί, κομπίνες, ρεμούλες και κατασπατάληση χρημάτων αλλά πουθένα δεν άκουσα κάποιον από τους πολιτικούς μας να αναλαμβάνει ευθύνη για αυτά που έχουν γίνει στην χώρα τα τελευταία 40 χρόνια και μας έχουν οδηγήση σε αυτή τη θέση.

Λέω μήπως, μήπως οι κύριοι βουλευτές, υπουργοί και λοιποί "παρά το ...." (συμπληρώστε το κενό με ότι καρεκλοκένταυρο θέλετε) θα έπρεπε πρώτα να κόψουν τις δικές τους *πραγματικές* αμοιβές 50% και μετά να ζητήσουν από όλους τους υπόλοιπους να "θυσιασούν" μισθούς και λοιπά άλλα ?
Όχι ότι θα σωθούμε έτσι, (γιατί και του χρόνου, αν δεν υπάρξει διακανονισμός τους χρέους, σεισάχθια το είπαν οι αρχαίοι τα ίδια και περισσότερα θα χρωστάμε) άλλα το να βάζουν όλοι αυτοί που έριξαν τη χώρα στα "βράχια" με τις πολιτκές τους,τις αποφάσεις τους και τις λοπές ρεμούλες τα τελευταία 40 χρόνια, τους "άλλους" να την ξελασπώσουν *πάλι*, και αυτοί να κάνουν εξεταστικές και να τσεπώνουν τα 300άρια άνα συνδρίαση είναι λίγο...γυφτιά.
Από την άλλη όμως, οι γύφτοι μια χαρά άνθρωποί είναι και δεν φταίνε σε τίποτα για την σημερινή κατάσταση, οπότε μάλλον θα πρέπει να αλλάξουμε τη λέξη από γυφτιά σε "βουλεφτιά" ή "πολιτεφτιά" γιατί μόνο έτσι μπορούμε να καταλάβουμε το πραγματικό νόημα του τι συμβαίνει.

Και για να μην παρεξηγούμαστε, όχι ότι είμαι αντίθετος στη μείωση της σπατάλης στο Δημόσιο, το κόψιμο των "επιδομάτων" στους αργόσχολους του Δημοσίου (έδω καταντήσαμε να παίρνουν επίδομα για να μην παίρνουν 'φακελάκι'και καλά) από τους οποίους οι μισοί τουλάχιστον θα έπρεπε να πάρουν πόδι και να πάνε σπίτια τους.

Δυστηχώς το μόνο που θα καταφέρουν οι πολιτικοί μας θα είναι μια τρύπα στο νερό, τα νούμερα δεν κάνουν λάθος.
Αν χρωστάς παραπάνω από αυτά που παράγεις δεν υπάρχει περίπτωση ποτέ να μπορέσεις να μειώσεις το χρέος.
Στην καλύτερη περίπτωση μπορείς να το μετατοπίσεις για "λίγο αργότερα" (όπως γίνονταν συστηματικά μέχρι τώρα) αλλά το τέλος θα είναι πάντα το ίδιο. Θα πτωχεύσης.

Είμαστε σαν το ασθενή που είναι στην εντατική και οι "συγγενείς και φίλοι" τον κρατάνε στη ζωή για να μπορέσει να τους γράψει το "οικοπεδάκι" και να πάρουν ότι μπρορούν πριν το "μοιραίο", και το "μοιραίο" ίσως είναι πιο κοντά από όσο εμείς νομίζουμε.

Οπότε, καλό κουράγιο Έλληνες.

Creating OpenOffice Horizontal Lines

Following is a 'shortcut' for OpenOffice to create horizontal lines

Horizontal lines can be created in Writer by typing a specific symbol three times on a line by itself and then pressing Enter.
The choice of symbol * - = # ~ _ decides the type of line that is created.

You can find more shortcuts and other OpenOfice related stuff here

Solving fax issues in Asterisk

"Asterisk","fax" and "problem" are three words you find often in forums and threads of people looking for help.
In the age of email,and pdf there are still large number of companies relying in fax to transmit documents.
The reasons are many, but the problems implementors face are common.

Bad quality of faxes with "missing" or malformed lines, large number of failed faxes etc.
Most of this issues come from two facts.
The first is that we are trying to use a medium that was primarily designed for human voice to carry high speed analog modem signals and second,that fax standards are not so "standard".

Putting asterisk to handle faxes can be both a blessing and a curse for those two reasons.

Asterisk has the ability to detect if an incoming (or outgoing) call is a fax and do a number of things, depending if its acting as a "pass-through" or receive the fax.

The first thing Asterisk does is try to detect if the call is a fax call and shutdown the echo canceler on that channel.
For this to work, asterisk listens for the CNG tone emitted by the calling fax and if found, disables the echo canceler in zaptel.

The CNG is in the CCITT (ITU-T) Recommendation T.30 and describes how FAX calls are established.

During the call setup/establishment process, there are two tone signals that are send:

CNG and CED

The CNG signal *may* be sent from the Originating FAX machine after dialing is complete.
The CNG signal consists of the transmission of 1100 Hz for 1/2 second, followed by a 3 second silent (2100 Hz OFF) period.

The CED signal *may* be sent by the Terminating FAX machine anywhere between 1.8 to 2.5 seconds AFTER answering the call. The CED signal consists of a 2100 Hz tone that is from 2.6 to 4 seconds in duration. The CED tone is useful for disabling any echo cancellers on the line.

(notice the *may*, don't you love clearly defined standards ;)

Asterisk then tries to redirect the call flow to the 'fax' extension, if its present in the context that is currently executing in the dialplan.
There you can use an asterisk application to receive faxes or redirect to another port etc.

Recently we faced a problem with one of our clients that is using an E1 for voice and fax calls.
The calls come in through the E1 and the faxes are connected to an 8 port analog card
Some of the faxes received had quality issues and we tried to figure out what the problem was.

After a lot of head scratching and countless tries we realized two things.
The PRI card was somehow "loosing" some frames and that the echo canceler, OSLEC in this case, was not shutting down when a fax was received.
The first issue was easy to solve, as we discovered that the PRI card was sharing the same IRQ as the network card, so switching the pci slot and making sure that the PRI did not share IRQ's with any other device.

The tricky part was to figure out why the echo canceler was not shutting down when the call was a fax call.
The problem with an E1 (or T1) is that the calls don't use "fixed" channel numbers as in a pstn card for example, where you know that number X is on channel Y.
The provider is sending the call to the first available channel and its up to the dialplan to determine where the call would be routed.
Shutting down the echo canceler manually for all of the E1 channels, solved the issue of the faxes but created 'random' problems of echo to the voice channels.

So i started looking in detail (i.e the code) how asterisk detects faxes.

What i realized is that asterisk needs a 3-5 seconds *after* the call is answered to determine if this is a fax call,shutdown the echo canceler and then jump to the 'fax' priority.
If you "bridge" the call to another zap channel (or to sip channel of an ATA for example) before that, then the EC stays on.
And this was exactly what we were doing in our case.
As soon as the call come in, we bridged it to the analog card, not giving enough time to asterisk to shutdown the EC.

So the problem can be solved by adding a delay of 5 seconds and let asterisk plenty of time to determine is this is a fax call or if your extension is a fax only extension (i.e you don't expect to receive any voice calls on that number) explicitly disable the echo canceller using the zapec(off) command in the dialplan.

[PRI_INCOMING]

exten => _XXXX,1,NoOP(Incoming call)
exten => _XXXX,n,answer()
;give the caller something to hear while we wait.
exten => _XXXX,n,Ringing
exten => _XXXX,n,Wait(5)

This was not a fax but a normal call so let's answer it
exten => _XXXX,n,Dial(SIP/,30,r)
exten => _XXXX,n,Hangup()

;This is were we land if asterisk detects a fax call
exten => fax,1,Goto(in_fax,s,1)

;FAX
[in_fax]
;turn the echo canceler off for this channel
exten => s,1,ZapEC(off)
;call the zap channel the fax is connected to
exten => s,n,Dial(ZAP,60,r,tT)
exten => s,n,Hangup

One drawback of having a single line handling both fax and voice is that detection can not be 100% accurate.
This can be either because Asterisk did not detect correctly the CNG or the that calling fax was not sending a CNG at all.

A quick note here is that all of the above are true for zaptel based channels (pri and pstn cards) plus bristuffed (bri) zaptel.
I am not 100% sure how other channels (Sangoma for example) handle this.

And one last advice.
To make the quality of of faxes even better make sure that you have the ECM mode of the fax turned on both for transmition and reception.

Get the md5 hash of a file from Python

Python has a module called haslib that provides secure hashes and message digests.
To get the md5 hash of a file all you have to do is this

import hashlib

in_file=open('path/to/file','rb').read()
hashlib.md5(in_file).hexdigest()

Life imitating art ?

With the pending financial crisis in Greece, and with the recognition that our poticians have the *MAJOR* blame for what's coming, i still could not figure out why there are so many negative reports about the Greek economy that makes borrowing even more expensive and thus hindering even more the efforts for a recovery.
Then i remembered one of the dialogs from the movie "Sneakers"

Cosmo: Posit: People think a bank might be financially shaky.
Martin Bishop: Consequence: People start to withdraw their money.
Cosmo: Result: Pretty soon it is financially shaky.
Martin Bishop: Conclusion: You can make banks fail.
Cosmo: Bzzt. I've already done that. Maybe you've heard about a few? Think bigger.
Martin Bishop: Stock market?
Cosmo: Yes.
Martin Bishop: Currency market?
Cosmo: Yes.
Martin Bishop: Commodities market?
Cosmo: Yes.
Martin Bishop: Small countries?

And these come out of movie script back in 1992...
Life imitating art ?

Reverting an SVN commit

Here is a one liner that helps a lot when trying to revert to an old commit after you realized that you have foobared your repository.

svn merge -c -R repository

where -R is the revision number you want to revert to.
What happens is that if R is negative it will be regarded as an inverse merge and the commit will be removed instead of added.

so

svn -c -99 http://svn.somewhere.com/trunk

will revert you trunk to revision 99

P.S Don't forget to commit after the merge if you want the reversal to be permanent

Using SAPI 4/5 voices with text to speech in Asterisk

For a while now i have been working on TTS (text to speech) on Asterisk for a startup i am running with two friends.
One of my biggest problems was that the available voices for Greek in linux are very limited,sounding like robots with a laryngitis problem in some cases.
And from what i understood (or actually hear) with the exception of English most of the other languages had similar problems.
Windows has a large number of commercial "voices" that use the SAPI interface like the AT&T natural voices, Nuance, and Loqundo, but i could not find any solution that would allow me to use them with Asterisk.
Plus i wanted something that could easily scale to hundreds of channels and be low costs or free as in beer.

So i used pyTTS and web2py to create, in essence, a web service that given a string returns a file with the spoken text.
This service runs on a Windows machine and provides a very simple API to allow selection of the language,voice and fine tuning that might be required.
Once the service is invoked it returns a wav file with the spoken text.

A simple python script on the asterisk side provides a library, called ast-SAPI, that handles the communication with the web service and returns a wav file.
it pretty much works the same way as flite when its told to create wav files.

At the moment the ast-sapi gets called by a Python AGI script.
The Python AGI receives the text and settings from the Asterisk dialplan, calls the library and then streams the resulting wav.
Also got an asterisk app build using flite's asterisk app as a template,to do the same from the dialplan and avoid the AGI. It works but there are some issues that need to be addressed.

The benefits of this solution are many.

I get good quality voices to use with asterisk and since this is similar to a web service is easy to scale and spread the load to many machines.

Its still in a prototype stage, needs more work and i already spotted some places where things need to be changed, but overall seems to work.

web2py - Getting the admin panel to work remotely

I have spend sometime looking at the web2py framework the past few days, as i was looking for a python based framework for a project.
My first impression is that it has some nice features but as with all "frameworks" the learning curve can be steep.

After going through the manual over the weekend i decided to take the plunge and install it in a dev machine.
This is a vm machine running Ubuntu server, so there is no local browsing capabilities everything is done over the network.
Got web2py installed, run the server and then tried to access the admin interface from my remote workstation and got this.

Admin is disabled because unsecure channel

According to the documentation the admin and appadmin only work from localhost and remotely via ssl (ssh tunnel or https)
Looking at what it takes to config Apache to test-run web2py i decided i need another simpler solution.

I found this blog post that had instructions on how to get web2py going with a self-signed certificate and gave it go.
(Note that there are some typos in instructions given in the blog, but its easy to figure them out)

So i created the certificates and tried to access web2py from https this time.
No i was getting a strange error in Firefox

SSL received a record that exceeded the maximum permissible length.

(Error code: ssl_error_rx_record_too_long)

The problem was that web2py was still sending http and not https to the browser, so probably the https was not working

I used the command line switches of web2py and provided the exact path to the certificates

python web2py.py -a pass -i 192.168.1.3 -p 8000 -c /etc/ssl/self_signed/server.crt -k /etc/ssl/self_signed/server.key

and got a new message form web2py

WARNING:root:OpenSSL libraries unavailable. SSL is OFF

It turned out that Openssl was installed on the dev machine but the python bindings were not.
so issuing a

sudo aptitude install python-openssl

solved this and remote access to web2py's admin console was available over https.