18 Feb 09 Asterisk AGI Programming – New Book

Well, it’s finally out – my new book that is

Some of you already know, but over the past year I’ve been busy writing a new book. This time it’s a book for Asterisk developers, especially tailored to PHP developers wishing to utilizing the PHPAGI framework. The book is out from Packt Publishing (Like my old AsteriskNOW book) and is updated with all the recent changes in Asterisk – including version 1.6.X and DAHDI.

If you like my work with Asterisk and would like to read more of my work, go ahead and get an electronic version of this book. I know it’s a little self promoting, by hey, it never hurts does it?

I’ve included a chapter on how to build a complete project from scratch, detailing the various analysis steps and various paradigms required to develop a fully functional Asterisk based application. I believe that even experienced Asterisk developers will benefit from this book.

Tags: AGI, Asterisk, Asterisk Book, Asterisk Developers, Book, GPL, open source, Packt Publishing, php, PHPAGI

21 Jan 09 Thoughts of virtualization – Part III

Ok, it’s been almost 24 hours since I started my serious playing around with Amazon EC2, and I can honestly say that I’m tired – however, I’m very pleased with my results. Like any other experiment, this one started with a requirement. The requirement was to install and operate one of the dialer frameworks I’ve written in the past year on an EC2 based instance. In order to evaluate the process, let’s start with our baseline installation, meaning, what am I using in the real-world:

Hardware and Software Specification

The original machine answered to the following specification: Quad Core CPU, 2GB of RAM, 250GB of Hard Drive. My original machine was running CentOS 5.2 with an x86_64 kernel installed. In terms of software installed, we had Asterisk 1.4.22.1, MySQL 5.X, PHP, FreePBX, Apache and my dialer framework.

Introducing Amazon EC2 AMI

An EC2 AMI is basically an image of a computer, contained with a single installation manifest on the Amazon cloud computing system. AMI’s provide for the simplest manner to start using EC2, as these usually include a pre-defined server installation, that usually has some stuff already installed.

Choosing your AMI

Amazon provides a multitude of AMI’s to work with, unfortunately, most of these are either out-dated and the sheer number of these makes the choice somewhat overwhelming. I decided to start working with a working AMI image of Fedora Core 8, that already had the LAMP stack installed – the one I used was:

The YUM Repository issue

For some strange reason, the AMI images contained in the Amazon repository are all unable to perform any updates to their installed RPM packages. It took me a while to understand what’s wrong, but in general, the fedora project had simply removed the old releases from their repository, so I had to go in and manually modify the /etc/yum.repos.d/ configuration files. For you convenience, here is the repos list that I’m using at this point:

YUM Repositories for Amazon EC2 Fedora images <- click this to download the file

[development]
name=Fedora Core $releasever - Development Tree
baseurl=http://archives.fedoraproject.org/pub/archive/fedora/linux/core/development/$basearch/
enabled=1
gpgcheck=0

[extras-development]
name=Fedora Extras $releasever - Development Tree
baseurl=http://archives.fedoraproject.org/pub/archive/fedora/linux/extras/development/$basearch/
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-extras
gpgcheck=0

[extras]
name=Fedora Extras $releasever - $basearch
baseurl=http://archives.fedoraproject.org/pub/archive/fedora/linux/extras/$releasever/$basearch/
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-extras
gpgcheck=1

[updates-testing]
name=Fedora Core $releasever - $basearch - Test Updates
baseurl=http://archives.fedoraproject.org/pub/archive/fedora/linux/core/updates/testing/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-test

[updates-released]
name=Fedora Core $releasever - $basearch - Released Updates
baseurl=http://archives.fedoraproject.org/pub/archive/fedora/linux/core/updates/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora

[base]
name=Fedora Core $releasever - $basearch - Base
baseurl=http://archives.fedoraproject.org/pub/archive/fedora/linux/core/$releasever/$basearch/os/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora

If you are trying to install an instance, you may use the above for any Fedora based AMI on EC2.

Creating my own AMI

While going about and building a single server is fun, I needed a way to create my own installation AMI once I’ve completed my modifications. The Amazon EC2 resources website gives out a whole lot of information, which can be somewhat confusing for the first time reader. So, I searched for a better way to construct my own AMI image. I found the following web page, which was really really helpful: Azeez’s Notes.

Azeez’s blog gives a very concise and to the point, step by step, guide to building your own AMI image – it got me up and running in less than 10 minute – WAY TO GO AZEEZ.

So far, so good …

So far my installed instances are working just fine and I’m currently operating a cluster of a few of these dialer systems. I’m in the process of checking what kind of mileage I’ll get from the EC2 instances, in comparison to a real hardware server – which is really interesting.

My main objective here is to be able to build a dialer-on-demand framework, which will enable my customers to increase their capacity utilizing a simple web interface to initiate my AMI instances. I’ll report back with additional information once I have it.

Tags: Amazon, Asterisk, Cloud computing, dialer, EC2, Virtualization, VMWARE

14 Oct 08 A2Billing+Apache vs. A2Billing+Lighttpd

Ok, it goes without saying: “A2Billing is one of the most complete Calling Card systems in the Asterisk market today.” – on the other hand, it is also true that: “A2Billing is one of the most complex and convoluted pieces of code ever written!”.

The combination of the above makes for a fairly combustible mixture, especially if you have a big system. Now, I recently ran into an issue, where PHP was litterally eating up almost 512MB of ram, in order to run the A2Billing reports. In it self, that didn’t make much sense to me. However, after inspecting the code, and realizing that A2Billing uses GD in run-time to generate images out of thousands of CDR records, it made perfect sense that it may just be eating up memory.

So, increasing the memory on PHP to go up to 512MB of RAM helps, but creates an interesting probelm. Whenever Apache will invoke a script, it will automatically consume a shitload of RAM, and for each time I intiate a new query, it will spawn a new Apache instance, and consume the same amount of memory. That said, after 6 queries of 512MB, about 50% of the machines RAM was already eaten up – and Apache will not free it!

At this point, I had 2 choices:

1. Go into the A2Billing code, change the GD code to work right or simply change it completely to something else (maybe flash).
2. Work around the problem with a mix of proper IT practices.

I admit that I hate quite a lot of things (I won’t list these here); however; nothing ranks up the list as modifying someone elses code, when I know for fact that it will be unmaintainable in the future. So, I choose option number 2.

I’ve being playing alot with Lighttpd lately, and got some really nice performance from it. So, I said to myself, this would be a great test to see if Lighttpd+FastCGI can solve the problem here. I had to work my way around lighttpd to do what I wanted and verify that my FastCGI server in Lighttpd doesn’t consume all memory, however, here is what I got working with A2Billing, and really nice.

Step 1: Enable the required modules:

server.modules              = (

                               "mod_access",

                               "mod_auth",

                               "mod_status",

                               "mod_fastcgi",

                               "mod_accesslog" )

Step 2: Enable the FastCGI Server

fastcgi.server             = ( ".php" =>

                               ( "localhost" =>

                                 (

                                   "socket" => "/var/run/lighttpd/php-fastcgi.socket",

                                   "bin-path" => "/usr/bin/php-cgi",

                                   "idle-timeout" => 30,

                                   "max-procs" => 1,

                                   "min-procs" => 1

                                 )

                               )

                            )

Step 3: Modify user permissions (required if you are using FreePBX)

server.username            = "asterisk"

server.groupname           = "asterisk"

Step 4: Setup authentication and authorization (optional)

#### auth module

## read authentication.txt for more info

auth.backend               = "htpasswd"

auth.backend.htpasswd.userfile = "/var/www/.htpasswd"

auth.require               = ( "/" =>

                               (

                                 "method"  => "basic",

                                 "realm"   => "A2Billing Management",

                                 "require" => "valid-user"

                               )

                             )

The above configuration made the interface spwan a single FastCGI, insuring that memory usage is never over utilized. I still need 512MB of RAM to run the scripts, but at least now it’s limited to only 512MB of RAM, out of a machine that has 16GB of RAM.

Tags: A2Billing, Apache, Asterisk, Authentication, Authorization, errord, FreePBX, htaccess, Lighttpd, memory, php

11 Oct 08 We’re melting, we’re melting…

I’m not much of an economist. I understand income and expenses, I understand more or less how the stock market works from time to time, but I’m no economist. I’ve been in the hi-tech industry since early 1997, which means, that I’ve seen this world being elevated to the level of godly hood, buried alive with the explosion of the dot.boom era, only to re-invent itself as a highly competitive telecom’s industry and the “Network is the Computer” manner of thinking.

About 2 weeks ago, a man I truely admire had written a post entitled: Economic Meltdown – Friend or Foe of Open Source? (written by Danny Windham, CEO of Digium). Danny talks about the various aspects of companies cutting down on their IT budgets and reversion to Open Source technologies, as a means to hedge needed cash. As I witnessed the Dot.Boom explosion of 2001, I can honestly say that 2001, at least as far as I can say, was the year that Open Source technologies had truly assimilated to the normal business practice of companies. Even beyond that, companies which were not based on Open Source and Linux/Unix based technologies were clearly not a valid option for investors.

One of the urban legends talks about Hotmail wishing to acquire an Israeli company called Commtouch, back then, a webmail company. Commtouch was purely based on Windows NT technologies back then, and while the business track of the negotiations were going fine, the deal collapsed as the technical sides, of a back then BSD based Hotmail, basically negated the deal. Now, I have no idea if the story is true or not, again, this is an urban myth, however, I believe myths are usually based on some portion of the truth - at least to an extent (even a small one).

While the adaptation of Open Source technologies is a must for companies, in order to survive the troubled waters of these time, it is highly dependant on the solution providers, consultants and IT outsourcing companies. In Israel, for example, it is very much a customary to hire outsourced IT assistance, on a part time basis, allowing you to cut your IT costs. When bringing in IT assistance, you’re not only bringing in IT assistance, you also bring in IT governance and IT policies – as an outsourced IT person will usually operate under the terms he had learned in his company. If his company is a M$ oriented company, no matter how Open Source will be crucial to the business, M$ will still rule the domain in that area. I believe that the first thing that needs to happen is for the consulting and IT outsourcing companies to realize that they need to change, first for their customers, then for themselves. If the small start-up companies and established companies will start falling, the consultants will simply dry out of work.

It is our responsibility as IT/Telecom/Development/Architecture/Design consultants to explain the aspects of using Open Source, assimilating it, integrating it, both the pros and the cons and then, assist our customers in doing so successfully – this is the only way to go.

For example, let us take examine the following scenario – a telephony service provider. Our telephony service provider is based on Asterisk as the switching environment, a purely M$ base for all the billing, web front-ends, databases, development and so on. Their entire IT infrastructure is fully owned by them, all running with ESX servers and other proprietary technologies. This company will be required to pay hundreds of thousands of dollars every year, for support purposes (and the year is coming close to an end), their expansion is highly dependant on 3rd parties and while they are self sufficient on the IT side, they are purely dependant on the technology side. Now, imagine that this company would have had their entire back-end based on Open Source technologies, let’s imagine PostgreSQL and Ruby on Rails as the web frame work, how much money will that save per year? now, let’s remove ESX out of the equation… let’s also remove non-Open storage solutions and replace those with CoRAID or something similar, we’ve saved hundreds of thousands of dollars on an annual basis. Radical? – YES! But drastic times call for drastic measures, and when drastic measures are required, Radical solutions are a must.

Tags: Asterisk, danny windham, digium, economy, ESX, meltdown, Microsoft, NASDAQ, Proprietary, VMWARE

15 Sep 08 Dialers – Myth or Truth?

Over the past 12 months, I’ve been heavily involved in the development of high-speed dialers. While many companies published a prolific number of automatic dialers (power, predictive, broadcast) – none of these companies ever announced there top speed dialing capability.

An so I ask myself: “What is the reason not to release these numbers publicly?” – interesting, isn’t it. So, I decided to experiment myself and see if I would release public numbers for my dialer. I’ve designed my dialer to be capable of generating upto 35 calls concurrently. My questions was this: “While the dialer is fully capable these numbers in a test scenario, will this number be reached in real life?”

So, I started experimenting with my code – doing a very simple test. I’ve installed my dialer on a Dual Quad Core system, 8GB RAM, and a RAID-1 array as the storage. Now, I’ve setup the system to generate up to 35 calls every second, allowing the server to sustain a total of 180 concurrent calls – 360 channels.

What happend was actually this: the dialer generated about 180 dials within a period of about 8 seconds, and then, simply waited till some of the calls completed in order to generate additional calls. Ok, that is understandable, however, what is the throughput of the dialer? So, I decided to do another test, I filled up my queue with a total of 50,000 records. In addition, I’ve arranged with my carrier to terminate the calls to on of their SigValue systems, to sustain the enormous number of inbounds. In addition, I asked them to perform a small measurement of the average call inits per second. I was shocked with the result!

While the dialer simply peaked upon startup, after 20 seconds of operations it started normalizing at around 6 call initiations per second. But how can that be? how can it be that a dialer capable of dialing 35 calls is slowed down to crawl? – the reason is simple, the latency imposed by the PSTN/VoIP network, the time it takes for the calls to terminate and the actual time the call is left on the air, while the dial happens. All these factors together had given me the notion that there is no practical top speed, as it is in direct relation to the demographics of the dialer operations.

Having said that, is it possible to devise a formula to calcualte this number on a per demographic basis? maybe on a per PSTN/VoIP carrier type? maybe a combination of the both? will a formula such as that will enable for the better creation of broadcast/predictive dialer? – no use dialing tons of number into a demographic that can’t handle it, while at the other side of the spectrum, what demographics will benifit from the utilization of an automated dialer?

Tags: Asterisk, demographic, dialer, predictive, PSTN, vicidial, VoIP