In my previous post I’ve announced the bootstrapping of a new PHP project, called “Project Stanley”. Project Stanley was an attempt at creating a Asterisk ARI developer kit, based upon the PHP programming language (yes, I call it a programming language).

Shortly after initiating the development and reaching a point where our code was actually able to do something, we realized that we’ve gone the wrong way. The wire frame we’ve created relied heavily on the Ellislabs CodeIgniter MVC framework. Now, don’t get me wrong, I love CodeIgniter – but, it was the wrong choice. It was wrong, because we were locking our developers into an MVC structure, that truly isn’t needed for something like this.

So, we’ve stopped working on Project Stanley (you can still find it on github if you really want to) and we migrated the code into the PHPARI project. PHPARI is a cleaner approach to providing a simple, to the point, ARI developer kit using PHP. It relies on 2 PHP external libraries – PHPWS and PEST.

PHPWS is a WebSocket client implementation in PHP, while PEST is a REST client implemented in PHP. Both are actively maintained and had been tested by multiple projects as stable and battle tested. We’ve also enabled PHPARI in packagist, you can look it up for installation. Make sure you use the dev-master part of the package, not the dev-develop – it’s unstable and may actually contain broken code.

Asterisk ARI – for a seasoned AGI/AMI developer like myself, ARI is a serious mind warp. Why is it a mind warp? simple, it’s all the things we wanted AGI to be, and the reliability we wanted AMI to have, minus all the work around we needed to do – in order to get similar functionality in the past.

So, is ARI truly a replacement for AGI/AMI? well… I think the true answer will be NO. Is a replacement for the Asterisk dialplan? well… I think the answer to that is NO as well. “Say, are you messed in the head? first you say “What AGI/AMI should have been”, and then you say it’s not a replacement? – are you mental?” – well, there are a few reasons why I claim it’s not a direct replacement, and I’ll detail these here.

In order to explain, I’ll give a few examples, using the “in-development” PHP ARI wireframe that I’m developing, called Stanley.

Synchronous vs. Asynchronous

ARI by definition is asynchronous. Keeping that in mind, in means that that any command you give it will get queued or spooled in some manner, and return back an immediate result. Just to illustrate it, let’s examine the following code segment:

$this->stasisLogger->notice("Stasis Start");
$lastResult = $this->channels->channel_playback($this->ari_endpoint, $messageData->channel->id, "sound:hello-world");
$this->stasisLogger->notice("Last result: " . $lastResult);
$lastResult = $this->channels->channel_playback($this->ari_endpoint, $messageData->channel->id, "sound:demo-congrats");
$this->stasisLogger->notice("Last result: " . $lastResult);

For all practical purposes, you should regard $this->stasisLogger as a simple logging object, and $this->channels as a model to initiate ARI Channel requests. If you use the above the code, and activate it from with a Stasis application, you would listen to the “hello-world” and “demo-congrats” segments. Now, let us examine the following code segment:

$this->stasisLogger->notice("Stasis Start");
$lastResult = $this->channels->channel_playback($this->ari_endpoint, $messageData->channel->id, "sound:hello-world");
$this->stasisLogger->notice("Last result: " . $lastResult);
$lastResult = $this->channels->channel_playback($this->ari_endpoint, $messageData->channel->id, "sound:demo-congrats");
$this->stasisLogger->notice("Last result: " . $lastResult);
$this->channels->channel_delete($this->ari_endpoint, $messageData->channel->id);

The only difference here is the last line. If you activate this code, you will hear the world “Hello”, immediately followed by a disconnect. “Wait a minute, what just happened? – wasn’t I supposed to hear everything?” – that’s exactly the point, the answer is NO! The asynch nature of ARI will simply queue the first 2 playback requests, while the hangup is performed almost immediately – the playback simply never get to be executed.

In other words, if you need something to be synchronous within the dialplan, you may need to work differently about it. If you are familiar with the Node.JS framework, you are fairly familiar with this issue.

ARI is for writing applications, not IVRs

When the Asterisk team created ARI, their idea was simple: “Don’t manage the queue application, simply write your own”. Same applies for managing multi party conference calls, call origination, etc. In 2009 I wrote a book about AGI programming, where I’ve explained the methodology for “Atomic AGI development“. The concept behind Atomic AGI was to contain small logic units in AGI scripts, and leave most of the heavy lifting to the dialplan. This methodology enables to create scaleable Asterisk platforms at fair ease, and introduce additional technologies, without going about and adding odd things into Asterisk.

ARI is meant to do something similar, in the form where you can go about and create your own logic, contain it into a singular application and activate when you require – for example, rewriting the queue application. One of the first applications that I’ve decided to re-write using ARI was a Radio broadcasting system that I’ve developed in 2006. The problem with that application was that I need to hold about 600 callers in a single queue, and attach them over to the broadcasting booth as required. Of course I needed to enable full call control, caller management, UI and more. Initially, I used MeetMe, MySQL, and AMI to do this. Later on it changed to MeetMe, Redis, AstManProxy and some other tools – but it never seemed to please me. The fact that I needed to maintain 2 MeetMe bridges, one for holding people and one for the actual broadcasting really bugged me. Yes, when Asterisk 1.8 came out I migrated to the Bridge application and yes, I updated bits and pieces here and there, but it was never what I wanted it to be.

When I started playing around with ARI, I said to myself – this is the perfect application to migrate to ARI. The only thing I needed was a simple Stasis application to read my state correctly, and that would be activated once the called is put into the waiting area – so in terms, I’ve developed a very simple queue application.

IVR heavy lifting was done using dialplan, but the actual service was done with ARI.

Blades and Bleeding Edge

Now, before you go about migrating all your existing code to ARI – you must remember this: If you walk on the bleeding edge, expect the blade to cut you here and there. Currently, I hadn’t yet seen any proper ARI wireframe available. I’ve seen some work done with Node.JS and Ruby, but I can’t say that I’ve taken a fancy to any of those. Honestly, my comfort zone is very much PHP and C/C++, what can I say, I’m old school.

When I started building the Stanley wireframe, it was fairly frustrating – simply because not everything was that much clear and clean. In addition, as Asterisk advances, ARI will change and advance as well. What ever you write, make sure it’s modular enough so you can change it as required.

 

Well, it’s been almost a month since I’ve started writing about the humbug project. Now, it’s time to actually get you people involved, at least in the initial levels. We are looking to add 10 additional members into the humbug call analytics suite. Currently available analytics during the alpha testing is inbound call analytics.

Our aim is to gather as much information as we can and as much user requests as we can, humbug is a community oriented project, thus it relies on community oriented input and feature requests. Participating members will  be granted access to the humbug analytics portal, allowing them to gather statistical information regarding their inbound call hits and their top ten DID numbers – we are working on additional statistics. As new stats will become available, we’ll role those out into the service as soon as possible.

In order to participate in the closed alpha testing, please send an email to alphatest at humbuglabs.org, and we’ll send you a short piece of dialplan code to insert into your Asterisk server. Technically speaking, we’ll send you a short AGI command that looks like this:

exten => _X.,n,AGI(agi://somehost/DataReceiver,some_unique_ident)

The above line needs to be inserted into any place you would like to generate call analytics from. We’ll also enclose configuration steps for FreePBX (and other FreePBX compatible distributions). We are hard at work for creating a FreePBX integrated module, so you can do a one-click install.

Reblog this post [with Zemanta]

Ok, it goes without saying: “A2Billing is one of the most complete Calling Card systems in the Asterisk market today.” – on the other hand, it is also true that: “A2Billing is one of the most complex and convoluted pieces of code ever written!”.

The combination of the above makes for a fairly combustible mixture, especially if you have a big system. Now, I recently ran into an issue, where PHP was litterally eating up almost 512MB of ram, in order to run the A2Billing reports. In it self, that didn’t make much sense to me. However, after inspecting the code, and realizing that A2Billing uses GD in run-time to generate images out of thousands of CDR records, it made perfect sense that it may just be eating up memory.

So, increasing the memory on PHP to go up to 512MB of RAM helps, but creates an interesting probelm. Whenever Apache will invoke a script, it will automatically consume a shitload of RAM, and for each time I intiate a new query, it will spawn a new Apache instance, and consume the same amount of memory. That said, after 6 queries of 512MB, about 50% of the machines RAM was already eaten up – and Apache will not free it!

At this point, I had 2 choices:

  1. Go into the A2Billing code, change the GD code to work right or simply change it completely to something else (maybe flash).
  2. Work around the problem with a mix of proper IT practices.

I admit that I hate quite a lot of things (I won’t list these here); however; nothing ranks up the list as modifying someone elses code, when I know for fact that it will be unmaintainable in the future. So, I choose option number 2.

I’ve being playing alot with Lighttpd lately, and got some really nice performance from it. So, I said to myself, this would be a great test to see if Lighttpd+FastCGI can solve the problem here. I had to work my way around lighttpd to do what I wanted and verify that my FastCGI server in Lighttpd doesn’t consume all memory, however, here is what I got working with A2Billing, and really nice.

Step 1: Enable the required modules:

server.modules              = (
                               "mod_access",
                               "mod_auth",
                               "mod_status",
                               "mod_fastcgi",
                               "mod_accesslog" )

Step 2: Enable the FastCGI Server

fastcgi.server             = ( ".php" =>
                               ( "localhost" =>
                                 (
                                   "socket" => "/var/run/lighttpd/php-fastcgi.socket",
                                   "bin-path" => "/usr/bin/php-cgi",
                                   "idle-timeout" => 30,
                                   "max-procs" => 1,
                                   "min-procs" => 1
                                 )
                               )
                            )

Step 3: Modify user permissions (required if you are using FreePBX)

server.username            = "asterisk"
server.groupname           = "asterisk"

Step 4: Setup authentication and authorization (optional)

#### auth module
## read authentication.txt for more info
auth.backend               = "htpasswd"
auth.backend.htpasswd.userfile = "/var/www/.htpasswd"
auth.require               = ( "/" =>
                               (
                                 "method"  => "basic",
                                 "realm"   => "A2Billing Management",
                                 "require" => "valid-user"
                               )
                             )

The above configuration made the interface spwan a single FastCGI, insuring that memory usage is never over utilized. I still need 512MB of RAM to run the scripts, but at least now it’s limited to only 512MB of RAM, out of a machine that has 16GB of RAM.

For a while now I’ve been toying around with the idea of utilizing Lighttpd for various web based applications. One of these application is my Automatic Dialer framework, also known as the GTD-API. The main issue with the GTD-API (besides that it is highly reliant on a MySQL database), is the fact that all requests have to be processed via XML-RPC HTTP post requests.

The main issue that I had was this: in a production scenario, a dialer management system will generate over 100 requests to the XML-RPC server. While Apache is fully capable of rendering services at such a speed, its increasing size and boilerplate automatically introduce a management issue. In addition, as I was trying to build a dialer appliance that can be used in any enterprise, the ever expanding Apache wasn’t a good choice.

While I was looking at both NginX and Lighttpd, the latter captured my eye, thanks to a simple advantage – The integration of FastCGI based PHP was so easy, that it almost troubling that I used Apache all these years.

At this point, once I got Lighhtpd working with my Dialer, I said to myself: “It would be really cool to go about and send status reports back from the dialer, directly to the web client activating the call. In addition, I really don’t want to go about and perform these updates to the database, then query the database – that would, literally, kill the MySQL server.

So, I implemented a local session storage area for each call, which updated the call status as it traverses. The information was stored on the hard drive, which allowed a better response time than the ever indexing MySQL server. The status reports were picked up from the Lighttpd server via an Ajax client (which I didn’t write – I suck at JS) – and it works quite well.

I wonder, can Lighttpd completely replace Apache? … time will tell…