Well, it’s been almost a month since I’ve started writing about the humbug project. Now, it’s time to actually get you people involved, at least in the initial levels. We are looking to add 10 additional members into the humbug call analytics suite. Currently available analytics during the alpha testing is inbound call analytics.

Our aim is to gather as much information as we can and as much user requests as we can, humbug is a community oriented project, thus it relies on community oriented input and feature requests. Participating members will  be granted access to the humbug analytics portal, allowing them to gather statistical information regarding their inbound call hits and their top ten DID numbers – we are working on additional statistics. As new stats will become available, we’ll role those out into the service as soon as possible.

In order to participate in the closed alpha testing, please send an email to alphatest at humbuglabs.org, and we’ll send you a short piece of dialplan code to insert into your Asterisk server. Technically speaking, we’ll send you a short AGI command that looks like this:

exten => _X.,n,AGI(agi://somehost/DataReceiver,some_unique_ident)

The above line needs to be inserted into any place you would like to generate call analytics from. We’ll also enclose configuration steps for FreePBX (and other FreePBX compatible distributions). We are hard at work for creating a FreePBX integrated module, so you can do a one-click install.

Ok, it goes without saying: “A2Billing is one of the most complete Calling Card systems in the Asterisk market today.” – on the other hand, it is also true that: “A2Billing is one of the most complex and convoluted pieces of code ever written!”.

The combination of the above makes for a fairly combustible mixture, especially if you have a big system. Now, I recently ran into an issue, where PHP was litterally eating up almost 512MB of ram, in order to run the A2Billing reports. In it self, that didn’t make much sense to me. However, after inspecting the code, and realizing that A2Billing uses GD in run-time to generate images out of thousands of CDR records, it made perfect sense that it may just be eating up memory.

So, increasing the memory on PHP to go up to 512MB of RAM helps, but creates an interesting probelm. Whenever Apache will invoke a script, it will automatically consume a shitload of RAM, and for each time I intiate a new query, it will spawn a new Apache instance, and consume the same amount of memory. That said, after 6 queries of 512MB, about 50% of the machines RAM was already eaten up – and Apache will not free it!

At this point, I had 2 choices:

1. Go into the A2Billing code, change the GD code to work right or simply change it completely to something else (maybe flash).
2. Work around the problem with a mix of proper IT practices.

I admit that I hate quite a lot of things (I won’t list these here); however; nothing ranks up the list as modifying someone elses code, when I know for fact that it will be unmaintainable in the future. So, I choose option number 2.

I’ve being playing alot with Lighttpd lately, and got some really nice performance from it. So, I said to myself, this would be a great test to see if Lighttpd+FastCGI can solve the problem here. I had to work my way around lighttpd to do what I wanted and verify that my FastCGI server in Lighttpd doesn’t consume all memory, however, here is what I got working with A2Billing, and really nice.

Step 1: Enable the required modules:

server.modules              = (

                               "mod_access",

                               "mod_auth",

                               "mod_status",

                               "mod_fastcgi",

                               "mod_accesslog" )

Step 2: Enable the FastCGI Server

fastcgi.server             = ( ".php" =>

                               ( "localhost" =>

                                 (

                                   "socket" => "/var/run/lighttpd/php-fastcgi.socket",

                                   "bin-path" => "/usr/bin/php-cgi",

                                   "idle-timeout" => 30,

                                   "max-procs" => 1,

                                   "min-procs" => 1

                                 )

                               )

                            )

Step 3: Modify user permissions (required if you are using FreePBX)

server.username            = "asterisk"

server.groupname           = "asterisk"

Step 4: Setup authentication and authorization (optional)

#### auth module

## read authentication.txt for more info

auth.backend               = "htpasswd"

auth.backend.htpasswd.userfile = "/var/www/.htpasswd"

auth.require               = ( "/" =>

                               (

                                 "method"  => "basic",

                                 "realm"   => "A2Billing Management",

                                 "require" => "valid-user"

                               )

                             )

The above configuration made the interface spwan a single FastCGI, insuring that memory usage is never over utilized. I still need 512MB of RAM to run the scripts, but at least now it’s limited to only 512MB of RAM, out of a machine that has 16GB of RAM.

For a while now I’ve been toying around with the idea of utilizing Lighttpd for various web based applications. One of these application is my Automatic Dialer framework, also known as the GTD-API. The main issue with the GTD-API (besides that it is highly reliant on a MySQL database), is the fact that all requests have to be processed via XML-RPC HTTP post requests.

The main issue that I had was this: in a production scenario, a dialer management system will generate over 100 requests to the XML-RPC server. While Apache is fully capable of rendering services at such a speed, its increasing size and boilerplate automatically introduce a management issue. In addition, as I was trying to build a dialer appliance that can be used in any enterprise, the ever expanding Apache wasn’t a good choice.

While I was looking at both NginX and Lighttpd, the latter captured my eye, thanks to a simple advantage – The integration of FastCGI based PHP was so easy, that it almost troubling that I used Apache all these years.

At this point, once I got Lighhtpd working with my Dialer, I said to myself: “It would be really cool to go about and send status reports back from the dialer, directly to the web client activating the call. In addition, I really don’t want to go about and perform these updates to the database, then query the database – that would, literally, kill the MySQL server.

So, I implemented a local session storage area for each call, which updated the call status as it traverses. The information was stored on the hard drive, which allowed a better response time than the ever indexing MySQL server. The status reports were picked up from the Lighttpd server via an Ajax client (which I didn’t write – I suck at JS) – and it works quite well.

I wonder, can Lighttpd completely replace Apache? … time will tell…

When people hear other people talk about “The Annoying Thing”, most probably the first thing that pops into their mind is the below video:

However, today I’d like to refer to another annoying phenom, the one that I call: “letting the piss get to your head – even if you didn’t do anything or contribute to anything!”.

As I am a consultant, I work with a multitude of customers, each one with their own craziness and weirdness. Now, there is nothing wrong with being weird or crazy, but being out blunt rude and insulting, that’s something completely different. For example, when I walk into a meeting, I have the best interest of my customer in mind, however, when that customer mistakes my willingness to assist and promote his company, mistakes that willingness to being a sucker – that’s annoying.

Why would an IP carrier tell something in the form of: “Haaaa… Are you kidding? they need to pay me for using their product, I’m promoting it!” – that’s plain rude. Fuck man, your founders built your entire company on this product, when they raised VC money and charged money from their customers, the makers of the product (an open source one), didn’t make a dime of from you. That statement is blunt out RUDE!

I think that the best option would be to go about an establish inside the Open Source license model a statement that says: “If the GPL product is used in conjunction with a service, that surpasses the size of X concurrent/registered users, the product shall no longer be considered a GPL product – but a comercially licensed product, requiring the user to pay royalties to the creator”. Why is this a good thing? – very simple, sustainability of Open Source. In one of my previous posts, I talked about the sustainability of Open Source – defining the border when a usage of an Open Source product is considered commercial – and requires a form of royalty payment to the project’s maintainer/creator is a crucial element in making sure the project continues and evolves.

Imagine for example, that the Apache foundation didn’t exist. The foundation supports and funds (partially) the development behind the Apache web server project and related projects. The project is capable of sustaining itself, as companies which are making commercial usage of Apache are taking an active part in the funding of the project. With the Apache project, there was no need for a definition of “Commerical Use”, as the Apache project started from a Commercial need. Other projects start from non-Commercial needs, but evolve into Commercial products – that case needs to be addressed.
 

There is nothing more annoying than an incomplete specification design, brought to you as a design document. Why does every person that is capable of operating a word processor at a beginner level and operating Visio at a level of a child, believe that they are able to produce a proper design specification or ever a proper requirements document?

I had just received a so-called “specification document” from a prospective customer, that had specified the required system as “a pre-paid IVR system for premium services”, that was accompanied by a half page flow chart that had 4 blocks on it, without any description of what the system does. Now, I called the customer, explaining to him that the document isn’t complete, and I’d like to meet with him in order to write a proper document – the following conversation took place:

Customer: I don’t think a meeting is needed, if you are too stupid to understand what I want – then I don’t think I want to work with you!.

Me: I’m very sorry that you think that I’m not good enough to carry your project, you are welcome to take your business to someone else who is more capable than I am.

Customer: Well, the thing is that you are my last hope, everyone said you can develop anything. I sent this document to a few companies, they all returned price quotes that were too high.

Me: They returned a high price quote because they have no idea what you want, so they gave you a price quote so high, so you would back off and not call them again. I want to help you do the project, but in order to do it, I need you to define your requirements better.

Customer: Are you familiar with the functionality of system X?

Me: Yes, that is a fairly complex system – development time is around 6-8 weeks, at minimum.

Customer: No, I need it working within 10 days, I already lost 6 weeks of getting price quotes.

Me: I’m sorry, but in this case, I can’t really help you, your requirements and time considerations are impractical.

At this point, the customer had started screaming on the phone – which was a point when I had bid the customer farewell and had disconnected the phone call, as I never conduct my business in a manner when people are screaming without listening to what they are shouting.

Now you tell me, was I logical in my response? or maybe the customer is right? why do customers have the impression that building complex systems is a matter of simply installing the computer, slapping a few pieces of code on the machine, test it a few times – and walla, a running system is ready for production?

Call me old fashioned, but I believe that each development should be backed by a proper specification design, a design document, implementation design, regression testing documents and acceptance tests – if you don’t agree, I’d love to hear about it.