As some of you know, over the past 9 months, I’ve been heavily involved in the establishment of Humbug. For those who may not know, Humbug is a Call Analytics and Fraud Analysis SAAS. Now, differing from many of the current telephony SAAS projects, we are not based on Amazon EC2 or some other public cloud infrastructure, we build our own cloud environment. Why do we build our own cloud? simple, we need to keep your data secured and confidential. At Humbug, we see ourselves as a cross between Google Analytics – in our ability to analyze and handle data and Verisign – in our security and confidentiality requirements and methodologies.

Question be asked, why do people trust Verisign to provide SSL certificates around the world. What makes Verisign’s CA better than a privately owned CA – the answer is simple, it’s a third party 2 entities can entrust at the same time. Humbug aims to provide the same lever of trust, simply because we regard your data as sacred and valuable.

Since about 2 months ago, we’ve been contacting various Asterisk integrators around the world, inviting them to evaluate Humbug services. Now, while some integrators and vendors were somewhat reluctant, others were more than happy to join. We now have over 250 monitored systems around the world, with system being monitored and analyzed in Israel, USA, UK, Brazil and more.

The thing that amazed me in regards to some of the integrators who decided not to participate was that they claimed: “we provide our customers our own brew of fraud analysis service, we don’t require your SAAS”. Now, while I can accept the fact that an integrator would offer such a SAAS as an in-house service, I can’t see why a customer would rely on these services. In my view, relying on your integrator to provide fraud analysis services is like relying on the integrator of your alarm system to provide hired guard services – it just doesn’t make any sense to me. Why doesn’t it make sense? in Hebrew we say: “Go prove that you have a sister”. Imagine that your PBX integrator offer you such a service, then, in some obscure manner, your PBX gets hijacked and you get slammed with 50K$ worth of phone calls to Somalia. Now, your integrator would say: “Hmmmmm… that’s odd, we didn’t even get those CDR events to our system… you really got hacked bad…” – sure, if you only rely on CDR records to do your analysis (which is what 99.9% of integrators do). There is much much much much more to fraud analysis than just CDR analysis – if it all began and finished with CDR analysis, then by far Cvidya, Verint, NICE and many others would have been made redundant.

Allowing your integrator to provide you with fraud analysis SAAS is like putting the fox to guard the hen house, when things louse up (and they may), he’s the first one to bail out saying: “It’s not my fault”.

Humbug takes a totally different approach to fraud analysis, specifically, in the way we regards the various PBX systems and integrators. We are vendor agnostic and integrator agnostic – we will provide you with the clear and concise information you require in order to make an educated decision as to how you were de-frauded (if de-frauded) and provide you a faster alerting and response time. Our recent adventures had lowered our fraud alert response time from 60 minutes, down to 14 minutes in some cases. Most fraud analysis system carry a 24-36 hour turn around time, by that time, you can be out of 50K$ – our aim is to lower that number to no more than a 100$ in the worst case. Ambitious? yes, down right crazy? probably so, but we always say: “Aim for the moon, you’ll land on a star!” – so we know we’ll get there.

The following post doesn’t really fit in line with the normal spirit of the blog, simply because it’s not funny nor directly related to technology. It’s called Business 2.0, as it relates to the ever problematic question any business owner has: “When should I grow and how?”.

As you may know, I’ve been a freelance Asterisk Platform developer since early 2003, turned to freelance development (Penguin for hire) around April 2007. Since that time, I’ve built systems and platforms for some of the better known brands around the world. Be it working directly with the customer or through a 3rd party (as a sub contractor) – I can easily say that I’ve completed over 120 different large scale projects within 3 years time. Now, when I refer to projects, I’m not referring to installing PBX systems, I don’t do that at all – I’m referring to highly complex application level development, creating some of the most innovative Asterisk based systems I’ve ever seen.

Image representing Jajah as depicted in CrunchBase

Image via CrunchBase


Image via Wikipedia

Putting aside everything, finalizing a rough estimate of 40 development projects on a yearly base, most of these performed solely by myself is a fairly challenging task. Sure, at times I’ll outsource some work to other freelancers like myself, specifically in fields where I’m not all that fluent (Database, Web Development, UI) – but yet, doing that means that I’m conducting 3 – 5 projects on a monthly basis.

After doing so for 3 years now, I can’t help but start thinking about expanding my business, taking it to the next level by hiring more people and building it up to a new level. Question remains for this: “How? What is the natural track of expanding your business?” – of course the simple answer would be: “Just hire another developer or two, and start doing more sales” – it’s not as simple as it sounds. After thinking about it for some time, I’ve concluded there are a few models of expansion:

Model 1: Organic Growth

Organic growth can be described as the simplest way of growth: “Hire a new guy and get more work in”. The problem with this model that it is fully reliant on your ability to sell more. However, as you concentrate on sales more, you take time from the development and delivery process – thus, the addition of the new developer is not a 100% addition, it’s actually 100% (developer) minus 40% (you) – so you are not at 200% capacity, you are 160% capacity. Surely 160 is 100, however, for the initial 6 months, till the guy learns the ropes, you are not at 160, you are actually at 80 – can you and your business sustain that?

Thus, the main issue with Organic growth is cash flow, can your business sustain the elevated expenses with less income for the period of transition? If the answer is NO, then you need a different method. If the answer is YES, then you are in the best place in the world, however, bear in mind that taking someone to work for you is a responsibility – people are not resources, they are human beings, with families and children – taking someone to work for you is like taking responsibility for their lives.

Model 2: The Partner

Panama Business and Investment
Image by thinkpanama via Flickr

A partnership with a person who is equally matched to you is always a good option. Technically speaking, it means that you are teaming with someone who generates as much work as you do and is capable of finalizing the work as fast and as good as you can. Yet, taking a partner doesn’t negate the requirement for a new employee or two. In this case, you may end up with too much sales with too little staff to deliver – that is a big problem.

Another issue with partners is the issue of trust. While most partners tend to rely on each other and trust each other, that trust can easily be broken (in most cases by stupid things). It’s enough for one partner to now carry its weight in sales/development to initiate a chain reaction, shortly ending in the partnership dissolving.

So, the partner is a good option, however, may prove to be problematic if the wrong partner is chosen – in addition, dissolving a partnership solely on these issues isn’t all that simple – and usually ends up in litigation and other judicial issues – YUCK!

Model 3: Un-intrusive Angel

Some people ragard Un-intrusive Angels as “Stupid Money” – an Angel investor that doesn’t interfere in your company business model and operations. In many cases, this is how start-up companies start – someone gives them a lump sum of money to start their business, signing off to own a portion of the new company.

An un-intrusive investor usually gives you the money and pays you a visit once every few months to see how his money is spent. Don’t expect to raise a whole lot from these people, usually you will get anything from 25K$ to around 250K$ – tops. If you are getting an investment from an Angel, make sure you plan your business carefully – and make sure your investor knows what he is getting into. The Angel is not a found piggy bank, he is a business man looking for profit – if you make sure his expectations of profits (time frame, amount, percentage, etc) are kept within the reason of your business – he will make an educated decision and invest accordingly. Promises like: “you’ll double your money in 3 years” are stupid – make sure it’s realistic and to the point. If you promise the moon, and reach a star – that’s a problem, if you promise the skies and hit a start – that’s wonderful.

Model 4: The Strategic-Intrusive Angel

Jeff pulver

Image by TheFemGeek via Flickr

A strategic angel is similar to the previous one in terms of funds, however, he is more capable in assisting your business meet its goals. Usually, it would be someone who is already a well established figure in your business sector, had made his money from previous companies and is now looking for new ideas and businesses. I call him an intrusive Angel, as sometimes he may have ideas as to where your business should go – and he will make sure you hear his ideas. You may regard it as annoying, but you should still listen to your Angel and pay him the respect he deserves.

Sometimes this Angel may invest in your business due to the fact that he has a hidden agenda. An agenda can be: The angel looks at your business and see a certain potential you are not planning, he’ll invest and try to re-direct your company to the agenda he sees. This is usually the case when your angel is invested into several endeavours that is either parallel to each other or may have orthogonal intersection points. These angels can be the builders of your business or the destroyers, it is up to you to make sure the latter doesn’t happen.


So, which model did I choose? – I didn’t choose yet, I’m still figuring it out myself. What ever the model may be, the choice isn’t simple nor straight forward. At best, whatever choice I’ll take will have a profound impact on my business and me – so I’ll need to weigh my options carefully. If you can think of an additional model, I’d love to hear about it – so just comment on this post.

Reblog this post [with Zemanta]

When most of us think about PBX systems, we usually associate these with cumbersome usage, confusing dialing codes and in most cases – a PBX system is automatically associated with the annoying task of transferring a call from one handset to another. Lately, I’ve been thinking deeply about how people use PBX systems, is this really the only way to use a PBX system? is there something else to the mix? can we really enrich one of the oldest operational paradigms in the world? – and for that matter, can the public be re-educated to assimilate a new breed of PBX systems or services?

Hardware-based IP phone
Image via Wikipedia

As to answering the question of re-educating the public, I guess I’ll have to leave that question to the head shrinks. As to answering the latter, enriching the PBX experience is both achievable and advisable. When I say enriching, I mainly talk about your ability to bring to the IP phone functionality usually not associated with it. Imagine to have the ability to receive a stock exchange RSS feed to your phones idle screen, notice that you stock is either rising or falling, and by the flick of a button – either sell or buy. We’ve all come accustomed to IP phones that look like the one of the right. A whole bunch of buttons, that in most cases have no direct use when our phone is utilized using a single account. However, these buttons can be externally re-assigned and re-programmed to achieve greater functionality – surpassing the normal behavior of just making phone calls.

The technology involved exists on almost every high-end IP phone on the market (well, at least those made by SNOM, Aastra, Cisco and Polycom – most of the Chinese makers don’t have this) – it’s called a Mini Browsers. Mini Browsers are exactly what they are called, these are simplified versions of your typical Internet browser. Some vendors had produced their own XML based Mini browser markup language (SNOM, Cisco, Aastra) while others had decided to provide a sub-set of XHTML (Polycom). The variations between the vendors are at the neck deep of the problems of using Mini Browsers, and that is that the formats are considerably different. Sure, SNOM had more or less adopted Cisco’s general structure, however, it still varies.

Through the utilization of this technology, it is possible to create phone based browser applications, that seem native to the phone user, as the general interface resembles the native phone interface. It is now the developers job to make the web interface displayed to the user as seamless and as native as possible, keeping in mind that the developer must remain agnostic to the information retrieval layer. Most companies leave their phone systems and these tasks to their system administrators and infrastructure team, however, this task is far beyond their capabilities and skill set. Creating an agnostic IP phone minibrowser dislplay layer, capable of utilizing multiple vendors and models, is a question of content management and content rendering, very must similar to the content transcoding problem that is common to the mobile content world – in other words, a sys-admin will create an ad-hoc solution, a programmer will create a proper, well structured, well designed solution that carry the enterprise beyond its initial needs and requirements.

A short example of how these interfaces work can be found here – on my company blog.

Reblog this post [with Zemanta]

As some of you may know, I’ll be attending the ITExpo in Miami Beach, Florida. The subject I’ll be lecturing about is “Virtualizing Asterisk”. However, I have to be honest, I really need to change the subject to be called “Asterisk in the Cloud“.

Ever since the introduction of Amazon EC2, people had been trying to get Asterisk to run properly inside an EC2 instance. While installing a vanilla Asterisk on any of the Fedora/RedHat variant instances in EC2 isn’t much of a hassle, getting the funky stuff to work is a little more tricky.

One of these tricky bits (which I hadn’t yet found a solution for) is the issue of supplying a timer for Asterisk’s MeetMe application. In the old days (prior to Asterisk 1.6), Asterisk required the utilization of a virtual timer driver, provided by Zaptel in the past and now the DAHDI framework. The problem is, that while you are fully capable of compiling and installing DAHDI on an Amazon EC2 instance – the problem starts once you want to use it.

A few words about Amazon EC2

For those not familiar with Amazon EC2, its general infrastructure is based upon the XEN virtualization project. XEN is a para-virtualization framework, meaning that is performs some of the work utilizing the underlying Operating System kernel and some of the work performed with a special Kernel in the virtualized Operating System instance. This poses an interesting issue with every type of application that relies on hardware resources and their emulation.

To learn more about the XEN project, go to

So, where’s the big deal?

So, if you can compile your code and run it in an instance, as long as you have the kernel headers and kernel source packages – you should be just fine – right? WRONG!

Amazon EC2 deploys its own Kernel binary image upon bootstrap, causing what ever compilation you may have done to the Kernel to go away (unless you’re creating a machine from real scratch). Another issue is a version skew between the installed Operating System kernel modules, the actual kernel and the installed compiler. For example, the instance that I was using had the XEN capable kernel compiled with gcc version 4.0.X, while the installed operating system was gcc version 4.1.X – so, no matter what I did to compile my kernel modules or binary kernel, I would always end up in a situation where loading the newly compiled kernel modules will generate an error.

Did I manage to solve it? – NOT YET. I’m still working on it, and I have to admit, that considering the fact that I have over 10 years of Linux experience and had compiled kernels from scratch many times, this one has gotten me a little baffled – I guess I’ll just need a few more nights and a case of Red-Bull to crack this one open.

So, what can we do with EC2?

In my view, EC2 + Asterisk is the ultimate IN/NGN services environment – and I have proof of that. A recent lab test that I did with one of my customers showed a viable commercial alternative to Sigvalue when using Asterisk and EC2 structures. The main reason for our belief in using EC2 was the following graph:

IN/NGN usage over 24 hours

IN/NGN usage over 24 hours

What we’ve noticed was that while our IN/NGN system was generating traffic, it’s general usage showed peak usage for a period of 2.5 hours, with a gradial increase and decrease over a period of almost 10 hours. Immediately that led us to a question: “Can we use Amazon EC2 to provide an automatd scaling facility for the IN/NGN system, allowing the system to reduce its size as required?”

To do this, we’ve devised the following IN/NGN system:

Amazon EC2 Enabled IN/NGN Platform

Amazon EC2 Enabled IN/NGN Platform

Our softswitch would have a static definition of routing calls to all our Asterisk servers, including our EC2 instances which had static Elastic IP numbers assigned to these. The EC2 Controller server was incharge of initiating the EC2 instances at the pre-defined times, mainly, 30 minutes prior to the projected increase in traffic. Once the controller reaches its due timer, it will automatically launch the EC2 instances required to sustain the inbound traffic.

For our tests, we’ve initiated 5 AMI instances, using the EC2 c1.medium instance. This instance basically includes 2 cores of an AMD opteron, about 8GB of RAM and about 160GB of Hard drive – more than enough. Initially, we’ve started spreading the load evenly across the servers, reaching about 80 concurrent channels per instance, and all was working just fine. We managed to reach a point where we were able to sustain a total of about 110 concurrent channels per instance, including the media handling – which is not too bad, considering that we are running inside a XEN instance. The one thing that made the entire environment extremely light weight is the GTx Suite of APIs for Asterisk. Thanks to the GTx Suite of APIs, scalability is fairly simple, as all application-layer logic is controlled from a central business logic engine, serving the Asterisk servers via an XML-RPC based web service. Thanks to Amazon, practically infinite, bandwidth allocation – the connections from the Asterisk servers to the US based central business logic was set at a whopping 25mSec, thus, there was no visible delay to the end user.

It is clear that the utilization of Asterisk and EC2 operational constructs can allow a carrier to establish their own IN/NGN environment. However, how these are designed, implemented and operated are at the hands of the carrier – and not a specific vendor. If the carriers around the world will take to this approach, time will tell. As a recent survey stated that 18% of the US PBX market is currently dominated by Open Source solution, having Digium dominate 85% of these 18% (~15%), I’m confident that we will see this combination of solutions in the near future.

OK, I’m fairly psyched out at this point, actually, I would say that I’m ecstatic. The first ever Asterisk Bootcamp training will be taking place on the 25th of May, in Ramat-Gan, and I’m the trainer 🙂

If you are not from Israel, you may regard this as: “What’s so special? Asterisk bootcamps are all over the place!” – but for me this is something really special. In 2002 I began a journey into the world of Asterisk, starting to develop various applications and voice systems based upon this wonderful piece of software. As time progressed, my interest in Asterisk grew, and thus, also my interest in the adaptation of Asterisk to Tier-1 telecom infrastructures.

In 2006 I’ve finalized my first Tier-1 Asterisk application, an International Operator Assisted Dialing platform. If you are from the telecom business, you are most probably familiar with the Nortel TOPS platform. The platform that I’ve developed replaced a Nortel TOPS system, that was installed at Bezeq International in Israel. This was the first ever Asterisk installation as a critical Tier-1 carrier service, within a Tier-1 carrier. Just to give you an idea of the big ho-ha, in Israel, this platform is a regulatory issue – no system, no license – no license, company go bye bye!

A year ago I left Atelis, the company that I help found and manage – a Digium channel and reseller in Israel and the UK. I’m now a freelance consultant/developer, focusing mainly on the integration of Asterisk in various applications and business structures. This training session, to me, is somewhat of a climactic event – as it bring my 5 years of Asterisk experience, into a situation where I can share this experience and know-how with a select group of people – who all share the same goal: they want to learn Asterisk to the fullest.

Training will begin on the 25th of May, and I will try to post each day of the bootcamp, telling you my thoughts, feelings and news from the bootcamp – I’ll even throw in some pictures to match 🙂