The Nir Simionovich blog
The rants and raves of a technogeek
Extreme Asterisk Cloud Performance – Part I
Jan 3rd
*** This post was originally posted at http://www.greenfieldtech.net
Here’s a challenging question for the Asterisk technical savvy of you… What is the top performance you can squeeze out of an Asterisk box, running on Amazon EC2 – or to that extent, a cloud infrastructure? If you scout the Internet, you may find various answers – however, most of them aren’t backed up by real numbers or real information,made accessible in a normal readable form.
Recently, we’ve become heavily involved in a project requiring massive usage of cloud based infrastructure. I won’t go into details as to what the project is or what we are doing, however, I felt that some interesting facts about Asterisk 11.0.1 and Cloud infrastructure can be shared with the rest of you.
Before we dig deep into the actual results, let’s talk about the various measurements usually associated with performance assessments of an Asterisk box, mainly, the machines load average. In order to continue, we must first understand what the Linux Load Average actually is. Most of you know load average as the below:
Most people know the load average as those 3 numbers, ranging from 0 to anything higher, and if the numbers reach a certain level – it’s bad. But the question is: “What is a good number? and what makes a number bad?” First, let’s understand what the number represents. Load average is an exponential average of all your machines processes. Running processes, sleeping processes, waiting processes and on Linux, also processes currently waiting for I/O access. Now, these number are directly correlated to the number of processors/cores your server has. In general terms, a machine with a single core, any number higher than 1 is considered bad – where 1 represents 100% of the resources being consumed. So, if your machine has 4 cores, the number 4 is your top most number – and from there it’s linear. Now, can we calculate HyperThreading into the equation, multiple CPU pipelines, SSD access – in Linux, all these come into play into that equation. In other words, we’ll never know what is the actual top limit, but working with a rule of thumb based upon the number of cores is a good practice – specifically if your operational environment is a virtualized one.
Now, there are 3 numbers in there – a 1 minute average, a 5 minute average and a 15 minute average. Technically speaking, the 1 minute average isn’t really interesting – as it is highly affected by context switches and process bootstrapping, thus, there is a good chande that its number will be higher than the “advised” number. The numbers that are more interesting are the 5 minute and 15 minute average. Technically speaking, if your machine’s load average is considerably higher than the advised at these, something is definitely wrong.
Southern Hospitality and Southern Cuisine
Oct 27th
This year’s Astricon took place in Atlanta, Georgia. Now, the only thing that I know about Atlanta was: it’s at the southern part of the United States, the Atlanta Asterisk Users Group is one of the biggest users’ group for Asterisk in the world, Bryan Johns of Digium is from the Atlanta area and that Southern Cuisine is considered to be very good.
Now, Southern Cuisine is really good (I’ll get to that shortly), but Southern Hospitality is something different altogether. Till today, I’ve been to several parts of the US. I’ve been to Manhattan, Boston, Phoenix, Washington, Denver, San Fransisco, San Jose, Detroit, Ann Arbor and Miami. Now, I have to admit, in none of the listed states had I ever been approached by the owner of the restaurant to check how the table is. In every “non-chain” restaurant I’ve been to, the owner just stepped over to our table to check out how we are doing. And this is not your ordinary “How’s the food? – good, bye” conversation, each time the owner seems to be genuinely interested in our well being, how we feel and even more.
So, southern hospitality gets my two thumbs up – there’s something to learn from it.
Now, regarding the cuisine. I didn’t have much time to enjoy the local cuisine, however, being in Atlanta with my colleague Eric (and his parents the came to visit all the way from Florida) surely helped to narrow things down in that respect. According to the information we had, one of the best places to eat authentic southern cuisine would be Mary Mac’s Tea House. Now, putting aside the multitude of pictures on the walls, depicting various movie starts and various public figures eating at Mari Mac’s, I have one thing to say: They know why they are eating there. Ranging from their crispy “Mudbugs”, through the amazing Sweet Potato Souffle up to their juicy Fried Chicken – what can I say, an experience to be remembered and hopefully experienced again in the future (gotta take my family there at some point in the future).
Earlier that week we’ve eaten at the “White Oak Kitchen & Cocktails” – it’s a brand new place and the food is amazing to say the least. We were a group of 6 people, including Eric Klein (GreenfieldTech), James Body (Truphone), Randy Reznik (vuc.me) and Tim Panton (Voxeo Labs). Each one of us had something else – we all enjoyed our food very much. The place isn’t on the “low cost” range, but definitely worth a visit, their steaks are awesome. In addition, I drank a beer, which I don’t recall its name, that resembled a HeffeWeiser in some form, although, it was tangier and slightly sour – which was very refreshing and new.
All-in-all, Atlanta gets the two thumbs up from me 
Updates, Astricon, Allo.com and more
Oct 26th
Ok, before all of you jump at my throat for not posting for a long time – I’m sorry. I can’t believe the last time I posted was around June, much has changed since then. So, let’s start with some updates… well, there are no big news updates. Since I left Humbug, I’ve been doing my best to keep busy with GreenfieldTech projects. We’ve successfully completed 10 different custom development projects since June and we’ve started a brand new services branch at GreenfieldTech – the VoIP Security Audits branch – but that’s a different post altogether.
So, I spoke at this years’ Astricon, which took place in Atlanta, Georgia!. This is my first time to the southern parts of the USA and might I add, Southern Hospitality is something you need to experience (I’ll write about that later). Much had changed at Digium in the past 2 years. Many of the long time project members had left to new grounds, Asterisk-SCF is now officially no longer in active development, in other words – this years’ Astricon was a fresh breeze of wind, specifically with new people at the driver seat and new ideas springing about.
So, let’s talk PRI cards. Once every often a company approaches me to evaluate their products. I do my best to be as impartial as I can – after all, Digium products are my favorite. However, I’m always happy to see a product that can compete nicely with Digium, simply because I believe it will make Digium products better and stronger. So, a couple of months ago, Allo.com approached me to evaluate their PRI card. I agreed, and they’ve sent me their PCI-e version of a quad span E1/T1 card.
So, let’s start with what I didn’t like about the card. People, it’s 2012, technology had progressed a great deal since the old Zaptel days, not to talk about the old Xilinx Spartan chips – comm’on, even the crappy Chinese boards don’t use that anymore – move on. Ok, let’s put aside the issue of the actual chips being used on the board – can someone PLEASE explain to me why I need to patch my DAHDI modules to support these cards? How shall I put it, patching DAHDI/ZAPTEL is so 2004. Make your card fully compatible with DAHDI, no patching, stream line the card with the DAHDI stock kernel module – OpenVOX did it, Yeastar did it (do an extent) – you want me to use your hardware, make it easy to install and simple to update and maintain.
OK, regardless of my somewhat reluctant feelings regarding drivers compatibility, I had the unit installed in a test gateway. It performed as I expected from a low-cost compatible. It held up nicely with normal traffic, but when I tried pushing 30 call initiations per second on the card, it heated up slightly and CPU spikes could be observed here and there. Now, in an office scenario – sure this card will do nicely – in a service provider scenario – I’ll think twice. Now, in the past I’ve received similar performance from other clone cards, so my estimate is that there is a group of engineers passing from one company to another, coming in with the know-how for a single design and they wrap that into a card.
Final word regarding the Allo.com card – not my favorite, but definitely a possibility for office environments. I have no idea how their other equipment holds up, but I hope that it holds as well, so that the office/smb market has a new option to choose from.
I will post my Astricon update later on.
Fare well Humbug Telecom Labs
Jun 25th
As I’ve publically announced a while back, I’m no longer an active member of the Humbug project. I’ve spent the better part of the past 30 months with Humbug, working on bringing something new to the world – a brand new approach to preventing fraud. In the world of social networking, reader driven news and crowd wizdom – Humbug, at least in my view, was to telecom fraud what facebook was for social networking. Ok, by no means am I comparing the two, I’m simply stating that it brought a fresh new approach to doing something, something that was usually related to mega-companies likes AT&T and Verizon down to mom-&-pop.
Am I right or am I wrong? I have no idea, only time will tell. I truly believe that the Humbug service is a wonderful idea. I truly believe that it is a needed service and that it carries great value to its users. I really hope that it does well. Contrary to what people believe, I have no grudge. Don’t get me wrong, I’m still in contact with people in the company on a weekly basis.
I guess that life is somewhat of a coffee shop, you come in, take your pick out of the menu and hope that you ordered well. If you didn’t like what you got in day one, you can always come back the next day and get a different item from the menu. Several friends asked me if I’m sad about leaving Humbug. Or to be more exact, if I have reservations about me departing the company. Well, honestly, I don’t. Sometimes, you need to take yourself out of the equation, in order to be able to look at the equation. What can I do, I’m a mathmatitian by nature, and I can’t really stop myself from looking at life from a mathematical/analytic point of view.
Right now I’m back to doing what I like best: consulting, writing code and most importantly, being a dad. When I left Humbug, I took a few days to myself, to stay at home and kind’a wind-down from everything. During that time off, I spent a great deal of time with my daughters. My only recommendation to all us Hi-Tech slaves, stop for a while, look around and see that life isn’t all in the office. A friend of mine recently came back from a visit at Facebook HQ. He said: “Facebook bought the old SUN HQ, who SUN employees used to refer to as SUN Quentin”. Facebook employees have no reason to leave the campus, they have everything inside. I think that if companies such as Facebook and Google would have been able to bring the employee’s entire family into campus – they would, only for him not to leave. Trust, leave your office for a bit, take some time off, it will do you good. I used to stay in the office everyday, till 1am in the morning. I’m the guy who used to believe that the entire world will collapse if I’ll go on vacation. Funny enough, it’s not true. When I left m-Wise, the company didn’t cave and when I left Humbug, the company didn’t cave.
Ok, enough babbling, I sound like a nostalgic 80 year old…
The natural evolution… of people and startups
Apr 2nd
It has been a while since I’ve written, I guess I didn’t have much to write about. Ok, I admit it, I had a shit load of stuff to write about, however, I simply never got around to actually do it. A full time job, two babies – or as they call it – real life, has somewhat managed to creep up to me and take its toll on my writing time. Over the course of the past few months, I’ve been heavily thinking about the role of the founder in a startup. Many people regard the founder of a startup as the CEO or some other key role in the company, but it’s not always like that. The question that I asked myself, and I have been for quite some time, is: “as a founder of a startup, what is the most important thing I need to know?” – I recently realized that the most important thing, is also the hardest thing to do.
As I see it now, the most important thing a founder needs to know how to do is: “When to walk away!” – Yes, you heard me right, understanding that you as a founder have to walk away from your creation is the hardest thing you’ll ever do. In many cases, walking away may seem to you like bailing out or giving up, but this is not the case. Sometimes, walking away means: “I’m entrusting my creation with someone else, simply because I can’t progress it any more”. It is very easy to become infatuated with your startup, it’s just like a baby. Initially, it’s just an idea, then after a long incubation time, suddenly it takes form. You hire people, slowly afterwards, you have a proof of concept, then you start selling, then different things can happen – some good, some bad. At that point in time, that’s a critical path, this is the time where you might find yourself at an impasse with your partners, investors, customers, VC’s, etc. What should you do? stay and fight? or just pack up and leave? – well, it all depends on your situation. I won’t detail the situation when you need to fight and stand your ground, as those are fairly easy. The harder ones relate to when to leave.
I would say, that if any of the following are upon you, it’s time to leave:
- Your investors are pushing toward an illogical track, although, they have no idea how the market nor the niche works.
- You ran into financial issues and leaving is a logical step in order to continue the life of the company.
- You are at an impasse with the CEO or other members of management, the issue can’t be bridged.
- You have lost trust with your partners.
At any of those, you must pack up and leave. The reasons are simple enough, all of those relate to direct trust between people. When trust is no longer there, it’s better to pack it up and move forward, simply because it won’t go to good places.
Picasa
Twitter
Facebook
LinkedIn
wikipedia
Youtube
RSS