The rants and raves of a technogeek
Posts tagged VMWARE
Thoughts of Virtualization – Part III – Multiple Asterisk Gateways
Dec 15th
While this post is titled “Thoughts of Virtualizaiton”, the applications described can be easily applied to non-VM type installations.
Virtualization is a wonderful tool, it enables rapid growth and rapid deployment of new servers and services. However, just like any other platform that tends to grow across the time line, it poses the same annoying issue of managing a large system, especially when dealing with Asterisk based installations.
Let us imagine the following scenario: A Calling Card company while utilizes 8 different Asterisk application servers, are utilizing a single Database servers cluster and are receiving inbound calls from various sources and load balanced across all Asterisk application servers. What I’ve described above is more or less the practice most (if not) all calling card operators deploy. No matter if the usage is A2Billing, MOR, ASTBILL – the methodology is more or less the same.
One of the bigger issues with such an installation is debugging of a running session, more over, the ability to debug a session after it is finished. This situation is caused by a simple, yet annoying issue, we are operating within a “zero-knowledge” system, where we have no precognition of where a specific call will be handled utilizing our cluster. Now, if you are an experienced sysadmin, you would most probably do the following:
- SSH to all your Asterisk servers.
- Tile all consoles on your desktop.
- Start the test – and hope your eyes are fast enough to capture the right gateway.
Well, this is the normal practice with most people – but I have to admit it’s kind of annoying. Now, let’s imagine that we are now building our system from scratch, we’re not using A2Billing or any of the other Open Source products, we simply build our own application framework. So, what do we need to do in order to keep track of our system correctly?
Step 1: Consolidate
Consolidate the messages coming from each of your gateways to a single logging facility. The best track would be to utilize some form of Syslog facility. For example, all the scripts and network services that I develop utilize a clear and concise interface to syslog. I usually re-direct the syslog facility that I use to an external server, thus, I get all the logs on a single syslog file system.
If you are worried about I/O issues on the syslog server, you can always create a “syslog-proxy” using tools such as memcached or others.
Step 2: Identify
Your syslog write function should always include a prefix indicating the name of the generating Asterisk server. For example, have something like the following prefix your syslog entry:
Dec 14 21:51:32 pbx [PBX01/6d6d6423a2244aa71980e5a5b437919e/check_pincode[22537]: agiParameters: check_pincode
While the syslog facility will include your generating hostname, when duplication VM’s, this would be a really good practice.
Step 3: Analyse
Once your logs are consolidated to a single environment, it should be fairly simple for you to go about and analyse these in a pre-defined routine. There is little to gain from analysing the logs on-the-fly, but analysing it every 5/10/15 minutes will prove worthwhile.
Step 4: Audit
Auditing is good – as long as you keep a clear view of what you audit and what you don’t. Audit key points in your application to a database can save you a whole lot of time of debugging – just make sure your audit is clear.
Keep the above in mind and you should be just fine creating any scale of platform.
Thoughts of virtualization – Part II
Nov 30th
I really like going to events at the GarageGeeks (www.garagegeeks.org) – apart from meeting with some friends, having a good laugh and enjoying the casual beer, the technical talks and the general subjects that are being attended at a gathering are always interesting.
This time, the Geeks had decided to focus on Cloud Computing, trying to introduce the somewhat highly evolving field of the IT/Computing industry to the visitors of Garage. Now, I have to admit that during the past few weeks I’ve been studying the various aspects of utilizing Amazon EC2 services, and I’ve come to the somewhat problematic conclusion: “Cloud computing is a wonderful solution, as long as you have clear and concise control over your computing cloud!”
The main problem with traditional cloud computing, as offered by Amazon and others, is that the physical boundaries of your system are somewhat unknown – not to you and in some cases, not to your provider either. Let us imagine a Tier based architecture platform, implemented utilizing Amazon EC2 technologies – our web front-end may be located in the US, the application server in the UK and the Database in Zimbabwe for all that matters – in accordance to EC2 and traditional cloud computing, that is fairly normal.While EC2 and some other providers had introduced the idea of a Zone, making sure that your virtual servers are all located in the same physical data center, other solutions are yet to follow.
The above design introduces an interesting “Critical Path” management problem, tied deep into the Tier based architecture design pattern. Companies such as GigaSpaces and others had made it their business to cash in on the problem, by providing a distributed, memory based, middle-ware – literally sweeping the “Critical Path” problem under the rug. However, as the platform will evolve, “Critical Path” issues have a tendency to re-appear, usually, a lot more serious than the original problem – due to new constraints and new operational paradigms within the operational construct of the cloud platform.
So, question be asked: “Is cloud computing good? Can you really build a full service based upon cloud computing services?” – The answer isn’t a straight forward yes or no. For example, while I have a couple of servers on EC2, serving me as development servers mainly, I’ve built my own little cloud environment on my own servers – used for production usage. I’ve used a mixture of OpenVZ servers tied together with my own provisional interface to create a seamless cloud, enabling me to replicate servers, control their operational structures at east and most important – expand my cloud rapidly by introducing new servers to my cloud.
Is my paradigm good for all purposes? I can’t really say – it works for me, it may not work for you. The thing is this, don’t expect cloud computing to come in and solve all your problems, you may run into new ones you are completely unfamiliar with.
We’re melting, we’re melting…
Oct 11th
I’m not much of an economist. I understand income and expenses, I understand more or less how the stock market works from time to time, but I’m no economist. I’ve been in the hi-tech industry since early 1997, which means, that I’ve seen this world being elevated to the level of godly hood, buried alive with the explosion of the dot.boom era, only to re-invent itself as a highly competitive telecom’s industry and the “Network is the Computer” manner of thinking.
About 2 weeks ago, a man I truely admire had written a post entitled: Economic Meltdown – Friend or Foe of Open Source? (written by Danny Windham, CEO of Digium). Danny talks about the various aspects of companies cutting down on their IT budgets and reversion to Open Source technologies, as a means to hedge needed cash. As I witnessed the Dot.Boom explosion of 2001, I can honestly say that 2001, at least as far as I can say, was the year that Open Source technologies had truly assimilated to the normal business practice of companies. Even beyond that, companies which were not based on Open Source and Linux/Unix based technologies were clearly not a valid option for investors.
One of the urban legends talks about Hotmail wishing to acquire an Israeli company called Commtouch, back then, a webmail company. Commtouch was purely based on Windows NT technologies back then, and while the business track of the negotiations were going fine, the deal collapsed as the technical sides, of a back then BSD based Hotmail, basically negated the deal. Now, I have no idea if the story is true or not, again, this is an urban myth, however, I believe myths are usually based on some portion of the truth - at least to an extent (even a small one).
While the adaptation of Open Source technologies is a must for companies, in order to survive the troubled waters of these time, it is highly dependant on the solution providers, consultants and IT outsourcing companies. In Israel, for example, it is very much a customary to hire outsourced IT assistance, on a part time basis, allowing you to cut your IT costs. When bringing in IT assistance, you’re not only bringing in IT assistance, you also bring in IT governance and IT policies – as an outsourced IT person will usually operate under the terms he had learned in his company. If his company is a M$ oriented company, no matter how Open Source will be crucial to the business, M$ will still rule the domain in that area. I believe that the first thing that needs to happen is for the consulting and IT outsourcing companies to realize that they need to change, first for their customers, then for themselves. If the small start-up companies and established companies will start falling, the consultants will simply dry out of work.
It is our responsibility as IT/Telecom/Development/Architecture/Design consultants to explain the aspects of using Open Source, assimilating it, integrating it, both the pros and the cons and then, assist our customers in doing so successfully – this is the only way to go.
For example, let us take examine the following scenario – a telephony service provider. Our telephony service provider is based on Asterisk as the switching environment, a purely M$ base for all the billing, web front-ends, databases, development and so on. Their entire IT infrastructure is fully owned by them, all running with ESX servers and other proprietary technologies. This company will be required to pay hundreds of thousands of dollars every year, for support purposes (and the year is coming close to an end), their expansion is highly dependant on 3rd parties and while they are self sufficient on the IT side, they are purely dependant on the technology side. Now, imagine that this company would have had their entire back-end based on Open Source technologies, let’s imagine PostgreSQL and Ruby on Rails as the web frame work, how much money will that save per year? now, let’s remove ESX out of the equation… let’s also remove non-Open storage solutions and replace those with CoRAID or something similar, we’ve saved hundreds of thousands of dollars on an annual basis. Radical? – YES! But drastic times call for drastic measures, and when drastic measures are required, Radical solutions are a must.




Picasa
Twitter
Facebook
LinkedIn
Youtube
RSS