A topic which is often discussed is the prevention of incoming spam but what about outgoing spam from shared web servers? The standard approach is a reactive one: the system administrator will respond to an abnormally high mail queue upon detection regarding a particular server and hopefully disable the outbreak of spam. There is also always the risk that the primary cause was not properly identified and the spam activity continues, thereby repeating a further cycle of the problem.
The biggest weakness endured by this approach is that the affected server could potentially have been blacklisted by one or more Real Time Blackhole Lists and this can severely hamper legitimate e – mail deliverability for other users on the server who share the same IP Address. Once listed, it can take hours or in some cases even days to become delisted.
The alternative method to stopping the spam would be to employ a spam filter such as SpamAssassin but for outgoing messages. If you would like to have such a setup on your cPanel based server, then it is very easy to implement as per the following reference.
All outgoing e – mail messages will be scanned for spam – like characteristics and discarded if they reach the configured determining threshold value. Any existing monitoring system check will also need to be changed to rather take into consideration that the e – mail queue size is no longer important, but rather the total number of instances determined as outgoing spam by the server’s mail logs.
There will be an increase in terms of resource usage / processing but the tradeoff in terms of maintaining good e – mail reputation should be well worth the resource penalty.
I want to take the opportunity in this blog post to wish everyone both a good holiday season, as well as New Year up ahead. This will likely be the last entry that I will be making before the start of 2014 and I hope that everyone can get some time to spend with their friends and loved ones.
Today I want to talk about strace. Many system administrators are aware of its usage but for those that aren’t familiar with it, an apt description from its Wikipedia page desribes it concisely below:
strace is a debugging utility for Linux and some other Unix-like systems to monitor the system calls used by a program and all the signals it receives, similar to “truss” utility in other Unix systems. This is made possible by a kernel feature known as ptrace.
I personally always find it useful when debugging hanging processes (commonly Apache, for example) but it can also be used for identifying questionable performance. One thing to keep in mind, however, is the overhead its usage incurrs. This can be demonstrated as follows:
time dd if=/dev/zero of=/dev/null bs=512k count=1024k
1048576+0 records in
1048576+0 records out
549755813888 bytes (550 GB) copied, 23.6136 s, 23.3 GB/s
time strace -c dd if=/dev/zero of=/dev/null bs=512k count=1024k
1048576+0 records in
1048576+0 records out
549755813888 bytes (550 GB) copied, 82.8322 s, 6.6 GB/s
As one can see by the results, there is a significant performance penalty and this should always be taken into account when strace is used as a means to troubleshoot and examine system bottlenecks.
See you all next year!
It’s been over half a year since my last post and I still remember my promise about adding a technical entry. What I want to write about today is how to avoid (or at least limit) false positives generated by the awesome uptime checker Pingdom.
For this example, I will be using arguably the most popular firewall used for standard cPanel web servers: ConfigServer Security and Firewall (available at http://configserver.com/cp/csf.html), commonly referred to as CSF.
In summary, here is the problem:
- Pingdom’s IP Addresses must be added to your firewall rules in order to avoid them from becoming blocked, otherwise this leads to false positives.
- Pingdom will occasionally add new IP Addresses / check locations and these must then be integrated with the firewall rules as quickly as possible.
- The use of a firewall allow list which is maintained by hand is far too labour intensive when multiple server configurations must be updated.
The solution? A search online reveals that Pingdom provides a realtime list of checker IP Addresses as an RSS feed: https://my.pingdom.com/probes/feed So far so good.
Further searching revealed a method involving the use of CSF’s “
GLOBAL_ALLOW” setting which is then used in conjunction with this RSS feed. CSF can add the IP Addresses to its allow list but this is not ideal in my opinion because a file will need to be hosted either from the local server’s htdocs or a remote web server which can be used as a source for all servers to gather the IP Addresses. This latter implementation poses security concerns because that central server will have influence over the firewall rules of all other servers which collect the Pingdom IP Address list from it.
A simple one liner below can be used as a cronjob instead presents a far more elegant solution:
/usr/bin/wget --quiet -O- https://my.pingdom.com/probes/feed | grep "pingdom:ip" | sed -e 's|</.*||' -e 's|.*>||' | xargs -n 1 /usr/sbin/csf -a
It would also be a good idea to list pingdom.com in
/var/cpanel/commondomains in order to prevent a user from creating it as a domain name and then using it as a mechanism for manipulating the server’s firewall rules.
Working remotely from home means that as a worker, you should be many more times more reliable than the traditional office worker. The excuse of heavy traffic or car failure can’t apply. Major events or infrastructure failures can indeed hinder work but you must be prepared at all times for any situation (even if it means only informing your employer of the problem).
The following is a very basic checklist for being a reliable remote worker:
- Regularly updated and printed copy of all employee contact details. If you work for a large business, then the list can be limited to key co workers and managers. If your employer gives you a laminated copy with branding, then they are doing it right.
- 2 internet connections. This is, in most cases, even more crucial than a UPS because power is commonly less likely to have periods of interruption than internet access. A cellphone can be used for this purpose.
- A UPS (Uninterruptible Power Supply) suitable for at least a 60 minute power outage. Energy efficient computers (such as Atom based all in one platforms) should be able to operate for a few hours in contrast to a standard desktop computer.
- 2 alarm clocks. The most convenient is to use a normal desk alarm clock in conjunction with your cellphone. Replace the batteries once per year on the day prior to time off from work.
- An offline / cloud copy of all needed software to carry out your work. This is to be used if your computer must be replaced.
A backup internet connection is the most important point above but also the simplest to have in place (the correct cellphone cable and data access are all that is needed). If you are an employer, ensure all of your workers have a backup internet connection and perform a test of it.
A quick glance over Amazon’s product listings reveals that there are good quality UPS units available for under $200 (closer to $100 if you are lucky enough to make the purchase while there is a discount). If you are an employer, you can easily implement a purchasing programme for employees who wish to purchase a UPS. The $200 business expense is minimal for the level of assurance that is returned. Even a split purchase agreement could be considered: after 1 or 2 years, the company can then pay the employee back the original portion they paid as a further incentive. This kind of thinking and planning is vital for all employers of remote workers.
On that note, this should be the last blog post for 2012 and I want wish all readers a very special festive season. While I am a technology enthusiast at heart, I also love the business and management side of the web hosting industry which is why my next blog post will be of a more technical nature, I promise.
What if I were to tell you that your web hosting business could reach out to most of your clients and that it wouldn’t even be considered spam? While it wouldn’t necessary reach every client, it almost certainly wouldn’t be blocked. It may seem to good to be true, but you are actually doing it without even really knowing. The answer would be e – mail auto responses.
The most underutilised and overlooked point of contact in the web hosting industry by far, automated e – mail messages are most commonly found in the form of support ticket acknowledgements. They can also be in the form of invoices and receipts. The retail industry have already caught on to this, very often adverts can be found on the opposite side to the printed receipt.
The benefits of regularly tweaking your automated e – mail messages can be staggering once understood properly. Customers often complain on forums that their billing or sales tickets are not answered quickly. These forum posts could have been avoided if the web host involved only had the foresight to modify their automated messages to contain the available hours per department.
These e – mail message modifications can also simply be good natured and non commercial, for example they could feature a positive New Year message. There is no reason for technical support tickets to be clinical and lack engagement – a season’s greeting message would serve only to boost the reader’s mood.
Let me give you one real world example:
Let’s say a web hosting business has recently implemented a firewall unblocker section which is made available from their client portal. This area can be used to unblock a customer’s IP Address which may have become blocked due to failed login attempts. A good portion of the daily support tickets which are received involve the tedious task of checking and unblocking these affected customers. We can help lower the number of these requests simply by modifying the automated support ticket responder which acknowledges support ticket submissions with a message asking the reader if they are aware of the newly available firewall unblocker.
Even if only a very conservative 10% of readers actually take the time to absorb this information, you have just lowered this particular type of support ticket by a noticeable margin in less time than it took to answer one firewall block ticket.
Special offers can also be advertised this way but be careful not to transform communication into spam. If you choose the route of advertising instead of a season greeting or snippet of useful information, then limit it to one offer and an exceptional one at that. You can also take advantage of issued receipts but do not advertise anything on your invoices – this will look very unprofessional.
Your customers are contacted regularly in the form of automated messages – be different from the rest and use this as an opportunity to engage your customer for the better.
If I asked you if you would purchase from a shop which stored your credit card details, would you still do business with them? I am fairly certain that most people would hesitate, regardless of how good their service is. It also would probably make no difference to your opinion if I told you how securely this was stored (be it in physical form in a safe location or encrypted).
Some businesses actually do this and the web hosting arena is no different. Included below are types of private information which web hosts commonly store in their billing system:
- Your control panel username and password
- Your credit card information
The first point is far more common then the second. The average non – technical web hosting operation (believe me, they make up the majority of service providers) would not even consider this as a security concern. Let me use an analogy:
Many people hide a second set of keys to their household under a potplant or rock as to avoid being locked out. What if I were to tell you that your web host was doing the exact same thing but with access to your e – mail and files? This is because the web host in question assumes it is more convenient for the customer to login to control panel via their customer portal instead of accessing it directly. Is it really worth the risk of having every single customer’s control panel username and password revealed in the event of a security breach?
There is absolutely no reason for any web host to store usernames and passwords in their billing or portal areas. It’s careless. There must only be one copy of a username and password and that is on the server itself.
My next point concerns storing credit card information. There is only one instance where this argument has a leg to stand on but is still unacceptable in my opinion: Cloud Computing. Cloud computing allows for hourly billing and in such a circumstance it would not be possible for a customer to re – enter their unsaved credit card information to make payment every hour. There is a solution to this, though and that is to rather have these hourly billing customers make lump sum deposits to their account.
Many web hosting customers pay yearly for their service – is it really worth while storing their credit card information simply so that payment can be more convenient once per year?
Ask your web host if they store your credit card information and / or control panel authentication details. If they answered yes then point them to this blog post.
When you think of live technical support, you often imagine live text or phone based discussions – it essentially means technical support which is given in real time. It can also correctly be assumed that there are basically only 2 forms of live technical support currently being offered in the web hosting industry:
While both of these can mostly be seen as ineffective (depending on how they are implemented), there is one further form of technical support which is almost totally ignored by the web hosting industry: Remote Desktop Support.
Remote desktop support is used extensively in the corporate IT support sector but not in the web hosting industry, which I find strange. It allows for a support technician to login and fix any client side configuration settings or perform all needed tests immediately and without any delays (such as informing a client on directions to perform a traceroute, for example).
A good example of an approach to offering this service is the use of TeamViewer. It requires no installation or technical setup and doesn’t even need any special open ports or port forwarding configuration. It also supports Windows, Apple and Linux based operating systems.
The implementation of this technology can be extremely flexible and powerful, for example all that you need to connect to the customer’s computer is a short set of numbers and then a further short password. You can probably already see where I am going with that:
- Technical support help desks can have fields for these needed values. This allows for immediate access and resolution for the customer.
- Customer records can have these values stored for automatic retrieval.
The service can also be offered at a premium, for example an extra $10 per month over and above the customer’s current monthly fee. There is no significant extra cost to the service provider because the TeamViewer service is hosted externally and the live support operators should have no problem in troubleshooting an e – mail client, so training is not essential.
There is no doubt in my mind that live technical support in the form of remote desktop support will become mainstream one day, all it will take is for an industry leader to see the bigger picture and offer it as a unique selling point. Competitors will soon follow the lead.
What I decided to write about today is a topic which has always intrigued me: business processes. I have had my fair share of jobs throughout my life (having sought employment from a young age) and it always brought me joy to discuss the various business processes / procedures with my employer and how they can be improved. To this day, I still see meet one of my very first employers for coffee, even after having left work there for nearly a decade.
Communication during a time of crisis commonly follows the same formula, that is a contact number is provided for employees to phone and state the emergency. Depending on how advanced the telephony setup is, it may either direct this call to one or even multiple contacts – hopefully reaching an individual which can help remedy the problem.
I personally believe this kind of setup is mostly ineffective and I will tell you why:
- Most of the time the call is going to only reach one person and if this person is unavailable then co workers will be left guessing as to what the situation might be.
- Assuming the intended person is reached, this individual must then contact other employees and notify them what is happening. This may even be prior to the start of remedying the crisis.
- A very expensive call rate will be charged should the employees be located overseas.
The alternative is fairly obvious but often ignored by organisations as a tool for communication. Many businesses seem to have forgotten the flexibility and power of SMSes (however dated the technology may currently be). SMS gateways can be developed to achieve almost any possible solution and there are even third party providers which can handle most of the technicalities on your behalf.
We can easily see the benefits of an SMS setup when using an example of a work – from – home employee’s computer or internet failure:
- One SMS message can immediately be forwardered to any number of co workers on shift, thereby keeping them up to date (in real time) as to what the situation is.
- This same SMS message can also be forwardered to workers which are not on shift, but rather on call / willing to work extra hours.
I am not saying that traditional phone / VOIP systems don’t have their place (because they definitely do), but it depends on the actual function which it is needed to serve. Taking into account that emergency situations specifically are often very simple to describe and therefore don’t require the overhead and inherent delays of voice communication.
The solution is to rather have any emergency communication instantly reach all important parties and SMS is the tool to achieve this. You could be even more prepared as a business though and offer both SMS and telephonic based communication mediums for use in a moment of crisis.
I recently purchased a D-Link 2500U because it allows telnet access to manage using command line. This feature is not often provided for ADSL modems (not to be confused with the routers with an integrated ADSL modem). Besides, I can’t resist a Linux powered device.
I then decided to experiment with the route command and split local and international traffic. There was surprisingly a Local Router project with a 2500U script, however their use of route proved to be incorrect in syntax.
Here is my modified script if any of you need it or are interested:
# Your router's IP Address:
# Your router's login user:
# Your router password:
# Your router's local interface:
# Download new list of local routes
wget "http://developers.locality.co.za/routes-rs.txt" -O localroutes.txt
while read i s
if [ "$i" != "#" ]; then
echo send \"route add `echo $i | sed 's/,/ /'` $if \\\\r\" >> localrouter.sh
echo expect \"# \" >> localrouter.sh
done < localroutes.txt
echo "#!/usr/bin/expect --" > localrouter.sh
# Perform login
cat >> localrouter.sh << EOF
send "open $host\r"
expect "ogin: "
expect "word: "
# Add routes
cat >> localrouter.sh << EOF
expect "$ "
# Run script
ifconfig can be used to obtain the needed (local) interface name.
It can be difficult to choose a good Virtual Private Server host with so many different web hosts available. Apart from an earlier post which demonstrated the use of a modified version of UnixBench (http://www.rskeens.com/?p=17), there is another quick and easy way to test a core component of a system: the disk I/O (Input / Output) throughput.
A quick test that I saw on Web Hosting Talk today used:
dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
Granted, this is only going to be a rudimentary test of your disk performance, but it can still be worthwhile to check out. My results of a VZ based VPS Node with no free slots are below:
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 3.4941 seconds, 307 MB/s
That’s fast considering my desktop with a single disk configuration only gets:
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 26.4561 s, 40.6 MB/s
Go on, bench your VPS (or desktop!) and see what results you get.