OzFclass

Classic DNS propagation error.

Checkable here: https://www.whatsmydns.net/#A/ozfclass.com

Not really sure exactly what the issue was but it all seems to be working fine now.



Please let me know if anyone is still having problems.

8-)

Navigation seems slow?

ShaneG wrote:Navigation seems slow?

In what way Shane?
Do you mean slow loading of pages?

For some days I have been noticing intermittent DNS resolutrion of http://www.ozfclass.com

I was able to resolve an IP address from my own name server that had it cached. The Telstra and IINET nameservers were not able to resolve for most of the last three days. When down, nslookup on numerous public name servers (including Googles) failed to resolve. Pinging the IP address was successful. I didn't try to connect (telnet) to port 80 to see if the web server was functioning but I imagine it was. I do not suspect browser caching is the main issue here.

I have seen this before - on my own DNS server - and it was due to a particular form of DDOS attack. While neither the DNS name server used by ozfclass.com nor my own servers are open recursive resolvers - that means neither is vulnerable to what we call a reflective UDP amplification attack - that doesn't mean that attackers don't try. What happens is this.

Someone out there decided to launch a DDOS (Distributed Denial of Service) attack on some poor victim - last Sep/Oct it was DYN in the USA who manage the delegations for Amazon, Facebook, and some other high profile sites. The DDOS attack takes the form of a small UDP packet that is sent to what (the attacker hopes) is an open recursive DNS resolver. That packet has a false (spoofed) source address - in my case it was Amazon (AWS) and Facebook. A small (say 80 byte) UDP/IP request for DNS information (UDP port 53) can result in a response back to the requester (the source IP address) of sometimes thousands of bytes. Hence the term "amplification". That it is going back to the source gives it the term "reflective". The idea is that if the nameserver is "open" then the intended victim (who's IP address has been spoofed) cops a truckload of responses back - that they didn't ask for in the first place! When these requests are being sent to literally thousands of name servers around the world all at the same time - of which a high percentage are "open" (that is, will honour the request) - it can overwhelm the resources of the victim. That is the intention.

Not being an open server does not mean that the attackers don't bother trying anyway. So the intermediary nameserver (the possible "reflector") itself cops a lot of small UDP packets that are ultimately ignored by the nameserver if it's not open - thus not honouring the requests. But it chews up considerable system resources. In my case if was taking 500,000+ every few hours from in excess of 65000+ different "sources" - that is, spoofed addresses within the Amazon and Facebook domains. I started blocking a lot of them at the firewall but ultimately the nameserver groaned nonetheless on those that still got through. The effect it had on the nameserver was intermittent visibility over the internet - just like I have been observing with the forum for the last few days. I repeat - neither my DNS nor the two being used by ozfclass are misconfigured to be "open". All this crap was chewing into my bandwidth something fierce - so while I was not the intended victim, I was sort of collateral damage.

There is little that can be done to mitigate against this sort of attack. Those with money can enlist the support of various third party DNS specialists who have fancy software to deal with this stuff - large companies and government departments can do this - with variable levels of success (remember the ABS?). There is talk of rate limiting modules being included in the BIND software (one of the most popular nameservers used by Linux and other flavours of UNIX). But even rate limiting is not really the answer - unless it is performed upstream from the affected servers. In my case I moved my domains onto a DNS run by a company large enough to have sufficient resources to deal with this crap.

I am not saying that what is affecting the DNS used by the forum is suffering from this. Just that it looks very much like what happened to me and therefore might be worth a sniff (by whoever is running the two nameservers used by ozfclass.com). I spent about a week getting on top of it - this sort of crap is a great time waster! I had better things to do at the time but I couldn't simply ignore what was happening.

Geoff.

Thanks Geoff.

Thank you for the post Geoff,

I'm sure it highlights a lot of the commonly unknown and/or unthought of "crap" that has to be dealt with when running any website,
BUT in this instance I can assure everyone it was nothing to do with any malicious attack.

Fact is, I have spent the past 3 days trying to find out why the DNS propagation has been so slow/intermittent, even after the problem was resolved I wasn't content in just accepting that it was fixed.

To explain the problem I need to go into a bit more detail, so please be patient with the long post.....


Firstly, I rent a server in a Sydney datacentre that currently runs about a dozen personal, friends, family and business websites in a shared hosting environment.
Shared hosting is the most common form of hosting in the world and is, in reality, the only option available without extremely deep resource pockets, particularly in Australia. (a dedicated server in Aus can cost between $500 to $5000 a month depending on the required resources)

Anyway, mid December my server was transferred to a new "state of the art" datacentre and hardware infrastructure. There were some minor changes needed on my side regarding IP addresses and nameserver alterations which were carried out as required.
After the change all sites currently running on the server (including the ozfclass.xyz demo site) were operating as they should, No Issues.

With the (limited) feedback we had received over a few weeks with the demo site it was decided we should go live with the new site. So the process was planned and carried out without any problems.

Within 3 hours I had full propagation and was only seeing the new site. I had contact with several others that were not having the same degree of success... and some that were.
This is normal with any nameserver change and can vary significantly between ISPs, so no "ALERT" buttons were going off until after 2 days when posts were coming about intermittent connection issues.

At this point I knew further investigation was required, although all signs were pointing towards a propagation issue, I had never, in 30 years had propagation this slow and intermittent. (yes, 30 years, I have been involved with online networking since before the internet was called the internet)

So to cut a really long story a little shorter.....

I double/triple checked that all server settings were as they should be, which they were, then contacted my supplier explaining the issue.

After several emails back and forth here is the end result.
=====================

Hello Graham,

Thank you for contacting us.

We have investigated further and it seems that this could possibly be due to a misconfiguration on our end. We have updated all records on our end now, and believe the issue should now be resolved.

=====================

So, all I can do is apologise to all that have had any problems and thank you to all that posted the issues, without these posts the problem would never come to light.

8-)

PS, yes I asked, but I'm sure they will never tell me what the "misconfiguration" was, as I'm sure it was something simple and probably avoidable.

Thanks Graham! (Fingers crossed )

It is extremely easy to misconfigure a zone file in a DNS setup. A missing period (.) in a name will do it - and be quite difficult to see. So will forgetting to update the serial number after a change - that will prevent a propagation to the secondary(s) and others.

Glad it's fixed - and that you have (sort of) an indication as to what happened.

Geoff.

Graham,
Problems all resolved for me.

Thanks for your time and effort in this upgrade!

Peter

There seems to be la lot of issues with the net recently.

About a fortnight ago, my wife had friends' messages to her Yahoo email bounced by several ISPs & just yesterday, the same started to happen to my Bigpond email from a friend who has am iprimus email. Seems to be no rhyme nor reason to it.

well done guys its working great now.