Battlefield 4 Why is 24/7 Shanghai always being DDOS'ed by night time?

[iL-MeowMix-Li]

I understand there was a server move by decision of the dedicated server provider from Dallas to Chicago for a "faster server" but honestly ever since the server move, ive noticed the 24/7 shanghai server being down a lot more often than it used to be back in the day, or even before the move.

Just want to know why the server has been going down a lot more often, and i know youre going to tell me to "just go play another server, its not a big deal" .... but dont get me wrong, its my favorite server and map to play and whenever im free to play the server seems to always be down!

I live in the west coast, so by the time im free later in the night to play Battlefield, most of the division leaders who control Server Procon cannot restart the server till the next morning.

Is there nothing that can be done? I feel like only the 24/7 shanghai server goes down most often its so frustrating.

 

[Gwiberr]

Firstly, i'd like to apologize for the recent DDoS attacks on Shanghai but I can assure you that Leadership are doing all they can to counter the problem. It is difficult to stay on top of the attacks and to keep the server up when it keeps going down. I have been on many times where someone from Leadership has come online early morning just to restart the server.

Please be patient and I can assure you that someday we will counter the issue

If you have any suggestions on how we could either improve the servers or even raise this matter directly towards Leadership, please use the DropBox:

http://www.edge-gamers.com/forums/forumdisplay.php?f=852

 

[iL-MeowMix-Li]

Firstly, i'd like to apologize for the recent DDoS attacks on Shanghai but I can assure you that Leadership are doing all they can to counter the problem. It is difficult to stay on top of the attacks and to keep the server up when it keeps going down. I have been on many times where someone from Leadership has come online early morning just to restart the server.

Please be patient and I can assure you that someday we will counter the issue

If you have any suggestions on how we could either improve the servers or even raise this matter directly towards Leadership, please use the DropBox:

http://www.edge-gamers.com/forums/forumdisplay.php?f=852

I am very patient, dont worry. I am just frustrated because it is my favorite server to play on. But i have other servers to play as well.

I honestly have no idea what i could do to help but if i think of something genius ill be sure to drop box it.

eGO seems like to be a huge community... there should be at least one or two people awake at this time to be able to restart the server? that person... is 100% up to leadership to elect though.

 

[DarkRaven]

Hello Meowmix,

We have spoken to our host and there is not much that can be done for ddos attacks at this time. The datacenter freely provides between 50 to 100 Gig of protection from ddos attacks. What that means is we are getting hit at higher rates than what those protections are capable of stopping. Sometimes the attacks are directed at our server sometimes we get the backend lag or shutdown from the datacenter being hit. As for the 8 hour shutdowns that is the new normal. We used to have only 4 hour downtimes but due to larger attacks hitting our servers they bumped it to 8 hours for most datacenters. The datacenter in Atlanta goes down for 24 hours for such attacks. The attacks mostly happen late at night to early morning from 10 pm EST to 3 am EST. Some are noticed by large lag spikes to the server others totally freeze and shutdown the server. If we are on and having large spikes we usually try to mitigate those attacks by shutting down a few minutes and then coming back online. Per the hosting that will not always work as some attacks are too large and too long. If your on late and notice it getting large spikes and freezing send us a pm here or TS. I try to be on late most nights until 1-3 am sometimes. I might not always be right in TS but send a message. As for the restarts we have to wait the full 8 hours from the initiation of the server null route. As i said sometimes the attacks are early and we can get it online 5-8 am EST some that are later in the am we get it up around 10-12 am/noon EST. Shanghai map is also on the Conquest pop maps server so maybe you can join that server if our regular shanghai is down to play and vote it into rotation. But dont be disappointed when you might have to play other maps. We are working on other security measures to mitigate these attacks but still going over solutions at this time. If you have any further questions contact us. I hope this explains more to you and others about these attacks.

 

[iL-MeowMix-Li]

Hello Meowmix,

We have spoken to our host and there is not much that can be done for ddos attacks at this time. The datacenter freely provides between 50 to 100 Gig of protection from ddos attacks. What that means is we are getting hit at higher rates than what those protections are capable of stopping. Sometimes the attacks are directed at our server sometimes we get the backend lag or shutdown from the datacenter being hit. As for the 8 hour shutdowns that is the new normal. We used to have only 4 hour downtimes but due to larger attacks hitting our servers they bumped it to 8 hours for most datacenters. The datacenter in Atlanta goes down for 24 hours for such attacks. The attacks mostly happen late at night to early morning from 10 pm EST to 3 am EST. Some are noticed by large lag spikes to the server others totally freeze and shutdown the server. If we are on and having large spikes we usually try to mitigate those attacks by shutting down a few minutes and then coming back online. Per the hosting that will not always work as some attacks are too large and too long. If your on late and notice it getting large spikes and freezing send us a pm here or TS. I try to be on late most nights until 1-3 am sometimes. I might not always be right in TS but send a message. As for the restarts we have to wait the full 8 hours from the initiation of the server null route. As i said sometimes the attacks are early and we can get it online 5-8 am EST some that are later in the am we get it up around 10-12 am/noon EST. Shanghai map is also on the Conquest pop maps server so maybe you can join that server if our regular shanghai is down to play and vote it into rotation. But dont be disappointed when you might have to play other maps. We are working on other security measures to mitigate these attacks but still going over solutions at this time. If you have any further questions contact us. I hope this explains more to you and others about these attacks.

I dont get disappointed at all! I like to play 24/7 Conquest sometimes and i often play Dragon Valley when i get the chance to find one populated.

Not a huge fan of locker, so i cant do that. Haha.

But yes this explains a lot more, thanks Raven. :thumbup:

 

[Simon]

Great explanation Raven! Perhaps a pinned post with just that would be good to help out anyone else who may have questions about the server status.

 

[Aetos17]

Great explanation Raven! Perhaps a pinned post with just that would be good to help out anyone else who may have questions about the server status.

Or we could find out who is doing it and send them a free t shirt that says "I tried to DDOS eGO and all I got was this free shirt."

 

[Gwiberr]

Or we could find out who is doing it and send them a free t shirt that says "I tried to DDOS eGO and all I got was this free shirt."

I like Aetos' idea better...

 

[1010101Binary]

Thanks Raven, if there's anything the community can do to help let us know! Would it be possible to get law enforcement involved? If the source of the DOS attack is the same IP Block every time, you may be able to contact their service provider (if in the United States) and get that customer shut down or put their ISP on legal notice. If its truly a DDOS (distributed denial) from dozens or hundreds of different IP Addresses, then I would imagine there's no much we can do. Is there anything we can glean from the source IP(s) of the attacker?

P.S. It may be one idea not to discuss our law enforcement approach to the attacker on public forums, as they may change their tactic, or give them information. Whatever the approach is, I hope we there is something we can do.

 

[itsfulloftuna]

If the server is not behind a firewall, then you call up the colocation provider and have them put it behind a firewall. Then you either have an ACL for ICMP traffic if it is needed for tracking purposes, or you have them deprioritize ICMP traffic when it reaches a certain percentage. There are lots of ways to prevent this.

 

[DarkRaven]

Hello Guys,

To Binary: The attacks have been true ddos attempts at both our servers and to the datacenter. Law enforcement/FBI would be an option only if we could track an IP back specifically which is very hard to do as they spoof ips and locations. Also such attack would have to do serious financial harm or cause economic loss. Game servers well in the eyes of hosting dont constitute that unless it deals with purchases of dedicated server and being a host. We have asked if the host could repay us for time lost which cannot happen either as they have a SLA or service level agreement that if we are ddosed they do not have to pay. On a side note some direct IPs have been blocked by our host but seems that they only shut them down for a few weeks to mitigate the attack. If we notice large lag spikes we attempt to restart at a rounds end to mitigate what could be an incoming attack building up. As i mentioned though sometimes they hit hard and fast without time to take such measures. Hackers are pretty tricky sometimes. We have moved to a higher ddos mitigation site but anyone can easily see from our battlelog page the location and ip. Hackers can trace those ips to where they are at as well to know how much to hit us with. Knowledge of how much protection datacenters and hosting companies provide can be found on the internet.

To Tuna: Thanks for the suggestions tuna and we can look into those. Mitigating ddos attacks of such magnitude take a lot of resources, bandwidth, and tools. Also it costs quite a bit of money. We will be looking at offsite options as well in tracking and mitigation.

 

[itsfulloftuna]

Hello Guys,

To Tuna: Thanks for the suggestions tuna and we can look into those. Mitigating ddos attacks of such magnitude take a lot of resources, bandwidth, and tools. Also it costs quite a bit of money. We will be looking at offsite options as well in tracking and mitigation.

Anything to help. To some colocations, traffic = money, and they can null route certain traffic to help stop the attacks, or, at the very least, reduce its impact. They have smart firewalls and even smarter IPS devices that can be installed/subscribed to identify certain disruptive patterns in traffic and create rules while analyzing packets to stop it before it even starts. If it costs money, then, so be it. I would also like to pretend that this particular data center would be interested in blocking the offending AUP traffic before it routes in.

I think overall, eGO would rather spend a few more dimes to ensure server stability as opposed to doing little and having players abandon playing on the servers ever again.

As far as FBI involvement, its won't really go that far. there are hundreds, if not thousands, of cases of DDoS'ing almost every day, and a lot comes down to the commercial level of ISP and their agreements with their subscribers. Almost always (in at least my experience) server and IP security is left up the the end user. Either they implement their own, or pay more money for theirs. Unless is damaging to the essential core of the US operations, they will probably get to it by 2018. Probably. Law enforcement is also a joke, because it will most certainly OOR for them, and will tell you to call the IP owners. Hell, in this day of age, (also from my experience) it would be surprising to get some type of LEO on the phone/email that even knows what you are talking about, let alone what to tell you.

Easiest way to stop the DoS/DDoS is to just keep it away from anything considered to be eGO leased IP space. Get to the colocation, where they can disperse a null route, and notify upstream providers and peering points for them to discard the traffic as well. If that can't be done, then have the gateway of the IP space create a rule to have any/all ICMP traffic dropped.They are a colocation, and when there is a CO, there is a route processor. They can easily fix this for you by looking at logs, or by killing the entire process all together.