A note about yesterday’s downtime

RSS
Godville, Sep 07 2011

Lots of you may have experienced problems when checking your heroes yesterday, as GV servers were not available or were returning error messages. First of all, we’re sorry that happened, and we want to let you know about the situation and what we plan to do in order to prevent it from happening again.

Our hosting provider informed us that due to some excavation work, a fiber optic cable that connects our datacenter (and 4 others) to the Internet had been cut, which caused everything to work really really slowly or sometimes not work at all. We tried to mitigate the issue by tuning up the servers, but that wasn’t enough to work around the underlying connectivity issue completely. After approximately 8 hours, the underlying connectivity issue was fixed by our hosting provider, which resolved the issues in accessing our servers.

Unfortunately it’s almost impossible to manually restore or rollback all the bad things that happened to your heroes while they were wandering around without supervision from their gods, but here’s what we’re going to do to prevent it in the future:
  • tune up timeouts in our mobile apps to better handle situations like this
  • fix some newly found bugs in our Android app related to error messages when servers are not available (some of them were already fixed in v2.5 available in Android Market, but this version is still going through the Amazon Appstore review process and not yet available to those who downloaded the app from the Amazon Appstore)
  • provision for additional server capacity to handle even more requests
Here are some tips for our players:
  • follow us on twitter @godvillegame to get the latest updates, especially at times when our site is not accessible
  • if you’re using our Android app and for some reason you were logged out of your account, but had never set a password: please try to login with your godname and an empty password
  • be aware that due to Amazon’s review process, app updates to Amazon Appstore are very likely to reach you several weeks later than the same update in Android Market

UPD

During yesterday’s non-stop endless patching & tuning session, we accidentally messed up one of the servers, which eventually backfired causing some of the accessibility issues observed today. As of now, this problem is resolved and everything should work fine. And to prevent this happening again, we promise to try get some sleep and be more careful even when our inbox is flooding and things are on fire.

If you don’t use twitter, here is the good old-fashioned RSS feed for blog posts.

Comments (80)
Trentos Sep 07 2011 12:20
Gravatar

Im still not regenerating godpower much like yesterday during the connectivity issues. The blog says everything is fine now. I’m not sure i’m receiving a problem free connection.

OhMG 6 Sep 07 2011 16:36
Gravatar

Thanks for the heads up.

Shortly Sep 07 2011 18:29
Gravatar

I am, unfortunately, an Amazon…is there a way to delete my Amazon app and re-load but still be able to keep my studly hero?

Sheisme33 Sep 07 2011 18:45
Gravatar

Nothing is working today. It won’t refresh at all in any category.. Hurry for a fix I’m so addicted!

Opythesustu Sep 07 2011 19:01
Gravatar

Thanks for the communication GV Gods. @shortly, if you have an account for your hero, you can delete the app, download the one from the market, and log in with your hero name/password.

Monkiegod Sep 07 2011 19:13
Gravatar

I had no problems yesterday, but cannot access my hero today, on the app, it simply doesn’t update the information (despite sending me notifications of godpower and special items etc), on my phones browser it says the browser is the problem and won’t load the page (used my phones browser 2 days ago to access a friend’s hero) and on my computer the page continually “refreshes” and won’t allow any interaction but all the correct information seems to be there.

Bomboy Sep 07 2011 19:29
Gravatar

The Internet is not a big truck, or something that you can just dump something on. It’s tubes. A series of fragile and destructible tubes. A backhoe is not the proper tool for maintaining these tubes.

http://godvillegame.com/superhero is working fine for me, but http://godvillegame.com/hero results in constantly getting “Failed to connect to the server in turbo mode (error code: 201).”

CodexAngel Sep 07 2011 20:00
Gravatar

I actually did delete and redownload from market, no dice, still not working. I had issues yesterday, but it worked later in the evening. Was still fine this morning, until about 2 hours ago.

Brian the Lazy 5 Sep 07 2011 20:04
Gravatar

Thanks for the explanation. I am working good on Android today (9/7). – Quando omni flunkus vici!

Bomboy Sep 07 2011 20:05
Gravatar

Disregard my previous comment about turbo mode not working. My work hates websockets. Also bunnies.

Khalii Sep 07 2011 20:07
Gravatar

I had problems yesterday then it straightened out -now ,for the last hour or so ,it’s acting the same as yesterday is the same for others?

Marthter 6 Sep 07 2011 20:11
Gravatar

Having new(?) problems today. Android app (latest version) not refreshing, turbo mode hosed on superhero page when using Google Chrome.

Number 20 Sep 07 2011 20:13
Gravatar

I am also experiencing issues identical to yesterday. Been that way for the last 2 hours or so

Childeric Sep 07 2011 20:16
Gravatar

Same here. Down since a couple of hours.

Isis Jane Sep 07 2011 20:22
Gravatar

Via iPhone app, everything is down. Forums, all. Not even an error message. No response to password retrieval request via godvillegame.com. He’s cold, hungry, lonely, and desperate! Mighty Ones, won’t you please send us a sign?...

Chase Hammer Sep 07 2011 20:22
Gravatar

still not working properly. its updating on the website but not on my phone. its 2:21pm and my diary on my phone is stuck on 11:49am. also regenerating godpower much and hero pretty much totally ignoring my voice commands (like not even acknowledging i sent them).

Khalii Sep 07 2011 20:35
Gravatar

Glad it’s not just me-hope it’s fixed soon- pretty addicted!

Lane1chick Sep 07 2011 22:40
Gravatar

Mine is still stuck on 6:35 am….not sure if it was from yesterday or today but no matter the amount of refreshes….uninstalls/reinstalls is fixing it. It is now 5:40 pm e.s.t. and I really miss my game!

Lane1chick Sep 07 2011 22:42
Gravatar

Mine is still stuck on 6:35 am….not sure if it was from yesterday or today but no matter the amount of refreshes….uninstalls/reinstalls is NOT fixing it. It is now 5:40 pm e.s.t. and I really miss my game! did the same yesterday as well in which reading your article explained that but still nothing today.

Burning Kittins Sep 07 2011 23:01
Gravatar

Yeah app is not refreshing anymore… been like this for a good few hours.

Please login to leave comment

back to all posts