Announcement

Collapse

Grendel's Revenge Free Accounts

On December 15th of 2012, Grendel's Revenge enabled free accounts for play. This announcement is following in the suit of TEC's own free account announcement, serving as a reminder to any new or returning players that are browsing the forums.

For all the details about what free accounts are, look no further than here: http://forum.skotos.net/showthread.php?t=95655

In order to log into GR for free, simply visit this page to sign into your Skotos account (ignoring any subscription notices it may give): https://www.skotos.net/user/login.php

Then click the following link for Zealotry/Firefox: zealotry:@GrendelsRevenge.skotos.net...login/Zealotry

Or this link for Alice/Internet Explorer: http://grendelsrevenge.skotos.net/mo...in/alice.shtml

Problems? Questions? You can email the GR staff group at grendel.staff@gmail.com
See more
See less

GR down until further notice [GR Up]

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Fangor
    replied
    While some attention is being paid to GR, perhaps staff could address a point I've raised in other threads, which is that when GR Free Accounts were announced the possibility of Free Accounts having two available character slots was raised - the time seems right for this to become a reality.

    Leave a comment:


  • cjmccoy
    replied
    Woo hoo! Thank you for everything! I can get my GR fix now and looks like all my characters are intact. Thank you for all the updates and hard work on your end.

    Leave a comment:


  • Smarmy
    replied
    Yippeee!!!

    Leave a comment:


  • ShannonA
    replied
    PS: Now that I can actually look at our Grendel's Revenge files, I suspect they would had to increase to just over 400G to be able to restore. That's because the sparse files on GR take up about 1.5G, but bloat to almost 200G if filled in (and there are two non-compressed copies: the live game and the backup).

    Leave a comment:


  • ShannonA
    replied
    I believe we're running again. Hopefully everything is well.

    The problem *was* the sparse files. I had them restore without any of the sparse files, which is fine because the nightly backup is actually in two parts: (1) I backup the sparse files to another directory; and (2) I make a compressed tar of those backed up files. So without any of the sparse files, I just unpacked my tar and rebuilt from there. I am thankful that I created a several times redundant backup system when I moved us to the cloud.

    The backup should be from 3am on Friday, which was the last backup before the power outage, so despite the annoying three and a half days of downtime, that's just the pretty standard seven hours rollback which we can expect to see when a machine is taken down without notice.

    For any regular players of the game (e.g., I'll see time spent on GR when I look at games played last week) who were put out by not being able to play over the weekend, feel free to drop me a line at shannona@skotos.net and include your Skotos account name and I'll send you over some StoryPoints as apology.

    Leave a comment:


  • Smarmy
    replied
    Originally posted by cjmccoy View Post
    Thank you very much for the explanation. Sounds like you are working as hard as you can to resolve the problem so thank you for that, we know your time is valuable. With luck, things will get back to normal soon.
    I second this. And thank you for working so hard on this issue. GR players often feel like the neglected step-child where support for the game is concerned. It warms my heart to know you are working so hard to restore our beloved game.

    Leave a comment:


  • ShannonA
    replied
    They've tried the restore three or four times, with escalating sizes. They are definitely saying the sparse files are a problem. They're also saying that they're ready to turn it over to a back-end engineer, who may take up to 24 hours to respond.

    So, probably tomorrow.

    Very frustrating that their backups appear to have a bug when dealing with sparse files, but as I said, they're hard to work with.

    Leave a comment:


  • Smarmy
    replied
    Originally posted by ShannonA View Post

    Our colo just bumped our disk size from 50G to 150G, and the restore went for about 10 minutes instead of 3 before failing. I have a suspicion that if they double it again, we'll get our restore correctly, and then I can resparsify the files.

    So, fingers are crossed.
    Can you request that they do so?

    Leave a comment:


  • Smarmy
    replied
    Originally posted by cjmccoy View Post
    At least some progress! All my fingers, toes, arms, and legs crossed. TY for the update again. BTW going upstairs with legs crossed is more difficult than I thought.
    You forget to cross your eyes! I am sure that should you do that, GR will be up and running pronto!

    Leave a comment:


  • cjmccoy
    replied
    At least some progress! All my fingers, toes, arms, and legs crossed. TY for the update again. BTW going upstairs with legs crossed is more difficult than I thought.

    Leave a comment:


  • ShannonA
    replied
    It is looking like this is a global problem with our Genesis backups. That's because the data files use something called "sparse data", which means that you have a data file that's mostly blank. I don't know why WAP opted to use them. I'd guess efficiency. But they've always been problematic, and we have to be careful when moving them around, to make sure they remain sparse. Otherwise, a file that is 14-15G can suddenly bloat to 109G (or at least that's what TEC would do right now).

    It looks like our colo's backup and restore functionality is *not* careful and that when trying to restore the data file it bloats it, which fills our disk. And no one knew this before because we've never done a full restore of TEC or GR, because we've got so many data file backups.

    Our colo just bumped our disk size from 50G to 150G, and the restore went for about 10 minutes instead of 3 before failing. I have a suspicion that if they double it again, we'll get our restore correctly, and then I can resparsify the files.

    So, fingers are crossed.

    Leave a comment:


  • cjmccoy
    replied
    Thank you very much for the explanation. Sounds like you are working as hard as you can to resolve the problem so thank you for that, we know your time is valuable. With luck, things will get back to normal soon.

    Leave a comment:


  • Smarmy
    replied
    Thank you, ShannonA for this detailed explanation.

    Leave a comment:


  • ShannonA
    replied
    This problem has nothing to do with Grendel's Revenge being a lightly supported game (though, indeed, it is). This has to do with problems of a jaw-dropping gravity and of a very frustrating longevity at our collocation facility.

    Here's the full synopsis:

    Friday morning, around 10am, our colo started having networking problems at their Fremont, California facility where all of our games are located.

    Around 11am, while fixing the problem, they somehow managed to reboot every machine in their facility, which is a catastrophe at a hosting facility. We had this happen once at our old facility and now once here. So, twice in about 15 years, which is frankly twice more than an unannounced, accidental reboot should happen, since colo facilities are supposed to have multiple redundancies to ensure this never occurs.

    The Genesis games do not respond well to reboots. The current data is always corrupted. Fortunately, we maintain backup data from 0-12 hours old on the same machine, and we can usually restore from that. That's what we did for TEC. Unfortunately, there was some deeper problem with Grendel's Revenge. Unannounced reboots can cause widespread data corruption on a hard drive, so I just went out to our backup. This has always been 100% reliable at our colo facility: we have never failed to cleanly restore from a backup, and we've had to perhaps half-a-dozen times in the last several years.

    I rebuilt from the ~5am Friday backup, which should contain the same game backup files as had been on the hard drive, and thus should have just been the same <12 hour rollback which is standard for a sudden reboot. Unfortunately, that backup would not reboot.

    I contact our colo and they took a full day to get back to me, which is also unheard of, but was apparently due to their creating such a widespread disaster with their sudden reboot, leaving them totally inundated with calls. They said they were working on it, and I requeried late Saturday night and got the same result.

    This morning, just as I was composing a new letter, I got the latest response from them: their backup team wanted me to give things a try. For the first time in almost three days I had access to our backups again. I discovered:
    • The Friday morning backup still results in an unbootable file system, and this time I was able to go deeper into it and discovered that only a fraction of our files are accessible.
    • Two other backups I tried (one from nine days ago and one from two years ago) just won't restore.

    I reported the continuing problems to our colo and am awaiting more responses. We are keeping on top of this as much as possible, but unfortunately at the moment we're dependent on our colo actually giving us a backup that works.

    The biggest problem (and biggest danger) is that I currently don't have the Grendel's Revenge game. You know, the actual code. Any backup could give that me, but I need one of them.

    If I get that in our most recent backup from Friday morning, no problem, things are exactly as I expected. If they can't deliver that backup, but can deliver any other backup that we have (there are three more), then I can merge that with a copy of a GR data file that I keep backed up over on another machine that keeps backups of local data files. These are slightly out of date; they're our last ditch emergencies. But I have one from the 14th.

    So that's where I am at currently. The lesson learned is probably that I should have a backup of the actual GR game files too, but honestly we have a number of games where if we lose their core infrastructure, the data files might not be enough. (As it happens, TEC is our one really safe game, because we have a full copy that we use for testing purposes, in addition to all the ways we backup data files.)

    Leave a comment:


  • justjoy
    replied
    There's no particular date when the maintenance will be over?

    Leave a comment:

Working...
X