Explanation of recent problems

Discussion in 'Announcements' started by sk89q, 20 February 2013.

Tags:
  1. sk89q

    sk89q Administrator
    Staff Member

    Joined:
    1 December 2011
    Messages:
    2.490
    Likes Received:
    1.504
    We run the server off of RAM so that any speed issues with the regular hard drives don't reflect on the server, and it has been fine like so for about a year now. Unfortunately, about 9 hours ago, the RAM disk we use was configured to only 8 gigabytes and the server used it all up. Now, this would not be so bad, but a decision was made by an administrator, not quite intentionally, to keep the server continually running for the next eight hours.

    Some chunks may have simply "disappeared." Please report this to a moderator and we will look into restoring it from a backup.

    [h2]Explanation[/h2]

    Does running out of disk space cause data loss? Not necessarily. Picture a chunk file as the example below, with three different chunks stored in different “slots.” The three squares represent disk space already “acquired” on the disk as one single .mca file, with the white areas representing acquired (from the PoV of a hard drive) but unused (from the PoV of Minecraft) space.

    chunk.png

    Sometimes a chunk needs to be enlarged, but only some activities do that. If you are placing down “simple” blocks like sandstone or cobble, you’re just changing a few bits of data in the chunk, so the chunk does not need to be resized. However, this is different when you add new entities (mobs, such as skeletons, or item drops, etc.) or new tile entities (more complex blocks, like chests, furnaces, etc.) because the existing space isn’t enough to store all the new data (think inventories!). Those changes require more disk space to store the chunk.

    chunk1.png

    But even then, even if the hard drive itself is out of space, sometimes it is OK. The middle chunk above still has “room” to expand. The white space available was already, as mentioned previously, acquired on the disk.

    The problems appear when the slot that the chunk is already contained within becomes too small to fit the newest version of that chunk. What happens is that a new free area has to be found in the file that can fit the chunk, but if that cannot be found, the chunk is placed at the end of the file. Placing data at the end of the file means that the entire file itself has to be enlarged, therefore requiring more actual disk space usage. When the hard disk is out of space, the extra data has nowhere to go. Because the header of the .mca file indexes where a particular chunk is stored, the index for this chunk may now point to this section of the file that cannot actually be written, causing the chunk to simply “disappear” on next load.

    chunk2.png

    An interesting tidbit of information is that this is how your regular hard drive works. When you save a file, it has to do the exact same thing: find a place to store the bits. If you've ever heard about "fragmentation," that is when, rather than simply moving the entire file to another place, the file is split across across several areas. Of course, while Minecraft can put more data at the end of the file, your hard drive has a fixed amount of total space. The chunk format used by Minecraft does not do fragmentation.
     
    Chaeris likes this.
  2. Chaeris

    Chaeris Active Member

    Joined:
    8 March 2012
    Messages:
    766
    Likes Received:
    89
    This is nicely explained, you should read, but if you are too lazy:

    The server ran out of memory!
     
  3. Cranimesao

    Cranimesao Administrator
    Staff Member

    Joined:
    1 December 2011
    Messages:
    618
    Likes Received:
    466
    Getting a message at 4am stating that the server has crashed and not looking into it immediately = bad. I was really tired though :(
     
  4. hsun324

    hsun324 Programmer, Gamer

    Joined:
    14 May 2012
    Messages:
    362
    Likes Received:
    90
    Yea, I mean I was one of the people immediately affected by this, but I know that the people who knew what was wrong were not going to be very happy getting a message from me at 1AM PST.
     
  5. Pathaleon

    Pathaleon Forum & Server Administrator
    Staff Member

    Joined:
    3 July 2012
    Messages:
    1.014
    Likes Received:
    670
    sowwy cran
     
  6. glitch80

    glitch80 Active Member

    Joined:
    26 March 2012
    Messages:
    345
    Likes Received:
    232
    What are we paying you for then?

    Oh, right. :)
     
    Shadow likes this.