Re-import, and data in edits

Discussion in 'Announcements and Rules' started by alethiophile, Dec 16, 2014.

  1. alethiophile

    alethiophile Shadowed Philosopher Administrator

    Joined:
    Apr 26, 2013
    Messages:
    2,367
    Likes Received:
    6,816
    As of now, there is the ability to re-import data from SMF in a limited manner, this time running the SMF data through a better input converter. This should fix many of the issues in old posts with corruption due to failed import. However, it will necessarily result in the loss of any edits made to the re-imported posts since the move to XF.

    It is possible to do this by small parts, choosing which posts to overwrite and which to leave. However, it's not easily possible to tell which posts have valuable edit data which should be retained, and which can be overwritten with their SMF versions. Moreover, once a post has been re-imported, to revert it will require a restoration from backup, which is an onerous procedure.

    Thus, asking the userbase: how many of you have made significant edits to old posts since moving to XF? If edits were made only in order to fix import errors, would re-importing in order to make the posts match their SMF version be acceptable? Would it be a problem if those edits were lost, to be replaced with the properly-converted contents of the SMF database?

    In order to resolve the archive issues, it's very likely such a re-import will take place. However, doing this it's possible to exclude posts at almost arbitrary granularity. Anyone desiring such should particularly note it here.
     
    Lement, kinglugia, Walkir and 3 others like this.
  2. Biigoh

    Biigoh What is this? Moderator

    Joined:
    Feb 19, 2013
    Messages:
    14,013
    Likes Received:
    28,821
    I kind of want to have it reimported. With the tags fixed/edited....

    Sure we'll need to edit the index or save a copy of the index of various quests stories and such...

    But it's for the greater good.

    Also please do a system wide notification before you do this?
     
  3. Least Devotee

    Least Devotee Versed in the lewd.

    Joined:
    May 9, 2014
    Messages:
    1,931
    Likes Received:
    2,124
    Well I'll have to request that any of my post on the first page of my quests not be changed, probably a few others I can't recall at the moment. Quests can be found Here and Here.
     
  4. Mr_John

    Mr_John Versed in the lewd.

    Joined:
    Sep 13, 2014
    Messages:
    1,529
    Likes Received:
    12,705
    If this does happen, I second the system wide notification. A time length of 2~3 days warning should give the vast majority of people time to save backups of their own.

    As for a personal opinion, I'm largely ambivalent. It's annoying to go through older posts and see broken links rendered into BBcode. It'd be a pain on a personal level to save each post and re-upload it when the restore happens, but I'll survive.

    EDIT; If we have the option, I would like my posts to be left alone during the transfer.
     
    Last edited: Dec 16, 2014
    Least Devotee likes this.
  5. a2znut

    a2znut Needs more handholding.

    Joined:
    Nov 12, 2013
    Messages:
    3,293
    Likes Received:
    2,602
    Please ignore all of my posts.
     
  6. alethiophile

    alethiophile Shadowed Philosopher Administrator

    Joined:
    Apr 26, 2013
    Messages:
    2,367
    Likes Received:
    6,816
    To clarify, the only way this will result in substantial data loss is if significant new content has been added to an old post via edit, that was not in the SMF forum. If a post was edited only to fix import issues, ideally, the re-import should result in an equivalent fixed post. (Also, if you have any posts which you do wish to be excluded, just note them here. I'm quite capable of leaving be individual posts.)

    The re-import itself will require several hours' downtime, and won't be happening for at least a week or so, while I get done with finals. There will definitely be a site-wide notice before it goes down, and a backup will be taken, so any unintentional losses can be restored.

    Edit: Also, as a bonus, the re-import automatically amends post links. Any links which pointed to SMF posts in SMF will afterward properly point to XF posts here. Thus, unless something strange is going on quest and story indexes should come out right without needing any modification.
     
  7. Valette-Serafina

    Valette-Serafina Shameless Shadow

    Joined:
    Feb 16, 2014
    Messages:
    3,845
    Likes Received:
    17,186
    I've edited the story posts and info sheets for In the Land of Lightning. Please don't touch my posts in that thread.
    Everything else, well. It'd be nice to have that formatting back.

    A heads up of at least three days'd still be appreciated, though.
     
    Plotvitalnpc and Least Devotee like this.
  8. ChandraMagic

    ChandraMagic Just a Bit Ditzy Moderator

    Joined:
    Feb 21, 2014
    Messages:
    3,748
    Likes Received:
    16,137
    Hrm, if you do do this, I'd ask you to leave my posts in my quest, Birth of a Succubus, alone, because I've already gone and fixed up the important posts in it and it would be a hassle to re-fix the front pages and such. Other than that, have at it.
     
    Last edited: Dec 16, 2014
  9. Mr_John

    Mr_John Versed in the lewd.

    Joined:
    Sep 13, 2014
    Messages:
    1,529
    Likes Received:
    12,705
    I'd like it if my Storybook was left untouched, but other than that I wouldn't have a problem.
     
  10. dalmedya

    dalmedya (unspecified)

    Joined:
    May 29, 2014
    Messages:
    3,831
    Likes Received:
    3,621
    Wouldn't opt-in make more sense than opt-out? Revert only posts that haven't been edited since the move, unless the poster specifically requests it?
     
  11. alethiophile

    alethiophile Shadowed Philosopher Administrator

    Joined:
    Apr 26, 2013
    Messages:
    2,367
    Likes Received:
    6,816
    Unfortunately, the edited-since data has been contaminated; some of my stopgap fixes, such as removing the   HTML entities which got scattered around posts, touched the edit field, and there have been other edits which were solely intended to fix corruption issues, and thus shouldn't necessarily block re-import. Using opt-in would miss the point: the reason for this is to fix the archives, all of them, including those by authors who are currently out of touch. My theory is that anyone who's actively editing old posts will be on the site enough to see the notice during the week or so I'll have it up, and come here to tell me.

    I could also set a cutoff point, e.g. December 5th (about a week after the transition, and after most of the fuss, including my mass search-and-replaces, had died down), and exclude any posts edited after that. Would that be preferable?
     
    Least Devotee likes this.
  12. Adyen

    Adyen Experienced.

    Joined:
    Jan 30, 2014
    Messages:
    3,945
    Likes Received:
    2,525
    I've edited my posts on the first page generally on stat-changes due to updates.

    If possible, I'd really like it if my threads were spared this, but if you must do it for the entire server, I'll have to save a copy offsite and then re-edit everything.
     
  13. Megaolix

    Megaolix Moderator

    Joined:
    Feb 19, 2013
    Messages:
    8,835
    Likes Received:
    11,748
    Well... No need to touch my posts for me.
     
  14. TotalAbsolutism

    TotalAbsolutism Magnificent Bastard

    Joined:
    Nov 27, 2014
    Messages:
    1,517
    Likes Received:
    7,737
    I came after the move, so I have no opinion. It oughtn't affect anything I've done, because of that, yes?
     
  15. Aleh

    Aleh GM, not hand-holder

    Joined:
    Feb 20, 2014
    Messages:
    2,549
    Likes Received:
    11,144
    While I've done a good bit of editing, my thread was started on XenForo. There shouldn't be an SMF backup to "restore" from.
     
  16. alethiophile

    alethiophile Shadowed Philosopher Administrator

    Joined:
    Apr 26, 2013
    Messages:
    2,367
    Likes Received:
    6,816
    I can't think of a circumstance under which this would be necessary. If I run it cutoff-style, then any posts of yours older than the migration that were not edited should come through in substantially identical form (barring corruption), and any that were edited would be ignored by default. (The only potential issue being with posts that were edited only in the week or so after migration but then not since, but it's my impression that most of the posts at issue are updated regularly with character stats and so on, and so would have edit dates more recent than that. Are these what you're referring to?)

    Both correct. Content that was originally posted to XF will not be touched in any event.
     
  17. GLH

    GLH Really?

    Joined:
    Mar 1, 2013
    Messages:
    843
    Likes Received:
    4,470
    The only areas that are going to be most affected by this are quests and story indexes.
     
  18. Adyen

    Adyen Experienced.

    Joined:
    Jan 30, 2014
    Messages:
    3,945
    Likes Received:
    2,525
    Well, you're the one working on the site. If you think it's fine then I'll trust ya. ;)
     
  19. Darkened

    Darkened Experienced.

    Joined:
    Mar 1, 2014
    Messages:
    3,532
    Likes Received:
    7,077
    Thanks for doing this! This is a great step to getting the archives perfect. I'd like to exclude my posts in these three threads though (or at least my posts on the first page of each). Since the edit I've manually tinkered with the look/tags/etc on each sheet a bit to use new tags and the like and would just as soon not have to do it again if it's not convenient.

    http://forum.questionablequesting.com/threads/i-am-so-totally-an-evil-overlord-original-setting.724/
    http://forum.questionablequesting.com/threads/wishin-aint-easy-zero-no-tsukaima.674/
    http://forum.questionablequesting.com/threads/miracle-magical-wars-magical-girl-survival-quest.772/
     
  20. Ampersandwich

    Ampersandwich Outl&ish Gr& Panj&rum

    Joined:
    Mar 5, 2014
    Messages:
    873
    Likes Received:
    1,622
    This is probably a good idea anyway; it gives authors the power to opt-out mechanically. (Just go to a post you want not to be touched and make a trivial edit to it.)
     
  21. HypoSoc

    HypoSoc Time, once consumed, has no meaning

    Joined:
    May 4, 2014
    Messages:
    2,423
    Likes Received:
    12,371
    December 5th cutoff is a good one.
     
  22. Nekraa

    Nekraa Nekraa Moderator

    Joined:
    Mar 2, 2013
    Messages:
    4,628
    Likes Received:
    10,397
    I have not done many edits to my posts. And the changes done are saved in word-files so there's no problem you change them.

    Haven't really looked at my post though and don't know if it's needed.
     
  23. Chibi-Reaper

    Chibi-Reaper D-Donations plz?

    Joined:
    Feb 19, 2013
    Messages:
    7,894
    Likes Received:
    71,870
    Um.

    Some of my OP posts for threads I fill with a lot of important information that I have my doubts anyone keeps constant track of, but that I edit relatively frequently. Like Crash, or Trader.

    Other hand, some of my OP posts are just a collection of index links because I didn't feel like making an archive right from the beginning, wasn't sure it'd be needed or that things would last long enough to require it. Like Desperate Measures.

    All in all, if there's a cut-off though, I can just make a quick edit. Most op posts aside, I've only really edited to fix image links as I find them broken, so...
     
  24. The Shadowmind

    The Shadowmind Experienced.

    Joined:
    May 29, 2014
    Messages:
    4,322
    Likes Received:
    19,105
  25. Valette-Serafina

    Valette-Serafina Shameless Shadow

    Joined:
    Feb 16, 2014
    Messages:
    3,845
    Likes Received:
    17,186
    Yo. If the re-import's still gonna be a thing, I'd like to keep my posts on the front page of nameless shadow quest as well as the ones on In the Land of Lightning.
     
    Plotvitalnpc likes this.
  26. alethiophile

    alethiophile Shadowed Philosopher Administrator

    Joined:
    Apr 26, 2013
    Messages:
    2,367
    Likes Received:
    6,816
    I've decided to use the date-cutoff method of excluding posts, probably with a value around December 5th. Therefore, any post which was last edited after that date will be preserved automatically. If you've already edited a target post after that date, you're fine; if you haven't, you can just make a trivial edit now in order to push the date to the right place.

    There are a few technical issues I need to resolve before doing the re-import, and though finals are now done I instead am in the midst of the much more pleasant but not significantly less time-eating pursuit of family Christmas celebrations. Thus, these have to be done in my spare time. Re-import may happen sometime in the few days before or after Christmas; in whatever case, there will be at least several days' warning with a sitewide notice before downtime happens.
     
  27. Trump-12

    Trump-12 Not too sore, are you?

    Joined:
    Jul 17, 2014
    Messages:
    272
    Likes Received:
    81
    My thread had significant changes to the first post, as well as next post links edited in at the end of each part.